Write caching causes slower performance?

Question

USB 2.0 External WD 2TB Elements HD.

Connected to Windows XP SP3 machine, write caching disabled by default. Worked fine.

Since it is intended to be a permanent fixture (never removed), and on a UPS (so power outage is not a worry), I decided to switch on write caching in Windows, to improve performance.

At first, I noticed that BURST write speed did indeed increase. However, sustained writes for more than a minute or so caused massive slowdowns in the entire system, and would cause the drive to almost lock up, causing any other I/O operations to fail. After a few minutes it would return to normal.

Disabled write caching, rebooted, and voila, everything is back to normal. What gives?

EDIT: Tested with a different WD internal drive connected via a 3rd-party IDE-to-USB cable, same results. Going to dig up an old Seagate and test it as well...

Answer 1

I can think of two possibilities.

Write caching causes Windows to start caching the data being written into memory. Since your harddrive is faster than the USB external (or techincally the USB interface), files start piling up in memory. Depending on your workload, this can push programs out of memory. So you now have a problem. Data is being read from the harddrive. But now Windows needs to page programs out of memory as the cache is filling up real fast. That can cause a lot of disk reads and writes.

This normally doesn't cause other I/O operations to fail. You should post the exact error messages. That's is rare and of huge concern as it could signal something worse than just an OS error message.

The second possibility is what happens when your external drive receives a flush cache command. Windows regularly sends a "cache flush" command to disks during longer IO write operations. This acts as a safeguard against data loss during power failures. When that happens, Windows stops sending data til the disk's cache is flushed. All this while, your source drive is still spinning, reading data into memory in preparation to send to the external.

There is a third possibility, but that lets start small.

BTW, that burst write speed is just to impress users and isn't real. Windows isn't sending data any faster then as it did during the middle and end of the IO write.

If you recall back to statistics class, what happens to the accuracy of your predictions as your sample size gets smaller?