1

Unix was originally designed to perform file system operations in memory, and to update the disk blocks only every 30 seconds during sync. This resulted in a much faster user experience than MSDOS many years ago that accessed the disk blocks directly for each file system operation. But today using Windows 7 I still see that copying lots of small files takes much longer than copying one big file of the same size.

For example: a Visual Studio C# Asp web project Publish directory contains 2000 files, total 150 MB, zipped (7Zip) 90 MB.

  • Copy the entire directory to the same disk: 30 seconds
  • Zip: 5 seconds
  • Copy one zipped file: 2 seconds
  • Unzip: 65 seconds. (On another machine: 20 seconds)

Computationally, unzipping should go faster than zipping. So the unzipping might be dominated by the file system performance.

Disk fragmentation might be a factor, and the reason another similar machine might be faster in handling many file system operations. But if disk fragmentation is a factor here, it might indicate that the user has to wait for each file system operation to finish before the next one starts, i.e. like old DOS.

  • How Windows copies a file has not changed. But I also don't believe Windows uses the disk entirely to copy a file. – Ramhound Jun 25 '14 at 13:03
  • Trying to find info to answer my own question I found that Windows does apply disk caching: [msdn.microsoft.com/en-us/library/windows/desktop/… . But the MSDN pages are full of remarks on how to avoid caching. And we are of course not talking about 256 KB objects here. – Roland Jun 25 '14 at 13:25
0

The behavior (uncached access) you describe hasn't been in use since Windows 3.11. However, Windows flushes its cache more aggressively that Unix. It's down to their respective roots. Unix machines were assumed to be built with reliable hardware and power. Windows PC's were cheap boxes, which crashed far more often. That wasn't just an OS problem, UPS'es are far less common on PC's and cheap IDE disks were less reliable on powerloss than expensive server SCSI disks. The ancient FAT format didn't help either - no journalling so massive corruption possibilities. NTFS is much better. What did you use for the test?

  • My machine has an NTFS formatted disk. – Roland Jun 25 '14 at 13:21
  • I believe that the reason was not just hardware. Unix has always been a multi-user multi-process system needing lots of disk operations. DOS started with almost no disk at all. There was not much point to write to a DOS-1 floppy that could only hold 64 file names or something similar. All subsequent DOS/Win versions tried to be backwards compatible, so today we have still drive letters. Still curious about caching today. – Roland Jun 25 '14 at 13:35
  • Contradicting myself: DOS stood for DISK Operating System. – Roland Jun 25 '14 at 13:37
  • 1
    Downvote issued for a statement that cannot be backed up by facts "Windows PC's were cheap boxes,which crashed far more often" is simply false and cannot be backed up. Personal Computers for nearly 2 decades didn't even use Windows, once Microsoft was in the picture, the reliability of the personal computer market was low for a varity of reasons none had anything to do directly with Windows or Microsoft. – Ramhound Jun 25 '14 at 13:42
  • @Ramhound: read the answer again. I'm contrasting Windows and Unix. That is why I can write "crash more often" - it's a comparison. I didn't say Windows PC "crash often". – MSalters Jun 25 '14 at 13:53
0

From the info I found myself (see above) and from contributors above, I conclude that modern Windows can do disk buffering/cache, but that this is often disabled. Reasons include that NTFS has no journalling. Windows performance might be of secondary importance than profits: quality versus time to market. With Unix/Linux, which are less commercial, quality/performance is much more important.

Coincidence: a coworker is currently zipping a few GB of files to try to copy faster, and is staring at a Windows Archive dialog box saying "10 seconds to go" for the last half hour, with a slowly growing zip file (not frozen). But that might also be a faulty disk... :-)

  • With exception of Mac osx which gives performance at way higher price – STEEL Jun 26 '14 at 2:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.