15

My boss was saying it takes longer to copy two files at the same time than doing one and then the other because the hard drive has to keep switching the head between the two locations on disk. Is this true? It seems to me that the operating system should be smart enough to fix this (i.e., it should know to copy them sequentially). Is this true when copying to a drive connected to a USB port?

EDIT: are there any other factors regarding the files system that factors in? For example does it make a difference copying 1 directory containing 10 files versus copying 10 files from all over the disk? I'm wondering only about one source medium and one destination medium (no reading from multiple drives).

4
  • Sometimes, sometimes not. Apr 26, 2013 at 19:47
  • I think this question could benefit from an actual benchmark, rather than the educated guesses by most of the answerers here. Anyone had a spinning disk and some spare time? I believe for SSD, the differences is likely to be negligible; while for harddisk, I believe the answer could range from major performance penalty to possibly slight benefit depending on the number and sizes of the files and the relative performance of the source and target disks (if they're different).
    – Lie Ryan
    Apr 27, 2013 at 6:44
  • 1
    In addition, if you're transferring over USB2, then there probably won't be any differences; typical harddisk read speed is around 50-100MB/s while the maximum speed of USB2 is 32MB/s, so the bottleneck is likely going to be the USB port rather than harddisk characteristic due to buffering. In other words, it's difficult to say one way or the other.
    – Lie Ryan
    Apr 27, 2013 at 6:52
  • @LieRyan it's too bad you didn't post the comment as an answer because it best fits the question I was intending to ask which was regarding copying from 1 hard disk to 1 external hard disk by USB 2.
    – Celeritas
    Apr 28, 2013 at 6:20

3 Answers 3

12

I will defer you to this question. It seems that running two copy operations on the same disk concurrently (though started through separate copy operations) would indeed take longer as a result of the latency generated by the head seeking back and forth between the competing operations.

If, however, the copy operations are started simultaneously, the large majority of modern operating systems are as you said smart enough to en-queue the transfers one after the other, and should yield a somewhat quicker copy time.

There are apparently a number of applications available to force the file copy operations to queue and execute sequentially such as Teracopy and FastCopy

Performing concurrent copy operations on multiple physical volumes is another matter entirely, however. As is transferring files through alternate protocols.

Regarding your question about copying to a drive connected to a USB port, it is heavily dependent upon the type of memory that the drive uses and the USB spec being utilized (certainly a noteworthy bottleneck for USB 1.0 and 2.0), as well as the previously mentioned factors regarding the copy source volume.

6
  • 1
    does that also not depend on processor capability of the system? Apr 26, 2013 at 18:27
  • @Raghunandan A fair point - I believe it totally depends on the circumstance. Multiple cores/hyperthreading would certainly enhance copy speeds if reading data from multiple physical volumes or SSD/RAMDisk, but in the scenario of a single HDD the copy speed's limiting factors would likely still reside in the IO speeds of the disk and the presence of head-movement latency (I'm sure there must a more appropriate term for that...).
    – bosco
    Apr 26, 2013 at 18:37
  • 3
    @Raghunandan - Might not be the OS so much as the file system (NTFS vs EXT4).
    – Enigma
    Apr 26, 2013 at 18:47
  • 2
    @Raghunandan PC-24000 DDR3 has a bandwidth of 24 GB/s and it's still not a limit for modern CPUs, so I don't think it's really a matter of CPU - even if HDD operations would be 100 times less effective than RAM operations, CPU could still handle at least 245 MB/s and that's still more than fastest SSD's bandwidth.
    – gronostaj
    Apr 26, 2013 at 18:53
  • 1
    @Raghunandan your linux partition may also be on a faster part of the disk than the Windows partition (e.g. outside v. inside)
    – ernie
    Apr 26, 2013 at 19:09
16

There are lots of factors here that could affect this.

  • Source drive - is this spinning disk or SSD? If spinning disk, the layout of the files could affect performance. As the two files are likely on different parts of the disk, this will incur head seek penalties. As you said, if you select two files at once, and initiate the copy that way, the OS will handle the copies sequentially.
  • File layout - fragmentation of the files (both on the source and the destination) could affect performance for non-SSD drives
  • Destination - if you've got two write streams writing to a single target, then you're back to the head seek issue (again, assuming not an SSD), and you could heavily interleave the files. I used to work for a company that made high performance storage, and one of the big issues for them was how many real-time streams of video they could read or write (2k video requires ~300 megs a second). Alternating the writes will slow down the copy process, and would also make reading the file back slower. Of course, if your disk is fragmented to start, your file will be written interleaved anyway.
  • single/multiple source/target - depending on if your files are all coming from one drive or all being written to a single drive, the head seek issue could be more or less
  • file size - for really small files, the head seek issue won't matter, as the head would need to seek to find the next file anyway (meaning that instead of going back and forth between files, the head would be reading files sequentially)

As for if the OS is smart enough to fix this, in general, they are. That is, if you copy multiple files simultaneously (e.g. think of selecting multiple files at once and drag-n-drop).

Of course, if you kick off two cp commands, then it's going to run the two commands, or in Windows if you copy/paste separate files, and get two "copy" progress windows on the screen, then the optimization won't occur. In this case, you've explicitly told the OS that you want to run the two copies simultaneously, so it's not going to decide one copy is more important than the other and start queuing them.

2
  • +1 Only thing missing is RAID configuration, alluded to with "single/multiple source/target". Apr 27, 2013 at 10:05
  • 1
    -1 most of what is said is unrelated to the question
    – Celeritas
    Apr 28, 2013 at 6:22
1

If you multiple select and copy - either from a GUI, or by using a command like "copy C:\folder* D:\folder\" - then the files WILL be copied sequentially anyway, rendering the question moot.

If you copy some files, and while they're copying, start copying more files - either by two separate operations from the GUI, or by two separate commands run simultaneously like "cp -r /usr/bin/ /opt/bin/ & cp -r /usr/local/bin/ /opt/local/bin/" - then your performance will almost certainly decrease - maybe by a little bit (high-end solid state drive, large files in both copy operations) or maybe by a TON (spinning disk, relatively small files in one or both operations). Better operating systems may mitigate the performance penalty to some degree - for example, modern Linux kernels use the CFQ (Completely Fair Queueing) I/O scheduler, which will to some degree "batch" block operations to increase efficiency - but you will still take a performance hit. In some cases the hit may be HUGE, as rotating disks are DRASTICALLY slower at random I/O than sequential I/O (potentially two orders of magnitude or more slower, which is why modern I/O schedulers try their best to mitigate the problem by batching and caching operations to minimize unnecessary seeks).

TL;DR - only run one copy operation at a time, regardless of how many individual files are included in that operation, if you're worried about performance.

Not the answer you're looking for? Browse other questions tagged .