I am transferring a bunch (20+) of large (1GB+) files to my external flash drive over USB 2.0. Is it quicker to just sling them all over at once (as in one at a time but not waiting for the previous transfer to finish) so that there are multiple transfers going on, or transfer one, wait for it to finish, transfer the next. The files are coming from a variety of locations so I can't do one single big transfer.
Are there any other advantages to one way or the other that are worth considering?
If they are all coming from different physical disks, but being written to a solid state drive (flash, SSD) then you can copy them simultaneously.
The physical head movement (latency) is what will slow a transfer from a spinning platter disk. If you were copying from a single HDD, then you would start a single big transfer rather than lots of small ones. The head going back and forth between two or more files that are being copied is what slows the transfer. Of course you would get the same effect if the HDD were heavily fragmented even if you were copying a single file.
If it was between solid state drives, then it would not matter.
If you don't want to wait around for each transfer to finish then use a copy queuing app like Teracopy.
Of course the flash drive write speed would be the bottleneck here, so it probably doesn't matter which way you do it. :)
If destination disk is an SSD, just keep in mind that SSD's are, for what's in concern here, damn fast. (*)
If destination is anything slower (from a flash with not-so-high-write-rate to a SATA3 HDD to a magnetic tape) you should queue the transfers, one at a time.
Your bottleneck will be on destination disk for sure
In HDDs this advised nowadays because, even though there is more than one head in a cylinder, they are not allowed to read nor write concurrently (it has shown no performance impact in real world scenario due to the fact that we can encounter random data in one of the heads that would not allow us to R/W in that head for most of the time being, and if the heads are not completely centered that could lead to disk corruption).
Common sense (and mathematics too!) say that if we have a very very source but we can make the transfer of files in parallel the head seek could (in not many scenarios though) be faster than serialization of the job.
(*) That is, as nnewton states, because there is no head movement. But should we not use the term latency: latency is head movement + disk rotation delay + read time, and in SSD that is simply random read time and does not depend on the previous read position as disk drives do
Using the built-in file copy utility in Windows they are both horribly slow. You may want to consider using an alternative application like TeraCopy.
If you were copying across IP protocol (FTP/SCP/Samba/etc) then there is less overhead to copy one file since there are multiple handshakes between each file.
In this case, since there are large files and few in quantity, it won't make much difference to the I/O bus what you do and it's probably easier to copy them as they are.
You are best off writing a single file at a time.
You’ll likely find that your flash-drive’s bandwidth will get saturated pretty easily. (The source drive is unlikely to be the bottleneck.) While USB 2.0 can transfer upto 480MBs, your flash-drive is probably limited to ~20MBs. This means that when you transfer a couple of files from a hard drive(s), it will max out the flash-drive’s write speed, and each file will be written slower since they have to share.
Also, by writing multiple files at the same time, you increase fragmentation because unless the space for the files is fully allocated as soon as the transfer starts, they will end up being broken into chunks as the files are transferred.
(Fortunately because flash-drives are solid-state, there is no head that has to thrash back and forth like it would if you transfer multiple files to a hard-disk.)
Another drawback of writing multiple files at the same time is that the file-copy dialogs only indicate the progress and remaining time for that individual file, and it becomes next to impossible to estimate the total time because it is not linear (you cannot simply add them). Transferring them together will give you much more accurate feedback for the whole transfer.
If the source files are from different sources (do you mean different hard-drives? CD/DVD? network drive? or just folders?), then what you can do is to use a dedicated file-copying tool instead of Windows’ built-in function. Some options include the Microsoft TechNet tool Robocopy, which also has a GUI front-end, RichCopy (also from TecNet) which is an enhanced tool based on Robocopy, and Teracopy, which specializes in making the transfer as efficient as possible (and has a free version).
OK since the OP specifically said he was copying them over USB 2.0 to one target I'd have to say one at a time, regardless of source.
S=Serial. Multiple file streams=Overhead.
But in practice...
When copying from multiple sources, the time lost to multiple streams would be more than made up by not having to build a copy 'queue' manually. And it's easier. ;)
Also, if you're copying in multiple streams and one fails, the rest will probably complete.
The electronic processing time saved by loading them individually would have to be appreciable to make up for the time lost between the physical act of waiting for each file to finish loading then moving the next one individually it would seem.
