I am using a 1.5TB Seagate USB Drive, formatted with NTFS, attached to a Linux Computer.
I am copying a 70GB File to the drive.
I have noticed that the file size of the copy has started increasing at a lower rate.
At the beginning, the file was growing at approximately 19MB/s
Now, with the destination file size at 60GB, the rate of growth is about 5MB/s
What are the factors that could contribute to this?
My guess is it's the cache (or buffer if you prefer).
The first chunk of data is written very fast into the cache (to be written to disk at leasure). The cache fills up, and so the transfer rate drops down to the actual transfer rate to the disk (as data is written to the disk from the cache space becomes available in the cache)
The gradual decrease in speed you see may be the average speed over the entire transfer. The longer the transfer goes on the higher the percentage of the transfer is at the lower actual speed, and so the lower the average transfer rate gets.
They say a picture paints 1,000 words, so here is one.
The blue line is the actual transfer rate, the red one the average since the start. (These are only example figures I made up from my head).
Hum, difficult to say.
Other operation on the source and/or dest, fragmented disk space on one or the other. Does it behave exactly the same way for each copy?
This is an issue I have seen with the way all the new green and low power drives like to park their heads every so often.
Apart from obvious cache aspect mentioned in above answers, Linux handles USB storage devices differently than local ones.
When you transfer data to "classic" disk (attached internally via SATA or externally via eSATA), Linux kernel uses DMA mode, which means that the whole transfer is done directly from RAM to disk controller, not involving CPU.
That's why SATA/eSATA transfers are fast.
However when you attach the same disk via USB, then each kilobyte of data is split to 64-byte packets that are exchanged between your computer and USB controller in your USB disk case. This has to be done using CPU, so not in DMA mode.
That's why USB is slower.
What you should also realize is that Linux is caching data in many aspects (not only disk/filesystem related) and on many layers. There are over 100 different caches.
So the bigger file you try to send to USB disk, the more caches are filled during the process. This is exactly the cause on non-linear slowing down of the transfers.
