43

I have 12 ~200 MB files. Copying one of them to another drive takes 20 seconds. When I try to copy all of them at once, it first says it's going to take 5 minutes, then it keeps slowing down into the hours and days.

I'm copying the files through Thunar in XFCE, but I remember having similar problems using the command line. Why does this occur and what can I do to remedy it, other than just copying the files one by one?

3
  • Have you checked the drive for errors? Are these files in sub-folders?
    – nwalke
    May 14, 2012 at 22:50
  • @tubaguy50035 The files are not in sub-folders. That being said, it's likely that the source drive may have errors. I'll check. May 15, 2012 at 3:40
  • @CodeGnome I did a basic cp -v sourcedir/* destdir. May 15, 2012 at 3:41

6 Answers 6

48

This is a known issue with the Linux kernel itself that has been known for ages, no one was tackled the issue yet.

there is currently no fix or known work-around, aside from using Windows which increases in speed and then normalizes. I find this bug to be a very high-priority issue, but the programmers do not feel the same way. It is this bug alone that is starting to drive me away from Linux.

Refs:

9
10

You should try rsync to monitor your transfer speed. For example:

rsync -PSauv sourcedir/* destdir/

If you actually see a drop in transfer speed, as opposed to files that are just big, then you need to look at a few other things.

  1. If your disk is I/O-bound, you will have problems. You often see this when writing heavily to a flash drive or an external USB device.

  2. Test both your disks with hdparm -Tt <device> to see if one is dramatically slower than it should be, or dramatically slower than the other device which can lead to I/O issues.

  3. Check your dmesg output and your SMART status to make sure you don't have any obvious disk errors. You might also consider badblocks -n if you suspect disk errors on a non-SMART drive.

There are lots of reasons why a disk can thrash or perform sub-optimally, but until you can measure the problem, you won't know what the problem is. :)

3

Remember that also the file copy process is somehow cached by Linux. If you take a single file (even if a 200MB one) and start copying it to another place, Linux caches, with great probability, the whole file in memory and writes it in its new location. Then, if you choose to copy that file again and with it a bunch of other files, then the first file is already cached and then probably skips the whole "read part", while the others are first read from and then written to their sources/destinations. This feature can even lead to infinite transfer speeds and similar oddities, so don't trust any measurement that could be invalidated by previous chaching mechanisms.

2

I tried to copy a 4 GB file to a fat32 usb stick.

after 1 GB has been copied, copy speed reduced from c. 20 MB/s to c. 0.5 MB/s.

I formatted the stick to ext4 but it didn't help.

Then I formatted the stick to NTFS as advised here, and had a dramatic improvement in speed:

$ rsync --progress movie.mkv /run/media/user/09D8E089029531DD/
movie.mkv
  3,759,209,408 100%    7.90MB/s    0:08:46 (xfr#1, to-chk=0/1)

PS: this doesn't answer the question but hopefully may solve the problem with speed.

0

The "hours and days" problem is related to having 12 copying processes simultaneously, which stresses the CPU, hence the odd estimated time. The time of copying 12x200=2400MB is surely reasonable, no matter what ETA is displayed in Thunar.

For more info during copying, some file managers provide more info than Thunar.

-3

Burn a copy of SliTaz onto a CD/USB.

It is designed for use on really ancient machines and runs really fast since it's in RAM. You don't need to do an install, just run it live. If you have a USB mounted, you will be able to see it, and then copy what you need to your computer. The transfer rates I have found to be stable and fast.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .