Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm running rsync to sync a directory onto my external USB HDD. It's about 150 gigs of data. 50000+ files I would guess.

It's running it's first sync at the moment, but its copying files at a rate of only 1-5 MB/s. That seems incredibly slow for a USB 2.0 enclosure. There are no other transfers happening on the drive either.

Here are the options I used:

rsync -avz --progress /mysourcefolder /mytargetfolder

I'm running Ubuntu Server 9.10.

share|improve this question
1  
are you sure you're getting a USB2 connection? does a (non-rsync) copy or other write operation run at normal speeds? if not, have you tried a copy/other write op with another USB port/cable? –  quack quixote Feb 17 '10 at 0:14
    
See also serverfault.com/questions/43014/… - there people also propose using two piped tar commands or cpio. –  Blaisorblade Feb 23 '13 at 17:32

4 Answers 4

up vote 18 down vote accepted

For the first sync just use

cp -a  /mysourcefolder /mytargetfolder

rsync only adds overhead when the destination is empty.

also.. the -z option is probably killing your performance, you shouldn't be using it if you are not transfering data over a slow link.

share|improve this answer
2  
rsync is so called because it's for remote synchronization and is not really appropriate for a locally-connected volume for this very reason. –  msanford Jun 27 '10 at 22:30
2  
It's supposed to be usable also for local transfers, and it's much more flexible. It's only possibly overkill for the first sync. –  Blaisorblade Feb 23 '13 at 17:31

If you're using rsync with a fast network or disk to disk in the same machine,

not using compression -z

and using --inplace

speeds it up to the performance of the harddrives or network

compression uses lots of CPU

not using inplace makes the harddrive thrash alot (it uses a temp file before creating the final)

compression & not using inplace is better for doing it over the internet (slow network)

also, you don't get wirespeed if ton & tons of small files, but it's still faster

share|improve this answer

Use the -W option. This disables delta/diff comparisons. When the file time/sizes differ, rsync copies the whole file.

Also remove the -z option. This is only useful for compressing network traffic.

Now rsync should be as fast as cp.

share|improve this answer
1  
Minor note: -z is only useful for low speed network traffic. If your network is fast enough, it'll slow things down, since you'll be limited by CPU. –  Hugo Jul 8 '13 at 2:28
1  
These tips vastly sped up the transfer of my files between two NAS devices, thanks! –  djhworld Sep 22 '13 at 13:30

You don't say what size distribution your files have. If there are many small files then this will reduce overall transfer rate by increasing head movement latency in both the source and destination drives as the tool opens new files and the OS keeps directory entries and other metadata (such as the filesystem's journal if you are using meta-data journaling like ext3/ext4 and NTFS do by default) up to date during the transfer. A file copy proces will only "get into its stride" for larger objects, when a simple bulk transfer is happening.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.