Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a big iso image which is currently being downloaded by a torrent client with space-reservation turned on: that means, file size is not changing while some chunks in in (4 Mib) are constantly changing because of a download.

At 90% download I do the initial rsync to save time later:

$ rsync -Ph DVD.iso /media/another-hdd/
sending incremental file list

DVD.iso
       2.60G 100%   40.23MB/s    0:01:01 (xfer#1, to-check=0/1)

sent 2.60G  bytes  received 73 bytes  34.59M bytes/sec
total size is 2.60G   speedup is 1.00

Then, when the file's fully downloaded, I rsync again:

total size is 2.60G   speedup is 1.00

Speedup=1 says delta-transfer was not used, although 90% of the file has not changed, target dir is on another FS and copying takes several minutes. Why doen't it try to speedup the transfer?! How can I force rsync to use delta-transfer?

share|improve this question
5  
What you are doing does not make any sense. The purpose of rsync is to speed up transferring files over the network, not locally. In order to find the differences, it has to read both the source and destination. In the time it takes to read the destination locally to find the differences, you may as well just do a normal copy. Just download the file to the destination in the first place instead of copying it. – psusi Jan 17 '11 at 15:35
1  
So it just does not use delta-xfer because, working locally, it's faster to copy than to calculate hashes? If yes — post the answer plz :) – kolypto Jan 17 '11 at 16:37
5  
Reading can be faster than writing to a local disk in certain circumstances. It can also reduce the wear on an SSD. This is certainly a valid question and the answer is quite valuable to me. – HRJ May 8 '13 at 11:56
1  
@psusi apart from HRJ's comment above, also consider the case when the target file has been reflinked (eg on btrfs or ocfs2). Minimizing writes during the sync can make an enormous difference to the overall space usage. – Jack Douglas Aug 23 '13 at 21:13
up vote 12 down vote accepted

According to the manpage, psusi is right:

-W, --whole-file: The transfer may be faster if this option is used when the bandwidth between the source and destination machines is higher than the bandwidth to disk (especially when the "disk" is actually a networked filesystem). This is the default when both the source and destination are specified as local paths, but only if no batch-writing option is in effect.

share|improve this answer
8  
Oh, thank you! I miseed this line :) To turn delta-trasfer on, use -no-W – kolypto Jan 18 '11 at 12:35
1  
On my system -no-W doesn't work only the long option -no-whole-file. My reason for needing this switch is when I'm setting up a backup and have large files (eg images) that don't have the same modification time. It is MUCH faster, speedup is 163.26, to sync these files using the delta-transfer on my local filesystem. – Jesse the Wind Wanderer Sep 21 '15 at 17:05

The straight-forward answer to this question is:

Use the --no-W flag to force delta compression, no matter local or remote.

Update: It looks like there is more to the story. The delta compression seems to be enabled only between receive and transmit process of rsync. When outputting the file to the file-system, rsync may still write out the whole file(s), even with delta compression on.

See "Wakan Tanka's" investigation here.

share|improve this answer
    
Thank you! Very helpful. – swietyy Mar 18 '15 at 14:32
    
--no-W always transfer whole file in my case. Please check unix.stackexchange.com/questions/291156/… – Wakan Tanka Jun 21 at 15:30
    
@WakanTanka That is interesting! I have updated my answer. – HRJ Jun 22 at 3:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .