Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I must not be the only one - I'm rsyncing .tar.gz files and notice that every time the full file gets rsynced rather than the differences. Reading into it it seems back in 1999 someone created an algorithm that fixed the issue http://svana.org/kleptog/rgzip.html (only 5% of data needed transferred)

Has this gone anywhere since, how do I create rsync friendly .tar.gz files?

share|improve this question

migrated from stackoverflow.com Oct 3 '12 at 6:21

This question came from our site for professional and enthusiast programmers.

4 Answers 4

up vote 13 down vote accepted

my gzip (on ubuntu and fedora) has the --rsyncable option. So create the tarballs using:

tar -c whatever/ | gzip --rsyncable > file.tar.gz
share|improve this answer

BeezNest has a pretty good explanation of the rsyncable option for gzip. In the author's test, this option added about 1% to the file size, but made it possible for rsync to transfer the updates to a gzipped file with over 1,300 times speedup.

For the gory details, see this discussion, which they cite.

share|improve this answer

I know that Ubuntu Linux applies a patch (gzip file) to gzip sources to allow for a --rsyncable flag. You can download that patch and use it yourself, or see if your distribution includes the patch.

share|improve this answer

http://smackerelofopinion.blogspot.com/2009/07/rsyncable-gzip.html

gzip --rsyncable  # need two dashes for long options
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.