Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I was reading several guides how combine btrfs snapshots with rsync to make an efficient backup solution with history. However it all depends on if rsync --inplace modifies only those portions of files that actually changed, or if it overwrites the whole file sequentially. If it writes the whole file then it seems that btrfs will always create a new copy of the file, which would make the idea much less efficient.

share|improve this question
    
How would it even know if it can avoid writing to the entire file? Doesn't it need to read the entire file first, to figure out what has changed? –  Mehrdad Apr 1 '13 at 3:16
    
@Mehrdad yes, it does, but reading the whole isn't a problem. If rsync reads the whole file and then seeks to and updates only those parts that are needed, btrfs will copy only these updated blocks. But if rsync reads and writes the whole file, then it'll be a problem. –  Petr Pudlák Apr 1 '13 at 14:36
    
@Mehrdad rsync does not only know that it may avoid writing the entire file, it manages to do so without copying it completely over the net. Clever little program. –  hirschhornsalz May 2 '13 at 11:24
add comment

4 Answers

If you pass rsync two local paths, it will default to using "--whole-file", and not delta-transfer. So, what you're looking for is "--no-whole-file". You also get delta-transfer if you requested '-c'.

Here's how you can verify:

$ mkdir a b
$ dd if=/dev/zero of=a/1 bs=1k count=64
$ dd if=/dev/zero of=a/1 bs=1k count=64
$ dd if=/dev/zero of=a/1 bs=1k count=64
$ rsync -av a/ b/
sending incremental file list
./
1
2
3

sent 196831 bytes  received 72 bytes  393806.00 bytes/sec
total size is 196608  speedup is 1.00

Then touch a file and re-sync

$ touch a/1
$ rsync -av --inplace a/ b/
sending incremental file list
1

sent 65662 bytes  received 31 bytes  131386.00 bytes/sec
total size is 196608  speedup is 2.99

You can verify it re-used the inode with "ls -li", but notice it sent a whole 64K bytes. Try again with --no-whole-file

$ touch a/1
$ rsync -av --inplace --no-whole-file a/ b/
sending incremental file list
1

sent 494 bytes  received 595 bytes  2178.00 bytes/sec
total size is 196608  speedup is 180.54

Now you've only sent 494 bytes. You could use strace to further verify if any of the file was written, but this shows it at least used delta-transfer.

share|improve this answer
add comment

--inplace overwrites only regions that have changed. Always use it when writing to Btrfs.

share|improve this answer
    
And do you have an evidence that shows it doesn't overwrite other parts of files? –  Petr Pudlák Oct 25 '13 at 13:00
add comment

From the man page:

          This  option  changes  how  rsync transfers a file when its data
          needs to be updated: instead of the default method of creating a
          new  copy  of  the file and moving it into place when it is com-
          plete, rsync instead writes the updated  data  directly  to  the
          destination file.

This leads me to believe that it writes over the file in its entirety-- I imagine it would be near impossible for rsync to work any other way.

share|improve this answer
    
After determining what parts need update, it could just seek to those parts and update them, instead of writing the entire file. –  Petr Pudlák Apr 1 '13 at 14:39
add comment

If you want rsync to modify only the places where the file hast changed, you must not use --inplace.

--inplace has serveral side effects, among them:

The efficiency of rsync’s delta-transfer algorithm may be reduced if some data in the destination file is overwritten before it can be copied to a position later in the file.

The default for rsync is to use its delta algorithm, which updates only differences between files. See man rsync and the technical report for more details.

share|improve this answer
1  
I think you're not correct, as rsync's man page says that --inplace "...instead of the default method of creating a new copy of the file and moving it into place when it is complete, rsync instead writes the updated data directly to the destination file." This means that without --inplace the whole file gets simply written somewhere else and then moved, which means that BTRFS's copy-on-write saves nothing. (I'm not concerned if the delta-transfer is less efficient, I'm concerned about the disk space is occupied when using BTRFS snapshots + COW.) –  Petr Pudlák May 2 '13 at 11:34
    
--inplace doesn't imply --whole-file. The quote you give explains an edge case, which would only be relevant with files that get data inserted at the beginning. –  Gabriel Oct 25 '13 at 12:29
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.