1

This is Mac to Mac (both on 10.12, Sierra) and I want to sync a lot of audio and video files from my local Mac to the server (the other Mac). The video files can be 1 or more GB each but although the audio files are smaller, it is still possible to watch the progress as they take some time, even over a gigabit network.

My question concerns the whole/partial/deltas operation of rsync. As far as I can tell, even using ssh as the transport, it should only copy the changes to files, but with only minor changes to files (like an ID3 tag in an MP3 audio file) the whole file gets resent. First it reports it is deleting a load of files and then it sends those same files again, the whole file it would appear. I can watch the percentage progress and it seems to take as long as copying a file of that size would normally take over the network. There does not appear to be anything gained using a clever algorithm to just transfer deltas.

Under what circumstances would rsync delete a file first and then send the whole file again over the network?

edited

--inplace I believe is slower and certainly less data secure in case of interruption. Also -z is unlikely to help in the case of audio and/or video content. Apart from exclusions etc, I'm just using -a option, with nv and --progress if I'm just testing. The problem is that in this case, a 1GB file with a minor change to an internal metadata tag should not take long to sync as only a very small amount of data needs sending, but I am watching the % progress and it takes at least as long as it would to simply copy the entire file. I can only assume that it is sending the entire file again, especially when it reports that it is deleting that file right at the start. If it really does delete it first like that, there's NO way it can then just send deltas. Is it really deleting those files first or just says that and actually just means that file needs updating?

1

Without seeing your rsync command line, it's hard to be sure what is going on, but check out man rsync and look into the --verbose, --inplace, --partial and --progress options.

Generally, I use a command like rsync -e ssh -avzP --inplace <src> <dst> to transfer files in-place via ssh with archive mode (a), verbose messages (v), compression (z) and partial+progress support (P).

-Rich Alloway (RogueWave)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.