I am using rsync to copy ~10GB of small files. What happens if the files in the source directory are modified while rsync is running?


rsync first scans the files and builds a list. so once the file is listed for sync, rsync will sync the latest change of file. but if the file is not in the list of files to be synced, which was built before starting the sync operation, then it will not sync it.

  • But what happens in following situation? rsync is started, rsync charges FileA as modified, rsync starts coping FileA but FileA is modified again whilst rsync is copying it. – wewa Apr 25 '20 at 15:55
  • @wewa RSYNC in itself does nothing to prevent that kind of data corruption, it can only read what it sees when it reads and does exactly that. If you have a use case running RSYNC concurrently with other writes to files, you need to deal with such things on your own by some kind of locking, snapshots supported using BTRFS/ZFS/LVM or something else fitting into your environment. Things heavily depend on who writes how into which files, e.g. their concrete format and alike. Better to be safe than sorry, though. – Thorsten Schöning Oct 31 '20 at 10:09

What happens if the files in the source directory are modified while rsync is running?

That depends if they are modified before rsync is running as well. As @Sachin said if they are not modified before the transfer then modifications during the transfer won't be picked up, however:

if you are modifying a file while it's being transferred, then probably the checksum will fail and it'll go round again. And if it goes around twice, and it still fails, then it prints a message saying; Error, checksum failed, file changed during transfer?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.