139

I see on the rsync man page that there are a number of delete options, but don't really understand the differences between them. What are the differences between these options?

 --del                   an alias for --delete-during
 --delete                delete extraneous files from dest dirs
 --delete-before         receiver deletes before transfer (default)
 --delete-during         receiver deletes during xfer, not before
 --delete-delay          find deletions during, delete after
 --delete-after          receiver deletes after transfer, not before
 --delete-excluded       also delete excluded files from dest dirs
  • 2
    Why --delete-before is described as (default)? I cannot understand what (default) means. – Marco Marsala Nov 4 '16 at 7:56
  • 1
    It's no longer specified as default in the man page for rsync version 3.1.2 protocol version 31. However, my guess on the version with the quoted text is that if you only use --delete rather than --delete-WHEN, then it's as if you specified --delete-WHEN where WHEN defaults to before. – Tom Hale Jan 14 '17 at 2:15
142
  • --del/--delete_during: Deletes files from the destination dir as they are copied (saves memory compared to --delete-before: --delete-before makes a separate scan to look for deleteables)

  • --delete: Deletes files in the destination directory if they don't exist in the source directory.

  • --delete-before: Delete files in the destination directory before copying file-with-same-name from source directory

  • --delete-during: Delete files in the destination directory WHILE copying file-with-same-name from source directory

  • --delete-delay: Mark deletes during transfer, but wait until transfer is complete

  • --delete-after: Receiver deletes after transfer, not before...If some other part of the rsync moved extra files elsewhere, you'd want this instead of --delete-delay, because --delete-delay decides what it's going to delete in the middle of transfer, whereas --delete-after checks the directory for files that should be deleted AFTER everything is finished.

  • --delete-excluded: Deletes files from the destination directory that are explicitly excluded from transferring from the source directory.

The point of rsync is not copying, it is archiving. This is an important distinction. Processing deleted/changed files is critical, and in many cases nuanced.

The --delete flag in particular is one I've seen screwed up many times. A lot of people use rsync to move files to low priority storage, and in this case you want the files you're moving to still EXIST in the destination directory. That's not what delete does: --delete makes sure that, when you delete a file from the source directory, it is ALSO deleted from your destination directory, so your destination doesn't get full of junk... Once saw a guy wipe out his backup by putting in a new drive, and not turning off his nightly rsync script. Script saw that the source dir was now empty, and it deleted every file in the destination dir, so they'd match.

Most of the other options are space or performance related. When you delete the files is important if you want to make sure the transfer is successful before you do anything, but if your device is too small to handle 2 copies of all the information, you need to delete as you go, etc. It's a little wacky because of its long history across multiple platforms: some options have been added so that people who were used to certain behaviour wouldn't be confused.

| improve this answer | |
  • 7
    So is --delete and --delete-before the same? man rsync doesn't say when --delete deletes files. – ppr Jul 3 '15 at 15:07
  • 1
    @ppr: Depends on the system. The reason it has so many specific options is because delete on Solaris != delete on HP-UX != delete on AIX != delete on Linux. – Satanicpuppy Jul 6 '15 at 14:39
  • When rsyncing a whole directory, does delete-after delete each file in between syncing, or only after the whole dir has been mirrored? I thought delete-after was sufficient for my needs but i'm getting no space left on device when backup up one of my drives. – Sridhar Sarnobat Aug 3 '15 at 21:56
  • 1
    @Sridhar-Sarnobat: Either. It moves everything it's going to move, and when it's done, it's starts deleting things. If you want it to delete before it's moved everything, use --delete-during – Satanicpuppy Aug 4 '15 at 18:58
  • 2
    After some experience, I'm finding that even delete-during doesn't always prevent no space left on device (on Linux). So delete-before is the only guaranteed way to avoid it (like when mirroring a drive that is almost full, and the destination drive is almost full too). – Sridhar Sarnobat Jan 26 '17 at 22:58
5

By default rsync doesn't delete any files at the destination side. To make rsync delete files at all, you need to use at least one of the delete options.

If you don't care when files are being deleted, just use --delete and leave the choice to rsync. You can combine --delete with other delete options (this doesn't conflict) but you don't have to as all other delete options already imply --delete.

--delete-before works as follows: rsync looks which files are present at the source and which files are present at the destination, deletes all files found at the destination but not at the source and then starts the actual synchronization. This order is useful if the destination has little storage space as it will first free up more disk space at the destination before starting transferring any new files. The downside is that rsync will require more memory to perform the operation and the whole operation is a two step process and thus slower.

--delete-during works as follows: rsync immediately starts to synchronize files and when it comes across a file that exists only at the destination, it is deleted. That way there is no speed penalty and also no additional memory is required. The downside is that it may happen that first a lot of new files are copied to the destinations before removed files are being deleted, so the destination may require much more disk storage space during the operation than it requires in the end once the entire operation is done.

--delete-after works as follows: First synchronize all files, then perform the same operation that --delete-before performs before the synchronization phase. This is the worst choice in most common cases as it requires most memory, most disk space at the destination, and it is slower as it is a two step process; basically it combines all disadvantages of the other two methods. This option mainly exists for the case that you are using "merge files" (what merge files are and how they work is beyond the scope of this answer). As these files may contain rules for files to be excluded during delete, new merge files must be copied before the deletion phase if their content shall be considered during the deletion phase. Unless that is a requirement, --delete-after has no advantage.

--delete-delay is a rather new option (it's not available in rsync 2.6.9, which is still the default in macOS 10.15 for example). It works like --delete-during, except that it won't delete files immediately but after the synchronization is done, so it is a hybrid of --delete-during and --delete-after. The advantages are that it is faster than --delete-after and still supports merge files correctly, the disadvantage is that it requires more memory during the synchronization.

--delete-excluded tells rsync to not just delete files that are missing at the source but to also delete files at the destination that were excluded from synchronization (--exclude or --exclude-from), regardless if these files would actually exist at the source or not.

| improve this answer | |
2

There are two things going on:

  1. Who does the deleting
  2. When it happens

Either the sender or the receiver can be instructed to do the deleting (I'm not sure why this matters). So when rsync from one computer connects to the rsync server on the other side, this determines who is effectively issuing the delete command.

When it happens is pretty easy... before means all the files are deleted, and THEN rsync copies over the files. during means as it goes through the list of files, it deletes them when it comes to them, and after means it waits until all the files are transferred over and then deletes the remote side. This matters only when the transfer gets interrupted.

| improve this answer | |
  • I'm pretty sure none of these options are related to deleting things from the sender's side. All of them (AFAIK) relate to deleting things from the receiver's side. Also, you say: "before means all the files are deleted, and THEN rsync copies over the files" — is this right? I'd expect --delete-before to delete only those files that needed to be gone, not all the files! But I guess it's possible that rsync deletes everything and then transfers over the "new" (possibly unchanged) data... – Quuxplusone May 15 '16 at 1:59
  • 1
    Ah, "delete from sender's side" is supported by rsync, but the name of that option is --remove-source-files. – Quuxplusone May 15 '16 at 2:02
2

One other point worth mentioning is that if your source directory ends with /*, then rsync will only consider those files and not the directory itself (and thus the absence of files that you want deleted on the destination).

If you're specifying a delete option above, but rsync looks like it isn't deleting, then check to make sure you're not accidentally globing and providing a list of files when you mean the directory itself.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.