I have a directory full of files that remain relatively static. However, some files are moved between subdirectories.

I create a full backup every month of this data because it's a lot of data. I'd like to start creating weekly or daily incremental backups.

If I need to recover halfway through the month, it's important that I restore the exact state of the original set using the last full backup with all incremental backups up to that point.

I currently use find to locate all modified files each day, and create a tar archive of those. However, this won't capture files deleted or moved.

Any ideas on how I can create incremental backup archives that will allow me to restore the data set to the exact point in time of the last incremental archive? Any files removed between the full backup and last incremental backup must not appear after restoration.



rsync does incremental copies by default and can do backups to both local and remote targets. It can also make deletions on the backup target for files that have been removed from the source. This way you have an exact copy of your source directory on the target that you can restore from. It can also compress data during the copy.

From the man page:

Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Rsync finds files that need to be transferred using a lqquick checkrq algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file's data does not need to be updated.

  • While I can get a full duplicate on a remote server with rsync, there's no way to create an incremental archive that captures the deletes. What I'd like to is to be able to have the full backup, followed by an incremental backup, that when "restored" together will delete the files that were deleted from the source between the full and the incremental. I don't think rsync can accomplish this. – Tharsan Mar 23 '14 at 2:59
  • rsync will do a full copy the first time, after which it will do incremental copies. If you delete on the source, it will delete on the target if the correct options are used. – MaQleod Mar 23 '14 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.