Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I used this command:

rsync -avP /source1/imgs /backup/  

to copy a directory onto another directory on the same server. It's about 18 TB of data. 23 million files. For building files only it took more than 3 days.

Now that I have a backup of my production directory, my question is: How can I update the backup directory with the changes happened on the production directory?

share|improve this question
up vote 3 down vote accepted

I such large filesystems, file based backup is not efficient, esp if that file system is active and changes are made often, as during the time it takes backup the data, it changed a lot of times, so it will be hard for you to determine which file copied when.

You might want to consider a real time back up, like using a distributed filesystem or a distributed block.

I personally use drbd, And I've set up lvm on top of it and using snapshots.

You can also setup a storage based on ZFS (I liked Nexenta), ZFS have a great snapshots features and there are few options to setup a live replication, if think one of them called luster, never tried that.

share|improve this answer

rsync to it again. rsync will do its best to figure out what has changed and only synchronize that. You can pass -n to perform a dry run so that you can verify.

share|improve this answer
@lgnacio: my problem is also building more than 23millions files taking lots of memory and time.. your answer mean i have to add -n to myt rsync command? – AlBouazizi May 6 '11 at 11:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .