Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Here is the thing ,

I have a folder on my server with 50 GB size and its' containing over 60000 file. I used rsync to transfer it to a mirror server and almost half of file has been transferred.Now i want to delete transferred file on main server.

Can this be done with rsync? I did read the help and find --delete option but these files are so important so i want to have an expert opinion thanks.

share|improve this question

migrated from stackoverflow.com Jul 17 '13 at 0:24

This question came from our site for professional and enthusiast programmers.

    
Just make sure you do a "dry run" with the -n option first. Also backups are a good idea. –  Paul R Jul 16 '13 at 20:28
    
I don't believe that rsync will "delete from source after transfer". You could write a script that transfers batches of files and then deletes after verifying successful transfer, though. –  twalberg Jul 16 '13 at 21:09

2 Answers 2

up vote 0 down vote accepted

rsync (checked with version 3.0.9) has an option called --remove-source-files which does what it says. If you only want to delete the transferred files and not transfer additional files which have not yet been transferred, you need to additionally use the option `--existing``.

Unfortunately it seems that rsync doesn't output which files it is deleting even if options --verbose --itemize-changes --stats are used.

Example

# create source and target dirs
mkdir /tmp/source
mkdir /tmp/target
# create a test file in source
touch /tmp/source/test
# rsync source and target
rsync --archive --itemize-changes --verbose --stats /tmp/source/ /tmp/target
# verify that test has been copied to target
[ -f /tmp/target/test ] && echo "Found" || echo "Not found"
# create another file in source
touch /tmp/source/test2
# delete files on source which are already existing on target
rsync --archive --itemize-changes --verbose --stats --remove-source-files --existing /tmp/source/ /tmp/target
# verify that test has been deleted on source
[ -f /tmp/source/test ] && echo "Found" || echo "Not found"
# verify that test2 still exists on source and was not transferred to target
[ -f /tmp/source/test2 ] && echo "Found" || echo "Not found"
[ -f /tmp/target/test2 ] && echo "Found" || echo "Not found"
share|improve this answer

As written before, rsync will not delete from the source, only on the destination.

In your case, I would generate MD5 hashes of the files on the mirror server, then check on the primary server if the hashes are correct and remove those files.

I.e.:

mirror$ find . -type f -print0 | xargs -0 md5sum > mirror.md5

..transfer mirror.md5 to primary server...

primary$ md5sum -c mirror.md5

Check for any FAILED files, then remove the files that have been transfered succesfully. You could automate it like this:

md5sum -c mirror.md5 | grep 'OK$' | sed -e 's/: OK$//' | while read FILE; do rm "$FILE"; done

This will filter all files with a good hash, chop off the 'OK' part from md5sum and remove the files one by one.

Needless to say, after this you don't want to use the --delete option from rsync to transfer the second half of your files...

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.