Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two copies of a folder


I want to merge them, doing the following:

If a file is only in src, I want it to be moved to dest

If a file is only in dest, I want it ignored IE left alone.

If a file is in both and has identical contents (IE same size and date), delete from src

If a file is in both and does not have identical contents, leave behind in src so I can manually merge them.

Only a very small number of files (between 0% and 5% of total files) should be in this last category, but I don't know how to separate the the in both and the same from in both, but different.

I've tried to figure out how to do this with rsync but to no avail so far.

share|improve this question
up vote 15 down vote accepted

I've only performed limited functionality testing, so please be careful with this command (--dry-run):

rsync -avPr --ignore-existing --remove-source-files src/ dest

Please note the trailing / as this will recurse into src instead of copying src itself, this should maintain your existing paths.

By using the --ignore-existing flag in combination with the --remove-source-files flag you will delete only files from src that are sync'ed from src to dest, that is files that did not previously exist in dest only.

For deleting non-sync'ed files, that is those that already existed in dest/ as in src/, you can use:

for file in `find src/ -type f`; do diff $file `echo $file | sed 's/src/dest/'` && rm $file || echo $file; done


find src -type f -exec bash -c 'cmp -s "$0" "${0/#src/dest}" && rm "$0"' {} \;

if filenames could contain whitespace/new lines/… Regarding Gilles' comment concerning special characters, that is certainly something to be mindful of and there are many solutions, the simplest would be to pass an -i to rm which will prompt before all deletion. Provided that src/, or its parent path, is provided to find, however, the fully qualified path should result in all file names being handled properly by both the diff and rm commands without quoting.

share|improve this answer
correction: that command will not remove files from src if an identical copy already exists in dest – Tok Nov 23 '10 at 18:52
Yeah :(. That's the part that I'm finding hard to figure out. – David Oneill Nov 23 '10 at 19:07
Well, the good news is that you can solve it independently without much hassle: for file in `find src/ -type f`; do diff $file `echo $file | sed 's/src/dest/'` && rm $file || echo $file; done (you can skip the || echo $file if you like, it is included for completeness) – Tok Nov 23 '10 at 19:16
Nifty: that's what I needed. Edit that into your answer, and I'll accept it! – David Oneill Nov 24 '10 at 0:33
@Tok: Your command will choke on file names that contain special characters (whitespace, \?*[, initial -). You need to use double quotes around variable substitutions, pass -- to utilities before file names, use find … -exec … instead of parsing the output of find. With an rm command in the mix, this is a recipe for disaster. – Gilles Nov 24 '10 at 1:06

unison is the tool you're looking for. Try unison-gtk if you prefer a gui. But I don't think it will delete similar files: unison try to have both directories identical. Nevertheless it will easyly 1) identify which files are to copy; 2) which ones needs manual merge.

share|improve this answer
It doesn't do exactly what the OP asks for, but it sounds like it accomplishes the OP's ultimate goal. +1 – Ryan Thompson Nov 24 '10 at 17:27
+1 Sadly, the server I'm running this on does not have unison installed, nor do I have the permissions to install it. But this might be a good answer to someone else. – David Oneill Nov 24 '10 at 21:37
You can download unison executable from…. Install it somewhere in your home directory, it's just one file. – JooMing Nov 29 '10 at 15:01

The following script should do things reasonably. It moves files from the source to the destination, never overwriting a file and creating directories as necessary. Source files that have a corresponding different file in the destination are left alone, as are files that are not regular files or directories (e.g. symbolic links). The files left over in the source are those for which there is a conflict. Beware, I haven't tested it at all.

cd src
find . -exec sh -c '
    set -- "/path/to/dest/$0"
    if [ -d "$0" ]; then #  the source is a directory 
      if ! [ -e "$1" ]; then
        mv -- "$0" "$1"  # move whole directory in one go
    elif ! [ -e "$0" ]; then  # the source doesn't exist after all
      :  # might happen if a whole directory was moved
    elif ! [ -e "$1" ]; then  # the destination doesn't exist
      mv -- "$0" "$1"
    elif [ -f "$1" ] && cmp -s -- "$0" "$1"; then  # identical files
      rm -- "$0"
  ' {} \;

Another approach would be to do a union mount one directory above the other, for example with funionfs or unionfs-fuse.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .