Synchronisation software to find files that have moved paths within a folder?

Question

Say I have a pictures folder which I reorganized on one computer. I'd like to use that directory as the base and compare it with another version on a backup drive. Will any Synchronisation/compare program find that a file in one folder has moved locations within a compare folder?

For instance, say I reorganized my pictures from trips into folders by year with the trips folders inside each year folder. If I use a regular compare utility I wind up with two copies of everything that's moved in different locations.

Answer 1

I don't know of any software that detects renames/moves across a whole folder hierarchy. That would be fairly slow, as it would have to scan all files to compare them.

The feature was once proposed as a patch for rsync (google for "rsync detect-renamed"), but apparently it was never accepted into rsync.

rsync does have a limited version of what you are looking for:

-y, --fuzzy

This option tells rsync that it should look for a basis file for any destination file that is missing. The current algorithm looks in the same directory as the destination file for either a file that has an identical size and modified-time, or a similarly-named file. If found, rsync uses the fuzzy basis file to try to speed up the transfer.

This will not work if you move files between directories, though.

That said, most sync programs do have the option to remove files in the destination if they are missing from the source, so a move should be replicated as new+delete, which requires more copying, but would work.

If you just want to compare drives, not sync them, you could just write a small script that lists all files with their filenames (and maybe a checksum), and import that into a spreadsheet or small db. Then you can sort by filename or checksum, and directly compare the lists of files. That might be the easiest route.

Answer 2

Yes, I found software that does this, better than any other: Super Flexible File Sunchronizer.

Listed under the features:

Detects moved files. If you reorganized your folders by moving files to different locations, Super Flexible File Synchronizer will detect this and quickly perform the same moves, for example, on your laptop. No extraneous copying!"

Answer 3

It all depends on how you set up the sync process.

Most sync programs have options on what to do if a file is/isn't in the source or is/isn't in the target. Properly set to completely echo a directory, a sync program will delete files in the target that are not in the source, and will copy from the source files that are not in the target or that are newer.

However, no sync program will detect that a file was renamed, in the sense that it will not rename the file in the target directory. It will just delete the file with the old name, and then copy the the file with the new name. This will take more time, of course, but sync programs don't have artificial intelligence.

That said, my own favorite sync product is FreeFileSync (go to end of the page).

Answer 4

Syncovery does the job if you have used it before you reorganize the source side:

• Detects Moved Files. If you reorganized your folders by moving files to different locations, Syncovery will detect this and quickly perform the same moves on the other side of the sync, rather than deleting and re-copying the files.

If you don't allow it to keep a database it is not possible for it to detect and handle moved files.

If you can use Linux, maybe my personal approach can then help:

Because I sync my files to a webdav mount, Synchronization to the webdav destination without move detection would mean large file operations.

I made a small script reorg_Remote_Dir_detect_moves.sh (on github) which is trying to detect the most moved files and then creates a new temporary shell-script with several commands to adjust the remote directory. Since I only take care of the file names the script is no perfect solution.

For safety, several files will be ignored:

• Files with same (beginning) names on every side
• Files which are only on the remote side

Skipped files will then be handled by your preferred sync tool (e.g. rsync, unison ...), which you have to use after running the temporary shell-script. So maybe my script is useful for someone. If so (to make it more clear) there are three steps:

1. Run the shell script reorg_Remote_Dir_detect_moves.sh (on github)
2. This will create the temporary shell-script /dev/shm/REORGRemoteMoveScript.sh
   => run this to do the moves (will be fast on mounted webdav)
3. Run your preferred sync tool (e.g. rsync, unison ...)

Answer 5

Try Allway Sync, it's free and you won't end up with duplicates. I use it too to backup my laptop on my main pc.

Answer 6

I heard Microsoft's SyncToy program has this capability.

or this tool by Risefly.

Answer 7

Syncthing and git-annex should do the job

• Git-Annex is as the name suggest based on git, moving and/or renaming a file will be recognize and so not duplicated.
• Syncthing will also sync the modification instead of duplicating the file

Both of those software are also able to keep version of your file and/or to keep deleted files for a certain period of time.

The main issue is you use a backup software while you tend to synchronize files, this is why you/we need a good file planning (how to store and how to name files) before starting backup and storage.

Answer 8

The BestSync (http://www.risefly.com) was the only thing has worked for me almost as expected.

It does detect the moves and renames, but not as you expect. Instead it represent a movement/rename as delete+copy. But at least it detects it AFTER the movement and rename has made by a 3d party software.