I'm reorganizing some files on my harddrive and have an old copy of a folder, what I want to do is to ensure that every file in folder A (the old copy) exists in folder B. Each folder contains at least 200 files, most of which are likely duplicates.

Caveat: I can't just check the filenames, I must also ensure that they have the same content (à la fdupes).

I checked the manpage for fdupes if there was some way to do the inverse operation (i.e. figure out which files don't have any copies) but came up empty handed, even if the functionality would exist I still have files that exist only in folder B.

These two questions were similar but were Windows only so not much help:

Thanks in advance.

  • I don't have fdupes installed. Does it produce a list of duplicate files? – choroba Dec 3 '15 at 11:37
  • @choroba Yes :) – erb Dec 3 '15 at 11:52

If you already have a list of duplicate files, just compare it to the list of all files to find the single ones:

fdupes -r dir1 dir2 | grep -vwFf- <(find dir1 dir2 -type f)
  • -w only mathces whole words (not substrings, so file abcd doesn't get matched by abc)
  • -F consideres the patterns to be literal, not regular expressions
  • -f uses the specified file (- in this case, i.e. standard input) as a list of expressions to match against.
  • -v as you probably know, shows the nonmatching lines.

This should output a list contains the files only found in one of the folders, it should be easy to verify that they are all in folder B by skimming or grepping for the name of folder A (below called dir1) like this:

fdupes -r dir1 dir2 | grep -vwFf- <(find dir1 dir2 -type f) | grep -F dir1
  • Worked wonders, just had to add the -r flag to fdupes, edited the post so that it recurses. Thanks! – erb Dec 3 '15 at 12:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.