I have looked for various free software to remove duplicate on my mac but could not find a good one. I have finally settled on command prompt to delete duplicate files but I can't get it to work either. May be I am missing something. I am familiar with command prompt but am not an expert on mac terminal esp find and grep. Can you someone provide a terminal command that will show duplicate files and will remove duplicates? I want it working on mac lion os x, otherwise I can find plenty online which does not work. A step by step tutorial will be helpful.

I have tried 1 and 2 and many others. None of them work. Throws some kind of errors.

find . -size 20 \! -type d -exec cksum {} \; | sort | tee /tmp/f.tmp | 
cut -f 1,2 -d ' ' | uniq -d | grep -hif - /tmp/f.tmp > dup.txt
  • The md5 solution looks rather nice. Which all sorts of errors did you get ? What is the output if you run part of the command (e.g. run only find /path/to/search/ -size 20 -type f -exec md5 '{}' ';' on a test folder. Does that part work. If you add the next stage, does it error. Etc etc. – Hennes May 18 '13 at 21:26
  • @Hennes I tried what you adviced. If I run the command in fragments, it runs fine. But the last part gives an error grep -hif - /tmp/f.tmp > dup.txt – hk_ May 20 '13 at 13:57
brew install fdupes
fdupes -rn .
  • -r is recursive, -n ignores empty files
  • Add -dN to delete all files except the first file

find . -type f ! -empty -exec md5 -r {} \+ | awk '++a[$1]!=1' | cut -d' ' -f2- | tr '\n' '\0' | xargs -0 rm

  • The md5 sum of empty files is the same
  • md5 -r (reverse) prints the checksum and filename in an easier to parse format
  • ++a[$1]!=1 prints all except the first line where the first column is duplicated

Related questions:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.