Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am using OSX 10.8.4 and would like to remove duplicate files in a single flat directory if an md5 hash matches.

I have looked up several answers but none of them work for me, I think because of differences in command syntax between terminal versions.

Things I have tried:


Finding and removing duplicate files in osx with a script

and some of these: http://www.commandlinefu.com/commands/view/3555/find-duplicate-files-based-on-size-first-then-md5-hash#comment

This approach feels closest:

find . -size 20 \! -type d -exec cksum {} \; | sort | tee /tmp/f.tmp | cut -f 1,2 -d ' ' | uniq -d | grep -hif - /tmp/f.tmp > duplicates.txt

But I get an error: grep: -: No such file or directory

Upon inspection, /tmp/f.tmp does exist. The duplicates.txt file is created but is empty.

How do I deduplicate these files?

share|improve this question
If you're not opposed to using a script, checkout dupinator, it is a python script which could accomplish this task. code.activestate.com/recipes/362459 –  spuder Aug 17 '13 at 18:17
Tried this. it finds no duplicates in a test folder which has intentional duplicates. Also comment 6 notes a data-losing bug: code.activestate.com/recipes/362459/#c6 –  Simple As Could Be Aug 17 '13 at 18:45
add comment

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.