Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

From: http://www.chriswrites.com/2012/02/how-to-find-and-delete-duplicate-files-in-mac-os-x/ How do I modify this to only delete the first version of the file it sees.

Open Terminal from Spotlight or the Utilities folder Change to the directory (folder) you want to search from (including sub-folders) using the cd command. At the command prompt type cd for example cd ~/Documents to change directory to your home Documents folder At the command prompt, type the following command:

find . -size 20 \! -type d -exec cksum {} \; | sort | tee /tmp/f.tmp | cut -f 1,2 -d ' ' | uniq -d | grep -hif – /tmp/f.tmp > duplicates.txt

This method uses a simple checksum to determine whether files are identical. The names of duplicate items will be listed in a file named duplicates.txt in the current directory. Open this to view the names of identical files There are now various ways to delete the duplicates. To delete all the files in the text file, at the command prompt type:

while read file; do rm "$file"; done < duplicates.txt
share|improve this question

migrated from stackoverflow.com Sep 30 '12 at 11:43

This question came from our site for professional and enthusiast programmers.

2 Answers 2

up vote 1 down vote accepted

Firstly, you'll have to reorder the first command line so the order of files found by the find command is maintained:

find . -size 20 ! -type d -exec cksum {} \; | tee /tmp/f.tmp | cut -f 1,2 -d ‘ ‘ | sort | uniq -d | grep -hif – /tmp/f.tmp > duplicates.txt

(Note: for testing purposes in my machine I used find . -type f -exec cksum {} \;)

Secondly, one way to print all but the first duplicate is by use of an auxiliary file, let's say /tmp/f2.tmp. Then we could do something like:

while read line; do
    checksum=$(echo "$line" | cut -f 1,2 -d' ')
    file=$(echo "$line" | cut -f 3 -d' ')

    if grep "$checksum" /tmp/f2.tmp > /dev/null; then
        # /tmp/f2.tmp already contains the checksum
        # print the file name
        # (printf is safer than echo, when for example "$file" starts with "-")
        printf %s\\n "$file"
    else
        echo "$checksum" >> /tmp/f2.tmp
    fi
done < duplicates.txt

Just make sure that /tmp/f2.tmp exists and is empty before you run this, for example through the following commands:

rm /tmp/f2.tmp
touch /tmp/f2.tmp

Hope this helps =)

share|improve this answer

Another option is to use fdupes:

brew install fdupes
fdupes -r .

fdupes -r . finds duplicate files recursively under the current directory. Add -d to be prompted which files to keep, or add -dN to always keep the first file and delete other files.

share|improve this answer
    
fdupes is awesome! Worked like a charm! Thanks bro.! –  racl101 Dec 3 at 6:40

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.