I have a directory with lots of images(100,000+). Many of these are duplicates/identical images but obviously all have different filenames. I need to find the images that have the most duplicates in this directory. For example file1.jpeg has 120 duplicates, file2.jpeg has 90 duplicates, etc.
I was thinking I would get the md5 of each file and do some kind of sort, but I'm fuzzy on the details. Can this be done with a shell script?
To be clear, I don't need to remove duplicates(yet), I need to find which files have the most copies.
I'm on OS X if that helps.