Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have several CDs/DVDs which have partially overlapping content (the overlapping files are identical, but have different names), and some of the files are on my hard disk. I need to get the remaining unique files copied to my hard disk.

I found a really good duplicate file finder, Duplicate Cleaner, which lets you select multiple folders and then finds duplicates by checksum (or file name, size, date) and is very fast, and free. It won't help me do what I want though, unless I just copied everything, and then deleted the duplicates - but I would have to do multiple cycles, as I don't have room to copy all the CDs/DVDs to my hard disk.

I found a couple of file sync programs, but they don't have the compare function - the file names must match. (I tried other duplicate file finders on CNET, but they aren't as good as Duplicate Cleaner, and also don't have the functionality I need.)

Thanks for any help.

share|improve this question

If you're willing to apply a little elbow grease then you can put together a script using find and some checksum tools in order to find the unique files.

share|improve this answer
I would think software like this would be popular, especially for people will lots of MP3's. – Carl Apr 12 '10 at 3:39
I don't think people really mind having five copies of the same thing spread across a few dozen discs, if the people around me are any indication. – Ignacio Vazquez-Abrams Apr 12 '10 at 3:46
Also, audio files should use fingerprinting to find duplicates, not checksumming. – Ignacio Vazquez-Abrams Apr 12 '10 at 3:46
The files aren't audio. I haven't seen any software that will do this (checksum, fingerprinting) with any file type. – Carl Apr 12 '10 at 12:57
Fingerprinting is going to be content-specific anyway. The same kind of algorithm that can determine whether 2 songs are the "same" isn't going to work for images or video. – afrazier Aug 11 '10 at 12:54

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.