I have several CDs/DVDs which have partially overlapping content (the overlapping files are identical, but have different names), and some of the files are on my hard disk. I need to get the remaining unique files copied to my hard disk.

I found a really good duplicate file finder, Duplicate Cleaner, which lets you select multiple folders and then finds duplicates by checksum (or file name, size, date) and is very fast, and free. It won't help me do what I want though, unless I just copied everything, and then deleted the duplicates - but I would have to do multiple cycles, as I don't have room to copy all the CDs/DVDs to my hard disk.

I found a couple of file sync programs, but they don't have the compare function - the file names must match. (I tried other duplicate file finders on CNET, but they aren't as good as Duplicate Cleaner, and also don't have the functionality I need.)

Thanks for any help.

1 Answer 1


If you're willing to apply a little elbow grease then you can put together a script using find and some checksum tools in order to find the unique files.

  • I would think software like this would be popular, especially for people will lots of MP3's.
    – Carl
    Apr 12, 2010 at 3:39
  • 2
    I don't think people really mind having five copies of the same thing spread across a few dozen discs, if the people around me are any indication. Apr 12, 2010 at 3:46
  • 1
    Also, audio files should use fingerprinting to find duplicates, not checksumming. Apr 12, 2010 at 3:46
  • The files aren't audio. I haven't seen any software that will do this (checksum, fingerprinting) with any file type.
    – Carl
    Apr 12, 2010 at 12:57
  • Fingerprinting is going to be content-specific anyway. The same kind of algorithm that can determine whether 2 songs are the "same" isn't going to work for images or video.
    – afrazier
    Aug 11, 2010 at 12:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .