Tell me more ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.
up vote 0 down vote favorite

On my Windows server, I have multiple folders containing hundreds if not thousands of PDFs (primarily documentation scans, scanned as images, non-OCR'ed).

I'm consolidating and reorganizing and need to de-duplicate these folders.

Now, I've already gone through with a filesize+CRC32/MD5/SHA-1 sweep to get the exact duplicates - that was easy.

However, now I need to get the ones that are effectively the same file but different quality. Additionally, I need to find the ones that are the same document but perhaps missing a page.

Is there any way to automate this process? I've tried DiffPDF as suggested here: Two PDF files, identical except for possibly different cover pages -- how to remove duplicates? but even with two pre-selected files (one approx. 12 MB, one approx. 6 MB, both with the exact same contents.), it could not do the job.

I'd prefer a Windows utility, but a Linux utility could also work.

My presumption is: page count + page extraction + image similarity dupe search could work in some sort of script. That's a LOT of pages though...

share|improve this question

closed as too localized by Nifle, Luke, Indrek, Randolph West, KronoS Sep 9 '12 at 18:10

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, see the FAQ.