On my Windows server, I have multiple folders containing hundreds if not thousands of PDFs (primarily documentation scans, scanned as images, non-OCR'ed).
I'm consolidating and reorganizing and need to de-duplicate these folders.
Now, I've already gone through with a filesize+CRC32/MD5/SHA-1 sweep to get the exact duplicates - that was easy.
However, now I need to get the ones that are effectively the same file but different quality. Additionally, I need to find the ones that are the same document but perhaps missing a page.
Is there any way to automate this process? I've tried DiffPDF as suggested here: Two PDF files, identical except for possibly different cover pages -- how to remove duplicates? but even with two pre-selected files (one approx. 12 MB, one approx. 6 MB, both with the exact same contents.), it could not do the job.
I'd prefer a Windows utility, but a Linux utility could also work.
My presumption is: page count + page extraction + image similarity dupe search could work in some sort of script. That's a LOT of pages though...