Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a bunch of PDF files which are articles downloaded from various Journal websites. The problem is, some of these PDF files are duplicates of one another. However, if I run a basic duplicate file finder, they won't show up, because journals have the (somewhat annoying) habit of putting a cover page in the PDF before the actual article. In other words, if I have two copies of article "X" by author Y, one downloaded from site A and the other site B, they aren't exact duplicates because one has cover page from A and the other has (different) cover page from B. So my question is:

Is it possible to check if two PDF files are duplicates,ignoring the first page?

Of course, this doesn't solve all my problems, since some journal websites don't put a cover page! So the best possible question would be:

Is it possible to check if two PDF files are duplicates,where one (or both) might have a cover page?

share|improve this question
up vote 1 down vote accepted

According to the PDF Spec there might be some things to help you.

First of all there is the ID, a unique identifier that hould not be changed after initial creation.

From the PDF spec linked above, Section 14.4:

File identifiers shall be defined by the optional ID entry in a PDF file’s trailer dictionary (see 7.5.5, “File Trailer” ). The ID entry is optional but should be used. The value of th is entry shall be an array of two byte strings. The first byte string shall be a permanent identifier based on the contents of the file at the time it was originally created and shall not change when the file is increm entally updated

Second of all you can take a look at 7.5.6 Figure 3. According to that Figur you should have an original Body.

For the first option there is a poppler function to extract the IDs (you are interested in the permanent one) , however I do not now how you could strip out the original body to compare those hashes.

share|improve this answer

DiffPDF can visually compare PDFs (I haven't tried it myself), also available as a portable app.

Even if you removed the cover page from all the articles with something like PDFTK, I very much doubt that the PDFs will be the same in a binary compare.

Maybe another way would be to rename each file to something unique in the articles, an article number perhaps, or a date including time? Then you could do a filename compare.

share|improve this answer
I should probably have clarified a bit: besides the cover page, the PDFs are identical. That is, before insertion of the cover page, they are the same binary file. – Steve D Apr 13 '11 at 2:39
Even so, re-saving them after removal of the cover page, I doubt they will be identical. You can give it a go, but if the PDF format saves something like a date in the file after the page removal, then it will be different. – Hydaral Apr 18 '11 at 1:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .