Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two identically sized files, and I need to do a binary diff to check whether they're the same.

I'm currently runnnig diff file1.img file2.img but it's taking quite a while to process my 4 GB files. Is this the most efficient way to do this?

share|improve this question

cmp is designed to find differences in binary files. You might also try checksumming (sum) and compare the hashes.

share|improve this answer

One of the most common ways of determining if two files are identical (assuming their sizes match) is using a program to create a "hash" (essentially a fingerprint) of a file. The most common ones are md5sum and sha1sum.

For example:

$ md5sum file1 file2
e0e7485b678a538c2815132de7f9e878  file1
4a14aace18d472709ccae3910af55955  file2

If you have many files that you need to check, for example if you are transferring a directory full of files from one system to another, you can redirect the output from the original system to a file, then md5sum/sha1sum can automatically use that file to tell you which files are different:

$ md5sum file1 file2 > MD5SUMS
... copy file1, file2, MD5SUMS across
$ md5sum --check MD5SUMS
file1: OK
file2: OK
share|improve this answer
MD5 is not always a reliable for this:… – Jon Cage Apr 29 '10 at 12:30
Actually, MD5 is reliable for basic integrity checks. It just isn't considered as cryptographically strong as we once thought. If you are worried about hackers, don't use MD5, but if you just want to know if some files copied from a CD got corrupted, or if your compiler spits out identical files every time, MD5 is more than adequate. – Adam Batkin Apr 29 '10 at 12:55
up vote 10 down vote accepted

Found a solution - the cmp tool which comes with most Linux flavours.

share|improve this answer

If I just want to know if they are the same, I prefer to use sha1sum if it's available, or md5 as a fallback.

If I want to know how different they are, or where they're different, one thing that works is to crank them both through od ('octal dump', which usually has a hex option) to make temporary files and then diff those.

share|improve this answer
If you want to know whether they are two files are the same, I don't think using sha1sum (or md5sum for that matter) can be more efficient than just diff (as this was the original question), because even if the two (large) files differ at the very beginning), you will read both of them entirely before knowing they differ. – Pierre Apr 10 '14 at 21:29
@Pierre BUT, hashing and crypto-signing works across remote devices. – Vasya Novikov Dec 15 '15 at 17:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .