Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I need to compare two binary files and get the output in the form

<fileoffset-hex> <file1-byte-hex> <file2-byte-hex>

for every different byte. So if file1.bin is

  00 90 00 11

in binary form and file2.bin is

  00 91 00 10

I want to get something like

  00000001 90 91
  00000003 11 10

What is the easiest way to accomplish the goal? Standard tool? Some third-party tool?

(Note: cmp -l should be killed with fire, it uses a decimal system for offsets and octal for bytes.)

share|improve this question
6  
you're basically looking for "binary diff". i can imagine some reeeally ugly commandline one-liner with od... –  quack quixote Mar 29 '10 at 15:36
1  
@quack quixote: What's ugly about a one-liner? ;) –  Bobby Mar 29 '10 at 16:50
    
xdelta.org works quite well. Perhaps it'd be worth having a look at it. –  thatjuan Oct 10 '13 at 5:31

8 Answers 8

up vote 63 down vote accepted

This will print the offset and bytes in hex:

cmp -l file1.bin file2.bin | gawk '{printf "%08X %02X %02X\n", $1, strtonum(0$2), strtonum(0$3)}'

Or do $1-1 to have the first printed offset start at 0.

cmp -l file1.bin file2.bin | gawk '{printf "%08X %02X %02X\n", $1-1, strtonum(0$2), strtonum(0$3)}'

Unfortunately, strtonum() is specific to GAWK, so you will need to use an octal-to-decimal conversion function. For example,

cmp -l file1.bin file2.bin | mawk 'function oct2dec(oct,     dec) {for (i = 1; i <= length(oct); i++) {dec *= 8; dec += substr(oct, i, 1)}; return dec} {printf "%08X %02X %02X\n", $1, oct2dec($2), oct2dec($3)}'

Broken out for readability:

cmp -l file1.bin file2.bin | 
mawk 'function oct2dec(oct,     dec) {
          for (i = 1; i <= length(oct); i++) {
              dec *= 8;
              dec += substr(oct, i, 1)
          };
          return dec
      }
      {
          printf "%08X %02X %02X\n", $1, oct2dec($2), oct2dec($3)
      }'
share|improve this answer
    
Unfortunately, this gives me awk: line 2: function strtonum never defined errors on Ubuntu 12.04. Specific AWK implementation perhaps? –  gertvdijk Jul 4 '13 at 15:59
1  
@gertvdijk: strtonum is specific to GAWK. I believe Ubuntu previously used GAWK as the default, but switched at some point to mawk. In any case, GAWK can be installed and set to the default (see also man update-alternatives). See my updated answer for a solution that doesn't require strtonum. –  Dennis Williamson Jul 4 '13 at 18:08
    
I added printing the original character and put it into the mc menu, I had to double the % signs: cmp -l %d/%f %D/%f | gawk '{printf "%%08X %%02X %%02X %%c %%c\n", $1-1, strtonum(0$2), strtonum(0$3), strtonum(0$2), strtonum(0$3)}' –  18446744073709551615 Apr 22 '14 at 10:10

Short answer

vimdiff <(xxd -c1 -p first.bin) <(xxd -c1 -p second.bin)

When using hexdumps and text diff to compare binary files, especially xxd, the additions and removals of bytes become shifts in addressing which might make it difficult to see. This method tells xxd to not output addresses, and to output only one byte per line, which in turn shows exactly which bytes were changed, added, or removed. You can find the addresses later by searching for the interesting sequences of bytes in a more "normal" hexdump (output of xxd first.bin).

share|improve this answer

Alternative to John Lawrence Aspden's answer:

diff <(od -An -tx1 -w1 file1) <(od -An -tx1 -w1 file2)

Output for a single NUL byte removal at the 100th byte of a large file:

100d99
< 00

Advantages over other proposed answers so far:

  • works well on byte addition / removal
  • od is POSIX, xxd is not (comes with Vim), and has -An to remove the address column without awk.

Command explanation:

  • -An removes the address column. This is important otherwise all lines would differ after a byte addition / removal.
  • -w1 puts one byte per line, so that diff can consume it.
  • -tx1 is the representation you want, change to any possible value, as long as you keep 1 byte per line.
share|improve this answer

There's a tool called DHEX which may do the job, and there's another tool called VBinDiff.

For a strictly command-line approach, try JDIFF.

share|improve this answer
4  
DHEX is awesome is comparing binaries is what you want to do. Feed it two files and it takes you right to a comparative view, highlighting to differences, with easy ability to move to the next difference. Also it's able to work with large terminals, which is very useful on widescreen monitors. –  Marcin Sep 8 '11 at 0:08
1  
I prefer VBinDiff. DHEX is using CPU even when idling, I think it's redrawing all the time or something. VBinDiff doesn't work with wide terminals though. But the addresses become weird with wide terminals anyway, since you have more than 16 bytes per row. –  Janus Troelsen Oct 17 '12 at 14:22
    
vbindiff lets us actually edit the file, thx! –  Aquarius Power Sep 16 '14 at 18:28
    
+1 But why do they say in jdiff -h: "Do not use jdiff directly on compressed files, such as zip, gzip, rar, ..." ?? –  Daniel Beauyat 2 days ago

It may not strictly answer the question, but I use this for diffing binaries:

gvim -d <(xxd -c 1 ~/file1.bin | awk '{print $2, $3}') <(xxd -c 1 ~/file2.bin | awk '{print $2, $3}')

It prints both files out as hex and ASCII values, one byte per line, and then uses Vim's diff facility to render them visually.

share|improve this answer

As ~quack (hehe) pointed out:

 % xxd b1 > b1.hex
 % xxd b2 > b2.hex

And then

 % diff b1.hex b2.hex

or

 % vimdiff b1.hex b2.hex
share|improve this answer
22  
In Bash: diff <(xxd b1) <(xxd b2) but the output format of this (or yours) is nowhere near what the OP asked for. –  Dennis Williamson Mar 29 '10 at 16:33
    
with vimdiff it is, it will color the bytes in the lines where the two 'files' differ –  akira Mar 30 '10 at 4:45
    
Aww, why didn't I think of that? And I'm sure I've used this technique in the past too. –  njd Mar 30 '10 at 17:37
    
Nice. I'm on an embedded system that uses BusyBox and there is no cmp, but hexdump + diff works like a charm. –  Robert Calhoun Mar 22 '13 at 3:47
    
This worked great for me (with opendiff on OS X instead of vimdiff) — the default view xxd provides keeps the diff engine on track comparing byte-by-byte. With plain (raw) hex simply column-fit with fold, diff would try to fold/group random stuff in the files I was comparing. –  natevw Nov 15 '14 at 23:26

I recommend IDA Pro to analyse the binary files. Comparison can then be done using a plug-in for IDA such as BinDiff.

share|improve this answer
    
I would have to buy to test it right? –  Aquarius Power Sep 16 '14 at 18:17
1  
@AquariusPower Not necessarily, IDA Pro does have a Free Version –  user3119546 Sep 19 '14 at 1:19
2  
but it is not for linux! :( –  Aquarius Power Sep 19 '14 at 20:08
    
I've used quite a few different free versions of IDA Pro under Wine without issues, if I recall correctly. –  doshea May 17 at 5:42

I'd recommend hexdump for dumping binary files to textual format and kdiff3 for diff viewing.

hexdump myfile1.bin > myfile1.hex
hexdump myfile2.bin > myfile2.hex
kdiff3 myfile1.hex myfile2.hex
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.