I need to compare two binary files and get the output in the form:

<fileoffset-hex> <file1-byte-hex> <file2-byte-hex>

for every different byte. So if file1.bin is

  00 90 00 11

in binary form and file2.bin is

  00 91 00 10

I want to get something like

  00000001 90 91
  00000003 11 10

Is there a way to do this in Linux? I know about cmp -l but it uses a decimal system for offsets and octal for bytes which I would like to avoid.

  • 9
    you're basically looking for "binary diff". i can imagine some reeeally ugly commandline one-liner with od... – quack quixote Mar 29 '10 at 15:36
  • 2
    @quack quixote: What's ugly about a one-liner? ;) – Bobby Mar 29 '10 at 16:50
  • xdelta.org works quite well. Perhaps it'd be worth having a look at it. – thatjuan Oct 10 '13 at 5:31
  • Because you can't answer this question (as you're not a user), I'm voting to close. A binary diff as explicitly requested here isn't at all useful, and I'm inclined to think you want something useful, if you insert one byte at the start of the file should all bytes be marked as being different? Without knowing that, this is simply too vague. – Evan Carroll Nov 9 at 4:02
  • 1
    @EvanCarroll If you think the question is off topic why are you answering it? – DavidPostill Nov 26 at 21:07

14 Answers 14

up vote 153 down vote accepted

This will print the offset and bytes in hex:

cmp -l file1.bin file2.bin | gawk '{printf "%08X %02X %02X\n", $1, strtonum(0$2), strtonum(0$3)}'

Or do $1-1 to have the first printed offset start at 0.

cmp -l file1.bin file2.bin | gawk '{printf "%08X %02X %02X\n", $1-1, strtonum(0$2), strtonum(0$3)}'

Unfortunately, strtonum() is specific to GAWK, so for other versions of awk—e.g., mawk—you will need to use an octal-to-decimal conversion function. For example,

cmp -l file1.bin file2.bin | mawk 'function oct2dec(oct,     dec) {for (i = 1; i <= length(oct); i++) {dec *= 8; dec += substr(oct, i, 1)}; return dec} {printf "%08X %02X %02X\n", $1, oct2dec($2), oct2dec($3)}'

Broken out for readability:

cmp -l file1.bin file2.bin |
    mawk 'function oct2dec(oct,    dec) {
              for (i = 1; i <= length(oct); i++) {
                  dec *= 8;
                  dec += substr(oct, i, 1)
              };
              return dec
          }
          {
              printf "%08X %02X %02X\n", $1, oct2dec($2), oct2dec($3)
          }'
  • 3
    @gertvdijk: strtonum is specific to GAWK. I believe Ubuntu previously used GAWK as the default, but switched at some point to mawk. In any case, GAWK can be installed and set to the default (see also man update-alternatives). See my updated answer for a solution that doesn't require strtonum. – Dennis Williamson Jul 4 '13 at 18:08

As ~quack pointed out:

 % xxd b1 > b1.hex
 % xxd b2 > b2.hex

And then

 % diff b1.hex b2.hex

or

 % vimdiff b1.hex b2.hex
  • 61
    In Bash: diff <(xxd b1) <(xxd b2) but the output format of this (or yours) is nowhere near what the OP asked for. – Dennis Williamson Mar 29 '10 at 16:33
  • 6
    with vimdiff it is, it will color the bytes in the lines where the two 'files' differ – akira Mar 30 '10 at 4:45
  • Aww, why didn't I think of that? And I'm sure I've used this technique in the past too. – njd Mar 30 '10 at 17:37
  • 1
    This worked great for me (with opendiff on OS X instead of vimdiff) — the default view xxd provides keeps the diff engine on track comparing byte-by-byte. With plain (raw) hex simply column-fit with fold, diff would try to fold/group random stuff in the files I was comparing. – natevw Nov 15 '14 at 23:26
  • 1
    This command does not work well for byte addition removal, as every line that follows will be misaligned and seen as modified by diff. The solution is to put 1 byte per line and remove the address column as proposed by John Lawrence Aspden and me. – Ciro Santilli 新疆改造中心 六四事件 法轮功 Apr 4 '15 at 20:38

Try diff in the following combination of zsh/bash process substitution and colordiff in CLI:

diff -y <(xxd foo1.bin) <(xxd foo2.bin) | colordiff

Where:

  • -y shows you differences side-by-side (optional)
  • xxd is CLI tool to create a hexdump output of the binary file
  • colordiff will colorize diff output (install via: sudo apt-get install colordiff)
  • add -W200 to diff for wider output

Hints:

  • if files are to big, add limit (e.g. -l1000) for each xxd

Sample output:

binary file output in terminal - diff -y <(xxd foo1.bin) <(xxd foo2.bin) | colordiff

  • 7
    Command can be simplified as colordiff -y <(xxd foo1.bin) <(xxd foo2.bin). – golem Nov 17 '15 at 22:46
  • 3
    If you don't have colordiff, this will do the same thing without colors: diff -y <(xxd foo1.bin) <(xxd foo2.bin) – Rock Lee Aug 4 '16 at 15:25
  • 4
    If you just want to know whether both files are actually the same, you can use the -q or --brief switch, which will only show output when the files differ. – Stefan van den Akker Oct 8 '16 at 11:14
  • 1
    create a function xxddiff for this with: xxddiff() ( f() ( xxd "$1" ; ); diff -y <(f "$1") <(f "$2") | colordiff; ) – rubo77 Nov 14 '16 at 6:35
  • 1
    great! still, diff -u <(xxd tinga.tgz) <(xxd dec.out.tinga.tgz) | vim - will do a job good enoug – ribamar Jun 1 at 14:27

There's a tool called DHEX which may do the job, and there's another tool called VBinDiff.

For a strictly command-line approach, try JDIFF.

  • 8
    DHEX is awesome is comparing binaries is what you want to do. Feed it two files and it takes you right to a comparative view, highlighting to differences, with easy ability to move to the next difference. Also it's able to work with large terminals, which is very useful on widescreen monitors. – Marcin Sep 8 '11 at 0:08
  • 6
    I prefer VBinDiff. DHEX is using CPU even when idling, I think it's redrawing all the time or something. VBinDiff doesn't work with wide terminals though. But the addresses become weird with wide terminals anyway, since you have more than 16 bytes per row. – Janus Troelsen Oct 17 '12 at 14:22
  • vbindiff lets us actually edit the file, thx! – Aquarius Power Sep 16 '14 at 18:28
  • 2
    @DanielBeauyat compressed files will be completely different after you encounter the first different byte. The output is not likely to be useful. – Mark Ransom Aug 5 '15 at 3:12
  • 2
    @1111161171159459134 jdiff is part of a "suite" of programs to sync and patch the differences found by jdiff. But, as Mark Ransom said, that would be generally not wise on compressed files; the exception is "synchronizable" compressed formats (like that produced by gzip --rsyncable), in which small differences in the uncompressed files should have a limited effect on the compressed file. – hmijail Feb 13 '16 at 11:12

Method that works for byte addition / deletion

diff <(od -An -tx1 -w1 -v file1) \
     <(od -An -tx1 -w1 -v file2)

Generate a test case with a single removal of byte 64:

for i in `seq 128`; do printf "%02x" "$i"; done | xxd -r -p > file1
for i in `seq 128`; do if [ "$i" -ne 64 ]; then printf "%02x" $i; fi; done | xxd -r -p > file2

Output:

64d63
<  40

If you also want to see the ASCII version of the character:

bdiff() (
  f() (
    od -An -tx1c -w1 -v "$1" | paste -d '' - -
  )
  diff <(f "$1") <(f "$2")
)

bdiff file1 file2

Output:

64d63
<   40   @

Tested on Ubuntu 16.04.

I prefer od over xxd because:

  • it is POSIX, xxd is not (comes with Vim)
  • has the -An to remove the address column without awk.

Command explanation:

  • -An removes the address column. This is important otherwise all lines would differ after a byte addition / removal.
  • -w1 puts one byte per line, so that diff can consume it. It is crucial to have one byte per line, or else every line after a deletion would become out of phase and differ. Unfortunately, this is not POSIX, but present in GNU.
  • -tx1 is the representation you want, change to any possible value, as long as you keep 1 byte per line.
  • -v prevents asterisk repetition abbreviation * which might interfere with the diff
  • paste -d '' - - joins every two lines. We need it because the hex and ASCII go into separate adjacent lines. Taken from: https://stackoverflow.com/questions/8987257/concatenating-every-other-line-with-the-next
  • we use parenthesis () to define bdiff instead of {} to limit the scope of the inner function f, see also: https://stackoverflow.com/questions/8426077/how-to-define-a-function-inside-another-function-in-bash

See also:

Short answer

vimdiff <(xxd -c1 -p first.bin) <(xxd -c1 -p second.bin)

When using hexdumps and text diff to compare binary files, especially xxd, the additions and removals of bytes become shifts in addressing which might make it difficult to see. This method tells xxd to not output addresses, and to output only one byte per line, which in turn shows exactly which bytes were changed, added, or removed. You can find the addresses later by searching for the interesting sequences of bytes in a more "normal" hexdump (output of xxd first.bin).

  • (Of course, one may use diff instead of vimdiff.) – VasyaNovikov Dec 15 '15 at 17:35

I'd recommend hexdump for dumping binary files to textual format and kdiff3 for diff viewing.

hexdump myfile1.bin > myfile1.hex
hexdump myfile2.bin > myfile2.hex
kdiff3 myfile1.hex myfile2.hex
  • 2
    Even here in bash kdiff3 <(hexdump myfile1.bin) <(hexdump myfile2.bin) with no need to create files myfile1.hex and myfile2.hex. – Hastur Jan 25 '16 at 14:34

The hexdiff is a program designed to do exactly what you're looking for.

Usage:

hexdiff file1 file2

It displays the hex (and 7-bit ASCII) of the two files one above the other, with any differences highlighted. Look at man hexdiff for the commands to move around in the file, and a simple q will quit.

  • 4
    But it does a pretty bad job when it comes to the comparing part. If you insert some bytes into a file, it will mark all byte afterwards as changes – Murmel Apr 27 '16 at 19:43
  • and hexdiff is not available via apt-get on Ubuntu 16.4 – rubo77 Nov 14 '16 at 6:38
  • @Murmel while I agree, isn't that what's being asked here? – Evan Carroll Nov 9 at 4:00
  • @EvanCarroll true, and hence I left a comment (only) and did not downvote – Murmel Nov 9 at 17:52
  • I also didn't down vote Mick, but I agree with you and answered here superuser.com/a/1373977/11116 because it seems likely that this bad question will get reformed or closed. – Evan Carroll Nov 9 at 19:06

It may not strictly answer the question, but I use this for diffing binaries:

gvim -d <(xxd -c 1 ~/file1.bin | awk '{print $2, $3}') <(xxd -c 1 ~/file2.bin | awk '{print $2, $3}')

It prints both files out as hex and ASCII values, one byte per line, and then uses Vim's diff facility to render them visually.

dhex http://www.dettus.net/dhex/

DHEX is a more than just another hex editor: It includes a diff mode, which can be used to easily and conveniently compare two binary files. Since it is based on ncurses and is themeable, it can run on any number of systems and scenarios. With its utilization of search logs, it is possible to track changes in different iterations of files easily.

  • Welcome to SuperUser! Although this software looks like it could solve the OP's problem, pure advertisement is strongly frowned upon on the Stack Exchange network. If you are affiliated to this software's editor, please disclose this fact. And try to rewrite your post so that it looks less like a commercial. Thank you. – Nathan.Eilisha Shiraini Aug 18 '17 at 13:31
  • I am not affiliated with dhex in any way. I copied the author's description into the post because there is minimum post length limit – Vincent Vega Aug 19 '17 at 13:59

You can use gvimdiff tool that is included in vim-gui-common package

sudo apt-get update

sudo apt-get install vim-gui-common

Then you can compare 2 hex files using following commands :

ubuntu> gvimdiff <hex-file1> <hex-file2>

Tha's all. Hope tha help !

https://security.googleblog.com/2016/03/bindiff-now-available-for-free.html

BinDiff is a great UI tool for comparing binary files that has been open sourced recently.

  • 3
    Can it be used on arbitrary binary files, though? That page seems to indicate that it's only useful for comparing executables that have been disassembled by Hex-Rays IDA Pro. – eswald Apr 29 '16 at 22:57

The go to open source product on Linux (and everything else) is Radare which provides radiff2 explicitly for this purpose. I voted to close this because myself and others have the same question, in the question you ask

for every different byte

That's insane though. Because as asked, if you insert one byte at the first byte in the file, you'd find every subsequent byte was different and so the diff would repeat the whole file, for an actual difference of one byte.

Slightly more practical is radiff -O. The -O is for ""Do code diffing with all bytes instead of just the fixed opcode bytes""

0x000000a4 0c01 => 3802 0x000000a4
0x000000a8 1401 => 3802 0x000000a8
0x000000ac 06 => 05 0x000000ac
0x000000b4 02 => 01 0x000000b4
0x000000b8 4c05 => 0020 0x000000b8
0x000000bc 4c95 => 00a0 0x000000bc
0x000000c0 4c95 => 00a0 0x000000c0

Like IDA Pro, Radare is a tool primary for binary analysis, you can also show delta diffing with -d, or display the disassembled bytes instead of hex with -D.

If you're asking these kind of questions though, check out

I recommend IDA Pro to analyse the binary files. Comparison can then be done using a plug-in for IDA such as BinDiff.

  • 1
    I would have to buy to test it right? – Aquarius Power Sep 16 '14 at 18:17
  • 1
    @AquariusPower Not necessarily, IDA Pro does have a Free Version – user3119546 Sep 19 '14 at 1:19
  • 7
    but it is not for linux! :( – Aquarius Power Sep 19 '14 at 20:08
  • I've used quite a few different free versions of IDA Pro under Wine without issues, if I recall correctly. – doshea May 17 '15 at 5:42
  • This has been open sourced recently security.googleblog.com/2016/03/… – Evgeny Mar 23 '16 at 20:17

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.