1

I have three binary files (memory dumps). Call them file1, file2, file3. I'm trying to debug some software, and I'm toggling a switch.

  • file1 = switch off

  • file2 = switch on

  • file3 = switch off

I need to know which bytes changed between file1 and file2, which then also changed back to the same values (as file1) in file3.

There are a lot of unrelated changes between file1, file2 so that diff alone is not enough for me to determine what is changing when I toggle this switch, I'm trying to identify the unique bytes of entropy that change from file1,2,3,…

I know there are tools such as xxd, diff, vimdiff, colordiff. I'm just not sure how best to use them for this problem.

  • 1
    "diff alone is not enough" – General note: tools that work with text (lines) cannot handle binary (i.e. non-text) input nicely. Even if they can in some cases, their output may not be what you expect. – Kamil Maciorowski Jun 25 at 0:33
  • Possible duplicate of How do I compare binary files in Linux? – Scott Jun 25 at 0:33
  • 1
    Possible duplicate of Three way binary comparison for savegame hacking.   Unfortunately, the only answer is a program that’s not free ($30 through $60, depending). – Scott Jun 25 at 0:33
  • Possible duplicate of diff for multiple files. – Scott Jun 25 at 0:33
  • I don’t fully understand what you want. Can you give an example? It should be long enough to illustrate all the aspects that are important, but short enough to be comprehensible.  See this for some guidance. – Scott Jun 25 at 0:34
0

To know bytes that changed between file1 and file2 and their respective values, use cmp -l:

cmp -l file1 file2 > changes12

Similarly for file3 and file2. The trick is to always investigate in the same direction (here: switch off to on), that's why I put file2 at the end:

cmp -l file3 file2 > changes32

Now you can find identical changes:

comm -12 changes12 changes32

and the output will be like (example):

     1629 152 112

which means the byte 1629 (decimal, numbering starts with 1) changed from 152 (octal) to 112 (octal).


Notes:

  • cmp -l in my Ubuntu "columnizes" its output. This means it may print lines with leading spaces, the width of the first column depends on the input size. I guess some implementations may not do this. There are few concerns:
    • If the original files are of different size (probably not your case), one cmp may generate wider first column than the other. In the context of later comm this is unacceptable. You can "decolumnize" the output by piping to awk '{print $1" "$2" "$3}'.
    • If the output is not "columnized" (or has been "decolumnized"), comm may complain the files are not sorted. You need sort (not sort -n) before you save to changesAB. This may generate somewhat unexpected order (e.g. differing byte 23 will appear after differing byte 100453) which may be fixed by piping the output of comm to sort -n.
  • changes* files may be huge. They are intermediate and temporary, so process substitution may be a good approach. This is non-POSIX though:

    # Korn shell syntax example
    comm -12 <(cmp -l file1 file2) <(cmp -l file3 file2)
    
  • The output from comm can be used with yet another and another output of cmp to filter out unrelated changes better:

    comm -12 changes12 changes32 > result1
    cmp -l file4 file5 | comm -12 - result1 > result2
    cmp -l file6 file5 | comm -12 - result2 > result3
    

    But remember:


Documentation:

  • Thank you Kamil. This is a very helpful response. – authur1234 Jul 2 at 11:37
0

I'm a Windows user and I've used Beyond Compare for many years for file compares, including comparing 3 files (2 at a time) to each other.

It appears that Beyond Compare also has a Linux distro, so you might want to check it out.

https://www.scootersoftware.com/download.php?zz=kb_linux_install

I know the Windows version has options to show just differences, as well as a lot of other features that might help you out. I can't remember if they have a 3 way compare or not.

I'm not dev or sales for them, I just like the software a lot.

0

Based on the response from Kamil, I used this to get what I needed:

cmp -l file1 file2 | awk '{print $1" "$2" "$3}' | sort > changes_12

cmp -l file3 file2 | awk '{print $1" "$2" "$3}' | sort > changes_32

comm -12 changes_12 changes_32 > common_changes

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.