20

I am currently working on a source code generation tool. To make sure that my changes do no introduce any new bugs, a diff between the output of the program before and after my changes would theoretically be a valuable tool.

However, this turns out to be harder than one might think, because the tool outputs lines where the order does not matter (like import statements, function declarations, …) in a semi-randomly ordered way. Because of this, the output of diff is cluttered with many changes that are in fact only lines moved to another position in the same file.

Is there a way to make diff ignore these moves and only output the lines that have really been added or removed?

4
  • Maybe it's easier to change your tool to generate functions and import declarations in a specific (e.g. lexicographical, if possible in your language) order?
    – Daniel Beck
    Dec 5, 2010 at 17:10
  • @Daniel Beck: See my comment to Gilles' answer below.
    – dnadlinger
    Dec 5, 2010 at 21:11
  • Old subject, but to summarize comments below, how would this diff tool be able to separate valid moves from invalid ones, as Order of instructions in code does matter, and cases where this is not true are limited (imports, declaration of functions and classes, etc.) ?
    – Joël
    Aug 4, 2014 at 12:05
  • @Joël: The answer is simply that I knew that the generator changes I had to test would not introduce any bugs related to changing the order of lines. Of course, you need an tool based on a parser for the target language to avoid false-positives in the general case (or simply a comprehensive test suite for your generator), but this was supposed to be a quick one-off check in addition to code review.
    – dnadlinger
    Aug 5, 2014 at 20:10

4 Answers 4

4

You could do a simple diff, store the result somewhere (to avoid another diff), loop through the lines in either version, then remove those from the other side.

This spawned a separate project for the working code. The code.

3
  • I'm not sure what that's supposed to do exactly, but it doesn't seem to produce the wanted results. As I understand the question, from the two examples in the code /tmp/old and /tmp/new no diff results would be wanted since there's just lines that have been moved around. This code however does produce results. Jan 7, 2011 at 14:54
  • Fixed the code.
    – l0b0
    Aug 24, 2011 at 13:37
  • Haven't tested the answer as I finished the merging process mentioned above a long time ago, but from a glance at the code it looks as if it could work.
    – dnadlinger
    Aug 5, 2014 at 20:04
9

You may try to sort'em first. Something like:

sort file-a > s-file-a
sort file-b > s-file-b
diff s-file-a s-file-b

Bash (and zsh) can do this in one line with process substitution

diff <(sort file-a) <(sort file-b)
4
  • 1
    This might be an option, but the generated diffs would not be very useful then, because I would lose all the line number and context information…
    – dnadlinger
    Sep 5, 2010 at 15:16
  • Even if I am still hoping for a better solution, I went with this approach for verifying the batch of changes I had been working on.
    – dnadlinger
    Dec 5, 2010 at 21:16
  • 2
    I can foresee where this would miss some changes. Sometimes order matters, sometimes it doesn't. You discard all context. Jun 26, 2012 at 21:30
  • For an ordering refactor where I wanted to make sure everything that did exist still does, this was exactly what I needed.
    – ntrrobng
    Apr 6, 2017 at 17:25
1

If the file is structured into sections, it is just the sections that are out of order, and there exists a regular expression that you can use to recognize the section header, you could csplit the files into their sections and then compare the sections pairwise.

For instance, I just did this on two MySQL dumps to compare them after some of the database names had changed case (and therefore the dump listed them in a different order):

csplit all-07sep2015-11:19:12.sql '/Current Database/-1' '{*}'  # split the dump made before the change, creating files xx00, xx01, ...
csplit -f yy all-07sep2015-12:26:12.sql '/Current Database/-1' '{*}' # split the dump made after the change, creating files yy00, yy01, ...
fgrep 'Current Database' xx?? yy?? | perl -lne 'BEGIN{my %foo}; /(^....).*`(.*)`/ and push(@{$foo{lc($2)}}, $1); END {printf("diff -di %s %s\n", @{$_}) for values %foo}' | sh -x | less  # match the pairs and compare them with diff
0

It sounds like you have control over the tool. Then make its output predictable: instead of emitting declarations in a semi-random order, use (say) alphabetical order as a last resort. This will not only have the benefit of removing useless cruft from diffs, but also of making the tool's output easier to read and verify for a human being.

1
  • Sorry, but this answer does not help me at all – if it was so easy, I would change it right away. Furthermore, I am currently merging changes from a project the generator was originally forked from, so adding such a rather far-reaching change would complicate that process even more…
    – dnadlinger
    Sep 5, 2010 at 17:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .