2

I have 2 TCL representations of a project that get generated by 2 different versions of the same tool, let's call them v1.tcl and v2.tcl

These logs are normally logically identical for all intents and purposes, with the exception of the order of the lines. Every line in v1.tcl will be found somewhere exactly once in v2.tcl when versions 1 and 2 of my project are identical.

I'd like to be able to identify whether someone has made a change in v2.tcl that needs to be backported to v1.tcl (or vice-versa) ... in other words I'd like to only see the lines in which don't match. For example:

  1. v1.tcl:

    foo1
    bar1
    hello1
    world1
    
  2. v2.tcl:

    hello1
    bar1
    foo2
    world1
    goodbye2
    
  3. "diff" return value:

    file1:1 foo1
    file2:3 foo2
    file2:5 goodbye2
    

Should I just write my own little script? Is there a tool that already does this?

4
  • @PimpJuiceIT I tired this, and since it's code, I sometimes get errors about grep: Trailing backslash. I'll check the man pages to see if I can get it to treat those as text... Sep 27 '18 at 4:03
  • @PimpJuiceIT just running grep -vf v2.tcl v1.tcl errors with "trailing backslash", even if I redirect it somewhere like above. Sep 27 '18 at 4:10
  • 1
    @PimpJuiceIT If the input files have lines ending in backslashes, grep gives the trailing backslash errors. I used grep's -F to fix those, and added your much shorter comment/answer to the end of mine. FYI I didn't use all awk solution, but did add bash/cut/grep to show the filename & lines
    – Xen2050
    Sep 28 '18 at 0:42
  • @Xen2050 Very nice solution, good explanation on all that too. I didn't have any Linux to ssh into to test myself from Linux so did the grep I commented on via Windows GNU packages or whatever. Sep 28 '18 at 1:48
5

If the lines are identical, and you only want to know if there's extra lines in one file or another, you could use sort & diff (and Process Substitution here):

$ diff -B <(sort v1.tcl) <(sort v2.tcl)
2c2,3
< foo1
---
> foo2
> goodbye2

With diff's -B to ignore blank lines. You could then use grep -n [pattern] file to find which line the pattern is at (maybe with one or a combination of grep, cut, sed, awk), if that matters.

Here's a more complete answer, showing the file and line number containing matches. Doesn't use sed or awk, just bash, cut, grep... here's it all (essentially) in one line:

diff -B <(sort v1.tcl) <(sort v2.tcl) | while read -r line; do if \
echo "$line" | grep -q "^<"; then grep -F -n -H \
"$(echo "$line"|cut -c3-)" v1.tcl ; elif echo "$line" | grep -q \
"^>"; then grep -F -n -H "$(echo "$line"|cut -c3-)" v2.tcl ; fi done

Or split into multiple lines:

diff -B <(sort v1.tcl) <(sort v2.tcl) | while read -r line
do
  if echo "$line" | grep -q "^<"
    then grep -F -n -H "$(echo "$line"|cut -c3-)" v1.tcl
  elif echo "$line" | grep -q "^>"
    then grep -F -n -H  "$(echo "$line"|cut -c3-)" v2.tcl
  fi
done

And depending on your input files (especially if you have lines with trailing backslashes), I'm using these options for read & grep:

  • read -r do not allow backslashes to escape any characters
  • grep -F Interpret PATTERN as a list of fixed strings (instead of regular expressions), separated by newlines, any of which is to be matched

Also, using Pimp Juice IT's comments, if there's trailing backslashes to lines in the input files grep gives the "file:line Trailing backslash" error. Using the -F option for grep to clear the trailing backslash error results in a much smaller grep-only solution:

grep -FvHn -f v2.tcl v1.tcl ;grep -FvHn -f v1.tcl v2.tcl

grep's options used:

  • -f Obtain patterns from FILE, one per line.
  • -F Interpret PATTERN as a list of fixed strings (instead of regular expressions), separated by newlines, any of which is to be matched
  • -v Invert the sense of matching, to select non-matching lines.
  • -H Print the file name for each match
  • -n Prefix each line of output with the 1-based line number within its input file.
1
  • Welcome :) The trailing backslashes was causing problems with my first draft (no -F & I think `-r``) answer too, though the sample text worked ok, and extra blank lines were causing weird problems also. But now everything seems fine
    – Xen2050
    Sep 29 '18 at 4:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.