How do I grep OUT the content of a file from another file
Say I have a file "file1" and it has
a
b
c
and another file "file2" that has
a
b
c
d
e
f
I would usually do
cat file2 | grep -v a | grep -v b | grep -v c
Is there a way to do this with the files
grep -v file2 file1
Use diff to find the differences and sed to select, format and print just the added lines:
diff file1 file2 | sed -n "/^>/{s/> //;p}"
Added:
If the files are in different orders, then sort them both first, then diff them.
sort file1 > file1.s
sort file2 > file2.s
diff file1.s file2.s | sed -n "/^>/{s/> //;p}"
If you're using bash, you can use <(...) process substitution:
diff <(sort file1) <(sort file2) | sed -n "/^>/{s/> //;p}"
GNU grep (and I imagine others also) can take the -f option:
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty
file contains zero patterns, and therefore matches
nothing. (-f is specified by POSIX.)
So, you could do this:
grep -vFf file1 file2
Following Nicole Hamilton's suggestion in the comments, I have added the '-F' option which causes grep to interpret its PATTERN (in this case, each line in file1) as fixed strings and not regular expressions:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by
newlines, any of which is to be matched. (-F is specified by
POSIX.)
I have also written a PERL script with a few more options:
$ list_compare.pl -h
USAGE: compare_lists.pl FILE1 FILE2
This script will compare FILE1 and FILE2, searching for the
contents of FILE1 in FILE2 (and NOT vice-versa). FILE one must
be one search pattern per line, the search pattern need only be
contained within one of the lines of FILE2.
OPTIONS:
-c : Print patterns COMMON to both files
-f : Search only the first characters (until the 1st space) of each line of
FILE2 for the search pattern given in FILE1. So, if FILE1 contains
"foo bar", only "foo" will be taken as a pattern (MUCH faster).
-d : Print duplicate entries
-m : Print patterns MISSING in FILE2 (default)
-h : Print this help and exit
-f option is a pretty good idea. But you'll likely want to use fgrep not grep so the lines you read with -f will be treated as literal text, not regex patterns.
Dec 9, 2012 at 17:06
grep -f and the reason I wrote that Perl script in the first place! Thanks, I had not realized that it was treating the file's lines as regexes. Changed the question to use grep -Ff (fgrep is depreciated says man grep).