How to search ocurrence of a word and absence of another using grep

Question

I have a file with content similar to:

google.com,9,AB+CD,nonAB+nonCD
youtube.com,9,AB+CD,AB+CD
facebook.com,20,AB+CD,nonCD

The number of columns is not fixed. But the first column is a URL, the second is a number, starting form the third is keywords separated by commas but they vary from site to another.

I want to count the number of URLs (lines) where I can control what keywords are in the line. For example,
1) AB+CD without nonAB and nonCD. Note: the word AB+CD can occur many times. 2) AB+CD without occurrence of nonCD (but it is ok if there is anything else)

How to search for a string in a line AND ensure the absence of another string. When I use:

grep 'AB+CD' test.txt > result.txt

It prints every line where 'AB+CD' is found.

What if I want to print the line where there is only 'AB+CD' to get:

youtube.com,9,AB+CD,AB+CD

Or there is 'AB+CD' with anything else except 'nonAB' to get:

youtube.com,9,AB+CD,AB+CD
facebook.com,20,AB+CD,nonCD

Answer 1

If you just wanted a plain text search without caring about columns, you can chain the inverted match grep -v like so:

cat input.txt | grep 'IncludedText' | grep -v 'ExcludedText'

If you want to do proper filtering by column, you'd want to use something like awk.

Answer 2

General tricks:

1. Lines containing foo plus lines containing bar (foo OR bar):

   grep -e foo -e bar
   

2. Lines containing foo and bar in the same line (foo AND bar):

   grep foo | grep bar
   

3. Lines not containing baz (NOT baz):

   grep -v baz
   

With these bricks you can build your logic. The problem is -v is not restricted to a single pattern, it's global to entire grep (at least in my Debian). This makes NOT (foo OR bar) possible:

grep -v -e foo -e bar

which is equivalent to (NOT foo) AND (NOT bar):

grep -v foo | grep -v bar

However NOT (foo AND bar) (logically equivalent to (NOT foo) OR (NOT bar)) is not that easy. We can try to obtain foo AND bar with a single (extended) grep:

2. Again lines containing foo and bar in the same line (foo AND bar):

   grep -E 'foo.*bar|bar.*foo'
   

Now to get NOT (foo AND bar):

grep -v -E 'foo.*bar|bar.*foo'

---

I'm not sure if the above is a complete system when dealing with more than two patterns. Still few of your problems are solvable with it. Example:

AB+CD without nonAB and nonCD

If I get you right it's AB+CD AND NOT (nonAB OR nonCD)

grep AB+CD | grep -v -e nonAB -e nonCD

---

Notice this request makes things complicated:

I want to print the line where there is only 'AB+CD'

One may say grep ,AB+CD,AB+CD will do, but since "the number of columns is not fixed", I guess you'd like to tell apart these two lines:

youtube.com,9,AB+CD,AB+CD,AB+CD
youtube.com,9,AB+CD,AB+CD,banana

In such cases you need more complicated regular expressions or other tools (like awk).

Answer 3

While You will get answer here, you should have a look at man grep (can be overwhelming) and some examples. For the time being, here goes the answer :

Using grep

grep "foobar" test.txt

will search for lines having word foobar in file test.txt and display all occurrences whereas ,

grep "foo" -v "bar" test.txt

will search for lines having word foo but not bar. We get this because of -v switch for which manpage explains :

-v, --invert-match
    Invert the sense of matching, to select non-matching lines.
    (-v is specified by POSIX .)

It simply means that It will search for lines having those words ( here bar ), but will exclude them in final display. Thus inverting the search.

Also, To count number of lines that matches the search, use -c switch :

-c, --count
    Suppress normal output; instead print a count of matching lines
    for each input file. With the -v, --invert-match option (see below),
    count non-matching lines. (-c is specified by POSIX .)

As a self-exercise, try your hands upon grep search on file foobar.

The Answer

Search for AB+CD ignoring nonAB and nonCD and count URLs :

grep "AB+CD" test | grep -cve "non"

where -v "non" will simply ignore both nonAB and nonCD as they both have non in them. And -c will give total count for the matches instead of printing them. To print matching lines, just ignore -c.

You may use it for separate inverts :

grep "AB+CD" test | grep -cve "nonAB\|nonCD"

where \| represents OR and means either of nonAB or nonCD exact word specified by -e switch.

---

Would advise you to see Kamil's answer, read manpages (you know the command) as much you can, try hard while searching stuffs online & serve the community. Feel free to add-in more details to answer.