Grepping grep output fails

Question

I'm trying to grep the output of ngrep. Unfortunately when I add another grep to the pipeline, I get no output at all. It can be some other command too - cat / grep / tee - everything breaks the chain. Example:

# this works:
$ ngrep -l -q -T -Wbyline -d any udp and port 5060 |
    egrep -B1 '^SIP/2.0 180'
--
U +1.469535 xxx:5060 -> xxx:5060
SIP/2.0 180 Ringing.
--
U +0.001384 xxx:5060 -> xxx:2048
SIP/2.0 180 Ringing.

but

#these don't:
$ ngrep -l -q -T -Wbyline -d any udp and port 5060 |
    egrep -B1 '^SIP/2.0 180' | egrep '^U'
$ ngrep -l -q -T -Wbyline -d any udp and port 5060 |
    egrep -B1 '^SIP/2.0 180' | cat
$ ngrep -l -q -T -Wbyline -d any udp and port 5060 |
    egrep -B1 '^SIP/2.0 180' | tee test

If I use cat somefile instead of ngrep at the start, everything works as expected. Any ideas what could go wrong here?

Answer 1

Have you got egrep aliased to anything that might modify the text, like grep --color might do? $ type egrep

Also check the output for "hidden" control characters using od -bc (as jch mentioned) or hd -C.

Answer 2

EDIT: I just ran into this myself with 'egrep' - was doing:

tail -f somefile|egrep 'somepattern'

Which would provide gratifying output every two seconds; but the following produced nothing:

tail -f somefile|egrep 'somepattern'|tr -d '%'

After some digging in the man page I found the '--line-buffered' option which produced output again!

Then I found this description of buffering in pipelines - it looks like what it boils down to is that some command line utils (eg: tail -f) routinely call fflush on stdout and others (cut, grep, etc) do not.

My wrong first reply is below

---

Usually when I run into these types of issues, one or more of the programs in the pipeline are putting output on multiple file descriptors - most often STDERR.

You might try adding '2>&1' before your first '|' in the pipeline to redirect STDERR to STDOUT.