Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm having problems with grep filtering of make output. In particular,

make target 2>&1 | grep -E --color=never "^make.*"

works as expected, but the following will print no output to the console:

make target 2>&1 | grep -E --color=never "^make.*" | cat

Am I missing something obvious? Why does the first command output but not the second? Is it to do with some sort of IO buffering? Or am I just being stupid?

[EDIT]: cat is just a minimum test case placeholder for the actual command I want to use.

[EDIT]: This does not seem to be a problem with grep, as replacing it with ack leads to the same behaviour.

[EDIT]: Script that cat is a placeholder for:

cat - \
 | grep -E --color=never "^.*warning:.*|^.*error:.*|^make.*[Ee]rror.*|^make.*" \
 | -r "^.*warning:.*" -f yellow -B \
 | -r "^.*error:.*" -f red -B \
 | -r "^make.*[Ee]rror.*" -f red -B \
 | -r "^make.*" -f magenta

[EDIT]: I think it's a buffering/IO issue. I'm leaving the build running over the w/e and will see if it eventually gets the output where it needs to!

share|improve this question
up vote 3 down vote accepted

Likely grep is detecting that it's not writing to a TTY, so it buffers more output, rather than operating in its usual line buffered mode (i.e. you see each line as it's found). This behavior is more efficient as it results in fewer write() system calls, but if you're a user who's waiting for pipelined output, then it may be a bit misleading.

If you're using GNU grep (and based on your linux tag I'm assuming you are), check out the --line-buffered option, which will force grep to work in the more familiar line buffered mode. This could technically decrease the performance of grep (as noted in the man page), but since you're looking at the output of a live build, I doubt it will make any difference in that regard.

share|improve this answer
Thanks, that fixed it. – brice Feb 6 '12 at 11:09
Woah, this answer SAVED me. Thanks! – pr1001 Aug 21 '12 at 21:49

Not sure what your command is supposed to do, cat any files that have make.* in their file name?

Try changing the cat for xargs cat?

share|improve this answer
Cat is just a placeholder for another set of commands. It fails to output anything regardless of what I replace it with. – brice Feb 3 '12 at 17:17
then try cat `make target 2>&1 | grep -E --color=never "^make.*"` – jwbensley Feb 3 '12 at 17:24
That didn't help. (aside: the make command takes several minutes) – brice Feb 3 '12 at 17:30
What about if you turned it into a for loop? for file in `make target 2>&1 | grep -E --color=never "^make.*"`; do cat $file; done – jwbensley Feb 3 '12 at 17:32
Javano: I'm actually trying to build a highlighting filter for make, rather than do something with the filename. I'm not iterating over lines as much as editing a bytestream. (besides the fact that make output could not be used like this. I've added the script that I use instead of cat for you to get an idea of what I mean. – brice Feb 3 '12 at 17:39

You might try experimenting with unbuffer, e.g.,

make target 2>&1 | unbuffer -p grep -E --color=never "^make.*" | cat
share|improve this answer
I'd not heard of unbuffer before. I'll check it out. it could come in handy. – brice Feb 6 '12 at 11:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .