I'm trying to pipe an active stream to another command (maybe a bash script), but it always wants to wait for the entire stream to finish before it processes the command. s.g. I can do this:

tail file.log | grep keyword | xargs -L 1 echo ...

and it correctly outputs the matching rows from file.log, preceded by "...". Obviously I don't want to just echo, that's just to simplify the example (I would replace that by a custom bash script). Now, when I try this:

tail -f file.log | grep keyword

this works, in that is outputs matching rows, then pauses until file.log changes, then it outputs any new matching lines. However, if I try this:

tail -f file.log | grep keyword | xargs -L 1 echo ...

It outputs nothing - I presume it's waiting for the stream opened by tail -f to end. I have tried this as well:

tail -f file.log | grep keyword | while read line; do echo...$line; end

with no change in behavior. I've read on other questions here that this should work. I'm running Ubuntu 14.04, is there some difference because of that? How can I get this to work?


You are right, xargs expects input from standard input, hence it is waiting for the stream to end. One way to make it work is by means of process substitution, because ...

Process substitution is a form of redirection where the input or output of a process (some sequence of commands) appear as a temporary file.

So you should try:

while IFS= read -r newline; do echo $newline | grep keyword | cat >> filename  ; done < <(tail -f file.log)

which dispenses with xargs altogether.

  • That works perfectly! I had seen the "< <(...)" syntax elsewhere, but didn't understand what it meant (or why the space in the middle was important). Thanks for the explanation! – lalitm Feb 12 '16 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.