Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Greeting,

I am trying to pipe tail -f into awk to monitor a logfile in realtime, but according to examples, there should be not problem but I can't get it to work.

here is the command I'm running

tail -f logfile.log | awk -F" " '{print $1, $2, $7, $8}'

But when I edit my file using nano add a line, it is not printed in real time, If I run the awk command directly, my new line appear in the result.

Thank you

share|improve this question

You don't see it in real time because, for purposes of efficiency, pipes are buffered. tail -f has to fill up the buffer, typically 4 kB, before the output is passed to awk.

A fix is to use the unbuffer command which is part of the expect package:

unbuffer tail -f logfile.log | awk -F" " '{print $1, $2, $7, $8}'

This tricks tail into thinking it is writing to an interactive terminal. As a result, it doesn't buffer.

For more, see http://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe

Alternatively, if you have GNU coreutils 7.5 or better, you can disable output buffering with the stdbuf command:

stdbuf -o0 tail -f logfile.log | awk -F" " '{print $1, $2, $7, $8}'
share|improve this answer
    
Sadly the stdbuf solution does not seem to work either. – gimpycpu Apr 16 '14 at 3:22
    
But you are right that if I add ~10 rows, the buffer is flushed. So while the code works, the unbuffered part doesn't – gimpycpu Apr 16 '14 at 3:29
    
I just fixed a typo in the stdbuf command. Other than that, there might be an additional issue with your editor not updating the file in a way that triggers tail. Can you try just appending to the file without the editor. For example, run date >>logfile.log a couple times in one window and see if awk, running in another window, prints the updates. – John1024 Apr 16 '14 at 3:52
    
Yes I did tried echo "My long string here" >> logfile.log from a different putty window. When I execute that line about 10 times the awk command will execute. – gimpycpu Apr 16 '14 at 3:59
    
@gimpycpu Using nano, I got various results depending on how I edited the file before saving. Using echo "My long string here" >> logfile.log, however, it works every time for me. – John1024 Apr 16 '14 at 4:00

You could un-buffer it by adding a while loop as given in the example below. I have tested this on a running log file and it worked for me.

tail -f input.log | while read a; do echo "$a" | awk -F" " '{print $1, $2, $7, $8}' >> output.log; done
share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .