Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want to use tail follow two log files, but one of the log files has too much data, so I want it filtered with grep.

tail -f file1 file2|grep mySearch

The issue with this is that both files are run through the grep or rather the output of tail is run through grep. Only file2 should be filtered with the grep of mySearch. Any ideas?

I have tried named pipes, process substitutions, and compound commands.

share|improve this question

3 Answers 3

up vote 1 down vote accepted

can you run the two tails in the background, piping the filtered and unfiltered output to a temp file.

Then use that temp file as the source for whatever you need to dos -

tail -f file1 >> temp &
tail -f file2 | grep mySearch >> temp &


do something with temp now
share|improve this answer
    
It looks like the second line in your solution ( the line with grep ) isn't working. If I tail just one file and grep the tail and send it to a file in the background - and then tail the temp file - the temp file never gets written to. Any suggestions. –  Brian Aug 11 '10 at 12:55
1  
Figured it out - had to add --line-buffered to my grep statement. But now I have different issues - like when my connection closes, tail is still running on the server even though I'm giving the -t option to ssh. Also, it seems like output from the log file being greped is inserted twice. Thanks for the help. –  Brian Aug 11 '10 at 13:32
    
Nevermind what I said about the processes hanging around. It must have been from prior testing. –  Brian Aug 11 '10 at 13:48
    
So, what is the status now? –  bryan Aug 11 '10 at 14:42
    
This is the command I ended up with: (tail -f ~/file1 >> temp &) ; (tail -f file2 | grep --line-buffered mySearch >> temp &) ; tail -f temp I needed to use the --line-buffered to get it to work. Thanks. –  Brian Aug 31 '10 at 18:14

Quite a few ways to skin a cat here. My personal all-time favorite though is MultiTail. It provides a myriad of ways to configure and display output from files, as well as a range of ways to filter that output via regex. You can filter 1 file for instance, while leaving another 4 unchanged. You can do an inverse filter as well, just like a grep -v.

A page of example commands can be found at http://www.vanheusden.com/multitail/examples.html

You may also like some screenshots of it in action. Multiple windows are controlled by ncurses.

If you are the programming type like me, you can create a personalized solution with something along the lines of Perl's File::Tail module using select on the separate handles.

share|improve this answer

Multitail sounds just like what I need.

This is something I've been doing, which is fine except that the log lines are not in exact temporal order..

-------------8<-------------

#!/bin/bash

function pidof_prog_file() {
 PGM=$1
 FIL=$2
 lsof -n 2>/dev/null| grep "${PGM}.*${FIL}" | tr -s ' ' | cut -d' ' -f 2
 }

T="tail -n 10 -f"

kill -TERM `pidof_prog_file tail application.log`
$T /var/www/protected/runtime/application.log | tr '\t' ' ' | tr -s ' ' | sed "s|^|PHP:|" &

kill -TERM `pidof_prog_file tail access.log`
$T /var/log/apache2/access.log | tr '\t' ' ' | tr -s ' ' | sed "s|^|HTTP:|" &

kill -TERM `pidof_prog_file tail catalina.out`
$T /var/log/tomcat6/catalina.out  | tr '\t' ' ' | tr -s ' ' | sed "s|^|JAVA:|" &

kill -TERM `pidof_prog_file ssh devj-db`
sudo -u $ME ssh $dbhost "$T /var/log/mysql/query.log"  | tr '\t' ' ' | tr -s ' ' | sed "s|^|SQL:|"
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.