Tail multiple files while piping one of the files through grep

Question

I want to use tail follow two log files, but one of the log files has too much data, so I want it filtered with grep.

tail -f file1 file2|grep mySearch

The issue with this is that both files are run through the grep or rather the output of tail is run through grep. Only file2 should be filtered with the grep of mySearch. Any ideas?

I have tried named pipes, process substitutions, and compound commands.

Answer 1

can you run the two tails in the background, piping the filtered and unfiltered output to a temp file.

Then use that temp file as the source for whatever you need to dos -

tail -f file1 >> temp &
tail -f file2 | grep mySearch >> temp &


do something with temp now

Answer 2

Quite a few ways to skin a cat here. My personal all-time favorite though is MultiTail. It provides a myriad of ways to configure and display output from files, as well as a range of ways to filter that output via regex. You can filter 1 file for instance, while leaving another 4 unchanged. You can do an inverse filter as well, just like a grep -v.

A page of example commands can be found at http://www.vanheusden.com/multitail/examples.html

You may also like some screenshots of it in action. Multiple windows are controlled by ncurses.

If you are the programming type like me, you can create a personalized solution with something along the lines of Perl's File::Tail module using select on the separate handles.

Answer 3

Multitail sounds just like what I need.

This is something I've been doing, which is fine except that the log lines are not in exact temporal order..

-------------8<-------------

#!/bin/bash

function pidof_prog_file() {
 PGM=$1
 FIL=$2
 lsof -n 2>/dev/null| grep "${PGM}.*${FIL}" | tr -s ' ' | cut -d' ' -f 2
 }

T="tail -n 10 -f"

kill -TERM `pidof_prog_file tail application.log`
$T /var/www/protected/runtime/application.log | tr '\t' ' ' | tr -s ' ' | sed "s|^|PHP:|" &

kill -TERM `pidof_prog_file tail access.log`
$T /var/log/apache2/access.log | tr '\t' ' ' | tr -s ' ' | sed "s|^|HTTP:|" &

kill -TERM `pidof_prog_file tail catalina.out`
$T /var/log/tomcat6/catalina.out  | tr '\t' ' ' | tr -s ' ' | sed "s|^|JAVA:|" &

kill -TERM `pidof_prog_file ssh devj-db`
sudo -u $ME ssh $dbhost "$T /var/log/mysql/query.log"  | tr '\t' ' ' | tr -s ' ' | sed "s|^|SQL:|"