Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am using tail -f to monitor a log file that is being actively written to. When a certain string is written to the log file, I want to quit the monitoring, and continue with the rest of my script.

Currently I am using:

tail -f logfile.log | grep -m 1 "Server Started"

When the string is found, grep quits as expected, but I need to find a way to make the tail command quit too so that the script can continue.

share|improve this question
    
I wonder on what Operating System the original poster was running. On a Linux RHEL5 system, I was surprised to find that the tail command simply dies once grep command has found the match and exited. –  ZaSter Sep 14 '13 at 1:27

11 Answers 11

up vote 12 down vote accepted

The accepted answer isn't working for me, plus it's confusing and it changes the log file.

I'm using something like this:

tail -f logfile.log | while read LOGLINE
do
   [[ "${LOGLINE}" == *"Server Started"* ]] && pkill -P $$ tail
done

If the log line matches the pattern, kill the "tail" started by this script.

Note: if you want to also view the output on the screen, either | tee /dev/tty or echo the line before testing in the while loop.

share|improve this answer
    
This works, but pkill is not specified by POSIX and isn't available everywhere. –  Richard Hansen Feb 27 '13 at 18:29

So after doing some testing, I found a quick 1-line way to make this work. It appears tail -f will quit when grep quits, but there's a catch. It appears to only be triggered if the file is opened and closed. I've accomplished this by appending the empty string to the file when grep finds the match.

tail -f logfile |grep -m 1 "Server Started" | xargs echo "" >> logfile \;

I'm not sure why the open/close of the file triggers tail to realize that the pipe is closed, so I wouldn't rely on this behavior. but it seems to work for now.

Reason it closes, look at the -F flag, versus the -f flag.

share|improve this answer
    
This works because appending to the logfile causes tail to output another line, but by then grep has exited (probably -- there's a race condition there). If grep has exited by the time tail writes another line, tail will get a SIGPIPE. That causes tail to exit right away. –  Richard Hansen Feb 27 '13 at 18:19
    
Disadvantages to this approach: (1) there's a race condition (it may not always exit immediately) (2) it requires write access to the log file (3) you must be OK with modifying the log file (4) you may corrupt the log file (5) it only works for tail (6) you can't easily tweak it to behave differently depending on differnt string matches ("server started" vs. "server start failed") because you can't easily get the return code of the middle stage of the pipeline. There is an alternative approach that avoids all of these problems -- see my answer. –  Richard Hansen Feb 27 '13 at 18:23

This will be a bit tricky since you will have to get into process control and signaling. More kludgey would be a two script solution using PID tracking. Better would be using named pipes like this.

What shell script are you using?

For a quick and dirty, one script solution - I would make a perl script using File:Tail

use File::Tail;
$file=File::Tail->new(name=>$name, maxinterval=>300, adjustafter=>7);
while (defined($line=$file->read)) {
    last if $line =~ /Server started/;
}

So rather than printing inside the while loop, you could filter for the string match and break out of the while loop to let your script continue.

Either of these should involve just a little learning to implement the watching flow control you are seeking.

share|improve this answer
    
using bash. my perl-fu is not that strong, but I'll give this a shot. –  Alex Hofsteede Apr 14 '11 at 17:03
    
Use pipes - they love bash and bash love them. (and your backup software will respect you when it hits one of your pipes) –  bmike Apr 27 '11 at 0:57
    
maxinterval=>300 means that it will check the file every five minutes. Since I know that my line will appear in the file momentarily, I'm using much more aggressive polling: maxinterval=>0.2, adjustafter=>10000 –  Stephen Ostermiller Sep 9 at 20:18

Alex i think this one will help you lot.

tail -f logfile |grep -m 1 "Server Started" | xargs echo "" >> /dev/null ;

this command will never give an entry on the logfile but will grep silently...

share|improve this answer
    
This won't work -- you have to append to logfile otherwise it could be an arbitrarily long time before tail outputs another line and detects that grep has died (via SIGPIPE). –  Richard Hansen Feb 27 '13 at 18:26

There are two ways to get tail to exit:

Approach #1: Force tail to write another line

You can force tail to write another line of output immediately after grep has found a match and exited. This will cause tail to get a SIGPIPE, causing it to exit. One way to do this is to modify the file being monitored by tail after grep exits.

Here is some example code:

tail -f logfile.log | grep -m 1 "Server Started" | { cat; echo >>logfile.log; }

In this example, cat won't exit until grep has closed its stdout, so tail is not likely to be able to write to the pipe before grep has had a chance to close its stdin. cat is used to propagate the standard output of grep unmodified.

There are several downsides to this approach:

  • If grep closes stdout before closing stdin, there will always be a race condition: grep closes stdout, triggering cat to exit, triggering echo, triggering tail to output a line. If this line is sent to grep before grep has had a chance to close stdin, tail won't get the SIGPIPE until it writes another line.
  • It requires write access to the log file.
  • You must be OK with modifying the log file.
  • You may corrupt the log file if you happen to write at the same time as another process (the writes may be interleaved, causing a newline to appear in the middle of a log message).
  • This approach is specific to tail—it won't work with other programs.
  • The third pipeline stage makes it hard to get access to the return code of the second pipeline stage (unless you're using a POSIX extension such as bash's PIPESTATUS array). This is not a big deal in this case because grep will always return 0, but in general the middle stage might be replaced with a different command whose return code you care about (e.g., something that returns 0 when "server started" is detected, 1 when "server failed to start" is detected).

The next approach avoids these limitations.

Approach #2: Kill tail

You can get tail to exit by sending it a signal like SIGTERM. The challenge is reliably knowing two things in the same place in code: tail's PID and whether grep has exited.

The second pipeline stage knows when grep has exited, so getting the PID to the second stage would solve the problem. The first pipeline stage can easily and reliably get tail's PID by backgrounding tail and reading $!. If the first pipeline stage could reliably communicate the PID to the second pipeline stage the problem would be solved. Unfortunately, the first pipeline stage can't simply set a variable to be read by the second pipeline stage because the two stages are in separate execution environments (subshells). The first stage could send the PID to the second stage via the pipe by echoing the PID, but this string will get mixed with tail's output. If the echo is interleaved with tail's output, it won't be possible to reliably distinguish tail's output from the PID string, even with a complex escaping scheme. The second stage could use pgrep, but that would be unreliable (you might match the wrong process) and non-POSIX (non-portable).

So instead, you can have the second pipeline stage notify the first pipeline stage (via a FIFO) that grep has exited and let the first stage kill tail. Here is some example code:

fifo=/tmp/notifyfifo
mkfifo "${fifo}" || exit 1
{
    # run tail in the background so that the shell can
    # kill tail when notified that grep has exited
    tail -f logfile.log &
    # remember tail's PID
    tailpid=$!
    # wait for notification that grep has exited
    read foo <${fifo}
    # grep has exited, time to go
    kill "${tailpid}"
} | {
    grep -m 1 "Server Started"
    # notify the first pipeline stage that grep is done
    echo >${fifo}
}
# clean up
rm "${fifo}"

The advantages to this approach are:

  • you don't need to modify the log file
  • the approach works for other utilities besides tail
  • it does not suffer from a race condition
  • you can easily get the return value of grep (or whatever command you're using in the second pipeline stage) by saving $? before running echo

The downside to this approach is managing the FIFO: You'll need to securely generate a temporary file name, and you'll need to ensure that the temporary FIFO is deleted even if the user hits Ctrl-C in the middle of the script. This can be done using a trap.

share|improve this answer

Here is a much better solution that does not require you to write to the logfile, which is very dangerous or even impossible in some cases.

sh -c 'tail -n +0 -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'

Currently it has only one side effect, tail process will remain in background until the next list is written to the log.

share|improve this answer
    
tail -n +0 -f starts from the beginning of the file. tail -n 0 -f starts from the end of the file. –  Stephen Ostermiller Sep 9 at 20:05
    
Another side effect I get: myscript.sh: line 14: 7845 Terminated sh -c 'tail... –  Stephen Ostermiller Sep 9 at 20:06
    
I believe that "next list" should be "next line" in this answer. –  Stephen Ostermiller Sep 9 at 20:07

Currently, as given, all of the tail -f solutions here run the risk of picking up a previously logged "Server Started" line (which may or may not be a problem in your specific case, depending on the number of lines logged and log file rotation/truncation).

Rather than over-complicate things, just use a smarter tail, as bmike showed with a perl snippit. The simplest solution is this retail which has integrated regex support with start and stop condition patterns:

retail -f -u "Server Started" server.log > /dev/null

This will follow the file like a normal tail -f until the first new instance of that string appears, then exit. (The -u option does not trigger on existing lines in the last 10 lines of the file when in normal "follow" mode.)


If you use GNU tail (from coreutils), the next simplest option is to use --pid and a FIFO (named pipe):

mkfifo ${FIFO:=serverlog.fifo.$$}
grep -q -m 1 "Server Started" ${FIFO}  &
tail -n 0 -f server.log  --pid $! >> ${FIFO}
rm ${FIFO}

A FIFO is used because the processes must be started separately in order to obtain and pass a PID. A FIFO still suffers from the same problem of hanging around for a timely write to cause tail to receive a SIGPIPE, use the --pid option so that tail exits when it notices that grep has terminated (conventionally used to monitor the writer process rather than the reader, but tail doesn't really care). Option -n 0 is used with tail so that old lines don't trigger a match.


Finally, you could use a stateful tail, this will store the current file offset so subsequent invocations only show new lines (it also handles file rotation). This example uses the old FWTK retail*:

retail "${LOGFILE:=server.log}" > /dev/null   # skip over current content
while true; do
    [ "${LOGFILE}" -nt ".${LOGFILE}.off" ] && 
       retail "${LOGFILE}" | grep -q "Server Started" && break
    sleep 2
done

* Note, same name, different program to the previous option.

Rather than have CPU-hogging loop, compare the timestamp of the file with the state file (.${LOGFILE}.off), and sleep. Use "-T" to specify the location of the state file if required, the above assumes the current directory. Feel free to skip that condition, or on Linux you could use the more efficient inotifywait instead:

retail "${LOGFILE:=server.log}" > /dev/null
while true; do
    inotifywait -qq "${LOGFILE}" && 
       retail "${LOGFILE}" | grep -q "Server Started" && break
done
share|improve this answer

If you're using Bash (at least, but it seems it's not defined by POSIX, so it may be missing in some shells), you can use the syntax

grep -m 1 "Server Started" <(tail -f logfile.log)

It works pretty much like the FIFO solutions already mentioned, but much simpler to write.

share|improve this answer
    
This works, but tail still running untill you send a SIGTERM (Ctrl+C, exit command, or kill it) –  mems Oct 6 at 12:17

Try to use inotify (inotifywait)

You set up inotifywait for any file change, then check the file with grep, if not found just rerun inotifywait, if found exit the loop... Smth like that

share|improve this answer
    
This way, the entire file would have to be rechecked every time something is written to it. Doesn't work well for log files. –  grawity Apr 13 '11 at 20:59
1  
Another way is to make two scripts: 1. tail -f logfile.log | grep -m 1 "Server Started" > /tmp/found 2. firstscript.sh& MYPID=$!; inotifywait -e MODIFY /tmp/found; kill -KILL -$MYPID –  Evengard Apr 13 '11 at 21:16
    
I'd love you to edit your answer to show capturing the PID and then using inotifywait - an elegant solution that would be easy to grasp for someone used to grep but needing a more sophisticated tool. –  bmike May 10 '11 at 20:47
    
A PID of what you would like to capture? I can try to make it if you explain a bit more what you want –  Evengard May 12 '11 at 20:31

wait for file to appear

while [ ! -f /path/to/the.file ] 
do sleep 2; done

wait for string to apper in file

while ! grep "the line you're searching for" /path/to/the.file  
do sleep 10; done

http://superuser.com/a/743693/129669

share|improve this answer

I can't imagine a cleaner solution than this one:

#!/usr/bin/env bash
# file : untail.sh
# usage: untail.sh logfile.log "Server Started"
(echo $BASHPID; tail -f $1) | while read LINE ; do
    if [ -z $TPID ]; then
        TPID=$LINE # the first line is used to store the previous subshell PID
    else
        echo "$LINE"; [[ "$LINE" == *"${*:2}"* ]] && kill -3 $TPID && break
    fi
done

ok, maybe the name can be subject to improvements...

Advantages:

  • it doesn't use any special utilities
  • it doesn't write to disk
  • it gracefully quits tail and closes the pipe
  • it is pretty short and easy to understand
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.