Monitoring a file until a string is found

Question

I am using tail -f to monitor a log file that is being actively written to. When a certain string is written to the log file, I want to quit the monitoring, and continue with the rest of my script.

Currently I am using:

tail -f logfile.log | grep -m 1 "Server Started"

When the string is found, grep quits as expected, but I need to find a way to make the tail quit too so that the script can continue.

Answer 1

Here is a simple one-liner. It doesn't need bash-specific or non-POSIX tricks, or even a named pipe. All you really need is to decouple the termination of tail from grep. That way, once grep ends, the script can continue even if tail hasn't ended yet. So this simple method will get you there:

( tail -f -n0 logfile.log & ) | grep -q "Server Started"

grep will block until it has found the string, whereupon it will exit. By making tail run from it's own sub-shell, we can place it in the background so it runs independently. Meanwhile, the main shell is free to continue execution of the script as soon as grep exits. tail will linger in its sub-shell until the next line has been written to the logfile, and then exit (possibly even after the main script has terminated). The main point is that the pipeline no longer waits for tail to terminate, so the pipeline exits as soon as grep exits.

Some minor tweaks:

• The option -n0 to tail makes it start reading from the current last line of logfile, in case the string exists earlier in the logfile.
• You might want to give tail -F rather than -f. It is not POSIX, but it allows tail to work even if the log is rotated while waiting.
• Option -q rather than -m1 makes grep quit after the first occurrence, but without printing out the trigger line. Also it is POSIX, which -m1 isn't.

Answer 2

The accepted answer isn't working for me, plus it's confusing and it changes the log file.

I'm using something like this:

tail -f logfile.log | while read LOGLINE
do
   [[ "${LOGLINE}" == *"Server Started"* ]] && pkill -P $$ tail
done

If the log line matches the pattern, kill the tail started by this script.

Note: if you want to also view the output on the screen, either | tee /dev/tty or echo the line before testing in the while loop.

Answer 3

If you're using Bash (at least, but it seems it's not defined by POSIX, so it may be missing in some shells), you can use the syntax

grep -m 1 "Server Started" <(tail -f logfile.log)

It works pretty much like the FIFO solutions already mentioned, but much simpler to write.

Answer 4

There are a few ways to get tail to exit:

Poor Approach: Force tail to write another line

You can force tail to write another line of output immediately after grep has found a match and exited. This will cause tail to get a SIGPIPE, causing it to exit. One way to do this is to modify the file being monitored by tail after grep exits.

Here is some example code:

tail -f logfile.log | grep -m 1 "Server Started" | { cat; echo >>logfile.log; }

In this example, cat won't exit until grep has closed its stdout, so tail is not likely to be able to write to the pipe before grep has had a chance to close its stdin. cat is used to propagate the standard output of grep unmodified.

This approach is relatively simple, but there are several downsides:

• If grep closes stdout before closing stdin, there will always be a race condition: grep closes stdout, triggering cat to exit, triggering echo, triggering tail to output a line. If this line is sent to grep before grep has had a chance to close stdin, tail won't get the SIGPIPE until it writes another line.
• It requires write access to the log file.
• You must be OK with modifying the log file.
• You may corrupt the log file if you happen to write at the same time as another process (the writes may be interleaved, causing a newline to appear in the middle of a log message).
• This approach is specific to tail—it won't work with other programs.
• The third pipeline stage makes it hard to get access to the return code of the second pipeline stage (unless you're using a POSIX extension such as bash's PIPESTATUS array). This is not a big deal in this case because grep will always return 0, but in general the middle stage might be replaced with a different command whose return code you care about (e.g., something that returns 0 when "server started" is detected, 1 when "server failed to start" is detected).

The next approaches avoid these limitations.

A Better Approach: Avoid Pipelines

You can use a FIFO to avoid the pipeline altogether, allowing execution to continue once grep returns. For example:

fifo=/tmp/tmpfifo.$$
mkfifo "${fifo}" || exit 1
tail -f logfile.log >${fifo} &
tailpid=$! # optional
grep -m 1 "Server Started" "${fifo}"
kill "${tailpid}" # optional
rm "${fifo}"

The lines marked with the comment # optional can be removed and the program will still work; tail will just linger until it reads another line of input or is killed by some other process.

The advantages to this approach are:

• you don't need to modify the log file
• the approach works for other utilities besides tail
• it does not suffer from a race condition
• you can easily get the return value of grep (or whatever alternative command you're using)

The downside to this approach is complexity, especially managing the FIFO: You'll need to securely generate a temporary file name, and you'll need to ensure that the temporary FIFO is deleted even if the user hits Ctrl-C in the middle of the script. This can be done using a trap.

Alternative Approach: Send a Message to Kill tail

You can get the tail pipeline stage to exit by sending it a signal like SIGTERM. The challenge is reliably knowing two things in the same place in code: tail's PID and whether grep has exited.

With a pipeline like tail -f ... | grep ..., it's easy to modify the first pipeline stage to save tail's PID in a variable by backgrounding tail and reading $!. It's also easy to modify the second pipeline stage to run kill when grep exits. The problem is that the two stages of the pipeline run in separate "execution environments" (in the terminology of the POSIX standard) so the second pipeline stage can't read any variables set by the first pipeline stage. Without using shell variables, either the second stage must somehow figure out tail's PID so that it can kill tail when grep returns, or the first stage must somehow be notified when grep returns.

The second stage could use pgrep to get tail's PID, but that would be unreliable (you might match the wrong process) and non-portable (pgrep is not specified by the POSIX standard).

The first stage could send the PID to the second stage via the pipe by echoing the PID, but this string will get mixed with tail's output. Demultiplexing the two may require a complex escaping scheme, depending on the output of tail.

You can use a FIFO to have the second pipeline stage notify the first pipeline stage when grep exits. Then the first stage can kill tail. Here is some example code:

fifo=/tmp/notifyfifo.$$
mkfifo "${fifo}" || exit 1
{
    # run tail in the background so that the shell can
    # kill tail when notified that grep has exited
    tail -f logfile.log &
    # remember tail's PID
    tailpid=$!
    # wait for notification that grep has exited
    read foo <${fifo}
    # grep has exited, time to go
    kill "${tailpid}"
} | {
    grep -m 1 "Server Started"
    # notify the first pipeline stage that grep is done
    echo >${fifo}
}
# clean up
rm "${fifo}"

This approach has all the pros and cons of the previous approach, except it's more complicated.

A Warning About Buffering

POSIX allows the stdin and stdout streams to be fully buffered, which means that tail's output might not be processed by grep for an arbitrarily long time. There shouldn't be any problems on GNU systems: GNU grep uses read(), which avoids all buffering, and GNU tail -f makes regular calls to fflush() when writing to stdout. Non-GNU systems may have to do something special to disable or regularly flush buffers.

Answer 5

Let me expand on @00prometheus answer (which is the best one).

Maybe you should use a timeout instead of waiting indefinitely.

The bash function below will block until the given search term appears or a given timeout is reached.

The exit status will be 0 if the string is found within the timeout.

wait_str() {
  local file="$1"; shift
  local search_term="$1"; shift
  local wait_time="${1:-5m}"; shift # 5 minutes as default timeout

  (timeout $wait_time tail -F -n0 "$file" &) | grep -q "$search_term" && return 0

  echo "Timeout of $wait_time reached. Unable to find '$search_term' in '$file'"
  return 1
}

Perhaps the log file doesn't exist yet just after launching your server. In that case, you should wait for it to appear before searching for the string:

wait_server() {
  echo "Waiting for server..."
  local server_log="$1"; shift
  local wait_time="$1"; shift

  wait_file "$server_log" 10 || { echo "Server log file missing: '$server_log'"; return 1; }

  wait_str "$server_log" "Server Started" "$wait_time"
}

wait_file() {
  local file="$1"; shift
  local wait_seconds="${1:-10}"; shift # 10 seconds as default timeout

  until test $((wait_seconds--)) -eq 0 -o -f "$file" ; do sleep 1; done

  ((++wait_seconds))
}

Here's how you can use it:

wait_server "/var/log/server.log" 5m && \
echo -e "\n-------------------------- Server READY --------------------------\n"

Answer 6

Currently, as given, all of the tail -f solutions here run the risk of picking up a previously logged "Server Started" line (which may or may not be a problem in your specific case, depending on the number of lines logged and log file rotation/truncation).

Rather than over-complicate things, just use a smarter tail, as bmike showed with a perl snippit. The simplest solution is this retail which has integrated regex support with start and stop condition patterns:

retail -f -u "Server Started" server.log > /dev/null

This will follow the file like a normal tail -f until the first new instance of that string appears, then exit. (The -u option does not trigger on existing lines in the last 10 lines of the file when in normal "follow" mode.)

---

If you use GNU tail (from coreutils), the next simplest option is to use --pid and a FIFO (named pipe):

mkfifo ${FIFO:=serverlog.fifo.$$}
grep -q -m 1 "Server Started" ${FIFO}  &
tail -n 0 -f server.log  --pid $! >> ${FIFO}
rm ${FIFO}

A FIFO is used because the processes must be started separately in order to obtain and pass a PID. A FIFO still suffers from the same problem of hanging around for a timely write to cause tail to receive a SIGPIPE, use the --pid option so that tail exits when it notices that grep has terminated (conventionally used to monitor the writer process rather than the reader, but tail doesn't really care). Option -n 0 is used with tail so that old lines don't trigger a match.

---

Finally, you could use a stateful tail, this will store the current file offset so subsequent invocations only show new lines (it also handles file rotation). This example uses the old FWTK retail*:

retail "${LOGFILE:=server.log}" > /dev/null   # skip over current content
while true; do
    [ "${LOGFILE}" -nt ".${LOGFILE}.off" ] && 
       retail "${LOGFILE}" | grep -q "Server Started" && break
    sleep 2
done

_{* Note, same name, different program to the previous option.}

Rather than have CPU-hogging loop, compare the timestamp of the file with the state file (.${LOGFILE}.off), and sleep. Use "-T" to specify the location of the state file if required, the above assumes the current directory. Feel free to skip that condition, or on Linux you could use the more efficient inotifywait instead:

retail "${LOGFILE:=server.log}" > /dev/null
while true; do
    inotifywait -qq "${LOGFILE}" && 
       retail "${LOGFILE}" | grep -q "Server Started" && break
done

Answer 7

So after doing some testing, I found a quick 1-line way to make this work. It appears tail -f will quit when grep quits, but there's a catch. It appears to only be triggered if the file is opened and closed. I've accomplished this by appending the empty string to the file when grep finds the match.

tail -f logfile |grep -m 1 "Server Started" | xargs echo "" >> logfile \;

I'm not sure why the open/close of the file triggers tail to realize that the pipe is closed, so I wouldn't rely on this behavior. but it seems to work for now.

Reason it closes, look at the -F flag, versus the -f flag.

Answer 8

This will be a bit tricky since you will have to get into process control and signaling. More kludgey would be a two script solution using PID tracking. Better would be using named pipes like this.

What shell script are you using?

For a quick and dirty, one script solution - I would make a perl script using File:Tail

use File::Tail;
$file=File::Tail->new(name=>$name, maxinterval=>300, adjustafter=>7);
while (defined($line=$file->read)) {
    last if $line =~ /Server started/;
}

So rather than printing inside the while loop, you could filter for the string match and break out of the while loop to let your script continue.

Either of these should involve just a little learning to implement the watching flow control you are seeking.

Answer 9

Read them all. tldr: decouple the termination of tail from grep.

The two forms most convenient are

( tail -f logfile.log & ) | grep -q "Server Started"

and if you have bash

grep -m 1 "Server Started" <(tail -f logfile.log)

But if that tail sitting in the background bothers you, there is nicer way than a fifo or any other answer here. Requires bash.

coproc grep -m 1 "Server Started"
tail -F /tmp/x --pid $COPROC_PID >&${COPROC[1]}

Or if it isn't tail which is outputing things,

coproc command that outputs
grep -m 1 "Server Started" <&${COPROC[0]}
kill $COPROC_PID

In bash, you can also get rid of the job control output by wrapping in ():

(coproc command that outputs; grep -m 1 "Server Started" ${COPROC[0]}; kill $COPROC_PID)

Answer 10

The tail command can be backgrounded and its pid echoed to the grep subshell. In the grep subshell a trap handler on EXIT can kill the tail command.

( (sleep 1; exec tail -f logfile.log) & echo $! ; wait ) | 
     (trap 'kill "$pid"' EXIT; pid="$(head -1)"; grep -m 1 "Server Started")

Answer 11

I can't imagine a cleaner solution than this one:

#!/usr/bin/env bash
# file : untail.sh
# usage: untail.sh logfile.log "Server Started"
(echo $BASHPID; tail -f $1) | while read LINE ; do
    if [ -z $TPID ]; then
        TPID=$LINE # the first line is used to store the previous subshell PID
    else
        echo "$LINE"; [[ "$LINE" == *"${*:2}"* ]] && kill -3 $TPID && break
    fi
done

ok, maybe the name can be subject to improvements...

Advantages:

• it doesn't use any special utilities
• it doesn't write to disk
• it gracefully quits tail and closes the pipe
• it is pretty short and easy to understand

Answer 12

wait for file to appear

while [ ! -f /path/to/the.file ] 
do sleep 2; done

wait for string to apper in file

while ! grep "the line you're searching for" /path/to/the.file  
do sleep 10; done

https://superuser.com/a/743693/129669

Answer 13

Here is a much better solution that does not require you to write to the logfile, which is very dangerous or even impossible in some cases.

sh -c 'tail -n +0 -f /tmp/foo | { sed "/EOF/ q" && kill $$ ;}'

Currently it has only one side effect, the tail process will remain in background until the next line is written to the log.

Answer 14

You don't necessary need tail to do that. I think the watch command is what you're looking for. The watch command monitors the output of a file and can be terminated with the -g option when the output changed.

watch -g grep -m 1 "Server Started" logfile.log && Yournextaction

Answer 15

The other solutions here have several issues:

• if the logging process is already down or goes down during the loop they will run indefinitely
• editing a log that should only be viewed
• unnecessarily writing an additional file
• not allowing for additional logic

Here is what I came up with using tomcat as an example (remove the hashes if you want to see the log while its starting):

function startTomcat {
    loggingProcessStartCommand="${CATALINA_HOME}/bin/startup.sh"
    loggingProcessOwner="root"
    loggingProcessCommandLinePattern="${JAVA_HOME}"
    logSearchString="org.apache.catalina.startup.Catalina.start Server startup"
    logFile="${CATALINA_BASE}/log/catalina.out"

    lineNumber="$(( $(wc -l "${logFile}" | awk '{print $1}') + 1 ))"
    ${loggingProcessStartCommand}
    while [[ -z "$(sed -n "${lineNumber}p" "${logFile}" | grep "${logSearchString}")" ]]; do
        [[ -z "$(ps -ef | grep "^${loggingProcessOwner} .* ${loggingProcessCommandLinePattern}" | grep -v grep)" ]] && { echo "[ERROR] Tomcat failed to start"; return 1; }
        [[ $(wc -l "${logFile}" | awk '{print $1}') -lt ${lineNumber} ]] && continue
        #sed -n "${lineNumber}p" "${logFile}"
        let lineNumber++
    done
    #sed -n "${lineNumber}p" "${logFile}"
    echo "[INFO] Tomcat has started"
}

Answer 16

Try to use inotify (inotifywait)

You set up inotifywait for any file change, then check the file with grep, if not found just rerun inotifywait, if found exit the loop... Smth like that

Answer 17

Alex i think this one will help you lot.

tail -f logfile |grep -m 1 "Server Started" | xargs echo "" >> /dev/null ;

this command will never give an entry on the logfile but will grep silently...

Answer 18

You want to leave as soon as the line is written, but you also want to leave after a timeout:

if (timeout 15s tail -F -n0 "stdout.log" &) | grep -q "The string that says the startup is successful" ; then
    echo "Application started with success."
else
    echo "Startup failed."
    tail stderr.log stdout.log
    exit 1
fi

Answer 19

I needed to write a line in log file after the line appeared in auth.log

got inspired by the answer above

tail -f logfile.log | while read LOGLINE
do
[[ "${LOGLINE}" == *"Server Started"* ]] && pkill -P $$ tail
done

for me worked next

tail -f -n0 /var/log/auth.log | while read LOGLINE;
do [[ "${LOGLINE}" == *"Removed session"* ]] && pkill -P $$ tail; 
echo "$(date "+%d-%m-%Y_%H:%M:%S") $(whoami) Screen_Locked poweroff" >> 
/var/log/lock-unlock-user.log; 
done

Answer 20

One liner

tail -f myfile.log | while read LINE; do echo $LINE; echo "$LINE" | grep -Fq 'Listening on 0.0.0.0' && pkill -P $$; done

Notes

• Based on the excellent answer by Rob https://superuser.com/a/449307/33705
• Doesn't use bash only double brackets [[]]
• This uses GNU grep to do the line matching.
• I did not see any tail processes running after this ends, but I might have missed it, correct me if I'm wrong.

Explanation

tail -f myfile.log | while read LINE;

will loop line by line from myfile.log and send it to the while which will store it in a variable $LINE.

echo $LINE;

Will print/echo the the line so that we can monitor it, I imagine this will lose any color information.

echo "$LINE" | grep -Fq 'Listening on 0.0.0.0' && pkill -P $$;

echo the LINE as a "string" and send the output to grep.

grep will then look for the string "Listening on 0.0.0.0" and when it matches, kill the current process.

• $$ is a special variable that returns the current process id.
• grep -F means "do plaintext matching", this skips doing regex and makes it faster.
• grep -q silences output and quits once there is a single match, returning a true or false that you can chain with &&

After that the shell will continue running commands as usual.

The cool thing about this approach is that we can use it for anything, even to monitor a docker container until a string is found.

docker logs container_name -f | while read LINE; do echo ${LINE}; echo "${LINE}" | grep -Fq 'Listening on 0.0.0.0' && pkill -P $$; done; echo "hello"

Answer 21

Inspired on the answer from Phio https://superuser.com/a/1052328/102262, I wrote this version on a function and that works both on Linux and MacOSX

wait_for_pattern() {
    local file="$1"
    local pattern="$2"
    # Two parts:
    # 1. tail follow the file in background, printing the PID and waiting for it
    # 2. add a trap to kill the tail PID on exit, read PID from first line, sed to match the first occurrence
    (
        (sleep 1; exec tail -n 1000 -f "${file}") & echo $! ; wait
    ) | (
        trap 'kill "$pid"' EXIT; pid="$(head -1)";
        sed -n "/${pattern}/{p;q;}"
    )
}

wait_for_pattern /tmp/foo Lib

Explanation:

1. starts a subprocess running in background
2. Subproccess sleep 1 to wait for the next piped process to read the pid
3. then execs into tail
4. print the PID of the background process
5. we add a wait to wait for tail in background to finish
6. The piped subprocess would setup a trap to kill the PID.
7. read the PID from first line
8. sed will match the pattern and exit on first occurrence. This sed syntax is compatible with macosx

Answer 22

how about this:

while true; do if [ ! -z $(grep "myRegEx" myLog.log) ]; then break; fi ; done