Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Trying to cram all of the following into the title was hard, so this is what I really mean:

I am doing a tail -F access.log to watch an Apache2 log in realtime. The first thing on each line is an IP address followed by a space. I also have an unchanging CSV file where each row is an IP address, comma, name. For example:

10.1.2.33,John Smith
10.1.2.190,Jane Doe

Deliverable: I want to follow my access log as before, but if the CSV happens to contain a row with the same IP address that the current logfile line starts with, then insert the person's name at the start of the line.

For example, a line like this:

10.1.2.33 - - [22/Aug/2013:13:41:24 +0000] "GET /index.php ...

Should instead render as:

John Smith 10.1.2.33 - - [22/Aug/2013:13:41:24 +0000] "GET /index.php ...

Ideally, I would like to do this entirely by piping some regex commands together. So far, my biggest issue is that grep doesn't accept a pattern on stdin, it expects the subject on stdin (unless I'm missing something). Also, I have no problem transforming the CSV into some better format if required. Thanks.

share|improve this question
add comment

3 Answers 3

up vote 1 down vote accepted

Here's a POSIX shell solution

tail -f access.log | while read -r line;do
    ip=${line%% *}
    name=$(grep -F "$ip" your_csv_file|cut -d, -f2)
    if [ -z "$name" ];then
        printf "%s\n" "$line"
    else
        printf "%s\n" "$name $line"
    fi
done

Edit

Two improvements made to the solution:

  • -r switch added to read so that it doesn't evaluate escape sequences in the lines it reads
  • -F switch added to grep so that it treats the IP as a fixed string rather than a regular expression.
share|improve this answer
add comment

I would do this with a little Perl script:

#!/usr/bin/env perl

## Open the first argument, the .csv file
 open($f,"<","$ARGV[0]"); 
## Collect the list of ips and names in the %ips hash
 while(<$f>){
    chomp;
    @a=split(/,/); 
    $ips{$a[0]}=$a[1]
 }
## Now open the second argument, the log file to be watched. The
## trick here is to open by piping through tail -F
 open($f,"-|", "tail -F $ARGV[1]");
 while(<$f>){
 ## Get the ip
  /^([\d\.]+)/; 
  $ip=$1; 
 ## Prepend the associated name from the .csv file if one is defined
  s/$ip/$ips{$ip} $ip/ if defined($ips{$ip});
 ## Print the line
  print;
}

Save that in your $PATH as ip2name.pl, make it executable (chmod +x ip2name.pl) and then run it like this:

ip2name.pl /home/foo/ips.csv /var/log/apache2/access.log
share|improve this answer
add comment

grep can get the patterns from stdin, but it cannot get the text from stdin at the same time. See the -f option. If your shell supports process substitution, you can simulate it with

produce_patterns | grep -f- <( produce_input )

For example:

( echo b; echo y ) | grep -f- <( for i in {a..z} ; do echo $i ; done )

Output:

b
y
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.