23

I have a log file, each line in the log is prepend with a date, like so:

2012-03-06 11:34:48,657 blah blah blah...

How do I grep this file and get only the lines from 8am to 11pm only?

My intention is I want to count the number of errors happening within 8am to 11pm.

3 Answers 3

26
egrep '^[^ ]+ (0[89]|1[0-9]|2[012]):'

Detailed explanation can be found in various regex (regular expression) tutorials; egrep uses "POSIX extended" syntax (man 7 regex).

  • The first ^ means "start of the line".

  • [^ ]+ just matches the date field, regardless of the actual date.

    • [...] means "any character between the brackets", so [89] will match either 8 or 9; [0-9] is any number, and [^ ] is anything except a space (because of the ^ inside brackets).

    • + means "one or more of the previous" (for example, a+ would match a, aaa, and aaaaaaaa).

    • So ^[^ ]+ will start with the beginning of line, and match as many non-space characters as it can.

  • (...|...|...) means "either of the given patterns", so (0[89]|1[0-9]|2[012]) means "either 0[89] or 1[0-9] or 2[012]". It will match all numbers from 08 to 22.


A somewhat better option is:

awk -F'[: ]' '$2 >= 8 && $2 <= 22 { print }'

The -F option splits every line into separate fields according to the [: ] regex (matching either : or a space), and the awk script checks the 2nd column (the hour).

2
  • Hi, it works.. but can you add a little explanation how it works? I dont get the ^[^ ]+ part..
    – Rosdi
    Jun 21, 2012 at 11:14
  • awk option is much easier! Jul 13, 2020 at 10:54
24

Why bother using grep? You can simply use sed.

example:

sed -n '/Jun 17 13:39:54/ , /Jun 18 10:50:28/p' kern.log

This will print all the logs between June 17 13:39:54 and June 18 10:50:28

4
  • 1
    I think the OP wants all logs between 8am and 11pm, bot just the ones from a specific day.
    – Dennis
    Jun 21, 2012 at 11:14
  • 1
    He can do that too. Eg: sed -n '/2012-3-06 11:34:48/,/2012-3-06 16:34:48/p' logfile.name
    – Nima
    Jun 21, 2012 at 11:16
  • 1
    Nasty typo. I meant: not the ones from a specific day.
    – Dennis
    Jun 21, 2012 at 11:29
  • 3
    The problem with this approach is that there has to be a line in the log file with that timestamp. In the above example, if there is no line with timestamp 2012-03-06 11:34:48, then nothing will print out. The awk approach above would work for such cases (i.e. you want all logs with hour between 8 and 11, but you don't know if there is a log entry at time 2012-03-06 08:00:00 or anything for that hour at all).
    – user650654
    Nov 19, 2014 at 21:03
0

There's actually a much easier way to do this.

Download/Documentation: autodrgrep.kl.sh

Command:

./autodrgrep.kl.sh   notchef   /tmp/client.log   '2016-05-08_08:00:00,2016-05-08_23:00:00'   'INFO'   'a2ensite'   5  10  -show

Explanation:

  • autodrgrep.kl.sh is the tool name.

  • notchef is an option that is passed to the tool to tell it what to do. In this particular case, it is telling the tool what type of log file /tmp/client.log is.

  • /tmp/client.log is of course the log file.

  • 2016-05-08_19:12:00,2016-05-08_21:13:00 is the range of date from within the log that you wish to scan

  • "INFO" is one of the strings that is in the lines of logs that you're interested in.

  • "a2ensite" is another string on the same line that you expect to find the "INFO" string on. Specifying these two strings (INFO and a2ensite) isolates and processes the lines you want a lot quicker, particularly if you're dealing with a huge log file.

  • 5 specifies Warning. By specifying 5, you're telling the program to alert as WARNING if there are at least 5 occurrences of the search strings you specified

  • 10 specifies Critical. By specifying 10, you're telling the program to alert as CRITICAL if there are at least 10 occurrences of the search strings you specified.

  • -show specifies what type of response you'll get. By specifying -shown, you're saying if anything is found that matches the specified patterns, output to screen.

Sample run:

# ./autodrgrep.kl.sh notchef  /tmp/client.log   '2016-05-08_19:12:00,2016-05-08_21:13:00' 'INFO' 'a2ensite'  5  10  -show

[2016-05-08 19:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:13:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 19:42:57-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:42:57-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:42:57-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:43:08-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:43:11-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:13:10-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:42:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:42:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:42:59-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:43:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:43:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 21:12:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 21:12:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 21:12:59-07:00] INFO: execute[a2ensite default] ran successfully
23
2---78720---23---ATWFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13) SEAGM

What if the user specified date range or time frame is not in the log?

Each run of the above command will always have a line (last line of the output) that either says "ATWFILF" or "ETWNFILF".

  • ATWFILF means that the actual date range or time frame you requested searched was found in the log. So this is very good.

  • ETWNFILF means the actual date range or time frame you requested searched was NOT found in the log. In this case, the closest time to the time you specified will be detected and used instead.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .