Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Tonight i was trying to look at my proxie's lines, and I came up with this, which seems all but KISS. How would you do it simply ?

 awk '{ trunc=sprintf("%.0f", $1) ;$1="" ; system("echo -n `date +%R -d\"@" trunc "\" `") ; print $0 ;}' </var/log/squid3/access.log

Lines in there begin with : 137816847.548 (hence the space delimiting next field, number totally random here)

share|improve this question
    
How about an example of a full log line and what you're hoping for as output? Looks like change the timestamp to a human readable date, and output the whole line? –  ernie Aug 7 '13 at 23:12
    
See @justbrowsing answer for an example of a line, thought the only significant part is the first field. What i want is indeed to be able to replace this field which is an UTC timestamp(with millisec) into a human readable date with choosable format. –  Kwaio Aug 8 '13 at 11:18
add comment

3 Answers

up vote 1 down vote accepted

Here is a small python script that should be quite robust and easily extended:

#!/usr/bin/env python

import sys, os
from datetime import datetime

try:
    if not os.path.isfile(sys.argv[1]):
        sys.stderr.write("[Error]: %s is not a file\n" % sys.argv[1])
        sys.exit(1)
except IndexError:
    sys.stderr.write("Usage: parse_squid_log.py access.log\n")
    sys.exit(1)

with open(sys.argv[1]) as log:
    for line in log:
        line = line.split(' ')
        line[0] = str(datetime.fromtimestamp(float(line[0])))
        print ' '.join(line).strip()

Save it to a file such a parse_squid_log and make sure it's executable chmod +x parse_squid_log and on your path.

Usage:

$ parse_squid_log
Usage: parse_squid_log.py access.log

$ parse_squid_log abc
[Error]: abc is not a file

$ parse_squid_log /var/log/squid3/access.log
2013-07-23 05:19:26.136000      929 185.168.101.131 TCP_MISS/200 28198 ...
2013-07-23 05:23:07.488000      0 10.42.0.14 TCP_IMS_HIT/304 278 GET ...
2014-05-08 13:49:47.488000      0 10.42.0.14 TCP_IMS_HIT/304 278 GET ...
share|improve this answer
    
I could have done this in Perl, but I had tried to be simple. After all your solution is what should be done by anyone in this need. –  Kwaio Aug 11 '13 at 19:12
add comment

Assuming the logfile is something like this:

1374553166.136    929 185.168.101.131 TCP_MISS/200 28198 GET http://www.amazon.de/gp/offer-listing/B000NOIW62/ref=sr_1_1_olp? - DIRECT/178.236.7.219 text/html
1374553387.488      0 10.42.0.14 TCP_IMS_HIT/304 278 GET http://g-ecx.images-amazon.com/images/G/01/da/creatives/sn130.png - NONE/- image/png
1399553387.488      0 10.42.0.14 TCP_IMS_HIT/304 278 GET http://g-ecx.images-amazon.com/images/G/01/da/creatives/sn130.png - NONE/- image/png

awk '{ cmd = ("date +%R -d @"$1); cmd | getline timestamp; close(cmd); $1=""; print timestamp$0 }' /var/log/squid3/access.log

Command breakdown:

  • date command set to convert epoch time to 24-hour hour and minute
  • set variable cmd to desired date command
  • execute cmd and pipe into variable timestamp
  • close system output
  • set column 1 to blank (to prevent duplication)
  • print converted time and the rest of the columns

OR

Better yet with only awk:

awk '{timestamp=strftime("%R",$1); $1=""; print timestamp$0}' /var/log/squid3/access.log

share|improve this answer
    
Indeed your solution seems to be better awk than mine, especially the latter. I'll keep the question unanswered for a week to see if someone comes up with a sexier, non-awksolution. If not, i'll take yours. +1 anyway –  Kwaio Aug 8 '13 at 11:19
add comment

Add logformat squid with parameters to /etc/squid/squid.conf, see man page. For example I use: logformat squid %tl IP:%>a %ru err_code Important is %tl, which mean, make timestamp in local time.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.