Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have hundreds of log files in a single directory, and I want an automated way to browse through the log files to count the number of times each command is used.

  • Log files are text files
  • There are 100+ different commands
  • they all start as "Command: " and then the name of the command.
  • It's one line per command
  • other lines should be ignored

So I'd like to know if there's a script or software that would do that and sort the results by most used commands, and display the frequency of each command. Something like statistics.

(edit: I'm on Win XP and I have no problems installing programs for this purpose)

share|improve this question
    
@Johannes Win XP, I never used PowerShell. edited the question. –  ino Jan 25 '10 at 16:41

2 Answers 2

up vote 2 down vote accepted

With Windows PowerShell you could do something along the lines of:

Get-Content *.log |
    Where-Object { $_.StartsWith('Command:') } |
    Group-Object {
        $null = $_ -match '^Command: (\w+)';
        $Matches[1]
    } |
    Select-Object Name,Count

For my test file this yields an output like

Name Count
---- -----
foo      2
bar      2
baz      1

Above code simply reads the log files line by line, pushing each line through the pipeline, it then filters the lines to only use those that start with “Command:”, indicating a command to follow. Then those lines are grouped into the individual commands. This is done by the regular expression

 ^Command: (\w+)

which matches the string “Command:” at the start of the line, followed by one or more word characters. This assumes the command name follows the colon and space immediately; adjust the regex accordingly if this is not the case. The command name is captured in a capture group which is used for grouping. After that only the name and frequency of the commands are selected.

The $null = part for the match is to suppress the output of the -match operator which would return always True here. We don't want to group by True bar but only by bar.

ETA: Depending on how exactly your input looks, you might want to tweak things a bit.

  • Allow empty commands:

     ^Command: (\w*)
    
  • Allow arbitrary non-space characters in command names (and empty commands):

     ^Command: ([^ ]*)
    
share|improve this answer
    
This works great, but when the "Command: " is empty or starts with an asterisk, it gives an error. Other than that it's perfect. any ideas on that? –  ino Jan 25 '10 at 17:20
    
@ume: Well, without details on how exactly your lines look this was a bit of guessing. To allow empty commands, replace the + in the regular expression by an asterisk *. To allow for more characters you can replace the \w by a dot . but that might match too much ... if spaces terminate your command names, then you can use [^ ] (note the space within the brackets) as a replacement for \w. –  Joey Jan 25 '10 at 17:27
    
It's perfect now. Thank you. –  ino Jan 25 '10 at 17:36

What OS?

This isn't a perfect match to your requirement, but...

grep -i -c Command *.log

will get you the number of occurrences broken down by file. You could then parse that output to get your totals. Dump it to a text file and pull it into Excel or something. Not hugely difficult, but admittedly not completely automated either. I often find that I spend more time searching for a completely automated solution than I do using something like this... You can get grep for Windows using cygwin.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.