Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a text file that has various words per line.
How can I find the 12 most frequent appearing lines in the file and display them?
I am not very good with scripting commands.

If I could get the command and an explanation so that I can understand how to use it and expand my knowledge on commands it would be great!

share|improve this question

2 Answers 2

up vote 10 down vote accepted

You can easily do this with built-in commands.

  • Feed the contents of the file sort. We need this for the next step.
  • This goes to uniq -c. It will count the unique occurrence of each line. If the similar lines are not adjacent, this wouldn't have worked without sorting before.
  • Then, feed it to another sort, which now sorts in reversed order (r) and based on the numeric (n) interpretation of the uniq output. We need the numeric option since otherwise, the space in front of the numbers would lead to wrong results (see GNU sort's help for more).
  • Finally, only show the first twelve lines with head.

The command would then be:

sort test.txt | uniq -c | sort -rn | head -n 12

The output here contains the actual count of the occurrences.

To only get the raw list of lines, you can pipe the output to sed:

sort test.txt | uniq -c | sort -rn | head -n 12 | sed -E 's/^ *[0-9]+ //g'

Example:

I'm not there very often
I'm not there very often
Look at me!
Look at me!
Look at me!
Hello there!
Hello there!
Hello there!
Hello there!
Hello there!
Hello there!

Output from the first command, but only selecting 2 from head:

6 Hello there!
3 Look at me!

Output from the second command:

Hello there!
Look at me!
share|improve this answer
    
You have to sort it before using uniq. –  cYrus Jan 29 '12 at 20:40
    
@slhck:Thank you!One question:The sort -rn sorts in reverse order using as sorting field the number next to each line produced by uniq -c?I thought that something like k1 or something like that would be used –  Jim Jan 29 '12 at 21:09
    
@Jim Exactly. r reverses, and n numerically sorts on the number produced by uniq. What exactly do you mean with k1? –  slhck Jan 29 '12 at 21:14
    
@slhck:I was trying to figure out these commands using man and I understood that a syntax using -k something must be used to choose the field to sort by –  Jim Jan 29 '12 at 21:16
    
@cYrus:What is the edge case that a sort is needed beforehand? –  Jim Jan 29 '12 at 21:17

If your distro have logtop

cat your_file | logtop

If your file is constantly growing, like a log file, try :

tail -f your_log | logtop
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.