74

In linux, I can grep a string from a file using grep mySearchString myFile.txt. How can I only get the result which are unique?

122

You can achieve this with the sort and uniq utilities.

example:

[john@awesome ~]$ echo -e "test\ntest\ntest\nanother test\ntest"
test
test
test
another test
test
[john@awesome ~]$ echo -e "test\ntest\ntest\nanother test\ntest" | sort | uniq
another test
test

depending on the data you may want to utilize some of the switches as well.

  • 9
    @John T - I would recommend to use sort before uniq in case the data are not ordered. Otherwise uniq won't completely work. – Studer Feb 21 '10 at 2:58
  • t now I can upvote ! You also helped me writing others scripts here ;) – Studer Feb 21 '10 at 3:17
  • 41
    Use sort -u instead of sort | uniq. It saves a process, reduces the total I/O, and reduces the total number of comparisons that have to be made. – Chris Johnsen Feb 22 '10 at 5:55
  • @ChrisJohnsen You should make that comment an answer as it's a better solution then the current given answer – Nico Van Belle May 24 '18 at 7:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.