84

In linux, I can grep a string from a file using grep mySearchString myFile.txt. How can I only get the result which are unique?

139

You can achieve this with the sort and uniq utilities.

example:

[john@awesome ~]$ echo -e "test\ntest\ntest\nanother test\ntest"
test
test
test
another test
test
[john@awesome ~]$ echo -e "test\ntest\ntest\nanother test\ntest" | sort | uniq
another test
test

depending on the data you may want to utilize some of the switches as well.

4
  • 11
    @John T - I would recommend to use sort before uniq in case the data are not ordered. Otherwise uniq won't completely work. – Studer Feb 21 '10 at 2:58
  • t now I can upvote ! You also helped me writing others scripts here ;) – Studer Feb 21 '10 at 3:17
  • 51
    Use sort -u instead of sort | uniq. It saves a process, reduces the total I/O, and reduces the total number of comparisons that have to be made. – Chris Johnsen Feb 22 '10 at 5:55
  • @ChrisJohnsen You should make that comment an answer as it's a better solution then the current given answer – Nico Van Belle May 24 '18 at 7:05
12

You can use:

grep -rohP "(mySearchString)" . | sort -u

-r: recursive

-o: only print matching part of the text

-h: don't print filenames

-P: Perl style regex (you may use -E instead depending on your case)

sort -u is better than sort | uniq, as @Chris Johnsen pointed out.

1
  • sort -R flag would use a random "sort" that only groups unique elements without actually sorting. I haven't benchmarked it, but it could be faster. – Dominykas Mostauskis Apr 3 '20 at 12:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.