98

In linux, I can grep a string from a file using grep mySearchString myFile.txt. How can I only get the result which are unique?

Improve this question

2 Answers 2

Reset to default
155

You can achieve this with the sort and uniq utilities.

example:

[john@awesome ~]$ echo -e "test\ntest\ntest\nanother test\ntest"
test
test
test
another test
test
[john@awesome ~]$ echo -e "test\ntest\ntest\nanother test\ntest" | sort | uniq
another test
test

depending on the data you may want to utilize some of the switches as well.

Improve this answer
4
  • 11
    @John T - I would recommend to use sort before uniq in case the data are not ordered. Otherwise uniq won't completely work.
    – Studer
    Commented Feb 21, 2010 at 2:58
  • t now I can upvote ! You also helped me writing others scripts here ;)
    – Studer
    Commented Feb 21, 2010 at 3:17
  • 56
    Use sort -u instead of sort | uniq. It saves a process, reduces the total I/O, and reduces the total number of comparisons that have to be made.
    – Chris Johnsen
    Commented Feb 22, 2010 at 5:55
  • @ChrisJohnsen You should make that comment an answer as it's a better solution then the current given answer
    – Nico Van Belle
    Commented May 24, 2018 at 7:05
19

You can use:

grep -rohP "(mySearchString)" . | sort -u

-r: recursive

-o: only print matching part of the text

-h: don't print filenames

-P: Perl style regex (you may use -E instead depending on your case)

sort -u is better than sort | uniq, as @Chris Johnsen pointed out.

Improve this answer
1
  • sort -R flag would use a random "sort" that only groups unique elements without actually sorting. I haven't benchmarked it, but it could be faster.
    – Dominykas Mostauskis
    Commented Apr 3, 2020 at 12:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .