Questions tagged [uniq]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
31 views

Sort lines in file by first field treated as number in shell

I have a file with the format shown below (a number + an IPv4 address). This format was generated by using uniq -c. I want to sort the lines after the first number in each of them. I would've gone for ...
2
votes
1answer
730 views

Grep on macOS: find unique occurrences of a capturing group in regular expression

I am on macOS and would like to get into using grep (or a similar tool) to find unique occurrences of a certain pattern in a codebase. For example, for finding all console.somemethod() calls in ...
-1
votes
2answers
570 views

Sorting files by “line content” frequency - print duplicates

Imagine there is a file - a b b b b c c d d d I want the output to be sorted based on frequency (I want the duplicate lines to be printed as well) as - b b b b d d d c c a
18
votes
4answers
16k views

Is there a Windows equivalent to the Unix uniq?

I need remove duplicate lines from a text file, it is simple in Linux using cat file.txt |sort | uniq when file.txt contains aaa bbb aaa ccc It will output aaa bbb ccc Is there a Windows ...
-1
votes
2answers
146 views

find number of teams from a file [closed]

I have started learning bash very recently and have to find the number of teams from a file, the file can be downloaded from this link. Below is a sample I understand that I have to use cut and uniq ...
4
votes
2answers
4k views

Remove non-duplicate lines in Linux

how can I remove non-duplicate lines from text file using any linux program linke sed, awk or any other? Example: abc bbc abc bbc ccc bbc Result: abc bbc abc bbc bbc Second list have removed ccc ...
-1
votes
1answer
319 views

How can I filter unique lines based on a substring?

If I have a set of URLs in a file like: http://example.com/do?foo=bar&etc=123 http://example.com/do?etc=456&foo=bar I want to filter this out so that there's only one url for each unique ...
3
votes
2answers
143 views

How do I make uniq only consider the first field?

I am using FreeBSD 3.2-RELEASE If I have some sorted text, like this last output— zikla13:Oct:20:22:34 zikla13:Oct:5:00:31 zikla14:Oct:17:22:01 zikla14:Oct:12:23:35 zikla14:Oct:12:23:34 zikla14:Oct:...
0
votes
1answer
915 views

Remove list of items from another file in bash

What would be the most efficient method (no repeated command execution) to remove items listed in one file from another file (unordered) ? One can easily get the list of non-matching items in the ...
1
vote
1answer
719 views

Grep matching lines but return matching key IDs in GnuPG output

Having this input: rsa2048/C7927B82 2015-08-30 rsa2048/FB2D99F9 2015-08-30 I want grep to only return the key ID, for example: C7927B92. Using the pattern from Grep characters before and after match?...
0
votes
1answer
1k views

Printing unique lines with uniq, OS X Yosemite

I am trying to print the unique entries in a column of a .csv file. For this, I tried the following: awk -F "," '{print $6}' dataCoarse.csv | uniq -u which just prints the 6th column as it is. There ...
0
votes
2answers
139 views

Sort on ID, then sort on date, then remove lines which have the older date?

I have a couple of tab delimited files. Each file is structured like so: ID Title Rating Date_Rated What I want to do is merge all these files into one, ...
0
votes
3answers
228 views

Filtering 2nd field from a data set and then using uniq on the output

I've got a dataset that goes like this: AAAAA 11111 Data1 AAAAA 11111 Data2 AAAAA 11111 Data3 AAAAA 11112 Data4 AAAAA 11112 Data5 AAAAA 11112 Data6 AAAAA 11112 Data7 AAAAA 11113 Data8 AAAAA 11114 ...
0
votes
1answer
65 views

Sort piped to uniq unexpected output

I have a file that looks like this: "string",192.168.1.0,,TX,192.168.1.2,,,100,0,0,10,10,10,, "string",192.168.1.0,,TX,192.168.1.1,,,100,0,0,10,10,10,, "string",192.168.1.0,,TX,192.168.1.4,,,100,0,0,...
1
vote
3answers
3k views

remove lines with duplicate words

I have a sorted file with lines like this word1 abca word1 abcb word1 abcc word2 abca word2 abcb word3 abbb ........... and i want to have something like this word1 abca word2 abca word3 ...
0
votes
1answer
530 views

How to print statistics from lines in a file

I often find myself computing percentages manually after filtering through a log file to find the proportion of X or Y in it. Can this easily be achieved via common CLI tools?
2
votes
1answer
726 views

Counting duplicates lines from a stream

I'm currently parsing apache logs with that command: tail -f /opt/apache/logs/access/gvh-access_log.1365638400 | grep specific.stuff. | awk '{print $12}' | cut -d/ -f3 > ~/logs The output is a ...
7
votes
3answers
2k views

Remove duplicates in each line of a file

How can I remove duplicates in each line, for example here? 1 1 1 2 1 2 3 5 5 4 1 2 3 3 I'd like to get this output: 1 2 3 5 4 1 2 3 There are lots of lines (100,000) and in each line I want ...
0
votes
2answers
5k views

linux sort -n uniq -c

I have a text file which contains about 50MB of text: valA valB valC valZ valB Whats the quickest way to: cat the file Display the unique appearences
0
votes
1answer
107 views

gnu `uniq -c`: display counts graphically as bars

With sort $file | uniq -c, it's possible to sum up files/input and show the number occurences like that: 13 foo 1 bar To get a better grasp of this data, I'd like to see them graphically ...
1
vote
1answer
408 views

gnu sort/uniq: sorting by number of times

How can I use GNU sort and uniq to have the most common occurrences on top instead of numerical or alphanumerical sorting? Example list.txt: 1 2 2 2 3 3 Since '2' occurs 3 times, should be on top, ...
0
votes
2answers
189 views

lines identical except for their first word, put the first word in a variable

I have file(sudoers) in which lines can be identical except for the first word in them (different users can execute the same set of commands). I am able to extract such a line using the command: # ...
0
votes
1answer
609 views

option -f for linux command

this is the content of test file: this this this that is that a that MAP that map that when I do uniq testCase -f1, the result is: this this this that which is reasonable, but if ...