The tag has no usage guidance.

learn more… | top users | synonyms

-1
votes
1answer
20 views

How can I filter unique lines based on a substring?

If I have a set of URLs in a file like: http://example.com/do?foo=bar&etc=123 http://example.com/do?etc=456&foo=bar I want to filter this out so that there's only one url for each unique ...
3
votes
2answers
40 views

How do I make uniq only consider the first field?

I am using FreeBSD 3.2-RELEASE If I have some sorted text, like this last output— zikla13:Oct:20:22:34 zikla13:Oct:5:00:31 zikla14:Oct:17:22:01 zikla14:Oct:12:23:35 zikla14:Oct:12:23:34 zikla14:Oct:...
0
votes
1answer
49 views

Remove list of items from another file in bash

What would be the most efficient method (no repeated command execution) to remove items listed in one file from another file (unordered) ? One can easily get the list of non-matching items in the ...
1
vote
1answer
94 views

Grep matching lines but return matching key IDs in GnuPG output

Having this input: rsa2048/C7927B82 2015-08-30 rsa2048/FB2D99F9 2015-08-30 I want grep to only return the key ID, for example: C7927B92. Using the pattern from Grep characters before and after ...
0
votes
1answer
79 views

Printing unique lines with uniq, OS X Yosemite

I am trying to print the unique entries in a column of a .csv file. For this, I tried the following: awk -F "," '{print $6}' dataCoarse.csv | uniq -u which just prints the 6th column as it is. ...
0
votes
2answers
63 views

Sort on ID, then sort on date, then remove lines which have the older date?

I have a couple of tab delimited files. Each file is structured like so: ID Title Rating Date_Rated What I want to do is merge all these files into one, ...
0
votes
3answers
29 views

Filtering 2nd field from a data set and then using uniq on the output

I've got a dataset that goes like this: AAAAA 11111 Data1 AAAAA 11111 Data2 AAAAA 11111 Data3 AAAAA 11112 Data4 AAAAA 11112 Data5 AAAAA 11112 Data6 AAAAA 11112 Data7 AAAAA 11113 Data8 AAAAA 11114 ...
0
votes
1answer
31 views

Sort piped to uniq unexpected output

I have a file that looks like this: "string",192.168.1.0,,TX,192.168.1.2,,,100,0,0,10,10,10,, "string",192.168.1.0,,TX,192.168.1.1,,,100,0,0,10,10,10,, "string",192.168.1.0,,TX,192.168.1.4,,,100,0,0,...
1
vote
3answers
569 views

remove lines with duplicate words

I have a sorted file with lines like this word1 abca word1 abcb word1 abcc word2 abca word2 abcb word3 abbb ........... and i want to have something like this word1 abca word2 abca word3 ...
0
votes
1answer
79 views

How to print statistics from lines in a file

I often find myself computing percentages manually after filtering through a log file to find the proportion of X or Y in it. Can this easily be achieved via common CLI tools?
1
vote
1answer
356 views

Counting duplicates lines from a stream

I'm currently parsing apache logs with that command: tail -f /opt/apache/logs/access/gvh-access_log.1365638400 | grep specific.stuff. | awk '{print $12}' | cut -d/ -f3 > ~/logs The output is a ...
6
votes
3answers
679 views

Remove duplicates in each line of a file

How can I remove duplicates in each line, for example here? 1 1 1 2 1 2 3 5 5 4 1 2 3 3 I'd like to get this output: 1 2 3 5 4 1 2 3 There are lots of lines (100,000) and in each line I want ...
0
votes
2answers
2k views

linux sort -n uniq -c

I have a text file which contains about 50MB of text: valA valB valC valZ valB Whats the quickest way to: cat the file Display the unique appearences
0
votes
1answer
95 views

gnu `uniq -c`: display counts graphically as bars

With sort $file | uniq -c, it's possible to sum up files/input and show the number occurences like that: 13 foo 1 bar To get a better grasp of this data, I'd like to see them graphically ...
1
vote
1answer
273 views

gnu sort/uniq: sorting by number of times

How can I use GNU sort and uniq to have the most common occurrences on top instead of numerical or alphanumerical sorting? Example list.txt: 1 2 2 2 3 3 Since '2' occurs 3 times, should be on top, ...
0
votes
2answers
145 views

lines identical except for their first word, put the first word in a variable

I have file(sudoers) in which lines can be identical except for the first word in them (different users can execute the same set of commands). I am able to extract such a line using the command: # ...
0
votes
1answer
323 views

option -f for linux command

this is the content of test file: this this this that is that a that MAP that map that when I do uniq testCase -f1, the result is: this this this that which is reasonable, but ...