0

I am trying to keep the first line that matches a string search and remove all following lines that have the same string matched.

Any ideas?

$ cat example-input.txt
Question one|some other text
Question two|dfgdfgdfgvd
Question one| dfg dfg dfg dfg
Question three|aa bb cc dd eee
Question one|zz aa BB yy qq
Question four|zz xx yy qq

cat example-input.txt | someuniqprogramoroptions "Question one" > example-output.txt

$ cat example-output.txt
Question one|some other text
Question two|dfgdfgdfgvd
Question three|aa bb cc dd eee
Question four|zz xx yy qq
$

UPDATE: thanks for the awk code G-Man, you're the man!

$ cat example-input.txt | ./awk-firstlines-only.sh
Question one|some other text
Question two|dfgdfgdfgvd
Question three|aa bb cc dd eee
Question four|zz xx yy qq
4
  • 1
    Welcome to SU, mike! Can you show please what you are trying (code), we can't read your mind! – Leo Chapiro Nov 11 '16 at 15:25
  • $ cat example-input.txt Question one|some other text Question two|dfgdfgdfgvd Question one| dfg dfg dfg dfg Question three|aa bb cc dd eee Question one|zz aa BB yy qq Question four|zz xx yy qq cat example-input.txt | someuniqueprogramand > example-output.txt $ cat example-output.txt Question one|some other text Question two|dfgdfgdfgvd Question three|aa bb cc dd eee Question four|zz xx yy qq $ – mike Nov 11 '16 at 19:16
  • Please do not respond in comments or by posting "answers" that aren't answers; edit your question to make it clearer and more complete. Give as precise an explanation as you can. – Scott Nov 11 '16 at 19:42
  • thanks, sorry this is my first question post on here, and still trying to figure out how to use it - thanks again – mike Nov 11 '16 at 19:56
2

Based on the example you’ve given, this awk command will produce the output you’re asking for:

awk '
    {
        i = index($0, "|")
        if (i == 0) {
                print "Error: line [" $0 "] does not have a \"|\" character."
        } else {
                prefix = substr($0, 1, i-1)
                if (++count[prefix] == 1) print
        }
    }'

The first two lines of code verify that each input line contains a |.  The next extracts the string before the first | character (e.g., “Question one”).  count is an associative array that we use to count how many times each prefix has appeared.  If this is #1 (i.e., the 1st appearance), print the line; otherwise, print nothing.

1
  • Or let defaults do most of the work: awk -F'|' '!count[$1]++' – dave_thompson_085 Nov 12 '16 at 16:28
0

If the first part has a fixed length, an alternative light solution is the command uniq combined with sort :

cat example-input.txt | sort | uniq -W 13

This is not very appropriate for your example, as you have a variable length and your file will be reordered, but this can be handy for similar jobs when you don't want to write a script.

0
perl -nle' /Question one/ and ($count++ or print) or print' example-input.txt

...on the input of OPs example will produce his output.

2
  • I believe OP was asking for more or less the exact opposite of this. – music2myear Jan 20 '17 at 17:43
  • The question didn't specify what to do with non-matching line. However his example with input and output do. – Kjetil S. Jan 21 '17 at 19:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.