Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a large text file with over 75 Million records. Each record consists of string tokens (attributes) and between each token there is a tab character. This is an example of two records:

43    ao489fer809    abcdefghikl    1.41    0

81    ijfij03iuff    abcdefghikl e89rt904i3 094utofgimf    0.13    1

So each record consists of 5 tokens (notice that the third token includes the space character).

Now let's suppose I have the string "abcdefghikl" -let's call it the key-. I am interested at the third token. So I want to compare the key to the third token and if the token is exactly the key (and doesn't just contain the key as my second record), to write the winning record to a new file. Any grep ideas?

share|improve this question
up vote 3 down vote accepted

I would use awk in this situation

awk -F"\t" '$3 == "abcdefghikl" {print}' filename > outputfile

Where you set the seperator to a tab, and the print only the lines where the third column matches the key.

share|improve this answer
Agreed, I was going to pipe sed in because I forgot how to set the separator. then direct to new file filename > newfile.txt – nerdwaller Oct 26 '12 at 19:22
@nerdwaller You can also set the field separator inside the script, but it doesn't make a difference afaik. – Bernhard Oct 26 '12 at 19:29
Works like a charm! – ddmichael Oct 26 '12 at 19:32
This doesn't work for the second of the two records. – RedGrittyBrick Oct 26 '12 at 19:37
@Bernhard Thanks, I just started learning the magic of awk & sed. I'll definitely note that one! – nerdwaller Oct 26 '12 at 19:38


$ cat -v td
43      ao489fer809     abcdefghikl     1.41    0
81      ijfij03iuff     abcdefghikl e89rt904i3 094utofgimf      0.13    1

$ grep -P '\tabcdefghikl\t' td
43      ao489fer809     abcdefghikl     1.41    0

or if finicky

$ grep -P '^[^\t]*\t[^\t]*\tabcdefghikl\t' td
43      ao489fer809     abcdefghikl     1.41    0
share|improve this answer
The awk example you give, does exactly what @ddmichael is looking for. – Bernhard Oct 26 '12 at 19:43
Ah, I completely misunderstood! Answer revised. – RedGrittyBrick Oct 26 '12 at 19:45
Your first example would also match if the key were in the second column. – Bernhard Oct 26 '12 at 19:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .