Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two files. One huge one (200.000+ lines) called 'db' and one big one (15.000+ lines) called 'indices'.

What is the quickest way of filtering out the lines in 'db' containing any index (anywhere on the line) from 'indices'

The solution I could think of is

for index in $(cat indices); do
  grep $index db >> selection

but this is taking a long time.

Is there a faster approach in bash, linux?

share|improve this question
up vote 5 down vote accepted
fgrep -f indices db 

should be faster.

share|improve this answer
Indeed, it is way, way faster. Thanks! – Peter Smit Feb 9 '10 at 9:41
"fgrep" usually isn't the fastest grep variant. You may find egrep is faster still. – njd Feb 10 '10 at 17:42
Are you sure about it? – Zsolt Botykai May 27 '10 at 6:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .