Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two files. One huge one (200.000+ lines) called 'db' and one big one (15.000+ lines) called 'indices'.

What is the quickest way of filtering out the lines in 'db' containing any index (anywhere on the line) from 'indices'

The solution I could think of is

for index in $(cat indices); do
  grep $index db >> selection

but this is taking a long time.

Is there a faster approach in bash, linux?

share|improve this question
up vote 5 down vote accepted
fgrep -f indices db 

should be faster.

share|improve this answer
Indeed, it is way, way faster. Thanks! – Peter Smit Feb 9 '10 at 9:41
"fgrep" usually isn't the fastest grep variant. You may find egrep is faster still. – njd Feb 10 '10 at 17:42
Are you sure about it? – Zsolt Botykai May 27 '10 at 6:47

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.