Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have two files. One huge one (200.000+ lines) called 'db' and one big one (15.000+ lines) called 'indices'.

What is the quickest way of filtering out the lines in 'db' containing any index (anywhere on the line) from 'indices'

The solution I could think of is

for index in $(cat indices); do
  grep $index db >> selection
done;

but this is taking a long time.

Is there a faster approach in bash, linux?

share|improve this question
add comment

1 Answer

up vote 5 down vote accepted
fgrep -f indices db 

should be faster.

share|improve this answer
    
Indeed, it is way, way faster. Thanks! –  Peter Smit Feb 9 '10 at 9:41
    
"fgrep" usually isn't the fastest grep variant. You may find egrep is faster still. –  njd Feb 10 '10 at 17:42
1  
Are you sure about it? –  Zsolt Botykai May 27 '10 at 6:47
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.