Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm trying to search a lot of XML files for certain strings but not other strings, and I'm having trouble putting a command together to do it. I only want it to list the file names that match the inclusion/exclusion criteria. I've been trying:

find . -name *.xml -exec grep -li "string1\|string2" {} \; | xargs grep -Li "string3\|string4"

But I'm having trouble because the file names that are returned from find have spaces in their names and the second grep breaks them all into little chunks and of course doesn't find such files. I've tried adding -0 to the xargs, and it removes the errors, but it says "File name too long" and only executes the first grep.

How do I tweak this command to have it work properly on files with spaces in the names?

share|improve this question
    
Doesnt this belong to super user? –  Cem Kalyoncu Jun 13 '11 at 13:34
2  
bash scripting is both programming related and system-maintenance related so I'd say it belongs at either site. –  aioobe Jun 13 '11 at 13:35
1  
Not on a unix box at the moment, but I'd guess (assuming GNU grep), you could try something like find . -name "*.xml" -exec grep -liZ "string1\|string2" {} \; | xargs -0 grep -Li "string3\|string4" (The Z flag to grep outputs zero terminators on its results list, which should be compatible with xargs -0) –  Matt Gibson Jun 13 '11 at 13:47
    
@pilcrow That won't quite work in this circumstance, because the -print0 will just be printing the filenames, not selecting only filenames matched by the first grep. It's the output of the first grep that's going through the pipe, which is why I think the -Z flag on that is what's needed. Hard to tell without a box to play with, though :) –  Matt Gibson Jun 13 '11 at 13:53
    
@Matt, yes, and there's more than one way to do it. :) –  pilcrow Jun 13 '11 at 13:56
show 1 more comment

migrated from stackoverflow.com Jun 13 '11 at 15:13

This question came from our site for professional and enthusiast programmers.

4 Answers

up vote 3 down vote accepted

There's more than one way to do it. This should keep the total number of processes executed to a minimum:

find . -name \*.xml -print0          \   # List of *.xml files (NUL-terminated)
  | xargs -0 grep -Zli 'string[12]'  \   # is input to first grep, which sends (NUL-term'd)
  | xargs -0 grep -Li  'string[34]'      # file list to second grep

Thanks Matt Gibson for reminding us of the -Z flag to GNU grep.

share|improve this answer
    
Yes, that's a nice method. @Cicerone -- also note that @pilcrow and I are both assuming you need to escape your *.xml somehow to avoid shell expansion. –  Matt Gibson Jun 13 '11 at 13:58
    
This one works too and it's extremely fast! –  Cicerone Cojocaru Jun 13 '11 at 14:16
add comment

You can do it all in one find which avoids all the problems with spaces in filenames. Something like

find . -exec grep -liq "string1\|string2" {} \; -not -exec grep -liq "string3\|string4" {} \; -print

The "-q" suppresses all grep output. The -exec primary returns true when the process exits with 0 status, as grep does when it finds a match, and the -not primary reverses that. So we directly place two conditions on the find, and as a result we only print the filenames that satisfy both -- no pipe required!

share|improve this answer
    
I was just working out the details of this when your answer appeared. –  Colin Fine Jun 13 '11 at 13:47
    
Sorry @Colin. I hate when that happens to me :) –  Ernest Friedman-Hill Jun 13 '11 at 13:48
    
This works too, thanks Ernest! –  Cicerone Cojocaru Jun 13 '11 at 14:12
    
+1 for a solution where it is highly likely that the file is still in the disk cache when the second grep goes to work. –  Ole Tange Jun 14 '11 at 14:25
add comment

As mentioned in my comment, I think all you're missing is the -Z flag on the first grep to go along with the -0 that you were trying on your xargs:

find . -name "*.xml" -exec grep -liZ "string1\|string2" {} \; | xargs -0 grep -Li "string3\|string4"
share|improve this answer
    
+1 for the -Z flag. –  pilcrow Jun 13 '11 at 14:08
    
This works very well! Thanks Matt! –  Cicerone Cojocaru Jun 13 '11 at 14:08
add comment

If the limiting factor is CPU (i.e. your disk is fast) and you have more CPU cores, you can use GNU Parallel:

find . -type f| parallel grep -Lq foo {} '||' grep -l bar {}

By running the two greps immediately after eachother chances are that the files are still in the disk cache. If the disk seeks are slow, you can add -j1 to disable the parallelism.

Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.