Tail and grep recursively?

Question

Trying to tail though a list of domains to search for a hacker code. All the files that I'm looking for are the same so I was trying something like this:

grep -rnl * -e "Ai9LbaFz7lC13SwzDxAYT72vwA"

Trying to see the file name / directory as well.

However that searches the entire file and that can be a waste of CPU and time so trying to write it with tail as I'm only needing to search a small section of the file. This worked:

tail -n10 * | grep --line-buffered -e '^==> .* <==$' -e 'Ai9LbaFz7lC13SwzDxAYT72vwA'

However tail doesn't seem to be recursive so it will only stay within it's dir. I am definitely no code master so I'm hoping someone can point me in the right direction.

Thanks.

Answer 1

The tail command has to read the entire file in order to calculate the last ten lines. It determines lines by looking for newline characters. So, it's reading the file looking for fixed strings.

Your grep, on the other hand, is reading the whole file looking for a fixed string. Sounds like pretty much the same thing. Granted, the grep is a tad more complex, but I'll bet the overhead of forking a new process and setting up the pipes is more than the grep overhead. You can, however, speed up grep a little by telling it that your pattern is a fixed string rather than a regexp. To do that, specify the "-F" option to grep (or use fgrep). If you have GNU grep, you can probably also use --mmap to speed things up.

Give grep -rnl * --mmap -F -e "Ai9LbaFz7lC13SwzDxAYT72vwA" a try and see if perhaps it's as fast or faster than your solution with tail. :)

Accuracy of the grep-only scan is another question, though, since this still isn't validating that the match is in the last 10 lines of the file. But I'm hoping to gloss over that part by suggesting the use of a post-processing command to count the lines in matching files or something. ;)

Answer 2

bash:

shopt -s globstar nullglob

for f in **; do
    if [[ -f $f ]] && tail -n 10 "$f" | grep -q --options 'pattern'; then
        echo "$f"
    fi
done