-1

I'm using these commands to to search in multiple pdfs, given a file path:

>>find /home/ad0x/Documents/Skola/Flervariabel/Tentor -name '*.pdf' -exec sh -c 'pdftotext "{}" - | grep --with-filename --label="{}" --color "phrase"' \;

Where phrase is the term you want to search for in the pdfs. This works as expected. I get all the occurrences of the word "volym". Output in terminal

When I try to do the same thing in a .sh script (search.sh)

#!/bin/bash
read -p "Enter term to search for: " phrase
find /home/ad0x/Documents/Skola/Flervariabel/Tentor -name '*.pdf' -exec sh -c 'pdftotext "{}" - | grep --with-filename --label="{}" --color "$phrase"' \;
echo "Search completed"

 >>./search.sh
 >>Enter term to search for:volym

It outputs every line in every pdf. The output: outputs every line in every pdf

I suspect it has something to with how read interprets the input, but I haven't found a solution to my problem online.

  • Changed it to single quotes, and now it works! Thank you! – ad0x Oct 9 '18 at 10:26
2

The direct culprit is $phrase in single quotes. This is not the only issue.

What happens

This is the relevant code (note I use ellipsis for the least interesting part; such line is meant to be understood by humans, not directly executed in a shell):

find … -exec sh -c 'pdftotext "{}" - | grep --with-filename --label="{}" --color "$phrase"' \;

The shell that interprets the script holds the value of the phrase variable; let's say the value is volym. In the above command everything that is in single quotes is left untouched because this is how single quoting works; so $phrase is not expanded yet. The shell only parses \ which informs it the following ; is not meant to separate commands, it should be treated as a command line argument for find.

When the find utility is run, this is what it sees as arguments (starting from 0th i.e. the find itself; one argument per line, except which denotes multiple less interesting arguments):

find
…
-exec
sh
-c
pdftotext "{}" - | grep --with-filename --label="{}" --color "$phrase"
;

Note the last but one line is one long argument.

Let's suppose foo.pdf is found and -exec is going to do its job. All arguments between -exec and ; become a new command after {} gets substituted by foo.pdf. The new command will be (again, starting from 0th argument; one argument per line):

sh
-c
pdftotext "foo.pdf" - | grep --with-filename --label="foo.pdf" --color "$phrase"

So sh runs, it gets -c and therefore knows the next argument should be run as if it was typed in the command line:

pdftotext "foo.pdf" - | grep --with-filename --label="foo.pdf" --color "$phrase"

This is the moment $phrase is expanded. It expands to nothing (the last word becomes "") because it hasn't been set in this shell. It would expand to volym if you exported the variable in your script; but you didn't. I wouldn't export though; in my opinion in this case exporting would unnecessarily pollute the environment.

Solution? Not yet

Putting $phrase outside of single quotes seems like a good idea. It will work in some cases. The most naive approach:

find … -exec sh -c 'pdftotext "{}" - | grep --with-filename --label="{}" --color "'$phrase'"' \;

It's flawed. With the phrase being " ; -exec rm "{} these are arguments our find will see:

find
…
-exec
sh
-c
pdftotext "{}" - | grep --with-filename --label="{}" --color ""
;
-exec
rm
"{}"
;

Your PDFs are gone. Artificial example? Maybe. Even if you're the only one using the script, such code injection vulnerability is nothing good.

This was because $phrase wasn't quoted at all. You probably know you should almost always put variables in double quotes. Let's do this. An improved approach:

find … -exec sh -c 'pdftotext "{}" - | grep --with-filename --label="{}" --color "'"$phrase"'"' \;

With the phrase being " ; -exec rm "{} this find will see:

find
…
-exec
sh
-c
pdftotext "{}" - | grep --with-filename --label="{}" --color "" ; -exec rm "{}"
;

Looks somewhat better; still flawed though, because for foo.pdf sh will try to run:

pdftotext "foo.pdf" - | grep --with-filename --label="foo.pdf" --color "" ; -exec rm "foo.pdf"

The last part will most likely throw an error because there is no -exec command. What if the phrase was " ; rm "{}? What if it was " ; rm -rf ~/".

There is more. Let the phrase be volym (quite safe) but name one of your PDFs "; rm -rf ~ #.pdf (this is possible in few filesystems, including ext family). After {}-s are substituted sh will run something like this:

pdftotext "/home/ad0x/…/"; rm -rf ~ #.pdf" - | grep …

I guesspdftotext will fail (irrelevant); then your files are gone; then # starts a comment, whatever.

Solution

This is the right way to pass your {} and $phrase to sh safely:

find … -exec sh -c 'pdftotext "$1" - | grep --with-filename --label="$1" --color "$2"' dummy {} "$phrase" \;

When this sh executes the given command string, $1 gets expanded to whatever find substituted for {}, $2 gets expanded to whatever the original shell substituted for $phrase. In the context of sh these parameters are properly quoted, so you can no more inject code. (This other answer of mine explains dummy).

Even now there's room for improvement. What if the phrase was -f? The grep part would eventually be:

grep --with-filename --label="…" --color "-f"

it would complain about the missing argument. Use -- to indicate end of options; -f after -- won't be treated as an option. The same applies to pdftotext (although in your particular case every path to PDF must begin with /home so it cannot be interpreted as an option; but in general $1 may expand to a string that looks like an option). Our sh invocation is already immune because sh takes options before a command string and our command string cannot be mistaken for an option (still sh -c -- 'pdftotext …' … will do no harm). More robust command:

find … -exec sh -c 'pdftotext -- "$1" - | grep --with-filename --label="$1" --color -- "$2"' dummy {} "$phrase" \;

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.