Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Using a loop such as

for i in `find . -name \*.txt` 

will break if some filenames have spaces in them.

What technique can I use to avoid this problem?

share|improve this question
1  
Note that files can also have newlines in their file name. That's why there's find -print0 and xargs -0. –  Daniel Beck Apr 8 '12 at 10:49
add comment

4 Answers

up vote 8 down vote accepted

Ideally you don't do it that way at all, because parsing filenames properly in a shell script is always difficult (fix it for spaces, you'll still have problems with other embedded characters, in particular newline). This is even listed as the first entry in the BashPitfalls page.

That said, there is a way to almost do what you want:

oIFS=$IFS
IFS=$'\n'

find . -name '*.txt' | while read -r i; do
  # use "$i" with whatever you're doing
done

IFS=$oIFS

Remember to also quote $i when using it, to avoid other things interpreting the spaces later. Also remember to set $IFS back after using it, because not doing so will cause bewildering errors later.

This does have one other caveat attached: what happens inside the while loop may take place in a subshell, depending on the exact shell you're using, so variable settings may not persist. The for loop version avoids that but at the price that, even if you apply the $IFS solution to avoid issues with spaces, you will then get in trouble if the find returns too many files.

At some point the correct fix for all of this becomes doing it in a language such as Perl or Python instead of shell.

share|improve this answer
    
Should the last line of your script be IFS="$oIFS" instead? –  Scott Wilson Apr 7 '12 at 23:18
    
Whoops, sorry yes –  geekosaur Apr 7 '12 at 23:19
    
I like the idea of just using Python to avoid all this. –  Scott Wilson Apr 7 '12 at 23:25
add comment

I disagree with the bash bahsers, because bash along with the *nix tool set is quite adept at handling files (including ones whose names have embedded whitespaces).

Actually find gives you fine grain contol over choosing which files to process... On the bash side, you really only need to realize that you must make you strings into bash words; typically by using "double quotes", or some other mechanism like using IFS, or find's {}

Note that in most/many situations you do not need to set and reset IFS; just use IFS locally as shown in the examples below. All three handle whitespace fine. Also you do not need a "standard" loop structure, because find's \; is effectively a loop; just put your loop logic into a bash function (if you aren't calling a standard tool).

IFS=$'\n' find ~/ -name '*.txt' -exec  function-or-util {} \;  

And, two more examples

IFS=$'\n' find ~/ -name '*.txt' -exec  printf 'Hello %s\n' {} \;  
IFS=$'\n' find ~/ -name '*.txt' -exec  echo {} \+ |sed 's/home//'  

'findalso allows you to pass multiple filenames as args to you script ..(if it suits your need: use+instead\;`)

share|improve this answer
    
There is some validity to both perspectives. When I was only working on my own files, I would just use find and not worry about it, because my files don't have spaces (or carriage returns!) in their names. But when you start working with other peoples' files, you have to use more robust techniques. –  Scott Wilson Apr 8 '12 at 11:54
add comment

Use find -print0 and pipe it to xargs -0, or write your own little C program and pipe it to your little C program. This is what -print0 and -0 were invented for.

Shell scripts aren't the best way to handle filenames with spaces in them: you can do it, but it gets clunky.

share|improve this answer
add comment

You can set the "internal field separator" (IFS) to something else than space for the loop argument splitting, e.g.

ORIGIFS=${IFS}
NL='
'
IFS=${NL}
for i in $(find . -name '*.txt'); do
    IFS=${ORIGIFS}
    #do stuff
done
IFS=${ORIGIFS}

I reset IFS after its use in find, mostly because it looks nice, I think. I haven't seen any problems in having it set to newline, but I think this is "cleaner".

Another method, depending on what you want to do with the output from find, is to either directly use -exec with the find command, or use -print0 and pipe it into xargs -0. In the first case find takes care of the file name escaping. In the -print0 case, find prints its output with a null separator, and then xargs splits on this. Since no file name can contain that character (what I know of), this is always safe as well. This it mostly useful in simple cases; and usually is not a great substitute for a full for loop.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.