Making BASH script `for` handle filenames with spaces (or workaround)

Question

Whilst I have been using BASH for several years, my experience with BASH scripting is relatively limited.

My code is as below. It should grab the entire directory structure from within the current directory and replicate it into $OUTDIR.

for DIR in `find . -type d -printf "\"%P\"\040"`
do
  echo mkdir -p \"${OUTPATH}${DIR}\"        # Using echo for debug; working script will simply execute mkdir
  echo Created $DIR
done

The problem is, here is a sample of my file structure:

$ ls
Expect The Impossible-Stellar Kart
Five Iron Frenzy - Cheeses...
Five Score and Seven Years Ago-Relient K
Hello-After Edmund
I Will Go-Starfield
Learning to Breathe-Switchfoot
MMHMM-Relient K

Note the spaces :-S And for takes parameters word by word, so my script's output looks something like this:

Creating directory structure...
mkdir -p "/myfiles/multimedia/samjmusicmp3test/Learning"
Created Learning
mkdir -p "/myfiles/multimedia/samjmusicmp3test/to"
Created to
mkdir -p "/myfiles/multimedia/samjmusicmp3test/Breathe-Switchfoot"
Created Breathe-Switchfoot

But I need it to grab whole filenames (one line at a time) from the output of find. I have also tried making find put double-quotes around each filename. But this doesn't help.

for DIR in `find . -type d -printf "\"%P\"\040"`

And output with this changed line:

Creating directory structure...
mkdir -p "/myfiles/multimedia/samjmusicmp3test/"""
Created ""
mkdir -p "/myfiles/multimedia/samjmusicmp3test/"Learning"
Created "Learning
mkdir -p "/myfiles/multimedia/samjmusicmp3test/to"
Created to
mkdir -p "/myfiles/multimedia/samjmusicmp3test/Breathe-Switchfoot""
Created Breathe-Switchfoot"

Now, I need some way that I can iterate through like this, because I also wish to run a more complicated command involving gstreamer on each file in a following similar structure. How should I be doing this?

Edit: I need a code structure which will allow me to run multiple lines of code for each directory/file/loop. Sorry if I was unclear.

Solution: I initially tried:

find . -type d | while read DIR
do
  mkdir -p "${OUTPATH}${DIR}"
  echo Created $DIR
done

This worked fine for the most part. However, I later found that since the pipe results in the while loop running in a subshell, any variables set in the loop were later unavailable which made implementing an error counter quite difficult. My final solution (from this answer on SO):

while read DIR
do
  mkdir -p "${OUTPATH}${DIR}"
  echo Created $DIR
done < <(find . -type d)

This later allowed me to conditionally increment variables within the loop which would remain available later in the script.

Answer 1

You need to pipe the find into a while loop.

find ... | while read -r dir
do
    something with "$dir"
done

Also, you won't need to use -printf in this case.

You can make this proof against files with newlines in their names, if you wish, by using a nullbyte delimiter (that being the only character which cannot appear in a *nix filepath):

find ... -print0 | while read -d '' -r dir
do
    something with "$dir"
done

You will also find using $() instead of backticks to be more versatile and easier. They can be nested much more easily and quoting can be done much more easily. This contrived example will illustrate these points:

echo "$(echo "$(echo "hello")")"

Try to do that with backticks.

Answer 2

See this answer I wrote a few days ago for an example of a script that handles filenames with spaces.

There's a slightly more convoluted (but more concise) way to achieve what you're trying to do though:

find . -type d -print0 | xargs -0 -I {} mkdir -p ../theredir/{}

-print0 tells find to separate the arguments with a null; the -0 to xargs tells it to expect arguments seperated by nulls. This means that it handles spaces fine.

-I {} tells xargs to replace the string {} with the filename. This also implies that only one filename should be used per commandline (xargs will normally stuff as many as will fit on the line)

The rest should be obvious.

Answer 3

The issue you're encountering is the for statement is responding to the find as separate arguments. The space delimiter. You need to use bash's IFS variable to not split on space.

Here is a link that explains how to do this.

The IFS internal variable

One way around this problem is to change Bash's internal IFS (Internal Field Separator) variable so that it splits fields by something other than the default whitespace (space, tab, newline), in this case, a comma.

#!/bin/bash
IFS=$';'

for I in `find -type d -printf \"%P\"\;`
do
   echo "== $I =="
done

Set your find to output your field delimiter after the %P and set your IFS appropriately. I picked semi-colon since it's highly unlikely to found in your filenames.

The other alternative is to call mkdir from the find directly via -exec do you can skip the for loop altogether. That's if you don't need to do any additional parsing.

Answer 4

If the body of your loop is more than a single command, it is possible to use xargs to drive a shell script:

export OUTPATH=/some/where/else/
find . -type d -print0 | xargs -0 bash -c 'for DIR in "$@"; do
  printf "mkdir -p %q\\n" "${OUTPATH}${DIR}"        # Using echo for debug; working script will simply execute mkdir
  echo Created $DIR
done' -

Be sure to include the trailing dash (or some other ‘word’) if the shell is of the Bourne/POSIX variety (it is used to set $0 in the shell script). Also, care must be taken with quoting, since the shell script is being written inside a quoted string instead of directly at the prompt.

Answer 5

in your updated question you have

mkdir -p \"${OUTPATH}${DIR}\"

this should be

mkdir -p "${OUTPATH}${DIR}"

Answer 6

find . -type d -exec mkdir -p "{}\040" ';' -exec echo "Created {}\040" ';'

Answer 7

or to make the whole thing much less complicated:

% rsync -av --include='*/' --exclude='*' SRC DST

this replicates the directory structure of SRC into DST.

Answer 8

If you have GNU Parallel http:// www.gnu.org/software/parallel/ installed you can do this:

find . -type d | parallel echo making {} ";" mkdir -p /tmp/outdir/{} ";" echo made {}

Watch the intro video for GNU Parallel to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ