I'm trying to echo all of the files in /foo while preserving the spaces within listed directories. Currently, a directory titled "bar1 bar2" will echo out as /path/to/foo/bar1/ then /bar2 then file.txt all on new lines. I need it to echo /path/to/foo/bar1 bar2/file.txt or /path/to/foo/bar1\ bar2/file.txt.

for file in $(find /path/to/foo/ -type f); do
  if [[ ... ]]; then
    echo "${file}"
  fi
done

Also tried:

for file in $(find /path/to/foo/ -type f | sed 's/ /\ /g'); do
  if [[ ... ]]; then
    echo "${file}"
    echo "$file" # variations i've tried
    echo $file   # variations i've tried
    echo ${file}
    otherCommand "${file}"
  fi
done
up vote 14 down vote accepted

The problem is not with $file, it's with your usage of $(find…).

Both regular variables and $(…) process substitutions are subject to exactly the same kind of word-splitting. If you put $(find…) in double quotes, you get a single word with all output. If you don't, it will be split up at any whitespace – not just newlines or some other magical boundary that you probably expected.

An important part of the above is that backslash escapes are not processed when expanding the variable. It's just split at spaces and that's it.

(In other words, quoting $file didn't help because it never had the correct value in the first place!)

If you want to process all files recursively, your options are:

  1. Read output from find in another way:

    find … -print | while read -r file; do
        echo "Got $file"
    done
    

    (Side note: File names may technically include newlines as well, so you might want to guard against that, as rare as it is:

    find … -print0 | while read -r -d "" file; do
        echo "Got $file"
    done
    
  2. For small amounts of files, use bash's extended wildcards:

    shopt -s globstar
    for file in /path/to/foo/**; do
        echo "Got $file"
    done
    

With large directories, the advantage of find | while read is that it's streamed – your script will process results as they come. Meanwhile, both $(find) and wildcards have to collect everything into memory first, and only then return the (possibly massive) results.

On the other hand, using a pipe affects the entire while loop (not just the read command), so if you want to run anything that wants to read keyboard input, you have to manually provide it with the original stdin, e.g. vim "$file" < /dev/tty.

  • 1
    Wow; your response time was incredible. Thank you so much. – super-frustrated Aug 24 '17 at 5:49
  • @super-frustrated Then you should mark it "accepted" – xenoid Aug 24 '17 at 6:44
  • Apologies. I don't use stackexchanges often. Marked it. – super-frustrated Aug 24 '17 at 6:49
  • 3
    It's best to use ... | IFS= read ...; without that, read will remove leading and trailing whitespace (which is rare, but perfectly legal in filenames). Also, note that the pipe makes the loop run in s subshell, so any variables set in it will not be available outside the loop. See BashFAQ #20: "How can I find and safely handle file names containing newlines, spaces or both?" for more tips and tricks. – Gordon Davisson Aug 25 '17 at 5:36
  • 1
    Leading spaces should be fairly obvious if you do ls or find -print. – G-Man Sep 12 '17 at 5:28

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.