16

I have a string like this:

"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"

I want to be able to split it like this:

aString that may haveSpaces IN IT
bar
foo
bamboo  
bam boo

How do I do that? (preferrably using a one-liner)

9
  • @DavidPostill the questions are quite different actually.
    – foxneSs
    Apr 17 '16 at 10:50
  • Not really, it's the same general problem.
    – DavidPostill
    Apr 17 '16 at 11:01
  • @DavidPostill - This is a much simpler problem: all it needs is for l in "aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"; do echo $l; done
    – AFH
    Apr 17 '16 at 11:24
  • @AFH lol. I just posted a much longer answer. The only difference in the output was that mine preserved the "s. I missed the fact that the OP doesn't need them in the output.
    – DavidPostill
    Apr 17 '16 at 11:27
6

When I saw David Postill's answer, I thought "there must be a simpler solution". After some experimenting I found the following works:-

string='"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"'
echo $string
eval 'for word in '$string'; do echo $word; done'

This works because eval expands the line (removing the quotes and expanding string) before executing the resultant line (which is the in-line answer):

for word in "aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"; do echo $word; done

An alternative which expands to the same line is:

eval "for word in $string; do echo \$word; done"

Here string is expanded within the double-quotes, but the $ must be escaped so that word in not expanded before the line is executed (in the other form the use of single-quotes has the same effect). The results are:-

[~/]$ string='"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"'
[~/]$ echo $string
"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"
[~/]$ eval 'for word in '$string'; do echo $word; done'
aString that may haveSpaces IN IT
bar
foo
bamboo
bam boo
[~/]$ eval "for word in $string; do echo \$word; done"
aString that may haveSpaces IN IT
bar
foo
bamboo
bam boo
8

The simplest solution is using making an array of the quoted args which you could then loop over if you like or pass directly to a command.

eval "array=($string)"

for arg in "${array[@]}"; do echo "$arg"; done   

p.s. Please comment if you find a simpler way without eval.

Edit:

Building on @Hubbitus' answer we have a fully sanitized and properly quoted version. Note: this is overkill and will actually leave additional backslashes in double or single quoted sections preceding most punctuation but is invulnerable to attack.

declare -a "array=($( echo "$string" | sed 's/[][`~!@#$%^&*():;<>.,?/\|{}=+-]/\\&/g' ))"

I leave it to the interested reader to modify as they see fit http://ideone.com/FUTHhj

1
  • I have a testcase that breaks this: string="bash -c 'echo \$USER'" which "leaves" the backslash. You sometimes need this for e.g ssh
    – Flamefire
    Jun 6 '19 at 15:18
3

It looks that xargs can do it pretty well:

$ a='"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"'
$ printf "%s" "$a" | xargs -n 1 printf "%s\n"
aString that may haveSpaces IN IT
bar
foo
bamboo
bam boo
2

How do I do that?

$ for l in "aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"; do echo $l; done
aString that may haveSpaces IN IT
bar
foo
bamboo
bam boo

What do I do if my string is in a bash variable?

The simple approach of using the bash string tokenizer will not work, as it splits on every space not just the ones outside quotes:

DavidPostill@Hal /f/test
$ cat ./test.sh
#! /bin/bash
string='"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"'
for word in $string; do echo "$word"; done

DavidPostill@Hal /f/test
$ ./test.sh
"aString
that
may
haveSpaces
IN
IT"
bar
foo
"bamboo"
"bam
boo"

To get around this the following shell script (splitstring.sh) shows one approach:

#! /bin/bash 
string=$(cat <<'EOF'
"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo" 
EOF
)
echo Source String: "$string"
results=()
result=''
inside=''
for (( i=0 ; i<${#string} ; i++ )) ; do
    char=${string:i:1}
    if [[ $inside ]] ; then
        if [[ $char == \\ ]] ; then
            if [[ $inside=='"' && ${string:i+1:1} == '"' ]] ; then
                let i++
                char=$inside
            fi
        elif [[ $char == $inside ]] ; then
            inside=''
        fi
    else
        if [[ $char == ["'"'"'] ]] ; then
            inside=$char
        elif [[ $char == ' ' ]] ; then
            char=''
            results+=("$result")
            result=''
        fi
    fi
    result+=$char
done
if [[ $inside ]] ; then
    echo Error parsing "$result"
    exit 1
fi

echo "Output strings:"
for r in "${results[@]}" ; do
    echo "$r" | sed "s/\"//g"
done

Output:

$ ./splitstring.sh
Source String: "aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"
Output strings:
aString that may haveSpaces IN IT
bar
foo
bamboo
bam boo

Source: StackOverflow answer Split a string only by spaces that are outside quotes by choroba. Script has been tweaked to match the requirements of the question.

0
2

You may do it with declare instead of eval, for example:

Instead of:

string='"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"'
echo "Initial string: $string"
eval 'for word in '$string'; do echo $word; done'

Do:

declare -a "array=($string)"
for item in "${array[@]}"; do echo "[$item]"; done

But please note, it is not much safer if input comes from user!

So, if you try it with say string like:

string='"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo" `hostname`'

You get hostname evaluated (there off course may be something like rm -rf /)!

Very-very simple attempt to guard it just replace chars like backtrick ` and $:

string='"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo" `hostname`'
declare -a "array=( $(echo $string | tr '`$<>' '????') )"
for item in "${array[@]}"; do echo "[$item]"; done

Now you got output like:

[aString that may haveSpaces IN IT]
[bar]
[foo]
[bamboo]
[bam boo]
[?hostname?]

More details about methods and pros and cons you may found in that good answer: https://stackoverflow.com/questions/17529220/why-should-eval-be-avoided-in-bash-and-what-should-i-use-instead/17529221#17529221

But there still leaved vector for attack. I very want have in bash method of string quote like in double quotes (") but without interpreting content.

0

Expanding on Oliver's answer, using xargs and declare the list can be translsted into an assignment expression safe to eval

echo "1 2 '3 4' 5" |  xargs bash -c 'declare -a array=("$@"); declare -p array' --
-1

use awk

echo '"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"' | awk 'BEGIN {FPAT = "([^ ]+)|(\"[^\"]+\")"}{for(i=1;i<=NF;i++){gsub("\"","",$i);print $i} }'
aString that may haveSpaces IN IT
bar
foo
bamboo
bam boo

Or convert the space to "%20" or "_", so it can be processed by next command throw pip:

echo '"aString that may haveSpaces IN IT" bar foo "bamboo" "bam boo"' | awk 'BEGIN {FPAT = "([^ ]+)|(\"[^\"]+\")"}{for(i=1;i<=NF;i++){gsub("\"","",$i);gsub(" ","_",$i)} print }'
aString_that_may_haveSpaces_IN_IT bar foo bamboo bam_boo

reference:Awk consider double quoted string as one token and ignore space in between

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.