Is there a way to make bash not eat newlines in the backtick substitution?

For example:

var=`echo line one && echo line two`
echo $var

line one line two

What I want is

var=`echo line one && echo line two` # plus some magic
echo $var

line one
line two

It's not an issue with backticks substitution, but with echo; you have to quote the variable to get the control characters working:

$ var=`echo line one && echo line two`
$ echo "$var"
line one
line two
| improve this answer | |
  • So, bash removes newlines while expanding a variable on the command-lien? And that behavior can be disabled by quoting the variable? – slowpoison Mar 23 '12 at 1:48
  • 7
    bash performs parameters expansion before executing a command, quoting is necessary to inform bash that you want to print a single string and not 4 words (line, one, line and two). Try: echo a <many spaces> b and echo "a <many spaces> b". Take a look to this answer too. – cYrus Mar 23 '12 at 2:15

Even though @cyrus is correct - it does not actually answer the whole question, and there is no explanation of what is happening.

So lets walk through it.

Newlines in string

First determine the byte sequence that is expected:

$ { echo a; echo b; } | xxd
0000000: 610a 620a                                a.b.

Now, use Command Substitution (Section 3.5.4) to attempt to capture this byte sequence:

$ x=$( echo a; echo b; )

Then, do a simple echo to verify the byte sequence:

$ echo $x | xxd
0000000: 6120 620a                                a b.

So it seems the first newline was replaced with a space, and the second newline was left intact. But why ?

Lets look at what is actually happening here:

First, bash will do Shell Parameter Expansion (Section 3.5.3)

The ‘$’ character introduces parameter expansion, command substitution, or arithmetic expansion. The parameter name or symbol to be expanded may be enclosed in braces, which are optional but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.

Then bash will do Word Splitting (Section 3.5.7)

The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.

The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words on these characters. If IFS is unset, or its value is exactly , ...

Next, bash will treat it as a Simple Command (Section 3.2.1)

A simple command is the kind of command encountered most often. It’s just a sequence of words separated by blanks, terminated by one of the shell’s control operators (see Definitions). The first word generally specifies a command to be executed, with the rest of the words being that command’s arguments.

The definition of blanks (Section 2 - Definitions)

blank A space or tab character.

Finally, bash invokes the echo (Section 4.2 - Bash Builtin Commands) internal command

... Output the args, separated by spaces, terminated with a newline. ...

So to summarise, the newlines are removed by Word Splitting, and then echo gets 2 args, "a", and "b", and then outputs them seperated by spaces and ending with a newline.

Doing what @cyrus suggested (and suppress the newline from echo with -n) the result is better:

$ echo -n "$x" | xxd
0000000: 610a 62                                  a.b

Newlines at end of string

Its still not perfect though, the trailing newline is gone. Looking closer at Command Substitution (Section 3.5.4):

Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted.

Now that its clear why the newline goes away, bash can be fooled into keeping it. To do this, add an additional string to the end, and remove it when using the variable:

$ x=$( echo a; echo b; echo -n extra )
$ echo -n "${x%extra}" | xxd
0000000: 610a 620a                                a.b.


Add extra part to end of output, and quote variables:

$ cat /no/file/here 2>&1 | xxd
0000000: 6361 743a 202f 6e6f 2f66 696c 652f 6865  cat: /no/file/he
0000010: 7265 3a20 4e6f 2073 7563 6820 6669 6c65  re: No such file
0000020: 206f 7220 6469 7265 6374 6f72 790a        or directory.
$ cat /no/file/here 2>&1 | cksum
3561909523 46
$ var=$( cat /no/file/here 2>&1; rc=${?}; echo extra; exit ${rc})
$ echo $?
$ echo -n "${var%extra}" | xxd
0000000: 6361 743a 202f 6e6f 2f66 696c 652f 6865  cat: /no/file/he
0000010: 7265 3a20 4e6f 2073 7563 6820 6669 6c65  re: No such file
0000020: 206f 7220 6469 7265 6374 6f72 790a        or directory.
$ echo -n "${var%extra}" | cksum
3561909523 46
| improve this answer | |

I would have put my small addition in a comment on the answer of cYrus, but I cannot yet comment on answers I think. So I'll duplicate part of cYrus his answer for completeness.

What happens in the first line of your code is that indeed the backticks substitution eats the trailing newlines of the output of the substituted command, as documented. But it does not eat the newline between your two lines.

What happens in the second line of code is that echo is executed with two arguments, the first and the second line. Quoting the expanded variable will fix that: echo "$var" This will preserve the newline between your line one and line two. And echo will add a trailing newline by default so that everything is okay.

Now if you wanted to preserve all trailing newlines in the output of some command substitution, a workaround is to add another line the output, which you can remove after the command substitution, see here .

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.