In brief, what is needed is to add a suffix to end of the output when capturing it, and remove it when using the variable, and always quote variables:
# add a suffix to the end of the output when capturing it
var=$( cat /no/file/here 2>&1; echo extra)
# when using the variable, remove the suffix, and quote the variable
echo -n "${var%extra}"
Compare it to the following, in which the output from the second command will not end with a newline.
var=$( cat /no/file/here 2>&1)
echo -n "${var}"
Explanation of what is happening
Newlines in string
First determine the byte sequence that is expected:
$ { echo a; echo b; } | xxd
0000000: 610a 620a a.b.
Now, use Command Substitution (Section 3.5.4) to attempt to capture this byte sequence:
$ x=$( echo a; echo b; )
Then, do a simple echo to verify the byte sequence:
$ echo $x | xxd
0000000: 6120 620a a b.
So it seems the first newline was replaced with a space, and the second newline was left intact. But why ?
Lets look at what is actually happening here:
First, bash will do Shell Parameter Expansion (Section 3.5.3)
The ‘$’ character introduces parameter expansion, command
substitution, or arithmetic expansion. The parameter name or symbol
to be expanded may be enclosed in braces, which are optional but
serve to protect the variable to be expanded from characters
immediately following it which could be interpreted as part of the
name.
Then bash will do Word Splitting (Section 3.5.7)
The shell scans the results of parameter expansion, command
substitution, and arithmetic expansion that did not occur within
double quotes for word splitting.
The shell treats each character of $IFS as a delimiter, and splits
the results of the other expansions into words on these characters.
If IFS is unset, or its value is exactly , ...
Next, bash will treat it as a Simple Command (Section 3.2.1)
A simple command is the kind of command encountered most often.
It’s just a sequence of words separated by blanks, terminated by
one of the shell’s control operators (see Definitions). The first
word generally specifies a command to be executed, with the rest of
the words being that command’s arguments.
The definition of blanks (Section 2 - Definitions)
blank A space or tab character.
Finally, bash invokes the echo (Section 4.2 - Bash Builtin Commands) internal command
... Output the args, separated by spaces, terminated with a newline. ...
So to summarise, the newlines are removed by Word Splitting, and then echo gets 2 args, "a", and "b", and then outputs them seperated by spaces and ending with a newline.
Doing what @cyrus suggested (and suppress the newline from echo with -n) the result is better:
$ echo -n "$x" | xxd
0000000: 610a 62 a.b
Newlines at end of string
Its still not perfect though, the trailing newline is gone. Looking closer at Command Substitution (Section 3.5.4):
Bash performs the expansion by executing command and replacing the
command substitution with the standard output of the command, with
any trailing newlines deleted.
Now that its clear why the newline goes away, bash can be fooled into keeping it. To do this, add an additional string to the end, and remove it when using the variable:
$ x=$( echo a; echo b; echo -n extra )
$ echo -n "${x%extra}" | xxd
0000000: 610a 620a a.b.
Another example
$ cat /no/file/here 2>&1 | xxd
0000000: 6361 743a 202f 6e6f 2f66 696c 652f 6865 cat: /no/file/he
0000010: 7265 3a20 4e6f 2073 7563 6820 6669 6c65 re: No such file
0000020: 206f 7220 6469 7265 6374 6f72 790a or directory.
$ cat /no/file/here 2>&1 | cksum
3561909523 46
$
$ var=$( cat /no/file/here 2>&1; rc=${?}; echo extra; exit ${rc})
$ echo $?
1
$
$ echo -n "${var%extra}" | xxd
0000000: 6361 743a 202f 6e6f 2f66 696c 652f 6865 cat: /no/file/he
0000010: 7265 3a20 4e6f 2073 7563 6820 6669 6c65 re: No such file
0000020: 206f 7220 6469 7265 6374 6f72 790a or directory.
$ echo -n "${var%extra}" | cksum
3561909523 46