4

When I pipe the output from docker exec to another command, it appears to mess up my terminal's line endings.

For example:

$ docker exec -it foo sh -c 'echo {\"a\":\"b\",\"c\":\"d\"}' | jq
{
   "a": "b",
              "c": "d"
                      }
                       $

But if I then immediately run the same command, in the same terminal, on the host:

$ sh -c 'echo {\"a\":\"b\",\"c\":\"d\"}' | jq
{
  "a": "b",
  "c": "d"
}

...as expected.

If I pipe the output through xxd, it appears to introduce a CR character as well as the LF (0d0a, rather than 0a); it also messes up the output from xxd:

$ docker exec -i -t foo sh -c 'echo {\"a\":\"b\",\"c\":\"d\"}' | xxd
00000000: 7b22 6122 3a22 6222 2c22 6322 3a22 6422  {"a":"b","c":"d"
                                                                   00000010: 7d0d 0a  

If I attempt to rule out docker by emitting the \r\n myself, it's fine:

$ printf "{\"a\":\"b\",\"c\":\"d\"}\r\n" | xxd
00000000: 7b22 6122 3a22 6222 2c22 6322 3a22 6422  {"a":"b","c":"d"
00000010: 7d0d 0a                                  }..
$

...which implies that it's docker exec.

Both my host and the container are running Linux. docker info reports Server Version: 18.03.1-ce. docker -v reports Docker version 18.03.1-ce, build 9ee9f40.

What did docker exec do to my terminal (temporarily), and how do I fix it?

  • What OS are you running these commands on? – Attie May 29 '18 at 13:06
  • Linux; updated the question. – Roger Lipscombe May 29 '18 at 13:07
  • Thanks .. and a follow up: what is the docker image that you're running? Can you reproduce with a basic ubuntu:latest or similar? (I can't) – Attie May 29 '18 at 13:07
  • 1
    Found it: It's due to the -t; if someone can write an answer explaining why, that'd be sufficient. – Roger Lipscombe May 29 '18 at 13:10
2

Very interesting...

docker run -d ubuntu:latest sleep infinity
$ docker exec -i sharp_einstein sh -c 'echo {\"a\":\"b\",\"c\":\"d\"}' | jq
{
  "a": "b",
  "c": "d"
}
$ docker exec -it sharp_einstein sh -c 'echo {\"a\":\"b\",\"c\":\"d\"}' | jq
{
  "a": "b",
  "c": "d"
}
$ docker exec -i -t sharp_einstein sh -c 'echo {\"a\":\"b\",\"c\":\"d\"}' | jq
{
   "a": "b",
              "c": "d"
                      }

Separating the -i and -t allow the issue to appear... (possbly a bug?)


Looking at the man page for docker exec:

  -t, --tty=true|false
      Allocate a pseudo-TTY. The default is false.

Pseudo-TTYs (PTYs) allow the system to interact with you (a human)... The result is that docker allocates a new PTY for the duration of the exec (inside the container), and things like how to produce a newline, the terminal height and width come into play - along with a load of other things.

I can't fully explain it, but but this isn't what you want from something that is feeding into a pipeline.


Compare also:

$ docker exec -i sharp_einstein sh -c 'stty'
stty: 'standard input': Inappropriate ioctl for device
$ docker exec -i -t sharp_einstein sh -c 'stty'
speed 38400 baud; line = 0;
-brkint -imaxbel
$ stty
speed 38400 baud; line = 0;
-brkint -imaxbel

stty is able to control the PTY's handling of carriage returns and newlines, though these options didn't have any effect here for me. From the man page:

  * [-]ocrnl
          translate carriage return to newline

  * [-]onlcr
          translate newline to carriage return-newline

There are similar observations with sst -t, for example: stair-stepping when using a pty (through ssh) and piping to more

  • 1
    Interesting. There's no difference between -i -t and -it on my installation; they're both broken. – Roger Lipscombe May 29 '18 at 13:58
  • Also: why is the behaviour of the pseudo-tty affecting the behaviour of my terminal's tty? jq runs in my tty. – Roger Lipscombe May 29 '18 at 13:59
  • Unfortunately that falls into my "I can't fully explain it"... I can at best offer a hypothesis that docker is playing with your local PTY to make things more "transparent"... I might dig into some sources later... – Attie May 29 '18 at 14:22
0

Did you try with "-n" on "echo -n" like :

$ docker exec -it upbeat_mendeleev sh -c 'echo {\"a\":\"b\",\"c\":\"d\"}' | jq .
{
   "a": "b",
              "c": "d"
                      }


$ docker exec -it upbeat_mendeleev sh -c 'echo -n {\"a\":\"b\",\"c\":\"d\"}' | jq .
{
  "a": "b",
  "c": "d"
}
  • 1
    I'm not actually using echo; this is just an example for the repro. So I can't add -n. – Roger Lipscombe May 29 '18 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.