3

Edit: meanwhile, I filed a bug with openSSH (https://bugzilla.mindrot.org/show_bug.cgi?id=2577). Should there be any outcome, I will post it here, too. New edit: this limitation will be solved in openSSH 7.3 (see here)

I use an SSH jump host/proxy at a location to connect to the remaining machines at this location. Everything works fine, except port forwardings. My .ssh/config (with explanations in-file):

## a list of hostnames otherwise not known to my laptop,
## i. e. not in /etc/hosts
## at home, in my local network, "ssh server" will connect me to the server
Host server
Host archpi
Host banana
Host uncle
Host router
Host dlna
Host osmc
Host vostro
Host xps
Host thor
Host hp

## the jump host
Host jump
    HostName my.dynamicdns.com
    Port 2222
    IdentityFile ~/.ssh/my-jump/remote_ed25519
    ControlMaster auto
    ControlPath ~/.ssh-cm-socket/%r@%h:%p
    ## the forwarding rule which does not work
    ## also, LocalForward x y:z does not work
    DynamicForward 1080

## abroad, this stanza together with the * stanza will connect me
## to the hosts above via the jump host.
## Yes, private ssh keys sit in subdirectories. %h will resolve as
## "server/remote" here, resulting in ~/.ssh/my-server/remote_ed25519;
## $(dirname %h) will simply be "server"
Host */remote
    IdentityFile ~/.ssh/my-%h_ed25519
    ProxyCommand ssh -W $(dirname %h):22 jump
    Ciphers arcfour

## at home, in my local network, "ssh server" will connect me to the server directly.
## abroad, via public wifi, and together with the */remote stanza, 
## "ssh server/remote" will connect me to the server via the jump host.
## This identity file seems to be somewhere else, but isn't. In this stanza,
## %h will be resolved as "server" to ~/.ssh/my-server/remote_ed25519
Host *
    Compression yes
    CompressionLevel 1
    ForwardAgent yes
    ServerAliveInterval 60
    User my
    IdentityFile ~/.ssh/my-%h/remote_ed25519

Again, everything works fine. I can connect to "server" in my local LAN and to "server/remote" from abroad. I can connect to "jump" via SSH, and the port forwarding will be established. Only not when connecting to another host via jump. Why? And, more important: how can I have the port forwarded in this ssh config?

Edit: Whenever I am directly connecting to "jump", ports will be forwarded. Whenever I am connecting to another host via "jump", no forwarding. From the logfiles, the only difference seems to be

Session via "jump", output of ssh -vvv router/remote

    [...]
    Authenticated to router/remote (via proxy).
debug2: fd 7 setting O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: proc
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: receive packet: type 91
debug2: callback start
debug1: Requesting authentication agent forwarding.
debug2: channel 0: request auth-agent-req@openssh.com confirm 0
debug3: send packet: type 98
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug3: Ignored env XDG_VTNR
debug3: Ignored env SSH_AGENT_PID
debug3: Ignored env XDG_SESSION_ID
debug3: Ignored env HOSTNAME
debug3: Ignored env QUBES_ENV_SOURCED
debug3: Ignored env TERM
debug3: Ignored env SHELL
debug3: Ignored env VTE_VERSION
debug3: Ignored env HISTSIZE
debug3: Ignored env QUBES_KEYMAP
debug3: Ignored env WINDOWID
debug3: Ignored env QUBES_USER_KEYMAP
debug3: Ignored env USER
debug3: Ignored env LS_COLORS
debug3: Ignored env SSH_AUTH_SOCK
debug3: Ignored env PATH
debug3: Ignored env MAIL
debug3: Ignored env QT_X11_NO_MITSHM
debug3: Ignored env UPDTYPE
debug3: Ignored env PWD
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env MODULEPATH
debug3: Ignored env LOADEDMODULES
debug3: Ignored env QREXEC_AGENT_PID
debug3: Ignored env SSH_ASKPASS
debug3: Ignored env HISTCONTROL
debug3: Ignored env XDG_SEAT
debug3: Ignored env SHLVL
debug3: Ignored env HOME
debug3: Ignored env GNOME_DESKTOP_SESSION_ID
debug3: Ignored env LOGNAME
debug3: Ignored env DBUS_SESSION_BUS_ADDRESS
debug3: Ignored env MODULESHOME
debug3: Ignored env VMTYPE
debug3: Ignored env LESSOPEN
debug3: Ignored env WINDOWPATH
debug3: Ignored env XDG_RUNTIME_DIR
debug3: Ignored env DISPLAY
debug3: Ignored env BASH_FUNC_module()
debug3: Ignored env BASH_FUNC_scl()
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
debug3: receive packet: type 96
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug2: channel 0: rcvd eow
debug2: channel 0: close_read
debug2: channel 0: input open -> closed
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug3: channel 0: will not send data after close
debug2: channel 0: obuf empty
debug2: channel 0: close_write
debug2: channel 0: output drain -> closed
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 r0 i3/0 o3/0 fd -1/-1 cc -1)

debug3: send packet: type 1
debug1: fd 2 clearing O_NONBLOCK

[user@personal ~]$ ss -lant4 | grep 1080
[user@personal ~]$ 

Session via "jump", output of ProxyCommand ssh -W $(dirname %h):22 jump 2>log.txt:

[...]
Authenticated to my.dynamicdns.com ([12.34.56.78]:2222).
debug3: ssh_init_stdio_forwarding: router:22
debug1: channel_connect_stdio_fwd router:22
debug1: channel 0: new [stdio-forward]
debug2: fd 4 setting O_NONBLOCK
debug2: fd 5 setting O_NONBLOCK
debug1: getpeername failed: Bad file descriptor
debug3: send packet: type 90
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x10
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: receive packet: type 91
debug2: callback start
debug2: callback done
debug2: channel 0: open confirm rwindow 2097152 rmax 32768
debug3: send packet: type 1
debug1: channel 0: free: direct-tcpip: listening port 0 for router port 22, connect from 127.0.0.1 port 65535 to UNKNOWN port 65536, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 direct-tcpip: listening port 0 for router port 22, connect from 127.0.0.1 port 65535 to UNKNOWN port 65536 (t4 r0 i0/0 o0/0 fd 4/5 cc -1)

debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
debug3: fd 2 is not O_NONBLOCK

Connection to "jump", output of ssh -vvv jump

[...]
Authenticated to my.dynamicdns.com ([12.34.56.78]:2222).
debug1: Local connections to localhost:1080 forwarded to remote address socks:0
debug1: Local forwarding listening on ::1 port 1080.
debug1: channel 0: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 1080.
debug1: channel 1: new [port listener]
debug1: channel 2: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: Requesting authentication agent forwarding.


[user@personal ~]$ ss -lant4 | grep 1080
LISTEN     0      128    127.0.0.1:1080                     *:*                  

All the forwarding is here in the log for the direct connection. The connection via jump only states debug1: getpeername failed: Bad file descriptor. Why is that and how can I have the port forwarded?

Edit 2: server logs. I attached the server logs (level 3) of host "jump" here. They are not too talkative regarding port forwarding. Presumably, because the port forwarding takes place on the client.

  • Forwarding over more jumps needs more care. Investigating verbose log will usually help. – Jakuje May 29 '16 at 19:30
  • Thanks for your comment. Could you please elaborate what you mean by "needs more care". I will update my question with additional information and the verbose logs I already investigated. – stueja May 30 '16 at 18:02
  • Thanks for the logs. I will check that in next days. It might be feature, or a bug, since I see some failures there. Manual page is not verbose enough in this case ... – Jakuje May 30 '16 at 21:07
0

Ok. I had the feeling that it needs some extra care, but just tested with local and dynamic port forwarding (on command-line, to the target host) and it works just fine for me.

Forwarding to jumbox only might need special care, because this connection goes only through, or you might not have enough permissions. Verifying the logs with added LogLevel DEBUG3 to the Host jump block might give you more ideas.

  • Unfortunately, server logs on the jump host, even in level 3, are very silent about port forwarding. I guess it is because it happens on the client. I will add them to my original question. – stueja May 30 '16 at 20:47
  • This is not logged on the server. It is mentioned only in the client logs. – Jakuje May 30 '16 at 21:01
  • Even log level 3 on the client does not provide more information to me than bad file descriptor , unfortunately. – stueja May 31 '16 at 4:52
  • I see you filled a bug already. I am guess that stdio-forward makes the session forwarding IO and after that you can not issue any other forwardings. But I believe it should be possible to do this earlier before. But lets see what upstream developers will come with. – Jakuje May 31 '16 at 7:32
  • Yes, this morning. Let's wait for them, I don't have any other explanation for this strange behaviour. – stueja May 31 '16 at 10:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.