SSH reverse tunnel netstat ports

Question

I've a question about SSH reverse tunnel.
I've an Ubuntu server with sshd installed.
I open an SSH reverse tunnel from a remote machine. On that machine the connection is restarted every time the tunnel is interrupted.

Whenever a tunnel is opened, on the SSH server I can see two connections when I am using netstat. One connection is listening to the internal server port. The other is listening on an external port. The latter one is the SSH tunnel.

As example:

given 203.0.113.10: my SSH server and 10.34.23.12 my remote PC, 5000 the port on my remote PC i would like to access to

ssh -R 1122:localhost:5000 [email protected]

Now server side I've two listening connections, which looks like this

tcp6    0    :::1122              :::*                 LISTEN
tcp6    0    0 203.0.113.10:22    10.34.23.12:62734    ESTABLISHED

I use/run the nc command from the server, to check for the tunnel to work

nc -z -v -w5 127.0.0.1 1122

Sometimes it happens that the external connection dies, so my netstat output will be

`tcp6    0    0 203.0.113.10:22    10.34.23.12:62734    ESTABLISHED`

Is there a way to check which tunnel does not have an external port listening?

I mean, is there a way to check when my 1122 port dies?
My solution would be kill the sshd process binded to the tunnel without any external port (10.34.23.12:62734). The problem is that I've an other SSH tunnel on this machine I would not like to kill, so killall sshd would not be an option.

Thank you!

Edit 1:

Possible solution: The netstat -lptun command (and the netstat -pant suggested by Paul) does the binding I need to try to solve this. Now I'm testing this solution in production.

#!/bin/sh
for pid in `lsof -i -n | egrep '\<ssh\>' | awk '{print $2}'`; do
  foundpid="$(netstat -lptun | grep :::11 | grep $pid)"
  if [ -z "$foundpid" ]
  then
    echo PID does not have any external port, killing the pid $pid
    kill $pid
  fi
done

Answer 1

What do you mean by "reverse" tunnel?

The -R parameter refers to a remotely-initiated tunnel, rather than the -L parameter which refers to a locally-initiated tunnel.

So, from the SSH client's perspective, the -R parameter will listen for traffic on the remote end (which means the SSH server). When the computer running the SSH server receives traffic on the specified port (1122), that computer will send that information through the SSH software. Specifically, it will send that information through the SSH tunnel. As the information gets sent through the SSH tunnel, it gets encrypted like all SSH traffic. Also, a little note is added to that information, which is (paraphrased here to say) "send the traffic to localhost port 5000".

When the traffic on the other end of the SSH tunnel (in this example, we are now talking about the computer running the SSH client) receives the traffic, it sees the note about where the traffic will go. So then that computer reaches out the localhost port 5000. That is the point of time that the host name of "localhost" gets resolved. So, in this example, the "localhost" will refer to the SSH client. (In contrast, if you used -L instead, using "localhost" like that would still be interpreted by the end receiving traffic over the tunnel, so the phrase "localhost" would be interpreted by the SSH server.)

Now, your best bet is to figure out why the tunnel keeps collapsing. I have usually used -L (local) tunnels, but I've just ignored them for many days before (sometimes) deciding to use one, and it works just fine. I would presume that the tunnel could be collapsed by either end. Check both sides for logs. You may want to check OpenSSH's documentation to see if there is some sort of timeout that you're dealing with.

If you want to go the unpreferable route of working around the problem (instead of fixing it), by killing the sshd and restarting, you could do that. As noted, you don't want to use killall sshd because there may be another sshd connection. You could try pkill to eliminate any program connecting to 203.0.113.10 (or is it 10.34.23.12?) or the port number (1122) but, again, that could lead to some false positives. You do have a safer way to work around the problem; it even involves your same proposed approach of restarting the server.

You could have sshd use a custom configuration file (-f configuration file), or you can specify configuration information on the command line (using -o). e.g.:

sshd -o PidFile /var/run/sshd-remPC-tunnel.pid

(Also, you would include your other desired options on that command line.)

This puts the process ID number in that file when you start the sshd. Then, you can:

kill $( cat /var/run/sshd-remPC-tunnel.pid )

(Note: You might need to muck with permissions. I am very intentionally not caring about that, in order to keep my answer simple. Make sure sshd can create the file, and you can successfully cat it when you desire to kill it. Run tests, and make whatever changes you need to.)

Note: You probably do NOT want to just auto-restart the tunnel every 5 minutes (to automate fixing the problem). Because, if you do have a successful connection, that would disrupt the successful connection. So I imagine this would be something intended to be done manually.