What is the overhead of SSH compared to telnet?

Question

I am curious: how big is the overhead SSH introduces compared to telnet?

Obviously there are different parts here:

1. Initial connection time overhead due to additional round trips for encryption
2. Is there a bandwidth overhead?
   • Is there an overhead in each packet transmitted
   • or do we only have some padding at the end?
3. CPU overhead on the client side
4. CPU overhead on the server side
   • How large is it?
   • How about newer CPUs which have AES modules?
   • And what about servers with dozens or hundreds of SSH connections?
   • is the per-connection CPU overhead of clients and servers symmetric?
5. Is there a memory overhead?
6. Did I forgot something?

This is more of a theoretical question. I am fully aware that it makes practically no difference in typical scenarios where a server maintains only a few ssh connections.

Answer 1

I'll give you quick answer since a lengthy explanation would easily fill a 90 min lecture or tow. These are little more than estimations since it is very much dependent on system load, the kind of connections you throw at the server and of course the hardware that is used.

1. Yes and no. There is a rekeying after 1 GB of data or 1 hour of connection time. Can also depend on whether keys are used for authentication. Diffie-Hellman is generally more expensive than RSA key exchange, but since it only happens once(ish) not a big deal.
2. Yes there is overhead: you have to add random padding of at least 4 byte (SSH2) And each data package gets an HMAC of some sort. Depending on what is used. I don't know how much exactly but it is less at max 33 byte (Full sha2 lengt) Max payload is 35000 - 4 byte padding. You get about 37/34996 = 0.001 % overhead or less.
3. CPU overhead is minimal. That was one of the goals for AES standard.
4. Same here. It is roughly symmetrical since the crypto operations are symmetrical (similar to multiplication and division are symmetrical), of course this requires knowing the keys. How many connections can a server handle? Depends on the connections. If you have simple terminal sessions: many. If someone blasts a 1 GBit connection at you and copies 10 GB of data via scp: using a standard intel i7 probably two or three at a time. Its likely that your NIC or storage bottle necks first.
5. Not sure. A little bit probably. I'd say not more than double the memory of a normal telnet session.

Answer 2

---

TL;DR

SSH bandwith overhead in tunneling mode is neglible

In this LAN testing scenario there were even a lot less pakets in the opposite direction, about 50 % less.

---

Details of how to get the bandwith overhead:

Setup:

On server running

iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------

On client we establish ssh connection to server ssh -L 5001:127.0.0.1:5001 [email protected] SSH Compression was not enabled.

To monitor traffic we add

sudo iptables -A INPUT -p tcp --src 192.168.178.55/32 -j ACCEPT
sudo iptables -A OUTPUT -p tcp --dst 192.168.178.55/32 -j ACCEPT

So first try direct connection:

iperf -c 192.168.178.55 -n 1000M
------------------------------------------------------------
Client connecting to 192.168.178.55, TCP port 5001
TCP window size: 83.1 KByte (default)
------------------------------------------------------------
[  3] local 192.168.178.21 port 60824 connected with 192.168.178.55 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-89.5 sec  1000 MBytes  93.8 Mbits/sec

First result

sudo iptables -L -v
Chain INPUT (policy ACCEPT 609 packets, 30850 bytes)
 pkts bytes target     prot opt in     out     source               destination
 342K   14M ACCEPT     tcp  --  any    any     192.168.178.55      anywhere

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 968 packets, 99600 bytes)
 pkts bytes target     prot opt in     out     source               destination
93424 1053M ACCEPT     tcp  --  any    any     anywhere             192.168.178.55

In between counters are flushed via iptables -Z 2nd try

sudo iptables -L -v
Chain INPUT (policy ACCEPT 387 packets, 19661 bytes)
 pkts bytes target     prot opt in     out     source               destination
 344K   14M ACCEPT     tcp  --  any    any     192.168.178.55      anywhere

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 626 packets, 64440 bytes)
 pkts bytes target     prot opt in     out     source               destination
93166 1053M ACCEPT     tcp  --  any    any     anywhere             192.168.178.55

Now ssh connection

iperf -c 127.0.0.1 -n 1000M
------------------------------------------------------------
Client connecting to 127.0.0.1, TCP port 5001
TCP window size: 2.50 MByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 38080 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-89.4 sec  1000 MBytes  93.8 Mbits/sec

First finding: Same speed eventhough we have a Raspberry PI 3B as Server and a Raspberry PI 4B as client, connected via wired LAN over 2 switches and PI 3B making a backup and thus sending a lot of data while the test (here we measured only data sending from PI4 to PI3B)

sudo iptables -L -v
Chain INPUT (policy ACCEPT 27327 packets, 1050M bytes)
 pkts bytes target     prot opt in     out     source               destination
 154K 6540K ACCEPT     tcp  --  any    any     192.168.178.55      anywhere

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 27791 packets, 1050M bytes)
 pkts bytes target     prot opt in     out     source               destination
79099 1053M ACCEPT     tcp  --  any    any     anywhere             192.168.178.55

With ssh inbetween we have even transfered less packets, total amount of data is the same. So bandwith overhead is neglible, especially compared with OpenVPN which has about 5 % with maximum packetsize https://serverfault.com/questions/249935/openvpn-performance

2nd try

sudo iptables -L -v
Chain INPUT (policy ACCEPT 27450 packets, 1050M bytes)
 pkts bytes target     prot opt in     out     source               destination
 190K 8025K ACCEPT     tcp  --  any    any     192.168.178.55      anywhere

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 27683 packets, 1050M bytes)
 pkts bytes target     prot opt in     out     source               destination
90036 1054M ACCEPT     tcp  --  any    any     anywhere             192.168.178.55

Here we have about 14 % more packets than in the first try, but still 3000 pakets less than with direct connection. We have transmitted 1MB more compared to direct connection. Thats in this case 0,09 % and since we measure at MB level it could be a lot less overhead.

I think the dramatically less pakets on input side reside due to the fact that ssh combines some TCP_ACKs and sends them combined. I was really astonished that there was less data transmitted via ssh Tunnel and searched for any mistake in this experiment, but couldn't find it.