Host A wants to ssh into Host B which is behind a router. We don't have access to the router of Host B, so we can't do port forwading.
Which other alternatives are there for allowing Host A to ssh into Host B?
One alternative that I've used for this purpose is to set up a VPN with an access point somewhere reachable by both Host A and Host B, i.e. somewhere on the internet. I suggest looking into OpenVPN and its examples. This way you can virtualize a network through which you can access any other hosts connected as if they were on the same network.
OpenVPN is pretty powerful, so providing a step-by-step guide would be beyond the scope of this answer.
There is one more alternative. Give host B regular IP address. Not NATTing, no RFC1918 IPs. Just a normal IP. Then (firewall rules allowing) everyone can ssh directly to host B.