I know that using SSL doesn't stop third parties from seeing what you're connecting to. Does it also not protect against third parties seeing what port you're connecting on?
Yes, they can.
To test this, I opened wireshark and browsed an SSL encrypted website.
Encapsulation is as follows:
Another proof of concept would be this: If the port numbers were encrypted, a layer 3 device such as a NATing router would have no idea where to forward the traffic to.