I'm trying to make a tunneling application to use SSH over a HTTPS connection, with a custom client and server at each end, where the custom client opens a unix socket that can communicate with the SSH server on the other end. The problem is I'm not sure how to get ssh to open a connection over the unix socket rather than specifiying a host and port. Is this possible or are there any command line arguments that can be set to do so?
Look at the ProxyCommand ProxyCommand option in the docs for ssh. It allows you to use pretty much anything that can redirect to stdio.
In your ~/.ssh/config file you want to add an entry like so
Host theDestination Hostname www.theDestination.com ProxyCommand ssh me@proxyBox.com nc %h %p User me
This will automatically tunnel your connection through proxybox.com to theDestination. One thing to note is that you will need to make sure you have nc installed on the proxyBox.