If you’re using Mac OS X 10.7 Lion or later, Terminal automatically restores windows and terminal sessions that were open when you Quit (see “Resume” on the What’s New page for Lion). If you created the SSH session with the Shell > New Remote Connection… command, Terminal will remember and create that connection again when restoring that terminal.
If this is what you’re encountering, just close that terminal window/tab and create a new one.
If new terminals also exhibit this behavior, you likely have a custom command in your preferences. Look at Preferences > Settings > [profile] > Shell > Startup > Run command and see if there’s a command in the text field. If so, remove it.
Also check Preferences > Startup > Shells open with: to see if there’s a custom command being used for the shell.