Comodo Personal Firewall is a free replacement for the Windows Firewall that will query whether you want an application to be able to make outbound connections the first time that application is used. You will also be able to allow programs to open incoming ports so you should be fine there too.
It's pretty comprehensive, you can (I believe) add the most commonly used applications during setup, so Firefox, IE and most email programs are not blocked by default.
The configuration is relatively straightforward and the baloon popups that appear when an application tries to access the network were quite informative when I used it last.
Unless you want Antivirus as well try to make sure that you only download the Firewall installer, their site makes it a bit too easy to get both by mistake.