You can check your firewall settings on your ubuntu system with
sudo iptables -L
By default ubuntu is pretty open so you don't have to manually open ports in the ubuntu firewall for the vnc server to work on the local network.
Windows XP has ports and programs that you can allow in the firewall. You can find the firewall settings under control panel. You should not have to open any ports in windows in order to connect to the linux vnc server.
Routers have a web interface that you can connect to. You can type ipconfig in the windows command prompt and take note of your default gateway. It might be something like 192.168.0.1 or 192.168.1.1. You can then type that address directly into a web browser address bar to get to the router login and settings. Routers are by default not set up to forward any ports... so you probably don't have to worry about people on the Internet having direct access to your ubuntu vnc server.
The default port for vnc is 5900. You can try connecting a vnc client on ubuntu to its self to see if the vnc server is running properly. This can also create a cool effect where your vnc client has the screen of your screen of your screen... inside it.