While creating an incoming TCP/IP or UDP connection we need for a firewall port to be opened or to get privilege for the specified port.
Why not for outgoing TCP/IP connections?
Actually outgoing TCP connections do need firewall to be opened as well. It is possible to set firewall in a way that it will block incoming and outgoing connection as well. Take a look on the above article about firewall.
There is such thing as default rule for firewall. Most secure configuration will have default rule to block everything, except few ports which are needed to be open. For outgoing connections default rule can be "allow everything" unless there is special security setup or you do not trust programs which run on the computer (or 'inside' the firewall) - in this case default rule would be to drop everything and allow connections only to allowed ports or addresses. And you will have to open firewall for outgoing connections as well.
So all that is just matter of firewall configuration and default firewall rules.
Some old-fashioned or simplistic firewalls from the 1990's may have only been able to apply rules to individual packets in isolation, but all modern firewalls are what we call Stateful Packet Inspection (SPI) firewalls. SPI firewalls remember the state of a traffic flow (such as remembering that they saw an outgoing TCP Synch that initiated a TCP session) and apply that state information to future allow/deny decisions (such as allowing other packets in that TCP session to flow in either direction, since it was initiated "from the inside").
Many home gateway router products don't even have fully-featured firewalls, but rely on the fact that NAT (specifically "NAPT", a.k.a. "PAT") acts as a kind of "poor man's SPI firewall". NAT gateways need port mappings because otherwise, when an incoming, initiated from the outside, connection attempt hits the NAT gateway's external IP address, it won't know what private side client to forward it to, since it doesn't have any state recorded. So it has to just drop it. Port mappings allow the NAT to know which client to forward certain kinds of incoming requests to.
As for getting privileges to put a listener on a given port on a host, some OSes restrict the ability to put listeners on the "Well Known" ports below 1024, or even on the "Registered" ports below 49152, but most OSes allow any normal user to put a listener on the "Ephemeral" ports above 49152. If you were frustrated that you didn't have privs to start up your own web server process on port 80 (in the "well known" port range) or 8080 (registered), try starting it up on 60080 (ephemeral).
Outgoing TCP and UDP connections are auto-assigned a port from the ephemeral range (some OSes may even use parts of the "registered" range for ephemeral ports as well).