There are two things that can be called "forwarding tables": a L2 switch's MAC→port mapping, and a L3 router's FIB. Based on the rest of your question, I'll assume you're actually asking about the latter. (If you really meant the former, see @harrymc's answer instead.)
1) Since the routing tables contain the shortest distance to the next hop router, why is there a need for a forwarding table to tell us which router interface to forward to ? How to reconcile the two concepts together ? or am i missing something
Routing protocols are usually described as having two databases:
The 'RIB' (routing information base) contains input used by the routing protocol to make decisions – e.g. it stores all candidate routes, distance vectors, link states, whatever. In BGP it would contain all route advertisements received from other BGP peers. RIB entries might not yet have direct associations between a route and its interface.
The 'FIB' (forwarding information base) is the result of the routing algorithm's decisions. It has only the specific routes that the algorithm has chosen, along with the resolved gateway addresses and devices. Basically it's the "routing table" that you know from Linux/BSD.
The RIB is kept internally by the routing protocol daemons, while the FIB is exported into the operating system's routing table (or into hardware-accelerated routing/forwarding chips). In other words, what you see by running ip route
on Linux is actually the FIB.
The separation exists as an optimization, because determining the best path is not fast; it might take quite a lot of CPU and memory (and it's not as simple as always using the "shortest" path).
However, instead of trying to calculate the shortest route for on every packet, it only needs to be done when the RIB contents actually change – so the routes are precomputed and put into the FIB, which only needs to do the single final step of looking up the longest matching prefix for each packet.
2) Routing algorithms need to determine the shortest route for a packet from a client to reach a server for example, does that mean that everytime a TCP connection is made to different servers, the shortest route is determined by looking at routers routing table.
No. As mentioned above, the "best" routes are precomputed whenever there's a change to the network topology. They are then loaded into the FIB so that individual connections only need to do a fast "longest prefix match" lookup.
The Internet is a packet-switched network, which means the route lookup isn't just done when a TCP connection is made – it's also done for every single packet later on. (And yes, this means even existing TCP connections can suddenly begin taking a different path than before.) This is in contrast with circuit-switched networks.
How exactly does that happen ? I am only aware that routers store the distance vectors of all routers in the network which is the shortest distance to all the routers in that network. How do they use that information to determine the shortest path for the packet ?
For the mechanisms determining the shortest path in general, Dijkstra's algorithm is a good start. It's the basis of OSPF and several other protocols.
That said, each routing protocol does it differently. BGP has its own rules, OSPF and IS-IS have their own, RIP and Babel have their own. (Some protocols are distance-vector, some are link-state, some are both at once. For example, OSPF is link-state within an area, but distance-vector between areas.) A router will even have global rules telling it to prefer OSPF routes over RIP, RIP over BGP, and so on.
Note that not all routers even run an automatic routing protocol at all. It's entirely possible for all routes in the FIB to be entered manually by the administrator.