The answer to your question is, as correctly pointed out by Kromey, it depends. It depends exactly on your router, and on the kind of VPN you wish to access.
However, I can suggest a solution that will certainly work, if you grant me the permission to try and fullfill your aim at large, instead of focusing on the answer to your exact question.
All you need to do is to buy a small pc (how small, in a second), use it as a gateway to the VPN, and then use this small pc, instead of your router, as the router on your network.
Now for the details:
the pc can be really small. To this aim, I use second-hand pcs bought on e-bay for 200$. But you can do even better, if you use a Raspberry Pi, which you can obtain for less than 40$.
It is easiest to do it in Linux (Raspian, for the Raspberry, is the simplest choice). In Linux, after setting up the VPN that connects this new pc to the VPN server, all you need to do is to use these two commands as root:
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -j MASQUERADE
In order to instruct the devices in your LAN to use this new pc as the new router, all you have to do is to change the default router IP address (details on how to do this depend on device, but basically it can be done for anything, pcs, printers, TVs, Blu-Ray players, you name it) to that of this new pc.
One of the advantages of this is that you can simultaneously have devices on the VPN or off it, and, for intance, you may use on your pc the VPN when you want to get around censorship, but use the normal router (which will be a bit faster) when you do not need to do that.