Setup routing and iptables for new VPN connection to redirect **only** ports 80 and 443

Question

I have a new VPN connection (using openvpn) to allow me to route around some ISP restrictions. Whilst it is working fine, it is taking all the traffic over the vpn. This is causing me issues for downloading (my internet connection is a lot faster than the vpn allows), and for remote access. I run an ssh server, and have a daemon running that allows me to schdule downloads via my phone.

I have my existing ethernet connection on eth0, and the new VPN connection on tun0.

I believe I need to setup the default route to use my existing eth0 connection on the 192.168.0.0/24 network, and set the default gateway to 192.168.0.1 (my knowledge is shaky as I haven't done this for a number of years). If that is correct, then I'm not exactly sure how to do it!. My current routing table is:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface    MSS   Window irtt
0.0.0.0         10.51.0.169     0.0.0.0         UG    0      0        0 tun0     0     0      0
10.51.0.1       10.51.0.169     255.255.255.255 UGH   0      0        0 tun0     0     0      0
10.51.0.169     0.0.0.0         255.255.255.255 UH    0      0        0 tun0     0     0      0
85.25.147.49    192.168.0.1     255.255.255.255 UGH   0      0        0 eth0     0     0      0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eth0     0     0      0
192.168.0.0     0.0.0.0         255.255.255.0   U     1      0        0 eth0     0     0      0

After fixing the routing, I believe I need to use iptables to configure prerouting or masquerading to force everything for destination port 80 or 443 over tun0. Again, I'm not exactly sure how to do this!

Everything I've found on the internet is trying to do something far more complicated, and trying to sort the wood from the trees is proving difficult.

Any help would be much appreciated.

UPDATE

So far, from the various sources, I've cobbled together the following:

#!/bin/sh

DEV1=eth0
IP1=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 192.`
GW1=192.168.0.1
TABLE1=internet
TABLE2=vpn
DEV2=tun0
IP2=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 10.`
GW2=`route -n | grep 'UG[ \t]' | awk '{print $2}'`

ip route flush table $TABLE1
ip route flush table $TABLE2
ip route show table main | grep -Ev ^default | while read ROUTE ; do
    ip route add table $TABLE1 $ROUTE
    ip route add table $TABLE2 $ROUTE
done
ip route add table $TABLE1 $GW1 dev $DEV1 src $IP1
ip route add table $TABLE2 $GW2 dev $DEV2 src $IP2
ip route add table $TABLE1 default via $GW1
ip route add table $TABLE2 default via $GW2

echo "1" > /proc/sys/net/ipv4/ip_forward
echo "1" > /proc/sys/net/ipv4/ip_dynaddr

ip rule add from $IP1 lookup $TABLE1
ip rule add from $IP2 lookup $TABLE2
ip rule add fwmark 1 lookup $TABLE1
ip rule add fwmark 2 lookup $TABLE2

iptables -t nat -A POSTROUTING -o $DEV1 -j SNAT --to-source $IP1
iptables -t nat -A POSTROUTING -o $DEV2 -j SNAT --to-source $IP2

iptables -t nat -A PREROUTING           -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables        -A OUTPUT               -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables -t nat -A PREROUTING -i $DEV1  -m state --state NEW                          -j CONNMARK --set-mark 1
iptables -t nat -A PREROUTING -i $DEV2  -m state --state NEW                          -j CONNMARK --set-mark 2
iptables -t nat -A PREROUTING           -m connmark --mark 1                          -j MARK --set-mark 1
iptables -t nat -A PREROUTING           -m connmark --mark 2                          -j MARK --set-mark 2
iptables -t nat -A PREROUTING           -m state --state NEW -m connmark ! --mark 0   -j CONNMARK --save-mark

iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport  80 -j CONNMARK --set-mark 2
iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport 443 -j CONNMARK --set-mark 2

route del default
route add default gw 192.168.0.1 eth0

Now this seems to be working. Except it isn't!

Connections to the blocked websites are going through, connections not on ports 80 and 443 are using the non-VPN connection.

However port 80 and 443 connections that aren't to the blocked websites are using the non-VPN connection too!

As the general goal has been reached, I'm relatively happy, but it would be nice to know why it isn't working exactly right.

Any ideas?

For reference, I now have 3 routing tables, main, internet, and vpn. The listing of them is as follows...

Main:

default via 192.168.0.1 dev eth0 
10.38.0.1 via 10.38.0.205 dev tun0 
10.38.0.205 dev tun0  proto kernel  scope link  src 10.38.0.206 
85.removed via 192.168.0.1 dev eth0 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.73  metric 1 

Internet:

default via 192.168.0.1 dev eth0 
10.38.0.1 via 10.38.0.205 dev tun0 
10.38.0.205 dev tun0  proto kernel  scope link  src 10.38.0.206 
85.removed via 192.168.0.1 dev eth0 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.73  metric 1 
192.168.0.1 dev eth0  scope link  src 192.168.0.73

VPN:

default via 10.38.0.205 dev tun0 
10.38.0.1 via 10.38.0.205 dev tun0 
10.38.0.205 dev tun0  proto kernel  scope link  src 10.38.0.206 
85.removed via 192.168.0.1 dev eth0 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.73  metric 1

Answer 1

So, most of this is above, but the whole solution was as follows:

Edit /etc/iproute2/rt_tables and add 2 lines at the bottom:

101 internet
102 vpn

You can give these tables other names that make more sense, just be consistent.

Then you need to create a script (I called it rt_setup) in /etc/init.d

#!/bin/sh

DEV1=eth0
IP1=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 192.`
GW1=192.168.0.1
TABLE1=internet
TABLE2=vpn
DEV2=tun0
IP2=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 10.`
GW2=`route -n | grep 'UG[ \t]' | awk '{print $2}'`

ip route flush table $TABLE1
ip route flush table $TABLE2
ip route show table main | grep -Ev ^default | while read ROUTE ; do
    ip route add table $TABLE1 $ROUTE
    ip route add table $TABLE2 $ROUTE
done
ip route add table $TABLE1 $GW1 dev $DEV1 src $IP1
ip route add table $TABLE2 $GW2 dev $DEV2 src $IP2
ip route add table $TABLE1 default via $GW1
ip route add table $TABLE2 default via $GW2

echo "1" > /proc/sys/net/ipv4/ip_forward
echo "1" > /proc/sys/net/ipv4/ip_dynaddr

ip rule add from $IP1 lookup $TABLE1
ip rule add from $IP2 lookup $TABLE2
ip rule add fwmark 1 lookup $TABLE1
ip rule add fwmark 2 lookup $TABLE2

iptables -t nat -A POSTROUTING -o $DEV1 -j SNAT --to-source $IP1
iptables -t nat -A POSTROUTING -o $DEV2 -j SNAT --to-source $IP2

iptables -t nat -A PREROUTING           -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables        -A OUTPUT               -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables -t nat -A PREROUTING -i $DEV1  -m state --state NEW                          -j CONNMARK --set-mark 1
iptables -t nat -A PREROUTING -i $DEV2  -m state --state NEW                          -j CONNMARK --set-mark 2
iptables -t nat -A PREROUTING           -m connmark --mark 1                          -j MARK --set-mark 1
iptables -t nat -A PREROUTING           -m connmark --mark 2                          -j MARK --set-mark 2
iptables -t nat -A PREROUTING           -m state --state NEW -m connmark ! --mark 0   -j CONNMARK --save-mark

iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport  80 -j CONNMARK --set-mark 2
iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport 443 -j CONNMARK --set-mark 2

route del default
route add default gw 192.168.0.1 eth0

Then, obviously, link it from /etc/rc2.d (I use ubuntu, runlevel may differ for you). Make sure you give it an S number higher than the openvpn link!

The script does a number of things. The top portion sets up the variables, with some perl and awk statements used to pick up the dynamic IPs and gateway addresses.The second section cleans up the tables that you setup in ipruote2, and copies the current routing table to them. It then creates two new routes, and two default gateways for them, with the VPN one going over the VPN, and the internet one going over my local network.

I'm not convinced the next 2 lines are necessary, but they enable ip forwarding for use in iptables.

Next the script creates some rules on where to look for traffic originating from the relevant IP address, and where to look if the traffic is specifically marked.

The POSTROUTING and PREROUTING ensures that traffic originating from an address gets the reply!

The final iptables PREROUTING is the part the tags the traffic, and ensures that anything going to ports 80 or 443 is marked to use Table 2 (VPN)

The final two lines remove the VPN gateway from the default routing table, and add back my local network gateway.

As it stands, the process works brilliantly. The VPN is started as the machine comes up, and this script is run a few seconds later (I may add a sleep statement just to ensure the VPN is fully initialised before running this script). My remote access connection (ssh etc.) work great. My connections out that don't go to ports 80 or 443 are using my local connection, but all web traffic is going over the VPN, and bypassing the controls put in place by my ISP!

As I said in my comment under my question, I wouldn't have even started looking at this route without the suggestion of @anttir. Off the back of that suggestion, the sites http://blog.khax.net/2009/11/28/multi-gateway-routing-with-iptables-and-iproute2/ and http://linux-ip.net/html/adv-multi-internet.html have been very useful (even if the code is not 100% complete!)

Answer 2

routing per protocol is a tad complicated. Usually routing table is used to check the gateway according to destination IP and use either the openvpn or the 192.168.0.1 default gateway.

It would be easier to set up e.g. Squid http proxy on the other end of the VPN and set browser to use the proxy.

You wouldn't use the iptables as it would change the destination IP of the HTTP connection and it would not work.

You could create a new routing table (/etc/iproute2/rt_tables) with default route set to the VPN endpoint, use iptables fwmark ( -j MARK ) to mark all the HTTP packets and then use ip rule to create a custom rule for the marked packages to use the new routing table.