Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a new VPN connection (using openvpn) to allow me to route around some ISP restrictions. Whilst it is working fine, it is taking all the traffic over the vpn. This is causing me issues for downloading (my internet connection is a lot faster than the vpn allows), and for remote access. I run an ssh server, and have a daemon running that allows me to schdule downloads via my phone.

I have my existing ethernet connection on eth0, and the new VPN connection on tun0.

I believe I need to setup the default route to use my existing eth0 connection on the 192.168.0.0/24 network, and set the default gateway to 192.168.0.1 (my knowledge is shaky as I haven't done this for a number of years). If that is correct, then I'm not exactly sure how to do it!. My current routing table is:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface    MSS   Window irtt
0.0.0.0         10.51.0.169     0.0.0.0         UG    0      0        0 tun0     0     0      0
10.51.0.1       10.51.0.169     255.255.255.255 UGH   0      0        0 tun0     0     0      0
10.51.0.169     0.0.0.0         255.255.255.255 UH    0      0        0 tun0     0     0      0
85.25.147.49    192.168.0.1     255.255.255.255 UGH   0      0        0 eth0     0     0      0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eth0     0     0      0
192.168.0.0     0.0.0.0         255.255.255.0   U     1      0        0 eth0     0     0      0

After fixing the routing, I believe I need to use iptables to configure prerouting or masquerading to force everything for destination port 80 or 443 over tun0. Again, I'm not exactly sure how to do this!

Everything I've found on the internet is trying to do something far more complicated, and trying to sort the wood from the trees is proving difficult.

Any help would be much appreciated.

UPDATE

So far, from the various sources, I've cobbled together the following:

#!/bin/sh

DEV1=eth0
IP1=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 192.`
GW1=192.168.0.1
TABLE1=internet
TABLE2=vpn
DEV2=tun0
IP2=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 10.`
GW2=`route -n | grep 'UG[ \t]' | awk '{print $2}'`

ip route flush table $TABLE1
ip route flush table $TABLE2
ip route show table main | grep -Ev ^default | while read ROUTE ; do
    ip route add table $TABLE1 $ROUTE
    ip route add table $TABLE2 $ROUTE
done
ip route add table $TABLE1 $GW1 dev $DEV1 src $IP1
ip route add table $TABLE2 $GW2 dev $DEV2 src $IP2
ip route add table $TABLE1 default via $GW1
ip route add table $TABLE2 default via $GW2

echo "1" > /proc/sys/net/ipv4/ip_forward
echo "1" > /proc/sys/net/ipv4/ip_dynaddr

ip rule add from $IP1 lookup $TABLE1
ip rule add from $IP2 lookup $TABLE2
ip rule add fwmark 1 lookup $TABLE1
ip rule add fwmark 2 lookup $TABLE2

iptables -t nat -A POSTROUTING -o $DEV1 -j SNAT --to-source $IP1
iptables -t nat -A POSTROUTING -o $DEV2 -j SNAT --to-source $IP2

iptables -t nat -A PREROUTING           -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables        -A OUTPUT               -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables -t nat -A PREROUTING -i $DEV1  -m state --state NEW                          -j CONNMARK --set-mark 1
iptables -t nat -A PREROUTING -i $DEV2  -m state --state NEW                          -j CONNMARK --set-mark 2
iptables -t nat -A PREROUTING           -m connmark --mark 1                          -j MARK --set-mark 1
iptables -t nat -A PREROUTING           -m connmark --mark 2                          -j MARK --set-mark 2
iptables -t nat -A PREROUTING           -m state --state NEW -m connmark ! --mark 0   -j CONNMARK --save-mark

iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport  80 -j CONNMARK --set-mark 2
iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport 443 -j CONNMARK --set-mark 2

route del default
route add default gw 192.168.0.1 eth0

Now this seems to be working. Except it isn't!

Connections to the blocked websites are going through, connections not on ports 80 and 443 are using the non-VPN connection.

However port 80 and 443 connections that aren't to the blocked websites are using the non-VPN connection too!

As the general goal has been reached, I'm relatively happy, but it would be nice to know why it isn't working exactly right.

Any ideas?

For reference, I now have 3 routing tables, main, internet, and vpn. The listing of them is as follows...

Main:

default via 192.168.0.1 dev eth0 
10.38.0.1 via 10.38.0.205 dev tun0 
10.38.0.205 dev tun0  proto kernel  scope link  src 10.38.0.206 
85.removed via 192.168.0.1 dev eth0 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.73  metric 1 

Internet:

default via 192.168.0.1 dev eth0 
10.38.0.1 via 10.38.0.205 dev tun0 
10.38.0.205 dev tun0  proto kernel  scope link  src 10.38.0.206 
85.removed via 192.168.0.1 dev eth0 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.73  metric 1 
192.168.0.1 dev eth0  scope link  src 192.168.0.73

VPN:

default via 10.38.0.205 dev tun0 
10.38.0.1 via 10.38.0.205 dev tun0 
10.38.0.205 dev tun0  proto kernel  scope link  src 10.38.0.206 
85.removed via 192.168.0.1 dev eth0 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.73  metric 1
share|improve this question
    
The above script does work as expected. I just expected to see traffic originating from the 10. address in netstat. However, all traffic originates from 192. , but port 80 and 443 is directed over VPN. I'll put the whole solution in an answer, but props to @anttir for suggesting iproute2 and fwmark. I hadn't come across those, and without them I'd still be banging my head against a brick wall! –  Steve Nov 9 '11 at 8:46

2 Answers 2

routing per protocol is a tad complicated. Usually routing table is used to check the gateway according to destination IP and use either the openvpn or the 192.168.0.1 default gateway.

It would be easier to set up e.g. Squid http proxy on the other end of the VPN and set browser to use the proxy.

You wouldn't use the iptables as it would change the destination IP of the HTTP connection and it would not work.

You could create a new routing table (/etc/iproute2/rt_tables) with default route set to the VPN endpoint, use iptables fwmark ( -j MARK ) to mark all the HTTP packets and then use ip rule to create a custom rule for the marked packages to use the new routing table.

share|improve this answer
    
Thanks for your help. I'll certainly take a look at those. Things are a little tricky in that I don't have control of the server end, just my side. Also, I can't use squid because I need to route https traffic over the connection, and squid doesn't work so well with that. –  Steve Nov 7 '11 at 21:13
    
How about routing just a few IPs over the VPN and the rest of the world outside the VPN? –  anttir Nov 8 '11 at 7:24
    
I did think of routing just a few IPs, but the list changes as the sites move around, and I need to make this easy to use for other users on the PC. I have started to look at setting up a new routing table, and marking the packets. The marking packets was the easy bit, it's the routing tables I'm not so sure of. I can set one up for VPN that looks right, and one up for everything else which looks right-ish, but I'm not sure what to do with main as that is still set to the defaults (Which is VPN). Still playing... –  Steve Nov 8 '11 at 13:46
    
linux-ip.net/html/adv-multi-internet.html ip rule show and ip rule add fwmark 4 table 4 priority 10000 –  anttir Nov 8 '11 at 15:05
    
Added an update to the original that details where I'm up to so far! AH! Just realised that I've edited your answer, not my question. Sorry! Haven't used this much, and didn't realise I could edit another person's answer! –  Steve Nov 8 '11 at 17:10
up vote 3 down vote accepted

So, most of this is above, but the whole solution was as follows:

Edit /etc/iproute2/rt_tables and add 2 lines at the bottom:

101 internet
102 vpn

You can give these tables other names that make more sense, just be consistent.

Then you need to create a script (I called it rt_setup) in /etc/init.d

#!/bin/sh

DEV1=eth0
IP1=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 192.`
GW1=192.168.0.1
TABLE1=internet
TABLE2=vpn
DEV2=tun0
IP2=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 10.`
GW2=`route -n | grep 'UG[ \t]' | awk '{print $2}'`

ip route flush table $TABLE1
ip route flush table $TABLE2
ip route show table main | grep -Ev ^default | while read ROUTE ; do
    ip route add table $TABLE1 $ROUTE
    ip route add table $TABLE2 $ROUTE
done
ip route add table $TABLE1 $GW1 dev $DEV1 src $IP1
ip route add table $TABLE2 $GW2 dev $DEV2 src $IP2
ip route add table $TABLE1 default via $GW1
ip route add table $TABLE2 default via $GW2

echo "1" > /proc/sys/net/ipv4/ip_forward
echo "1" > /proc/sys/net/ipv4/ip_dynaddr

ip rule add from $IP1 lookup $TABLE1
ip rule add from $IP2 lookup $TABLE2
ip rule add fwmark 1 lookup $TABLE1
ip rule add fwmark 2 lookup $TABLE2

iptables -t nat -A POSTROUTING -o $DEV1 -j SNAT --to-source $IP1
iptables -t nat -A POSTROUTING -o $DEV2 -j SNAT --to-source $IP2

iptables -t nat -A PREROUTING           -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables        -A OUTPUT               -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables -t nat -A PREROUTING -i $DEV1  -m state --state NEW                          -j CONNMARK --set-mark 1
iptables -t nat -A PREROUTING -i $DEV2  -m state --state NEW                          -j CONNMARK --set-mark 2
iptables -t nat -A PREROUTING           -m connmark --mark 1                          -j MARK --set-mark 1
iptables -t nat -A PREROUTING           -m connmark --mark 2                          -j MARK --set-mark 2
iptables -t nat -A PREROUTING           -m state --state NEW -m connmark ! --mark 0   -j CONNMARK --save-mark

iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport  80 -j CONNMARK --set-mark 2
iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport 443 -j CONNMARK --set-mark 2

route del default
route add default gw 192.168.0.1 eth0

Then, obviously, link it from /etc/rc2.d (I use ubuntu, runlevel may differ for you). Make sure you give it an S number higher than the openvpn link!

The script does a number of things. The top portion sets up the variables, with some perl and awk statements used to pick up the dynamic IPs and gateway addresses.The second section cleans up the tables that you setup in ipruote2, and copies the current routing table to them. It then creates two new routes, and two default gateways for them, with the VPN one going over the VPN, and the internet one going over my local network.

I'm not convinced the next 2 lines are necessary, but they enable ip forwarding for use in iptables.

Next the script creates some rules on where to look for traffic originating from the relevant IP address, and where to look if the traffic is specifically marked.

The POSTROUTING and PREROUTING ensures that traffic originating from an address gets the reply!

The final iptables PREROUTING is the part the tags the traffic, and ensures that anything going to ports 80 or 443 is marked to use Table 2 (VPN)

The final two lines remove the VPN gateway from the default routing table, and add back my local network gateway.

As it stands, the process works brilliantly. The VPN is started as the machine comes up, and this script is run a few seconds later (I may add a sleep statement just to ensure the VPN is fully initialised before running this script). My remote access connection (ssh etc.) work great. My connections out that don't go to ports 80 or 443 are using my local connection, but all web traffic is going over the VPN, and bypassing the controls put in place by my ISP!

As I said in my comment under my question, I wouldn't have even started looking at this route without the suggestion of @anttir. Off the back of that suggestion, the sites http://blog.khax.net/2009/11/28/multi-gateway-routing-with-iptables-and-iproute2/ and http://linux-ip.net/html/adv-multi-internet.html have been very useful (even if the code is not 100% complete!)

share|improve this answer
1  
One final addition, I did add a sleep 20 to the top of the script as the openvpn connection wasn't completing in time. I also added echo 2 > /proc/sys/net/ipv4/conf/tun0/rp_filter to the script as it's necessary to disable reverse packet filter for tun0. When a reply comes back from tun0 with source address S, the reverse packet filter checks "if I was to route a packet to address S, and it wouldn't go through tun0, I will drop the packet" - and because when doing this lookup there is no valid fwmark, it determines the route would be the usual default route, so it drops the packet. –  Steve Nov 11 '11 at 10:10
    
I had to edit your script to make it work for me. The last line route add default gw 192.168.0.1 eth0 seemed to route port 80/443 traffic through the local gateway instead of the tun0 as intended. Changing the last line to route add default tun0 seems to do the trick for me. –  user127121 Apr 8 '12 at 6:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.