1

So I know there is another question on here that I used as a guide as it was super helpful! (Setup routing and iptables for new VPN connection to redirect **only** ports 80 and 443) Only my goal is a bit different. I am running a headless gui-less install of Ubuntu Server 12.04 that is being used for a variety of different purposes... I would like all traffic to travel un-prohibited through my ISP except for my transmission traffic. I have a VPN i subscribe to that allows me access for which I only want to direct a single port's traffic to. I am currently using a modified version of the code from the above link. My current code is below:

#!/bin/sh

sleep 200

DEV1=eth0
IP1=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 192.`
GW1=10.0.1.1
TABLE1=open
TABLE2=vpn
DEV2=tun0
IP2=`ifconfig|perl -nE'/dr:(\S+)/&&say$1'|grep 10.`
GW2=`route -n | grep 'UG[ \t]' | awk '{print $2}'`

ip route flush table $TABLE1
ip route flush table $TABLE2
ip route show table main | grep -Ev ^default | while read ROUTE ; do
    ip route add table $TABLE1 $ROUTE
    ip route add table $TABLE2 $ROUTE
done
ip route add table $TABLE1 $GW1 dev $DEV1 src $IP1
ip route add table $TABLE2 $GW2 dev $DEV2 src $IP2
ip route add table $TABLE1 default via $GW1
ip route add table $TABLE2 default via $GW2

echo "1" > /proc/sys/net/ipv4/ip_forward
echo "1" > /proc/sys/net/ipv4/ip_dynaddr
echo "2" > /proc/sys/net/ipv4/conf/tun0/rp_filter

ip rule add from $IP1 lookup $TABLE1
ip rule add from $IP2 lookup $TABLE2
ip rule add fwmark 1 lookup $TABLE1
ip rule add fwmark 2 lookup $TABLE2

iptables -t nat -A POSTROUTING -o $DEV1 -j SNAT --to-source $IP1
iptables -t nat -A POSTROUTING -o $DEV2 -j SNAT --to-source $IP2

iptables -t nat -A PREROUTING           -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables        -A OUTPUT               -m state --state ESTABLISHED,RELATED          -j CONNMARK --restore-mark
iptables -t nat -A PREROUTING -i $DEV1  -m state --state NEW                          -j CONNMARK --set-mark 1
iptables -t nat -A PREROUTING -i $DEV2  -m state --state NEW                          -j CONNMARK --set-mark 2
iptables -t nat -A PREROUTING           -m connmark --mark 1                          -j MARK --set-mark 1
iptables -t nat -A PREROUTING           -m connmark --mark 2                          -j MARK --set-mark 2
iptables -t nat -A PREROUTING           -m state --state NEW -m connmark ! --mark 0   -j CONNMARK --save-mark

iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p tcp --dport  44447 -j CONNMARK --set-mark 2
iptables -t mangle -A PREROUTING -i $DEV2 -m state --state NEW -p udp --dport 44447 -j CONNMARK --set-mark 2

route del default
ip route del 0.0.0.0/1
ip route del 128.0.0.0/1
route add default gw $GW1 eth0

I took into account the original poster's own comments, modified it to my IP configuration and port needs... extended the sleep to ensure the OpenVPN configuration had occured... And then also deleted two routes that I believe were added by my VPN provider for a fallback incase the default route failed... Now everything seems to be okay except a few things...

  1. traceroutes fail... completely...
    $ traceroute yahoo.com
    traceroute to yahoo.com (206.190.36.45), 30 hops max, 60 byte packets
     1  * * *
     2  * * *
     3  * * *
     4  * * *
     5  * * *
  1. ping results in 100% packet loss
    $ ping google.com
    PING google.com (173.194.43.46) 56(84) bytes of data.
    ^C
    --- google.com ping statistics ---
    119 packets transmitted, 0 received, 100% packet loss, time 118945ms

I don't know what is causing this???

$ nslookup
> google.com
Server:     10.0.1.1
Address:    10.0.1.1#53

Non-authoritative answer:
Name:   google.com
Address: 173.194.43.46
Name:   google.com
Address: 173.194.43.38
Name:   google.com
Address: 173.194.43.35
Name:   google.com
Address: 173.194.43.41
Name:   google.com
Address: 173.194.43.39
Name:   google.com
Address: 173.194.43.34
Name:   google.com
Address: 173.194.43.36
Name:   google.com
Address: 173.194.43.37
Name:   google.com
Address: 173.194.43.32
Name:   google.com
Address: 173.194.43.40
Name:   google.com
Address: 173.194.43.33

route table below:

$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         Rolands-AirPort 0.0.0.0         UG    0      0        0 eth0
default         Rolands-AirPort 0.0.0.0         UG    100    0        0 eth0
10.0.1.0        *               255.255.255.255 UH    0      0        0 eth0
10.0.1.0        *               255.255.255.0   U     0      0        0 eth0
10.4.0.1        10.4.49.21      255.255.255.255 UGH   0      0        0 tun0
10.4.49.21      *               255.255.255.255 UH    0      0        0 tun0
hosted-by.lease Rolands-AirPort 255.255.255.255 UGH   0      0        0 eth0

Any help would be greatly appreciated!

2
  • I looked further into my route tables and I found that for some reason the VPN was also pushing a route into my "open" table to override my defaults to my ISP. I am adding these two lines to my script to successfully resolve the issue. ip route del table open 0.0.0.0/1 ip route del table open 128.0.0.0/1
    – user230409
    Jun 12, 2013 at 19:10
  • well my routing is still not working right... I now have successful pings and traceroutes to external ip's so I know my traffic is defaulting to the open table or using the main table... However my transmission client does not seem to be using the VPN and as such I am not connectable. I can see that transmission is announcing my external IP from my ISP and not the IP from my VPN. I am attempting to overcome this by adding ip route add table vpn default dev tun0 because there was no default route in my VPN table... if anyone has any ideas please help!
    – user230409
    Jun 12, 2013 at 21:29

2 Answers 2

1

Suppose that there is a problem to redirect only packets destined to remote port Y of some internet server and only those packets owned by user with ID 100Z. We should properly utilize both MANGLE and NAT tables for redirecting specific ports opened by this user because rules from MANGLE table applies before NAT and before rerouting (e.g. "ip route" rules) procedure in contrast with rules from NAT table which applies after rerouting. In an example below NAT rules used only for sending packets and MANGE for sending and receiving for convenience.

There is could be following local configuration of interfaces: "tun0" interface for VPN with local machine address 10.???.???.??? and peer (gateway) 10.???.???.(???+1) and real hardware interface eth0 with local machine address 192.168.XXX.XXX and gateway 192.168.XXX.1.

We can now apply following rules to split-redirect traffic for desired user.

Part 1. Output of locally-generated (outbound connection) packages.

  • Mark outgoing locally generated packages with connection-specific mark for future rerouting with different rt_table.
/sbin/iptables -t mangle -A OUTPUT -s 192.168.XXX.XXX -p tcp --dport Y -m owner --uid-owner 100Z -j CONNMARK --set-mark 2
/sbin/iptables -t mangle -A OUTPUT -s 192.168.XXX.XXX -p udp --dport Y -m owner --uid-owner 100Z -j CONNMARK --set-mark 2
  • Translate connection-specific markers to package-specific markers.
/sbin/iptables -t mangle -A OUTPUT -s 192.168.XXX.XXX -p tcp --dport Y -m owner --uid-owner 100Z -m connmark --mark 2 -j MARK --set-mark 2
/sbin/iptables -t mangle -A OUTPUT -s 192.168.XXX.XXX -p udp --dport Y -m owner --uid-owner 100Z -m connmark --mark 2 -j MARK --set-mark 2
  • Save marker for further restoring it for all tracked connections. This can be implemented in NAT POSTROUTING table only - it's just a precaution to not loose markers after connection traverse the routing table.
/sbin/iptables -t mangle -A OUTPUT -s 192.168.XXX.XXX -p tcp -m owner --uid-owner 100Z -m tcp --dport Y -m connmark ! --mark 0 -j CONNMARK --save-mark
/sbin/iptables -t mangle -A OUTPUT -s 192.168.XXX.XXX -p udp -m owner --uid-owner 100Z -m udp --dport Y -m connmark ! --mark 0 -j CONNMARK --save-mark

Normally connection-specific markers should be saved after packages rerouted and reached NAT POSTROUTING table

/sbin/iptables -t nat -A POSTROUTING -s 192.168.XXX.XXX -o tun0 -p tcp --dport Y -m owner --uid-owner 100Z -m connmark --mark 2 -j CONNMARK --save-mark
/sbin/iptables -t nat -A POSTROUTING -s 192.168.XXX.XXX -o tun0 -p udp --dport Y -m owner --uid-owner 100Z -m connmark --mark 2 -j CONNMARK --save-mark
  • Replace source with proper VPN-related local address for tun0 interface.
/sbin/iptables -t nat -A POSTROUTING -s 192.168.XXX.XXX -o tun0 -p tcp --dport Y -m owner --uid-owner 100Z -m connmark --mark 2 -j SNAT --to-source 10.???.???.???
/sbin/iptables -t nat -A POSTROUTING -s 192.168.XXX.XXX -o tun0 -p udp --dport Y -m owner --uid-owner 100Z -m connmark --mark 2 -j SNAT --to-source 10.???.???.???

Part 2. Input of connections related to locally-generated (outbound connection) packages.

  • Restore connection-related markers for input packets before they go to the routing table.
/sbin/iptables -t mangle -A PREROUTING -i tun0 -p tcp -j CONNMARK --restore-mark
/sbin/iptables -t mangle -A PREROUTING -i tun0 -p udp -j CONNMARK --restore-mark

Part 3. Optional optimization

Other than above rules there is a strong desire in optimizing marking process to prevent or minimize redundant and repeated checks with marking. The most general approach for this task is to distinguish rules between NEW and RELATED,ESTABLISHED connection and use below template for optimal remarking already marked connections: * Restore old markers (you probably won't do this for NEW connections because they are obviously wasn't marked earlier):

/sbin/iptables -A OUTPUT -t mangle -j CONNMARK --restore-mark
  • Skip already marked connections:
/sbin/iptables -A OUTPUT -t mangle -m mark ! --mark 0 -j ACCEPT
  • Mark new (usually marking only NEW connection should be enough) connections:
/sbin/iptables -A OUTPUT -m mark --mark 0 -p tcp --dport 21 -t mangle -j MARK --set-mark 1
/sbin/iptables -A OUTPUT -m mark --mark 0 -p tcp --dport 80 -t mangle -j MARK --set-mark 2
  • Mark other packages with a "dummy" marker (that is not used anywhere else at all):
/sbin/iptables -A OUTPUT -m mark --mark 0 -t mangle -p tcp -j MARK --set-mark 3
  • Finally save markers:
/sbin/iptables -A OUTPUT -t mangle -j CONNMARK --save-mark

NOTE: the above settings are not a mandatory part of netfilter configuration for split tunnel - it is just a template (i.e. proper suggestion) for the way to minimize redundant filter checks. It also can be POSTROUTING instead of OUTPUT, but only one place is recommended for markings, do not need to implement it twice.

Part 4. Create table aliases in rt_tables.

echo 2 vpn >/etc/iproute2/rt_tables
echo 3 novpn >/etc/iproute2/rt_tables

Part 5. Configuring routing tables for VPN connections.

This is a standard configuration for VPN that allows to redirect all local sources (0.0.0.0/1 and 128.0.0.0/1 subnets include the whole IP range for local addresses) and also provides backward compatibility for returning packages (i.e. the "default" route).

  • Table with VPN connection
ip route flush table vpn
ip route add 10.???.???.(???+1) dev tun0 src 10.???.???.??? table vpn
ip route add 0.0.0.0/1 dev tun0 via 10.???.???.(???+1) table vpn
ip route add 128.0.0.0/1 dev tun0 via 10.???.???.(???+1) table vpn
ip route add 192.168.0.0/16 src 192.168.XXX.XXX dev eth0 table vpn
ip route add default via 192.168.XXX.1 dev eth0 table vpn
  • The other table called "novpn". It's goal to provide direct routing bypassing VPN connection.
ip route flush table novpn
ip route add 192.168.0.0/16 src 192.168.XXX.XXX dev eth0 table novpn
ip route add default via 192.168.XXX.1 dev eth0 table novpn

Part 6. Configuring routing rules (it is better to follow rule order below)

ip rule add from all lookup novpn
ip rule add from all fwmark 2 lookup vpn
ip rule add from 10.XXX.XXX.XXX lookup vpn

Part 7. Final steps

Now it is time to flush main routing table (there is no need for it while split tunnel is working):

ip route flush table main

The last step in configuration is to enable IP forwarding and dynamic IP addresses for sockets (client application connections) in Linux kernel:

echo 1 >/proc/sys/net/ipv4/ip_forward
echo 1 >/proc/sys/net/ipv4/ip_dynaddr

Now we can launch OpenVPN client with "--route-noexec" and/or "--ifconfig-noexec" (whether you already know or still not receive push message from VPN server with your tun0 interface configuration) parameters. If there are troubles with tun0 addresses - it is better to not use "--ifconfig-noexec" and let OpenVPN client set tun0 addresses for you. After that you just need to delete some old rules from "ip route" and "/sbin/iptables" and replace them with similar ones containing proper tun0 addresses (refer to all lines with 10.XXX.XXX.XXX or 10.XXX.XXX.(XXX+1) from above).

openvpn --config ./<your_config>.ovpn --route-noexec --ifconfig-noexec --auth-nocache

Those are only rules for outbound connections. The main difference between inbound and outbound configurations: inbound rules should be implemented mostly in "mangle" table, not "nat", and vice-verse for outbound as shown above. There is an exception - for output of both locally-generated (outbound) and answered (inbound connection) packages rules must be placed in "mangle" table due to netfilter architecture implementation (for some reason "nat" table will intercept packages only after the rerouting procedure - consult Wikipedia netfilter graph or netfilter official documentation).

0

Well I have my work around... I don't know why I wasn't able to get this to work but basically what I found was that even with my routing configured this way, the nature of bittorrent led my ISP provided public IP being announced... which is a major problem for connectivity. To fix my issue I stopped the transmission-daemon and altered the /etc/transmission-daemon/settings.json file to bind on the ipv4 address of the tun0 interface. This is not an ideal solution as it is not dynamic. However, the ip address of my vpn is static enough that this shouldn't be too much of an issue. If anyone has a better answer please let me know!

please note that the issues of 100% packet loss with ping and no traceroute available was fixed by manually removing routes pushed by the VPN provider (within the script).

Forcing the transmission traffic over the VPN was a matter of adjusting a setting within the client to bind on the IP address of the tun0 interface. This is not a proper solution but works none the less. I am continuing to use the script so that the default interface is not the VPN.

1
  • are you by chance using openvpn client ? if so there's a very easy way to do this let me know and if so I'll give you the info to make it work. I do similar on my home router with 3 vpn connections. Nov 20, 2013 at 4:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .