Multiple OpenVPN Clients & Binding Interfaces with bind.so - Does not work

Question

Edit
I solved the problem through writing a script to generate routing tables and rules, after following @MariusMatutiae's link to an introduction to policy routing.

---

I'm attempting to run two OpenVPN clients on my Raspberry Pi running Minibian, and bind a specific application (get_iplayer) to one of the VPNs using bind.so as explained here and here. I originally followed the guide here, installing iproute and downloading and compiling bind.so as instructed there.

The VPNs
I am using config files provided by Private Internet Access.

One VPN points to their Switzerland server, uses udp, and I have set the option dev tun0, as I wish this to be the main tunnel through which all traffic goes through except that which I declare explicitly through the use of of bind.so. This tunnel works fine, all traffic seems to go through it.

The second VPN points to the London, UK server, uses tcp and has the option dev tun1 set to act as the second tunnel. This tunnel appears work fine when ran by itself. I can run get_iplayer correctly.

The problem occurs when I run both instances at the same time. No traffic seems to go through the tun1 interface, even when I attempt to use bind.so and the LD_PRELOAD approach as explained in the links above.

bind.so
As far as I'm aware I have compiled bind.so properly, copied it to /usr/lib, etc. Frustratingly, I did get it work once but I have no idea how that occurred.

Commands
I have been using ip route to find the gateway address; I'm confident I am using the correct ip address. e.g.:

$ ip route
0.0.0.0/1 via 10.30.1.17 dev tun1 
0.0.0.0/1 via 10.198.1.5 dev tun0 
default via 192.168.1.254 dev eth0 
10.30.1.1 via 10.30.1.17 dev tun1 
10.30.1.17 dev tun1  proto kernel  scope link  src 10.30.1.18 
10.198.1.1 via 10.198.1.5 dev tun0 
10.198.1.5 dev tun0  proto kernel  scope link  src 10.198.1.6 
104.238.169.140 via 192.168.1.254 dev eth0 
128.0.0.0/1 via 10.30.1.17 dev tun1 
128.0.0.0/1 via 10.198.1.5 dev tun0 
179.43.177.66 via 192.168.1.254 dev eth0 
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.84 

Then running:

BIND_ADDR="10.30.1.17" LD_PRELOAD=/usr/lib/bind.so get_iplayer --type=tv

results in no connection and nothing in the log for the UK VPN.

Stopping the Switzerland VPN and running the same get_iplayer command results in a connection and information being downloaded. ip route gives the following:

$ ip route
0.0.0.0/1 via 10.30.1.17 dev tun1 
default via 192.168.1.254 dev eth0 
10.30.1.1 via 10.30.1.17 dev tun1 
10.30.1.17 dev tun1  proto kernel  scope link  src 10.30.1.18
104.238.169.119 via 192.168.1.254 dev eth0 
128.0.0.0/1 via 10.30.1.17 dev tun1 
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.84

So there's no change in ip address or anything odd happening as far as I can tell with routing when opening or closing the different VPNs.

I'm at a loss as to why bind.so seems to be having no effect. There is no output to the terminal to show whether its successful or not, and I'm not sure where to look for a log, if it is outputting anything (the output in the terminal is for the command, i.e. get_iplayer).

Obviously I could run cron jobs/scripts to open and close the VPNs to allow me to run get_iplayer successfully/through the UK VPN, but I would much rather keep both VPNs open, have all of my traffic going through the tun0 interface and only use tun1 for get_player when I need to with bind.so.

Can anyone provide any help? If we cannot solve this issue, some help towards writing specific routing tables or rules for the get_iplayer process would be great.

Thanks.

Answer 1

bind.so is not responsible for this problem, which is instead one of routing: in every routing table there can be only one default gateway, but every instance of a VPN tries to set the gateway (and the routing in general) as it best suits it, resulting in tables with lines like:

  0.0.0.0/1 via 10.30.1.17 dev tun1 
  0.0.0.0/1 via 10.198.1.5 dev tun0 

What is the kernel to do with this? Should it route thru tun0 or thru tun1? The answer is: it does not really route, your pc cannot connect.

Ways out:

1. Setup two routing tables, with a rule instructing the kernel when to use either of them. This is called policy or source routing, you find a brief introduction to it here. This is tricky, not because of policy routing per se (which is a cinch), but because you will have to setup the routing of one of the two instances of the OpenVPN yourself, to make sure the routing rules are added to the other routing table. But it can be done, because OpenVPN provides a statement,

--route-nopull

When used with --client or --pull, accept options pushed by server EXCEPT for routes.

which allows you to setup the routing as per your needs.

2. Setup a network namespace where you start one instance of OpenVPN, and the programs that use it. You find an introduction to network namespaces here. That also requires some work, but it is mostly rote work. If you want, the following script (very simple-minded!) that I wrote works out of the box.

EDIT

Since you are working from an ssh session, I had to modify my script slightly.

You can start it with

     newnsssh NameOfNNS start

but you will still find yourself inside the default network namespace, NNS for short. Before accessing it, it is best to open up a terminal:

     xterm &

This will open a graphic terminal provided you connected to the RPI by means of

    ssh -Y me@rpi

and provided the $DISPLAY environment variable is set to:

  export DISPLAY=localhost:10.0

After having opened the xterm, go into it, and issue the following command:

   sudo ip netns exec NameOfNNS bash

The new prompt is in the new NNS; to check,

    ip netns identify $$

If nothing is returned, you are in the default NNS, otherwise you will be shown NameOfNNS. You can now start the OpenVPN

      openvpn --config /path/to/config/file &
      su YourName

and you are done. Now from within this xterm all programs will be routed by this instance of the OpenVPN, while all programs started outside this xterm will be started outside the OpenVPN, or thru the other instance of the OpenVPN, if you happen to have enacted another one.

When you are done, just close the xterm and, in the ssh session,

    newnsssh NameOfNNS stop

That's all.

#!/bin/bash

#
# This script will setup an internal network 10.173.N.0/24; if this causes
# any conflict, change the statement below.

export IP_BASE=10.173

# It will open an xterm window in the new network namespace; if anything
# else is required, change the statement below.


# The script will temporarily activate ip forwarding for you. If you
# do not wish to retain this feature, you will have to issue, at the 
# end of this session, the command
# echo 0 > /proc/sys/net/ipv4/ip_forward 
# yourself. 

export WHEREIS=/usr/bin/whereis

# First of all, check that the script is run by root:

[ "root" != "$USER" ] && exec sudo $0 "$@"

if [ $# != 2 ]; then 
    echo "Usage $0 name action"
    echo "where name is the network namespace name,"
    echo " and action is one of start| stop| reload."
    exit 1
 fi

 # Do we have all it takes?

 IERROR1=0
 IERROR2=0

 export IP=$($WHEREIS -b ip | /usr/bin/awk '{print $2}')
 export IPTABLES=$($WHEREIS -b iptables | /usr/bin/awk '{print $2}')

 if [ x$IP = x ] ; then
    echo "please install the iproute2 package"
    IERROR1=1
 fi

 if [ x$IPTABLES = x ] ; then
    echo "please install the iptables package"
    IERROR2=1
 fi


 if [[ $IERROR1 == 0 && $IERROR2 == 0 ]] 
    then
    :   
 else
    exit 1
 fi


 prelim() {

 # Perform some preliminary setup. First, clear the proposed 
 # namespace name of blank characters; then create a directory
 # for logging info, and a pid file in it. Lastly, 
 # enable IPv4 forwarding. 

    VAR=$1
    export NNSNAME=${VAR//[[:space:]]}

    export OUTDIR=/var/log/newns/$NNSNAME

    if [ ! -d $OUTDIR ]; then
        /bin/mkdir -p $OUTDIR
    fi
    export PID=$OUTDIR/pid$NNSNAME


    ICOUNTER=1
    export Nns=$ICOUNTER 
    if [ $Nns == 1 ]; then
        echo 1 > /proc/sys/net/ipv4/ip_forward
    fi
}

start_nns() {

# Check whether a namespace with the same name already exists. 

$IP netns list | /bin/grep $1 2> /dev/null
if [ $? == 0 ]; then 
    echo "Network namespace $1 already exists,"
    echo "please choose another name"
    exit 1
fi

# Here we take care of DNS

/bin/mkdir -p /etc/netns/$1
echo "nameserver 8.8.8.8" > /etc/netns/$1/resolv.conf
echo "nameserver 8.8.4.4" >> /etc/netns/$1/resolv.conf

# The following creates the new namespace, the veth interfaces, and
# the bridge between veth1 and a new virtual interface, tap0.
# It also assigns an IP address to the bridge, and brings everything up

$IP netns add $1
$IP link add veth-a$1 type veth peer name veth-b$1
$IP link set veth-a$1 up
$IP tuntap add tap$1 mode tap user root
$IP link set tap$1 up
$IP link add br$1 type bridge
$IP link set tap$1 master br$1
$IP link set veth-a$1 master br$1
$IP addr add $IP_BASE.$Nns.1/24 dev br$1
$IP link set br$1 up

# We need to enable NAT on the default namespace

$IPTABLES -t nat -A POSTROUTING -j MASQUERADE

# This assigns the other end of the tunnel, veth2, to the new 
# namespace, gives it an IP address in the same net as the bridge above, 
# brings up this and the (essential) lo interface, sets up the 
# routing table by assigning the bridge interface in the default namespace
# as the default gateway, creates a new terminal in the new namespace and 
# stores its pid for the purpose of tearing it cleanly, later. 

$IP link set veth-b$1 netns $1
$IP netns exec $1 $IP addr add $IP_BASE.$Nns.2/24 dev veth-b$1
$IP netns exec $1 $IP link set veth-b$1 up
$IP netns exec $1 $IP link set dev lo up
$IP netns exec $1 $IP route add default via $IP_BASE.$Nns.1 
ln -s /proc/1/ns/net /var/run/netns/default 2> /dev/null
#   $IP netns exec $1 bash & $IP netns exec $1 echo "$!" > $PID
}

stop_nns() {

# Check that the namespace to be torn down really exists

$IP netns list | /bin/grep $1 2>&1 1> /dev/null
if [ ! $? == 0 ]; then 
    echo "Network namespace $1 does not exist,"
    echo "please choose another name"
    exit 1
fi

# This kills the terminal in the separate namespace, 
# removes the file and the directory where it is stored, and tears down
# all virtual interfaces (veth1, tap0, the bridge, veth2 is automatically
# torn down when veth1 is), and the NAT rule of iptables. 


rm /var/run/netns/default
$IP link set br$1 down
$IP link del br$1
$IP netns del $1
$IP link set veth-a$1 down
$IP link del veth-a$1
$IP link set tap$1 down
$IP link del tap$1
$IPTABLES -t nat -D POSTROUTING -j MASQUERADE
/bin/rm /etc/netns/$1/resolv.conf
    /bin/rmdir /etc/netns/$1
}


case $2 in
  start)
    prelim "$1"
    start_nns $NNSNAME
    ;;
  stop)
    prelim "$1"
    stop_nns $NNSNAME
    ;;
  reload)
    prelim "$1"
    stop_nns $NNSNAME
    prelim "$1"
    start_nns $NNSNAME
    ;;
 *) 
 # This removes the absolute path from the command name

    NAME1=$0
    NAMESHORT=${NAME1##*/}

    echo "Usage:" $NAMESHORT "name action,"
    echo "where name is the name of the network namespace,"
    echo "and action is one of start|stop|reload"
    ;;
esac