0

I am trying to set up an Ubuntu 17.10 VM to use multiple VPN connections, and to load balance connections between them. I am trying to do that by running multiple openvpn clients, and trying to mark connections with iptables/connmark and using mode random to mark different connections with different values, then matching these with different VPN routing tables. However, I am having no success, so I have tried to simplify the situation to try to understand what is going wrong.

I have reduced the scenario to a single VPN connection, with all my routing logic in a dedicated routing table, and trying to mark all packets to use that routing table.

If I force all traffic to use that table, it works ok. If I try to mark all packets and then match that mark to use the table, it doesn't work.

Here are the rules that work:

0:      from all lookup local
10:     from all lookup VPN2
32766:  from all lookup main
32767:  from all lookup default

and here are the rules that do not work:

0:      from all lookup local
10:     from all fwmark 0x14 lookup VPN2
32766:  from all lookup main
32767:  from all lookup default

And here is my iptables mangle table that should mark everything:

# iptables -nvL -t mangle
Chain PREROUTING (policy ACCEPT 6785 packets, 464K bytes)
 pkts bytes target     prot opt in     out     source               destination
 8013  545K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore
 7981  543K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK set 0x14
 7958  541K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save

Chain INPUT (policy ACCEPT 6773 packets, 463K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 4382 packets, 1132K bytes)
 pkts bytes target     prot opt in     out     source               destination
 4636 1159K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore
 4516 1146K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK set 0x14
 4382 1132K CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save

Chain POSTROUTING (policy ACCEPT 4151 packets, 1116K bytes)
 pkts bytes target     prot opt in     out     source               destination

I understand my rules are a bit pointless here (the restore, before marking anyway) but I have just been trying to get it to mark everything to understand where the problem is.

Like I say, if I have everything use the VPN2 routing table, it works fine - and I can see connections marked with 20 (0x14) using conntrack -L

But if I try to have the routing based on the mark, it doesn't work, and the connection attempt isn't even visible in conntrack -L, even though I see the SYN_SENT status in netstat. I also see the source IP is not my VPN tun interface, so the table is not being used.

Any advice to debug this is greatly appreciated.

Many thanks

  • from what I'm understanding: when mangle is used in output it triggers a reroute check. that's this 2nd routing check that will use the fwmark rule. But the IP was already chosen at the 1st routing check. you could try doing also a nat. anyway you should concentrate on testing prerouting (from an other vm or network namespace) before testing output when output doesn't behave as expected. Packet flow in Netfilter and General Networking – A.B May 27 '18 at 18:10
  • I still haven't figured it out completely yet, but I think you have put me on the right track with the 'rerouting' you have mentioned. I found this post, and I think my filter table is filtering the packets before the rerouting happens: linuxquestions.org/questions/linux-networking-3/… – Danny Trigo May 28 '18 at 14:37
  • Many many thanks. I have now managed to get connections randomly sent down different interfaces by temporarily removing all my filters (not sure how I'll re-add those), adding an SNAT rule in the nat/output chain, and setting this to 2 to prevent filtering of returning packets: /proc/sys/net/ipv4/conf/tun1/rp_filter. Thanks again – Danny Trigo May 28 '18 at 15:03

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.