I have sooo struggled with this so here's a COMPLETE solution. It's tested on Ubuntu 15 & 16. You can especially use it with OpenVPN to route certain apps outside the VPN tunnel interface.
The complete "cgroup" solution
How that works?
- Linux kernel will put the app into a control group. Network traffic from apps in this cgroup will be identified by their class ID at the network controller level.
- iptables will mark this traffic and force it to exit with the right IP
- ip route will process marked traffic in a different routing table, with a default route to whatever gateway IP you want.
I've made a novpn.sh script to automate dependencies install & run. Tested on Ubuntu.
Start your VPN first.
# If you don't use eth0, edit the script setting.
sudo chmod +x novpn.sh
./novpn.sh traceroute www.google.com
First, install cgroup support and tools:
sudo apt-get install cgroup-lite cgmanager cgroup-tools
Reboot (may not be necessary).
You need iptables 1.6.0+. Get iptables 1.6.0 release source, extract it, then run this (
--disable-nftables flag will avoid errors) from iptables source dir:
sudo apt-get install dh-autoreconf bison flex
./configure --prefix=/usr \
sudo make install
Now, the real config. Define a control group named
novpn. Processes in this cgroup will have a classid of
echo 0x00110011 > net_cls.classid
Now, we'll suppose the interface you want to use for the specific app is
eth0 with a gateway IP of
10.0.0.1. Replace these by what you really want (get the info from
ip route). Run still as root:
# Add mark 11 on packets of classid 0x00110011
iptables -t mangle -A OUTPUT -m cgroup --cgroup 0x00110011 -j MARK --set-mark 11
# Force the packets to exit through eth0 with NAT
iptables -t nat -A POSTROUTING -m cgroup --cgroup 0x00110011 -o eth0 -j MASQUERADE
# Define a new "novpn" routing table
# DO THIS JUST ONCE !
echo 11 novpn >> /etc/iproute2/rt_tables
# Packets with mark 11 will use novpn
ip rule add fwmark 11 table novpn
# Novpn has a default gateway to the interface you want to use
ip route add default via 10.0.0.1 table novpn
# Unset reverse path filtering for all interfaces, or at least for "eth0" and "all"
for i in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $i; done
Finally, run your app on the specific interface:
sudo cgcreate -t $USER:$USER -a $USER:$USER -g net_cls:novpn
cgexec -g net_cls:novpn traceroute www.google.com
# Close all Firefox windows first
cgexec -g net_cls:novpn firefox
Or if you want to move an already running process to the cgroup, well... you can't! That seems to be due to the NAT (masquerade) function:
iptables -nvL -t nat doesn't match when the cgroup is switched, but
iptables -nvL -t mangle does match.
# Get PID of the process (we'll then suppose it's 1234)
# Add to cgroup - THIS DOESN'T WORK! Silently fails to produce the final result.
sudo echo 1234 > /sys/fs/cgroup/net_cls/novpn/tasks
# Remove - but this works...
sudo echo 1234 > /sys/fs/cgroup/net_cls
Credits: No answer were working as expected, but a mix of them did: chripell answer evolware article Per process routing take 2: using cgroups, iptables and policy routing, How do I make a specific process NOT going through a OpenVPN connection?, Kill switch for OpenVPN on the basis of iptables