3

I want to route a traffic from a specific program through a WireGuard VPN. I set the wireguard interface, config, routings, etc. To do routing through the VPN only for a single program, I run the program with a different group (sudo -g GROUPNAME) and use

iptables -t mangle -A OUTPUT -o owner --gid-owner GROUPNAME -j MARK --set-mark MARK` to set

to mark to the outgoing packet. Then, I add a rule

ip add rule fwmark MARK table TABLE

so that the packets going from group GROUPNAME are routed using another routing table TABLE.

In the TABLE I add routes:

ip add VPN_SUBNET/24 dev wg0 table TABLE
ip add VPN_SERVER_IP via INTERNET_GATEWAY table TABLE
ip add default via VPN_GATEWAY table TABLE

With this configuration all pings and requests fails. Using tcpdump -i wg0 -n I found that packets are actually sent from a wrong IP address, from the public IP address of my server instead of my internal IP address in the VPN network, as I expected. To verify this I added iptables -t nat -A POSTROUTING -o wg0 -j MASQUERADE, so that all the wrong source addresses that are sent through the VPN interface are translated into the correct VPN source address. And it works!

Trying to find out why without masquerading the packets are sent with a wrong source address, I tried to add a routing in the default routing table (without fwmark filtering) for the IP address to which I am trying to send a request:

ip add SERVER_IP via VPN_GATEWAY

And it also worked even without the POSTROUTING trick. But of course now the whole system routes all the packets sent to SERVER_IP through the VPN, I don't need that.

So now I'm just curious, why the packet that is marked to use another routing table, seemingly still first uses the default routing table, sets the default source address and only then uses the other routing table to obtain the gateway though which the packet would be sent to the destination?

And it there any way I can make the packets sent from the GROUPNAME group to be routed immediately with the correct routing table, so with the correct source address, without the seemingly unnecessary MASQUERADE trick?

1 Answer 1

3

First an explanation of what you are experiencing and why it behaves like that:

When the application generates the packet, the fwmark is not set yet. Since the application does not specify a source IP, the kernel assigns it one based on the default routing table, since the OUTPUT rules giving it the fwmark didn't run yet. That's why it gets your public IP assigned as source. Later, the OUTPUT rule affects only the fwmark and doesn't touch the source IP of course. See this for how the kernel selects a source IP when a packet is created). Notice that it you change the global default route, the kernel will give new packets the other source IP, as you experience, since this default route is what it looks at.

Now what you can do...

Obviously you found one solution, which is to add masquerading on the outgoing packets, which is very normal to do and not a "trick". Of course, since in your case you have a single source address, you can use a static SNAT instead, like this:

iptables -t nat -A POSTROUTING -s <YOUR_PUBILC_IP> -j SNAT --to <YOUR_VPN_IP>

If your "public IP" is not really public, but you are behind a NAT anyway (in your defalt table), then you can keep your source IP unmodified and instead add a route for your "public IP" (which is the source of your packets) in the other side of your VPN to know how to route to that one back through the tunnel again. Of course, since you called it public, in your case I assume it is really public and not an option for you, but adding this here for completeness since it is a common situation.

Notice how these two solutions are kind of "symmetric", in one case you have a NAT on your public interface, in the other one you need to make a NAT through your tunnel.

Now, if for some reason you really need the source IP to be correct from the beginning (in that case, please edit your question to give some more details why you think something is wrong with a NAT/MASQUERADING), you would have to do that in the actual program you are running. Many programs have a native option for that (e.g. `nc -s <source_ip> ...>), or if it is code you have access to you can modify it to add a hardcoded source IP, or better add a command line argument. If you don't have access to the code, some other things that come now to my mind are much more "trickery" than simply adding a NAT.

5
  • Would setting a prefsrc on the chosen route help with the situation? (If I remember correctly, it triggers a second routing/PBR lookup.) I'm using WireGuard with PBR, and I don't recall having needed to use SNAT. Apr 26, 2023 at 5:12
  • 1
    Not really familiar with the prefsrc option, so probably worth a try, but I wouldn't expect it to work either, because the first routing decision happens without the mark set, so the kernel would add a source based on the main routing table anyway. When it comes to a reroute-check after the mark is set, the packet has already received a source IP from the kernel and I don't expect prefsrc to do anything at that point.
    – gepa
    Apr 26, 2023 at 7:35
  • Regarding your case where Wireguard with PBR works, you are probably doing PBR based on some policy rules that are available during the initial routing decision (e.g. destination IP or port), in contrast to the case of the OP where it is based on the mark which get set in the mangle OUTPUT chain which happens AFTER the initial routing.
    – gepa
    Apr 26, 2023 at 7:37
  • Hmm, yes, in my case the program itself sets the packet mark on its packets, so that might be why the preferred source (default via 192.168.10.1 src 192.168.10.244) takes effect. Apr 26, 2023 at 7:56
  • Other possible method: forcing a SO_MARK socket option. If an application can do this (it also requires privileges), obviously it could choose to bind instead. But with much more elbow grease involving cgroups and ebpf, it's possible to force the mark on the application, thus getting a correct source address including for the UDP service case without the application knowing this was done to it. I just don't know how to do this, but I know it's possible since this commit: git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/…
    – A.B
    Apr 26, 2023 at 18:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .