Making routing decisions based on UID using nftables

Question

I am trying to route packets based on the uid of of the originating process.

I know that PREROUTING is not hit for outgoing packets. In iptables there where a MANGLE table where you could place those rules inside the OUTPUT chain. nftables do not have a MANGLE hook or something. There is only the OUTPUT hook.

Is this a limitation of nftables that there is no way to mark outgoing packets that originate on the host before the routing decision?

nftables v0.7 (Scrooge McDuck)
Linux 4.14.5-1-ARCH #1 SMP PREEMPT Sun Dec 10 14:50:30 UTC 2017 x86_64 GNU/Linux

Answer 1

The naming conventions in nftables might have misled you into thinking the feature doesn't exist.

From nftables' wiki:

route, which is used to reroute packets if any relevant IP header field or the packet mark is modified. If you are familiar with iptables, this chain type provides equivalent semantics to the mangle table but only for the output hook (for other hooks use type filter instead). This is supported by the ip and ip6 table families.

So doing all this from an LXC container, using debian stretch's kernel 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3 (2017-12-03), I'll make uid 1001 the only user able to connect to internet (here Google's public DNS using TCP):

# ip route
default via 10.0.3.1 dev eth0 
10.0.3.0/24 dev eth0 proto kernel scope link src 10.0.3.66 
# pkill -9 dhclient
# ip route del default
# ip route add default via 10.0.3.2 dev eth0 # 10.0.3.2 doesn't exist, but a default route is required for the routing code to trigger the fwmark rule at all, else there's a direct "Network is unreachable" with no packet generated.
# ip route add default via 10.0.3.1 table 10
# ip rule add fwmark 1 table 10

The canonical naming for mangle below can be found here (or /usr/share/doc ...). -150 is the value of NF_IP_PRI_MANGLE

# nft add table ip mangle
# nft 'add chain ip mangle output { type route hook output priority -150; }'
# nft add rule ip mangle output skuid 1001 counter mark set 1
# id
uid=0(root) gid=0(root) groups=0(root)
# nc -v -n -z 8.8.8.8 53 # will timeout on ARP request to 10.0.3.2
nc: connect to 8.8.8.8 port 53 (tcp) failed: No route to host
# su - test
$ id
uid=1001(test) gid=1001(test) groupes=1001(test)
$ nc -v -n -z 8.8.8.8 53
Connection to 8.8.8.8 53 port [tcp/*] succeeded!

Last note. There's a comment about using a network namespace to simplify configuration, and I can only concur: it would have been harder to give this example without a container (with its own network namespace). Also for example, if there is a suid root command using the network, depending on how the command is working, it might trigger or not the mark and route.

UPDATE: some packets initiated by the user's connection don't belong to the user (anymore). This can happen for the last ACK acknowledging a FIN packet at the end of a connection, and more is often the case for RST packets when a connection is prematurely aborted. These packets won't match an owner, thus won't receive a mark, won't be rerouted, and this might lead to timeouts at the end of a transfer. To avoid it, this should be combined with other features like ct mark set mark/mark set ct mark (equivalent of iptables' CONNMARK). More informations about conntrack marks and equivalent iptables issues and solutions there:

nftables' mark and conntrack mark

Netfilter Connmark

Iptables: matching outgoing traffic with conntrack and owner. Works with strange drops

Is it possible to force fwmark reflection in arbitrary-TCP reply packets?

iptables packets without uid

Timeouts did happen in the previous "successful" test with nc above, but since the command returned, it wasn't visible without network capture. The timeouts occured on the peer server side. To solve it, keep the same routing commands, but replace the nft rules above with:

# nft add table ip mangle
# nft 'add chain ip mangle output { type route hook output priority -150; }'
# nft flush chain ip mangle output
# nft add rule ip mangle output mark set ct mark counter
# nft add rule ip mangle output mark ne 0 counter accept
# nft add rule ip mangle output skuid 1001 counter mark set 1
# nft add rule ip mangle output ct mark set mark counter

The counters are just for debugging.

Answer 2

nft add rule inet filter output meta skuid 1004 drop

or:

chain output {
    type filter hook output priority 0; policy accept;
    meta skuid t drop
}