How to pass traffic through other Interfaces

Question

I have total 4 Interfaces on my VM where enp0s3 is the primary Interface and others are the secondary Interfaces

ip a output

[root@vm]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 xdp/id:1 qdisc sfq state UP group default qlen 1000
    link/ether 02:00:17:01:c1:5d brd ff:ff:ff:ff:ff:ff
    inet 100.10.10.100/24 brd 100.10.10.255 scope global dynamic enp0s3
       valid_lft 81440sec preferred_lft 81440sec
    inet6 fe80::17ff:fe01:c15d/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 02:00:17:01:6c:5d brd ff:ff:ff:ff:ff:ff
    inet 100.10.10.101/24 scope global enp1s0
       valid_lft forever preferred_lft forever
4: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 02:00:17:00:a2:f3 brd ff:ff:ff:ff:ff:ff
    inet 100.10.10.102/24 scope global enp2s0
       valid_lft forever preferred_lft forever
5: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 02:00:17:01:a7:4e brd ff:ff:ff:ff:ff:ff
    inet 100.10.10.103/24 scope global enp3s0
       valid_lft forever preferred_lft forever

I am redirecting incoming traffic from ports 9100, 9101, 9102, 9103 to some specific ips using the rules

iptables -t nat -A PREROUTING -p udp --dport 9100 -j DNAT --to 1.1.1.1:9100
iptables -t nat -A POSTROUTING -p udp -d 1.1.1.1 --dport 9100 -j MASQUERADE

iptables -t nat -A PREROUTING -p udp --dport 9101 -j DNAT --to 2.2.2.2:9101
iptables -t nat -A POSTROUTING -p udp -d 2.2.2.2 --dport 9101 -j MASQUERADE

iptables -t nat -A PREROUTING -p udp --dport 9102 -j DNAT --to 3.3.3.3:9102
iptables -t nat -A POSTROUTING -p udp -d 3.3.3.3 --dport 9102 -j MASQUERADE

iptables -t nat -A PREROUTING -p udp --dport 9103 -j DNAT --to 4.4.4.4:9103
iptables -t nat -A POSTROUTING -p udp -d 4.4.4.4 --dport 9103 -j MASQUERADE

iptables -t nat -A PREROUTING -p udp --dport 1245 -j DNAT --to 5.5.5.5:1245
iptables -t nat -A POSTROUTING -p udp -d 5.5.5.5 --dport 1245 -j MASQUERADE

I wanted 1.1.1.1 & 2.2.2.2 shall pass traffic from enp2s0 and 3.3.3.3 & 4.4.4.4 shall pass traffic from enp3s0

IP 5.5.5.5 it wanted the traffic to pass from enp1s0 so I used a commad

ip route add 100.10.10.0/24 dev enp1s0 scope link metric 1000 table main
ip route add 5.5.5.5/32 via 100.10.10.1 dev enp1s0 table main

It Worked

But when I use the same rules for other interfaces It doesn't work I used these rules from below

ip route add 100.10.10.0/24 dev enp2s0 scope link metric 1000 table main
ip route add 1.1.1.1/32 via 100.10.10.1 dev enp2s0 table main

ip route add 100.10.10.0/24 dev enp2s0 scope link metric 1000 table main
ip route add 2.2.2.2/32 via 100.10.10.1 dev enp2s0 table main

ip route add 100.10.10.0/24 dev enp3s0 scope link metric 1000 table main
ip route add 3.3.3.3/32 via 100.10.10.1 dev enp3s0 table main

ip route add 100.10.10.0/24 dev enp3s0 scope link metric 1000 table main
ip route add 4.4.4.4/32 via 100.10.10.1 dev enp3s0 table main

Can someone please help me in this case. I am not good at routing. Thanks in Advance

Answer 1

Assuming enp0s3 is considered as the "default" interface (i.e., the one you want to use for destinations, LAN or WAN, that are not specifically "assigned" to an interface), when you add/configure address on the remaining interfaces, you should use the prefix length /32 instead of /24 (or, you can probably use the flag noprefixroute instead; I have no idea what's the potential pros of that though):

ip a add 100.10.10.101/32 dev enp1s0
ip a add 100.10.10.102/32 dev enp2s0
ip a add 100.10.10.103/32 dev enp3s0

then you can add the desired indirect routes with the onlink flag:

ip r add 1.1.1.1 via 100.10.10.1 dev enp2s0 src 100.10.10.102 onlink
ip r add 2.2.2.2 via 100.10.10.1 dev enp2s0 src 100.10.10.102 onlink
ip r add 3.3.3.3 via 100.10.10.1 dev enp3s0 src 100.10.10.103 onlink
ip r add 4.4.4.4 via 100.10.10.1 dev enp3s0 src 100.10.10.103 onlink
ip r add 5.5.5.5 via 100.10.10.1 dev enp1s0 src 100.10.10.101 onlink

(apparently when you use onlink, the src 100.10.10.x part is crucial for the traffics to use the address configured on the corresponding interface as source address, so make sure you don't omit that)

---

Alternatively, you can make use of destination-based policy routing instead.

Create extra route tables with routes that would make use of the extra interfaces:

ip r add 100.10.10.0/24 dev enp1s0 table 101
ip r add default via 100.10.10.1 table 101
ip r add 100.10.10.0/24 dev enp2s0 table 102
ip r add default via 100.10.10.1 table 102
ip r add 100.10.10.0/24 dev enp3s0 table 103
ip r add default via 100.10.10.1 table 103

(the indirect routes above will have the corresponding dev part set according to the direct route in the same table that covers their nexthop 100.10.10.1, which is why you can omit that, but certainly you can specify it explicitly as well)

Then add ip rules that causes traffics with certain destination addresses to make use of these tables:

ip ru add to 1.1.1.1 lookup 102
ip ru add to 2.2.2.2 lookup 102
ip ru add to 3.3.3.3 lookup 103
ip ru add to 4.4.4.4 lookup 103
ip ru add to 5.5.5.5 lookup 101