0

tldr; A LAN device is able to get 2 separate internet connection (1 NATted from ISP Router, 1 from manual PPP connection). Able to separate ISP-WAN and PPP-WAN IPv4 into 2 routes using ip rules but not IPv6.


Apologies for the lengthy post. My requirement is to have 2 separate routing tables to connect to the internet through 2 separate routes.

I will be sharing how I am managing this with IPv4 - however IPv6 is where I am unable to figure out the steps.

I have a rather unique setup where I am able to get a global IPv6 address from my ISP in 2 ways.
My ISP provides an FTTH modem+router (RT) that uses PPPoE to connect to internet.

My LAN is on 192.168.5.0/24 and the ISP RT is the default Gateway at 192.168.5.1

I have a raspberry Pi RPi connected to RT using single interface br0 at IP 192.168.5.22 which is where I am performing this setup.

RT has an option to enable PPPoE passthrough that lets my Raspberry Pi also request for a PPP connection and hence,

1 - I can get a public IP on ppp0 for direct Internet access
2 - I can also access the internet through gateway 192.168.5.1 of RT


Working IPv4 Setup

<WAN1> <WAN2>
    |    |
    |    `-RT - <PPP Public IP 1>
    |      `-br_lan - 192.168.5.1/24
    |        `-Laptop
    |          `-eth0 - 192.168.5.100/24 (default gateway: 192.168.5.1)
    |        `-RPi 
    |          `-eth0 - 192.168.5.22/24 (default gateway: 192.168.5.1 in table 111)
    `------------`-ppp0 - <PPP PUBLIC IP 2> (default gateway in table `main`)

ppp0 interface is the default gateway on RPi's main routing table:

# ip route show
default dev ppp0 scope link
192.168.5.0/24 dev br0 proto dhcp scope link src 192.168.5.22 metric 204
<redacted hop ip> dev ppp0 proto kernel scope link src <redacted PPP-WAN ip>

A new separate routing table contains the route to ISP-WAN with RT as gateway

# ip route show table 111
default via 192.168.5.1 dev br0 proto static
192.168.5.0/24 dev br0 proto static

A rule is set for all incoming connections from LAN to use ISP-WAN gateway

# ip rule show
0:      from all lookup local
0:      from 192.168.5.0/24 lookup 111

curl -4 ifconfig.co from RPi does return PPP-WAN public IP ✅ Any device on LAN using RPi as gateway gets the ISP-WAN public IP ✅

All traffic originating from Pi will use ppp0 and if needed I can add new rules to use ISP-WAN for any VPN interfaces :)
The corresponding systemd-networkd config:

# cat 11-br0.network
[Match]
Name=br0

[Network]
#DHCP=ipv6
DHCP=no
Address=192.168.5.22/24
Address=fdXX:22/64
DNS=1.1.1.1
#IPv6AcceptRA=yes
#IPv6PrivacyExtensions=true
IPv6AcceptRA=no

# Seperate LAN and PPP traffic in different tables
[Route]
Destination=0.0.0.0/0
Gateway=192.168.5.1
Table=111

[Route]
Destination=192.168.5.0/24
Table=111

[RoutingPolicyRule]
From=192.168.5.0/24
Table=111

[Link]
MACAddress=HH:MM:XX:YY:ZZ

The IPv6 part

There are 2 ways I can get IPv6 Access on RPi:
1. Through DHCPv6 requests on ppp0 with help of wide-dhcpv6-client.service (Ask Ubuntu link that describes this method) /etc/wide-dhcpv6/dhcp6c.conf:

# Default dhpc6c configuration: it assumes the address is autoconfigured using
# router advertisements.

profile default
{
  information-only;

  request domain-name-servers;
  request domain-name;

  script "/etc/wide-dhcpv6/dhcp6c-script";
};

interface ppp0 {
    # Request Prefix Delegation on ppp0, and give the received prefix id 0
    send ia-pd 0;
};

id-assoc pd 0 {
    prefix-interface br0 {
        # Assign subnet 1 to br0
        sla-len 0;
        sla-id 1;
        ifid-random;
    };
};

Resulting ip addr & routing table:

4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.22/24 brd 192.168.5.255 scope global br0
       valid_lft forever preferred_lft forever
#Global Public
    inet6 2401::XX/64 scope global
       valid_lft forever preferred_lft forever
#ULA
    inet6 fdXX::Y/64 scope global
       valid_lft forever preferred_lft forever
#Link local
    inet6 fe80::Y/64 scope link
       valid_lft forever preferred_lft forever

5: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc pfifo_fast state UNKNOWN group default qlen 3
    link/ppp
    inet <Public IPv4> peer <IPv4>/32 scope global ppp0
       valid_lft forever preferred_lft forever
    inet6 fe80::<ppp0 link local> peer fe80::123:46ff:fe07:0508/128 scope link
       valid_lft forever preferred_lft forever

# Route
::1 dev lo proto kernel metric 256 pref medium
<PPP IPv6 Prefix>::/64 dev br0 proto kernel metric 256 pref medium
fdXX:22::/64 dev br0 proto kernel metric 256 pref medium
fe80::123:46ff:fe07:0508 dev ppp0 proto kernel metric 256 pref medium
fe80::<ppp0 link local> dev ppp0 proto kernel metric 256 pref medium
fe80::/64 dev br0 proto kernel metric 256 pref medium
default via fe80::123:46ff:fe07:0508 dev ppp0 proto ra metric 1024 expires 1616sec hoplimit 64 pref medium

With this, anything from or through RPi (like VPN clients) will go out through ppp0 using the WAN prefix provided by PPP

2. Through RT's Router Advertisements on br0 using following change in [Network]:

# /etc/systemd/network/11-br0.network
[Network]
DHCP=ipv6
DNS=1.1.1.1
IPv6AcceptRA=yes
IPv6PrivacyExtensions=true

With this, all traffic from LAN as well as RPi both will use RT's ISP-WANv6 prefix.

The problem

If I enable any of the 2 methods, I cannot enable the other.
If I am using ISP-WAN, and I try to get an IPv6 from ppp0, wide-dhcpv6-client.service will throw client6_init: bind: Address already in use.

I tried to setup separate routing table for LAN br0 by a rule that routes all traffic from RT's ULA use the new table:

# Setting a static ULA + using DHCP to get Prefix from `RT`
[Network]
DHCP=ipv6
Address=fdZZ::1/64
IPv6AcceptRA=yes
IPv6PrivacyExtensions=true

# Route all outgoing traffic through the gateway provided by RT's RA
[Route]
Destination=::/0
Gateway=_ipv6ra
Table=111

# All traffic to LAN devices that use RT's ULA 
[Route]
Destination=fdXX::/64
Table=111

# All LAN devices (which use RT's ULA) will use the above routes configured in table 111
[RoutingPolicyRule]
From=fdXX::/64
Table=111

Other than the link local address fe80:: I don't get which address is in use. Tried removing teh link-local address from br0 but no luck.

The Global Prefix from ISP-WANv6 and ppp0 are both different, The ULA Prefix from RT is also different than the static ULA I configured manually.

In IPv4 I can easily create a separate route with a different default gateway, but unable to figure the equivalent in IPv6.

I'd need to be able to have 2 paths to the IPv6 internet, albeit using separate tables
And a rule that tells which route to take based on the originating source.
Any help would be greatly appreciated!

Disclosure: I have also posted this on serverfault - however not sure if it was the right place so reposting here

1
  • Could you post the actual system state, i.e. both IPv6 routing tables from ip -6 route show table X as well as the IPv6 policy rules from ip -6 rule? Aug 14, 2022 at 9:10

1 Answer 1

0

wide-dhcpv6-client.service will throw client6_init: bind: Address already in use.

This most likely means you have two DHCPv6 clients running on the same interface (i.e. both the WIDE client and systemd-networkd are probably trying to speak on ppp0, or something like that).

While DHCPv4 clients use "raw" sockets to send/receive the DHCP packets and therefore do their own packet filtering, that's no longer the case in DHCPv6. All DHCPv6 clients use ordinary UDP sockets and must be able to bind a socket to your link-local address at UDP port 546 (i.e. it's in fact [fe80::X]:546 that might be "already in use").

Try running the WIDE client from command line with verbose or debug modes enabled to see whether it's trying to use other interfaces than the one you've configured.

In IPv4 I can easily create a separate route with a different default gateway, but unable to figure the equivalent in IPv6.

It's exactly the same in IPv6.

Well, Linux does have an easier way to do this in IPv6 – you can in fact have routes that match source as well as destination – but ignoring that, it has literally the same "policy routing" feature for IPv4 and IPv6 both.

Do keep in mind that the two protocols have separate lists of policy rules, i.e. ip -4 rule only affects IPv4 and ip -6 rule only affects IPv6. (Systemd-networkd's [RoutingPolicyRule] determines which ruleset to use based on From= or To= address.)

If the rules are present in ip -6 rule, make sure they're actually correct (e.g. for some reason your rule uses fdXX::/64 but your address is fdZZ::1/64) and in the correct order (policy rules are matched top-to-bottom, not longest-prefix, so you may need to specify a custom Priority=).


The alternative to policy routing: Linux's IPv6 stack lets you simply use [Route] Source= in the main routing table, so you can have multiple routes matching the same destination (e.g. ::/0) but matching different source prefixes.

2
  • I actually do have 2 DHCPv6 clients. and this does seem to be the issue. Both of them are on br0 where systemd-networkd listens to RAs from ISP router, whereas WIDE-DHCPv6 service listens to RAs coming from the ppp0 connection. I did a little more digging and what I see is my Default route using the source address of ISP's global prefix when it should actually use PPP0's provided global prefix. With this ping6 works but anything else on the internet fails. This looks to me like Linux mixing up the RAs. Ill need to organize things before I reply more with my findings. THanks!
    – RuMAN S
    Aug 15, 2022 at 2:55
  • I think I managed to figure it out. In the /etc/wide-dhcpv6/dhcp6c.conf, I removed the Prefix Delegation blocks and replaced it with send ia-na 0; and id-assoc na 0 { }; So now my ppp0 interface just contains a public /128 IPv6, and br0 gets the RAs from RT (giving a /64 prefix) which I can then just isolate to a separate table. Now my Pi uses ppp0 and any external clients like VPN clients are now using the separate routing table for br0/ISP-WAN routes. I will test a bit more before marking it as solved. THanks a lot for your help! It made me look at the right direction :)
    – RuMAN S
    Aug 15, 2022 at 6:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .