Create ethernet loop network - how to configure routes

Question

I have a network that has to be configured as a loop. It consists on 3 nodes each of which has two interfaces. The diagram below explains it.

+--->(eth0) Node 1 (eth1)--->(eth0) Node 2 (eth1)--->(eth0) Node 3 (eth1)--->+
|    10.0.3.1     10.0.1.1  10.0.1.2     10.0.2.2  10.0.2.3      10.0.3.3    |
+--<----------------------------<--------------------------------------------+

I want to make a ping from Node 1 to Node 3 so the request goes through Node2 and the reply goes directly to Node 1 from Node 3.

node1$ ping 10.0.2.3

I have configured the nodes as:

node1# route add -net 10.0.2.0/24 gw 10.0.1.2

node2# route add -net 10.0.3.0/24 gw 10.0.2.3

node3# route add -net 10.0.1.0/24 gw 10.0.3.1

When running the ping, the request from Node 1 arrives at Node 3. However Node 3 doesn't reply, it does not generate even the reply (at least that I can capture with wireshark).

Could you please give me some hint?

TA

Answer 1

When a node picks the source IP address for a packet it's originating, unless otherwise constrained, it typically picks the IP address "closest" to the next hop on the route to the packet's destination.

From Node 1's point of view, the next hop to Node 3 is 10.0.1.2. The Node 1 IP address closest to 10.0.1.2 is 10.0.1.1, not 10.0.3.1. (An IP address on the same subnet as the destination is considered "closer" to the destination than an IP address not on the same subnet.)

Check the source IP address of the ping. Most likely, it's 10.0.1.1, not 10.0.3.1. If Node 3 has no route to 10.0.1.1, it can't reply.

Answer 2

The nodes are correctly failing to repeat to prevent runaway bridge broadcasting. I recommend you run Spanning Tree Protocol. This will allow you to put fully functioning routes between all the nodes. I cannot think of another way to do this unless you are willing to limit connectivity between certain links at layer 2 or layer 3.

Answer 3

(/sbin/route is deprecated, use ip route instead).

If you want your packets to only travel in one direction, that mean every node need to use the next node as their gateway, whatever the destination is.

node1# ip route add 10.0.0.0/22 via 10.0.1.2
node2# ip route add 10.0.0.0/22 via 10.0.2.3
node3# ip route add 10.0.0.0/22 via 10.0.3.1

However, on most distributions, reverse path filtering is often enabled by default. Reverse path filtering is a filter for incoming packet which tries to check that an answer to this packet would go to the same interface the packet is received. This is a good thing on common network where routing is symetric, but in your case, you need asymetric routing, so you might just disable reverse path filtering on the interface where you receive packets, or at least reduce it so it does just check if the source IP is routable :

node1# sysctl -w net.ipv4.conf.eth0.rp_filter=2
node2# sysctl -w net.ipv4.conf.eth0.rp_filter=2
node3# sysctl -w net.ipv4.conf.eth0.rp_filter=2

Documentation about reverse path filtering and other knobs are available in the kernel documentation, at Documentation/networking/ip-sysctl.txt. Where to find that depends on your distribution (or just browse the web for ip-sysctl.txt).

Happy looping!