1

I have several servers (B, C, ...) which are in one place and one machine (A), which is behind a regular old home router. I want all of them to be able to connect to each other. It seems to me that OpenVPN is a great place to start here, so I set up a tunnel (tun0) between A and B over the internet. This is what the situation looks like:

 10.8.0.4                 10.8.0.1
----------  10.8.0.0/24   ---------- 
|        |                |        | 
|   A   tun0 ----------- tun0  B   | 10.128.140.204 
|        |                |        | 
----------                -- eth1 -- 
                              |
                              | 10.128.0.0/16
                              |
                          -- eth1 --
                          |        |
                          |    C   | 10.128.13.224
                          |        |
                          ----------

Specifically, I need A to be able to reach C under the address 10.128.13.224 and C to be able to reach A under the address 10.8.0.4. This means that NAT won't do - most resources talk about setting up NAT on B so A can see C - and this is where it seems to get tricky.

So this is what I've been trying to do now:

  • tell A to use 10.8.0.1 as gateway for 10.128.0.0/16
  • tell B to use 10.128.140.204 as gateway for 10.8.0.0/24
  • get B to act as that gateway

Routing table on A:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         [home router]   0.0.0.0         UG    0      0        0 enp0s3
10.8.0.0        *               255.255.255.0   U     0      0        0 tun0
10.128.0.0      localhost       255.255.0.0     UG    0      0        0 tun0
[home ip]       *               255.255.255.0   U     0      0        0 enp0s3

A# ip route get 10.128.13.224
10.128.13.224 via 10.8.0.1 dev tun0  src 10.8.0.4
    cache

Looks great. Routing table on C:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    0      0        0 eth0
10.8.0.0        10.128.140.204  255.255.255.0   UG    0      0        0 eth1
10.13.0.0       *               255.255.0.0     U     0      0        0 eth0
10.128.0.0      *               255.255.0.0     U     0      0        0 eth1
[public IP]     *               255.255.255.0   U     0      0        0 eth0

C# ip route get 10.8.0.4
10.8.0.4 via 10.128.140.204 dev eth1  src 10.128.13.224
    cache

Great. Enable IP forwarding on B:

B# cat /proc/sys/net/ipv4/ip_forward
1

I guess the default ACCEPT everything rules in iptables would also work, but for good measure, I limited for FORWARD chain a bit:

B# iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-A FORWARD -s 10.128.0.0/16 -d 10.8.0.0/24 -i eth1 -o tun0 -j ACCEPT
-A FORWARD -s 10.8.0.0/24 -d 10.128.0.0/16 -i tun0 -o eth1 -j ACCEPT

Well, they don't. A# ping 10.128.41.180 does nothing and neither does C# ping 10.8.0.4.

IF I add NAT as follows:

B# iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A POSTROUTING -s 10.8.0.0/24 -d 10.128.0.0/16 -o eth1 -j MASQUERADE

then a ping from A to C works.

A# ping 10.128.13.224
PING 10.128.13.224 (10.128.13.224) 56(84) bytes of data.
64 bytes from 10.128.13.224: icmp_seq=1 ttl=63 time=120 ms
...

This is not what I want to do, but I guess it means that something is working. Another curious thing is that if I configure the MASQUERADE the other way round, a ping from C to A does not work, so I guess there could be a clue here.

Now I'm starting to wonder if what I want to do even can be done or if I have some basic misunderstanding of how networking works. Is this even possible? Am I overlooking something basic? I'd appreciate any hints or resources.

Improve this question
  • Why do you want to limit forwarding in the first place? Is the reason that is reachable from the outside, and you want to prevent indiscrimenate forwarding to A and C? If yes, what happens if there are not restrictions on forwarding (just to test)? What happens if you just restrict by source and destination, and don't do connectiont tracking? (I'm actually not sure if connection tracking will allow pings to succeed, may guess would be that this is the problem). – dirkt Nov 20 '16 at 17:55
  • @dirkt Correct, since B is reachable from the internet, I would like to restrict forwarding. If I remove all FORWARD rules and switch the FORWARD policy to ACCEPT, the result is identical (i.e. not working, but sort of working with the NAT rule as described above). I've removed the connection tracking since it is not important to my rules anyway, see updated post. – Tillerino Nov 20 '16 at 18:48
  • A# ping 10.128.41.180 while in theory should work that IP doesn't show up in your diagram. Thanks for putting the work into that diagram! What does the routing table on B look like and what happens if you do the ip route get on B? Can you ping both sides from B? – Seth Nov 21 '16 at 12:03
  • @Seth Oh you're right. Sorry, as I said there are more servers on that network and I copied the wrong output. After staging the entire setting on a bunch of virtual machines at home, I figured out the problem: the network doesn't allow foreign source addresses on IP packets, see the answer I posted. Thanks for your reply! – Tillerino Nov 21 '16 at 12:13
1

Turns out that the above will work beautifully in a regular setting. The servers (B, C, ...) however were in a network which doesn't allow sending packets which have a source address that doesn't match the sender's IP address:

When pinging C from A, A sends an IP packet inside an Ethernet frame over tun0 to B which roughly looks like this:

----------------------------------------------------
| source: A's tun0 MAC | destination: B's tun0 MAC |
| -------------------------------------------------|
| | source: 10.8.0.4 | destination: 10.128.13.224 ||
| | payload: PING                                 ||
| -------------------------------------------------|
----------------------------------------------------

B then checks its routing and sends the following over eth1:

----------------------------------------------------
| source: B's eth1 MAC | destination: C's eth1 MAC |
| -------------------------------------------------|
| | source: 10.8.0.4 | destination: 10.128.13.224 ||
| | payload: PING                                 ||
| -------------------------------------------------|
----------------------------------------------------

The network inspects this and finds that the source address 10.8.0.4 does not match B's eth1 address 10.128.140.204 and doesn't transmit the frame. That's why a plain gateway won't work on this network.

Improve this answer
  • Wow this would've been hard to figure out/didn't cross my mind with the information provided. Don't forget you can mark your own answer as one! – Seth Nov 21 '16 at 12:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.