8

Abstract:

I have (I presume routing) issues when adding a secondary ISP to my existing network setup: incoming traffic through Router1 is not answered, but local traffic and incoming through Router0 is working fine.

How can I keep the parts that currently function well working, while making incoming traffic through Router1 work?

Elaboration:

I've sketched a diagram below with the bare essentials of the situation (in practice there are more devices on each LAN, but they do not matter).

This is the situation:

  • I have two internal networks: LAN0 is 192.168.x.0/24 and LAN1 is 192.168.y.0/24. Both work fine for internal traffic (for instance http using cURL).
  • LAN0 has always been connected through Router0 and ISP0 to the Internet.
  • LAN1 always had Router1, but is now connected through ISP1 to the Internet.
  • Machines only on LAN0 and having a default route through Router0 work fine for outgoing and incoming traffic.
  • Machines only on LAN1 and having a default route through Router1 work fine for outgoing and incoming traffic.
  • Internal traffic on LAN0 and LAN1 has always worked fine.
  • Incoming traffic through Router1 for WindowsB arrives correctly: I can connect to it over RDP from WindowsC.
  • Incoming traffic through Router1 for LinuxB arrives (according to tcpdump), but not answered back as a curl http://e.f.g.h fron LinuxC shows with a tcpdump on LinuxB shows:

It shows only packets that - according to the tcpdump output format - have a SYN flag set:

LinuxB:/tmp/LinuxB.eth1.80 # tcpdump -i eth1 'port 80'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
13:35:19.489779 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1287047182 ecr 0,sackOK,eol], length 0
13:35:19.788841 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1287047478 ecr 0,sackOK,eol], length 0
13:35:19.888835 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1287047578 ecr 0,sackOK,eol], length 0
13:35:19.989412 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1287047678 ecr 0,sackOK,eol], length 0
13:35:20.089685 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1287047778 ecr 0,sackOK,eol], length 0
13:35:20.190836 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1287047877 ecr 0,sackOK,eol], length 0
13:35:20.392123 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1287048072 ecr 0,sackOK,eol], length 0
13:35:20.693692 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,sackOK,eol], length 0
13:35:21.197162 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,sackOK,eol], length 0
13:35:22.204134 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,sackOK,eol], length 0
13:35:24.115961 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,sackOK,eol], length 0
13:35:27.852374 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,sackOK,eol], length 0
13:35:31.967049 IP i.j.k.l.57512 > 192.168.y.2.http: Flags [S], seq 816356596, win 65535, options [mss 1460,sackOK,eol], length 0

This is the LinuxB route table:

LinuxB:/tmp/LinuxB.eth1.80 # route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.x.1     0.0.0.0         UG    0      0        0 eth0
loopback        *               255.0.0.0       U     0      0        0 lo
link-local      *               255.255.0.0     U     0      0        0 eth0
192.168.x.0     *               255.255.255.0   U     0      0        0 eth0
192.168.x.0     *               255.255.255.0   U     0      0        0 eth1

Since connecting over RDP from WindowsC to WindowsB works fine, I resume this is indeed a routing problem. This is the WindowsB route table:

C:\temp>route print
===========================================================================
Interface List
0x1 ........................... MS TCP Loopback interface
0x2 ...00 0c 29 35 77 e1 ...... AMD PCNET Family PCI Ethernet Adapter - Packet Scheduler Miniport
0x3 ...00 0c 29 35 77 eb ...... VMware Accelerated AMD PCNet Adapter - Packet Scheduler Miniport
===========================================================================
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      192.168.x.1     192.168.x.4      10
          0.0.0.0          0.0.0.0      192.168.y.1     192.168.y.4       5
        127.0.0.0        255.0.0.0        127.0.0.1       127.0.0.1       1
      192.168.x.0    255.255.255.0      192.168.x.4     192.168.x.4      10
      192.168.x.4  255.255.255.255        127.0.0.1       127.0.0.1      10
    192.168.x.255  255.255.255.255      192.168.x.4     192.168.x.4      10
      192.168.y.0    255.255.255.0      192.168.y.4     192.168.y.4      10
      192.168.y.4  255.255.255.255        127.0.0.1       127.0.0.1      10
    192.168.y.255  255.255.255.255      192.168.y.4     192.168.y.4      10
        224.0.0.0        240.0.0.0      192.168.x.4     192.168.x.4      10
        224.0.0.0        240.0.0.0      192.168.y.4     192.168.y.4      10
  255.255.255.255  255.255.255.255      192.168.x.4     192.168.x.4       1
  255.255.255.255  255.255.255.255      192.168.y.4     192.168.y.4       1
Default Gateway:       192.168.y.1
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0      192.168.y.1       5
          0.0.0.0          0.0.0.0      192.168.x.1      10

So how can I get the routing on LinuxB to be like this:

  • keep default route on LinuxB to 192.168.x.1 so outgoing traffic keeps using Router0/ISP0
  • keep answering incoming requests coming from LAN0 on LAN0
  • keep answering incoming requests coming from LAN1 on LAN1
  • keep answering incoming requests through Router0 (a.b.c.d/192.168.x.1) via 192.168.x.1
  • start answering incoming requests through Router1 (e.f.g.h/192.168.y.1) via 192.168.y.1
  • bonus: have Router1 fail-over or load-balance with Router0

Postscript:

The PNG image below is generated on UML text through the free online PlantUML engine. If you want to see the original UML text, paste the PNG image link into this PlantUML form, then press Submit.

enter image description here

Improve this question
4
  • Just found an interesting answer at unix.stackexchange.com/a/23345/69111 which I'm investigating to see if it applies to my case, and how to implement it using yast.
    – Jeroen Wiert Pluimers
    Jul 18, 2014 at 5:48
  • 1
    Yast seems out for doing complex routing and route seems to be deprecated in favour of ip. See opensuse.14.x6.nabble.com/yast2-advanced-routing-td3083578.html and suse.com/documentation/sles11/book_sle_admin/data/…
    – Jeroen Wiert Pluimers
    Jul 18, 2014 at 6:06
  • 1
    Your linked solution at unix.stackexchange is going in the right direction, and would solve the linux problem -- the linux node has two separate routing tables, and uses the appropriate table depending on which interface the connection was initiated on. I am not aware of any such solution for Windows. Unless there is a real need for two subnets, I would get rid of one and use one router for both ISPs. You can buy routers that can deal with two WANs properly, or you could use another Linux system to do it, which would simplify the routing on the systems behind it.
    – David Mackintosh
    Jul 18, 2014 at 13:20
  • @DavidMackintosh in the future I will. This is temporary: fiber is actually DSL, and going to move to a different fiber address. The machine is DNS and MX, so I want it to be accessible from two ISPs to have continuity as I'm not sure if DSL will be shutdown before or after fiber becomes active.
    – Jeroen Wiert Pluimers
    Jul 18, 2014 at 20:13

1 Answer 1

Reset to default
1

I had a shell script for doing something like that a long-long time ago but, sorry, could find it. So I can only give you the pointers to the solutions I implemented back then. I'm writting mostly from memory so some examples are missing:

  1. I had one routing table per uplink (ip route ... table 101, ip route ... table 102). That goes into /etc/iproute2/rt_tables.

    101 isp1 102 isp2

    You need to setup those tables also:

    ip route add default via $Gateway1 dev $Interface1 table isp1 ip route add default via $Gateway2 dev $Interface2 table isp2

    #Don't forget the default table:

    ip route add default via $DefaultGateway dev $DefaultInterface

  2. Enable iptables connection tracking (modprobe nf_conntrack)

  3. Set an iptables rule for NEW incoming connections to -j MARK the packets somehow (ie: 0x201, 0x202)

  4. Set an ip rule that ensure that traffic going out trhough an interface uses the right routing table

    ip rule add from $Ip1 table isp1 ip rule add from $Ip2 table isp2

  5. Set an ip rule (ip rule add ...) stating that "packets marked 0x201 shall lookup routing table 201", one rule for every uplink.

With everything in place you should be able to receive and initiate connections with any wan uplink and even balance outgoing connections.

Those would be the basics. Iptables + "ip route" + "ip rule" and you're good to go.

Improve this answer
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.