1

I am trying to setup an isolated virtual network on Linux with 2 virtual devices connected through bridge (running in ubuntu linux on wsl2).

My end goal is educational: to learn more about layer3 networking. Right now I'm trying to write C code that operates on raw sockets: I want to have 1 application that sends out packets and another receiving it. Before I'm running my own C code that I don't know if written correctly I want to have a proof that my linux virtual devices are setup correctly by using tcpdump and ping -I test_eth1 -b (broadcast ping on ethernet socket -I test_eth1). My assumption is that ping command will put a packet on test_eth1 and tcpdump should receive it on test_eth0 because test_eth0 and test_eth1 are connected with virtual switch.

I have created 2 tap devices (test_eth0, test_eth1) connected through bridge (test_switch) like so:

  ip tuntap add dev test_eth0 mode tap
  ip tuntap add dev test_eth1 mode tap

  ip link set dev test_eth0 address 00:11:22:33:44:55
  ip link set dev test_eth1 address 00:11:22:33:44:56

  ip link add name test_switch type bridge
  ip link set dev test_eth0 master test_switch
  ip link set dev test_eth1 master test_switch

  ip link set dev test_eth0 up
  ip link set dev test_eth1 up
  ip link set dev test_switch up

To test this setup I've started tcpdump on test_eth0

  tcpdump -i test_eth0 -p -e -A -vv

and running ping broadcast on test_eth1

  ping -I test_eth1 -b 255.255.255.255

I would expect tcpdump to be able to see ping packets. But tcpdump doesn't capture anything.

brctl show test_switch output shows

bridge name     bridge id               STP enabled     interfaces
test_switch             8000.001122334455       no              test_eth0
                                                        test_eth1

appears that switch has enabled=no, but why?

Output for brctl showstp test_swtich:

$ sudo brctl showstp test_switch
test_switch
 bridge id              8000.001122334455
 designated root        8000.001122334455
 root port                 0                    path cost                  0
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay             0.00                 bridge forward delay       0.00
 ageing time             300.00
 hello timer               0.00                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                 146.05
 flags


test_eth0 (1)
 port id                8001                    state                  disabled
 designated root        8000.001122334455       path cost                100
 designated bridge      8000.001122334455       message age timer          0.00
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.00
 flags

test_eth1 (2)
 port id                8002                    state                  disabled
 designated root        8000.001122334455       path cost                100
 designated bridge      8000.001122334455       message age timer          0.00
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.00
 flags

11
  • 1
    @John Creating a virtual broadcast domain isn't even the same thing as Bridged networking. The latter refers to a setup in which you enslave a physical NIC to a bridge. When it isn't bridged networking, whether the virtual BD is isolated in any sense depends on L3 forwarding and so on. Don't spread FUDs before you stop mixing up things.
    – Tom Yan
    Apr 27, 2023 at 1:47
  • 1
    @Ski I don't think you want tap in this case. AFAIK tap needs a program (e.g. qemu, openvpn) to "catch" and what flows out of the "tap". On the host itself the interface is the just the "angle" of the "tap" (and using tcpdump on it is like watching the "angle"). You probably want to try veth instead of tap if you don't want to use some (if that's possible at all) VMs inside the WSL to perform your test. (I'm not a native English speaker. I hope you could get what I'm trying to convey.)
    – Tom Yan
    Apr 27, 2023 at 1:57
  • 1
    @John The OP is NOT even talking about bridged networking. Period.
    – Tom Yan
    Apr 27, 2023 at 2:02
  • 1
    @Ski For the record, while veth itself doesn't require netns to work (and / in the sense that broadcast ping + tcpdump will give you expected result), but in case you are going to actually assign IPs and try unicast ping, it might not work since it appears that the host won't reply to ARP requests sent/triggered to itself this way (as in, ping with -I). I don't know the rationale behind but in that case you can use netns additonally. (And without -I and netns it would just take the loopback path, which you may already know.)
    – Tom Yan
    Apr 27, 2023 at 3:01
  • 1
    @John: Are you assuming that "bridged" always refers to virtual machines? It absolutely doesn't. The term existed for years before desktop VMs even became a thing; it's the term for devices such as Ethernet switches. VMs only use the term "bridged networking" because they describe behavior that emulates a bridge. Apr 27, 2023 at 10:41

1 Answer 1

4

The real problem is that you're trying to send packets in the wrong direction.

Network interfaces naturally have two ends – e.g. an Ethernet interface is "eth0" on one end and a RJ45 jack on the other; packets that arrive on the RJ45 connector are released through eth0. And sending packets through eth0 has them leave through the RJ45 connector – they are not looped back through the same eth0.

The same applies to TUN/TAP interfaces, which are meant to connect to a program (e.g. some kind of a VPN tool) instead of a physical cable. Anything you send through the "test_eth0" or "test_eth1" interfaces does not go to the bridge, because that is already the side that is facing the bridge – instead the packets go through the interface, towards the program waiting on the other end (which, in your case, there isn't any.)

enter image description here

If packets were to arrive from the program connected to the TAP interface (or from the physical RJ45 port, if eth0 had been in the bridge), then they would indeed reach the bridge and would be forwarded to the other bridge-port.

So in other words, TAP interfaces cannot be used this way. You most likely want two veth-pairs instead – as those do come in pairs with their "cables" connected, any packets sent via veth0A would arrive at veth0B and enter the bridge. (See Tom Yan's comments for more details though, especially the part about network namespaces.)

appears that switch has enabled=no, but why?

Because STP (Spanning Tree Protocol) is in fact not enabled on the bridge.

STP is only a loop-avoidance protocol (and a mostly-obsolete version at that; modern networks would use RSTP) – it is not necessary for the bridge to function. Most "virtual-only" bridges actually keep STP disabled, especially as the obsolete version implemented by Linux uses very long learning timeouts (the 30-second delay before ports become usable).

It is more important here that the individual ports are shown as "disabled". They are disabled because you're trying to use 'tap' interfaces, and as it was already mentioned in the comments, a 'tap' interface is designed to be connected to some program that would handle the packets – without any handler, it is the equivalent of an Ethernet interface without a cable connected, i.e. not able to send packets anywhere, and therefore it will report "NO-CARRIER" in ip link.

3
  • Excellent answer! I was coming to conclude that ping (and tcpdump probably too) was not doing the thing I assumed it would do in my setup but couldn't figure out what was wrong exactly. I assume there probably exists other tools that I could use to test this setup that would attach to TAP interface (just like VPN application) from the other end and could send a sample packet the other direction and I will explore the veth approach too - though I still think TAP is what I actually want to explore here as I want to keep it low lev. I have now enough knowledge to continue exploring, big thanks!
    – Ski
    Apr 27, 2023 at 11:29
  • @Ski: Yes, you could also experiment with injecting packets through TAP interfaces specifically, but it won't be through "raw sockets" exactly – here's an example program that I've written for forwarding packets between a TAP interface and a remote UDP endpoint; instead of a raw socket, the program needs to open /dev/net/tun and write() the packet, then that packet pops out through tap0 (or test_eth0 as in your case). Apr 27, 2023 at 11:32
  • +1 for the excellent graph Jan 11 at 12:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .