Good afternoon all:
Here is the relevant infrastructure setup I'm using:
Modem--->Router-->Router (192.168.1.1)-->wireless bridge (ddwrt; static 192.168.1.254)-->Proxmox Server (static 192.168.1.101)
On this server, there are two virtual bridges - vmbr0 which is bridged to eth0 (the port used to connect the server to the wireless bridge) and vmbr1 - which is configured as blank.
On this proxmox server, I currently have three VMs - Centos7, debian stable box, and a pfsense-2.4.x install. The pfsense box is configured as follows:
vtnet0 = virtual nic attached to vmbr0 (bridged to eth0 which is uplink for entire server) vtnet1 = virtual nic attached to vmbr1 (no configuration for this vswitch vmbr1)
WAN = vtnet0 (virtual nic attached to the bridged connection to eth0 which connects upstream) WAN: IPv4/DHCP: (192.168.1.* address from upstream router's DHCP)
LAN = vtnet1 (virtual nic attached to virtual switch without configuration on proxmox host); LAN = IPv4 STATIC 172.16.0.1/12
I've gotten everything set up, and the CentOS and Debian boxes are pulling DHCP addresses from the 172.16.0.0/12 network just fine... However, when I look at the DHCP leases I notice that some of my roommates devices have actually pulled their DHCP from the pfsense box. As in I'm seeing "$(RoomMates)Iphone" showing up with a DHCP lease in the 172.16.0.0/12 network...
Their iphone is connecting to the main router AP, which should be assigning these clients an IP of 192.168.1.x. Instead, the iphone (as well as other clients, like desktops) are pulling an IP from 172.16.0.1 and connecting outwards from that. The funny thing is some clients get DHCP from the 192.168.1.1 router just fine - e.g. there are clients on both DHCP servers, and I'm trying to figure out how to keep all clients outside of VMs to get their DHCP lease from 192.168.1.0 network.
When I disable the pfsense box, the following happens: The CentOS and debian boxes (which have their vnics attached to vmbr1 - the unconfigured vswitch), pull an IP from 192.168.1.1... Since they are on vmbr1, they should not be able to reach that network as it's not bridged to eth0 (or any real NIC for that matter).
I have tried the following settings, to no avail: -block all RFC1918 traffic on WAN interface -same as above but for LAN interface -played with firewall settings for hours -Used wireshark on desktop attached to 18.104.22.168 network to analyze dhcp, ip.addr == $(172.16./12 IP of pfsense), and just generally sifting through what I see. Nothing.
Can not block all wan traffic to LAN interface, as I need the VMs on LAN interface network to access 192.16. network and internet beyond demarc.
Static 172.16. addressing for the vms works, but I'd like to have dhcp turned on as it's fun to play with and rather convenient for a frequently changing environment.
TL;DR: - Trying to create 172.16.0.0/12 network within Proxmox. Trying to seperate them so that if pfsense box goes down, no traffic occurs between vmbr0 and vmbr1. Also trying to ensure that clients from upstream (anything other than VMs from proxmox server) do not use pfsense box as their DHCP server (which shouldn't be happening, because again the traffic is coming through the WAN interface on the pfsense install...)
Please help, any hints are useful. More than happy to provide logs, output, whatever is needed. Been pulling my hair out over this all weekend and it's frustrating as I've made this setup work before in the homelab.