In my lab, I have an OPNsense firewall, a Ubiquiti 10GbE Edgeswitch and a Ubiquiti 48x 1GbE Edgeswitch for networking. I have 4x ESXi hosts (Cisco UCS C-series) that I'm trying to get vLAN 200 configured for management and be able to have my other vLANs, as well as the management vLAN 200, set to be networks in vCenter (vLAN 50,70,80,90, 200, etc). I should note that anything connected to my APs that are in vLAN 110 or 120 work as designed, as well as vLANs configured for my virtual machines on my old hosts (70,80,90, etc.). The firewall rules for these vLANs are nearly identical to the vLAN 200 rule I have posted below.
Rundown of swith ports:
SW01 - int 0/3,0/4 = ESXi management and VM networks
SW01 - int 0/13 = Downlink to int 0/49 on SW02
SW01 - int 0/15,0/16 = Uplink to OPNsense firewall
SW02 - int 0/5 = IPMI/CIMC for OOB of the ESXi host (should only be vLAN 200)
SW02 - int 0/6 = ESXi management and VM networks standby
SW02 - int 0/49 = Uplink to SW01
From the ESXi management console (Let's say ESXI02 - 10.0.200.14) on vLAN 200, I can ping the gateway (10.0.200.1 as well as the LAN gateway, 10.0.0.1), can ping other devices on my LAN and my other vLANs (10.0.70.0/24, 10.0.80.0/24, etc), can resolve DNS, but I can't ping or access anything beyond my OPNsense firewall (WAN no workie).
Rules in OPNsense for vLAN 200:
Proto Source Port Destination Port Gateway Description * VLAN10_MGMT net * VLAN10_MGMT address * * Any host on this subnet CAN access the gateway (provides internet access) TCP/UDP VLAN10_MGMT net * Servers_DNS 53 (DNS) * Pass DNS to local pi-hole DNS servers ICMP VLAN10_MGMT net * * * * Allow Ping/ICMP * LAN address * VLAN10_MGMT net * * TEMPORARY - Allow LAN access * VLAN10_MGMT net * * * * Any host on this subnet CAN access anything (this last rule enables internet access).
VII-SW01 - 10GbE Edgeswitch Config (relevant info):
(VII-SW01) #show ru !Current Configuration: ! !System Description "EdgeSwitch 16-Port 10G, 1.8.2, Linux 3.6.5, 1.0.0.4872137" !System Software Version "1.8.2" !System Up Time "35 days 22 hrs 30 mins 33 secs" !Additional Packages QOS,IPv6 Management,Routing !Current SNTP Synchronized Time: Jun 24 19:52:50 2019 UTC ! hostname "VII-SW01" network protocol none network parms 10.0.0.101 255.255.248.0 10.0.0.1 no network ipv6 enable vlan database vlan 10,20,50,70,80,90,100,110,120,130,200 vlan name 10 "VLAN10_MGMT" vlan name 20 "VLAN20_Storage" vlan name 50 "VLAN50_Users" vlan name 70 "VLAN70_DL" vlan name 80 "VLAN80_WEB" vlan name 90 "VLAN90_RemoteAccess" vlan name 100 "VLAN100_Guest" vlan name 110 "VLAN110_Wiferino" vlan name 120 "VLAN120_IOT" vlan name 130 "VLAN130_CAMS" vlan name 200 "v200_MGMT" exit ip http session hard-timeout 168 ip http session soft-timeout 60 ip http secure-session soft-timeout 60 sshcon timeout 160 configure clock timezone -5 minutes 0 zone "CST" ip name server 10.0.1.14 10.0.1.15 logging email logging email from-addr XXXXXXXXXXXXXXXX logging email message-type urgent to-addr XXXXXXXXXXXXXXXX logging email message-type non-urgent to-addr XXXXXXXXXXXXXXXX mail-server "XXXXXXXXXXXXXXXX" port 465 security tlsv1 username XXXXXXXXXXXXXXXX password XXXXXXXXXXXXXXXX exit username "admin" password XXXXXXXXXXXXXXXX level 15 encrypted username "XXXXXXXXXXXXXXXX" password XXXXXXXXXXXXXXXX level 15 encrypted no username "ubnt" line console exit line telnet exit line ssh exit snmp-server sysname "VII-SW01" snmp-server location "Office Rack" ! snmp-server community "XXXXXXXXXXXXXXXX" ro switchport protected 0 name 'UNMS' interface 0/3 description 'VII-ESXI02' vlan participation exclude 1 vlan participation include 10,20,50,70,80,90,100,110,120,130,200 vlan tagging 10,20,50,70,80,90,100,110,120,130,200 exit interface 0/4 description 'VII-ESXI02' vlan participation exclude 1 vlan participation include 10,20,50,70,80,90,100,110,120,130,200 vlan tagging 10,20,50,70,80,90,100,110,120,130,200 exit interface 0/13 description 'Downlink to VII-SW02' vlan participation include 10,20,50,70,80,90,100,110,120,130,200 vlan tagging 10,20,50,70,80,90,100,110,120,130,200 exit interface 0/15 description 'Uplink to Firewall' switchport mode trunk vlan participation include 10,20,50,70,80,90,100,110,120,130,200 vlan tagging 10,20,50,70,80,90,100,110,120,130,200 exit interface 0/16 description 'Uplink to Firewall' switchport mode trunk vlan participation include 10,20,50,70,80,90,100,110,120,130,200 vlan tagging 10,20,50,70,80,90,100,110,120,130,200 exit
VII-SW02 - 1GbE Edgeswitch Config (relevant info):
(VII-SW02) #show ru !Current Configuration: ! !System Description "EdgeSwitch 48-Port Lite, 1.8.2, Linux 3.6.5-1b505fb7, 0.0.0.0000000" !System Software Version "1.8.2" !System Up Time "0 days 3 hrs 59 mins 29 secs" !Additional Packages QOS,IPv6 Management,Routing !Current SNTP Synchronized Time: Jun 24 19:58:00 2019 UTC ! hostname "VII-SW02" network protocol none network parms 10.0.0.102 255.255.248.0 10.0.0.1 no network ipv6 enable vlan database vlan 10,20,50,70,80,90,100,110,120,130,200 vlan name 10 "VLAN10_MGMT" vlan name 20 "VLAN20_Storage" vlan name 50 "VLAN50_Users" vlan name 70 "VLAN70_DL" vlan name 80 "VLAN80_WEB" vlan name 90 "VLAN90_RemoteAccess" vlan name 100 "VLAN100_Guest" vlan name 110 "VLAN110_Wiferino" vlan name 120 "VLAN120_IOT" vlan name 130 "VLAN130_CAMS" vlan name 200 "v200_MGMT" exit ip http session hard-timeout 168 ip http session soft-timeout 60 ip http secure-session soft-timeout 60 sshcon timeout 160 configure clock timezone -5 minutes 0 zone "CST" ip name server 10.0.1.14 10.0.1.15 logging email logging email from-addr XXXXXXXXXXXXXXXX logging email message-type urgent to-addr XXXXXXXXXXXXXXXX logging email message-type non-urgent to-addr XXXXXXXXXXXXXXXX mail-server "XXXXXXXXXXXXXXXX" port 465 security tlsv1 username XXXXXXXXXXXXXXXX password XXXXXXXXXXXXXXXX exit username "admin" password XXXXXXXXXXXXXXXX level 15 encrypted username "XXXXXXXXXXXXXXXX" password XXXXXXXXXXXXXXXX level 15 encrypted no username "ubnt" line console exit line telnet exit line ssh exit snmp-server sysname "VII-SW02" snmp-server location "Office Rack" ! snmp-server community "XXXXXXXXXXXXXXXX" ro interface 0/5 description 'VII-ESXI02-CIMC v200' vlan pvid 200 vlan participation exclude 1,10,20,50,70,80,90,100,110,120,130 vlan participation include 200 exit interface 0/6 description 'VII-ESXI02' vlan pvid 200 vlan participation exclude 1 vlan participation include 10,20,50,70,80,90,100,110,120,130,200 vlan tagging 10,20,50,70,80,90,100,110,120,130,200 exit interface 0/49 description 'Uplink to SW01' vlan participation include 10,20,50,70,80,90,100,110,120,130,200 vlan tagging 10,20,50,70,80,90,100,110,120,130,200 exit
What am I missing in my configs to get vLAN 200 to connect to the internet? And how do I ensure that the ESXi management and the hosted VMs and VM networks are able to connect and route appropriately?
No comments:
Post a Comment