Tuesday, October 19, 2021

Mystery double VLAN tag in VWLC on ESXi

Hello /r/Networking!

I'm setting up a learning lab and have an instance of VyOS working in an ESXi VM using a VGT trunk (VLAN 4095) and it's working fine. But I'm stuck trying to set up dynamic VLANs in Cisco's VWLC in another VM on the same hypervisor: data doesn't flow out properly from the management interface. Dynamic VLANs aside, I'm able to provision a WAP using CAPWAP configured via DHCP option 43 from VyOS; I can also log in using RADIUS and use the wifi connection. I suspect the issue stems from some VLAN tagging weirdness. Any and all help is appreciated. Thanks in advance :)

The setup

This lab is running on a laptop with a single NIC; VWLC seems to indicate the use of a dedicated vSwitch necessitating another NIC but a) this seems to be to avoid having promiscuous interfaces on the primary vSwitch (I'm not terribly concerned about security for this lab) and b) the issues I'm seeing don't seem to be related to this configuration. Please correct me if I'm wrong about this :)

The VWLC service port (172.28.8.65/27) is on a vSwitch access port (VLAN 31) and works just fine - I can initiate connections to/from the service port (ICMP, TCP). The VWLC management interface (172.28.8.98/25) is on a promiscuous vNIC on a VGT trunk and is using tagged VLAN 41. I can ping the management interface from another machine in VLAN 41 -- but the reverse is not true.

What I've tried

In addition to the connectivity tests above, I've done some packet dumps to try figure out what's going on. I used a VM on a VGT trunk on the same vSwitch to record these packet dumps.

These are ICMP captures are between the router's interface on VLAN 41 (172.28.8.97/25) and the VWLC's management interface.

Router pinging VWLC. Wireshark says that the frame has one 802.1Q chunk inside (the response from the VWLC does as well and it is received properly by the router).

0000 02 00 00 76 55 4e 00 0c 29 d1 a0 16 81 00 00 29 0010 08 00 45 00 00 54 29 98 40 00 40 01 a8 15 ac 1c 0020 08 61 ac 1c 08 62 08 00 8f c0 0f 38 00 01 ae 94 0030 6f 61 00 00 00 00 7b 3d 01 00 00 00 00 00 10 11 0040 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 0050 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 0060 32 33 34 35 36 37 

VWLC pinging router (via CLI). Wireshark says that the frame has two 802.1Q chunks inside. The router does not respond to this. Similarly, pinging other IP addresses outside of the management subnet are properly directed to the gateway, but with an extra 802.1Q encapsulation so they're ignored by VyOS. I have no idea where this extra 802.1Q tag is coming from. And it's the same as the management tag (VLAN 41). My understanding of VGT/VST/EST suggests that EST won't help here.

0000 00 0c 29 d1 a0 16 02 00 00 76 55 4e 81 00 00 29 0010 81 00 00 29 08 00 45 00 00 68 1d d1 00 00 40 01 0020 f3 64 ac 1c 08 62 ac 80 08 61 08 00 15 d5 d8 04 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0040 00 00 00 00 00 00 1c 1d 1e 1f 20 21 22 23 24 25 0050 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 0060 36 37 38 39 3a 3b 3c 3d 3e 3f 40 41 42 43 44 45 0070 46 47 48 49 4a 4b 4c 4d 4e 4f 50 51 52 53 

Edit: I forgot add: Packet capturing on the VWLC itself does not show the extra tag.

Oddly, the VWLC, when pinging other machines on the subnet, does not send out any ICMP packets, not even for broadcast.



No comments:

Post a Comment