I'm running into a bit of a conundrum with a new 3-node Nutanix cluster I'm setting up. I've got a vCenter installed on the hosts with a dvSwitch plugging into a pair of Cisco C6509Es in a VSS, 1 NIC per host to each physical chassis. Each interface and the port-channel are configured as trunks allowing my 2 management VLANs and the DMZ VLANs. My original issue cropped up while setting up a LAG in vCenter and the LACP port-channel for one host on the physical switch. After moving both the host's NICs to the LAG, traffic would flow up until both interfaces were up and established in the port-channel. Turn one interface off, everything flows. Have only one interface in the port-channel, it's all happy. Have both interfaces up and in the port-channel, management traffic gets dropped and I lose access to the ESX host. Same holds true for flipping which port you're playing with. At this point in time, there were only Nutanix management servers and the vCenter server on the cluster, all of which are in the same VLAN/portgroup as the ESX hosts vmkernels.
While waiting to hear from VMware for their take on this, I decided to migrate a test machine to a different host in the new cluster to test the other VLANs/portgroups. It's at this point that I find what I suppose is the VRF-centric issue, and may also be causing the LAG/LACP issue. For my test VM, if it's in one of my DMZ VLANs, all of which are associated with a DMZ VRF on the physical switch, traffic is all fine and dandy. If I move them into management VLANs (the same vlan as the Nutanix CVMs and vCenter, and one for backups) which uses the default route table on the physical switch, no traffic is passed.
Since VRFs are assigned to the VLANs, having VLANs using the default route table and the VRF shouldn't be an issue, right?
Relevant physical switch configs:
PrimColo_SW01#show etherchannel summ Flags: D - down P - bundled in port-channel I - stand-alone s - suspended H - Hot-standby (LACP only) R - Layer3 S - Layer2 U - in use N - not in use, no aggregation f - failed to allocate aggregator
M - not in use, no aggregation due to minimum links not met m - not in use, port not aggregated due to minimum links not met u - unsuitable for bundling d - default port w - waiting to be aggregated Number of channel-groups in use: 8 Number of aggregators: 8 Group Port-channel Protocol Ports ------+-------------+-----------+----------------------------------------------- 13 Po13(SU) LACP Gi1/4/14(P) Gi2/4/14(P) PrimColo_SW01#show etherchannel port-channel [...] Group: 13 ---------- Port-channels in the group: ---------------------- Port-channel: Po13 (Primary Aggregator) ------------ Age of the Port-channel = 16d:19h:05m:24s Logical slot/port = 46/8 Number of ports = 2 HotStandBy port = null Port state = Port-channel Ag-Inuse Protocol = LACP Fast-switchover = disabled Load share deferral = disabled Ports in the Port-channel: Index Load Port EC state No of bits ------+------+------------+------------------+----------- 1 FF Gi1/4/14 Active 8 0 FF Gi2/4/14 Active 8 Time since last port bundled: 0d:00h:01m:07s Gi1/4/14 Time since last port Un-bundled: 0d:16h:23m:52s Gi1/4/14 PrimColo_SW01#show lacp neigh Flags: S - Device is requesting Slow LACPDUs F - Device is requesting Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode [...] Channel group 13 neighbors Partner's information: Partner Partner LACP Partner Partner Partner Partner Partner Port Flags State Port Priority Admin Key Oper Key Port Number Port State Gi1/4/14 SA bndl 255 0x0 0x9 0x8001 0x3D Gi2/4/14 SA bndl 255 0x0 0x9 0x8000 0x3D PrimColo_SW01#show etherchannel load-balance EtherChannel Load-Balancing Configuration: src-dst-ip vlan included mpls label-ip EtherChannel Load-Balancing Addresses Used Per-Protocol: Non-IP: Source XOR Destination MAC address IPv4: Source XOR Destination IP address IPv6: Source XOR Destination IP address MPLS: Label or IP interface Port-channel13 description *** NTX-ESX-DMZ03 *** switchport switchport trunk encapsulation dot1q switchport trunk allowed vlan 130,145,600,701,750,760,770,800,810,820 switchport mode trunk speed 1000 duplex full spanning-tree portfast edge trunk end
No comments:
Post a Comment