Network diagrams: https://imgur.com/a/ZmZP2jN Switches 11,12,14,16 are Nexus 3000. Switch 3 is a stack of 3750's(to be decommissioned). We are adding a new DC2 for redundancy. I have a dark fiber between the two DC's using CWDM to get a a few 10GB channels(synchronous SAN replication). I have a connection at each site to our ISP's MPLS network that connects to all our remote sites. If i disable the Primary link in DC1 to the ISP MPLS i get about 50% packet loss. If i disable the port-channel on switch12 highlighted in Red, while the Primary link is shut, everything works fine. I have check all ethernet ports,links,port-channels and routing tables a couple of times. Their are no issues routing between the DC's or from servers connected to the Nk3's. No errors or anything meaningful in the logs. I have done isolation tests on the VPC between the DC's and don't seem to get any errors.
My guess is that switch12 is not forwarding(Routing traffic) traffic to switch11. I can't figure out why. i have looked at https://www.cisco.com/c/en/us/support/docs/ip/ip-routing/118997-technote-nexus-00.html and as far as i can tell it is a supported Topology.
Any ideas?
Switch configs(redacted and sorry about the poor formatting): [switch3]# config 15.0(2)SE10 interface GigabitEthernet2/0/47 description [ISP MPLS] no switchport ip address [ISP MPLS] 255.255.255.248 interface vlan [VPC routing vlan to switch 14/16] ip address [VPC routing vlan to switch 14/16]/24 interface Port-channel2 switchport trunk encapsulation dot1q switchport mode trunk interface TenGigabitEthernet1/0/1 description [switch14] switchport trunk encapsulation dot1q switchport mode trunk channel-group 2 mode active interface TenGigabitEthernet2/0/1 description [switch16] switchport trunk encapsulation dot1q switchport mode trunk channel-group 2 mode active router bgp 65010 redistribute connected redistribute static neighbor [ISP MPLS] remote-as [ISP AS] neighbor [ISP MPLS] transport path-mtu-discovery neighbor [ISP MPLS] version 4 neighbor [ISP MPLS] soft-reconfiguration inbound neighbor [router] remote-as [Our_Public_AS] neighbor [router] transport path-mtu-discovery neighbor [router] version 4 neighbor [router] soft-reconfiguration inbound neighbor [switch14] remote-as 65010 neighbor [switch14] transport path-mtu-discovery neighbor [switch14] version 4 neighbor [switch14] next-hop-self neighbor [switch14] soft-reconfiguration inbound neighbor [switch16] remote-as 65010 neighbor [switch16] transport path-mtu-discovery neighbor [switch16] version 4 neighbor [switch16] next-hop-self neighbor [switch16] soft-reconfiguration inbound [switch14]# config version 7.0(3)I7(1) interface vlan [dc - to dc routing vlan] ip address [dc - to dc routing vlan]/24 interface vlan [Peer-link routing vlan] ip address [Peer-link routing vlan]/24 feature vpc vpc domain 1 peer-keepalive destination [switch16] source [switch14] peer-gateway layer3 peer-router interface port-channel1 vpc peer-link interface port-channel26 vpc 26 interface Ethernet1/45 - 48 description [DC1 to DC 2 link] switchport mode trunk switchport trunk allowed vlan [group of vlans including bgp routing vlan] channel-group 26 mode active interface Ethernet1/49/1 - 4 description [peer-link] switchport mode trunk channel-group 1 mode active interface Ethernet1/51/1 - 4 description [peer-link] switchport mode trunk channel-group 1 mode active router bgp 65010 address-family ipv4 unicast redistribute direct route-map REDIST-ALL neighbor [switch12] remote-as 65043 address-family ipv4 unicast soft-reconfiguration inbound always neighbor [switch11] remote-as 65043 address-family ipv4 unicast soft-reconfiguration inbound always neighbor [switch3] remote-as 65010 address-family ipv4 unicast next-hop-self soft-reconfiguration inbound always neighbor [switch16] remote-as 65010 address-family ipv4 unicast next-hop-self soft-reconfiguration inbound always [switch16]# config version 7.0(3)I7(1) interface vlan [dc - to dc routing vlan] ip address [dc - to dc routing vlan]/24 interface vlan [Peer-link routing vlan] ip address [Peer-link routing vlan]/24 feature vpc vpc domain 1 role priority 1 peer-keepalive destination [switch14] source [switch16] peer-gateway layer3 peer-router interface port-channel1 vpc peer-link interface port-channel26 vpc 26 interface Ethernet1/45 - 48 description [DC1 to DC 2 link] switchport mode trunk switchport trunk allowed vlan [group of vlans including bgp routing vlan] channel-group 26 mode active interface Ethernet1/49/1 - 4 description [peer-link] switchport mode trunk channel-group 1 mode active interface Ethernet1/51/1 - 4 description [peer-link] switchport mode trunk channel-group 1 mode active router bgp 65010 neighbor [switch11] remote-as 65043 address-family ipv4 unicast soft-reconfiguration inbound always neighbor [switch12] remote-as 65043 address-family ipv4 unicast soft-reconfiguration inbound always neighbor [switch3] remote-as 65010 address-family ipv4 unicast next-hop-self soft-reconfiguration inbound always neighbor [switch14] remote-as 65010 address-family ipv4 unicast next-hop-self soft-reconfiguration inbound always [switch11]# config version 7.0(3)I7(3) interface Ethernet1/4 description [ISP MPLS] no switchport ip address [ISP MPLS]/30 interface vlan [dc - to dc routing vlan] ip address [dc - to dc routing vlan]/24 interface vlan [Peer-link routing vlan] ip address [Peer-link routing vlan]/24 feature vpc vpc domain 2 role priority 40 peer-keepalive destination [switch12] source [switch11] peer-gateway layer3 peer-router interface port-channel1 vpc peer-link interface port-channel26 vpc 26 interface Ethernet1/45 - 48 description [DC1 to DC 2 link] switchport mode trunk switchport trunk allowed vlan [group of vlans including bgp routing vlan] channel-group 26 mode active interface Ethernet1/49/1 - 4 description [peer-link] switchport mode trunk channel-group 1 mode active interface Ethernet1/51/1 - 4 description [peer-link] switchport mode trunk channel-group 1 mode active router bgp 65043 neighbor [switch12] remote-as 65043 address-family ipv4 unicast next-hop-self soft-reconfiguration inbound always neighbor [switch14] remote-as 65010 address-family ipv4 unicast soft-reconfiguration inbound always neighbor [switch16] remote-as 65010 address-family ipv4 unicast soft-reconfiguration inbound always neighbor [ISP MPLS] remote-as [ISP AS] address-family ipv4 unicast soft-reconfiguration inbound always [switch12]# config version 7.0(3)I7(3) interface vlan [dc - to dc routing vlan] ip address [dc - to dc routing vlan]/24 interface vlan [Peer-link routing vlan] ip address [Peer-link routing vlan]/24 feature vpc vpc domain 2 peer-keepalive destination [switch11] source [switch12] peer-gateway layer3 peer-router interface port-channel1 vpc peer-link interface port-channel26 vpc 26 interface Ethernet1/45 - 48 description [DC1 to DC 2 link] switchport mode trunk switchport trunk allowed vlan [group of vlans including bgp routing vlan] channel-group 26 mode active interface Ethernet1/49/1 - 4 description [peer-link] switchport mode trunk channel-group 1 mode active interface Ethernet1/51/1 - 4 description [peer-link] switchport mode trunk channel-group 1 mode active router bgp 65043 address-family ipv4 unicast neighbor [switch11] remote-as 65043 address-family ipv4 unicast next-hop-self soft-reconfiguration inbound always neighbor [switch14] remote-as 65010 address-family ipv4 unicast soft-reconfiguration inbound always neighbor [switch16] remote-as 65010 address-family ipv4 unicast soft-reconfiguration inbound always
No comments:
Post a Comment