Friday, February 14, 2020

Cisco took down my router, but how?

Hello All,

We had a Cisco TAC call yesterday in order to troubleshoot a VPN tunnel. Phase 1 comes up, but phase 2 never negotiates. So through a Webex I let their engineer work on the router and he wanted to create object groups to match the style of access lists on the other side of this tunnel. I didn't think it makes a difference, but I'm all for building test access lists to test this theory out, so that's what he was going to do. I sat on the call and watched him type and had a Putty log running. I was remote through a gateway to my desktop in the office. After he created a new test access list and modified the dynmap for this particular client to use the new access list, we lost all external connectivity. I was disconnected from my session and users in the office reported they no longer had internet access.
Since I was remote, I had to call a coworker to just reboot the entire device. Once it was back online I was able to get back in.

Below is the Putty log for changes that the Cisco engineer made. I am not very familiar with Cisco IOS, but I'm fairly certain creating a new ACL and calling that ACL in a dynmap should not affect anything but that dynmap. I'm hoping someone here can look at this output and tell me where we screwed up so I can avoid doing that in the future. We have another call scheduled with Cisco, but I'm a little nervous to let them make any changes until I understand why this went down.

Any thoughts are greatly appreciated! :-)

I have sanitized the log so that none of our IPs are in this log, nor any of the names we actually use. I also left spacing alone as it's off in a couple of places and I'm not sure if that is relevant.

ROUTER#conf t Enter configuration commands, one per line. End with CNTL/Z. ROUTER(config)#object-group network Vendor2Client ROUTER(config-network-group)#network-object host 1.1.1.151 ^ % Invalid input detected at '^' marker. ROUTER(config-network-group)#network-object host 1.1.1.152 ^ % Invalid input detected at '^' marker. ROUTER(config-network-group)#network-object host 1.1.1.153 ^ % Invalid input detected at '^' marker. ROUTER(config-network-group)#network-object host 1.1.1.148 ^ % Invalid input detected at '^' marker. ROUTER(config-network-group)#network-object host 1.1.1.149 ^ % Invalid input detected at '^' marker. ROUTER(config-network-group)#host host 1.1.1.151 ROUTER(config-network-group)# host 1.1.1.152 ROUTER(config-network-group)# host 1.1.1.153 ROUTER(config-network-group)# host 1.1.1.148 ROUTER(config-network-group)# host 1.1.1.149 ROUTER(config-network-group)#host 1.1.1.150 ROUTER(config-network-group)#exit ROUTER(config)#no object-group network Vendor2Client ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)#object-group network Client2Vendorobject-group network Client2Vendor ROUTER(config-network-group)#host 1.1.1.151 ROUTER(config-network-group)# host 1.1.1.152 ROUTER(config-network-group)# host 1.1.1.153 ROUTER(config-network-group)# host 1.1.1.148 ROUTER(config-network-group)# host 1.1.1.149 ROUTER(config-network-group)#host 1.1.1.150 ROUTER(config-network-group)# ROUTER(config-network-group)# ROUTER(config-network-group)# ROUTER(config-network-group)#exit ROUTER(config)# ROUTER(config)# ROUTER(config)#object-group network Vendor2Client ROUTER(config-network-group)#host 192.168.15.150 ROUTER(config-network-group)#host 192.168.15.149 ROUTER(config-network-group)# ROUTER(config-network-group)# ROUTER(config-network-group)# ROUTER(config-network-group)# ROUTER(config-network-group)# ROUTER(config-network-group)# ROUTER(config-network-group)#exit ROUTER(config)#access-list test_test extended permit icmp object-group Vendor2Client object-group Client2Vendor ^ % Invalid input detected at '^' marker. ROUTER(config)#ip access-list ? extended Extended Access List helper Access List acts on helper-address log-update Control access list log updates logging Control access list logging match-local-traffic Enable ACL matching for locally generated traffic persistent enable persistency across reload resequence Resequence Access List role-based Role-based Access List standard Standard Access List ROUTER(config)#ip access-list e x t ROUTER(config)#ip access-list ext e ROUTER(config)#ip access-list extended test_test ? <cr> ROUTER(config)#ip access-list extended test_test ROUTER(config-ext-nacl)#permit ? <0-255> An IP protocol number ahp Authentication Header Protocol eigrp Cisco's EIGRP routing protocol esp Encapsulation Security Payload gre Cisco's GRE tunneling icmp Internet Control Message Protocol igmp Internet Gateway Message Protocol ip Any Internet Protocol ipinip IP in IP tunneling nos KA9Q NOS compatible IP over IP tunneling object-group Service object group ospf OSPF routing protocol pcp Payload Compression Protocol pim Protocol Independent Multicast tcp Transmission Control Protocol udp User Datagram Protocol ROUTER(config-ext-nacl)#permit icmp ROUTER(config-ext-nacl)#permit icmp ? A.B.C.D Source address any Any source host host A single source host object-group Source network object group ROUTER(config-ext-nacl)#permit icmp obj ROUTER(config-ext-nacl)#permit icmp object-group Vendor2Client object-group Client2Vendor ROUTER(config-ext-nacl)# ROUTER(config-ext-nacl)# ROUTER(config-ext-nacl)#exit ROUTER(config)# ROUTER(config)# ROUTER(config)# ROUTER(config)#c crypto map dynmap 95 ipsec-isakmp ROUTER(config-crypto-map)#no match address VPN-CLIENT-ACL ROUTER(config-crypto-map)# ROUTER(config-crypto-map)#match add ROUTER(config-crypto-map)#match address test_test ROUTER(config-crypto-map)# ROUTER(config-crypto-map)# ROUTER(config-crypto-map)#exit ROUTER(config)#exit ROUTER#sh run | be 


No comments:

Post a Comment