Tuesday, September 10, 2019

ASA Ipsec tunnel degrades on a schedule and it's driving me mad.

Hello,

We have an issue that has lingered on for about 2 weeks, where a site connected via an ASA IPSEC tunnel will stop passing about 50% of traffic after hours. The circuit is nowhere near full capacity when the issue starts. It's a 500Mbs DIA circuit - and after hours traffic might crack 5Mbs. The ASA won't process that much traffic anyway. The ISP on each side if verified as solid - no issues there. And like clockwork at 7:40am each morning - the issue with dropping traffic through the tunnel just goes away. Only to return about 12 hours later. Additionally it does not get fixed on weekends or holidays - when it goes bad on Friday, it stays that way until Monday (unless it's Labor day, it seems.) This same configuration has worked without issue for about 4 months. So this is a newfound problem.

I thought it was an environment issue (building HVAC shutting off after hours) but have verified temps on the equipment is well within the normal range. Cisco TAC spent 3 hours working on the issue and blames the ISP. ISP finds no issues (and I'm on their side - as I can verify that tunnel traffic is the only thing they ever seem to drop.)

I'm out of ideas. Anyone seen anything like this before? Is there rogue packet inspection in our path that only breaks things after hours?



No comments:

Post a Comment