So after a 17 hour day of dealing with this I thought perhaps this group might find some value in it. Would have saved it for rant wednesday but I'll still be drinking by then.
Background: We had to transition very quickly to work from home like everyone. Which meant lots of VPNs (like everyone) However maybe unlike a lot of people we have gear that cannot run Global Protect so instead we were using the XAUTH feature of the GP Gateways to run these
Problem #1 - Turns out there is a 2k limit on XAUTH. Have a 7k series whose spec sheet says it can handle 60k vpn clients. Nope 2k. But it lumps XAUTH in with the other vpns on the KB article... Nope 2k.
Problem #2 - PaloAlto has decided for reasons I won't fathom to not disclose a critical bug related to XAUTH and all PanOs 9.0 releases. PAN-150646. The way the tech described it:
3rd party vpn clients can't connect with error exceed max-user most likely because inactivity TTL causing stale ike-sa and device capacity is reached. It's caused by the IKE-SA's not being torn down when the timeout occurs. Eventually the box reaches maximum
Course since it wasn't listed among known issues maybe you did an upgrade to 9.0.x a week ago and because it takes time to build up to that maximum the upgrade went fine. The next couple days were fine... then a week after upgrade your phone is blowing up with ticket escalations about critical gear not being able to connect to VPN with zero helpful error logs and your best solution is to downgrade back to 8.1.x and pull out the whiskey.
Good luck all...
No comments:
Post a Comment