Saturday, June 29, 2019

Cisco Unified Communications Upgrade Woes

I know I'm probably preaching to the choir here when it comes to this, but why can't Cisco UC upgrades just be straightforward? We are going through an upgrade of everything UC related (UCCX, CUC, CUCM, IM&P, CER). This weekend I had a maintenance window to upgrade our UCCX pair and although the new software successfully installed on the inactive partition, the switch version failed due to database issues. At this point, my maintenance window is getting close to ending so I do not have much time to troubleshoot. As a result, I decide to stop and wait for another maintenance window. Fast forward a few hours later and I'm getting calls saying that our help desk customers are sitting on hold indefinitely even though there are agents available. Upon investigating, something occurred when I rolled back the upgrade that caused some sort of internal system issue (and mind you, I rolled back following Cisco's documentation to the letter). I debugged my scripts and all was working as intended. It seemed like UCCX was having issues maintaining state information for agents and CSQs. Although the fix for this was simply a reboot of the primary and secondary nodes, it is still infuriating that something as simple as a direct software upgrade and backing out of said upgrade is anything but straightforward and almost always involves TAC since none of us really have access to the internals of these applications. I mean, I had to get TAC to obtain root access once just so they can run a script as a workaround so I could get a secondary node to properly install.

My coworker who manages our ISE deployment has also had his share of problems when it comes to performing upgrades on ISE.

/endrant

EDIT: Sorry if my post seems like a mess, still haven't gotten any sleep.



No comments:

Post a Comment