How to Buy time during an Outage (Corporate Politics and you Pt.1)

A few friends have encouraged me to post stuff like this as they found it helpful. Most posts are are of a technical nature, but the hardest part of being a lead NE isn't technical.

Communication, Budgeting, planning equipment refreshes, outage management, project management are key skills to being a lead NE.

I'd like to contribute what I believe are key phrases and concepts to survive as an NE in any environment. While I worked at VARs and an ISP, most of my XP is at an enterprise. So that is who this is targeted at.

How to buy time during a Network Outage;

People outright panic during outages as we know. I've been cursed at by CIOs, PMs, my own Manager when these have occurred. What you have to convey during this time is a calm, "just the facts" demeanor that acknowledges the issue and their concerns while notifying them you're working on it.

Example phrases to use when this happens.

• "We're aware of the issue and still investigating the root cause, updates every 45 minutes"
• "We're aware of the issue and working with the vendor to resolve it"
• "We're still investigating the issue and have no ETA but will keep you updated."
• "We've identified the issue and are working to resolve it as soon as possible, multiple vendors are engaged."

If you don't at minimum acknowledge the issue and their feelings things will go downhill, fast.**, it makes it harder on everyone. It shouldn't work like this, but it does usually.**

Once the outage is over

Control the narrative (or it'll be controlled for you).

In more toxic environment, they're looking for a vendor, person, or ISP to blame/fire. In others they want to understand root cause to prevent it in the future. This is actually an opportunity sometimes.

Assuming the outage wasn't fatfinger related.

Explain, and build relationships

You can tie this back to your yearly architecture reviews and budget requests. "X failed because my funding was cut for Y refresh", "We requested an outage window to update this devices old code, but none was given". If you can tie the technical failure back to your challenges with your business it can give them motivation to listen to you and fund you appropriately. If they don't care, that's pretty much a red flag and you need to bounce. Companies like that usually end up relying on VARs tbh.

As part of your yearly budget requests, and architecture reviews you need to identify all of the areas where equipment is under-designed, needs redundancy or refresh. If they don't fund your requests, and you have no options, you need to convey the risk in a respectful manner. This is key post outage.

Lastly,

Don't take it personally, people get mean when the network goes down. Especially at Hospitals and Financial Institutions. Do a Headspace, take some CBD, and don't let the VPs, Middle Managers, and Yes-men, push all the blame on you and leave you out to dry. That's assuming you follow process around changes, communicate and set expectations around outages and listen to your peers.