Little bit of background: The company I am at is growing rapidly, and they acknowledge that there is a large amount of technical debt in all areas that has or will soon start severely limiting the growth potential. An example of this would be that a majority of the routing infrastructure is held together with static routes, and there are about 1,600 routes defined in our core equipment. It's probably not that bad, but for the size we are, it just feels bad. So company realizes that this is going to be a problem and is investing heavily into modernizing. We've hired a large amount of new staff which is good, but also causes an issue because none of the current staff have time to train them on the environment. Current staff are basically rushing from fire to fire and no one has time to take the time to improve things.
That's where my problems start coming in. I have been heavily investing in knowledge and am transitioning to a higher level network role. I'm taking CCIE level courses, ACI, DNAC training, etc, basically anything that could help us. The company now wants me to start implementing changes that would better the business, somewhat carte blanche. That's great and exciting, until the realization sets in that no one else can manage any of this. This is made more problematic because of the odd set of knowledge our team has.
There are 3 levels of engineer: 1) Can perform basic L2 troubleshooting, understands what a static route is, and that is pretty much it. 2) More knowledge, understands portfast should probably only be on edge ports. They know what BGP does, but only configure it based on templates of currently working configs. Can also setup VPNs but don't understand what all the checkboxes in ASDM are used for. Can also configure VPC on a Nexus. 3) Knows BGP well, muddles through EIGRP and OSPF but don't really touch it because we don't use it. Knows how to twist some advanced knobs in L2 and make L3 perform well.
My problem is that I'm supposed to train everyone level 2 and up how to troubleshoot whatever I implement for a bus plan. For example, say I'm the only one that knows how to setup ACI, and understand the underlying technologies to be able to confidently dig into it. After presenting it, the company decides it is necessary to fulfill their DC strategy. I've put say 80 hours into researching the material, a couple hundred into labbing, plus knowing all the background knowledge necessary to know how it all fits together. This is problematic when I have to run every engineer through the equivalent of a CCNP bootcamp to even start to scratch the surface of what we're doing.
My question is how can I reasonably accomplish this goal? I know documentation will be a large part of it, but the documentation is worthless for L2s when things start to even mildly deviate from the checklist. Our on-call rotates through all levels of staff, and how am I supposed to help the L1 engineers even know where to begin to identify a problem without having to immediately call another engineer because they understand no part of what is going on? The company wants to move forward but the things I am supposed to implement look a lot like this to the rest of our team. We could bring in VARs, but then that gets things even more black boxed for staff. Where do I even begin with this?
No comments:
Post a Comment