Earlier this year I got a job that I'm barely qualified for (I made a post about it here, you guys were great) and inherited an environment that was . . . messy. I told them it was going to cost half a million dollars to fix, and they were like "K" and now I'm looking at some new core switches and a big honkin' firewall that I'm bringing online during a three-day outage starting tomorrow.
And fucking hell I'm just a jangly ball of nerves. I'm taking the entire company offline to do this cut-over/migration, everything is going hard down. Everything. My checklist is like a hundred items long, I've got a preemptive case open with Palo Alto, I've got detailed visio diagrams for exactly what gets plugged into where, the commands are already written and ready to be pasted into the switches, I've got the fiber run and the SFPs are in place (just waiting for me to push them in and no shut the interfaces), I've got some pro services on standby in case anything goes wrong.
But fuck fuck fuck man there's still so much I don't know. Like I spent most of yesterday re-watching the OSPF lab videos from my old CCNA course, and I'm pretty sure it all should work the way I think it will but maybe I missed some kind of really obscure command somewhere. There's a LOT of stuff going on in our edge firewalls (both set up by "that guy" three engineers ago who didn't document a single thing and even Palo Alto has looked at the configs and been like "wow . . . he's doing some weird stuff with the Policy Based Forwarding") that I've only halfway got a handle on. I had to spend like a solid week going through the configs menu by menu and every day I was finding something new that I was like "Oh, wow. Yeah that would have fucked me if I don't change that during the migration". Even today I was STILL finding things! Like the fact that our Service Route Configuration has the firewall getting updates through one of the other VLAN interfaces that I'm planning on removing (rather than the management interface like the usually do). Gotta remember to change that, add it to the list. Now there's 101 things on my checklist of things to do.
I don't know how I'll be able to sleep tonight. I've been obsessively planning every step of this process for the last three months and now it's just one day away and it's kinda like when you're a kid and Christmas is the next day? Like man, I just want to be hitting "go" on all this stuff already! But then it's tempered by the impending sense of dread that I'm going to reconfigure it all and get it all patched in and nothing's going to work and I'll be caught in a routing loop somewhere and I'll end up in the fetal position crying on the floor of the datacenter.
And this is only Phase 1! I get to do this again in May! :D
Fuck man this is gonna be awesome. I can't wait. But I'm so nervous.
No comments:
Post a Comment