Long story short, our organization has a lot of coin invested in storage infrastructure. We also used to have a dedicated storage admin, who ran storage mostly end-to-end for years. A lot of training was invested in him, and a lot of resources have been spent in his infrastructure. Roughly $400K in two new storage arrays and OM3 fiber and new director switches to move to 16G FC from 4G. You'll never guess what happened -- he left. That migration was maybe 10% complete--literally out of 200 or so hosts, less than 20 have been migrated. And yes, the rest of those hosts are still sitting on ancient 4G FC connections to gear that hasn't been supported in years.
All of that, and he didn't document much of anything, and the group has just left both sets of infrastructure running without a thought about next steps forward, really. Surprised? Neither am I.
That 16G FC gear is now end of support apparently, and the sysadmin team has no plan forward. The manager of that team has approached my network team about a project to transition storage networking into our scope. I'm not very excited about inheriting their stinky mess, but our VM hosts running our internal monitoring systems are somewhat held hostage.
I was involved in a sidebar chat in the hallway, sounds like the scope they're looking for us to manage is just the storage networking infrastructure. They would still be responsible for the arrays themselves, and we would be tying in the hosts.
Our shop runs Nexus gear in the data center already right now anyways. Our infrastructure is about at max scaling that we can handle right now (just two 5548's with 12 FEX pairs). I've already spun up a few FC ports on a test 5548, plumbed some FC optics in, built a couple of VSANs -- seems fairly straight forward but I'm sure that's not all there is to it.
We would obviously need to scale way up on our Nexus infrastructure, and repurposing a lot of the wasted OM3 fiber plant would be a great way to do this. Most of the servers are already running converged NICs, so they could theoretically be ready for Ethernet and FCoE rather than requiring direct FC.
What would you do in my shoes? I can tell you now that we won't get extra staff for this. They've already closed the position vacated by the storage engineer as unfilled. I'm inclined to help, but I really don't want more fingers pointed at my group's direction when our SAs begin to blame us for BS storage latency issues. This group has--several times over the years--hurled crap at us for "slowness issues" for low transfer throughput from a 1G FEX to 15 year old HP DL380 servers with spinning disks still in production. This decision would be ~50% technical and ~50% political.
And in the end, who knows I may not have a choice in the matter anyway...
No comments:
Post a Comment