FMCv/FTD code upgrade experience / rant

Figured this might be useful as an FYI for some people.

1x FMCv

2x standalone FTDs, one per DC (no h/a)

Went from 6.2.3 to 6.4.0, upgraded FMCv first (no probs), reloaded, then did FTD. FTD's both had come up with no management interface access. FORTUNATELY we were smart enough to have oob terminal servers that gave us console access, saving us a long drive. TAC had no idea what happened, so I did some t-shooting on my own while TAC was on the line and figured out I could hit the mgmtif from within it's own network, so obviously routing was broken.

Anyways, I removed the routing config in place (TAC was fumbling over commands trying to do this), and replaced it with the proper routing config via "expert" mode, or the linux shell, and wallah, I had mgmt access and FMCv saw the FTD. Prior to me identifying the issue, TAC said that waiting 2 hours for an FTD upgrade was "normal behavior" and that "if the console had output, we just needed to wait". *smh* Also, it wasn't a fluke, this happened on both FTDs.

We also noticed that if you don't run a deployment after upgrading FMCv, you will run into a fatal error during the FTD upgrade at around 11% and you will have to kick the upgrade off again. ICMP inspection also disabled itself during the upgrade which made me think I was blackholing traffic after BGP stood up and dropped the default route into our core.

Overall a terrible experience with terrible support. This platform requires you to basically learn about how shit it is then work around it. I guess this is the future for us, so we may as well get used to it :(

​

.:Edit:.

Before I catch flak for going to a ".0" version, you have to before you can jump to the latest hotfix. I tried a jump directly to 6.4.0.7 and you get prompted to upgrade to 6.4.0 first.