Wednesday, August 1, 2018

Does anyone run pure BGP on core of network

First, the diagram.

Above is a quick high level view of the network at the enterprise I work at. Historically the entire enterprise was run using OSPF. When I showed up they were OSPF peering to provider L3 MPLS circuits on both sides to remote sites, everything in area 0. What wasn't OSPF'd was static routed. And everything else was layer 2. There was a huge L2 domain sprawling across 2 datacenters (they're only like a mile apart so we have a ton of dark fiber between them).

Our enterprise has grown quite a bit but the network never developed because of older engineers clinging to what they new. Until this year the core of the network was part of a L2 spanning ring loop that would occasionally roll and tank the entire enterprise.

We're just finishing up a project that's converting the 2 datacenters to logically separate eBGP CLOS fabrics using Arista ToRs. EVPN+VXLAN overlay for physical servers and NSX for the large virtual environment. We're starting to turn our attention to the rest of the network now.

This is where me and some of my colleagues have philosophical differences. We all agree that we want to run eBGP across all our circuits to our remote sites and allow them to run their own IGP on prem. This will allow us to better integrate acquired sites without a complete overhaul (something that happens often).

Where we differ on is how the core and campus parts of the network should integrate with everything else. We're split about 50/50.

I would like to see each functional portion of the network split off into it's own AS number and isolate IGP to that area. So on the above diagram each of the areas outlined in blue I would see as the border between BGP and whatever IGP is run in that area of the network. I would run the 2 core routers as 1 AS and then eBGP peer to each of the functional sections.

One of our colleagues in particular does not like this design. Coming from a service provider background he wants us to put everything on the above diagram in a single AS, running iBGP on top of OSPF with route reflectors.

The main argument I have for my design is fault isolation and organizational benefits. As well as reduced complexity (no RR, only 1 routing protocol). Mostly I'm applying the same concepts we used in the datacenter to the core of the network, using pure BGP as an IGP, separate ASs, etc..

His big argument is it's extremely standardized and he has never seen a network run eBGP internally.

Any thoughts on either? Has anyone seen a network using eBGP in this fashion or am I crazy? My colleague is really big on whitepapers but it's hard to find good whitepapers that show an enterprise. Lots of whitepapers showing campus networks, or datacenters, but not a lot that I can find that show the interconnection of them all and how to design a core.



No comments:

Post a Comment