Wednesday, February 21, 2018

Looking for a Juniper Edge Router suggestion

Hello fellow redditors,

Currently I'm shopping for an Edge Router for a small DC. This router needs mainly to:

  • Receive 2 full BGP feeds (v4 and v6), eventually we'll need to receive a third one.
  • Process around 4 Gbps concurrent traffic (in + out), roughly 250k pps
  • Provide visibility through netflow/sflow
  • Connect back to 5 iBGP peers and get around 100 additional local routes
  • OSPF single area
  • Connect via eBGP to a scrubbing center via GRE tunnels (no encryption)
  • QoS for out critical traffic mainly (not for controlling usage by tenants)
  • Around 3 VRFs
  • 2 - 4 1GE ports
  • 2 - 4 10GE ports

For fun a small history of why I'm looking for this

Basically we're a Cisco + MikroTik shop, life was all good with our setup (roughly 6000 VMs distributed over 150 servers) 24/7 no issues at all for the past 2 years, we use Cisco at the Core and Access layer and MikroTiks at the Edge layer.

Well we had 2 events that are making us change the MikroTiks at the Edge, basically we were DDoSed (not the first time, but the first time the DDoS crippled our routers), this attack was a Low BW one (less than 250 mbps) and low PPS (less than 200k pps), the attack made our routers go to 100% CPU usage making them behave really bad, since those are software based routers the CPU spiking like this locks them almost 100%. The attack itself wasn't targeted directly to the routers but to one of our servers.

They didn't reboot and we managed to get them back online with help from the upstreams but we don't want this to happen again.

We were victims in the past of 6 Gbps/500k pps DDoS attacks and the routers didn't sweat at all (CCR1036 for those who may want to know), so at first got confused as to why this "smallish" attack made the router go like this, but upon further investigation we found out that the router's network process goes crazy if every single packet of those 200k pps comes with a randomized source or destination port, as in, every packet is treated as a different new connection.

We knew this was a possibility if we made the routers use a lot of services, but we basically deployed these with:

  • Conntrack disabled
  • Only firewall rules to protect access to it
  • Only newflow operational (which I know will stress the router a lot if such situation but we need it like this)

We don't hate those routers, we actually love those, 2 years working non-stop with no issues at all getting full BGP feeds from v4/v6 and using netflow, surviving "normal" DDoS, etc. But guess it's just a normal limitation of software based routers...

So here I'm, currently looking for a router that's hardware (ASIC) based.

End of the history

Anyway we could go with Cisco's ASR1000X line for this, but we'd like to try and test Juniper, I've toyed with these before and I do LOVE the CLI but I have to admit I am a bit confused about their MX line, that seems to be the one that fits us, they pack the datasheets for all the line with not all the information I'd like to have.

Any Juniper operator here that could provider some suggestion as to what hardware should we take a look into?

Thank you very much in advance.



No comments:

Post a Comment