Wednesday, September 18, 2019

Replacing 4 x 10 Gbps with 1 x 100 Gbps - how do you split up into "virtual" links, and guarantee bandwidth?

I'm setting up some small VM test labs, running Proxmox (KVM) and Ceph in a 3-node HA configuration.

Currently, we're using SFP+ for our ToR switches, running 10Gbase-LR.

Each VM node has four SFP+ ports:

  • 1 x 10Gbps for VM traffic
  • 2 x 10Gbps (LACP) for Ceph traffic
  • 1 x 10Gbps for Corosync (i.e. heartbeat for the HA).

I'm looking at upgrading the lab with new servers and switches to 100 Gbps (QSFP28) switches. Yaysies!

The ToR switches are likely to be either EdgeCore (running Cumulus) or Aristas.

However, the new servers we're using only offer a single 100 Gbps port...

Previously, we were splitting it across 4 x 10Gbps to avoid network contention etc.

If we put everything over a single 100 Gbps - is there some way we could divide up the 100 Gbps connection into say, three arbitrary "virtual" connections?

How would we do this?

And is there some QoS we could use (or similar feature) to guarantee bandwidth, even under saturation?



No comments:

Post a Comment