Thursday, April 26, 2018

Mellanox Connectx3 40gbe - Can't get past 20gbps from Windows VM to Windows VM

Scenario: Two Windows Server 2012 R2 virtual machines, one on each of 2 Esxi hosts. Hosts are directly attached to each other via connectx3 infiniband cables (card models and cables confirmed to be compatible via mellanox) I have spent hours with Vmware & Mellanox support. EsxiHost1 to Esxhost2, via SSH, DOES achieve 40gbs using iperf directly in the CLI of the esxihost, so I know the drivers should be good. I have tried the following:

  • I have tried PCI passthrough, allowing me to achieve 20gbps

  • When using VMXnet3 i can't push past 17 or 18gps

  • I have tried many small tweaks in windows including Jumbo packets/TCP offload/etc etc under the "configure" section of the network adapter

Help me achieve 40gbps throughput (please)!



No comments:

Post a Comment