Saturday, January 27, 2018

How to isolate network throughput issue

I'm running into a network issue I've isolated away from systems to the core which consists of a N3k and a C2960s. iperf3 is reporting consistent 10G connectivity between both NICs on both systems in storage, however when transferring data It initiates at over 400MB/s and then trickles to 20M. I've tested this using a RAM disk to hosts connected to the 2960 and to the N3k, and get the same behavior, however the 2960 connection initiates at 100MB/s. Here's a quick screen capture of the behavior, this happens when copying disk to disk within the RAMDisk or flash storage only over network, local on the storage system is fast. https://streamable.com/yy7s3 The upper limit here even when reading/writing to RAM is telling me it's a network issue (CPU is idle during this test), but I don't have it in my vocabulary to describe what could happen.

If this helps, the connections related to the storage nodes are simply two 10G, with individual IP to N3K, 1 1Gb connected to the 2960S and the 2960S connects over LACP to the N3K. This issue happened before th 1Gb connections, they were added when upgrading the NXOS on the N3K recently during reboot.

Something tells me this is a loop, but I'm not sure where to start testing there. This issue happened weather the devices were in a bond0 or separate IPs . Any point in the right direction on a description, and I'll investigate further will be appreciated.



No comments:

Post a Comment