Monday, August 24, 2020

Unable to get link lights on Chelsio T520-SO using SFP+

Hi all,

I'm new to working with fiber so please bear with me.

I've got a Dell R520 that I've just installed a Chelsio T520-SO fiber adapter into. The server is running Ubuntu 18.04. I've got a compatible Chelsio SM10G-SR:10G transceiver on the adapter side and the switch (HP5406) which is using a HP short-range 10G transceiver (genuine j9150a). Between the transceivers, I have a 3-meter aqua fiber cable. I have another server with the exact same transceivers, T520-SO Chelsio card, and the same exact cable that the previous administrator installed, which is running great. I've even tried swapping everything between the existing server and the new server, but have been unable to establish an active link or get link lights while in Ubuntu.

Ubuntu sees the card and interfaces just fine. When I was in the Dell UEFI BIOS and viewed the network adapters, the switch link light lit up. Not sure if this meant the link was good or not.

I've installed the Chelsio Unified Wire drivers, configured link speeds to 10G, disabled auto-negotiation, and disabled FEC per documentation found here, I.3 and I.4 sections: https://service.chelsio.com/store2/T5/Unified%20Wire/Linux/ChelsioUwire-3.13.0.1/Chelsio-UnifiedWire-Linux-UserGuide.pdf

Here's some additional information:

:/lib/firmware/cxgb4# lshw -short | grep network /0/100/1/0 eno1 network NetXtreme II BCM5716 Gigabit Ethernet /0/100/1/0.1 eno2 network NetXtreme II BCM5716 Gigabit Ethernet /0/100/a/0 network T520-SO Unified Wire Ethernet Controller /0/100/a/0.1 network T520-SO Unified Wire Ethernet Controller /0/100/a/0.2 network T520-SO Unified Wire Ethernet Controller /0/100/a/0.3 network T520-SO Unified Wire Ethernet Controller /0/100/a/0.4 enp5s0f4 network T520-SO Unified Wire Ethernet Controller /1 bond0 network Ethernet interface /2 enp5s0f4d1 network Ethernet interface ethtool enp5s0f4d1 Settings for enp5s0f4d1: Supported ports: [ FIBRE ] Supported link modes: 1000baseT/Full 10000baseT/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: Yes Supported FEC modes: None Advertised link modes: 1000baseT/Full 10000baseT/Full Advertised pause frame use: Symmetric Advertised auto-negotiation: Yes Advertised FEC modes: None Link partner advertised link modes: Not reported Link partner advertised pause frame use: Symmetric Link partner advertised auto-negotiation: No Link partner advertised FEC modes: None Speed: Unknown! Duplex: Full Port: Other PHYAD: 255 Transceiver: internal Auto-negotiation: on Current message level: 0x000000ff (255) drv probe link timer ifdown ifup rx_err tx_err Link detected: no /etc/network/interfaces auto lo iface lo inet loopback auto enp5s0f4d1 iface enp5s0f4d1 inet static address 10.1.5.4 subnet 255.255.255.0 gateway 10.1.5.1 

Any clue where to go next? No network activity and it seems to assign it's IP just fine.



No comments:

Post a Comment