Friday, February 26, 2021

A plea for suggestion. Ports dropping connectivity.

Hello everyone, I work with a library that is having very odd connectivity issues. Its not a complicated situation but its probably going to be hard to convey the overall issue. So here it goes :

Problem : We have 40 or so Dell Optiplex 9010's that are used for public access computers. 20 Dell Optiplex 90-somethings that are used for staff computers. "Not sure the exact model but what is important is that they are not 9010's" What we experience is that random public computers "and ONLY the public computers" will drop their ethernet connection randomly. So no link lights at all and the switches thinks nothing is plugged in. There is no consistency with what computer screws up or when. We will have no problems for a month and then randomly 2 computers will stop working. No consistency with what computers screw up it just seems to be luck of the draw.

Current fix : Re-seat cable at the computer - re-seat at the patch panel - re-seat at the switch - restart the computers - Administratively shut down the port and then bring it back up. One of these will generally fix it.

The complicated : We never had a issue with any of these computers until we did three things "unfortunately all at the same upgrade time" We upgraded the libraries switches from Cisco 2950 "the old 10/100 switches" to Cisco 3650 - We moved the network closet to a bigger room "located 100feet away from original closest" and extended the old closest to the new room with cat6a cabling.

The complicated 2 : Initially we focused on the cabling because the switches think nothing is wrong and re-seating the cables fixes the problem most of the time. The existing wiring is cat5e and although we do know the wire pathing ect.. we have no true idea of cable length on the existing wiring. Our guess is a average of 150ft and then we extended it another 100ft with cat6a to the new closet. We looked around a few days ago and saw massive service loops in the ceiling so it is a concern that we are cracking that 300ft mark but my gut/experience tells me that cables that are to long do not show this kind of problem and generally I will see negotiation problems on overly long runs.

The mind blow : So we are faced with no pattern on the computers - no pattern on the fix - and no pattern on time frame. However yesterday we had a computer that would not come up for anything. We swapped cables around with the computer next to it and the network connection comes right up, plug the other computer in to the dead line and nothing. Swap them around and the one cable works but the original does not. Jiggle all the cables, nothing. port shutdown and port enable, nothing. As we stared at a rack in confusion I grabbed a short cable from the table and plugged in a laptop to the inactive port at the switch and boom fires right up. WTF.... Unplug the laptop and plug the original cable back in a boom it works.... WHAT THE F!

The mind melt : So if its only the Optiplex 9010 "the staff computer have NEVER done this and we have a handful at greater run length than the public computers" then it has to be the network card on the pc's? But if the network card is mad then why does swapping cables fix it. If its something with the switch then why does it only happen to the 9010's? If its something with the cabling then why is it completely random? We get no errors on the switch as far as port stats go and we see no reconnections ect.. from the pc. Just here one moment and gone the next.

What kind of rubs me the wrong way is that the cisco switches report nothing at all wrong. No flapping, no errors, no port negotiation attemps, no STP attempts just f your cable there is nothing plugged in.

Sorry for the wall of text guys but I am just throwing crap at the wall and seeing what bounces back. This network is setup to the tits with redundancy and properly installed equipment. Everything is E-Rate funded so no expenses spared when installing the cabling. Grounding everywhere, grounded patch panels, shielded cabling , ground bars in the rack, cable trays in the ceiling.

Side note : we have even switched public computers to the second cisco switch in the stack and it still happened. We even replaced one of the switches with a hot spare and it still happened. IT HAS TO BE THE CABLING OR PC's but whaaaaaattt theeee....



No comments:

Post a Comment