Cisco 2960X (5) stack pinging intermittently - but devices connected to it are fine.

This one has me scratching my head.

I have a stack of 5 2960X 48-port switches in an IDF connected via 10gig fiber to a 3850 in the MDF. While there are 2 fiber connections, they aren't set up as etherchannel and one of the two is shutdown on the 3850 - don't ask me why, I didn't do this. I guess someone wanted a manual failover or something.

Not sure if related: Yesterday we upgraded our AT&T MPLS network to an AT&T AVPN network. We did the upgrade at about 5:30 PM, I left the building at about 7:15 PM. Basically I just swung the patch cord going from the old MPLS router to the new AVPN router which was programmed with the same IP. Everything worked fine. New router is physically connected via GigE to one of my 3850's in the MDF.

At just after 8PM we started getting "down" alerts from our PRTG monitor. I'm getting about 54% loss of packets when I do a ping stream to it - not alternating, but it will work for a few minutes, then it will stop for a few, then it will be sporadic, it's all over the place.

I tried rebooting the whole stack during the night, just for kicks. It seemed fine for a few minutes after and I went to bed, but then it started up again. Nothing is appearing in the log on the switch, I thought maybe I was dealing with an IP conflict but if so, it's not being logged. But I keep thinking there must be some sort of ARP issue here? Not sure. Last night I also tried shutting down the active fiber uplink to the IDF and no shutting the other. Everything came up but in the same state where I was losing about half my pings so I set it back the way it was.

All of the devices connected to the switch - AP's on the same VLAN, PC's and phones on other VLANs - are all fine. People are on their phones and the voice quality is perfect. But out of the blue I can't reliably ping the switch. And since PRTG keeps reporting it down, up, down, up, my management is asking why it's down or flapping and I keep having to explain that the users are fine.

Thoughts?

sh ver Cisco IOS Software, C2960X Software (C2960X-UNIVERSALK9-M), Version 15.2(4)E8, RELEASE SOFTWARE (fc3) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2019 by Cisco Systems, Inc. Compiled Fri 15-Mar-19 10:55 by prod_rel_team

ROM: Bootstrap program is C2960X boot loader BOOTLDR: C2960X Boot Loader (C2960X-HBOOT-M) Version 15.2(4r)E3, RELEASE SOFTWARE (fc4)

SHC-DAL-IDF-SW uptime is 10 hours, 17 minutes System returned to ROM by power-on System restarted at 15:54:32 CDT5 Wed Aug 11 2021 System image file is "flash:/c2960x-universalk9-mz.152-4.E8.bin" Last reload reason: Reload command

...

Switch Ports Model SW Version SW Image

---

• 1 52 WS-C2960X-48FPD-L 15.2(4)E8 C2960X-UNIVERSALK9-M
  2 52 WS-C2960X-48FPD-L 15.2(4)E8 C2960X-UNIVERSALK9-M
  3 52 WS-C2960X-48FPD-L 15.2(4)E8 C2960X-UNIVERSALK9-M
  4 52 WS-C2960X-48FPD-L 15.2(4)E8 C2960X-UNIVERSALK9-M
  5 52 WS-C2960X-48FPD-L 15.2(4)E8 C2960X-UNIVERSALK9-M