Wednesday, January 10, 2018

occasional latency spikes on cisco wireless network

This is a new network, run on a 5508 controller using sw version 8.3.133. APs are 3802i. There are currently 22 APs connected to this controller. There are only 2 SSIDs broadcasting 5ghz and 1 that is 2.4 only.

This network is at an assisted living center. We use some devices for cardiac monitoring of residents. The devices stream data to a monitoring station, basically showing vital signs, etc as a graph. Occasionally there is a small break in connectivity that causes gaps in the graph. This causes a problem that's too complicated to explain. I've traced the cause of the problem to a small latency spike on the wireless network. Typically we see 2-8ms latency, with the occasional spikes to 25-80ms. The devices use 802.11a, and when this happens they are generally connected at 54mb.

When this happens, all devices connected to a given AP see the spike at the same time. It doesn't happen on all APs at the same time. Sometimes it doesn't happen at all, sometimes the spikes occur every 5-7 seconds. The issue will happen on one AP for 10 minutes, and then another AP for 5 minutes, and then maybe back or off to another AP. Sometimes it's 2-3 APs at a time. RSSI at the device measured on the device and also with an android phone is -45 to -60db at the time this issue happens.

I'm trying to find any more advanced ways to troubleshoot exactly what is causing that latency spike on the AP - everything on the controller looks good, SNR is good, there are no obvious problems. At present the APs experiencing this only have 2-5 clients associated, most are idle or are streaming this graph data.

The APs are connected to a 3560x running older 12.2 software. Updating that is in the plans, but isn't going to be anytime soon due to downtime intolerance.

I see the latency spikes on A, AC, and AN connected devices. It's something happening on/to the AP iself periodically. I can see the latency spike when pinging the AP from the switch it's plugged into.

This problem does not occur when the same devices are connected to 1140 series or 2702 APs using older software (8.0 something) on a different 5508 controller using an identically configured SSID.

Does anyone have, or know of, any kind of advanced monitoring/troubleshooting tools for cisco APs? I realize that in the wireless world I'm splitting hairs here, as this performance would be considered excellent by most, but I think there's a specific cause for this issue, I just can't find it.



No comments:

Post a Comment