Monday, December 21, 2020

Why did this cause a broadcast storm (presumably)?

Juniper shop. Two buildings, connected by primarily dark fiber and wireless point-to-point (PTP) backup. Switch ports connected to PTP access points are normally disabled when dark fiber is operational. Dark fiber went down so we enabled PTP ports. Dark fiber came back before we were ready and this essentially brought our network to its knees for a few minutes. Logs show that storm control was in effect on the PTP port on Switch1. I believe the broadcast storm then caused OSPF to be unable to reach neighbors and caused a failover.

I realize this isn't an ideal design for several reasons, but mainly just trying to understand what happened here. My guess is that the difference in link speed made it so the PTP link couldn't keep up with all of the broadcasts, but I never saw storm control in effect on the other switches (though I could have missed it). Ideally we would just have the Juniper switch monitor the dark fiber interfaces and automatically bring up the PTP ports when fiber was down, but this requires additional licensing on these models.

Diagram



No comments:

Post a Comment