We've been having circuits drop for 20-30 seconds at a time ever since the Fastly incident the other day, and not limited to one region or one particular network. Circuits completely independent from each other region-wise, all the way from the Mid West to the West Coast.
Entire swaths of customer equipment has been going offline in some areas, generally one state at a time, and then coming back online after a few minutes.
It's like the entire Internet has a chaos monkey running through it, and we can't pinpoint it to one circuit, one area, or one region; with any rhyme or reason.
Then when everything seems normally, we start to see insane jitter running through the network, up to 5% packet loss, then back to normal. Of course, everything internal or on our own fabric is unaffected.
I haven't seen reports anywhere of this, but now I'm going through things with a fine-toothed comb; wondering where this chaos monkey is, and how it's affecting systems at random that have almost nothing to do with each other.
No comments:
Post a Comment