Friday, July 31, 2020

C9500 Switch Architecture, ASICs, and TCAM MAC capacity

Hey Gang,

I've been away from switch route for a long time and heavily focused on WiFi for most of my career. I have an implementation which relies heavily on centralized forwarding where all the client MACs end up getting dumped into a single switch port, or port channel.

A fellow network engineer tipped me off that the specs on a datasheet for most switches regarding the mac table size, is usually a little more convoluted than what they're telling you. If a switch has a table size of 56K, that may be further split up and restricted to certain groups of ports. So I've gone down the rabit hole to understand a little more about switch/router architecture.

https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9500-series-switches/nb-06-cat9500-ser-data-sheet-cte-en.html

Looking at the C9500 Datasheet, "Table 11. ASIC template descriptions" i see that based on which template I apply I can get either 82K or 32K of capacity for MAC table entries. That looks pretty straight forward to me, but my friend's tip got me digging a little deeper.

https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9500-series-switches/white-paper-c11-741484.html#Switchdesign

Looking at the "Figure 3. C9500-32QC/48Y4C/24Y4C board layout" I see 4 ASICs total, appearing to be responsible for different groups of ports below them.

https://www.ciscolive.com/c/dam/r/ciscolive/emea/docs/2018/pdf/BRKARC-2035.pdf

Scouring around I found a random slideshow with a hint that the UADP 2.0 XL has a 54K TCAM table size. So this must mean the numbers above are sliced and diced (L2, L3, Sec?) then added up to make up an aggregate of some sort, but I haven't found anything that clearly indicates the MAC table size for specific groups of ports associate to each of these ASICs in the switch.

Why am I digging into this so deep? 32K is a lot of MAC addresses, but 32k / 4 ASICS = 8K which will likely be a problem for my deployment. Definitely not today with COVID isolation measures, but definitely some day when things normalize.



No comments:

Post a Comment