Friday, June 1, 2018

Weird networking behavior with a gateway between a switch and CMC module, on Dell VRTX equipment. Am I doing something wrong?

Diagram:

https://tinyurl.com/ycb4dxgb


Here's the simplest way I know how to explain how the internal networking of this Dell VRTX works.

So, you have your 4 blades and your network switch module that have integrated remote management controllers, for the blades it's iDRAC and for the switch it's an internal OOB port that I can't actually physically plug into anything else.

According to Dell documentation these ports communicate with the CMC module of the enclosure. So technically the CMC is like a really dumb switch in some sense.

On my RT-N66U, I created 2 separate VLANs to segment the management interfaces and the rest of the interfaces of the switch and gave said interfaces 10.1.0.0 /24 addresses for management and a simple 169.254.255.1-2 /30 address set just to bridge the router and the switch module.

In the diagram I have the management interfaces set on a the range from .2 to .7, with a default gateway of 10.1.0.1. These all work fine.

However, the weird behavior is when I try to do static routes on the switch module.

The switch module supports layer 3 static routing. One primary issue is that in the Dell VRTX CMC, to access the web GUI I can set the IP for the OOB interface on the switch module. In this instance, it would show as 10.1.0.3 with a default gateway of 10.1.0.1. The odd thing is that for the switch, it sets the "ip default-gateway" property to this value and all of traffic ends up defaulting to this route which I don't necessarily want.

What i've tried:

  • I remove the default-gateway and I add my own static routes. Problem is, 10.1.0.3 should technically be a directly connected interface so I can't really route back to 10.1.0.0 with any gateway that makes sense.
  • If I do a default route of 0.0.0.0 0.0.0.0 10.1.0.1 which is the equivalent of a default gateway AND I add my other static routes for my VMs to get back out to my other networks, I am unable to access the web interface of the switch module for some reason. So here's a quick sample of what some configs look like and the behavior:

Able to access web GUI but cannot ping/access devices in 10.2.0.0 subnet


show run ... ip default-gateway 10.1.0.1 ... show ip route ... S 0.0.0.0/0 [1/1] via 10.1.0.1, 00:01:15, oob C 10.1.0.0/24 is directly connected, oob C 10.2.0.0/24 is directly connected, vlan 3 C 169.254.255.0/30 is directly connected, gi0/1 ... 

Add in a static route, lose access to web GUI but can ping/access devices in 10.2.0.0 subnet


show run ... ip default-gateway 10.1.0.1 ip route 192.168.1.0 /24 169.254.255.1 ... show ip route ... S 0.0.0.0/0 [1/1] via 10.1.0.1, 00:04:32, oob C 10.1.0.0/24 is directly connected, oob C 10.2.0.0/24 is directly connected, vlan 3 C 169.254.255.0/30 is directly connected, gi0/1 S 192.168.1.0/24 [1/1] via 169.254.255.1, 00:01:32, gi0/1 ... 

Lastly, if I remove the ip default-gateway and add my own static routes, the behavior is essentially the same as the first one above.

This embedded / internal port sharing setup is weird to me and I'm just wondering if I'm missing something stupid simple here. I've done a similar setup on a Cisco 3560E and Quanta LB6M and had no issues at all.

Thanks.



No comments:

Post a Comment