Hi folks. I’m at the end of my rope with my lack of advanced knowledge surrounding routing, and I need some help. I have struggled lately grasping routing concepts that should probably be general knowledge to me by now; so needless to say, I’m feeling a bit defeated and frustrated. I’m a visual learner by default, and networking is often hard for me because “concepts” are explained well with diagrams and visuals, but the actual programming of switches/routers, etc. are often not explained with any helpful visuals. Anyways…
I’m trying to get a brand new Layer 3 (L3) network up and running. We are using a core stack of some Netgear M4300s. I have successfully configured the internal network for VLAN/subnet routing. I have not gone through and implemented any ACLs yet, so at present, all VLANs can successfully ping one another with no issues. Internal subnet routing is working just fine.
My problem is the uplink to our gateway. We have a bit of a unique situation (well maybe not), and I fear I am overthinking something, or missing something minor. (I literally spent 12 hours on this yesterday trying to figure out what I was doing wrong—so long that my arm physically hurts from being on the mouse and googling things all day.) We have an intermediate link that will be used to carry traffic over a buried fiber (private) line to our actual ISP equipment; we are using two identical L2/L3 Extreme Networks switches to carry the connection over the fiber (this will sound silly, but yes, that is literally all they are doing is creating an L2 “conduit” between our buildings; they are dedicated because they were funded by E-Rate and can be used for nothing else other than our internet connection transport). Our link looks a little like this:
[Internal Network] <---> [Netgear Layer 3 Switch] <---> [Extreme L2/L3 Switch] <---> [private fiber] <---> [Extreme L2/L3 Switch] <---> [ISP Equipment w/ Cloud Fortinet FW] <--->|<---> (public internet)
I have this set up in a test environment right now because our actual building this is being installed in (we are a rural educational cooperative) is not ready yet. I’m trying to be proactive so that this stuff is ready to go and can just be racked up when the network closets are done.
I do not have access to the Extreme switches, as those are being installed on-site by the vendor we purchased from (and the building isn’t ready for that yet), so in my test environment, I’m currently going from the Netgear L3 Switch to my physical pfSense box (firewall/gateway).
I am unable to successfully ping out from any internal subnets to the pfSense gateway or back into the internal subnets from the pfSense gateway. The ONLY thing I can ping is between the two ports on the respective devices (I can ping from the assigned uplink port on the L3 Netgear switch to the receiving port on the L2 Extreme switch—and vice versa).
I wish to “terminate” the VLANs in the Layer 3 Netgear switch; I have no desire to forward those VLANs to the Extreme switches for any VLAN routing there because the VLANs won’t be used on those switches at all (again, they are just connecting our fiber at both ends).
Here is what I have done so far:
- Enabled routing on the Netgear, globally (i.e. turned Layer 3 on; it is off by default).
- Created all needed VLANs--with VIPs (virtual IP addresses) that assign each VLAN to their own appropriate subnet, then enabled routing for each VLAN, and assigned my ip helper addresses for DHCP addressing (which is working fine, btw). VLAN10 (which is where the networking mgmt plane now resides) has an IP of 172.30.0.10/21 (this is the IP address for the switch). I will list the rest below VLANs below.
- Moved switches (mgmt IPs) off of native VLAN1 and onto VLAN10 for security reasons.
- Assigned an IP address to the physical port (on the Netgear switch) that is being used as the uplink port: 10.0.0.2/30. This port is 1/0/12. I have enabled routing for this port. It is set as a “general” port (not trunk since I don’t want to pass the VLANs to the upstream gateway). This port is in VLAN1, as the pfSense box cannot do a native VLAN change on its ports (it can see VLAN tags, but it can’t have its own port moved off of VLAN1). Since this port, along with all other VIPs are routing-enabled, I assume that this being in VLAN1 is no issue. (This would make sense since I can ping from an internal subnet—i.e. 172.30.XX.XX/2—to the physical uplink port of 10.0.0.2/30 with no issue.)
- Assigned an IP address to a free NIC interface/port on my pfSense box: 10.0.0.1/30. (I also created one firewall rule for this interface that allows ANY traffic from the Source Network (10.0.0.1/30) to ALL destinations (this is exactly how I have my homelab setup, as well, and I’ll explain that in a sidenote down below).
- Created a new default IP Route on the Netgear switch as follows: 0.0.0.0 (net address) 0.0.0.0 (netmask) 10.0.0.1 (next hop address). After this Route is created, this then shows up in the IP routes list with the additional column of “next hop interface” = 1/0/12.
- When creating the IP address on the physical port (IP 10.0.0.2/30 on interface 1/0/12) it automatically created the following IP route in the table: 10.0.0.0 (net address), 255.255.255.252 (netmask), 10.0.0.2 (next hop address), 1/0/12 (next hop interface)
- RIP is on by default; OSPF is off by default. I know very little about either of these or if I even need them for this setup, but it looked like I should have at least one protocol on, so I chose to leave the default RIP enabled. I’ve tried it with it disabled but saw no change that I could tell.
Based on everything I have tried to read up on the internet and in the Netgear M4300 guide, this should be all that is needed to make the upstream gateway active.
Now, I’m fully aware that there could be something odd with my pfSense box that may not be the case when this setup is moved to production with the Extreme switches, but the fact that I can’t ping out to the gateway from the routed internal network OR ping back in to the routed internal network from the gateway is troubling. I have to be missing something simple, right?
(Sidenote: on this pfSense box, I have another interface that I have in an exact same setup—different port IP address & subnet of course—to act as the upstream gateway for my homelab equip (homelab uses different subnets then what I’m using here, so no conflicts there) and it works just fine. I’m using a Layer 2 HP switch with one of its ports going to the upstream pfSense gateway, with the rest of my home lab network running on that HP switch; nothing in this setup above is using the HP switch, btw. Anyways, the point in me explaining all of this is that pfSense works fine as the upstream gateway for my homelab. I know that the pfSense box can be the upstream gateway and provide the internet connection to the downstream equipment. The only difference is that my homelab is not a routed VLAN network.)
Can anyone, just by reading this, pinpoint something stupid/minor that I forgot that is breaking this setup? Is there some kind of NAT that I need to setup on the pfSense? (And if so, can you explain it to me like I’m 5—ELI5—and also explain if (and how?) I’ll need to set that up on the production Extreme switches?
Here are the rest of my routed VLANs:
· VLAN10: 172.30.0.0/21 with VIP of 172.30.0.10 (networking management plane VLAN; yes, I’m aware this doesn’t match the “pattern” of my other subnets, but is valid nonetheless)
· VLAN20: 172.30.8.0/21 with VIP of 172.30.15.254 (“servers” VLAN; DHCP resides here that hands out addresses for all subnets)
· VLAN30: 172.30.16.0/21 with VIP of 172.30.23.254
· VLAN 40,50,60,70,100,200,300, and 500 follow the same exact patter as above for VLAN20 and 30.
I have been doing most of the setup in CLI for this (easier for bulk changes) and then using the web GUI to verify my settings along the way. Here are some shots of the Web GUI for the items I laid out above.
I feel like I’m in a black hole right now trying to figure this out. I appreciate anyone who takes the time to ready through all this crap to help me out. One of the reasons I have been so detailed here is the complete lack of information I could find online for this exact scenario. Maybe I was not searching for the correct things, but I’m hoping this post will get picked up by the ol’ search bots for someone in the future who finds themselves in this same situation.
Thanks again, all.
(P.S. – I leave for vacation on Sunday; I likely won’t be able to reply until I return, but I will come back to reply to any questions posed AND to update the fix for this when I do get it figured out so that this thread has a SOLUTION for future reference.)
X-Post with /r/NETGEAR
No comments:
Post a Comment