Expanding network to a new site and using their ISP for failover.

Hi,

to begin with: I originally posted this in r/HomeNetworking but didn't get a single reply. I guess this was too advanced for them. I'm usually the guy that answers posts there but this is even for me too new.

Short version:

I currently have 3 sites (connected via NanoStations), one EdgeRouter 4, and a bunch of VLANS. This ER4 is the single point for inter-VLAN-routing and internet access. I want to connect another Family and their business to our network so we can share internal traffic but their private/business traffic uses their ISP and not ours.

Miscellaneous:

I have a good knowledge about VLAN and firewall rules but only basic knowledge about NAT rules and no L3 and static routing knowledge. I fear that I need to use the latter for this project though. I am an IT-Student and worked a little bit as a 2nd level support for MS networks but got laid off until covid disappears. I have very good computer knowledge and networking is my hobby. My "home lab" is my running config though so I can only do stuff in the night but I'm kinda nocturnal anyway.

(just to be clear, we're the same family, but operate different businesses. We get along but don't want to share everything)I belong to Family 1, so if you find words like "us, me, our" then they refer to Family 1."their" is Family 2.

Long Version:

Here's a simplified network diagram of the network: https://imgur.com/a/rkgwr2aThere's also a description on the picture which I have written before I wrote this post. Some information may be new but I try to include everything in more details here. The diagram does not depict every single device. Only network devices and important ones for this project.

Current Situation:

• All Sites but "E" are connected as depicted.
• Everything runs over the EdgeRouter 4 on "Site A". "Site D" is owned by "Family 2" but uses our internet because they're right next to our property ("Site B") and don't need much bandwidth.
• "Family 2" also owns "Site E", which is a restaurant.
• The links always carry all VLANS because we have 9 different SSIDs for the VLANS (except for "VLAN 60" which is wired only) and all APs broadcast them.
• Network equipment is on "VLAN 10". I see it as a "management" VLAN. My devices are on "VLAN 50" and firewall rules allows them to access "VLAN 10".

Additional info:

• Family 1 has static IP addresses.
• Family 2 currently does not.
• Family 1 has 80 down, 15 up.
• Family 2 currently has 25 down, 10 up (but it varies a lot).

The Hardware:

• An EdgeRouter 4 is used as the router.
• The switches are currently from different vendors. I'm planning on buying some EdgeSwitches for better integration with UNMS. (As soon as I gather 10 devices for their hosted service)The one at "Site B" will most likely be replaced with one as we need a new switch for expanding "Site B" anyway and then we'll move the current Zyxel Switch one link downwards to the new building. I'm not certain whether it needs L3 features (EdgeSwitch 8). You guys tell me please!
• The Access Points are tp-link EAPs. The management controller is located at Site A.
• All the network equipment at "Site E" does not exist yet. They currently run on the ISP Modem/Wireless-Router Combo and a dumb switch for the cash register.

The goals:

• "Site E" still has all VLANS from Family 1 so Employees and us can roam around BUT traffic from "Family 2" VLANS (which do not exist yet) should use their ISP, including VLAN 70.Optionally: All traffic happening on their site goes through their ISP.
• "Site E" needs to operate even if the Link to "Site B" drops.(This might be an issue if they use the Pi-hole on "VLAN 10". I'm not certain what happens to the access points if they lose connection to the controller. I believe they would still work though.)If the link is down, the VLANS from Family 1 may go down as well. Ideally they'd use the Family 2 ISP, but not required. Guests however should stay online and I strongly believe that they require access to the EAP management controller for the captive portal. That might need a site-2-site VPN connection incase the link fails. (I hope that the link is stable though, so just keep it in mind, but don't see it as a requirement just yet)
• There's a guest network on all sites. As the sites are all on the same street, guests should be able to roam around between the sites. Almost the whole street is covered by our access points. Traffic happening at Site.
• I, on "Site A" and "VLAN 50", need to be able to manage the whole network and access all devices.

Last words:

If you believe I should clarify some points then please say so. I hope I haven't forgotten anything.

Mandatory: English is not my native language, you know how it works...

I will answer all questions. I know how hard it is to follow the description of a network that is not yours. If something is unclear, please let me know.

​

Thanks for reading this and hopefully I get some constructive ideas/solutions from you guys.