Is there any way to do this without PBR?

Let’s say we have our own ASN, and two public /24’s of our own to advertise out. For the sake of example we’ll just use 1.0.0.0/24 and 2.0.0.0/24.

At our data center we’ll connect to two different ISP’s using our ASN, and advertise both subnets to both ISP’s.

I have been asked to configure the network in order to meet the following requirements.

• Any company owned device that our firewall team NAT’s to a 1.0.0.0/24 address must use Circuit A for all Internet access. Both sending and return traffic from remote endpoints must use Circuit A only.

• Any company owned device that our firewall team NAT’s to a 2.0.0.0/24 address must use Circuit B for all Internet access. Both sending and return traffic from remote endpoints must use Circuit B only.

• In the event that Circuit A or B fails, traffic will fail over to the other circuit and continue to work. So if Circuit A fails, then 1.0.0.0/24 will fail over to Circuit B and share that link with the 2.0.0.0/24 traffic. And vice versa

At first glance I told leadership “I can do this,” and set about designing the bgp stuff with the two providers.

By using AS-PATH prepending, I can in theory make 1.0.0.0/24 preferred to the outside world on Circuit A. I can do the same on Circuit B for the other subnet.

If the prepending is allowed to stand, it should accomplish what Circuit a particular subnet routes to from the outside world, and still allow it to fail over to the other subnet in the event of a circuit outage.

Perfect. Mission accomplished.

There’s only one... little detail I left out. Egress traffic from our network to the Internet.

How can I force traffic arriving at the edge from the firewalls with a source ip in 1.0.0.0/24 out Circuit A and traffic arriving at the edge from the firewalls with a source ip in 2.0.0.0/24 out Circuit B?

Uh oh.

I know one can control egress traffic with tools like LOCAL-PREF, or WEIGHT. But those are destination-based tools. You pick a subnet being advertised to your AS, and assign it a higher LOCAL-PREF coming from either ISP A or B, and your AS will send outbound traffic destined to that subnet to the peer with the higher LOCAL-PREF for example.

There’s only one problem. The endpoints sourcing traffic from either 1.0.0.0/24 and 2.0.0.0/24 all talk to the exact same IP’s out there, almost 1 for 1. So it’s utterly impossible to do any kind of destination based routing to send their egress traffic out the correct circuit.

The only method I can even fathom with my small imagination is a PBR route-map on the interface from the firewall, doing source address based forwarding with verify reachability.

This leaves a very sour taste in my mouth. Because PBR is only meant to solve temporary problems, not to be implemented permenantly in production. This PBR would also touch every packet leaving our network, so performance is a key concern.

How can I accomplish the three specific requirements above without using PBR? There simply must be a way!