Load Balancing ipsec tunnels between active/active firewalls in a cloud environment

Our current Azure setup is such that all incoming traffic hits a public load balancer first, and then gets forwarded to one of two palo alto firewalls in the backend pool.

This is great from a resiliency standpoint since its easily scalable and makes it easier to do maintenance with no downtime...in theory.

The problem:

Unfortunately, we also have about a dozen VPN tunnels that terminate on one of the backend firewalls. In my mind, this single point of failure makes the setup more fragile than it should be, since any downtime on this particular firewall will necessarily cause a service disruption.

My hypothesis:

To fix this, I was thinking of moving the VPN peer IP from the firewall and onto the public load balancer. That way, remote sites will be able to establish tunnels to any firewall in the backend pool instead of just this one

My question:

Does this design even make sense? Is this the best way to solve this problem?

It seems reasonable to me, but I haven't been able to find any documentation on this kind of design. Frankly, I don't know what I don't know.