Help with "Slow" connection on SDWAN

So the company I work at is in the middle of migrating from MPLS to SDWAN (Viptela). Currently we have a large global group all migrated successfully with no issues mostly in a dual TLOC setup (1x mpls and 1x biz-internet per site). Starting with a few sites here in Australia, I'm setting up single biz-internet sites and migrating them completely away from MPLS (MPLS still in site as a fallback but not setup in SDWAN as a TLOC).

The control connections all work fine and the tunnels are up perfectly. All non-SDWAN traffic goes back to the closest datacenter (assigned using Centralized Policies) where it can then go to an MPLS site or out to the internet through a NGFW.

Latency is perfect, traceroutes and pings show sub 5ms to 8.8.8.8, RDP and other traffic is perfect, etc.

Here's the weird part. Some of the single TLOC sites have absolutely awful HTTP/HTTPS traffic. I'm talking 20+ Seconds of "Establishing Secure Connection" in Google Chrome before loading the website. Setting a static route pointing the traffic out the MPLS instead then returns the traffic to its lightning fast speed. This doesn't affect every site either. We have 3 sites - 2 of them are having issues, and 1 of them is totally fine. The site that is fine is using the exact same Viptela template as a site that's having issues.

Here's a crude diagram of the setup: https://i.imgur.com/558piBW.png

Wireshark shows a ton of TCP Retransmissions however there is no packet loss anywhere in the connection. The only thing I can blame is SDWAN as an almost identical traffic path through MPLS shows zero issues but I'm at a loss on how to troubleshoot and resolve it.

Where do I even start looking? I've stripped out so much of the template so that there's no shaping, no QoS, no sslproxy, etc, and the issue still occurs.

Any help would be appreciated.