Inconsistent Network Traffic

I am suffering through one of the most confusing issues I've encountered since I became a sysadmin. I've a diploma in IP engineering, but have primarily done sysadmin over the last 20 years, so I'm rusty-ish but think I have a good foundation.

We recently put in a dedicated link between our head office & a remote site. Previously, communication was handled by a site-to-site VPN tunnel. Traffic was routed:

Head Office core switch - HO Firewall - VPN tunnel over internet - SiteA firewall - SiteA core switch

Now, traffic will be routed:

HO core switch - dedicated link - SiteA core switch

The HO core switch is an HP ProCurve 5412ZL, while the SiteA core switch is an HP Aruba 2920. SiteA originally had a 2910, but we came across this problem and decided to replace it with a 2920, in case it was the switch. (it wasn't)

When I change the routing table on the switches so that traffic is sent over the dedicated link, low level traffic like ping works in all situations, but higher level protocols like RDP and SMB are all over the place. For example, I can RDP from servers at HO to *some* SiteA servers, but not others, even if they're on the same subnet (or even hosted on the same hyper-v server). In addition, some protocols work to a server, but not other protocols: I can RDP to a Domain Controller we have out there, but replication to/from it fails.

Even ping, which is successful between every server I've tried, is a little odd: RTT will be 5-6ms, but every so often, I'll get a single RTT of 235-237ms.

We have this same setup (HO core switch - dedicated link - SiteB HP 2920 core switch) at another site in our organization, and there are no problems there. I'd like to think I've set everything up properly, but I'm willing to consider all options.

As mentioned, what really throws me is that something like RDP will work fine to Server1 at the DR site, but not Server2, even though they're on the same Hyper-V server and can both be pinged. How can I isolate a) what the problem is exactly and b) where it's occurring?