VPN behind ISP modems

I know this may be a stupid question but is it possible to VPN site to site when both firewall is behind the modem provided by the ISP?

What I have is one site has a fiber line (100/20) that has an UBNT USG Pro and the other site has a DSL (10/2.5). I was thinking linking them via a VPN tunnel. My understanding of it is that those firewalls must have a public address on one of their interface. Unfortunately, my ISP modems for both sites face the wan and my firewalls have 192.168.1.0/24 and 192.168.2.0/ addresses. Behind my firewall, my local network starts at 172.16.x.x/23 on both sites.

My question is, is it possible for VPN when both firewall don't have public facing address? So I have to enable port forwarding?

Thanks!