IPTV always buffers except on VPN (TCP not UDP)

Whenever I try to open IPTV streaming on any of my devices, and regardless of the protocol (TS or HLS) it always buffers like mad. I have an 150mbps internet currently.

I notice that when I’m on VPN it doesn’t buffer, although it will only do so when I connect with TCP and not UDP. This is the case with OpenVPN or OpenConnect. V2Ray also doesn’t buffer.

My question: Is it possible to not get buffering without VPN?

FYI I’m using Unifi Dream Machine as my router.

Thanks in advance!