Cisco NAT stops working

I have an ISR 4331 with latest IOS XE setup with some VRF lite, inner VRF routing, NATing, IPsec tunnels... Any ways.

The problem is with the NAT.

I have 2 NATs setup.

1 is outbound to provider A using a dynamic pool off a /25. This works fine

The second is outbound to provider B using the interface IP and NAT overload. This works for awhile and after a few hours the desktops that were connecting to resources behind Provider B stop working. If I try it on a desktop that wasn't using those resources it works when the others arent working.

The only way I've been able to fix it is to do a 'clear IP nat translation *'.

I can post config later.