Slow transfer speed to Cisco Devices (ISR 4331 & Cat 3850)

I am transferring new IOS images to my routers and switches and the download speed is really slow (between 80-130 kB). Sites are connected over DMVPN with most sites at 100Mb/s+ connections. I am using SCP to transfer the files.