SSH to remote server doesn't work through Netgear Router as access point - why?

I have a Verizon router that has a wired connection though the walls to a room where I have a netgear router setup as an access point. HTTP traffic works fine wirelessly and wired through that netgear router. However, I can't ssh to a remote server that way. But if I connect to the wired connection directly through the Verizon router, I can SSH out.

When it doesn't work, I get this after about 30 seconds

packet_write_wait: Connection to <IP> port 22: Broken pipe

The admin settings don't allow much. I have ACL turned off which is the default. Is it happening because it's an access point? I want the verizon router to hand out the IP's which is why I have it setup like this.