FTP Logon Attempt Restriction exceeded, not restricting IP address

Windows Server 2012/IIS 8.0 has FTP logon attempts restrictions set to 5 with a time limit of 180 seconds and set to deny IP address.

We are being hit with dictionary attacks dozens of times within the span of 2 minutes, and IIS isn't blocking that IP address.

What should I be checking to troubleshoot this? It seems like it should be pretty straightforward.