Does a switch need to be synced with an NTP server in order to maintain SSH?

I read without an NTP server configured, NTP won't operate correctly and won't read the RSA keys. Why is this? Can I set the time locally and still be able to SSH?