we are talking about RSA keys - you don’t memorize your RSA keys
if you rely on memorizing all your passwords, I assume that means you have ample password reuse, which is a million times worse than using a different less-secure password on every site