I discovered the answer while writing the question.
If you do not have a public key file which corresponds to your private key (
id_rsa.pub in this example), then
ssh-keygen -l will calculate the public key from your private key, and use this calculated public key to calculate the public fingerprint, as expected.
However, if you DO have a public key file (e.g.
ssh-keygen -l will not bother to calculate the public key, and will instead use the public key file to get the fingerprint.
This is weird behavior, since all the data you could need is in the private key, and it doesn't take long to calculate the public key from the private one. It also doesn't check if the private and public keys actually match. However, this odd behavior is actually documented in the man page for
ssh-keygen. So, basically, yes, there is a cache: the cache is the public key file (e.g.
You can safely delete the public key file and regenerate it with
ssh-keygen -y -f id_rsa > id_rsa.pub.