There are a few things that could be at play:
On the "Server"
Have you verified that the public key is in the ~/.ssh/authorized_keys
and ~/.ssh/authorized_keys2
of the vagrant
user?
A simple way to add:
cat id_aribtratry_name.pub >> ~/.ssh/authorized_keys{,2}
(Shell expansion should do both, but may as well double check the files).
On the Host
Nothing above suggests to me that you used Puttygen to get the key for Putty, also you can configure putty to skip asking you the other things.
Using Puttygen
Open Puttygen: File > Load Private Key (in the pop-up dialog, you will need to select "All Files") to load your key.
Leave everything as it is and click "Save private key"
It will prompt you to confirm you don't want a password, I will leave that option up to you. Locally, not really a problem. But a server online - I would suggest it.
Save it wherever as id_whatever.ppk
Set up Putty (saves you time later)
Enter the IP and Port (As you have been doing)
On the left, navigate to Connection > SSH > Auth, find your private key file in the "browse" dialog.
Navigate back to Connection > Data, specify your Auto-login username
Finally, go back to Session, and you can specify a name to save the session under.
After all that, you should be able to login. That is, assuming the IP is correct. To avoid confusion, I personally would use the actual IP of the machine. But if the LO works through the other port, that is fine to use.