The strength of your computer system(s) is only as strong as the weakest link.
Using SSH keys is a far stronger approach than username/password, but if username and password is still an option then that is your weakest link. If you were to disable password authentication on all the other computers, it would be marginally safer only due to the fact that your exposure is now limited to one computer that is vulnerable to a brute force password attack.
If you have a set of trusted computers (meaning you are the only person that has root access to this box), I would recommend installing your public key on each of these computers, and then using what's called "SSH Agent Forwarding" to allow your starting computer to SSH and authenticate with every computer along the way. This way, your starting computer will act as the master and provide credentials as necessary, unfortunately if you're using a computer that is 'new' you don't have this luxury.
Even if you aren't using Ubuntu, this is a good article to read: https://help.ubuntu.com/community/SSH/OpenSSH/Configuring#Disable_Password_Authentication
I haven't tried this myself, but 2-factor auth may be a good solution to your password/foreign computer use-case: https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-two-factor-authentication
At the very least I would recommend you to use a password manager (such as mSecure) and cryptographically secure random generated password of sizeable length.
Disclaimer: I'm not computer security expert, I've just picked up a thing or two over the years.