I recently constructed a small Beowulf cluster out of some old computers all running Linux Ubuntu 12.04 LTS and have been following the instructions on this page:


So far all has been going well as far as nfs and pinging the other nodes but now I can't get ssh to work without a password. I spent many hours reading other pages on people with similar problem yet none of the things that helped them seem to help me (changing to 600 mode, editing sshd_configure file, etc).

eventually I just moved on to the rest of the steps hoping I might just be able to enter the password at the execution of the task and that does work if it's only one computer running the task (because it asks for mpiuser@node1's password and I can just enter it in) but as soon as I try running the task (the cpi program mentioned later in that document for testing the cluster) on more that one node it (not to surprisingly) asks for the password for both nodes, which is the same, and I enter it but then it just freezes, I think it has something to do with how it asks (it just lists them: password for mpiuser@node1: password for mpiuser@node2: password for mpiuser@node3: etc.)

I've even tried uninstalling and re-installing ssh and nothing happened, the ssh-keygen command seems to work fine (it says everything everyone else's supposedly says) but then it doesn't do anything. I was able to get ssh to work with a password because I found that ssh's default password is the root password so I just changed that but otherwise the ssh-keygen command doesn't seem to do anything.

Any ideas?


Upon typing

"ssh -v mpiuser@node1" it says:

      master@master:~$ su mpiuser


      mpiuser@master:/home/master$ cd

      mpiuser@master:~$ ssh -v mpiuser@node1

    OpenSSH_5.9p1 Debian-5ubuntu1.1, OpenSSL 1.0.1 14 Mar 2012

    debug1: Reading configuration data /etc/ssh/ssh_config

    debug1: /etc/ssh/ssh_config line 19: Applying options for *

    debug1: Connecting to node1 [] port 22.

    debug1: Connection established.

    debug1: identity file /home/mpiuser/.ssh/id_rsa type 1

    debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048

    debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048

    debug1: identity file /home/mpiuser/.ssh/id_rsa-cert type -1

    debug1: identity file /home/mpiuser/.ssh/id_dsa type -1

    debug1: identity file /home/mpiuser/.ssh/id_dsa-cert type -1

    debug1: identity file /home/mpiuser/.ssh/id_ecdsa type -1

    debug1: identity file /home/mpiuser/.ssh/id_ecdsa-cert type -1

    debug1: Remote protocol version 2.0, remote software version OpenSSH_5.9p1 Debian-5ubuntu1.1

    debug1: match: OpenSSH_5.9p1 Debian-5ubuntu1.1 pat OpenSSH*

    debug1: Enabling compatibility mode for protocol 2.0

    debug1: Local version string SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1.1

    debug1: SSH2_MSG_KEXINIT sent

    debug1: SSH2_MSG_KEXINIT received

    debug1: kex: server->client aes128-ctr hmac-md5 none

    debug1: kex: client->server aes128-ctr hmac-md5 none

    debug1: sending SSH2_MSG_KEX_ECDH_INIT

    debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

    debug1: Server host key: ECDSA 54:0e:8b:52:4d:41:08:fe:0b:bc:95:e5:93:42:59:40

    debug1: Host 'node1' is known and matches the ECDSA host key.

    debug1: Found key in /home/mpiuser/.ssh/known_hosts:1

    debug1: ssh_ecdsa_verify: signature correct

    debug1: SSH2_MSG_NEWKEYS sent

    debug1: expecting SSH2_MSG_NEWKEYS

    debug1: SSH2_MSG_NEWKEYS received

    debug1: Roaming not allowed by server

    debug1: SSH2_MSG_SERVICE_REQUEST sent

    debug1: SSH2_MSG_SERVICE_ACCEPT received

    debug1: Authentications that can continue: publickey,password

    debug1: Next authentication method: publickey

    debug1: Offering RSA public key: /home/mpiuser/.ssh/id_rsa

    debug1: Authentications that can continue: publickey,password

    debug1: Trying private key: /home/mpiuser/.ssh/id_dsa

    debug1: Trying private key: /home/mpiuser/.ssh/id_ecdsa

    debug1: Next authentication method: password

    mpiuser@node1's password: 
  • Are you using public key or password authentication? Are you running as user "mpiuser"? Do you have ssh-agent running, loaded w/ a valid key, and accessible? Use "ssh -v mpiuser@node1" to see what your client is seeing. – jeberle Nov 30 '13 at 18:36
  • I'm not sure what you mean by public key or password authentication, when I do the ssh-keygen command it says now creating public/private key if that answers your question. I am running as user "mpiuser" as it said to do so in that document. For now ssh seems to be running fine with the default (root) password as a valid key but as I mentioned that only works for one computer at a time. When I type in "ssh -v mpiuser@node1" it says the following (see next post) – user3052738 Nov 30 '13 at 18:42
  • there, I added it to the post – user3052738 Nov 30 '13 at 18:45
  • you need to append your public ssh key to the authorized_keys files of the server – hek2mgl Nov 30 '13 at 18:54
  • i tryed the ssh-copy-id localhost command, is it something different from that that I have to do? – user3052738 Nov 30 '13 at 18:56

Well, these lines:

debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/mpiuser/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/mpiuser/.ssh/id_dsa
debug1: Trying private key: /home/mpiuser/.ssh/id_ecdsa
debug1: Next authentication method: password

tell you that the server allows both publickey and password based logins, so the client offers up 3 public keys (id_rsa, id_dsa, and id_ecdsa) and the server is rejecting all three, so it falls back to password authentication.

If you expect that one of these keys should work, then you need to look at the server and see why it is rejecting them. Look at the authorized_keys file on the server and make sure it has the correct public keys corresponding to the private keys on the client, and make sure it has the correct permissions -- must be owned by the user you're logging in as (mpiuser?) and not writable by anyone else.

| improve this answer | |
  • All the keys files should have 600 permissions, and at least one of them should definitely be working, I'll trying doing the ssh-keygen a couple more times on some of those other files but so far it still asks for a password. The /home/mpiuser/.ssh file is shared among all the nodes so just doing the keygen on the master to these files should be working. I double checked, all the files are still being shared (I added a new text file and it properly showed up on all the nodes – user3052738 Nov 30 '13 at 19:41
  • I just re-did the keygen for all 3 of the files you just said it checked and still it's reverting to the password, the ssh -v mpiuser@node1 still looks the same – user3052738 Nov 30 '13 at 19:43
  • Assuming you ran ssh-copy-id after ssh-keygen to add the client keys to the authorized_keys on the server, you need to look at the server logs to see why it's rejecting them - set LogLevel DEBUG in /etc/sshd_config, restart the ssh server (make sure to undo that afterwards). Logs should turn up in /var/log/ somewhere - typically auth or daemon or secure. – Gabe Dec 2 '13 at 8:27
  • I didn't find an sshd_config file anywhere in the etc folder – Eric Dec 3 '13 at 0:50

Use the ssh-copy-id command to copy the pub key to the destination server. Delete the .ssh directory in that server too. Looks like a permissions issue for the keys, or the .ssh directory. Let the ssh-copy-id to do the task for you and it will work.

You can have a look at the command, it's a simple script.

| improve this answer | |
  • Here, maybe this will help because I'm new to all this and am not very good with terminal; I'm sharing the /home/mpiuser folder on all the nodes through nfs and that all works fine as far as I can tell. Right now there are 4 nodes and one master. If I'm on the master logged in as mpiuser and trying to log into one of the nodes through ssh (say node1) what would be the input for destination server be and where should I delete the .ssh directory since everything is shared? I would really appreciate it if you could just write the command I would put into terminal. – Eric Dec 1 '13 at 2:36

I finally figured it out! Turns out ssh had a permission problem with the entire '/home/mpiuser' folder and therefore wouldn't work for any of the nodes. I just edited "StrictModes" to "no" in the "sshd_config" file and now everything works.

Thank you everyone for all your help!

| improve this answer | |

I've got the same issue. Check your files and directories permission on both client and remote

 chmod 700 /home/user
 chmod 700 ~/.ssh
 chmod 600 ~/.ssh/authorized_keys
 chmod 600 ~/.ssh/config
 chmod 600 ~/.ssh/privatekey
 chmod 644 ~/.ssh/publickey.pub

It works for me.

| improve this answer | |

I have same problem as the user. The issue is caused by permission setting indeed. Before my entire '/home/xxx' folder permission is drwxrwxrwx (777). Right after I changed permission to drwx------ (700). It works! Notes: assume you append public key to authorized_keys and "sshd_config" file are correct.

Let me know if you have any issues. :)

| improve this answer | |
  • A HUGE thanks for this, I looked for a solution for two hours before finding this one. – ClementNerma May 27 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.