28

For some reason one of my ssh keys "just broke" - it just stopped working:

$ ssh-add ./id_rsa
Error loading key "./id_rsa": invalid format

Copying the key inside a clean VM, the key does work. Even with the exact same ssh version (OpenSSH_7.8p1, OpenSSL 1.1.0i-fips 14 Aug 2018 on Fedora 28). So it must be related to some config on my system I assume.

# cat ./id_rsa
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,...

...
-----END RSA PRIVATE KEY-----

Also peculiar: GNOME somehow manages to add the key on login with seahorse. Then ssh-add -L does list the key but it is not usable:

sign_and_send_pubkey: signing failed: agent refused operation
24

Traditionally OpenSSH used the same private key format is identical to the older PEM format used by OpenSSL. (Because it uses OpenSSL for parsing the key, it will accept the newer PKCS#8 format as well.)

So the issue can be one of:

  1. Your OpenSSL version refuses to load this key format. Perhaps it has accidentally enabled FIPS mode and refuses any algorithms except those part of its original FIPS validation?

    Try loading the key into the openssl command-line tool (which, yes, might also be linked to a different libcrypto, and you should check with ldd):

    openssl rsa -noout -text < id_rsa
    openssl pkey -noout -text < id_rsa
    

    Try converting it to PKCS#8 format:

    umask 077
    openssl pkey < id_rsa > id_rsa.pkcs8
    ssh-add id_rsa.pkcs8
    
  2. Your OpenSSH has been built without OpenSSL support. Even though ssh -V says the support was enabled, that does not automatically mean the ssh-add binary is the same – it might come from a different partial installation.

    Use type -a ssh and type -a ssh-add to compare installation locations.

    Once you know the path, use ldd /usr/bin/ssh-add to verify that it's linked to libcrypto.so (the OpenSSL cryptographic library).


If nothing works at all, try converting your key to the new OpenSSH-proprietary format using... PuTTY. Install the putty package for Fedora, and use:

puttygen id_rsa -o id_rsa.newformat -O private-openssh-new
ssh-add id_rsa.newformat

Also peculiar: GNOME somehow manages to add the key on login with seahorse.

Older GNOME Keyring versions have an internal copy of the SSH agent code and are independent from the system OpenSSH. So they will accept keys that your OpenSSH won't. (But on the other hand, this means severe lagging in terms of feature support (such as Ed25519 keys), and the latest GNOME Keyring just uses the system ssh-agent instead.)

8
  • 1
    Thanks for the lengthy answer! 1. opening the key with openssl does work: ``` openssl rsa -noout -text < id_rsa openssl pkey -noout -text < id_rsa ``` Also I have other ssh keys that have the same header and work fine. 2. converting to converting it to PKCS#8 format does work. I can use the key in PKCS#8 3. Then other keys wouldn't work, or wouldn't they? 4. Regarding GNOME, it is the current version of GNOME and it runs the OpenSSH agent (as confirmed by ps) --- So I do have a workaround, thanks! Still wondering WHY. What is wrong / what happened... Oct 29 '18 at 8:40
  • Same exact thing happened to me. No manual change regarding ssh (only culprit could be the command heroku keys:add but that should operate on remote; is the only command regarding keys that I recently run). Regenerating the key as pkcs8 works for me as well and restored my ability connectivity. The invalid-format id_rsa.pub last modify is dated 2018. – I quote you: Still wondering WHY. What is wrong / what happened... 🤷‍♂️ Jul 17 '19 at 15:39
  • Wondering if it might be related to virus infections; or to the SSD starting to die? It seriously worries me, to not know the cause. Jul 17 '19 at 15:40
  • 1
    This happened to me when I upgraded to OpenSSH 8 on Windows 10. I ended up using PuTTYgen to export the private key in the new format, and it started working again.
    – Matt Mills
    Mar 4 '20 at 17:54
  • use puttygen to export key . Menu>Conversions>openssh key and save it somewhere
    – Dr Deo
    Jun 7 '20 at 13:16
20

I was getting the same error message when passing in the private key through a CI pipeline variable in Gitlab.

The error was caused by not having a newline character at the end of the variable and was fixed by manually adding it.

10

In my case, the problem was caused by incorrect end of line characters in id_rsa file. After copying file content, Windows text editor wanted to help me and converted EOLs to CR LF.

4
  • 1
    Although tempting, don't use the clipboard. Copy the file using WinSCP.
    – woter324
    Feb 25 '20 at 23:55
  • 1
    @woter324, note that as of 2020 only 46% of developers use Windows
    – vladkras
    Jul 28 '21 at 7:53
  • After EOL Conversion, I also needed to append a new line to the end Dec 1 '21 at 12:32
  • @vladkras While I would personally prefer never to use Windows again, I am required to. Another way to look at this statistic is that as of 2020, nearly half of developers use Windows. That makes this a very valid response in my opinion.
    – slbass
    Dec 28 '21 at 15:30
2

I recently had this problem, and in my case it was due to having an invalid certificate (i.e. $HOME/.ssh/id_rsa-cert.pub), which confusingly gave this same error even though my private key was still valid and SSH continued to work.

It was fixed by either removing the invalid (in my case, zero-sized) cert file, or replacing it with a valid certificate, as the case may be.

3
  • So, how did you fix it? Apr 20 '20 at 5:35
  • 1
    It was fixed by either removing the invalid (in my case, zero-sized) cert file, or replacing it with a valid certificate, as the case may be. Apr 20 '20 at 11:42
  • 1
    This needs to be part of the answer post, to make it an actual answer rather than a statement or comment. Apr 20 '20 at 19:00
2

In my case, I just copied id_rsa private key but not the public part id_rsa.pub. It worked but complained with 'invalid format' each time I did server operations. Copying id_rsa.pub as well solved the problem.

1
  • This solution fixed for me
    – Skatox
    Jul 14 '20 at 4:03
2

In my case, on Windows, the solution was to use the Puttygen option Conversions > Export SSH key (force new file format)

0

Oh boy, just went through doing this for a team member... and the problem proved to be puttygen not doing things the nice way. I was finally able to fix it by using Windows Powershell and the command:

ssh-keygen -t rsa

it was an instant fix !

1
  • with this command you are creating a new key. The problem is that the current key is valid but the system trying to use complains about the format
    – Purefan
    Oct 26 '21 at 21:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.