27

There is a very similar question that if answered may have been the answer to this question. Unfortunately it's a case of "no need to answer the question I asked because the problem wasn't what I thought it was."

The Setup

  1. The server bastion.ec2 accepts an ssh connection from my workstation via ssh -i mykey.pem [email protected]
  2. The server service1.ec2 accepts ssh connections only from bastion.ec2 via ssh -i sharedkey.pem [email protected]

The Requirements

  1. Both keys are only on my workstation so I can't actually do the 2nd command without copying the key over
  2. For security reasons I want to use ssh-agent forwarding rather than copy ssh keys over to bastion.ec2

The Solution

This is where you come in. How can I forward a different key for the 2nd connection?

If shareduser had mykey.pub in it's ~/.ssh/authorized_keys this would work:

ssh -i mykey.pem [email protected] ssh [email protected]

However, I don't want every user to have to put their public key in every server.

2 Answers 2

32

Step 1

Make sure your local agent is ready

Just because you can ssh into your bastion server without specifying your key path or getting prompted for the password, doesn't mean that your ssh agent is running and holding your key. Some modern OSes (eg: OSX) handle this for you.

On your local machine

$ ssh-add -L
ssh-rsa ObahfCbvagGbLbhSbeHfvatEBG13== ~/.ssh/mykey.pem
ssh-rsa LbhNerWhfgJnnlGbbPyrireEBG13== ~/.ssh/sharedkey.pem

fig.1

That means your agent is running and has your key.

$ ssh-add -L
The agent has no identities.

fig.2

That means you haven't added any keys to your agent. Fix that with:

ssh-add ~/.ssh/mykey.pem ~/.ssh/sharedkey.pem

fig.3

Step 2

Make sure your remote agent is ready

SSH into your bastion server and repeat the check from fig.1 & fig.2. However, the error you are more likely to get is this:

$ ssh-add -L
Could not open a connection to your authentication agent.

fig.4

That most likely means that your SSH client is not forwarding your authentication agent connection.

You can force this with the -A flag (as long as the sshd config on the server allows it, which is the default).

$ ssh -A bastion.ec2

fig.5

Step 3

Make sure you're using the right keys

If you have added keys to your agent, your agent is forwarding, and your remote agent lists your local keys. There is only two likely reasons you are not getting a connection. Either you are not using the right key or you are not using the right username.

Output the public counterpart to your private key:

$ cd
$ cd .ssh
$ ssh-keygen -y -f mykey.pem
ssh-rsa ObahfCbvagGbLbhSbeHfvatEBG13
$ ssh-keygen -y -f sharedkey.pem
ssh-rsa LbhNerWhfgJnnlGbbPyrireEBG13

fig.6

These should be the same as what you were seeing from ssh-add -L up to the == near the end.

Now one way or another you have to get into the box you are failing to connect to and look at the content of the $HOME/.ssh/authorized_keys file for the user you are trying to connect as. You need make sure the public key you output with the command above is in that file on a line by itself. You can't trust that the sharedkey.pub that the Bro 2 cubes emailed you is right. Verify! This may require getting someone else who can SSH in as that user to get you the authorized_keys file or getting root access. If you've come this far and it still isn't working, you are beyond taking shortcuts.

Step 4

Make it easy

Hopefully the steps above have gotten you in. Now let's make this headache go away for as long as you are using this workstation.

Configure your local ssh-client

Host *
    # A lot of people put an IdentityFile line in this Host * section.
    # Don't do that unless you will use only 1 key everywhere forever.
    #IdentityFile id_rsa

Host bastion.ec2
    # You want to make sure you always forward your agent to this host.
    # But don't forward to untrusted hosts. So don't put it in Host *
    ForwardAgent yes
    # Go a head and put the IP here in case DNS ever fails you.
    # Comment it out if you want. Having it recorded is a good backup.
    HostName 172.31.0.1
    # You don't want to create a proxy loop later, so be explicit here.
    ProxyCommand none
    # SSH should try using all keys in your .ssh folder, but if you
    # know you want this key, being explicit speeds authentication.
    IdentityFile ~/.ssh/mykey.pem

# Connect effortlessly by hostname or IP address
# This assumes that your internal DNS uses the fake TLD ec2
# This assumes that 172.31.0.0 is your C-Class subnet
Host *.ec2 172.31.*
    # This command says proxy all ssh connections through bastion as if
    # you had done an ssh -A
    ProxyCommand ssh -W %h:%p bastion.ec2
    ForwardAgent yes
    # These next lines are documentation you leave as a love letter to
    # your future self when all else fails or you have to help a
    # coworker and decide to look at your own config.
    # ssh-add ~/.ssh/*.pem
    # ssh -At bastion.ecs ssh [email protected]

fig.7

If you take nothing else away from fig.7 it should be the proper use of ProxyCommand & ForwardAgent.

Auto populate your .bash_profile

You don't want to have to do ssh-add manually every time you log into your machine. ~/.bash_profile is a script that runs each time you login**. Put the line from fig. 3 in there and you should always have your agent ready.

** Don't confuse this with .bashrc which runs for each new [interactive] terminal. Your agent continues running even if you close all your terminal sessions. No Need to reload your keys

Alternative to using .bash_profile

I have also created a gist that add an OSX/macOS Launch Agent. You can use that method for starting your ssh-agent on boot. It is very easy to install:

curl -sSL https://gist.github.com/RichardBronosky/429a8fff2687a16959294bcee336dd2a/raw/install.sh | bash
2
  • 1
    Step 2: You cannot force ForwardAgent yes with -A if the host disallows it.
    – dlamblin
    Jul 11, 2017 at 1:52
  • 1
    Thanks @dlamblin, I added the relevant info and a link to the sshd documentation. Jun 26, 2018 at 19:09
2

This is where you come in. How can I forward a different key for the 2nd connection?

You do not forward the keys. You forward agent and it can have completely independent keys than you are using for authentication in the first jump. Check the keys in your agent using ssh-add -L.

But even better, run the connection over ProxyCommand ssh -W %h:%p [email protected], which will avoid the need to forward the agent, the heap or command-line options and a need to authenticate from the immediate host.

You can simply put that in your config (see man ssh_config).

2
  • 1
    Luckily I actually know what you are talking about here, but the average user would not. You argue semantics with your "you forward agents, not keys" thing. You gave me enough to go on that I was able to solve this so I want to give you the chance to correct your answer before I add my own. Hint: ssh-add mykey.pem sharedkey.pem then confirm with ssh-add -L then ssh -At [email protected] ssh [email protected] Oct 31, 2016 at 15:49
  • Great writeup. Note OpenSSH 8.2 does not have allowagentforwarding as the default.
    – ndemarco
    Nov 2, 2022 at 13:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .