3

I've been getting a connection timed out since some days in 1 of our EC2 instance.

I was able to connect to SSH previously but all of a sudden it stopped working. Server is still up and running, HTTPS works fine and i can reach the services through the browser but can't SSH.

Here are the steps i tried so far:

  • SSH from PAC (linux SSH client) using the key pair: Getting a time out. This was working before, the connection did not change
  • SSH from another EC2 instance using the public host. Getting a timed out
  • SSH from another EC2 instance using the private IP. Getting a timed out
  • I then stopped/started the instance, got a new IP, changed the info in route 53, the site is back up and running but SSH with the above steps still does not work.
  • I checked the security groups (in case it changed somehow) and the port 22 is allowed inbound. That same security group is used on another instance that works fine.
  • I also added my IP to the group just in case and it still did not work. I have 4 instances in the same availability zone but the problematic one has a different IP range (it was different before as well) Problematic Public IP starts with: 35.182. Other working 3 instances Public IP starts with: 99.79.

I followed the troubleshooting steps located at the link below: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html

Error connecting to your instance: Connection timed out

  1. Check your security group rules. You need a security group rule that allows inbound traffic from your public IPv4 address on the proper port. As mentionned above, the security group attached has the port 22 inbound allowed
  2. Check the route table for the subnet. You need a route that sends all traffic destined outside the VPC to the internet gateway for the VPC. There is an internet gateway attached to my VPC (same VPC as my other instances)
  3. Check the network access control list (ACL) for the subnet. The network ACLs must allow inbound and outbound traffic from your local IP address on the proper port. The default network ACL allows all inbound and outbound traffic. Checked and the default settings are used, all traffic and ports are allowed for 0.0.0.0/0
  4. If your computer is on a corporate network, ask your network administrator whether the internal firewall allows inbound and outbound traffic from your computer on port 22 (for Linux instances) or port 3389 (for Windows instances). If you have a firewall on your computer, verify that it allows inbound and outbound traffic from your computer on port 22 (for Linux instances) or port 3389 (for Windows instances). I can connect to the other instances, this is not relevant.
  5. Check that your instance has a public IPv4 address. If not, you can associate an Elastic IP address with your instance. For more information, see Elastic IP Addresses. The instance contains a public ipv4 address, also has a public DNS, this is where i can see a difference between this instance and the other ones, this particular instance is the only one with a public ip starting with: 35.182. while the others all start with 99.79. This was not a problem before, since the IP was also different, could it be related somehow?
  6. Check the CPU load on your instance. CPU load and everything else is normal, no maintained peaks.

Here's even some more stuff i've done:

  • telnet on port 22 to a working instance, no problem works like a charm, but telnet to the problematic instance does not work, it simply hangs.
  • iptables were not modified (as far as i'm aware, i'm the only one that can connect to the instances and i did not do it)
  • checked the key pair owner and file permissions are at the expected values
  • Tried SSH with the terminal instead of the client, same results
  • Checked the system logs (this can be obtained though the EC2 dashboard) and there's no errors, system boots fine, apache2 is running and the site is accessible.

The server is running ubuntu 18.04, update and default upgrades we done.

I'm not sure what i can do anymore, but if anybody has any suggestions or need more info i'll be glad to provide them.

Thank you.

1

Check if Outbound connections on port 22 are allowed via the AWS Firewall and with the iptables on the machine. You can check all iptables rules using iptables -nvL .

Make sure that the Output chain in iptables is set to ALLOW instead of DROP, or make sure a proper rule allowing new, established and related connections for the Output chain is configured.

Or, do a netstat -npl and check where exactly SSH is listening on. If you've bound it to a Non-elastic IP address that has since changed, you'd have an issue.

EDIT: Since you mentioned you have no access via SSH to issue commands, you may use the AWS console to issue these and observe their output:

https://docs.aws.amazon.com/systems-manager/latest/userguide/rc-console.html

| improve this answer | |
  • These are valid troubleshooting steps but as i mentioned i can't connect to the machine at all with SSH therefore i have no way of checking iptables rules or doing a netstat, unless you have a method of doing it through through the AWS dashboard? – Cedric Guindon May 2 '19 at 10:40
  • Sure, have a look at: docs.aws.amazon.com/systems-manager/latest/userguide/… A bit finicky, but it will have to do in this situation. – nick064 May 3 '19 at 11:18
  • Thank you, i think that solves my problem, do you mind adding that info to your answer so i can mark it as answered? – Cedric Guindon May 3 '19 at 13:49
  • Done, I added to the original answer. Good luck with this :) – nick064 May 3 '19 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.