So, I / we have brand new Fedora Core 28 installation we've been trying to bring online. The installation went perfectly so far as we could tell. It has two network cards, one for an internal net, one with a fixed IP address. It's a server, so its primary job is to run a web server,
httpd. And it has to be accessible from other than the console, so it's running
At first, everything seems OK. The first net use was to use yum to install many packages, there was no problem. We added
gnome to get a GUI, added
firefox to get a web browser and then used that to help get data for troubleshooting. Connections on the inner net are fine. Connecting with
SSH is not a problem and at least one account was set up to use encryption keys to avoid a password. Connecting to the new web server seems fine too, using both internal and external IPs. So, we felt we had confirmed the
ssh and web server were configured properly. ...Initially, we didn't know anything was wrong.
When we got around to connecting from the outside world is when things "went sideways." Attempts to connect with
SSH just failed outright and connections to the web server also appeared to fail outright as well.
FIRST attempt was with the firewall. How's it configured? Is it
firewalld? How do we ensure we can get through from the outside? We found that somehow we'd configured the exception in
/etc/firewalld/zones - the default zone is
"FedoraServer.xml" in our case - as
httpd, and we noticed on a reboot that it complained about that and changed it to
http, with which we no longer noticed any complaint. Still no joy.
nmap! This is a port scanner for Linux systems to look at any network target's open ports. This seemed to show we were not blocked by the firewall. We confirmed
iptables was not in use and it was
firewalld, so we turned it off and still no joy.
Then we considered the older equipment. Twisted pair ethernet ports are bi-directional devices so was it possible inbound on the exterior port was good while outbound was bad? OR, maybe, there was a bad port on the switch? So, we tried swapping them, internal and external. No change.
We then considered that MAYBE the router was filtering, even though it shouldn't. The new box was a replacement for an older box that was retired, and the old one was fully functional, but who knows? Maybe the router has gone crazy? So, we went to log into it and see. But, unfortunately, nobody could find the router's password.
Instead, since the router only knows the systems by their external IPs, we shut down a fully-functional system and had the new one pick up the shut-down system's IP address. This eliminates the router as a potential cause, at least we thought so. However, the problems persisted.
It was suggested that maybe the router is at fault as perhaps it was grabbing MAC addresses, and so to eliminate that possibility, we rebooted it. Again, no improvement.
I then had the insight to ask our external tester to please increase the verbosity of ssh connection diagnostic data by adding, increasingly, -v to the command line (you can do up to three), and our internal sys-admin to please check the Apache logs. And this was very helpful. We found that the external perception was
ssh was getting to our
sshd daemon but the connections were being dropped and also the web server logs were showing correct inbound connections from just the places we expected, and logging "200" as the result, which means
httpd thought the web clients were getting their pages. But it wasn't true.
OK, so now what?