How to effectively debug HTTP request issue on my web server?

Question

I'm having a strange issue with my web server across my LAN. HTTP requests spin forever but don't give a meaningful error code.

This issue happens on my ASUS router and my Cisco router, but paradoxically does not happen on my old DLink router. Some analysis of the differences below.

[Update Feb19 10:30PM] I've tried switching from Apache to the Nginx server (based on Kevin's comment), and they display an identical result. Appears to be TCP packet issues independent of server (see the tcpdumps below). I've also experimented with the TCP datapush size and find the probability of failure goes down with request size. When less than ~800 bytes, it manages to load about half the time after looping 5-10 before getting that rare ACK. At ~2000 bytes, it rarely but sometimes works. At ~6000 bytes, no way. The DLink router works instantly on any of these file sizes. I've checked to see if any interfering packets were coming on the loopback interface, nothing there. I've switched cables/ports (but not NIC's, only have the one).

I've tried to use telnet to do a manual HTTP request for a simple page:

telnet 192.168.0.101 80
GET /index.html HTTP/1.0

The same issue appears there, hanging for several seconds.

Netstat output:

simon@fire:~$ sudo netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1056/sshd       
tcp        0      0 0.0.0.0:23              0.0.0.0:*               LISTEN      1147/xinetd     
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1482/sendmail: MTA:
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      1064/mysqld
tcp        0      0 127.0.0.1:587           0.0.0.0:*               LISTEN         1482/sendmail: MTA:
tcp6       0      0 :::21                   :::*                    LISTEN      1066/vsftpd     
tcp6       0      0 :::22                   :::*                    LISTEN      1056/sshd       
tcp6       0      0 :::80                   :::*                    LISTEN      1234/apache2

Shutting down Apache and trying netcat works immediately across the same port:

simon@fire:~$ sudo service apache2 stop
simon@fire:~$ { echo -ne "HTTP/1.0 200 OK\r\nContent-Length: 13\r\n\r\n"; echo "Hello World!"; } | sudo nc -l 80
GET / HTTP/1.1
Host: 192.168.0.101
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
(On client --> Works immediately)

The tcpdump dump seems to indicate a missing "ACK 2921" packet following the Push packet. Then the server repeats the Push packet, still no ACK. Then it proceeds to do a series of seq 1:1461 ACKs before a finishing F packet, seeming to give up.

I'm no expert on tcp packets but this is from analyzing the failing case (ASUS router) vs successful case (Dlink router). On the DLink, there's a clear ACK 2921 packet. Prior to that, both show the same output.

One obvious difference is that with the DLink, the server is seeing the client by its IP (192.168.0.104), whereas on the ASUS the server is seeing the client by its hostname (Gin) - but experimenting the 3rd router (Cisco) rules this out as part of the problem as it also sees the client by IP, but behaves the same as the ASUS otherwise.

simon@fire:~$ sudo tcpdump -i eth0 port 80 or port 443
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes

19:43:37.815448 IP Gin.60309 > fire.http: Flags [S], seq 4284943091, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
19:43:37.815490 IP fire.http > Gin.60309: Flags [S.], seq 1854464822, ack 4284943092, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
19:43:37.815684 IP Gin.60309 > fire.http: Flags [.], ack 1, win 16425, length 0
19:43:37.816701 IP Gin.60309 > fire.http: Flags [P.], seq 1:344, ack 1, win 16425, length 343: HTTP: GET / HTTP/1.1
19:43:37.816729 IP fire.http > Gin.60309: Flags [.], ack 344, win 237, length 0
19:43:37.818341 IP fire.http > Gin.60309: Flags [.], seq 1:2921, ack 344, win 237, length 2920: HTTP: HTTP/1.1 200 OK
19:43:37.818357 IP fire.http > Gin.60309: Flags [P.], seq 2921:4155, ack 344, win 237, length 1234: HTTP

-- There seems to be a missing [.] ack 2921 packet here!? Then the server tries again --

19:43:37.827311 IP fire.http > Gin.60309: Flags [P.], seq 2921:4155, ack 344, win 237, length 1234: HTTP
19:43:38.051329 IP fire.http > Gin.60309: Flags [.], seq 1:1461, ack 344, win 237, length 1460: HTTP: HTTP/1.1 200 OK
19:43:38.499322 IP fire.http > Gin.60309: Flags [.], seq 1:1461, ack 344, win 237, length 1460: HTTP: HTTP/1.1 200 OK
19:43:39.395318 IP fire.http > Gin.60309: Flags [.], seq 1:1461, ack 344, win 237, length 1460: HTTP: HTTP/1.1 200 OK
19:43:41.191327 IP fire.http > Gin.60309: Flags [.], seq 1:1461, ack 344, win 237, length 1460: HTTP: HTTP/1.1 200 OK
19:43:42.823524 IP fire.http > Gin.60309: Flags [F.], seq 4155, ack 344, win 237, length 0
19:43:42.823736 IP Gin.60309 > fire.http: Flags [.], ack 1, win 16425, length 0
19:43:44.783318 IP fire.http > Gin.60309: Flags [.], seq 1:1461, ack 344, win 237, length 1460: HTTP: HTTP/1.1 200 OK
19:43:51.967338 IP fire.http > Gin.60309: Flags [.], seq 1:1461, ack 344, win 237, length 1460: HTTP: HTTP/1.1 200 OK
19:43:52.825832 IP Gin.60309 > fire.http: Flags [.], seq 343:344, ack 1, win 16425, length 1: HTTP
19:43:52.825875 IP fire.http > Gin.60309: Flags [.], ack 344, win 237, options [nop,nop,sack 1 {343:344}], length 0
19:44:02.826426 IP Gin.60309 > fire.http: Flags [.], seq 343:344, ack 1, win 16425, length 1: HTTP
19:44:02.826464 IP fire.http > Gin.60309: Flags [.], ack 344, win 237, options [nop,nop,sack 1 {343:344}], length 0
19:44:06.335330 IP fire.http > Gin.60309: Flags [.], seq 1:1461, ack 344, win 237, length 1460: HTTP: HTTP/1.1 200 OK
19:44:12.827029 IP Gin.60309 > fire.http: Flags [.], seq 343:344, ack 1, win 16425, length 1: HTTP
19:44:12.827067 IP fire.http > Gin.60309: Flags [.], ack 344, win 237, options [nop,nop,sack 1 {343:344}], length 0
19:44:22.827563 IP Gin.60309 > fire.http: Flags [.], seq 343:344, ack 1, win 16425, length 1: HTTP
19:44:22.827592 IP fire.http > Gin.60309: Flags [.], ack 344, win 237, options [nop,nop,sack 1 {343:344}], length 0
19:44:32.828133 IP Gin.60309 > fire.http: Flags [.], seq 343:344, ack 1, win 16425, length 1: HTTP
19:44:32.828157 IP fire.http > Gin.60309: Flags [.], ack 344, win 237, options [nop,nop,sack 1 {343:344}], length 0
19:44:35.039322 IP fire.http > Gin.60309: Flags [.], seq 1:1461, ack 344, win 237, length 1460: HTTP: HTTP/1.1 200 OK
19:44:42.828692 IP Gin.60309 > fire.http: Flags [.], seq 343:344, ack 1, win 16425, length 1: HTTP
19:44:42.828734 IP fire.http > Gin.60309: Flags [.], ack 344, win 237, options [nop,nop,sack 1 {343:344}], length 0
19:45:08.571555 IP Gin.60309 > fire.http: Flags [F.], seq 344, ack 1, win 16425, length 0
19:45:08.571581 IP fire.http > Gin.60309: Flags [.], ack 345, win 237, length 0

(Pressed CTRL C after 30 seconds)
^C
32 packets captured
32 packets received by filter
0 packets dropped by kernel

The tcpdump when using the DLink router (working fine) scenario is below:

simon@fire:/sbin$ sudo tcpdump -B 4096 -i eth0 port 80 or port 443
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:48:20.209920 IP 192.168.0.104.53696 > fire.http: Flags [S], seq 1043812686, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
21:48:20.209954 IP fire.http > 192.168.0.104.53696: Flags [S.], seq 639467520, ack 1043812687, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
21:48:20.210290 IP 192.168.0.104.53696 > fire.http: Flags [.], ack 1, win 16425, length 0
21:48:20.211575 IP 192.168.0.104.53696 > fire.http: Flags [P.], seq 1:344, ack 1, win 16425, length 343: HTTP: GET / HTTP/1.1
21:48:20.211603 IP fire.http > 192.168.0.104.53696: Flags [.], ack 344, win 237, length 0
21:48:20.213190 IP fire.http > 192.168.0.104.53696: Flags [.], seq 1:2921, ack 344, win 237, length 2920: HTTP: HTTP/1.1 200 OK
21:48:20.213199 IP fire.http > 192.168.0.104.53696: Flags [P.], seq 2921:4155, ack 344, win 237, length 1234: HTTP
21:48:20.213959 IP 192.168.0.104.53696 > fire.http: Flags [.], ack 2921, win 16425, length 0
21:48:20.415317 IP fire.http > 192.168.0.104.53696: Flags [P.], seq 2921:4155, ack 344, win 237, length 1234: HTTP
21:48:20.416066 IP 192.168.0.104.53696 > fire.http: Flags [.], ack 4155, win 16116, options [nop,nop,sack 1 {2921:4155}], length 0
21:48:25.215387 IP fire.http > 192.168.0.104.53696: Flags [F.], seq 4155, ack 344, win 237, length 0
21:48:25.215864 IP 192.168.0.104.53696 > fire.http: Flags [.], ack 4156, win 16116, length 0
21:48:25.216086 IP 192.168.0.104.53696 > fire.http: Flags [F.], seq 344, ack 4156, win 16116, length 0
21:48:25.216093 IP fire.http > 192.168.0.104.53696: Flags [.], ack 345, win 237, length 0
^C
14 packets captured
14 packets received by filter
0 packets dropped by kernel

I also checked the firewall, found no packets hitting the "Deny" rules, and also executed the recommended iptables command but it didn't help: /sbin/iptables -I INPUT -p tcp --dport 80 -j ACCEPT

ifconfig output:

eth0  Link encap:Ethernet  HWaddr 00:16:e6:85:ad:a1  
      inet addr:192.168.0.101  Bcast:192.168.0.255  Mask:255.255.255.0
      inet6 addr: fe80::216:e6ff:fe85:ada1/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:4660 errors:0 dropped:2 overruns:0 frame:0
      TX packets:1769 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000 
      RX bytes:399847 (399.8 KB)  TX bytes:210999 (210.9 KB)
      Interrupt:17 

lo    Link encap:Local Loopback  
      inet addr:127.0.0.1  Mask:255.0.0.0
      inet6 addr: ::1/128 Scope:Host
      UP LOOPBACK RUNNING  MTU:65536  Metric:1
      RX packets:47 errors:0 dropped:0 overruns:0 frame:0
      TX packets:47 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0 
      RX bytes:15869 (15.8 KB)  TX bytes:15869 (15.8 KB)

iptables output:

~$ sudo iptables -L -n
Chain INPUT (policy DROP)
target     prot opt source               destination         
f2b-sshd   tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
ufw-before-logging-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-before-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-logging-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-reject-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-track-input  all  --  0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy DROP)
target     prot opt source               destination         
ufw-before-logging-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-before-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-logging-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-reject-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-track-forward  all  --  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ufw-before-logging-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-before-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-logging-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-reject-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-track-output  all  --  0.0.0.0/0            0.0.0.0/0           

Chain f2b-sshd (1 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-after-forward (1 references)
target     prot opt source               destination         

Chain ufw-after-input (1 references)
target     prot opt source               destination         
ufw-skip-to-policy-input  udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:137
ufw-skip-to-policy-input  udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:138
ufw-skip-to-policy-input  tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:139
ufw-skip-to-policy-input  tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:445
ufw-skip-to-policy-input  udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:67
ufw-skip-to-policy-input  udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:68
ufw-skip-to-policy-input  all  --  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type BROADCAST

Chain ufw-after-logging-forward (1 references)
target     prot opt source               destination         
LOG        all  --  0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw-after-logging-input (1 references)
target     prot opt source               destination         
LOG        all  --  0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw-after-logging-output (1 references)
target     prot opt source               destination         

Chain ufw-after-output (1 references)
target     prot opt source               destination         

Chain ufw-before-forward (1 references)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 3
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 4
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 11
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 12
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 8
ufw-user-forward  all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-before-input (1 references)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
ufw-logging-deny  all  --  0.0.0.0/0            0.0.0.0/0            ctstate INVALID
DROP       all  --  0.0.0.0/0            0.0.0.0/0            ctstate INVALID
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 3
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 4
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 11
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 12
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 8
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            udp spt:67 dpt:68
ufw-not-local  all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     udp  --  0.0.0.0/0            224.0.0.251          udp dpt:5353
ACCEPT     udp  --  0.0.0.0/0            239.255.255.250      udp dpt:1900
ufw-user-input  all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-before-logging-forward (1 references)
target     prot opt source               destination         

Chain ufw-before-logging-input (1 references)
target     prot opt source               destination         

Chain ufw-before-logging-output (1 references)
target     prot opt source               destination         

Chain ufw-before-output (1 references)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
ufw-user-output  all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-logging-allow (0 references)
target     prot opt source               destination         
LOG        all  --  0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW ALLOW] "

Chain ufw-logging-deny (2 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            0.0.0.0/0            ctstate INVALID limit: avg 3/min burst 10
LOG        all  --  0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] "

Chain ufw-not-local (1 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
RETURN     all  --  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type MULTICAST
RETURN     all  --  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type BROADCAST
ufw-logging-deny  all  --  0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 10
DROP       all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-reject-forward (1 references)
target     prot opt source               destination         

Chain ufw-reject-input (1 references)
target     prot opt source               destination         

Chain ufw-reject-output (1 references)
target     prot opt source               destination         

Chain ufw-skip-to-policy-forward (0 references)
target     prot opt source               destination         
DROP       all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-skip-to-policy-input (7 references)
target     prot opt source               destination         
DROP       all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-skip-to-policy-output (0 references)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-track-forward (1 references)
target     prot opt source               destination         

Chain ufw-track-input (1 references)
target     prot opt source               destination         

Chain ufw-track-output (1 references)
target     prot opt source               destination         
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            ctstate NEW
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            ctstate NEW

Chain ufw-user-forward (1 references)
target     prot opt source               destination         

Chain ufw-user-input (1 references)
target     prot opt source               destination         
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443 /* 'dapp_Nginx%20Full' */
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:80 /* 'dapp_Nginx%20HTTP' */

Chain ufw-user-limit (0 references)
target     prot opt source               destination         
LOG        all  --  0.0.0.0/0            0.0.0.0/0            limit: avg 3/min burst 5 LOG flags 0 level 4 prefix "[UFW LIMIT BLOCK] "
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain ufw-user-limit-accept (0 references)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-user-logging-forward (0 references)
target     prot opt source               destination         

Chain ufw-user-logging-input (0 references)
target     prot opt source               destination         

Chain ufw-user-logging-output (0 references)
target     prot opt source               destination         

Chain ufw-user-output (1 references)
target     prot opt source               destination

Similar issue happens if I try to access internet from the server on port 80 (extremely slow, spins or sometimes can load a simple page after a very long time).

This is a LAMP server (Ubuntu 16.4).

Thanks in advance!

telnet in 21st century ??? You are brave... If this server facing wild internet then I strongly advise you switch to SSH. About your issue, run sudo netstat -lntp and add its result to your question — Alex, Feb 19, 2017 at 7:24
Thanks Alex - I added the netstat -lntp output to the question. Anything interesting in there? :) — Horseman, Feb 19, 2017 at 20:05
@Alex - telnet is not being used here to issue commands but just to issue a raw HTTP request. While nc is a bit better for this it's no security risk. The security risk is in running a telnet server, not using the telnet command to manually send data through a socket. — LawrenceC, Feb 19, 2017 at 22:06
@LawrenceC Agree, when I see when people still using telnet servers nowadays, it affect me as a red cloth in a bull fights, but according to netstat output, OP's server listening for incoming connections on port 23, so it's still the case. Anyway, I appreciate your comment ;) before netstat output it sounds like not really good comment from my side ! — Alex, Feb 19, 2017 at 23:40
ahh crap yeah, now that I actually looked closely, you are right. — LawrenceC, Feb 20, 2017 at 2:13

Kevin · Accepted Answer · 2017-02-19 21:42:33Z

1

Have you tried running tcpdump on the server?

sudo tcpdump -i eth0 port 80 or port 443

If you don't see the packets you may need to double check your firewall.

You can also shutdown apache and run a temporary netcat webserver on port 80 with

{ echo -ne "HTTP/1.0 200 OK\r\nContent-Length: 13\r\n\r\n"; echo "Hello World!"; } | sudo nc -l 80

Edit:

You can use tcpdump to write to a file and then transfer and open it on a client machine with wireshark to see what information is being transferred in the packets.

sudo tcpdump -i eth0 port 80 or port 443 -w httpdebug.pcap

Then copy that file to a machine with wireshark. If you need to install wireshark on a windows machine you can skip installing winpcap.

Some other steps you can try:

Having apache serve a file with just "Hello World!" or something very short in it.
Using the netcat webserver to try and host your index.html

{ echo -ne "HTTP/1.0 200 OK\r\nContent-Length: $(wc -c <index.html)\r\n\r\n"; cat index.html; } | sudo nc -l 80

If the latter works there may either be a lot of render blocking content or some misconfiguration in apache. I would try purging and reinstalling apache, rather than searching for the config issue.

edited Feb 19, 2017 at 21:42

answered Feb 19, 2017 at 6:53

Kevin

1766 bronze badges

+1 for the check firewall bit - Try running "/sbin/iptables -I INPUT -p tcp --dport 80 -j ACCEPT on the server to allow it to accept traffic on port 80 until rebooted/the firewall is restarted.
– davidgo
Feb 19, 2017 at 7:23
Thanks very much Kevin and Davidgo - I added the tcpdump, and netcat info to the description. Also tried iptables (no help). netcat works on port 80. TCPdump seems to show it repeating the same request over and over... but no actual dropped packets?!
– Horseman
Feb 19, 2017 at 20:06
This isn't necessarily a good answer (fixing your issue), but have you considered running nginx either in front of or replacing apache. I use nginx and found it much easier to configure and use. It is also (supposedly) much more efficient at serving static content.
– Kevin
Feb 19, 2017 at 21:45
Kevin, I haven't heard of nginx, I'll look it up but hoping i can also resolve this with apache as well.
– Horseman
Feb 19, 2017 at 22:16
For the tcpdump, it seems there's a missing ACK 2921 to correspond to the push packet... and interestingly on a different router where the server sees the client just by its IP and note its "Gin" hostname, everything works, and t I'll update the desc with some more info on this. 19:43:37.818357 IP fire.http > Gin.60309: Flags [P.], seq 2921:4155, ack 344, win 237, length 1234: HTTP -- Missing ACK here for for 2921, and it never comes. -- 19:43:37.827311 IP fire.http > Gin.60309: Flags [P.], seq 2921:4155, ack 344, win 237, length 1234: HTTP
– Horseman
Feb 19, 2017 at 22:23

| Show 2 more comments

Alex · Accepted Answer · 2017-02-19 23:28:57Z

0

According to sudo netstat -lntp output, your apache's setup don't listening on IPv4.

You need to add in your apache's configuration:

#NameVirtualHost *:80
Listen 0.0.0.0:80

If you wold use SSL protocol on your site then add also:

<IfModule ssl_module>
#   NameVirtualHost *:443
    Listen 0.0.0.0:443
</IfModule>

<IfModule mod_gnutls.c>
#   NameVirtualHost *:443
    Listen 0.0.0.0:443
</IfModule>

These settings applies to Apache 2.4 + versions. If you using 2.2 versions then add also

NameVirtualHost *:80
NameVirtualHost *:443

in (apache|httpd|ports)\.conf

answered Feb 19, 2017 at 23:28

Alex

6,2271 gold badge17 silver badges25 bronze badges

I updated it as you mentioned, but it didn't help. And on researching, it looks like both IPv4 and v6 work in the previous setting but netstat just represents it as tcp6, some info about this here link
– Horseman
Feb 20, 2017 at 2:50
@Horseman " it looks like both IPv4 and v6 work in the previous setting but netstat just represents it as tcp6" - No, it shouldn't be. If you used Listen 0.0.0.0:80 it effectively disable listener on TCP6. Did you restated your apache server (sudo service apache2 restart) after you added changes I mentioned ?
– Alex
Feb 20, 2017 at 6:02
yes, restarted the server. The problem does not appear to be related to ipv4/v6. (have since tried the nginx server with both ipv4/v6 enabled as well)
– Horseman
Feb 20, 2017 at 6:27
@Horseman Are you able to get any output from this command: wget -O - http://127.0.0.1 ? Could you also add output of ifconfig and iptables -L -n
– Alex
Feb 20, 2017 at 6:42
I added the ifconfig and iptables output above. Using wget -O - 127.0.0.1 seems to work well, displays the html page in full right away. Thanks for continuing to give suggestions, you're my lifeline :o)
– Horseman
Feb 21, 2017 at 4:25

Add a comment |

Stack Exchange Network

How to effectively debug HTTP request issue on my web server?

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
apache-http-server
http
ubuntu-16.04
lamp
.

Hot Network Questions

How to effectively debug HTTP request issue on my web server?

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged apache-http-serverhttpubuntu-16.04lamp.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
apache-http-server
http
ubuntu-16.04
lamp
.