Curl response hangs

Question

I have a site set up in a virtual box. I am able to access it just fine from a browser on my machine, but I have trouble accessing it via CURL while SSH'ed in to the box.

When I try, curl hangs before displaying the response and exiting.

This is what I run: curl -vvv site1.dev/

This is the output it gives:

* Hostname was NOT found in DNS cache
*   Trying 192.168.10.10...
* Connected to site1.dev (192.168.10.10) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: site1.dev
> Accept: */*
> 
< HTTP/1.1 200 OK
* Server nginx/1.9.7 is not blacklisted
< Server: nginx/1.9.7
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Cache-Control: no-cache
< Date: Fri, 08 Apr 2016 16:47:30 GMT
< 
* Connection #0 to host site1.dev left intact
hi

The request portion is sent off right away, but the response hangs for a number of seconds (looks like 120ish), and then curl exits with that message: * Connection #0 to host site1.dev left intact

That is followed by the appropriate body of the response, "hi".

I'm a little lost -- any pointers would be appreciated.

Edit April 11: I've tried wget and see a similar result (response hangs). I suspect it is a network config issue.

In case it is relevant, here is some of the port config for the virtual box.

==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
==> default: Forwarding ports...
    default: 80 => 8000 (adapter 1)
    default: 443 => 44300 (adapter 1)
    default: 3306 => 33060 (adapter 1)
    default: 5432 => 54320 (adapter 1)
    default: 22 => 2222 (adapter 1)

EDIT April 12:

So... I decided to destroy this vagrant box and start fresh...doing this has resolved the problem.

I suspect that I changed/broke something over the course of the past several months. Starting over, with the vanilla box settings, has corrected that problem.

Answer 1

Perhaps nginx is configured to do ip resolution of incoming requests and is taking time to resolve the incoming connection before actually answering the request.

A couple pointers though, you'll want to check the following on 192.168.10.10.

1. Verify name servers are correct within /etc/resolv.conf
2. If #1 resolv settings are are correct for primary and secondary nameservers, verify that 192.168.10.10 has the ability to resolve hosts. (simple nslookup to google.com is a good test for this, if timeout occurs then then this could be part of the issue)
3. Ensure that your nginx server has the ability to query name servers externally or internally via firewall (port 53 tcp/udp)
4. Look for potential resolver settings within nginx configuration options and or set resolution timeout setting where applicable and restart nginx.
5. If its still an issue attempt to add the host of the incoming request connection within /etc/hosts on 192.168.10.10.

Let me know how that works for ya..

Thanks for posting.

Answer 2

Remark: You can get more detailed timing information by using time curl -vvv site1.dev/.

I note that the server reply contains Connection: keep-alive, which means that the server is configured to use HTTP keep-alive :

HTTP persistent connection, also called HTTP keep-alive, or HTTP connection reuse, is the idea of using a single TCP connection to send and receive multiple HTTP requests/responses, as opposed to opening a new connection for every single request/response pair. The newer HTTP/2 protocol uses the same idea and takes it further to allow multiple concurrent requests/responses to be multiplexed over a single connection.

The server is therefore keeping the connection open in the expectation of serving more requests. In this way, a browser can use the same TCP connection to receive an HTML page and immediately request the linked images without the need to establish new connections.

The curl.1 man page specifies the parameter --no-keepalive :

Disables the use of keepalive messages on the TCP connection, as by default curl enables them.

You may also parameter similarly the nginx server Module ngx_http_core_module keepalive_timeout :

Syntax:   keepalive_timeout timeout [header_timeout];
Default:  keepalive_timeout 75s;
Context:  http, server, location

The first parameter sets a timeout during which a keep-alive client connection will stay open on the server side. The zero value disables keep-alive client connections. The optional second parameter sets a value in the “Keep-Alive: timeout=time” response header field. Two parameters may differ.

The “Keep-Alive: timeout=time” header field is recognized by Mozilla and Konqueror. MSIE closes keep-alive connections by itself in about 60 seconds.