Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sometimes wget or curl randomly hangs in the middle of get request in Ubuntu 10.10:

curl "http://api.indeed.com/ads/apisearch?publisher=7570038743238473"
wget "http://api.indeed.com/ads/apisearch?publisher=7570038743238473" -O - -d

Seems the problem is in some low level library that does the request, but not in headers, cookies or caches.

Any ideas?

share|improve this question
    
For that specific URL, how often is "sometimes"? –  RedGrittyBrick Nov 29 '10 at 15:00

3 Answers 3

This URL fetches without errors or hangs for me. As wget and curl are, generally, these low-level libraries, and they only share the HTTP/TCP protocol, the error can be in two places:

  1. The request can be mishandled on your side. I've experienced symptoms like this several times, and looks like that they can be caused by a node on your route to the server which has MTU less than yours (often it is your provider's VPN, if they use it), and because of that packets get dropped (I don't know what happens there exactly, as they should be fragmented in theory).

    A quick check is to lower the MTU locally on your Ethernet/WiFi link like this:

    sudo ip link set $IFACE mtu 1300
    

    where $IFACE is your network interface (eth0, wlan0 or such).

  2. The request can be staled by server. The server can either be overloaded, misconfigured or just blocking your requests selectively. This is the less probable variant, and it cannot be diagnosed easily.

share|improve this answer

My ideas:

  • The server is occasionally overloaded.
  • Use a sniffer (such as Wireshark) to see what is happening when it hangs.
  • Use strace to see what wget/curl are doing when it hangs.
share|improve this answer

I'd use tcpdump to see where exactly it hangs. You might find regularities.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.