Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have the following problem: when I retrieve a page from Hackage, I get a large delay (about 30 seconds). Further requests are fast, but if I don't connect to it during a couple of minutes, the problem comes back.

What's interesting about this problem is:

  • it is specific to this particular site (Hackage) — I don't get a similar problem with any other site (and I visit quite a few);
  • it seems to be specific to my ISP — when I connect from other places, there's no such problem;
  • it's not related to DNS or connectivity problems — in fact, the TCP connection is established quickly; it's the HTTP response that takes too long, as can be seen from the following sample packet capture:

      1 0.000000000 192.168.1.101 -> 66.193.37.204 TCP 66 41518 > http [SYN] Seq=0 Win=13600 Len=0 MSS=1360 SACK_PERM=1 WS=16
      2 0.205708000 66.193.37.204 -> 192.168.1.101 TCP 66 http > 41518 [SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1440 SACK_PERM=1 WS=128
      3 0.205759000 192.168.1.101 -> 66.193.37.204 TCP 54 41518 > http [ACK] Seq=1 Ack=1 Win=13600 Len=0
      4 0.205846000 192.168.1.101 -> 66.193.37.204 HTTP 158 GET /packages/hackage.html HTTP/1.1 
      5 0.406461000 66.193.37.204 -> 192.168.1.101 TCP 54 http > 41518 [ACK] Seq=1 Ack=105 Win=5888 Len=0
      6 28.433860000 66.193.37.204 -> 192.168.1.101 TCP 1494 [TCP segment of a reassembled PDU]
      7 28.433904000 192.168.1.101 -> 66.193.37.204 TCP 54 41518 > http [ACK] Seq=105 Ack=1441 Win=16480 Len=0
      8 28.434211000 66.193.37.204 -> 192.168.1.101 HTTP 1404 HTTP/1.1 200 OK  (text/html)
      9 28.434228000 192.168.1.101 -> 66.193.37.204 TCP 54 41518 > http [ACK] Seq=105 Ack=2791 Win=19360 Len=0
     10 28.434437000 192.168.1.101 -> 66.193.37.204 TCP 54 41518 > http [FIN, ACK] Seq=105 Ack=2791 Win=19360 Len=0
     11 28.635146000 66.193.37.204 -> 192.168.1.101 TCP 54 http > 41518 [FIN, ACK] Seq=2791 Ack=106 Win=5888 Len=0
     12 28.635191000 192.168.1.101 -> 66.193.37.204 TCP 54 41518 > http [ACK] Seq=106 Ack=2792 Win=19360 Len=0
    

    (packet capture in pcap-ng format). This capture shows what happens during a simple curl http://hackage.haskell.org/packages/hackage.html.

It also doesn't matter that I'm behind a router — it's the same when I connect directly. The connection type is PPPoE.

I reproduced the problem on 3 computers that run Linux and Windows.

How to diagnose such a problem?

share|improve this question
    
Hi, I think that you need to use a browser with developer tools enabled to see the HTTP level dialog rather than the IP level dialog. We need to see what is causing the delay and you can only do this by looking at the total set of HTTP interactions for the page. Instead, you could use GMetrix. –  Julian Knight Feb 25 '13 at 21:32
    
Running GMetrix on the site gave me pretty good results with a couple of significant expections that might point you in the right direction. –  Julian Knight Feb 25 '13 at 21:34
    
@JulianKnight: there's a link to the full capture file in the question — it has all the information –  Roman Cheplyaka Feb 25 '13 at 21:55
    
Your link is a PCAP, I'm referring to something at a much higher level. Please report back using either a browser-based developer analysis or GMetrix or both. –  Julian Knight Feb 25 '13 at 22:58
1  
@JulianKnight: let me repeat — CSS is irrelevant here, and we're talking about a 30 seconds delay for a single HTTP request. –  Roman Cheplyaka Feb 26 '13 at 7:38
show 9 more comments

4 Answers

up vote 5 down vote accepted
+100

"30 seconds" and "after two minutes" are a dead ringer for a DNS issue to me.

If we suppose that the page you are connecting to does something like a DNS query on the connecting IP, and that query fails for some reason, you would see:

  • TCP connection almost instantaneous since the server is not doing DNS checks
  • the script runs a DNS query and gets stuck.
  • after 30 seconds the default timeout expires and the script goes on (you are now "Unknown")
  • on subsequent queries, the negative DNS hit is still cached and stage 1 is passed in next to no time
  • after negative timeout expires (RFC 2308), and that is anything between 2 and 5 minutes, a new query is issued on the next connect, and the story repeats.

...and these are exactly the symptoms you are describing.

You could try running a DNS query from another ISP (say, ISP2) on the IP you get from ISP1. It is not 100% proof, but I expect a high likelihood that the query will take 30 seconds to complete. That would mean that ISP1 DNS server is having problems answering to queries from the outside.

Another possible cause could be ISP1's DNS being firewalled out by Hackage for some (likely mistaken) reason (in my outfit, the reason would be "a trigger-happy netadmin", and I could name names). In that case you would have a much harder time diagnosing, for any tests through ISP2 would return nothing unusual; you'd have to escalate this to Hackage.

share|improve this answer
    
This looks very plausible! Let me verify it. –  Roman Cheplyaka Feb 28 '13 at 10:58
    
For the first cause, I tried going to haskell using an anonymous proxy and it was fast, which might possibly indicate that this cause is unlikely. For the second one, the same pause is then to be expected when accessing haskell from any ISP, so it is also unlikely. DNS might still be the cause, but it might be more complicated to explain. –  harrymc Feb 28 '13 at 12:08
    
@harrymc: it's very simple, actually. My ISP's DNS servers that are responsible for reverse DNS are down. So, attempts to do reverse resolving time out. Try this: dig +trace -x 80.90.233.38. I'm 95% sure that this is the cause, just waiting for confirmation that hackage indeed performs reverse DNS lookups. –  Roman Cheplyaka Feb 28 '13 at 16:35
add comment

Problem sounds like an issue with "MTU". If you google "windows setting mtu" you should come up with a number of responses which will show you how to test this theory, and lower your MTU as appropriate. (If you were using a Linux router I could produce an IPTables command to do this dynamically for you, but I don't "do" Windows.)

share|improve this answer
    
According to the Wireshark guide, the "TCP segment of a reassembled PDU" doesn't in fact correspond to IP fragmentation but rather just indicates that the response validly contains multiple packets as you would expect from a web page. –  Julian Knight Feb 25 '13 at 21:28
    
It doesn't seem to be MTU. I tested this by connecting directly via ethernet and setting mtu to 1000. The problem persisted. –  Roman Cheplyaka Feb 25 '13 at 22:05
add comment

I have repeated your packets capture, which look this way on my end :

capture image

Effectively there is a minor undetectable pause while the packet is reassembled, but nowhere as long as yours. I have also verified all the IP-addresses and the HTML, and everything is correct and looks extremely simple and harmless.

In short, there is no reason for this delay, as far as the Internet is concerned. The conclusion is that there is a problem with your ISP.

What you can do to narrow-down the possibilities is :

  1. Try connecting to another haskell.org package and see if there is a similar delay
  2. Try using another router from your place with several computers using different network adapters
  3. Try to have somebody in your area that uses the same ISP repeat the connection
  4. Try to have somebody in your area that uses another ISP repeat the connection
  5. With this information, if you still have no explanation for this delay, contact the Support of your ISP to ask what's going on.

[EDIT]

I noticed that haskell.org sends an ETag, so that explains why the first access is slow but the next ones are fast: Because for as long as the ETag is valid, the page actually comes from your browser's cache.

The weird part here is why the ISP is not slow when transmitting an ETag request. An explanation might be that for a limited time they satisfy the request from their own cache, rather than going to haskell.org.

share|improve this answer
    
1. This is the same for all hackage pages. 2. As I said, I tried this on several computers and with several routers (and without one). 4. The problem doesn't exist if I use another ISP in my area. –  Roman Cheplyaka Feb 28 '13 at 9:49
    
Now, the ISP problem indeed looks like the only plausible solution, but what kind of problem can it be? They probably do not even suspect about the existence of hackage, so it can't be intentional. If I tell them, "hey, this one site doesn't work for me (but all the others do)", they won't listen. –  Roman Cheplyaka Feb 28 '13 at 9:52
    
I added above an explanation why only the first access is slow. Point 3 still needs an answer before talking to the ISP. Their problem might be related to security software that they employ, being for some reason very slow to check the validity of haskell.org. –  harrymc Feb 28 '13 at 11:48
    
Etag is irrelevant, since I use curl for testing. Anyway, the answer about reverse dns is most probably the correct one. –  Roman Cheplyaka Feb 28 '13 at 11:52
add comment

It sounds like a server issue. It loaded fast for me. To test if the server dislikes you, try accessing it from a proxy, such as TOR or HideMyAss.com. If it is fast, then there is a problem between haskell.org and your house.

Another test you can run is to find a resource on that sight such as HTML file, CSS file, or XML file, and pass that link to an HTML validator, etc. If the 3rd party services takes a long time to fetch, then it is a problem with the server.

Another test: clear your DNS cache. It could be looking up haskell.org's IP address takes a long time. ipconfig /flushdns. Also try ping hackage.haskell.org from the command line to see how long it takes to look up the IP address.

Another test: open a private browsing session with Chrome (and others) to avoid sending cookies.

Another test: Open F12 in Chrome or Opera, go to Network tab, and then go to site to see the time for each resource.

share|improve this answer
    
When using a proxy the problem goes away. Your other suggestions are already addressed in the question itself. –  Roman Cheplyaka Feb 28 '13 at 7:47
    
The server doesn't like you. It is throttling your IP for whatever reason. There is nothing you can do. –  Chloe Feb 28 '13 at 8:20
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.