0

I refresh a few pages concurrently in a browser (any browser, Safari, Chrome, etc) and they suddenly get stuck, waiting for "connection". lsof shows a number of connections in status SYN_SENT. Their number goes up every second, up to 100-150. This happens for 10-20 seconds. Then they all disappear and web pages finally get loaded.

What this could be? I'm at home, on a regular home Internet connection.

3
  • What happens if you just refresh one page, then two pages, etc? The reason I ask is because most OSes limit the number of new TCP connections that can be created at any given time. Also can you use netstat to see how many connections are in 'CLOSE_WAIT' status when this problem happens? You should also read this Stackoverflow post. I can't help wondering if you have another application running which is opening/closing a lot of TCP connections simultaneously.
    – smashingly
    Jul 31 '14 at 1:28
  • Actually, can you please run netstat -p tcp -n and post the results here?
    – smashingly
    Jul 31 '14 at 1:30
  • Sorry, I meant TIME_WAIT state, not CLOSE_WAIT. The latter is when the local application hasn't closed the connection yet. The former is of more interest here, although any of the closing states (CLOSE_WAIT, LAST_ACK, FIN_WAIT_1, FIN_WAIT_2, CLOSING and TIME_WAIT) may tell us something useful.
    – smashingly
    Jul 31 '14 at 3:43
1

EDIT: At first I thought this would be a function of how many browser tabs you are opening, and therefore you'd be suffering from "normal" functioning of your TCP/IP stack - rate-limiting of new TCP connections (and running out of available ephemeral ports). But I re-read your original post and noticed that you said SYN_SENT connections are piling up. This means that your machine IS managing to create lots of connections at once. I'll leave the old text of my original answer intact (if for no other reason than that I spent hours researching it, and it's pretty interesting stuff) but I just wanted to add this edit at the start to avoid reader-frustration. The end of my post covers router limitations, and I suspect this is the problem. You could try the same test on a different internet connection (3G using an iPhone's Personal Hotspot, for example).


Original post...

I just did an experiment - I used Chrome's Developer Tools window to monitor a page-load, whilst loading www.ft.com. I then went through every single line on the Network tab of the Developer Tools window and hand-copied each unique host/server accessed when loading www.ft.com's main page. In total, 71 unique servers were accessed:

ft.com
s1.ft-static.com
navigation.webservices.ft.com
s4.media.ft.com
s2.ft-static.com
im.ft-static.com
www.googleadservices.com
www.ft-static.com
amch.questionmarket.com
reg.ft-static.com
service.maxymiser.net
personalisation.ft.com
pong.qubitproducts.com
track.ft.com
stats.ft.com
www.googletagservices.com
js.revsci.net
cdn.krxd.net
partner.googleadservices.com
admin.brightcove.com
mostpopular.sp.ft-static.com
googleads.g.doubleclick.net
fastft.ftdata.co.uk
widget-cdn.rpxnow.com
static.chartbeat.com
pubads.g.doubleclick.net
www.google.com
4235225.fls.doubleclick.net
ads.rubiconproject.com
ping.chartbeat.net
pagead2.googlesyndication.com
c.brightcove.com
ads.revsci.net
media.ft.com
pix04.revsci.net
beacon.krxd.net
secure.fastclick.net
leadback.advertising.com
ib.adnxs.com
api.adsymptotic.com
optimized-by.rubiconproject.com
cdn.quilt.janrain.com
s1.test.ft-static.com
www.facebook.com
b.scorecardresearch.com
tap2-cdn.rubiconproject.com
b.scorecardresearch.com
im.test.ft-static.com
clamo.ftdata.co.uk
assets.rubiconproject.com
tap.rubiconproject.com
x.bidswitch.net
rp.gwallet.com
rc.d.chango.com
i.w55c.net
tags.bluekai.com
rbp.mxptint.net
ads.p161.net
magnetic.t.domdex.com
ads.mediade.sk
ad.turn.com
video.ft.com
pool.adizio.com
cdn.trn.com
p.brilig.com
s.ixiaa.com
api.bizographics.com
metrics.brightcove.com
goku.brightcove.com
www.google-analytics.com
brightcove.vo.llnwd.net

It gets worse though! Note that many of those hosts were accessed multiple times, but let's assume the web browser (or OS) is intelligent enough to keep TCP connections open for a few seconds and reuse them. Using lsof -n -p 8420 | grep -c "IPv4" (Chrome is using PID 8420, and I have no other pages open in Chrome; I cleared the cache completely and disabled AdBlock etc beforehand). Whilst reloading the page, I simply upArrowed on the lsof command repeatedly, with the above piped grep giving me a count, i.e.:

$ lsof -n -p 8420 | grep -c "IPv4"
42
$ lsof -n -p 8420 | grep -c "IPv4"
64
$ lsof -n -p 8420 | grep -c "IPv4"
75
$ lsof -n -p 8420 | grep -c "IPv4"
85
$ lsof -n -p 8420 | grep -c "IPv4"
111
$ lsof -n -p 8420 | grep -c "IPv4"
129
$ lsof -n -p 8420 | grep -c "IPv4"
127
$ lsof -n -p 8420 | grep -c "IPv4"
128
$ lsof -n -p 8420 | grep -c "IPv4"
129
$ lsof -n -p 8420 | grep -c "IPv4"
128
$ lsof -n -p 8420 | grep -c "IPv4"
128
$

As you can see, loading a single web page resulted in a peak of 129 TCP connections. Whether or not they were in ESTABLISHED or some closing state like CLOSE_WAIT is irrelevant - those source-ports are not able to be used at that point and will (by default) take 60-480 seconds to be returned to the pool (RFC793 originally specified a MSL (Max Segment Lifetime) of 4 minutes but I think nowadays 60-120 seconds is the default). In Windows XP and Vista there were only a few thousand ephemeral ports available (by default), so as you can see, it would be very easy for your machine to run out of available ports. *It is important to note that as you can see from the first lsof execution, my system already had 42 connections open, so this web page has resulted in 87 new connections being opened. I ran it a couple of times to ensure other apps (email client, etc) weren't temporarily boosting those numbers during my testing.

There are also other limits - I can only speak about Linux here as I haven't had time to research this on Windows (I've already spent a few hours looking into this problem and I need to eat!) ... but in Linux (and OSX) there is a limit on the maximum number of file descriptors per process, and that limit is set in a couple of places. The limit is 256. Bear in mind some web browsers launch one process per tab, so I'm not sure if you would be hitting this wall, but it would be easy enough to change the limits and re-test. Google for "ulimit max file descriptors" and "launchctl maxfiles" to find both places where it is set. Also see this Stackoverflow post.

I know from my pre-OSX days that Windows XP had a rate-limit on the creation of new TCP connections. This was to reduce the spreading of worms like Sasser, by limiting how rapidly they could create new TCP connections, but it was also to prevent OS resources being swallowed up - each new TCP connection requires resources which are usually be PRE-allocated by the OS during boot-up (to allocate/destroy buffers every time a TCP connection is opened/closed, would take longer, so the OS creates a connection table in advance). If you look at the configuration page for Torrent clients (such as Transmission), their default global limit for new TCP connections is 120, with a limit-per-torrent of 60. Their user manual recommends sticking to 120, and I've seen my web-browsing performance really take a dive if I set it to say 200-240. It's the TCP MSL that really limits things here - the time it takes for a TCP port in TIME_WAIT to be returned to the pool.

My gut says that you're being limited by the fact that each of your browser tabs will be creating a lot of new TCP connections, and by loading say 15 tabs at once, you are asking the OS to open a lot of TCP connections all at the same time. When you re-run your test, ensure you wait at least 4 minutes between test runs, and use netstat -n -p tcp (in Windows just use netstat -no). If you're using *nix, use lsof -n -i | grep -c beforehand to see how many connections your computer already has open.

LASTLY... check your router/gateway. I've seen a lot of BusyBox-based ADSL routers in the past that have a max limit of 1024 concurrent TCP connections (in any state, i.e. established, close_wait, etC). One client running a P2P download was enough to grind the router completely to a halt - and the symptom was exactly as you describe, but even if you only tried to load one web page. 1024 connections could be eaten up within minutes by p2p simply because all you have to do is open and close 1024 connections within 4 minutes and no new connections would be allowed. The common complaint was "when I run eMule, even with a slow download speed limit, everyone in my home complains that the internet becomes almost unusable". The reason a home internet router gives a s*** about TCP connections is because they almost always run NAT/PAT and often have stateful/SPI firewalls - both of which require the tracking of all connections. That said, routers made in the last few years should be better at handling this.

Hope that helps.

2
  • I turned off "IP Flood Detection" option in my router. Everything works fine now :)
    – yegor256
    Aug 6 '14 at 7:54
  • Wow, that's great! I've never seen that feature on a router (well, not a consumer grade router anyway). As my tests above showed, modern website design has changed a lot even in 5 years, with so many different hosts being hit just to view one web page, and I think router firmware still needs to catch up. I wonder how many other people out there get similar issues... glad we could figure this one out on Superuser and hopefully other people with the same issue will land here and be able to fix it. All the best.
    – smashingly
    Aug 7 '14 at 22:06
0

This means your computer has sent a connection request (SYN) packet to the target machine, but has not received an answer (ACK or NAK, positive or negative, respectively) yet. This is a so-called half-open connection. It’ll go away when the connect timeout expires.

So either your connection is unreliable or the remote server is.

/edit: Of course, there shouldn’t be hundreds of them. No browser would do that. You sure it’s not some torrent client? ;)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.