Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm getting the error relating to port exhaustion on my server ("Only one usage of each socket address (protocol/network address/port) is normally permitted").

So, I checked with netstat -n, and it lists about 84000 connections.

Based on the local port, and the fact that the foreign IP address is myself, I can see that for example 3654 of them relate to a program running on the same server that I killed over 24 hours ago (yesterday morning, in fact). They're all in the TIME_WAIT state, on foreign ports between 49165 and 65533. At least some of the same foreign port numbers show up as local ports too, for example:

TCP    <my_ip>:14046  <my_ip>:49165  TIME_WAIT
...
TCP    <my_ip>:49165  <other_ip>:80  TIME_WAIT

14046 is the port that a program on my server listens to, and that the program I killed yesterday used to repeatedly connect to. host <other_ip> gives me something at compute-1.amazonaws.com, I don't recognise the IP. I can't get anything out of it from my browser. So I can't quite figure out what on that machine would have connected to port 80 on , but there are various things that have run on the same machine recently that consume web services, so I guess it could be a redirect or similar from one of those.

Separate runs of netstat more than 4 minutes apart show that both connections are sticking around in the list.

Is this expected? I thought sockets were supposed to finally close after 4 minutes or so, but does netstat normally list them for much longer? Are these ports really occupied, and if so is there anything I can do to release them? Preferably without rebooting, but if I have to then I have to.

OS is Windows Server Enterprise, running on a Rackspace VM.

[Edit: hmm, should I have asked this on serverfault?]

[Another edit: a reboot seemed to fix it. All the old connections gone, and new connections of the same type disappearing in a few minutes as expected. I would guess therefore that the network stack somehow got into a malfunctioning state.]

share|improve this question
    
FWIW, the issue outlined here doesn't seem to be something specific to an enterprise server environment (could happen on any system), so I'd keep it here. –  Darth Android Aug 4 '11 at 19:31
    
@Darth: that's what I figured - the port exhaustion is because I'm running server-type things, but the fact that the ports stick around may well not be. I just would never have noticed if it had been a few hundred rather than a few tens of thousands. –  Steve Jessop Aug 5 '11 at 8:40
add comment

1 Answer

Take a look at the following article from Microsoft: Avoiding TCP/IP Port Exhaustion.

One of the suggestions it has is to check your registry setting for the TcpTimedWaitDelay key under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters. That entry controls "the length of time that a connection stays in the TIME_WAIT state when the connection is being closed". Perhaps you've got a very long value somehow .. the default is 240 seconds.

Also, you may want to check and see if you can trace and see which processes are causing the TIME_WAIT connections and see if there is a configuration setting in it to close ports quicker.

share|improve this answer
    
Thanks, there's no TcpTimedWaitDelay value set. I know exactly what processes caused the connections on port 14046. The port 80 could be a few different things, but they all use Python's urllib or httplib to consume web services. A bug in my code or in Python, not closing the sockets cleanly, I'd sort of understand. I don't know how I'd even go about writing a bug that sets the TIME_WAIT time to (at least) several days, though... –  Steve Jessop Aug 5 '11 at 8:39
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.