Is it possible to force wget to use one connection?

Question

I'm trying to download about 20GB of data with FTP with my personal account (no scp available and tar-ing is not possible).

I'm using the following standard command:

 wget -m ftp://user:[email protected]/db/

It works some time and then fails while authenticating for a download.

When trying to connect with a FTP client to the server, it responds with:

530 Sorry, the maximum number of clients (15) for this user are already connected.

So it seems like wget opens many connections. Is it possible to use only one connection to the server with wget?

Answer 1

By default, wget uses passive FTP, i.e., it uses one control connection and opens a new one for every file transfer.

In active FTP, the server opens the necessary connections. If you can accept incoming connections, that's the preferred way of transferring files via FTP.

wget only downloads one file at a time, so it should only use two connections. It looks like the connections don't get closed properly. Try if setting the transfer mode to "active" fixes that.

To enable active FTP, use the --no-passive-ftp switch.

Answer 2

I think wget just works that way. In the past when I needed to mirror stuff over FTP I have used lftp.

Answer 3

wget doesn't do any parallelism, but connections might drop and be counted by the server for a while.

--retry-connrefused --waitretry=30 should help. I'll let you look at wget(1) if those are not clear.