1

I see this option:

--waitretry=seconds

If you don't want Wget to wait between every retrieval, but only between retries of failed downloads, you can use this option. Wget will use linear backoff, waiting 1 second after the first failure on a given file, then waiting 2 seconds after the second failure on that file, up to the maximum number of seconds you specify. Therefore, a value of 10 will actually make Wget wait up to (1 + 2 + ... + 10) = 55 seconds per file.

By default, Wget will assume a value of 10 seconds.

But it doesn't (a) it doesn't seem to be increasing the time between retry attempts, and (b) I don't want to set a "max" amount of time to wait, but a minimum.

--timeout=seconds

Set the network timeout to seconds seconds. This is equivalent to specifying --dns-timeout, --connect-timeout, and --read-timeout, all at the same time.

When interacting with the network, Wget can check for timeout and abort the operation if it takes too long. This prevents anomalies like hanging reads and infinite connects. The only timeout enabled by default is a 900-second read timeout. Setting a timeout to 0 disables it altogether. Unless you know what you are doing, it is best not to change the default timeout settings.

All timeout-related options accept decimal values, as well as subsecond values. For example, 0.1 seconds is a legal (though unwise) choice of timeout. Subsecond timeouts are useful for checking server response times or for testing network latency.

This seems to set the amount of time before giving up.

So how do I say "wait 10 minutes before trying again if the connection times out"?

1 Answer 1

2

You could disable retrying (-t 0) and check wget return value. For example while using bash:

failures=0
while [ $failures -lt 10 ]; do
 wget -t 0 http://www.example.com/asdf
 if [ "$?" = 0 ]; then
  break
 else
  let failures=failures+1
  sleep 600
 fi
done

This will check return value of wget and sleep 600 seconds, if wget failed. wget returns 0 if file was downloaded (and saved) successfully. This small script runs only ten times - of course you can change argument on while loop. Otherwise for example script will try to download same file indefinitely, even if it just does not exist. Running using different shell requires small changes (you can't use "let" for instance, but usually expr is good replacement).

2
  • Hrm... it's a recursive job though. It would have to start all over every time.
    – mpen
    Jan 27, 2011 at 18:53
  • Then you are out of luck at least with wget. Of course if you can split recursive job to few smaller jobs, then starting over is smaller problem.
    – Olli
    Jan 28, 2011 at 12:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .