Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How can i force wget to stop after X seconds? I have a script that downloads images, from time to time it gets stuck and refuses to "timeout"

what i've tried:

--tries=3 --connect-timeout=30

from ps aux:

root     26543  0.0  0.0  38636  1656 ?        S    20:40   0:00 wget -nc --tries=3 --connect-timeout=30 --restrict-file-names=nocontrol -O 18112012/image.jpg http://site/image.jpg
share|improve this question

migrated from stackoverflow.com Dec 5 '12 at 21:17

This question came from our site for professional and enthusiast programmers.

    
Have you tried the --timeout (or -T) option? – gniourf_gniourf Dec 5 '12 at 19:04
    
yes... everything from wget man – teslasimus Dec 5 '12 at 19:12
    
Are you sure you tried wget -nc --tries=3 -T30 --restrict-file-names=nocontrol -O 18112012/image.jpg http://site/image.jpg? I've never had problems like you're describing. – gniourf_gniourf Dec 5 '12 at 19:17
up vote 0 down vote accepted

You can run the wget command as a background process and send a SIGKILL to forcibly kill it after sleeping for a certain amount of time.

wget ... &
wget_pid=$!
counter=0
timeout=60
while [[ -n $(ps -e) | grep "$wget_pid") && "$counter" -lt "$timeout" ]]
do
    sleep 1
    counter=$(($counter+1))
done
if [[ -n $(ps -e) | grep "$wget_pid") ]]; then
    kill -s SIGKILL "$wget_pid"
fi

Explanation:

  • wget ... & - the & notation at the end runs the command in the background as opposed to the foreground
  • wget_pid=$! - $! is a special shell variable that contains the process id of the most recently executed command. Here we save it to a variable called wget_pid.
  • while [[ -n $(ps -e) | grep "$wget_pid") && "$counter" -lt "$timeout" ]] - Look for the process every one second, if it's still there, keep waiting until a timeout limit.
  • kill -s SIGKILL "$wget_pid" - We use kill to forcibly kill the wget process running in the background by sending it a SIGKILL signal.
share|improve this answer
    
very nice! thanks chen – teslasimus Dec 5 '12 at 19:03
1  
What if wget succeeds? You'll throw randomly a SIGKILL? – gniourf_gniourf Dec 5 '12 at 19:08
    
how can i skip "sleep" if file has been already downloaded? – teslasimus Dec 5 '12 at 19:09
1  
Rewrote the script to make it a bit more robust and address the skipping of sleep times issue – sampson-chen Dec 5 '12 at 19:17
    
@gniourf_gniourf is there evidence that it's harmful to send a SIGKILL to nothing? – kojiro Dec 5 '12 at 19:25

Easiest way is to use the timeout(1) command, part of GNU coreutils, so available pretty much anywhere bash is installed:

timeout 60 wget ..various wget args..

or if you want to hard-kill wget if its running too long:

timeout -s KILL 60 wget ..various wget args..
share|improve this answer
    
You do know bash is installed on every Mac, which do not have a GNU userland, right? – kojiro Dec 5 '12 at 19:22
1  
@kojiro: The question is tagged ubuntu which includes GNU coreutils, and it can readily be installed on a mac if you want it. – Chris Dodd Dec 5 '12 at 19:24
    
Hmm, maybe it should've been migrated to askubuntu.com. >;) – kojiro Dec 5 '12 at 21:24
1  
This is definitely the easiest and safest way. man timeout will show other options. I would personally not use the -s KILL option directly, but maybe use the -k option, something like timeout -k 5 60 wget ..various wget args.. (to give a 5 extra seconds chance to wget before being KILLed). This answer deserves +1. – gniourf_gniourf Dec 6 '12 at 7:49

Community wiki because this is mostly a copy of sampson-chen's answer, but I wanted to point out a couple of things:

wget ... &
# Strictly speaking you can just use the job number,
# which is probably %1, but saving the pid is also fine.
wget_pid=$! 
counter=0
timeout=60
# use kill -0 to check if a pid is still running
while kill -0 "$wget_pid" && (( counter < timeout )); do
    sleep 1
    (( counter++ ))
done
# if killing nothing is distasteful, use kill -0 one more time.
# also think a SIGKILL is overkill since the question doesn't imply wget needs it.
kill -0 "$wget_pid" && kill "$wget_pid"
share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .