Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

How can i force wget to stop after X seconds? I have a script that downloads images, from time to time it gets stuck and refuses to "timeout"

what i've tried:

--tries=3 --connect-timeout=30

from ps aux:

root     26543  0.0  0.0  38636  1656 ?        S    20:40   0:00 wget -nc --tries=3 --connect-timeout=30 --restrict-file-names=nocontrol -O 18112012/image.jpg http://site/image.jpg
share|improve this question

migrated from stackoverflow.com Dec 5 '12 at 21:17

This question came from our site for professional and enthusiast programmers.

    
Have you tried the --timeout (or -T) option? –  gniourf_gniourf Dec 5 '12 at 19:04
    
yes... everything from wget man –  teslasimus Dec 5 '12 at 19:12
    
Are you sure you tried wget -nc --tries=3 -T30 --restrict-file-names=nocontrol -O 18112012/image.jpg http://site/image.jpg? I've never had problems like you're describing. –  gniourf_gniourf Dec 5 '12 at 19:17

3 Answers 3

up vote 0 down vote accepted

You can run the wget command as a background process and send a SIGKILL to forcibly kill it after sleeping for a certain amount of time.

wget ... &
wget_pid=$!
counter=0
timeout=60
while [[ -n $(ps -e) | grep "$wget_pid") && "$counter" -lt "$timeout" ]]
do
    sleep 1
    counter=$(($counter+1))
done
if [[ -n $(ps -e) | grep "$wget_pid") ]]; then
    kill -s SIGKILL "$wget_pid"
fi

Explanation:

  • wget ... & - the & notation at the end runs the command in the background as opposed to the foreground
  • wget_pid=$! - $! is a special shell variable that contains the process id of the most recently executed command. Here we save it to a variable called wget_pid.
  • while [[ -n $(ps -e) | grep "$wget_pid") && "$counter" -lt "$timeout" ]] - Look for the process every one second, if it's still there, keep waiting until a timeout limit.
  • kill -s SIGKILL "$wget_pid" - We use kill to forcibly kill the wget process running in the background by sending it a SIGKILL signal.
share|improve this answer
    
very nice! thanks chen –  teslasimus Dec 5 '12 at 19:03
1  
What if wget succeeds? You'll throw randomly a SIGKILL? –  gniourf_gniourf Dec 5 '12 at 19:08
    
how can i skip "sleep" if file has been already downloaded? –  teslasimus Dec 5 '12 at 19:09
1  
Rewrote the script to make it a bit more robust and address the skipping of sleep times issue –  sampson-chen Dec 5 '12 at 19:17
    
@gniourf_gniourf is there evidence that it's harmful to send a SIGKILL to nothing? –  kojiro Dec 5 '12 at 19:25

Community wiki because this is mostly a copy of sampson-chen's answer, but I wanted to point out a couple of things:

wget ... &
# Strictly speaking you can just use the job number,
# which is probably %1, but saving the pid is also fine.
wget_pid=$! 
counter=0
timeout=60
# use kill -0 to check if a pid is still running
while kill -0 "$wget_pid" && (( counter < timeout )); do
    sleep 1
    (( counter++ ))
done
# if killing nothing is distasteful, use kill -0 one more time.
# also think a SIGKILL is overkill since the question doesn't imply wget needs it.
kill -0 "$wget_pid" && kill "$wget_pid"
share|improve this answer

Easiest way is to use the timeout(1) command, part of GNU coreutils, so available pretty much anywhere bash is installed:

timeout 60 wget ..various wget args..

or if you want to hard-kill wget if its running too long:

timeout -s KILL 60 wget ..various wget args..
share|improve this answer
    
You do know bash is installed on every Mac, which do not have a GNU userland, right? –  kojiro Dec 5 '12 at 19:22
    
@kojiro: The question is tagged ubuntu which includes GNU coreutils, and it can readily be installed on a mac if you want it. –  Chris Dodd Dec 5 '12 at 19:24
    
Hmm, maybe it should've been migrated to askubuntu.com. >;) –  kojiro Dec 5 '12 at 21:24
1  
This is definitely the easiest and safest way. man timeout will show other options. I would personally not use the -s KILL option directly, but maybe use the -k option, something like timeout -k 5 60 wget ..various wget args.. (to give a 5 extra seconds chance to wget before being KILLed). This answer deserves +1. –  gniourf_gniourf Dec 6 '12 at 7:49

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.