0

I want to iterate through a list of bash scripts with a cron-job. I.e. every two hours I want to start another bash script. My idea was to set a new environment variable, each time the cronjob script reads this environment variable and calls the corresponding script and and increments the environment variable.

I tried this, and it works if I call test.sh from the terminal.

# test.sh
JOB_COUNTER=$((JOB_COUNTER + 1))
echo $JOB_COUNTER
export JOB_COUNTER=$JOB_COUNTER

If I put call it from within crontab, $JOB_COUNTER does not seem to get incremented (I check it with echo $JOB_COUNTER)

Context: I want to add 50 new jobs to our cluster every two hours, because if I would submit all jobs together, the scheduler does start too many jobs in parallel and most time out because the I/O is too slow. If a cronjob is not what one would use, I am open for other ideas.

2
  • Wouldn't it be simpler to write a simple Python (or even shell) script, which just runs through the list? Just do a sleep() in the script - that doesn't take any CPU time.
    – jcoppens
    Apr 9, 2015 at 4:48
  • Where do you initialize JOB_COUNTER? Also, remember that cron creates a new shell for each execution. You need to make JOB_COUNTER persistent, e. g. write it to the cron's user .bashrc.
    – bjanssen
    Apr 9, 2015 at 5:38

1 Answer 1

0

each test.sh called by cron is a new process, so environnement variable are dropped.

Try

# test.sh
JOB_COUNTER=$(< $HOME/jobcounter.txt )
JOB_COUNTER=$((JOB_COUNTER + 1))
echo $JOB_COUNTER > $HOME/jobcounter.txt
echo $JOB_COUNTER
# JOB_COUNTER can be used here 

beware that you cannot use JOB_COUNTER in calling shell unless test.sh is sourced ( . ./test.sh )

you will have to initialize and reset $HOME/jobcounter.txt

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .