I want to run a multi-parallel task in one bash file like the example code below,
for i in 1 2 3 4 5 6 7 8; do
setsid python /tmp/t.py ${i} 1>>/tmp/1.log 2>&1 &
done
wait # first wait
echo "next wait"
for i in 9 10 11 12 13 14 15 16; do
setsid python /tmp/t.py ${i} 1>>/tmp/1.log 2>&1 &
done
wait # second wait
As you can see it, is wait
possible to do this? I want to run the first 8 tasks and then wait
all the tasks to finish, then spawn the next 8 tasks because the RAM is limited, I cannot run all the 16 tasks in one round.
source
it? Have you seen this answer? Doest.py
fork? I guess your problem is the code doesn't wait when you need it to, right? (If it did, you wouldn't ask if it's possible). Simple examples do wait, your code may not. Please edit and provide MCVE. Have you considered/tried GNUparallel
with-j
?setsid
, should I alsowait
?