I am using curl to get a URL then writing to a file, like this:
urls=(
'https://www.example1.com'
'https://www.example2.com'
)
for i in ${urls[@]}; do
curl $i &
done
echo 'stuff'
I have deliberately simplified the code, so the exact problem can be tackled.
Output:
stuff
$curlContents1
$curlContents2
I know why this happens, it’s running asynchronously.
What I want to know
- I want to run this async cmd with the output the same as it would be if I had ran it sync.
- This is because running it async gives a nice speed boost
Desired output:
$curlContents1
$curlContents2
stuff
more info
- my actual problem is a bit different…
What I’m doing is downloading videos then taking the last part of the URL and using it as the file name, how can I use parallel
in this example?
The write happens before the download as the downloads are the most time consuming part
arr=(
'https://www.example1.com/stccdtu.mp4’
'https://www.example2.com/dyubdf.mp4’
)
for i in ${arr[@]}; do
curl $i > `echo $i | sed s'#https://www.example[0-9].com/##'g` &
done
output:
ll
0 stccdtu.mp4
0 dyubdf.mp4