Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am looking to download multiple files from a server using cURL from the Terminal on a Mac. I know that using the {} operator, I can load one file after another but I am looking to download all (10) of them using multiple connections at the same time. Is there a way to do this (note: Not using PHP which seem to have a bunch of answers here)?

share|improve this question

migrated from stackoverflow.com May 4 '11 at 12:09

This question came from our site for professional and enthusiast programmers.

    
Flagged for relocation to superuser site. – ewindisch May 3 '11 at 21:16
up vote 2 down vote accepted

You can do this from your shell:

cat <<URLS | xargs -P5 -n1 curl -O
http://www.example.com/
http://www.example.net/
ftp://ftp.example.org/
URLS

This will download one file per curl process, with a maximum of 5 concurrent curl workers/processes.

Note that with xargs, it will separate the input by both spaces and new-lines. Since your URLs shouldn't contain spaces (or should have them encoded), this shouldn't be too significant.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .