I have 100,000 URLs of small files to download. Would like to use 10 threads and pipelining is a must. I concatenate the result to one file. Current approach is:
cat URLS | xargs -P5 -- curl >> OUTPUT
Is there a better option that will show progress of the whole operation? Must work from the command line.