up vote 2 down vote favorite
share [g+] share [fb]

I need to download 30000+ .mp3 files! each of them are ~200 kb..
I tried to download all of them or in 6 sets (5000 files each time) with Internet Download Manager but it freezes.. Which download manager could handle this much files?

Links:
http://www.everyayah.com/data/Alafasy_64kbps/

http://www.everyayah.com/data/Maher_AlMuaiqly_64kbps/

link|improve this question
1  
Post the url, I'll get them with linux, 7z them (store archive) and upload it somewhere. If that's a solution. As an answer: Go with wget. If possible. The URL would help us a lot. – Shiki Oct 7 '11 at 23:14
Wow I feel like I'm asking too much. I've updated the post above with 2 links each containing 6300+ links. Thanks for the offer Shiki – Omar Oct 8 '11 at 21:16
If you can manage it, ask someone on the other end to zip them up for you. You won't get much additional compression (if any), but they will transfer much faster as a unit. – Joel Coehoorn Oct 8 '11 at 21:23
You are not asking too much, we just want to help. :) – Shiki Oct 8 '11 at 21:33
feedback

5 Answers

up vote 3 down vote

I routinely do 3k <.5MB files, and I use Free Download Manager.

I can't say if there is better or more light weight solutions out there, but, this works and does what I need - I go page by page, right click, add all to IDM, then when I finished, I highlight them all in IDM and do start!

I have never done 30k all in one go, but, I hope this helps!

link|improve this answer
FDM is a good general purpose download manager. 30k might be pushing it, though... – ta.speot.is Oct 8 '11 at 9:22
I tried FDM and its okay though it fatal-crashes a lot when I add the links, I tried to add 6000 links each time and its still crashing :( – Omar Oct 8 '11 at 21:10
@Omar - Do you get impatient or do you leave it running? 6k is a hell of a lot for any program... possibly leave it running to get it's way through your list. – William Hilsum Oct 8 '11 at 21:20
@WilliamHilsum Haha no I obviously leave it running for a long time though when I do anything after a looong time I get the fatal-crash dialog – Omar Oct 8 '11 at 21:23
@Omar - It depends what "fatal crash" dialog it is... if it is a single threaded application, the UI will be unresponsive and look like it has crashed - it will most likely come back to life after finishing... but, if you click on it, it will appear dead/unresponsive/crashed... If you have time to test, give it a 6k list and leave it overnight and see if it has worked by morning. If it hasn't I am out of ideas and clearly this program is not for you! – William Hilsum Oct 8 '11 at 21:26
show 1 more comment
feedback
up vote 2 down vote

As @Shiki already said: wget

wget -i will read all URLs from external files and process them

link|improve this answer
feedback
up vote 1 down vote

I use curl (http://curl.haxx.se/). It has a bunch of good features for mass downloads. It allows you to specify many files on one invocation, like this:

CURL.EXE http://example.com/file[1-10].mp3

That will download "file1.mp3", "file2.mp3", and so on.

I've used it to download many files, but not 5,000. However, in principle it should work.

link|improve this answer
feedback
up vote 0 down vote

Jdownloader http://jdownloader.org/download/index Is an interesting, multi-platform Java downloader that is being developed continually. It is Not lightweight at all, and could be a security risk for all I know. 30k would probably have to be done in sections. but it has a way to backup the list, and load lists in. Its main design seems to be more about downloading from "public servers", but it works good for normal stuff.

I have seen 5000 items in it, but that was because I asked it to analise a link that went on forever, ooops.

link|improve this answer
feedback
up vote 0 down vote

You can try Internet Download Manager

link|improve this answer
Did you read the questions?! – William Hilsum Oct 8 '11 at 11:21
@William oops sorry, meant download specifically 6.07 build 12...its stable – V413HAV Oct 8 '11 at 11:43
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.