0

I'm trying to download a favourite website with lots of pictures, its around 10 gigabytes in size.

I'm using sitesucker right to download it and I've noticed that although I have a 10Mbps download speed, it's taking something like two days to download it and it's still running. My question is, is downloading/mirroring a website more to do with the download speed or with the software's speed having to save and compile all the links?

I'm thinking of switching over to wget if there's more of a speed improvement; would that work?

Improve this question
1

Since you're getting data from another host, you need to have in mind your bandwidth and the host bandwidth. If the webpage server has good bandwidth, you should download it faster than if its a slower server.

Also if there are lots of files, that also makes the whole process longer, since you would always have little pauses in between files download with wget.

Improve this answer
2
  • So if the site is 10GB in size and my download speed is around 10Mbps, then although in theory I should finish it in around 2 hours, this would actually take longer right? – user2028856 Mar 27 '13 at 14:19
  • Right, you should take 2 hours to download one file of 10Gb, but since you're usually downloading lots of smaller files, it will take longer. For each file, you need to check if it exists and then download it. – Peter Mar 27 '13 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.