0

I am downloading 180,000 files and downloading it for almost a day - just half finish and are all images sizing between 50kb - 100kb is that normal? I am using filezilla? my ISP is 3MBs

The reason I asked is when I download via http or through cPanel, e.g. zip the file and download it, it would only take me like an hour or less since the company does not provide cPanel access, I have to do it through FTP

1

Yes, this can be quite normal. FTP is not a very nice protocol, and there is lots of going backwards and forwards for each file - and this latency can add up very quickly.

Downloading via a ZIP file is way more efficient because it can optimise the transfer for a single big data block, and it can also compress the data. You have not specified the data, but I would guess that the data is very compressible.

Some maths: 180k files @75k each is 13.5 gigs of data. At 3 MB per second (which sounds suspect, ISP's typically quote megabits, not megabytes), without compression, this would take 1 to 1.5 hours to actually transfer the data. If the speed is 3 megabits, the time would be 10-15 hours... and this excludes the latency which would be a very big factor. You have not advised the latency between the sites, but lets say its 10ms round trip (which would be on the low end - you would be looking at a, at least an additional 30 minutes. I imagine in reality you would be looking at an extra 2-4 hours).

  • Do you have ssh access to the system with the files ? If so, if its a Unix based system we can still transfer the data efficiently. [ If you do have ssh access, and can advise your OS and confirm the remote side uses Linux, we can advise on how to compress and download efficiently ] – davidgo Mar 23 '16 at 7:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.