Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

So why are FTP uploads soo slow? I am using filezila as a client.

I have like 10 mb in 1000 files and I can upload each individual file with 300-500kb/s yet the whole upload is incredibily slow due to the queueing process that occurs as files are uploaded. For every singe file the client performs all kind of commands and connection operations before actually uploading.

Is there no way to skip over these commands? I am new to ftp clients/uploads/websites etc Is this standard practice? Is this just the way ftp uploads work? Don't you get bored waiting like 20 minutes for 8-10 mb?

How can I efficiently upload 100 mb or more?

share|improve this question
6  
This doesn't appear to be a programming question stackoverflow.com/faq Voted to close and move it... –  John K Jun 13 '11 at 17:43
    
Transferring a large number of small files is much less efficient for any protocol, not just FTP. It's just much more noticeable over the internet (or a WAN) because the latency makes the command overhead swamp the actual transfer rate. –  afrazier Jun 14 '11 at 17:59
add comment

migrated from stackoverflow.com Jun 13 '11 at 18:07

This question came from our site for professional and enthusiast programmers.

3 Answers

Sadly, this is the way that FTP functions. To efficiently transfer lots of small files, either archive them locally, transmit the entire archive via ftp, and then unarchive the files on the remote machine, or turn on simultaneous uploads, where the client is uploading 10 files at once. This will help to fully saturate your upload link.

share|improve this answer
    
with an avg. of 100kb per file they are not small files. –  Captain Giraffe Jun 13 '11 at 17:53
    
@Captain Giraffe: On most WAN/Internet links, 100 kB is still small enough that latency swamps throughput when it comes to uploading. –  afrazier Jun 14 '11 at 17:56
add comment

Have you tried compressing the files locally then uncompressing them on the server? Then you'd only have to transfer one small(er) file.

If it's applicable, you could also only copy files that have changed since your previous upload. Tools like rsync (if you have ssh access) and robocopy (if it's a windows server) could help you do this.

share|improve this answer
add comment

I use Auto FTP Manager. It runs multiple simultaneous FTP transfers so the entire bandwidth link can be used:

Auto FTP Manager makes it easy to schedule and automate your FTP transfers. Use Auto FTP Manager to connect to any FTP server and automatically upload and download files. Plan and automate your workflow. Let your PC move or synchronize files between PC to FTP Server, PC to PC, and FTP Server to FTP Server, automatically according to a schedule...

... Auto FTP Manager is multi-threaded, allowing you to open connections to multiple FTP servers at the same time. The program can transfer files in the background while you work on other tasks.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.