Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm downloading a large number of small files (over 800, each around 5k). I'm doing this with lftp's mget command. But it's rather slow. Each file takes roughly 1s. Is there a tool/way to speed this up?

Thanks for all the reply but I only have ftp access to this server

share|improve this question
If it is slow due to the number of files: Either tar/zip them and download them in one go, or use a stream (e.g. tar piped though netcat to untar) – Hennes Apr 7 '13 at 11:04
@Hennes: can you tar them on the ftp server? How can you do ftp download through netcat? – lang2 Apr 7 '13 at 11:05
Can you log in on the server running FTP? If you can try this: On the receiving host: nc -l 7654 | tar x, on the server which also runs ftp: tar cf - | nc -p receiving_host 7654. Mind you, that is not over FTP, but FTP needs do die. It is old, and should have been replaced decades ago. – Hennes Apr 7 '13 at 11:09

If you have command line access to the server, log on and bundle them into one file with zip, gzip, bzip or tar (whichever is available, and can be reversed on the destination -- you didn't say what OSs are involved). This reduces the number of files _and _ the total number of bytes to be transferred.

If you only have FTP access to the server: some servers are configured to deliver a zip/gzip archive (which they build on the fly) if you just ask for it. E.g., if your files are in, try fetching It might not work, but it's worth a try.

share|improve this answer
This thing about servers automatically compressing folders is a nice idea, but does that actually happen??? Can you provide an example of a server that does this? – foobarbecue Jun 30 '15 at 21:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .