Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I often have to transfer 4000 small files to a remote server (around 20 MB of data), which I currently only have FTP access to. This operation takes 30 minutes but if I were to send one file sized 20 MB it would take 2 seconds or so. I am looking for an alternative which is faster, and which for example zips the 4000 files into one file, sends the zip file, and unzips it at the remote machine. Are there any file transfer servers that do this?

Note that I have access to installing the alternative server on the remote machine, so i am not limitted to the FTP server.

Thank you very much in advance Jeeji

share|improve this question

migrated from Oct 20 '10 at 18:08

This question came from our site for professional and enthusiast programmers.

This should really be on Superuser, unless you are trying to do this in code - in which case, you need to put your code in the question. – adam Oct 20 '10 at 13:32
This should probably be moved to indeed. It would be useful to say what kind of client/server environment you're using (Windows, Linux, ...)? – Bruno Oct 20 '10 at 13:36

Zip them up and send them to the server and then use a scheduled task on the receiving side to find the zips and unzip them locally. Using a file system watcher you can detect when a zip is created, wait till it's done uploading (i.e., is no longer modified) and then unzip it.

share|improve this answer

You say you only have FTP access but then say you have the ability to install an alternate server. Can you install and use rsync?

Rsync will run over ssh and was designed to handle the "lots of tiny files" problem by sending them as a continuous stream of data and not as discrete chunks each with their own overhead.

share|improve this answer

You should have a look at weex or use rsync over the mountpoint created with CurlFtpFS

share|improve this answer
Good idea, I tried the same solution yesterday. CurlFtpFS is painfully slow, which makes very impratical to use cp or rsync (at least the first full copy) over there. But the solution is clean and worth a try anyway. – Paolo Aug 25 '15 at 18:44

Thanks for your suggestions. I thought of solving the problem myself as you suggest using a file system monitor that triggers when a zip file is copied. However, it requires some programming, since I need it to run as a service (so that it starts with no user logged on), and I need it to return some status of the result. All this is possible to program, but it would however be a lot easier if there is an off the shelf solution.

I will look into rsync. It sounds exactly as the application I am looking for.

share|improve this answer
Jihad - associate your Stack Overflow and Super User accounts and you'll regain ownership of the question – ChrisF Oct 22 '10 at 18:29

A simple shell script on the server end will take care of this for you. Pick a folder in which every zip file that goes in gets uncompressed. Write a script to find the zip files, and unzip them.

share|improve this answer
For flexibility, I'd write it so that it'd wait until it found a and a foo.txt, where the foo.txt contains an absolute path stating where to unload the .zip. Put some marker in the .txt file so the script won't try to read an only half-uploaded txt and put the zip in some screwed up location. – doppelgreener Oct 21 '10 at 5:09

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.