Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've stared mirroring the files and directories on an FTP server to my external HDD using a Raspberry Pi.

I used the following commands:


mirror --use-pget-n=8 /

The files are downloading, however it seems to be running quite slow. After about 10 hours of running, only 139GB has been downloaded. Whenever I ran the download using Filezilla on my desktop (running Linux Mint) I downloaded 200GB in 4 hours.

I was unaware of how many segments each file should be downloaded in, so I choose 8. I'm not entirely sure of the benefit splitting the file into segments however.

My download speed according to peaks at around 40Mb/sec and is usually around 30Mb/sec.

Are there any parameters I could use to improve performance, or is it down the Raspberry Pi's hardware?

I would archive all the files and download them in one go, only I don't have any other access to the server.


share|improve this question
Is there a specific reason you don't want rsync? – terdon Jan 16 '13 at 22:14
I don't have any other access other than FTP as I don't own the server. The files are all public, but I only have access via FTP. – Torvero Jan 16 '13 at 22:31
Ah, OK. I would first compare the speed of the desktop and RPi by downloading the same file a few times. I mean run the same FTP command on both machines. That way you'll know if it is a hardware limitation. Also, some of the programs suggested on this thread. might help. – terdon Jan 16 '13 at 22:41
up vote 1 down vote accepted

The fastest way do download the files from that FTP server would be to run lftp on the server and download the files over the loopback interface.

Your question is imprecise (leave that aside for now).

Speedtest gives bits per second. 40 Mbit/s is 5 MByte. 5 MB/s * 3600 sec * 4 = 72k MB or roughly 70 Giga bytes. If your speed was 40 MB/s (320 Mbit/s), then you would have downloaded roughly 560 giga bytes in four hours. I don't think even the newest consumer grade hard drives can write that fast, so you would need an SSD drive. Are those even available in 500 or more gigs?

You are seeing a bottleneck of 47 MB/sec with the described configuration and the most likely source of that is (if not the hard drive itself) the USB connection of the external hard drive. However, you did not state that the drive was connected via USB. The RasPi could be downloading to a network drive, for all I know. In that case, you would still be limited by the USB 2 theoretical 60MB/s limit because the ethernet network on the RasPi is actually a USB adapter.

My comments are inconclusive, but I think it's reasonable to conclude that the RasPi hardware is to blame for the bottleneck.

File segmentation becomes more important when you are downloading over an unstable connection (or UDP). If your net does not frequently fall out while one file is being transferred, using application level segmentation will give you nothing but a few extra check sums. Actually, in all likelihood, you will not even see those.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .