1

I'm trying to download from the remote server to local laptop a big file (~5G), but the download stops at 40% of the size (2.4G).

I'm using this rsync command:

rsync --bwlimit=1200 -rh --progress -e 'ssh -p port' server:/path/to/file Downloads/

I also tried to lower the mtu on the server from 1500 to 1492, with no success.

Any suggestions? :D

4
  • How much RAM is available to the machine? That sounds typical of an out of memory issue in rsync. Windows versions of rsync used to be particularly prone to that issue. Jun 1, 2015 at 20:26
  • @JulianKnight Thanks for your comment, the server has 2Gb of Ram. The client is a Mac OSX w/ 8GB of RAM.
    – rdbisme
    Jun 1, 2015 at 22:04
  • I wonder how much virtual memory you have configured, probably not enough. I still suspect you are hitting memory limits. I would recommend keeping a close eye on the stats as you run it. Jun 1, 2015 at 23:21
  • @JulianKnight I really did not set the virtual memory. Just having the server as it configured at first installation...
    – rdbisme
    Jun 1, 2015 at 23:32

2 Answers 2

0

I do not believe this to be a network issue.

Much more likely to be a memory issue. With only 2GB of RAM server and running RSYNC on a 5G file, I'd suspect the server first. You can monitor the server memory hopefully as you are running it.

The client side could have issues too. I gave up on RSYNC for Windows for example because it always crashed on large files no matter how much RAM you had.

I'd have to question whether rsync was the right tool for the job on such large files. It depends on the file of course but unless any updates are amenable to rsync's diff engine, there is probably no point. A straight transfer may be better. You might even use something like zip or tar to split the file so you could send it in parts and reassemble at the other end.

3
  • Thanks for your answer. I'll check the stats so far. As straight transfer what do you mean? How should I transfer that file instead of ssh-related commands?
    – rdbisme
    Jun 1, 2015 at 23:31
  • Use SCP or SFTP both of which use SSH under the skin. I normally use WinSCP on Windows, you will need to find something for Mac. Jun 2, 2015 at 5:53
  • thanks for your answer. I thought it was clear anyway I get the same behavior even with scp: stall at 40%...
    – rdbisme
    Jun 2, 2015 at 6:29
0

please try the following:

  • lower your MTU to 1492

sudo ifconfig eth0 mtu 1492

  • disable TCP SACK

sudo sysctl -w net.ipv4.tcp_sack=0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.