rsync vs rsh, or alternatives

Question

I have been using a plain 'ole rsync command to move large amounts of data off of some hard drives on a LAN to another computer for analysis. Usually just a one-time transfer. The format of rsync that I have been using is:

rsync -ave ssh foo@bar:/source dest

I need to improve the speed of my transfers as much as possible. Someone recently mentioned setting up rsh-server and copying the files through RSH. Not having an encryption overhead can be very helpful. I suppose I have these options:

• cp, scp, etc.
• rsync via rsh
• rsync via ssh, arcfour encryption for reducing speed
• rsyncd, no encryption, some benefits by not having to build the file list

I'm curious as to which I can expect the best performance from. I know, every case is different. Are there any better alternatives?

Some notes:

Ease of install is great. Setting up an RSH server is a bit of a pain. Rsyncd setup is reasonable. Security is not a big concern.

All are 7200 rpm drives, gigabit network with speedy CPU (at least dual core i7 2.0ghz).

Answer 1

At work, we use rsyncd to backup servers on the same LAN. I think it gives maximum transfer speed. Almost no protocol overhead.

However, same as rsync over SSH, it will take time to build the file list. Building the file list is very fast, unless for very huge file trees.

rsyncd setup is very easy and powerful. See man rsyncd.conf

Answer 2

you write 'Security is not a big concern' and 'Not having an encryption overhead can be very helpful'.

You then should use rsync without encrypting overhead. Just use a double colon (::) separator after a host specification. See man rsync for details. For 'a one-time transfer' a temporary wildcard setup on the server side may be helpful as outlined here: File copy utility or file manager with each byte report while copying?

Answer 3

TLDR

This will resume partial transfers, and add compression for faster transfers

rsync --partial --progress --archive --compress --compress-choice=zstd --compress-level=9 --checksum-choice=xxh3 user@host:~/my_file.txt .

Flag explanation

--partial: Resume partial file transfer

--progress: Show transfer progress and ETA

--archive: Preserve file attributes

--compress: Enable compression (zlib by default)

--compress-choice=zstd: Enable zstd compression (faster and better compression than zlib)

--compress-level=9: Increase the compression level from the default of 3 (tradeoff vs maximum of 19)

--checksum-choice=xxh3: Use xxh3 hashing algorithm (very fast)

Answer 4

If you need to improve speed of transfer , use command with -z options , its for compression.

rsync -avze ssh foo@bar:/source dest

The compression is done using zlib libraries,so the transferred number of bytes will be less.