How do I specify a large list of files I need to sync to rsync?

Question

I have a list of files and destinations that I need to sync with rsync but I do not want to run rsync several times for that (connection time is big due to ssh usage in my case).

Example:

   ~/aaa    remote:/~/aaa
   ~/bbb    remote:/~/otherplace/bbb
   …

As you can imagine the list is bigger and in my case I do have to run the same set with multiple remote servers.

How can I implement this easily? I want rsync to start once per server.

Answer 1

I did a quick fix for this once when I wanted to crowbar this behavior into an existing rsync script that only transferred to a specific folder: I made a catalog of symlinks for this purpose on both sides and told rsync to dereference links (-L) during transfer. On my side, the links were dereferenced into their contents and transferred to the directory on the remote side, where they were put into the symlinks that pointed to the directories where I wanted them.

This is actually not the worst way to specify files for backup. It is versatile and easily customizable without changing configuration files (well, the structure of symlinks is the configuration file, in a way).

This specific example is not a good solution if you want to not dereference links for transfer in certain cases, though.

Answer 2

If ssh is the problem, then maybe you can speed things up by reusing the ssh connection, put this in ~/.ssh/config or some other relevant ssh config file:

Host *
  ControlMaster auto
  ControlPath /tmp/%r@%h:%p