1

I recently upgraded to Rtools-3.5 (ancillary to this question) which includes rsync-3.1.3. On a different laptop using Rtools-3.4 (rsync-3.1.1), a simple command of rsync -aHAX remote:/path/to/dir/* . works just fine, but with 3.1.3 it does not. To troubleshoot, I copied the rsync-3.1.1 executable (with accompanying cyg*.dll) from Rtools-3.4, so now I have two sets of executables: 3.1.1 (old, working) and 3.1.3 (new, not working). The 3.1.1 version works.

The ssh configuration is identical, ssh-agent is working fine (via KeePass/KeeAgent), and ssh-ing (from Git for Windows, version OpenSSH_7.7p1, OpenSSL 1.0.2o 27 Mar 2018 if it matters) to the remote server works without issue.

I thought another question "https://unix.stackexchange.com/questions/389908" would be similar-enough, but in that question, all versions of rsync are the same.

Older version:

$ ~/rs/rsync --version
rsync  version 3.0.9  protocol version 30
Copyright (C) 1996-2011 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    no socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes

Newer version:

$ /c/Rtools/bin/rsync --version
rsync  version 3.1.3  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, no xattrs, iconv, symtimes, prealloc

I try rsync -e 'ssh -vv' ... and nothing seems amuck: the log reports no problems, the keys are accepted, and it ends with:

debug2: input_userauth_pk_ok: fp SHA256:g2VNSDTf5fNp46rZUo4GmAVi84gSNA9nNbT9KKfGXP0
debug1: Enabling compression at level 6.
debug1: Authentication succeeded (publickey).
Authenticated to my.remote.host ([11.22.33.44]:22).
dup() in/out/err failed
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.3]

The same command with 3.1.1 works fine. That error often indicates that the remote dir/file does not exist, but that's obviously not the case (since 3.1.1 works).

It appears that Rtools has changed their compilation options, because comparing the capabilities of the two reveals some differences:

## Versions:       3.1.1                          3.1.3
$ diff -uw <( ~/rs/rsync --help) <( /c/Rtools/bin/rsync --help) | head -n 20
--- /dev/fd/63  2018-08-12 17:48:25.000000000 -0700
+++ /dev/fd/62  2018-08-12 17:48:25.000000000 -0700
@@ -1,10 +1,10 @@
-rsync  version 3.0.9  protocol version 30
-Copyright (C) 1996-2011 by Andrew Tridgell, Wayne Davison, and others.
+rsync  version 3.1.3  protocol version 31
+Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
 Web site: http://rsync.samba.org/
 Capabilities:
     64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
-    no socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
-    append, ACLs, xattrs, iconv, symtimes
+    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
+    append, ACLs, no xattrs, iconv, symtimes, prealloc

So they've added socketpairs and removed xattrs. Other pages I've read lead me to believe it has to do with xattrs, but I'm not wholly convinced of this, and I don't know how to really test it.

The remote end is:

$ rsync --help | head -n 20
rsync  version 3.1.1  protocol version 31
Copyright (C) 1996-2014 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc

Further suggesting that xattrs might be the key.

Looking at Rsync Security Advisories, they suggest three things that seem possibly related:

If you are using a version of rsync older than 3.1.3 as a client and receiving xattrs from an rsync server that you might not fully trust, a malicious (modified) server could send a non-null-terminated xattr name to overflow the xattr read buffer.

If you are running a writable rsync daemon older than 3.1.3, you should add a rule "refuse options = protect-args" if you don't fully trust the users who are sending you files.

If you're using a version of rsync older than 3.1.2 as a client and receiving files from an rsync server that you might not fully trust, this version adds extra checking to the file list to prevent the sender from tweaking the paths and/or the transfer requests in a way that could cause a file to be received outside the transfer destination.

I've tried some combinations of variables (e.g., -X) to try to remediate this, but can find nothing to get the 3.1.3 version to work. In the interim, I can use 3.1.1 to do what I need to do, but what else can I check and/or change on the command line to get things working?

(win10_64, GfW-2.18.0, remote is ubuntu-16.04 with rsync-3.1.1)

  • Can you rsync the other way? (If you reverse the source and destination, you can run the otherwise-identical command on the destination machine and it will pull the data from the source machine.) – Jim MacKenzie Aug 13 '18 at 3:31
  • Interesting thought, I hadn't considered it. Primarily, though, I'm not going to try to rsync from ubuntu (client) to windows (server) ... I believe it's possible to run some form of ssh/rsyncd on windows, but that's well outside of my design. Thanks! – r2evans Aug 13 '18 at 4:14
  • I've no experience in doing it with Windows, but if your machine can run sshd, it should be possible to run clients on it remotely, making it possible. – Jim MacKenzie Aug 13 '18 at 4:18
  • Thank you, no. The fact that I'm running into this problem means I will run into it with others' computers, and I believe that asking everybody else to run an sshd server on their windows box is a hack-fix for this problem. Also, I'm familiar-enough with running things on windows that I would not trust running this, as much a statement about my comfort-level with servers on linux versus comfort-level with servers on windows. – r2evans Aug 13 '18 at 4:22
  • I was thinking of it more as a diagnostic tool. It might help you learn more about your problem. I didn't intend it as a permanent solution, although Unix-like to Unix-like OS, it might work well. – Jim MacKenzie Aug 13 '18 at 4:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.