8

Is it possible with rsync to not create directories on destination?

Imagine I have that source :

a/
a/x.txt
b/
b/y.txt

And that I have this destination :

a/
a/z.txt

The wanted result of rsync source destination :

a/
a/x.txt
a/z.txt

Of course my real situation involves thousand files/folders structure and I don't want solutions involving explicit list of synced folders, which I can do. I'm looking for a clean way just to prevent any folder creation on destination. By exclude or filtering... That could even be something outside rsync, like a hack with permissions if rsync can't do this...


For information, this is really easy to get this kind of situations, in my case I have:

  • A server with 2 disks, let's say A & B. And a local drive C.
  • I usually use rsync to sync (and merge) remote A & B into local C.
  • Then sometimes I just want to sync back some C files into A and B. (Just new Files... not non-existing folders on destination)
3
  • rsync alone cant do this, I think (the option --ignore-non-existing would skip files and folders). You need to execute multiple steps to achieve that.
    – mana
    Jul 28, 2011 at 10:31
  • that's what I'm afraid of... but that reallly sounds strange that rsync can't do this. I do not think it is technically hard to implement and that it is a common need. But maybe I'm wrong. Next step w/o an answer > rsync mailing list...
    – Vincent
    Jul 28, 2011 at 11:17
  • I'd be interested in seeing the solution if you find one. I'm out of ideas myself atm.
    – jw013
    Aug 3, 2011 at 10:01

5 Answers 5

2

Try the following:

rsync -av src:/dir/to/files/* /dest/dir

If you have multiple nested subdirectories, you'll need to do something like this first:

find /dir/to/files -type d -print

Run that list of directories through a loop and call rsync each time. Of course, assuming you're doing this on a remote machine, you'll need to issue the find via ssh and store the results in an array and then execute the rsync loop.

1

I implemented this for rsync now, and submitted the patch to upstream: https://lists.samba.org/archive/rsync/2015-November/030455.html

2
  • What ever happened to this? Sep 23, 2023 at 5:48
  • Upstream rsync developers never responded to me. I've had several patches submitted and just no response from upstream at all.
    – robbat2
    Sep 24, 2023 at 6:05
0

maybe do a normal rsync and then in a second step delete all newly created folders ... sound dangerous.

To increase the safety of this operation you can use another user than the one on the destination system:

rsync $src $specialuser@server:$destination

Deleting the remotely created folders by that user:

ssh $normalUser@destination "find $destination -type d -user $specialuser -exec rm -r {} \;"

And then set the permissions back to normal:

ssh $normalUser@destination "chown -R $defaultUser:$defaultGroup $destination"

damn, this looks dirty ... this needs some serious improvements :D

1
  • 1
    good idea, but as you said, and as I was expecting from these kind of solutions... looks a bit dirty/dangerous! :) I'm really surprised that rsync does not look like capable of this OTB.
    – Vincent
    Jul 28, 2011 at 17:43
0

You could generate a set of filter rules on the target machine in the following way:

find /destdir/ -type d -printf 'show /%P/\nshow /%P/*\n'; echo 'hide *'

You could trun this into command line arguments or write the result to a file and include that using the merge filter rule. You could probably even do something like this:

rsync srcdir/ target:/destdir/ --filter='merge '<(ssh target \
    find /destdir/ -type d -printf 'show /%P/\nshow /%P/*\n'; echo 'hide *')

The idea is to give a list of things you want to be able to synchronize (i.e. show to rsync). This list includes existing directories and files directly therein. Anything outside those directories will be hidden from the transfer, i.e. not created on the target.

Any directory or file which exists on the target but not on the source might be deleted if you also specify --delete. If that is not what you want, you might want to use include and exclude instead of show and hide.

All of this is untested, so I won't be blamed for any data loss this causes you.

0

I'd recommend creating an include_file file on your target using a script to append a /* to each line of the result of:

find [target_dir] -type d >[target_dir]/include.file

Then, you can use rsync's --include_file=[target_dir]/include.file to specify including only the files that match a pattern in that file -- which should be a complete list of folders on the target under the point you specified.

If you create a script to do this every time, your list in include.file will always be current. Otherwise, you might decide to just refresh them (one on each server you want to rsync back to) occasionally.

If I get time later, I'll post a usable python script here, but this should give you a good path forward.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .