rsync: avoiding deleting files that do not belong to a certain user

Question

I am using rsync to keep a large shared document folder in sync with my laptop. More specifically, I am rsyncing FROM my laptop TO the shared repo. What I would like to do is rsyncing only those files that have been created/modified by my user. Being a shared repository other people might have created other directories or files.

I have browsed through the various rsync options but I haven't found what I am looking for. I can --exclude or --delete-excluded but only giving patterns to match the filename, not any other metadata (or can I?).

An alternative, but somewhat cumbersome solution could be launch a remote script like find -not -perm $USER > excluded_files and rsync excluding those, but I'd rather not do that.

Any ideas?

update: to clarify, the data on my laptop will be only part of the data on the server. Here is an example

• my laptop contains dir_1 and dir_2
• I rsync them to the server; now the server contains two dirs
• another user logs onto the server and creates a new directory dir_3
• I mess up with my files, now will rsync again but need to use --delete
• I want to --delete ONLY the files that I created on the first place, i.e. those on dir_1 and dir_2, not dir_3

Answer 1

I assume that keeping a copy of all the files on your laptop is not an option. Otherwise, it would be a lot easier.

The right tool here is a version control tool. But I realize that unless all users have a minimum amount of technical sophistication, getting them to use version control tools is not a realistic suggestion. And even then you'd have to carefully pick what you check out on your laptop.

You might still use version control. On the shared machine, check in all of your files, and leave other people's files alone. Keep a separate checkout on your laptop. Commit and check out whenever you switch machines. The more I think about it, the more I like this option: it doesn't require any coding or even any fragile scripting.

---

You do two-way synchronization. Rsync is not good at this: if you forget to run it, or accidentally run it in the wrong direction, you're likely to lose work with no warning. Unison is a good tool for two-way synchronization: it always synchronizes bidirectionally, and it complains if a file has changed on both sides. It's open source, and well integrated into both unix and Windows platforms.

Unfortunately, unison doesn't have an option to ignore files by owner, any more than rsync does. You can generate an ignore list with find -user, but it gets messy, and there's a race condition if someone else adds or delete files between the run of find and the run of unison or rsync.

You could patch rsync or unison to add the option to ignore files based on their metadata. I just had a quick look at the source, and in both cases the exclusion code is strongly tied to strings, not to directory entries. It looks doable, but it's not a trivial patch.

---

You could create a view of the filesystem that contains only your files. I don't have a complete, seamless solution, but here are a couple of ideas to get started.

• If the shared machine supports hard links (all unices do, and Windows does since NT4 as long as you use NTFS), it's easy enough to create a copy of the repository containing only hard links of your files:

  cd /shared/repository
  find . -user "$USER" -exec sh -c 'mkdir -p "/my/view/${0%/*}" &&
                                ln "$0" "/my/view/$0"' {} \;
  

  or with zsh:

  autoload zmv
  zmv -L -Q '/shared/repository/(**/)(*)(u$UID)' "/my/view/$1$2"
  

  You'd have to be careful not to break hard links when editing or synchronizing. I don't know how rsync and unison would cope. After the initial creation, a file in the shared repository that is owned by you and has a link count of 1 is to be deleted, and a file in your view with a link count of 1 is to be linked in the shared repository. This is all quite clumsy, and I would welcome suggestions for improvement.

• If the shared machine supports FUSE, you could use it to create a live view of the shared repository containing only your files. I don't know of any existing FUSE filesystem that can do it, but bindfs comes close (I haven't looked at the code to evaluate the coding difficulty).

• You could use LD_PRELOAD to override readdir (or a Windows equivalent) when running rsync or unison, so that it would only return files owned by you.