12

I have a source directory that looks like the following (with numerous subdirectories inside of the src and bin dirs):

  • top_dir
    • proj1
      • src/
      • bin/
      • scripts/
      • somefile.txt
    • proj2
      • src/
      • bin/
      • target
        • bin/
      • file.xml

I'd like to rsync just the bin dirs directly under the project directories (i.e., ignore top_dir/proj2/target/bin/) and their contents to the destination directory, so that it looks like this:

  • top_dir
    • proj1
      • bin
    • proj2
      • bin

I could probably make it work by explicitly listing out each bin directory (there are about 15 in my real scenario) in an include-from file, but I was thinking there must be a way to say that I want to sync all the top_dir/*/bin directories.

2 Answers 2

23

Possibility 1

Using the shell expansion1 instead of rsync's filter rules, you can do as follows:

a. To include all bin dirs in the top_dir hierarchy:

rsync -av -R top_dir/**/bin/ destination_path

Here -R is a rsync2 parameter standing for relative. man rsync explains it as

Use relative paths. This means that the full path names specified on the command line are sent to the server rather than just the last parts of the filenames. This is particularly useful when you want to send several different directories at the same time.

but in essence it means that the directory structure is preserved.

** is a shell globbing pattern and searches recursively for directories named bin under top_dir. So with the above example the command line gets expanded to:

rsync -av -R top_dir/proj1/bin/ top_dir/proj2/bin/ destination_path

b. If you only want to include bin dirs one level below the project dir, just use a single * (that works with "ancient" shells too!):

rsync -av -R top_dir/*/bin/ destination_path

1 You need a recent bash version (>4) and activate recursive globbing via shopts -s globstar. With zsh recursive globbing is default.

2 rsync version v3.2.3 protocol version 31 on Linux.

Possibility 2

Only using rsync's filter rules you need:

a. To include all bin dirs in the top_dir hierarchy:

rsync -av -m --include='**/' --include='**/bin/**' --exclude='*' top_dir/ destination_path

The first include rule might not be obvious3 but guarantees that all directories (note the trailing /) are searched, the second adds everything in bin directories, and the exclude rule excludes everything else. -m makes sure that empty directories are not copied.

b. To include only one level below the project dir:

rsync -av -m  --include='/*/' --include='/*/bin/***' --exclude='*' top_dir/ destination_path

Using *** syntax of rsync >= 2.6.7. The leading / in the patterns represent top_dir, otherwise the patterns are matched against the end of the file path.


3 Search for /some/path/this-file-will-not-be-found in man rsync

5
  • Possibility 1 worked for me, although I had to change ** to *, since there are nested bin directories I don't want to include - see the updated source tree above. I'm hoping to get possibility 2 to work, since although I have zsh, this will probably get used by others who don't have zsh or a new enough version of bash. Unfortunately, the include/exclude filter expressions you provided seem to match everything, at least with v2.6.9 of rsync. Jun 30, 2014 at 19:48
  • Actually, the Possibility 2 filter expressions are closer than I realized. They were including all of the directories, but not the files within the non-bin dirs. If I add -m, it's almost right. The command rsync -avm --include='**/' --include='**/bin/**' --exclude='*' top_dir/ dest_dir still includes the nested bin dir. Even if I change the second pattern to '*/bin/**' or 'bin/**', it still picks up the nested bin dir. Jun 30, 2014 at 20:00
  • @MattPassell Then I would definitively use Poss. 1. The * should work in every Un*x shell used today. I updated my answer with appropriate rsync rule, too.
    – mpy
    Jun 30, 2014 at 20:59
  • @XavierStuvw: Thanks for your language polishing, however I reverted the explanaition of -R as I double-checked with my version (which I included now in the answer) that -R (capital R) means relative, small -r is recursive. What version/OS are you using?
    – mpy
    Feb 18 at 18:03
  • My apologies for the blunder. You are absolutely right @mpy and my wires had to be completely crossed. Thanks for pointing me to that. Feb 19 at 11:39
0

You can exclude all src directories using --exclude src/

1
  • Sorry. I hadn't made it clear that there are other directories and files there. I've edited the file tree to clarify that. Jun 30, 2014 at 16:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.