chained commands to create directories on a remote server

Question

To get rsync cronjob working, I need to first create the dir tree on the remote server before running rsync. The local directories shall mirrored (w/o content) to the remote server. I'm strugglying to do that.

What I need is stg like this :

find /home/local_directories/onedir/* -type d -exec sh -c "ls -dlts {}"  \; | 
    cut -d" " -f10 | uniq | ssh user@server mkdir -p /tmp/{xargs}  

The hard point is how to get the last pipe running ?

EDIT: the rsync command is like this (tried also with -R):

/usr/bin/rsync -e ssh -arz --stats --force /home/local_directories/onedir/ user@server:port:/exitsing/path/to/remote/dir/  

The source tree is a svn hotcopy (/conf, /db, /hooks, /dav, ...)

Thx

Answer 1

It's not clear why you need to make all those directories using mkdir.

The only directory you should need to create is the destination directory itself, and that's only if it doesn't already exist. rsync will create all the subdirs if you use the -a or -v option.

Something like this should be sufficient:

destdir="/home/local_directories/onedir.bak"  # or wherever it should go
ssh user@server "mkdir \"$destdir\""
rsync -a /home/local_directories/onedir/ user@server:"$destdir"

Or more correctly (only making the directory if it doesn't exist):

destdir="/home/local_directories/onedir.bak"
ssh user@server 'destdir="'"$destdir"'"; test -d "$destdir" || mkdir -p "$destdir"'
rsync -a /home/local_directories/onedir/ user@server:"$destdir"

---

But, if you really have to do this, and for some reason the ls -ltds really is required, then you would have to fix your {xargs} bit to get it running.

The most correct and efficient way I can think of to do the last bit is:

find ... |
    ssh user@server 'while IFS=$'"\n"' read -r dir; do test -d "$dir" || mkdir -p "$dir"; done'

The idea behind this is that we only start find and ssh once. We start find and ssh together in a pipeline, with find sending its output to the remote server over ssh.

find produces one line of output per file, then the while read loop on the other side reads one line at a time, and does something with it (in this case, makes the directory).

If you were using xargs, it would be running ssh multiple times, which will make things slower.

Better, but a bit more complicated, is:

find ... -print0 |
    ssh user@server 'while read -r -d "$(printf "\000")" -r dir; do test -d "$dir" || mkdir -p "$dir"; done'

It's better because it handles weird characters in files, including newlines, which are very rare, and a bad idea, but better to handle them just in case.

---

So as far as I see it, you should just be doing something like:

find /home/local_directories/onedir/ -type d -print0 |
    ssh user@server 'while read -r -d "$(printf "\000")" -r dir; do test -d "$dir" || mkdir -p "$dir"; done'

Answer 2

I don't quite understand what you're trying to achieve here. I use rsync all the time and have never had to pre-make any directories on the server.

rsync -avP /home/local_directories/onedir/ user@server:/where/to/sync/to/

should just mirror the whole tree under onedir from local to remote.