Transfer using linux ssh and maintaining permissions

Question

I need to transfer files across ssh to another server. The file structures are identical on both sides. I have used scp -r but that does not retain the orginal file/dir permissions. rsync does the job of keeping the permissions in tact but does not delete the files on the destination side if I want to overwrite them because of changes. I know rsync will write the changes when the source files are newer but I need it to just copy everything reguardless of the date (ie replace destination directory with the one I am moving) without having to shell into the destination first and manually delete the dir.

I heard tar can do this but I can not seem to get it to work without errors. The syntax is

tar -cf - /directory/directory | ssh host.name tar -xf - C /destination_directory

Any help would be appreciated.

Answer 1

rsync can delete files on the destination using the --delete syntax. If you want it to overwrite a file in it's entirety versus it's partial sync implementation, try --whole-file.

Excerpts from http://www.samba.org/ftp/rsync/rsync.html:

-W, --whole-file With this option rsync's delta-transfer algorithm is not used and the whole file is sent as-is instead. The transfer may be faster if this option is used when the bandwidth between the source and destination machines is higher than the bandwidth to disk (especially when the "disk" is actually a networked filesystem). This is the default when both the source and destination are specified as local paths, but only if no batch-writing option is in effect.

--delete This tells rsync to delete extraneous files from the receiving side (ones that aren't on the sending side), but only for the directories that are being synchronized. You must have asked rsync to send the whole directory (e.g. "dir" or "dir/") without using a wildcard for the directory's contents (e.g. "dir/*") since the wildcard is expanded by the shell and rsync thus gets a request to transfer individual files, not the files' parent directory. Files that are excluded from the transfer are also excluded from being deleted unless you use the --delete-excluded option or mark the rules as only matching on the sending side (see the include/exclude modifiers in the FILTER RULES section).

Answer 2

As Eddie pointed out, rsync --delete -Ha will do the job for you. Make sure to use the --dry-run/-n option before you actually run it to avoid accidentally deleting everything in the wrong directory. Rsync interprets your directories differently depending on whether you append a trailing slash (see man rsync and search for "trailing"), so it's easy to get things wrong if you're not paying attention.

I also use tar sometimes like you're doing. I think you're simply missing a dash before the C in your command above. As a rule of thumb, you can only omit the dashes in the first parameter. (But you could also use cd instead of the awkward -C syntax.) With the tar on your source side you're creating a extra leading directory/ in the path names in your archive, so I'd cd first on the source side as well. The -f - is redundant by the way (tar uses stdin/stdout by default). So here's what my piped tars tend to look like:

cd /some/dir; tar c my_dir | ssh host 'cd /the/target/dir; rm -rf my_dir; tar x'

Use sudo tar on the host side to preserve ownership when extracting. (Tip: Use cz/xz instead of c/x for gzipped transfer, or pipe through lzop and lzop -d for real-time compression if you're transferring over Ethernet cable.)