I'm using rsync to essential fetch files off a server, then delete the files from the server once I have them locally. The full command I'm running is below.

This does successfully delete the files on the source server, however empty directories still remain. I don't get any messages or errors. All output is normal. Perhaps this is the intended functionality.

How can I tell rsync to clean everything, including the directories?

rsync --progress -vrzh --remove-source-files

Version is 3.0.9 on both ends.

up vote 12 down vote accepted

The behavior of --remove-source-files that you observe is exactly that specified by man rsync:

--remove-source-files

   This tells rsync to remove from the sending side the files (meaning non-directories) that are a part of the transfer and have been successfully duplicated on the receiving side.

There is no specific command to remove the directories, as these two discussions in StackExchange and ServerFault clearly show. The solution suggested there is to issue two separate commands:

 rsync -av --ignore-existing --remove-source-files source/ destination/ && \
 rsync -av --delete `mktemp -d`/ source/ 

The last piece of the command suggested in these two posts,

 rmdir source/

which is needed to remove the (now emptied) source directory has this form in these posts because the OPs and the answers are using rsync to move large amounts of files within the same machine. In your case you will have to do this manually.

  • 3
    The rsync --delete suggestion is dangerous, because it ignores the possibility that the rsync wasn't completed, or there are new files at the source. @slhck's find method below is much safer. – Sai Jul 5 '17 at 9:08

The manpage even says:

--remove-source-files   sender removes synchronized files (non-dirs)

If you want to remove empty directories in your source, if there are still files left, do a:

find . -depth -type d -empty -delete

But for an empty root directory, a rm -rf <directory> will of course suffice.

  • 4
    yeah, this is the only solution. it's kind of a silly missing feature of rsync... rsync knows when it's processed the last file in a directory... it's easy enough to remove the dir too if it's empty. – Erik Aronesty Jun 23 '14 at 19:10
  • 4
    Beware that issuing "rm -rf" is race-condition prone and I discourage it. – Raúl Salinas-Monteagudo Mar 24 '15 at 16:38

Using "rm -rf" has an inherent race condition, you could namely delete files that were just created between the rsync and the rm invocations.

I prefer to use:

rsync --remove-source-files -a server:incoming/ incoming/ &&

ssh server find incoming -type d -delete

This will NOT remove the directories if they are not empty.

  • 2
    The rm -rf will also remove files that were not transferred for some reason. – Kristian Jun 16 '16 at 21:23

-m, --prune-empty-dirs prune empty directory chains from file-list

--force force deletion of dirs even if not empty

  • That just prevents rsync from copying empty dirs. It doesn't delete empty dirs. – Navin Jul 23 '17 at 4:45

Remove source files, then remove directories to be safe.

# given this scenario where you generate folders 2014-01-01 etc.. that have an archive myfile.tar.gz
pushd $(mktemp -d)
mkdir 201{4..6}-{01..12}-{01..31}
for i in $(ls); do; touch $i/myfile.tar.gz;done;
# find and rsync on 10 CPU threads directories that match ./2015-*
find /tmp/tmp.yjDyF1jN70/src -type d -name '2015-*' | \
parallel \
--jobs 10 \
--progress \
--eta \
--round-robin \
rsync \
--hard-links \
--archive --verbose --protect-args \
--remove-source-files \
{} /tmp/tmp.yjDyF1jN70/dest
# now safely remove empty directories only
for i in $(ls /tmp/tmp.yjDyF1jN70/src); do; rmdir /tmp/tmp.yjDyF1jN70/src/$i; done;

More on GNU Parallel

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.