17

I'm trying to determine the size of the files that would be newly copied when syncing two folders by running rsync in dry mode and then summing up the sizes of the files listed in the output of rsync.

Currently I'm stuck at prefixing the files by their parent folder. I found out how to prefix lines using sed and how to escape using sed, but I'm having troubles combining those two.

This is how far I got:

source="/my/source/folder/"
target="/my/target/folder/"
escaped=`echo "$source" | sed -e 's/[\/&]/\\//g'`
du `rsync -ahnv $source $target | tail -n +2 | head -n -3 | sed "s/^/$escaped/"` | awk '{i+=$1} END {print i}'

This is the output I get from bash -x myscript.sh

+ source=/my/source/folder/
+ target=/my/target/folder
++ echo /my/source/folder/
++ sed -e 's/[\/&]/\//g'
+ escaped=/my/source/folder/
+ awk '{i+=$1} END {print i}'
++ rsync -ahnv /my/source/folder/ /my/target/folder/
++ sed 's/^//my/source/folder//'
++ head -n -3
++ tail -n +2
sed: -e expression #1, char 8: unknown option to `s'
+ du
80268

Any ideas on how to properly escape would be highly appreciated.

2 Answers 2

34

You don't actually need to bother with escaping the /. From the GNU sed manual:

The / characters may be uniformly replaced by any other single character within any given s command.

For example:

echo 'foobar' | sed -e 's#foo#bar#'

gives the output

barbar
2
  • 4
    This should be the first thing in the man page for sed
    – Clearer
    Dec 25, 2014 at 2:22
  • Any as in 'any' *? Jul 9, 2015 at 12:00
0

If you are trying to tally up the data, you will likely want to skip over any possible subdirectories. This may not be applicable, but it's always best to avoid any weirdness. Here is my source directory. These files were all created with touch, so they take up 0 space.

$ find source
source
source/5832
source/5832/5832_1.png
source/5832/5832_2.png
source/5831
source/5831/5831_1.png
source/5830
source/5830/5830_3.png
source/5830/5830_1.png
source/5830/5830_2.png

You shouldn't need to escape anywhere near as much as you are doing. The sed block in here will do the following:
1d; delete the first line of 'sending incremental file list'
/^$/,$d; delete the blank line that signifies end of file list and everything after it.
//$/d delete any line ending in a '/', which would be the directories
Note that you can escape the slashes with backslashes (to answer the actual question).

$ du $(rsync -ahnv source target | sed '1d;/^$/,$d;/\/$/d')
0   source/5830/5830_1.png
0   source/5830/5830_2.png
0   source/5830/5830_3.png
0   source/5831/5831_1.png
0   source/5832/5832_1.png
0   source/5832/5832_2.png

And for extra bonus points, you can even tally up the results:

$ du -s $(rsync -ahnv source target | sed '1d;/^$/,$d;/\/$/d') | awk '{x+=$1;print $0} END {print "sum: "x}'
0   source/5830/5830_1.png
0   source/5830/5830_2.png
0   source/5830/5830_3.png
0   source/5831/5831_1.png
0   source/5832/5832_1.png
0   source/5832/5832_2.png
sum: 0

If you are doing this with the directory already fully synced, it will return funny results, though:

76  .
sum: 76

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .