I prefer to use tar over ssh as opposed to scp for several reasons, e.g.
tar -czf - file1 file2 ... | ssh host "cd ~/destination && tar -xzf -"
However, I sometimes miss the scp progress bar, especially when copying large files.
I learned about the pv command which looks promising, however I am having trouble getting it to work as desired. For uncompressed archives, the following reports an incorrect completion percentage because the resulting tar archive is generally larger than its individual files:
tar -cf - <files> \
| pv -s $(du -sb <files> \
| awk '{sum+=$1} END {print sum}' \
| ssh host "cat - > files.tgz";
Thus the completion percentage in various tests ranges from 105% complete all the way up to 250% complete (larger inputs result in significantly larger output files). On the other hand, using compression results in a completion percentage less than 100%. These problems stem from the fact that I am telling pv the size of the original input files, not the actual size the output generated by tar which I am transferring.
Essentially, my question is this: Is there a way that I can more accurately measure the size of the tar output and send that value as an argument to pv -- an alternative to using du -- OR is there a better way to measure the progress of the transfer -- an alternative to using pv?
edit split the second command onto multiple lines
