In the case where you are rsync'ing only one file and that file is encrypted or compressed, the only bandwidth you would likely save is that of not needing to transfer it at all if unchanged.
However, if you had a directory full of ZIP or JPEG or GPG files, rsync still only transfers those files that have changed, and is a great way to easily transfer only new files.
Note: I find it useful to rsync the uncompressed data whenever possible, and then compress it for storage on both sides of the link if necessary. In this manner, you can save yourself the transfer bandwidth. ie:
rsync -avz . remote:/tmp/somefile
ssh remote 'zip -r somefile.zip /tmp/somefile'
YMMV of course.
OT: with its backup options, I find rsync useful even when it does not save bandwidth as it will create backup copies of replaced files, allowing me to retrieve historical copies easily.
Follow-up: this applies to all formats where compression or encryption is involved, but I'm not familiar with Mathematica users.