I've read that sometimes files like .mp3's or compressed images aren't worth compressing (with zlib for example) because they are already compressed and they'd waste more CPU resources than save space. Are there any recommended libraries out there that can determine if a file is worth compressing before being transferred?
Compression in general works by removing redundant constructs out of the data stream in a reversible manner.
One of the measures you can use to find how much of the date can be considered redundant is autocorrelation where the lower it is the less the signal will be compressible (exceptions do exist depending on the algorithm used).
However the lowest order algorithm to find the autocorrelation is of the order of
In StoreBackup, you can define rules. A rule can be a combination of different checks, eg. depending on the file suffix. But there also is a special function, COMPRESSION_CHECK, which decides depending on the contents of a file. It does not make much sense to check every file with COMPRESSION_CHECK, therefore you can define eg. the following:
This is a behavior described in the referenced document.
(You could eg. also define rules that additionally never compresses files from a special person or group for easier restore with a file browser, but that's not related your questions)