Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am frequently moving large files around from server to server.

UNIXes comes with standard gzip and bzip2 compressors. However, what open options there exist beyond these, especially ones utilizing multiple cores? I am aware of pzip2 which I am using currently.

Are there any up-to-date benchmarks and charts how fast/effective compressors are to see what's the tradeoff between time and the file size?

An open source alternative, which is distribution packaged and supported across Linux/OSX, preferred. Windows support unneeded.

share|improve this question
    
When you say pzip2, I assume you mean pbzip2? –  Croad Langshan Feb 5 '12 at 16:24

2 Answers 2

up vote 2 down vote accepted

You don't say much about your priorities, but it sounds like compression time for large files is your main priority?

This claims to be suited to that case (see the -l flag):

http://manpages.ubuntu.com/manpages/lucid/man1/lrzip.1.html

On the other end of the tradeoff (high compression ratio, long compression time), some notable software distributions have started using this recently:

https://en.wikipedia.org/wiki/Xz

http://tukaani.org/xz/

Re benchmarking: why not run some quick tests youself? Benchmarks are often misleading because results can vary a lot depending on the precise workload and test conditions. Still, here's a report in German (try google translate) about somebody else's benchmarking:

http://www.linux-community.de/Internal/Artikel/Print-Artikel/LinuxUser/2010/10/Aktuelle-Komprimierprogramme-in-der-Uebersicht/(printView)/true

share|improve this answer

Also worth looking at are pigz for parallelism and 7zip if Windows support does rear its head.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.