Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am trying to compress large datasets (from 15-400 GB each). I am looking for the best compression software for Mac that has

  1. High compression rate.
  2. Is fast

I have tried 7zX, which has excellent compression rate on Ultra settings, but is very slow. I wonder if I can change the settings so it

  1. uses all cores I have
  2. the max RAM
  3. and max CPU.

So I can shorten that time. Or if there is another software that offers this option. Of, course if anybody can recommend any better software, I'd be happy to here from it.

Also about workflows on how to deal with big files that need to be compressed/decompressed later.

Thanks.

share|improve this question
    
Some algorithms are notoriously more space or time efficient than others, but there is no one best compression software. At the very least, you'll need to specify what sort of data we're talking about, preferably by posting a sample. –  Marcks Thomas Jun 19 '12 at 10:11
    
mostly video, images and data with a lot (!) of small files –  usersuper Jun 19 '12 at 10:18
    
Compressed video or uncompressed, compressed images or uncompressed, what kind of data, text files or binary? If the videos and images are already compressed (e.g. MPEG 4 and JPEG) I would go for the fastest compression (pigz would be a candidate), since you will not gain much. –  Marco Jun 19 '12 at 10:29
    
It seems like you want to use 7z and just wait for it to finish. Of course you do understand any sort of compression means you have to uncompress the files also right? In other words at such a small size you might just want to store the data raw. Although the compression can give you one thing you need data verification that the archive isn't corrupt. –  Ramhound Jun 19 '12 at 10:57
    
uncompressed everything. –  usersuper Jun 19 '12 at 10:58

1 Answer 1

up vote 2 down vote accepted

There are versions of bzip (pigz) and bzip2 (pbzip2) which are multi-threaded. Both will use 100% of all cores.

Using the maximum RAM doesn't make sense and the algorithms will take as much as necessary to run, not more.

Which algorithm is best, depends on your use case and data, but pigz will probably be one of the fastest and 7z one of the slowest, but with better compression ratio.

share|improve this answer
    
pigz is very fast, but compression is not great (even on comp. level 7) and if I do not specify a file, but a directory it does not compress and I get: "is a directory --skipping". –  usersuper Jun 19 '12 at 11:24
    
The fastest algorithm will probably never be the one with the best ratio. There is no all-in-one algorithm suitable for everything. bzip and pbzip2 (inclusive the multi-threaded variants) are pure compression tools, they don't create archives. You can use tar (tar cf directory.tar <directory>) to first create an uncompressed archive and then run pigz on the tar archive. –  Marco Jun 19 '12 at 11:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.