Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have to pack circa 7,000,000 small files. I'm looking for an efficient way to do this.

Is tar the best way? What about benchmark tests or any other suggestion?

share|improve this question

migrated from Nov 26 '09 at 14:58

This question came from our site for professional and enthusiast programmers.

In one directory? – Mark Byers Nov 26 '09 at 14:00
Efficient in what way? Compresionspeed? Filesize? etc. – Grumpy ol' bear Nov 26 '09 at 20:23

Some people like 7-Zip. For example that's how Wikipedia packages up complete images.

share|improve this answer
I'd go with 7zip. – Grumpy ol' bear Nov 26 '09 at 20:50
Indeed. I'd use 7zip for EVERYTHING, except system backup on Linux. Since the files lose their attributes. That stays in .tar.gz , "but for everything else.." :~) – Shiki Jun 30 '10 at 9:01

TAR is actually doesn't have it's own compression. TAR is just merging directories/files into one big file. The speed of creation of tar archive depends more on filesystem and I/O subsystem than on tar itself. The best way to get the highest compression ratio is to use 7zip (7zr command on *nix/Linux). The bigger dictionary size you choose - the smaller archive you receive.

For Linux you can try:

7zr a -t7z -m0=lzma -mx=9 -mfb=64 -md=128m -ms=on /path/to/put/archive.7z folder_to_compress
share|improve this answer

jar is also very nice, as it's a zip (compression) of the files. If not for the manifest inclusion, it would be close to perfect (as jar seems to exist just about everywhere now)

share|improve this answer
fyi, a jar file is a zip file w/ an automatically added manifest. No difference. – basszero Nov 26 '09 at 19:59
and that's different from my answer how? – KevinDTimm Feb 11 '10 at 16:17
why would you use jar over zip? – lfaraone Aug 20 '10 at 13:33

You must log in to answer this question.