4

I have to pack circa 7,000,000 small files. I'm looking for an efficient way to do this.

Is tar the best way? What about benchmark tests or any other suggestion?

2
  • 1
    In one directory?
    – Mark Byers
    Nov 26, 2009 at 14:00
  • 2
    Efficient in what way? Compresionspeed? Filesize? etc. Nov 26, 2009 at 20:23

3 Answers 3

5

Some people like 7-Zip. For example that's how Wikipedia packages up complete images.

2
  • I'd go with 7zip. Nov 26, 2009 at 20:50
  • Indeed. I'd use 7zip for EVERYTHING, except system backup on Linux. Since the files lose their attributes. That stays in .tar.gz , "but for everything else.." :~)
    – Apache
    Jun 30, 2010 at 9:01
3

TAR is actually doesn't have it's own compression. TAR is just merging directories/files into one big file. The speed of creation of tar archive depends more on filesystem and I/O subsystem than on tar itself. The best way to get the highest compression ratio is to use 7zip (7zr command on *nix/Linux). The bigger dictionary size you choose - the smaller archive you receive.

For Linux you can try:

7zr a -t7z -m0=lzma -mx=9 -mfb=64 -md=128m -ms=on /path/to/put/archive.7z folder_to_compress
0

jar is also very nice, as it's a zip (compression) of the files. If not for the manifest inclusion, it would be close to perfect (as jar seems to exist just about everywhere now)

3
  • 2
    fyi, a jar file is a zip file w/ an automatically added manifest. No difference.
    – basszero
    Nov 26, 2009 at 19:59
  • and that's different from my answer how?
    – KevinDTimm
    Feb 11, 2010 at 16:17
  • 1
    why would you use jar over zip?
    – lfaraone
    Aug 20, 2010 at 13:33

You must log in to answer this question.