up vote 3 down vote favorite
share [g+] share [fb]

I have to pack circa 7,000,000 small files. I'm looking for an efficient way to do this.

Is tar the best way? What about benchmark tests or any other suggestion?

link|improve this question
1  
In one directory? – Mark Byers Nov 26 '09 at 14:00
2  
Efficient in what way? Compresionspeed? Filesize? etc. – Don Salva Nov 26 '09 at 20:23
feedback

migrated from stackoverflow.com Nov 26 '09 at 14:58

This question came from our site for professional and enthusiast programmers.

3 Answers

up vote 5 down vote

Some people like 7-Zip. For example that's how Wikipedia packages up complete images.

link|improve this answer
I'd go with 7zip. – Don Salva Nov 26 '09 at 20:50
Indeed. I'd use 7zip for EVERYTHING, except system backup on Linux. Since the files lose their attributes. That stays in .tar.gz , "but for everything else.." :~) – Shiki Jun 30 '10 at 9:01
feedback
up vote 2 down vote

TAR is actually doesn't have it's own compression. TAR is just merging directories/files into one big file. The speed of creation of tar archive depends more on filesystem and I/O subsystem than on tar itself. The best way to get the highest compression ratio is to use 7zip (7zr command on *nix/Linux). The bigger dictionary size you choose - the smaller archive you receive.

For Linux you can try:

7zr a -t7z -m0=lzma -mx=9 -mfb=64 -md=128m -ms=on /path/to/put/archive.7z folder_to_compress
link|improve this answer
feedback
up vote 0 down vote

jar is also very nice, as it's a zip (compression) of the files. If not for the manifest inclusion, it would be close to perfect (as jar seems to exist just about everywhere now)

link|improve this answer
2  
fyi, a jar file is a zip file w/ an automatically added manifest. No difference. – basszero Nov 26 '09 at 19:59
and that's different from my answer how? – KevinDTimm Feb 11 '10 at 16:17
1  
why would you use jar over zip? – lfaraone Aug 20 '10 at 13:33
feedback

Your Answer

 
or
required, but never shown
discard