Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have to pack circa 7,000,000 small files. I'm looking for an efficient way to do this.

Is tar the best way? What about benchmark tests or any other suggestion?

share|improve this question

migrated from stackoverflow.com Nov 26 '09 at 14:58

This question came from our site for professional and enthusiast programmers.

1  
In one directory? –  Mark Byers Nov 26 '09 at 14:00
2  
Efficient in what way? Compresionspeed? Filesize? etc. –  Don Salva Nov 26 '09 at 20:23
add comment

3 Answers

Some people like 7-Zip. For example that's how Wikipedia packages up complete images.

share|improve this answer
    
I'd go with 7zip. –  Don Salva Nov 26 '09 at 20:50
    
Indeed. I'd use 7zip for EVERYTHING, except system backup on Linux. Since the files lose their attributes. That stays in .tar.gz , "but for everything else.." :~) –  Shiki Jun 30 '10 at 9:01
add comment

TAR is actually doesn't have it's own compression. TAR is just merging directories/files into one big file. The speed of creation of tar archive depends more on filesystem and I/O subsystem than on tar itself. The best way to get the highest compression ratio is to use 7zip (7zr command on *nix/Linux). The bigger dictionary size you choose - the smaller archive you receive.

For Linux you can try:

7zr a -t7z -m0=lzma -mx=9 -mfb=64 -md=128m -ms=on /path/to/put/archive.7z folder_to_compress
share|improve this answer
add comment

jar is also very nice, as it's a zip (compression) of the files. If not for the manifest inclusion, it would be close to perfect (as jar seems to exist just about everywhere now)

share|improve this answer
2  
fyi, a jar file is a zip file w/ an automatically added manifest. No difference. –  basszero Nov 26 '09 at 19:59
    
and that's different from my answer how? –  KevinDTimm Feb 11 '10 at 16:17
1  
why would you use jar over zip? –  lfaraone Aug 20 '10 at 13:33
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.