I tried to use .rar and I get a great rate of compression. However it take a lot of time to show the folder structure when I open the .rar file (tried different programs), it looks it is needed to read every file before displaying the folder structure. (I need to compress like 1 million files)
What compression format should I use so I can open the archive and extract some file on it fast when needed?
There will be trade off between speed/compression ratio.
You may want to view the blog post, Linux Compression Comparison (GZIP vs BZIP2 vs LZMA vs ZIP vs Compress). The author's results reflect my general experiences with the various compression algorithms. Pay specific attention to the decompression graph.
The ZIP format provides the best platform compatibility (both Windows and OS X support zip natively). However, its compression ratio will likely be worse than RAR.
The GZIP format is a very popular choice in the Linux world. It provides very quick compression while average compression rates. Mac OS X supports gzip archives natively, however with Windows you need an application such as 7zip.
The BZIP2 and LZMA formats tend to take a much longer time to compress and decompress files, however they tend to yield much better compression ratios. I think OS X supports both of these formats natively, again with Windows, you will need to use something like 7zip.
The type of files being compressed makes a huge difference in compression time, decompression, and compression ratio and differs for each format.
Ultimately you will need to try various formats and see which meets your needs tho best.
