Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'd like to create a (compressed) archive of a folder which contains ~350,000 items totalling ~17 GB.

Is .tar.gz a sane format to use, or would .zip, .rar, or .7z work better for handling that much data and that many files?

Edit: just came across this SF question; it looks like 7-zip (with bzip2 compression) is the way to go.

share|improve this question
up vote 2 down vote accepted

It all depends on how you are wanting to access the archived files.

If you are only ever going to restore all the files in one go, then tar + a compression system (gzip, bzip2, xz, whatever) will be fine and will probably give you the best compression.

If you will want to extract a subset of the files at any point then stay away from tar. Tar has no central index of the contents, and so will have to read through the entire archive to extract the few files you want. In this situation something like ZIP or RAR would be much more sensible.

share|improve this answer
This is just a basic backup in case I screw something up while upgrading an old Linux install, so indexed content is probably a Good Thing for this use case. Is 7z also indexed (I can only imagine the answer is "yes")? Any idea of which format will be faster to build? – Matt Ball Apr 14 '11 at 18:26
ZIP and 7z are both pretty similar technologies (7z is a super-set of ZIP) and both are indexed. Most of the time taken will be disk access, so there won't be much to choose between them speed wise. – Majenko Apr 14 '11 at 18:29
"stay away from tar" - not quite so. With pixz-made archives (archive.tar.xz) you can easily extract just a single file. – kubanczyk Jun 29 '14 at 13:33
No you can't. Tar is a tape stream format, xz is a compression algorithm. You have to decompress the xz stream whilst streaming the tar data until you reach the file. There is no way of "jumping in" and just decompressing the one file like you can with things like Zip, where each file is compressed individually. – Majenko Jun 29 '14 at 14:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .