Mounting a .7zip file containing .tar

Question

I have downloaded wikipedia in a compressed form. It is a .7z file, which contains a .tar file. In compressed form it is only 14 GB. If I uncompress it, it has 208 GB in it. And I don't have that much space on my hard drive.

I guess there is a way to use it in its compressed form, by using some sort of mount tools.

I am on Windows Vista.

What to do?

^{ServerFault question reference.}

Answer 1

I think you should extract the tar.7z file once to its whole 200GB+ size,
Then you can re-zip the base directory as a pure 7z archive.

It will probably not exceed 14GB by far.

You can then browse the new 7z archive more easily.
^{this will need one go of un-zip, un-tar and re-zip that will take disk space and compute time though.}

Answer 2

Any decent Orhtodox File Manager allows you to browse through archives. I know Total Commander has 7zip plugin that will allow you to browse the 7z archive as a directory.

Answer 3

On a Unix box (or with cygwin (Unix for windows), a Unix environment), you could pipe the file, and get a listing out of it, or probably make a zip file of the whole thing, which would allow random access.

In some ways, tar is a feature, as it was meant for tape drives (one way linear access).

Get a Linux vm and a virtual env (say, vmware server?), or cygwin, and have fun.

7z -options_to_extract_to_console/pipe wiki9.tar.7z | tar t > listing_of_files.txt

would give you a list of what was there

Then

7z -same_options_as_before wiki9.tar.7z | tar -xT list_of_files_I_want

and it should write them (possibly someplace weird).

I think. I don't have the listing for 7z switches on Unix handy right now.

Or to make it a zip archive:

7z -same_options_as_before wiki9.tar.7z | tar -x | zip wiki.zip -

Note that it will likely be 2-4 times larger (better to stick the files together and then compress than compress one at a time, and 7z uses better compression).

Expect it to take hours.

Consider borrowing an external drive for a few days :-D or upgrade your HD, heck under $100 for internal or external over 200gb, even laptop HD.

Answer 4

Open it with 7Zip? I will only decompress what you tell him to...might take some time, though.

Edit: Wait a minute, I just realized that this is possibly a dump idea because of the size. I don't know how 7Zip exactly works, but I'd guess that he's just extracting the file list, without caching the whole content.

Answer 5

You basically cannot accomplish this. 7zip's compression style doesn't lend itself to opening random sections without opening the whole file. If I recall correctly, it's a stream system.

For example, this is different than with .zip archives which will compress each file individually (with whatever algorithm they fancy), and .bz2 will use a fixed (typically) 900kb block size. Both of those forms can be used to seek to an individual space without full compression.

And, just to add insult in case I'm wrong about the .7zip story: Tar files don't store their file information in a central location, so you have to expand that too. See http://en.wikipedia.org/wiki/Tar%5F%28file%5Fformat%29#Random%5Faccess

You'll essentially have to find the disk space if you want to use those files.

Answer 6

FYI, the Wikipedia data dumps are typically just one XML file.

A few years ago I wrote a Wikipedia data dump parser in C: wikiq. It converts data dump streams into a tab-separated revision history format amenable to use in statistical processing programs.

7za e -so <file> | ./wikiq | <tab-separated processing program>

If you throw away text and comment data (as it does by default), processing is actually bounded by decompression (20-30mb/s).