Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a huge tar file (about 500G) and I wan't to extract just a single file from it.
However, when I run tar -xvf file.tgz path/to/file it seems like it is still loading the whole contents into memory, and takes over an hour to extract. I've also tried to use --exclude=ignore.txt where ignore.txt is list of patterns in an attempt to stop it from traversing futile paths, but that doesn't seem to work.

Perhaps I don't understand tar... Is there a way to quickly extract the file?

share|improve this question
    
I am wondering about the same. The file I am looking for is found quickly and extracted - and then I need to wait for an hour for the rest of the achieve to be processed :o( –  maasha Sep 29 at 7:55

3 Answers 3

up vote 3 down vote accepted

Unfortunately, in order to unpack single member of .tar.gz archive you have to process whole archive, and not much you can do to fix it.

This is where .zip (and some other formats like .rar) archives work much better, because zip format has central directory of all files contained in it with direct offsets pointing to the middle of the zip file, so archive members can be quickly extracted without processing whole thing.

You might ask why processing .tar.gz is so slow?

.tar.gz (often shortened as .tgz) is simply .tar archive compressed with gzip compressor. gzip is streaming compressor that can only work with one file. If you want to get any part of gzip stream, you have to uncompress it as a whole, and this is what really kills it for .tar.gz (and for .tar.bz2, .tar.xz and other similar formats based on .tar).

.tar format is actually very, very simple. It is simply stream of 512-byte file or directory headers (name, size, etc), each followed by file or directory contents (padded to 512 block size with 0 bytes if necessary). When you observe totally null 512 block for a header, this means end of .tar archive.

Some people think that even .tar archive members cannot be accessed quickly, but this is not quite true. If .tar archive contains few big files, you actually can quickly seek into next header, and thus you can find necessary archive member in few seeks (but still could require as many seeks as there are archive members). If your .tar archive contains of lots of tiny files, this means quick member retrieval becomes effectively impossible even for uncompressed .tar.

share|improve this answer
    
gzip can stream uncompressed data, it doesn't have to undo the whole thing. But, since .tar is short for tape archive, you do need to traverse the whole file until you find the file you are looking for. Although tar will keep looking because there might be another, later copy later on in the tar file. –  kurtm Oct 8 '13 at 4:35

Unfortunately, the tar file format contains no centralized table of contents - so the archive must be read sequentially to locate a particular file. It was originally designed for tape backups ("tar" comes from tape archive), which wouldn't have supported such an operation in any case.

So, you'll probably just have to wait.

share|improve this answer

I've opened various tar files (including tgz) on windows using the common zip file tools. (I just now confirmed that I can extract a single file from a tgz archive using WinRAR)

It might be worth your while to do that just to extract the one file, then move it (scp, thumb drive, etc) over to your linux machine.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.