I would like to create tar gzip archive, but do it the reverse manner of what is most commonly done -- have the files in the archive be compressed individually rather than compress the entire archive: that way it retains the seekable property it should have. It makes much more sense to me, and I don't know why this has not been favored.
I have some ideas on how to do this:
- use an archive tool with compression, such as 7zip
- write my own script to do this by compressing the files first, then feeding them to tar (already found a handy Python script to write a tar file and the Python gzip library seems easy to use.
However, ideally, I would like to continue to use tar for this, as it is a familiar, de-facto tool for archiving where I work. tar has the
--to-command switch, which allows piping extracted files to a program. If I had a symmetric command such as
--from-command I would easily implement my wish with:
tar cf my_archive.tar file1 file2 --from-command=gzip tar xf my_archive.tar --to-command=gunzip
My motivation comes from dealing with archives containing a large number of large files. I currently tar-gzip them, but then extracting any files from the archive takes a long time - it needs to be decompressed before tar can access the file, and it does so in a serial manner!
So here are my questions:
- Is there an evident way to achieve this that I am disregarding?
- Has anyone already written a tool to do with, specifically with
- If one would call
gzipand standard methods of archiving and compressing in Linux, what would be the equivalent, popular method for archiving with compression in the manner I mentioned about (i.e. not
- Is there another way I am overlooking to circumvent the large amount of time it takes to extract a file from a large tar-gzipped archive?
I realize that I need to re-phrase and refine my question. Especially since, as Robin Hood pointed out, there are existing rather easy solutions to create compressed archives (namely,
zip). So here it is:
Is there are way to use
tar that allows true random-access to the archive while still keeping it compressed? If not, is there another
tar-replacement for Linux (that is built with the same rationale, and, ideally, with support for the same command-line options), that does achieve this?
Right now I can replace
tar in a general sense with
zip, by changing:
tar c path/to/file1 path/to/file2 | gzip > arc.tar.gz gunzip < arc.tar.gz | tar x
zip -qr - path/to/file1 path/to/file2 > arc.zip unzip -qoX test.zip
However, this has the disadvantage that it does not support all the options that
tar does for archiving, namely:
- piping each extracted file individually into a pipe (the
unzipdoes not accept an archive in standard input.
funzipdoes, however - it only outputs the first file in the archive
So it's rather limiting.