5

Maybe a variation of the whole question would be: Are there any other archiving tools than tar that preserve Linux file permissions and user/group flags?

I want to create archives of a directory tree on my ubuntu box. The directory trees are large, a first run with tar cvpzf archive.tgz /home/foo/bar yielded a 5GB archive.

I also want to keep all permissions and other flags and special files.

I'm fine with a 5GB archive, however to look inside that archive -- since it is a compressed tar archive -- the whole 5GB have to be decompressed first! (Or so it appears when opening it with the archive viewer -- I'm happy to be corrected.)

So I need a way to "backup" a directory (tree) while preserving full filesystem attributes and right and creating an archive with an "index" that hasn't to be decompressed for browsing its contents.

Edit: Specifying what I think of as an archive, after Whalley's answer :

An archive is either a single file, or a (small) set of files that carries full and complete information within this file/s. That is, it can live on any filesystem (size permitting), it can be burnt onto a DVD, you can split it (after which the point of this question is really lost - but still), ...

Improve this question

4 Answers 4

Reset to default
2

Do you know about tar's t function for printing out the contents of the archive? You can use it on a gzip compressed archive like this:

tar ztvf archive.tgz

And it will print out the files in a long list like format including timestamps and sizes. It turns out that if you use two v options when creating the archive that it will long list out the files (instead of just plain listing them) as it adds them to the archive, so you can do something like this to auto create an index while you're backing up in one go:

tar cvvpzf archive.tgz /home/foo/bar > archive.tgz.index

Note the use of two v options. Many other commands allow you to increase verbosity this way (ssh for example).

Improve this answer
1
  • Interesting idea!
    – Martin
    Commented Nov 5, 2010 at 18:43
4

I've run into this problem on multiple occasions. Here's what I've figured out.

ZIP
.zip files support unix permissions and symbolic links if created using the zip utility installed on most Linux systems. However, it doesn't support all the extensions supported by tar, such as hard links and extended attributes. But you can extract files from ZIP archives without decompressing the whole archive up to that point first, so that's a win.

SquashFS
If you're looking for a mountable compressed read-only filesystem, the top contender is currently SquashFS. But there's a catch... well, a few catches, really. Specifically:

  • SquashFS is relatively new, and not as widely supported as tools like tar.
  • There are two on-disk formats out there produced by two different versions of the tool which are not compatible with each other. Make sure you get the right one.
  • There is no FUSE driver -- you need in-kernel support for your version of squashfs in order to mount images.
  • The supporting tools (mksquashfs and unsquashfs) are fairly simple and don't support the wide range of options you see with tar.
  • There's not a lot of documentation.
Improve this answer
2
  • in many situations squashfs is the best solution, when you need extract file from archive very fast. it is enough you mount as loop squashfs file & simply copy it to restoring place. many other archive format takes long time for seeking. for example tar format needs seek all arvhive for extracting one file. zip is better but what about attributes? and ISO formats family doesn't support compression. then I recommend using zip (if it stores attributes) or squashfs (squashfs-tools unsquashfs mksquashfs). If you don't want temp mount squashfs, you can use 7zip tools for extracting without mount.
    – Znik
    Commented Nov 27, 2015 at 14:59
  • I like SquashFS a lot, unfortunately it is not designed as a long term archive file system but for use with Live-DVDs. The catch is that future kernel drivers do not need to be backward compatible. I had the case when I couldn't mount my SquashFS images any more after updating the kernel. I had to use unsquashfs to extract them, loosing the benefits of mount. Also there seems no updates since 2011!
    – Martin Scharrer
    Commented Feb 2, 2019 at 9:35
2

The main problem seems to be that

... to look inside that archive ... the whole 5GB have to be decompressed first! (Or so it appears when opening it with the archive viewer ...)

It's true that it needs to be decompressed, but that does not mean it's necessary to "unpack" all the files to the filesystem.

To extract a single or multiple files, tar will decompress the whole archive, but discard anything except the files listed as arguments.
In a similar way, listing the file names with -t (--list) needs to decompress the whole archive, but discards all the file content.

To understand why there is no index at the start of the archive, take into account that tar is originally made to work with magnetic tape devices; Hence the name tar - like tape archiver. On a tape, you want to add the next file at the end, you do not want to rewind to the beginning for writing an index.

Improve this answer
1

You may want to look into fusecompress, which "provides a mountable Linux file system which transparently compress its content. Files stored in this file system are compressed on the fly and Fuse allows to create a transparent interface between compressed files and user applications."

To the user the stored files can be accessed as if they are not compressed.

But if you have the space, why not just copy the directory tree? Simpler.

Another option is to copy the directory trees to a usb drive. That would allow easy access and would give protection from a disk crash.

Improve this answer
2
  • Careful with the USB drive - make sure you know what filesystem is on it - they tend to come pre-formatted with a Windows filesystem of some sort, and those don't tend to support unix permissions.
    – Michael Kohne
    Commented Oct 31, 2010 at 20:30
  • Simple copy is not an option when you want to make sure the "archive" can be moved around as transparently as possible. (USB stick NTFS file systems etc.)
    – Martin
    Commented Nov 2, 2010 at 7:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .