3

I am trying to back up my /home directory of my Ubuntu laptop to a external storage drive formatted with NTFS. As NTFS does not support directory permissions, I tried using tar for creating the backup:

tar -cvpf backup.tar /home

After finishing the backup, I found out that the access to parts of the archive (e.g. listing the contents of a single directory in the archive), Is horribly slow.

tar -tf backup.tar someuser/Documents

As the overall /home directory is not that small (about 120 GB) I assume this is because tar is optimized for securing files to a backup tape and not for real disks as no index is created for the containing files [1]. Please mind that space is not a problem and therefore I haven't compressed the tar file!

As there are already files on the NTFS drive, I would like to avoid reformatting the drive and creating a ext4 partition for the backup.

How can I store and read my backups quicker?

Zip and 7z doesn't seem to preserve directory permissions, so they seem not to be suitable for the job. I can't believe that there is no archive format that has support for UNIX style permissions and allows fast access of single files in the archive...

[1] http://www.cyberciti.biz/faq/extracting-single-file-directory-from-tarball-2/

1
  • tar does have to read the whole archive before listing all the files. If you're just extracting a single specified file, you may get lucky and it will be at the start rather than the end, so could be quicker. PS. This really is on-topic "I have <problem-x> that I don't know how to solve. I've already tried X, Y, Z, but those programs don't work because this or that. How do I do this?"
    – Xen2050
    Apr 7, 2016 at 18:31

1 Answer 1

0

I've got a couple ideas...

Use ZIP

The ZIP format does store permission info (at least Info-ZIP implementations, as used by Mint/Ubuntu/Debian), but apparently not users/groups (at least the version on Mint "This is Zip 3.0 (July 5th 2008), by Info-ZIP."). If the files are all owned by your user with no special user or groups, then zip should work well enough.

Tested with a directory, using Archive Manager 3.10.2.1 too, and it preserved the permissions of the files & directories OK. The user:group is apparently set to you:you.

See Wikipedia about Zip's file format:

The Info-ZIP implementations of the .ZIP format adds support for Unix filesystem features, such as user and group IDs, file permissions, and support for symbolic links. The Apache Ant implementation is aware of these to the extent that it can create files with predefined Unix permissions. The Info-ZIP implementations also know how to use the error correction capabilities built into the .ZIP compression format. Some programs (such as IZArc) do not, and will fail on a file that has errors.

The Info-ZIP Windows tools also support NTFS filesystem permissions, and will make an attempt to translate from NTFS permissions to Unix permissions or vice versa when extracting files. This can result in potentially unintended combinations, e.g. .exe files being created on NTFS volumes with executable permission denied.

7z appears to store file & directory permissions OK too, and no user/group info either.

Use a filesystem-in-a-file

Especially since space is no issue, you can create a single very large file and treat it as a big partition, format it to ext (or your favourite linux format) to store all your files in.

Once mounted, Linux will treat it like another partition and you can copy & read files to it preserving all permissions/users.

The steps are basically

  1. Create a big empty/zero file. head works, or dd etc. This example creates a 5M file full of zeros:

    head -c 5M /dev/zero > bigfile
    

    (replace 5M for whatever size you want, using whatever multiplier suffix you like: "b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024, and so on for T, P, E, Z, Y.")

  2. Format the file. Some instructions say to set up a loop file first, with losetup, but modern distributions can do the loop part themselves.

    $ mke2fs -v bigfile
    mke2fs 1.42.9 (4-Feb-2014)
    bigfile is not a block special device.
    Proceed anyway? (y,n) y
    ...
    Writing superblocks and filesystem accounting information: done
    
  3. Mount the filesystem & use it

    $ sudo mount -v bigfile mountpoint
    mount: enabling autoclear loopdev flag
    mount: going to use the loop device /dev/loop1
    mount: you didn't specify a filesystem type for /dev/loop1
           I will try type ext2
    /home/user/bigfile on /home/user/mountpoint type ext2 (rw)
    

I've seen some programs for Windows that can read ext partitions, but I don't know for sure if any can do it from a file like this, so the files may not be accessible from Windows.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .