2

Is there any tools that allows this use case:

  1. I have a LVM2 partition vg-vol;
  2. I periodically create snapshots of it, save them (create_new_snapshot --directory=/media/external_storage/snapshots/ --input=/dev/mapper/vg-vol_snapshot --name=qwerty), and remove. Only changed blocks are expected to be saved by the tool. The tool is not expected to be LVM-specific, just accept a single big file which is expected to share many blocks with previous such big file. Good thing if compression (with random access preserved) supported...

  3. I can mount (using FUSE or NBD or some other thing) any saved snapshot without fully unpacking it anywhere.

    touch /root/mountpoint
view_external_snapshot --mountpoint=/root/mountpoint --directory=/media/external_storage/snapshots/ --name=qwerty
mount -o loop -t reiserfs /root/mountpoint /root/tmpmnt

The closest thing is rdiff (not rdiff-backup), but I'm not sure if I can access the content of base+delta without unpacking (rdiff patch) it first...

Other related thing is cloop. It accepts big file as input, compresses it and allows to access as block device after. But there I can't store deltas, only full snapshot...

Note: the question is not about filesystem-level backup (like in duplicity, rdiff-backup, rsnapshot).

Improve this question

3 Answers 3

Reset to default
1

If you need a solid solution (and probably you do) for protecting your data, try ZFS, which has deduping, transparent compression and block-level mirroring. There is both a kernel module and a FUSE overlay for Linux.

EDIT: as discussed in chat, another solution would be to use squashfs and take advantage of the append mode by using dated directories for base and increments. You could mount the image in loopback and use rdiff to find actual deltas. However, I think it will be much slower than the ZFS approach. After all, advanced filesystems like ZFS and BTRFS were born to fill those gaps.

Improve this answer
2
  • 2
    Also BTRFS has these features, but it is simply not ready for production.
    – Stefano Sanfilippo
    May 5, 2013 at 11:29
  • Solid solution is not strictly required (that's why I also thinking about writing that myself) - this is for personal data. But keeping the entire thing simple is probably required.
    – Vi.
    May 5, 2013 at 11:43
1

Rolled my own: https://github.com/vi/forsnapshotfs

It allows sequentially writing new files (reusing blocks from previous version of the file), uses compression and allows mounting (with simple in-memory copy-on-write) such files.

$ fsfs-write . root_20130326          < /dev/mapper/inside-root_20130326
Completed. new: 9297920   reused: 0         hashcoll: 0     zero: 0  dblref: 0
$ fsfs-write . root_20130625 20130326 < /dev/mapper/inside-root_20130625
Completed. new: 1376582   reused: 7921338   hashcoll: 5380  zero: 0  dblref: 0

# mkdir m m2
# fsfs-mount . m
# mount -o loop,ro   m/root_20130326   m2
Improve this answer
1

It can also be done with the help of bup. It seems to support this use case.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .