2

I recently built a system with a btrfs root partition. The most compelling reason for adopting btrfs compared to ext4, in my case, was live copy-on-write snapshots, with near-zero latency. Compared to ext4, when a full system backup entailed taking down the system, mounting from a live distribution, and building a partclone image on removable media, the promise of snapshots is that the snapshot can be captured on the backup media while the system is live.

To my astonishment, no tool exists, however, that will capture the entire snapshot to a single file on external media, such that if the system is restored from that file, all applications will have the same view of the filesystem as before the crash (except that the other snapshots or subvolumes are missing).

The documentation suggests mirroring the directory tree using a tool such as rsync, or using btrfs-send/btrfs-receive to capture the incremental changes on another system. In the first case, I have always found it nearly impossible to recreate all the metadata in a file tree exactly by mirroring a file tree rather than imaging the file system, and have little optimism that a restore would be very smooth. I always find that some metadata, be it permissions, timestamps, hidden files, or so on, are not properly captured. The problem is compounded when the transfer occurs across file systems of different types. The other suggestion assumes another btrfs file system is available, which is not always the case.

Are any suggestions available for saving or restoring a volume-level image, similar to a partclone file, but representing only a select subvolume?

  • The problem here is that identifying what is a subvolume relies on metadata that you by definition don't want to copy (you only want a subvolume), so I don't think such a tool would be easy to write: It would have to modify this metadata, and keep track while doing so. – dirkt Jan 20 at 6:54
0

btrfs send and btrfs receive are exactly the tools you need.

I have always found it nearly impossible to recreate all the metadata in a file tree exactly by mirroring a file tree rather than imaging the file system, and have little optimism that a restore would be very smooth. I always find that some metadata, be it permissions, timestamps, hidden files, or so on, are not properly captured.

Have you tried the tools? I have had no such problems with btrfs. btrfs send and btrfs receive do not work with files, they work with entire subvolumes, having access to all Btrfs metadata natively.

The problem is compounded when the transfer occurs across file systems of different types. The other suggestion assumes another Btrfs file system is available, which is not always the case.

You will need another Btrfs filesystem only if you want to replicate the subvolume there. This is often desirable, so you can mount it and get access to its files. And this is unavoidable, if you want to backup your subvolumes incrementally. You need the exact previous snapshot(s) in both places to incrementally send the next one.

With full dump, however, you get a standalone stream, which is exactly an image (but not a browsable backup, unless you btrfs restore it). Work with it like with any other stream:

btrfs send /source/subvolume >/another/filesystem/subvolume-image   # just a file
# (or you can gzip it and/or send with nc on the fly, whatever)
# then later
</another/filesystem/subvolume-image btrfs receive /some/btrfs/directory

Where /some/btrfs/directory may belong to the same Btrfs filesystem as /source.

  • To be clear, my requirements are 1) capture an entire, single subvolume, 2) into a single file (or small group), 3) that can be stored as regular files on any external FS of any type. The purpose is crash recovery after equipment failure (akin to partclone for full partitions), not live, browsable, replicated instances. I have not tried the send command because I understood that it is only useful for exchange between two live btrfs partitions. Are you saying that I can use the command as in your example to capture a minimal but full dump of all data and metadata for a single subvolume? – epl Jan 20 at 7:57
  • @epl Are you saying that I can use the command as in your example to capture a minimal but full dump of all data and metadata for a single subvolume? – I believe so. Yes. – Kamil Maciorowski Jan 20 at 9:31
  • That is very nice. The tutorials indicate that btrfs has no solution for backups, and other tools such as rsync are recommended. I wish that they instead would indicate that btrfs has a great backup system. To create a backup, one first simply makes a snapshot, and then dumps the snapshot to a file using btrfs send. To test this approach online, I suppose I could very easily dump a snapshot to a file and then it restore to an empty subvolume from that file? – epl Jan 21 at 1:34
  • @epl btrfs send looks OK when it comes to a full copy of a single subvolume, but for many of them sharing data blocks, and for incremental backups, the situation gets complicated. See this answer. In such case storing backups as files leads to problems: (1) Restoring must be done in proper sequence. (2) Incremental backups require you to store new files, while you cannot really delete old ones. (3) Non-incremental backup transfers some data again. // Unless the file holds a Btrfs filesystem you mount and btrfs restore to on the fly. – Kamil Maciorowski Jan 21 at 5:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.