1

I'm planning out how to proceed with getting the storage of all my Debian PCs at home moved to ZFS. I previously considered this but the more I think about it, the more I wonder if it's going to limit my setup.

The Requirements

Consider I have three computers, all running Debian. One would be at least a file server as described in the other post hosting documents, photos, videos, etc. This file server can be up 24 hours a day and has several TB of storage.

Ideally, the backup drive for the file server would not be mounted live 24 hours a day. In fact, it would be great if I could have it in a separate box that only powers up 1-4x per day just to collect ZFS snapshots from the primary drive in the file server, and then shuts back down or goes to sleep (assuming that's not too taxing for the drive). This setup could also open the door to moving the backup box out to someone else's house for a remote backup.

The others are standard Debian workstations with local hard drives, likely running other VMs using VMware or VirtualBox (i.e. likely suitable candidates for zvols).

Let's say these "clients" are never reliably available. One is a laptop that might be out of the house at random times of the day, and the other is a desktop PC that might be turned on at 8 a.m. one day while at 3 p.m. the following day. The storage on these boxes ranges from a couple hundred GB of storage up to 1 TB on the laptop.

My primary concern is keeping the data safe i.e. frequent backups (anywhere from hourly to daily) onto a second drive that can be rotated out every so often (e.g. every 4-8 weeks).

I'm not too worried about redundancy on any of these as I can afford the downtime if there is a failure, but if I get it for free, I won't object to it.

I'm also not too concerned about speed as my current nightly dirvish/rsync backups during off-hours are already slow, so any improvements by moving to ZFS will give me the desired speed boost for free.

I believe that's it for my requirements.

Question 1: Is there a better way to accomplish this than what I have drafted below?

The Plan

I was thinking of setting each of the three computers' hard drives with ZFS-on-Linux for storing data. Each box would be configured for capturing ZFS snapshots every hour with some sort of method to the madness of expiring older snapshots.

Now that I have the snapshots, I need a place to send them.

For the "clients" mentioned above, since the file server is available to them whenever they're around, I was thinking of simply having them zfs send their snapshots to the file server.

Question 2: What happens to the snapshots taken by the laptop while it was away from home? If the laptop takes snapshots A and B while on the network and sends them to the file server, then snapshots C and D while off of the network, when it returns in time to take snapshot E, then is it sufficient for it to send only snapshot E to the file server, or does it need to keep track of C and D as well and send all three to the file server?

Question 3: What should the file server do with these snapshots? Is it better space-wise for the file server to zfs receive them into read-only data sets, or does it not matter if it simply hangs onto incremental dumps?

I'm leaning toward running zfs receive on them as I get them; I'd hate to have to replay through potentially thousands of incremental dumps when I need to restore unless someone has a better idea.

If there are no objections so far, then this will get me to the point of my other post because I'll have one single drive that I need to back up on a rotating schedule.

In that post, I asked about setting up two drives in a mirrored vdev configuration similar to RAID1.

This would seem to require both drives living in the same box. While I'm not opposed to that, I'd prefer to have them separated as stated above.

If I choose to do this, it seems it would be just as simple to set this up as I did with the relationship between the file server and its clients.

Question 4: Am I correct? Is it really just this simple?

Finally, the rotating backup. If this separate, dedicated backup computer is running a Debian OS off of something simple such as a small SD card or USB flash drive (i.e. the hard drive itself is purely ZFS data with no OS on it), I would like to be able to swap out that drive with another drive of the same capacity every few weeks.

If I don't proceed with the mirrored vdev configuration and instead simply create separate zpools--individually, one for each drive--then...

Question 5: Is there anything I need to watch out for each time I'm swapping the drives? Will it be as simple as actually swapping the drives, or will I need to go edit any /etc files before/after I make the change?

closed as too broad by Twisty Impersonator, Burgi, Kamil Maciorowski, Josh, music2myear Oct 26 '17 at 22:01

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • Please limit your post to a single question. It is difficult to select a single correct answer when you have at least 5 questions in your post. – music2myear Oct 26 '17 at 22:01