I have a little home server (Ubuntu Server 17.10) and I'd like to setup some backups for it. My main points are:

  • The server has two hard drives. The first one (an SSD) is partitioned with the EFI partition and an LVM space; inside there are two LVM volumes, and I want to backup one of them. I left enough space to make a snapshot volume when needed. The second one has two partitions, and I want to mount the second one when needed to store the backups. These two partitions on the second HDD are not part of the LVM space
  • The server is not on 24/7
  • I'd to have some backup like this:
    • Every day at 1.00* perform an incremental backup; if the backup is not performed because the PC is off, perform it when ready
    • Keep at most 5* daily incremental backups. If one day the backup was not performed because the PC was off all day, there should always be 5* backups.
    • Every month there should be one full backup (i.e. not incremental), to protect against data corruption
    • Keep at most 3* monthly full backups. The same concept explained in point 2 applies (3* backups, not the backups of the last 3* months)

Note: the time and amount of backups (noted by a *) can be changed.

For instance, today is the 14/03. On 11/03 the PC was off all day, so in the database there should be:

Incremental backups: 14/03, 13/03, 12/03, 10/03, 09/03 (09/03 can also be complete if necessary) Complete backups: 01/03, 01/02, 01/01

Now, I'm quite new to this, so I started looking around and I found Attic and then Borg. But I can't understand if what I want to achieve is doable with borg or not, or if there are easier ways to do it rather than using this program (considering that I'm on LVM so maybe I can use a snapshot to "track" changes or.. whatever).

So... Can what I described be implemented in a single (or double) borg script? Do you recommend other tools (or a plain copy)? Or you think that the strategy I described is useless/not enough?

Thank you

NOTE: I mistakenly opened this on serverfault, and since I could not move it and there were no comments nor answers, I preferred to delete the old one and open it anew here in order to speed up the transition and avoid leaving this task to the moderators

EDIT: Maybe I explained not in the proper way, so I'm making some more lines in this question.

My experience with "incremental" snapshots comes from Virtualbox. In this case, when you remove a snapshot the modifications get merged.

In order to avoid writing too much, I'm making an example using at most three daily backups.

Assume we are in the condition

  • Daily backup 12/03 - Full copy of file system on 12/03
  • Daily backup 13/03 - Modifications between 12/03 and 13/03
  • Daily backup 14/03 - Modifications between 13/03 and 14/03

Today is a new day, so the backup starts. But there are already 3 daily backups. So the 12/03 backup and the 13/03 get merged (the 13/03 differences get applied on the base version), and a new incremental backup is made. This way you lose the differences between 12 and 13, but the backup remains "full":

  • Daily backup 13/03 - Full copy of file system on 13/03
  • Daily backup 14/03 - Modifications between 13/03 and 14/03
  • Daily backup 15/03 - Modifications between 14/03 and 15/03

I think this is the normal behavior of borg's prune (correct me if I'm wrong, please)

Another option (which is more desirable in my opinion) would be to merge the incremental backups from the monthly one. I mean something like:

  • Monthly backup 01/03 - Full copy of file system on 01/03
  • Daily backup 12/03 - Modifications between 01/03 and 12/03
  • Daily backup 13/03 - Modifications between 12/03 and 13/03
  • Daily backup 14/03 - Modifications between 13/03 and 14/03

When pruning, the 12 and 13 get merged (so it becomes Daily backup 13/03 - Modifications between 01/03 and 13/03) and a new one is created.

The last option, less desirable in my opinion (but if you think it is better you can argument this) is to keep daily backups relative to the monthly one.

  • Monthly backup 01/03 - Full copy of file system on 01/03
  • Daily backup 12/03 - Modifications between 01/03 and 12/03
  • Daily backup 13/03 - Modifications between 01/03 and 13/03
  • Daily backup 14/03 - Modifications between 01/03 and 14/03

In this case it's simpler: just remove the daily backup without looking behind you.

  • Typically when people talk about a full backup once a month, then incremental backups each day, they are each relative to the day before, so if you only kept the most recent 5, after day 5 you would have some backed up data that you no longer have a backup of and would lose if you had to restore. Or did you mean that the incremental backups should be relative to the most recent full backup, not to the last incremental one? – psusi Mar 15 '18 at 16:40
  • @psusi For the moment the only "incremental" stuff I have a small of experience with is Virtualbox snapshots. In any case no, I don't mean a "lossy" way of backing up. I'm editing the question to further explain this – frarugi87 Mar 15 '18 at 18:14

You can't really "merge" backups. It isn't quite what you asked for, but this may fit your needs. The backup system I use is a cron job involving multilevel dump in a tower of hannoi pattern. I believe you can use anacron to make sure runs scheduled for when the system is off are performed when it next is on.

set -e
declare -a LEVELMAP=(1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5)
DATE=`date +%-d`
echo Performing a level $LEVEL dump
lvcreate -s -n snap devserv2/root -L 1g
dump -$LEVEL -quz9 -b 1024 -f /backup/dump.$LEVEL /dev/mapper/devserv2-snap
lvremove -f devserv2/snap

dump has the concept of levels. A level 0 dump contains everything. A level 1 dump contains everything that has changed since the last level 0. A level 2 has everything since the last level 0 or level 1, whichever is more recent, and so on. By performing the listed level dumps on each day of the month, you get a level 1 on the 1st and 17th of the month. These contain everything since the last level 0, which I do manually every couple of months. The rest of the days of the month you use levels 2-5. That means that at any time, you have 3 levels of backups that contain changes since the 1st or 17th of the month, and frequently changed files will have multiple versions you can go back to in each of those backups, varying in age from 1 to several days.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.