I'm trying to come up with some backup system which would allow me to transfer the minimal amount of data every time a new backup is made.
Currently, I'm doing an incremental backup via duplicity over to an smb mount. I'm limited with that to around 1MB/s at the moment (wifi/ndiswrapper being silly, but still - I'm not going to get more than 5x speedup by fixing it).
Unfortunately the removed, modified, etc. data makes it hard to keep the incremental backups forever. I can't easily flatten them to a full backup and taking a full copy of 40+GB is taking a bit long now.
There are a couple of things I'd like to achieve with the new system and a couple that I don't care about much:
- I'd like to have a couple of checkpoints, but probably not more than 2 or 3
- I don't care about encryption since the the storage is located on local network
- Compression would be good, but I guess I can just use filesystem/device compression
- I want to be able to completely get rid of all but the latest backup without forcing me to do a full copy again
This seems to be doable with rsync + loop device compression + fs snapshotting... but unfortunately my device doesn't really support snapshots. It's just got the standard debian's list of supported filesystems and I don't really want to use btrfs for backups.
Are there any comparable solutions? How else can I approach it?
Edit: I think that the snapshot part can be actually done with lvm read-only snapshots. That would also give another layer of protection from badly-behaving apps.