I do in such cases when one need quick access to backuped content - incremental backup using rsync
with --link-dest=
option.
It runs pretty fast, don't effect previous backup copies that you can have as many as you want and backup don't take a lot of space since backup copies are hard links to existent files. In this case access to backup is instant and actual file transfer takes a seconds even on huge amount of files since rsync
copied only new files.
#!/bin/sh
srcDir='/importunt/data' # Use full path
bkpDir='/backups' # Use full path
cd "${bkpDir}"
previousDir="$(ls -td -- */ | head -n 1 | awk -F'/' '{print $1}')" # Get most newest directory
currentDir="$(date '+%Y-%m-%dT%H;%M;%S')"
[ -n "${previousDir}" ] && {
rsync_opts="-aPvz --safe-links --link-dest=${bkpDir}/${previousDir} --exclude=*.mp3"
} || {
rsync_opts="-aPvz --safe-links --exclude=*.mp3"
}
mkdir -m 770 "${currentDir}"
rsync ${rsync_opts} "${srcDir}" "${bkpDir}"/"${currentDir}"
Basically such solution create exact snapshot in time, so restoration of files are pretty easy.
Don't be scary if you using du
on /backups
directory when it show increasing size on each update, if you will use df
you will find that actual space isn't reduced. That is how hard links counted on Linux and FreeBSD, so no worry. To be make sure I don't lied you can check inode on some file in incremental backup with ls -i file
. You will find that the same file in all directories has the same inode which is means rsync
duplicate only filenames with hard links but all of them pointing to the same content.
One more advantage of this method is that you can delete oldest backup directories in any order, - latest, intermediate or the oldest ones.
Script above is simplified example. If content in incremental backup supposed to be edited then you shouldn't use mechanism of ls -t
to detect newest previous directory in backup but instead save ${currentDir}
to some file and restore to ${previousDir}
on subsequent call.
Since rsync
supports transfer over ssh
you can move incremental backup to remote machine with the same efficiency, the only changes will be synchronized.