Apparently the proper term for this is that I want a backup that is "atomic." The --delay-updates
switch is termed in the man
page, "a little more atomic," doing all the deletions and moves, right at the end. It worked for me, like this:
HOME="/home/me"
BACKUPFOLDER="$HOME/Backup/"
DIRECTORYLOG="$BACKUPFOLDER/<directory>.log"
NICERSYNC="nice -n 19 ionice -c 3 /usr/bin/rsync"
REMOTEHOST="[email protected]"
REMOTEBACKUPFOLDER="/home/me/Backup"
# partial-dir must be relative when using it with temp-dir
RSYNCSWITCHES="-acv --itemize-changes --delay-updates --delete-delay --partial-dir=<directory>-partial --temp-dir=$REMOTEBACKUPFOLDER/<directory>-temp"
$NICERSYNC $RSYNCSWITCHES "$BACKUPFOLDER/<directory>" $REMOTEHOST:\"$REMOTEBACKUPFOLDER\" &>> "$DIRECTORYLOG"
Though I don't really understand the details, when --partial-dir
is used with --temp-dir
in the context of --delay-updates
, it needs to be given a relative path.
There is also a script, atomic-rsync
in the /usr/share/rsync/scripts
directory. Quoting from the man
page:
See also the "atomic-rsync" perl script in the "support" subdir for an
update algorithm that is even more atomic (it uses --link-dest and a
parallel hierarchy of files).
The problem for me is, that's only for pulling to a local directory, rather than pushing to a remote. I've scripted one that should do the job properly, below. After the first successful backup, <directory>
will always exist on the remote as a complete backup. Interrupted backups will appear as <directory>.1
, <directory>.2
, etc. Any one that rsync
's to completion, will be moved to <directory>
, and the other <directory>.n
will be deleted.
HOME="/home/me"
BACKUPFOLDER="$HOME/Backup/<directory>"
DIRECTORYLOG="$BACKUPFOLDER/<directory>.log"
LOCALFOLDER="/home/dev/Backup"
LOCALSUBFOLDER="Restic/<directory>"
LOCALSOURCE="$LOCALFOLDER/$LOCALSUBFOLDER"
REMOTEHOST="[email protected]"
REMOTEHOME="/home/me"
REMOTEFOLDER="Backup"
REMOTESUBFOLDER="<directory>"
REMOTETARGET="$REMOTEFOLDER/$REMOTESUBFOLDER"
NICERSYNC="nice -n 19 ionice -c 3 /usr/bin/rsync"
SWITCHES="-ac --partial-dir=\"$REMOTEHOME/$REMOTETARGET-partial\""
errormsgandexit() {
if [[ $1 -ne 0 ]]; then
echo "$2"
exit $1
fi
}
ssherrormsgandexit() {
SSHCONNECTERROR=255
if [[ $1 -eq $SSHCONNECTERROR ]]; then
echo "$2"
exit $1
fi
}
echo -e "\n$(date) Rsync <directory>\n--------------------" >> "$DIRECTORYLOG"
# If TARGET exists, we use that for links:
ssh $REMOTEHOST "[[ -d \"$REMOTETARGET\" ]]" && SWITCHLINKS="--link-dest=\"$REMOTEHOME/$REMOTETARGET\""
ssherrormsgandexit $? "ssh connect failure while checking $REMOTETARGET directory." >> "$DIRECTORYLOG"
# Find the smallest n such that TARGET.n doesn't exist. use TARGET.1-(n-1) for links.
declare -i n=1
while ssh $REMOTEHOST "[[ -d \"$REMOTETARGET.$n\" ]]"; ! ((RETURN=$?)); do
SWITCHLINKS="$SWITCHLINKS --link-dest=\"$REMOTEHOME/$REMOTETARGET.$((n++))\""
done
ssherrormsgandexit $RETURN "ssh connect failure while checking $REMOTETARGET.$n directory." >> "$DIRECTORYLOG"
SWITCHESLINKS="$SWITCHES $SWITCHLINKS"
# now I am free to copy to TARGET.n:
$NICERSYNC $SWITCHESLINKS "$LOCALSOURCE/" $REMOTEHOST:\"$REMOTETARGET.$n\" >> "$DIRECTORYLOG"
errormsgandexit $? "rsync error, code $?." # >> "$DIRECTORYLOG"
# if rsync was successful, just keep the most recent complete TARGET
ssh $REMOTEHOST "([[ ! -d \"$REMOTETARGET\" ]] || rm -r \"$REMOTETARGET\") && \
mv \"$REMOTETARGET.$n\" \"$REMOTETARGET\" && \
([[ ! -d \"$REMOTETARGET.1\" ]] || rm -r \"$REMOTETARGET.\"*)" >> "$DIRECTORYLOG"
RETURN=$?
ssherrormsgandexit $RETURN "ssh connect failure while reordering $REMOTETARGET\* directories." >> "$DIRECTORYLOG"
errormsgandexit $RETURN "error when reorganizing $REMOTETARGET* directories." >> "$DIRECTORYLOG"