Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

On my Linux machine I'm using a home made backup script which is actually a few rsync calls. I've tested my restores and everything seems to be working, but are there any possible problems with this setup?

My main concern is atomicity of this backup. As far as I know, files in Linux aren't locked. Can files be copied if only partially written? Is it possible that databases, xml files or any other frequently written files that have a structure or a syntax could end up broken or unusable at a backup location?

share|improve this question
up vote 3 down vote accepted

For an atomic backup you ideally need to stop all write access to the area being backup up which means stopping all services that might write to them.

If you use LVM then the snapshot facility makes this much less onerous as you only need to stop services for the length of time it takes to create the read-only snapshot which is almost instant. You take the backup from the snapshot and then remove it until a new one is needed for the next backup run. See for more detail.

Don't leave the snapshot(s) active any longer than you need to though, as there are performance implications (though these are usually far less of a problem then having to have a full downtime period while the backups run).

share|improve this answer
Some services, such as databases, have facilities for 24x7 operations (hot log archive, transaction logs, online dumps). Many other services do not keep critical files open (e.g. xinetd, xdhcpd); they do write to files but these are usually very quick; the LVM copy would ensure these files are completely valid though I've never had problems without that. – Kevin Brock Feb 5 '10 at 11:03

Regarding databases sometimes is safer to dump the data and then backup the dump. For example, MySql comes with this tool mysqldump that can accomplish this.

I use Bacula to do my backups, and I have it configured to run mysqldump to dump the databases into a directory for the dumps and then bacula backups this directory. I do something similar for SVN.

I don't have the restore procedure automatized, but at least I have the data in a format easy to import into MySql and probably into other databases since the dump file is just SQL

share|improve this answer

Yes, they can still be copied. Like suggested above I would dump the databases into a file, bzip them for good compression then rsync them to wherever

# mysqldump --all-databases -u user -p | bzip2 -c > mysqlbackup.sql.bz2
# <rsync stuff here>

Then to load it into the new database

# bzip2 -d mysqlbackup.sql.bz2
# mysql -u user -p < mysqlbackup.sql

To further ease any paranoia I sometimes revert to lsof -F | grep <keyword> to see if the file I want to transfer is actually in use, or open. If lsof returns null then I know I'm right to continue. You could use it to search for an open MySQL table as MySQL is writing to it. Once lsof returns null you can continue transferring files.

share|improve this answer
Depending on how little your database changes between backup runs (in most cases a very small amount of data is added/updated/removed between regular backups, relative to the total data size), if you are backing up over an external (i.e. slow) network link you may well find rsyncing the pure backup file more efficient then compressing with bzip or similar first. After bzip2 compression it is most likely that rsync will send the entire file and that may well be bigger than sending the differences between the uncompressed files. – David Spillett Feb 3 '10 at 15:06

According to this page, you should back the database to a file and then stop it before the rsync. It's not ideal as you have to backup the whole database every time. A specialised database backup procedure is best.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .