Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I use back-in-time to backup my Linux installation. It serves as an advanced wrapper for the rsync command.

Today I tried to add /var/log to the list of folders to be backed up and it caused some serious performance problems. The job seems to stuck on a particular file and the CPU usage of the rsync parent process reaches 100%. I then used lsof to see which file caused the problem and it seems to be the /var/log directory.

I did some googling and some experiments with the different rsync options and found --checksum to be the offender. Without the parameter, an incremental backup finishes properly in minutes. With it, the process will stuck when rsync tries to sync a constantly changing log file. This kind of make sense, but it still seems to be a bug to me.

Am I using the option correctly? Is there a workaround for this?

share|improve this question
up vote 0 down vote accepted

You might consider performing a log rotation before the sync. Skip the active files, and just backup the archives. You could even make the rsync an postrotate action after the logs are normally rotated.

share|improve this answer
    
Yeah, I've been trying to do that, but back-in-time only allow me to change a small set of options. I can't specify the minimum modification age, so I may need to manually build an exclusion list which doesn't seem to be a robust method. – billc.cn Nov 20 '11 at 21:05

Keep in mind that the checksum is only there to verify that the file transfer has completed successfully.. a means to ensure the file is not corrupt if you will.

This may not be the answer you're looking for but --checksum is going to be slow. There is no way around it. If I were you I would create a quick bash script to compress all of your logs before you run the backup.

Something quick would suffice:

#!/bin/sh
for file in /path/to/log/dir/*;
do;
zip `basename $file`.zip *;
rm -f "$file";
done;

Then run your backup. This would save rsync from creating checksums of possibly huge log files anyway.

Be nice... I know it's not the answer you were looking for.

Good luck

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .