Are there technical reasons why specific directories can or should be excluded from backup in Linux, and what are those directories?

Question

I am doing a backup of my Linux CentOS 6 using Rsapshot.

I already have daily backups that do only critical things like databases and emails.

However, I'd like to set-up a "total" backup where nearly the whole machine will be mirrored. This is for "just in case" paranoia.

I understand that some directories just hold temporary files, or are re-created at each bootup, or have other considerations that suggest there is no benefit to backing them up. In an "off-the-shelf" CentOS 6 installation, for which directories is there no benefit to backing them up, and what is the technical reason applicable to each of these?

p.s. CentOS filesystem hierarchy

Answer 1

Anything in /tmp, /run or /shm you don't need to backup. Exclude /dev/* from backup as it's created on each boot, and /proc/* and /sys/* as those are virtual filesystems.

/var/log typically houses log files, consider removing old log files before backup.

Check which applications are using /var/spool and /var/cache - you may not need to backup something there depending on your needs. For example, if you are running squid and restoring a system you may want to recreate the squid cache anyway, so don't bother backing it up.

Everything else, you should backup.

Answer 2

General Thoughts

A couple of general thoughts, first. I assume your objective in excluding unnecessary directories is to save backup space and file transfer time. Don't worry too much about excluding every last unnecessary snippet. It may not save significant space or time.

• If you use incremental backups, it will save only files that have changed, and a lot of stuff rarely changes. So even if you backup some unnecessary directories, it may not add much content after the initial backup.

• Some directories contain mainly empty directories and/or small files. Particularly if the backup uses some form of archive file, these directories take up little space. When selecting directories for exclusion, look at their size. It may not be worth messing with exclusion of directories that take little space. There's a chance that some future software may stick something there that you may be unaware of, in which case it would then never get backed up.

• If you don't use incremental backup, you can save some space and time by structuring the content to separate things that change from things that don't. This is easy to do in your home directory even if you don't want to mess with the system directories. Make a new backup of the fixed content only as needed.

Specific Suggestions

LawrenceC focused on system directories. You can also apply your directory exclusion objective within the /home directory, and there can be some substantial chunks depending on what you have loaded (I assume that CentOS is similar to other Linux distros there):

Exclusions

• .local/share/Trash (your deleted files, unless you want to save them)
• Search for directories named cache or .cache. These are temporary files.
• Search for directories named things like log or Crash Reports. These often contain records that grow forever of activity or problems. You might want to review them, but if the information is ancient and you aren't having problems, there isn't a lot of value in preserving them.

Periodic Backup, Only

There are some directories that can get pretty large and may not change, depending on how you use them, or the changes may not make much difference if you are doing an emergency restoration. These could include a VM or WINE (and/or PlayOnLinux). If you have these and they are large and don't have critical day-to-day changes, you could back these up separately on a less-frequent schedule.

Segregate Files

If you are a collector or saver of videos, music, images, or historical emails, these collections can get big. Often, you add to them but the previous files don't change. If you do full backups, these can eat up significant space in each backup. One way to save backup space and time is to segregate the "historical" files in a separate directory. Back those up separately on an as-needed basis.

Answer 3

• Browser caches: Caches of firefox and chrome/chromium should be excluded since they are useless for backup purposes:

  exclude /home/username/.cache/mozilla/firefox/*.default/cache2/entries/*
  exclude /home/username/.cache/mozilla/firefox/*.default/cache2/doomed/*
  exclude /home/username/.cache/chromium/*
  

• Google-earth cache: Can take a lot of place and again useless for backup purposes:

  exclude /home/username/.googleearth/Cache/unified_cache_leveldb_leveldb2/*
  

• Virtual machine VDI/VMDK/VHD's: Even a small change inside a VM requires the whole file to be copied. Instead a great tool like vdfuse (https://github.com/Thorsten-Sick/vdfuse) can be utilized to mount the VM disk images, and add to the the source path for incremental backup. An emptied copy of the VM disk image can be kept for recovery purposes (mount and copy files).