Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm looking for a way to HOT back up (externally, via USB or LAN) my virtual machines while the machines are still active & running. I have number of challenges:

  1. Dealing with large variety of GUEST OS'es ranging from Windows 2000 - 2008 R2, Centos, SUSE, RedHat, Ubuntu, etc.
  2. Dealing with multiple physical host machines
  3. Dealing with large files (obviously) - in the 10's to 100's of GB's
  4. Machines need to stay active / running DURING BACKUP PROCESS.

Ideally this would be like RAID where data is made redundant, but one of the "drives" is external to the host machine. Maybe?

Each of the host machines currently has internal RAID, but that's not enough in the event that the entire machine gets wiped out because there's no external backup. Furthermore it becomes a problem because each host machine might have a different configuration - CPU / RAM, motherboard, etc.

After a great deal of research (although I'm still relatively fresh on the DFS / storage scene), it seems like I'm looking at creating some kind of clustering / distributed file system environment?

Any pointers would be very helpful.

Edit: I intentionally did not mention host or VM platform as I'm open to the right solution that will work for the needs described. I have experience with Linux and Windows, multiple VM platforms. Don't really care about cost at the moment. Trying to understand what solutions are available.

share|improve this question
    
I just realized this question might be better on ServerFault. Can someone vote to move it there? –  Joshua Jan 27 '12 at 19:22
add comment

2 Answers

There are several obstacles you're facing:

Ensuring that your backups are consistent

On the filesystem level, this is pretty easy. What I do is have all virtual machines storage on LVM volumes (on Linux), and during backup create a snapshot of the relevant volume and backup that snapshot (mount the filesystem temporarily on the virtualization host, copy files, etc.) I'm using dirvish with some custom scripts for backup.

On the application level, it is impossible to insure data integrity without cooperation of the relevant application. If you have a database running inside your VM, you need to instruct that database to flush data to disk, lock tables, create your LVM snapshot and then unlock the database. No need to keep locks during backup since you're backing up the snapshot, which is atomic. Obviously, this step cannot be generalized, and you need to carefully evaluate what's running inside each of your VMs. This problem is the most serious one, and clustering, distributed filesystems, etc., are not going to help you here, unfortunately.

Managing storage

The backups are going to be pretty expensive in terms of network traffic, if you want all machines backed up into one place. I'm using locally connected eSATA hard disks in each virtualization host (I have just 2). I keep one set of disks offsite, and other connected to hosts, and rotate them regularly.

share|improve this answer
    
I do not believe this addresses the issue of backing up while machines are active / running. –  Joshua Jan 19 '12 at 16:13
    
These are OS level issues, they are irrelevant to VMs. The backups are performed under the OS at the hypervisor level. –  skub Jan 19 '12 at 16:32
    
@Josh: LVM Snapshots are created while the VMs are running. They let you do atomic backups while VMs are running, e.g. any changes done inside the VM are not reflected in the snapshot that you're currently backing up. –  haimg Jan 19 '12 at 16:54
add comment

All the major VM vendors provide solutions to back up VMs while they are running.

For instance, if you run vSphere and use VMWare's VDR (or even Veeam) the hypervisor will quiesce the file system while keeping the VM running. You may notice some sluggishness on the VM, but it's services will not stop during a back up. Obviously you're going to want to follow the recommended hardware guidelines to ensure full uptime.

If you use the VMWare solution, you can use iSCSI targets to backup your data. So, for example, you can buy a cheap netgear NAS (~$700) and backup machines to that and carry them off site. You can also use real-time backup software that vendors are selling now. Basically it mirrors all changes to an external appliance and then just sends the deltas of the changes.

Some companies (like mine) back up the data over the VPN to an external site. We haven't added any real-time backups yet, but we're working with our vendors to get that all worked out. We're not a company that likes to send our data to the cloud.

Clearly there are mission critical systems that run on VMs in the enterprise that cannot go down and need to be backed up. Every vendor is going to have a solution for this.

share|improve this answer
    
Good luck backing up reliably anything non-trivial using these tools. See for example this: hiperlogic.com/blog/?p=803 -- and these steps are needed for a simple MySQL backup... –  haimg Jan 19 '12 at 16:51
    
No one should rely on snapshots for a database backup - this is why there is redundancy. A snapshot isn't going to backup all the transactions you lost between the time you took the snapshot and the failure, nor should you expect it to. For instance, Citibank isn't just dropping things to tape 2 times a day and hoping for the best. They have redundant systems distributed across a huge network. –  skub Jan 19 '12 at 20:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.