Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I use vmware fusion to prototype and model enterprise infrastructure. This question is intended to solicit an optimal method for backing up (preferably incrementally) VMs.

My VM directory is now around 250 gig, mounted on my onboard SSD, and I need to back it up occasionally but it takes forever as vmware seems to touch all the files in a .vmwarevm directory so that any reasonable backup solution will regard them as dirty. I've also read that they are sparse files so there should be optimisations available. I have tried

1) rsync -azP  --exclude-from=xlist.lst /Users/pcarr01/VM/ /Volumes/SD128GB

rsync to local thunderbolt SSD - no compression and too slow, IO did not seem to match figures quoted for hardware

2) rsync rsync -aP * admin@persistence::VMs

rsync to Netgear ReadyNAS duo - the NAS ,maxes out on NAS CPU quite raipdly and nothing much happens, takes over 24 hours for full backup

3) tar based solutions

#!/bin/bash
# backup tars to target ignoring vms that are older than backups
# args target where target is name of volume [/Volumes/???]
[[ -z $1 ]] && {
    "No target specified"
    exit 1
}
target=$1
dst_path="/Volumes/${target}/vmtar/"
path="/Users/pcarr01/VM/"
ls -d /Users/pcarr01/VM/*.vmwarevm | grep -vf ${path}/xlist.lst | while read file
do
    dst=${file/$path/$dst_path}.tgz
    echo backing up $file to $dst
    [[ ! -e "$dst" ]]  && echo creating new tar && tar -czf "$dst" "$file"  &
    sleep 3
    [[ `stat -f "%m" "$dst"` -lt `stat -f "%m" "$file"` ]] && echo updating tar && tar -czf "$dst" "$file" &
done

this backs up concurrently and takes less than 2 hours only cause bsdtar on the mac appears to be single threaded and the bottleneck is the one thread @ 100% for each tar, so it never gets near the IO for the backup SSD on the thunderbolt

Any better ideas?

share|improve this question

migrated from serverfault.com Aug 12 '13 at 4:09

This question came from our site for professional system and network administrators.

1  
I use VMware Fusion too, but I don't give a toss about my VMs, as I'll just rebuild/redeploy them. –  Tom O'Connor Aug 9 '13 at 14:31
    
Would this be a laptop? Would backing up to an internal drive not be an option? I'd also wonder about the performance and overhead of thunderbolt. There's just so many possible elements here –  Journeyman Geek Aug 9 '13 at 14:39
    
2  
By the way, don't use rsync compression on local copies... –  ewwhite Aug 9 '13 at 14:51
    
Thx guys, I need to maintain state, it is a laptop & yes, thunderbolt is vastly over-hyped, and @ewwhite you are quite correct on the compression. –  Paddy Carroll Aug 9 '13 at 19:57

2 Answers 2

Typically, really important VM's should be moved off the workstation and to a dedicated server.

On My Mac, I turn on the auto snapshot feature inside fusion, and then let crashplan backup to my NAS. I can get away with this because I don't make a lot of changes to my VM's.

You could try a multithreaded compression program such as pigz.

share|improve this answer
    
Thanks for the info on pigz! v cool, will compile and try. I cant use a dedicated server on any form of public transport however :) –  Paddy Carroll Aug 9 '13 at 19:52
    
Yes, local VM's are faster. If you have tethering on your phone, then hosting the VM on an VPS, and ssh-ing into it remotely may work. (Assuming you don't need a gui). –  spuder Aug 9 '13 at 19:55
    
as long as the other commuters can tolerate the swearing when I lose the link, thats a lot of swearing from Hampshire to London :( –  Paddy Carroll Aug 9 '13 at 19:59

Most people don't back up their workstation-hosted virtual machines. The options for this are limited and the use cases are narrow.

Time Machine and rsync-style backups won't be helpful from an incremental perspective.

Unfortunately, you haven't given any information about the operating systems running as guests on your Macintosh desktop/laptop. Your best case is to worry about the in-VM data; e.g. your configuration and data directories. You can also run native backups. If using Windows, it's quite easy to back up to a share somewhere. If Linux, also quite easy.

It's worth reading VMware's best practices guide for this, however.

share|improve this answer
    
The VMs are typically W2k8R2, W7, Debian/Ubuntu 6&7 RHEL 5+ & Centos. Some of these have data stores and some dev environments but it is mainly Application Servers, databases, some enterprise middleware and not a little systems management infrastructure. So my use case is quite specific, if my laptop gets left on to of the car or nicked I will have a backup. I don't need the hassle of many backup solutions on one box. –  Paddy Carroll Aug 9 '13 at 19:15
    
Your best option is to take full block-level backups of your Mac. –  ewwhite Aug 9 '13 at 19:29
    
So, a crap answer to a crap question :) –  Paddy Carroll Aug 9 '13 at 19:45
    
+1 for backing up the in-vm data if it's really important (treat them like physical machines) -- I know it doesn't achieve what you probably want (the ability to restore your VMs quickly and easily), but you often get stuck in a Bad Place trying to back up workstation virtualization solutions... –  voretaq7 Aug 9 '13 at 19:53
    
@PaddyCarroll Okay... But that's a lot to store on your laptop. Do you have any server-based options available? –  ewwhite Aug 9 '13 at 23:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.