I use vmware fusion to prototype and model enterprise infrastructure. This question is intended to solicit an optimal method for backing up (preferably incrementally) VMs.

My VM directory is now around 250 gig, mounted on my onboard SSD, and I need to back it up occasionally but it takes forever as vmware seems to touch all the files in a .vmwarevm directory so that any reasonable backup solution will regard them as dirty. I've also read that they are sparse files so there should be optimisations available. I have tried

1) rsync -azP  --exclude-from=xlist.lst /Users/pcarr01/VM/ /Volumes/SD128GB

rsync to local thunderbolt SSD - no compression and too slow, IO did not seem to match figures quoted for hardware

2) rsync rsync -aP * admin@persistence::VMs

rsync to Netgear ReadyNAS duo - the NAS ,maxes out on NAS CPU quite raipdly and nothing much happens, takes over 24 hours for full backup

3) tar based solutions

# backup tars to target ignoring vms that are older than backups
# args target where target is name of volume [/Volumes/???]
[[ -z $1 ]] && {
    "No target specified"
    exit 1
ls -d /Users/pcarr01/VM/*.vmwarevm | grep -vf ${path}/xlist.lst | while read file
    echo backing up $file to $dst
    [[ ! -e "$dst" ]]  && echo creating new tar && tar -czf "$dst" "$file"  &
    sleep 3
    [[ `stat -f "%m" "$dst"` -lt `stat -f "%m" "$file"` ]] && echo updating tar && tar -czf "$dst" "$file" &

this backs up concurrently and takes less than 2 hours only cause bsdtar on the mac appears to be single threaded and the bottleneck is the one thread @ 100% for each tar, so it never gets near the IO for the backup SSD on the thunderbolt

Any better ideas?

  • 1
    I use VMware Fusion too, but I don't give a toss about my VMs, as I'll just rebuild/redeploy them. Aug 9 '13 at 14:31
  • Would this be a laptop? Would backing up to an internal drive not be an option? I'd also wonder about the performance and overhead of thunderbolt. There's just so many possible elements here
    – Journeyman Geek
    Aug 9 '13 at 14:39
  • 2
    By the way, don't use rsync compression on local copies...
    – ewwhite
    Aug 9 '13 at 14:51
  • Thx guys, I need to maintain state, it is a laptop & yes, thunderbolt is vastly over-hyped, and @ewwhite you are quite correct on the compression. Aug 9 '13 at 19:57

Most people don't back up their workstation-hosted virtual machines. The options for this are limited and the use cases are narrow.

Time Machine and rsync-style backups won't be helpful from an incremental perspective.

Unfortunately, you haven't given any information about the operating systems running as guests on your Macintosh desktop/laptop. Your best case is to worry about the in-VM data; e.g. your configuration and data directories. You can also run native backups. If using Windows, it's quite easy to back up to a share somewhere. If Linux, also quite easy.

It's worth reading VMware's best practices guide for this, however.

  • The VMs are typically W2k8R2, W7, Debian/Ubuntu 6&7 RHEL 5+ & Centos. Some of these have data stores and some dev environments but it is mainly Application Servers, databases, some enterprise middleware and not a little systems management infrastructure. So my use case is quite specific, if my laptop gets left on to of the car or nicked I will have a backup. I don't need the hassle of many backup solutions on one box. Aug 9 '13 at 19:15
  • Your best option is to take full block-level backups of your Mac.
    – ewwhite
    Aug 9 '13 at 19:29
  • So, a crap answer to a crap question :) Aug 9 '13 at 19:45
  • +1 for backing up the in-vm data if it's really important (treat them like physical machines) -- I know it doesn't achieve what you probably want (the ability to restore your VMs quickly and easily), but you often get stuck in a Bad Place trying to back up workstation virtualization solutions...
    – voretaq7
    Aug 9 '13 at 19:53
  • @PaddyCarroll Okay... But that's a lot to store on your laptop. Do you have any server-based options available?
    – ewwhite
    Aug 9 '13 at 23:47

Typically, really important VM's should be moved off the workstation and to a dedicated server.

On My Mac, I turn on the auto snapshot feature inside fusion, and then let crashplan backup to my NAS. I can get away with this because I don't make a lot of changes to my VM's.

You could try a multithreaded compression program such as pigz.

  • Thanks for the info on pigz! v cool, will compile and try. I cant use a dedicated server on any form of public transport however :) Aug 9 '13 at 19:52
  • Yes, local VM's are faster. If you have tethering on your phone, then hosting the VM on an VPS, and ssh-ing into it remotely may work. (Assuming you don't need a gui).
    – spuder
    Aug 9 '13 at 19:55
  • as long as the other commuters can tolerate the swearing when I lose the link, thats a lot of swearing from Hampshire to London :( Aug 9 '13 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.