Backing up multiple vmware VMs from a mac

Question

I use vmware fusion to prototype and model enterprise infrastructure. This question is intended to solicit an optimal method for backing up (preferably incrementally) VMs.

My VM directory is now around 250 gig, mounted on my onboard SSD, and I need to back it up occasionally but it takes forever as vmware seems to touch all the files in a .vmwarevm directory so that any reasonable backup solution will regard them as dirty. I've also read that they are sparse files so there should be optimisations available. I have tried

1) rsync -azP  --exclude-from=xlist.lst /Users/pcarr01/VM/ /Volumes/SD128GB

rsync to local thunderbolt SSD - no compression and too slow, IO did not seem to match figures quoted for hardware

2) rsync rsync -aP * admin@persistence::VMs

rsync to Netgear ReadyNAS duo - the NAS ,maxes out on NAS CPU quite raipdly and nothing much happens, takes over 24 hours for full backup

3) tar based solutions

#!/bin/bash
# backup tars to target ignoring vms that are older than backups
# args target where target is name of volume [/Volumes/???]
[[ -z $1 ]] && {
    "No target specified"
    exit 1
}
target=$1
dst_path="/Volumes/${target}/vmtar/"
path="/Users/pcarr01/VM/"
ls -d /Users/pcarr01/VM/*.vmwarevm | grep -vf ${path}/xlist.lst | while read file
do
    dst=${file/$path/$dst_path}.tgz
    echo backing up $file to $dst
    [[ ! -e "$dst" ]]  && echo creating new tar && tar -czf "$dst" "$file"  &
    sleep 3
    [[ `stat -f "%m" "$dst"` -lt `stat -f "%m" "$file"` ]] && echo updating tar && tar -czf "$dst" "$file" &
done

this backs up concurrently and takes less than 2 hours only cause bsdtar on the mac appears to be single threaded and the bottleneck is the one thread @ 100% for each tar, so it never gets near the IO for the backup SSD on the thunderbolt

Any better ideas?

Answer 1

Most people don't back up their workstation-hosted virtual machines. The options for this are limited and the use cases are narrow.

Time Machine and rsync-style backups won't be helpful from an incremental perspective.

Unfortunately, you haven't given any information about the operating systems running as guests on your Macintosh desktop/laptop. Your best case is to worry about the in-VM data; e.g. your configuration and data directories. You can also run native backups. If using Windows, it's quite easy to back up to a share somewhere. If Linux, also quite easy.

It's worth reading VMware's best practices guide for this, however.

Answer 2

Typically, really important VM's should be moved off the workstation and to a dedicated server.

On My Mac, I turn on the auto snapshot feature inside fusion, and then let crashplan backup to my NAS. I can get away with this because I don't make a lot of changes to my VM's.

You could try a multithreaded compression program such as pigz.