Sign up ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have an OmniOS file server with a ZFS raidz2 array that I want to back up to another machine over gigabit ethernet. The other machine is a Linux (Ubuntu 12.04) system with non-ecc RAM connected to a pile of USB 3.0 external hard drives. I want to use ZFS on the backup machine's external hard drives and ZFS send/receive to backup the data. Data integrity and prevention of silent file corruption are the top priorities. Ideally, I'd like to mount the external hard drives from the linux machine on to the OmniOS server as if they were local disks (ie: block level access to actual hard drives) so that all of the zfs checksumming is done on the OmniOS machine, because it has 8 CPU cores and ECC RAM, while the Linux machine is a "desktop" (HTPC) dual core with non-ECC RAM.

Can this be done using iSCSI? Can this be done using ATA over Ethernet? (the external HDD's show up in linux as SATA disks, ie: /dev/sdb, /dev/sdc, /dev/sdd, etc)

If I were to use ZFS on Linux on the backup machine, how large of a risk am I taking with my backed up data in relation to silent corruption?

Thanks for any and all input.

Background Info:

I am in the middle of building a virtualized "everything" server and am now trying to come up with a better backup solution. The server will run OmniOS (an opensolaris derivative) as a built-in virtualized SAN, which will keep all of my data on a RAIDz2 (ZFS version of RAID6, two disk redundancy) array. The server will also run bunch of other stuff, like a LAMP stack and a MythTV Backend for recording TV shows from a rack of cable boxes nearby, etc. It's primarily a media server full of music, pictures, movies and TV show recordings.

I am very concerned about data integrity. Given the amount of data I have and the amount that it is manipulated (transcoding for web streaming, internal streaming of movies to many local media display systems, constant additions of gigabytes worth of photos, etc) silent corruption is highly likely. That is why I'm running ZFS everything, ECC RAM, etc.

I have offsite (and BluRay) backups of everything that is mission critical (my work, my memories, etc) but I need to implement an on-site backup for some of the media (I don't have enough internet connection bandwidth to push 80 Gigabytes of HD TV shows to a remote server every day).

I have a HTPC running Ubuntu Linux with MythTv frontend and XBMC (I was indecisive) about 600 feet away from the main server room, on another floor of the building. It is connected back to the server room with Gigabit Ethernet (it passes though the network switch on that floor, so 600 feet of Ethernet cable is not an issue). It has a collection of 2 TB external hard drives (WD MyBook Essentials, if you're curious) connected via USB 3.0 hidden in a locked wall unit. I would like to back up everything to those hard drives.

share|improve this question

1 Answer 1

how large of a risk am I taking with my backed up data in relation to silent corruption?

ZFS takes data integrity very seriously and my personal experience with ZFS send/receive is that it's robust. I would suggest you implement your send/receive in the most straight forward manner and use scrubbing to make sure that any corruption is detected.

I regularly use ZFS send/receive to backup a RAIDZ to external USB 1TB 2.5" consumer drives.

Only once have I encountered an error (detected by scrubbing) on the backup disk. The scrub was done some significant time after the send/receive so I can't be sure whether the error was introduced during the transfer or occurred some time after the data was already on the disk, but I strongly suspect the latter.

The backup disk was a single disk with no RAIDZ, snapshots or multiple copies, ZFS couldn't repair the single corrupted block but it was trivial to fix manually by copying the original file.

share|improve this answer
I agree that ZFS send/receive appears to be quite robust, at least on server hardware. My question related to the risk I'm taking was primarily about the lack of ECC ram on the "backup" machine. ZFS does a lot of work with your data in memory before committing it to disk. I would use mirrored backup drives for redundancy (as opposed to RAID) to prevent non-ECC backed parity calculations (more risky IMHO). Was the machine managing your consumer external hard drive solution equipped with ECC RAM? Glad to hear someone is already using consumer drives with ZFS as a backup medium with some success. –  Jay Sep 24 '13 at 13:21
@Jay, yes the machine was using ECC RAM. I agree with you that ECC is desirable in the backup machine but it won't protect your data as it moves over the network or after it's written to the disk. –  Mike Fitzpatrick Sep 24 '13 at 21:39
Jeff Bonwick's blog offers some insight into ZFS' end-to-end data integrity. Quote: "A ZFS storage pool is really just a tree of blocks. ZFS provides fault isolation between data and checksum by storing the checksum of each block in its parent block pointer -- not in the block itself. Every block in the tree contains the checksums for all its children, so the entire pool is self-validating." –  Mike Fitzpatrick Sep 24 '13 at 21:53
I agree with your approach. I'm no ZFS guru, but my sense is that the risk of having a zpool on a desktop machine with non-ECC will be lesser than adding the complexity of mounting the remote drives to the server over ethernet. ECC mitigates the less-likely issue of memory corruption--and scrubbing a pool with multiple copies is likely to catch any issues due to drive corruption. –  STW Oct 29 '14 at 19:30

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.