3

I do not trust any RAID types for home use. I've got 2 Terabytes of Data that I want to copy (duplicate) to about 4-6 drives for redundancy so that I will never lose these pics and videos I've created of my family. I know that 100% failure prevention is impossible. And yes I know the obvious "copy to many sources such as online storage, DVDs, etc.".

However let's get back to hard disks for now. I am not focusing on anything else for this thread.

So I do not trust RAID for home use, and so if I am going to copy new pictures or video to a primary disk, it would be nice if I could find some kind of controller card that would essentially do a copy on demand to the other drives so that I'd have complete duplicates of anything I do to my primary drive.

My understanding is that RAID 1 does this but then you still have the problem of it being a RAID issue...that you're still being dependent on an array which is not what I want.

I merely want the action of RAID 1 (meaning I write to disk A, write the same to disk b, c, d,.etc.) without the reliance or dependency on any stupid failure prone array.

I think ideally if I can set up a box, put about 6 drives in it and somehow maybe get a couple of controller cards that when I write to a designated drive...or delete, or whatever, it duplicates that action realtime to the other x drives.

Anyone seen anything that can do this such as a card (I want to build my own box and drives, etc. to do this) outside of something like this: http://www.aleratec.com/alhddcrhadid.html or this http://www.abcusinc.com/mm5/merchant.mvc?Screen=CTGY&Category_Code=HDD but I want to do it myself over a controller card(s) in my own box that I will be building?

Obviously a RAID controller card is not what I'm after here. I'm after hopefully a card that allows me to plug in lets say 4 drives internally to it and somehow it duplicates from a designated primary drive to the other 3. I'm not sure if such a thing exists.

Basically I want to build my own server doing what that box does...and put an ASUS board in it, etc. I want complete control over this but I need to find some sort of plan using a card or something that does this duplication without the raid array dependency.

8 Answers 8

12

RAID1 is exactly what you want. Buy a well known controller card so if it craps out you can replace it. That way:

  1. Hard drive fails? Replace it, RAID will rebuild
  2. Controller fails? Replace it, RAID will be intact

My understanding is you can get RAID controllers that keep the configuration on the controller, so the disks are 1:1 duplicates without proprietary information on the disk. This means you can pull it out and put it into another PC.

I don't see how this would be any more "failure prone" than a disk duplicator. With a disk duplicator:

  1. Hard drive fails? Replace it, disk will duplicate later
  2. Duplicator fails? Replace it, disk data will be intact

What happens when your primary hard drive you are duplicating gets a bad sector, and when you are duplicating the duplicator can't read it and writes 0's to your other hard drives? With RAID you would be warned and while the data would be lost on one disk it is still intact on the others.

4
  • I can vouch for this. With RAID 1 there should be no proprietary data that will prevent it booting on the disk. At work we've used RAID 1 arrays, and you can literally unplug a drive from the controller and plug it in to the main system (without the controller) and the drive will work without problem, plug it back into the controller and tell it which is the primary and the array gets rebuilt.
    – Mokubai
    Jan 4, 2010 at 9:39
  • Just make sure you don't buy the same batch of hard drives at the same time. I had two drives fail at the same time. 1st from a mechanical problem, then 2nd the primary drive fail when copying the data to a new drive. Thankfully I had a backup of all the data. Though make sure your RAID 1 supports more than 2 drives. I don't recommend mother board on-board raid controllers. Get a dedicated card.
    – Chad
    May 16, 2011 at 6:33
  • It is much more failure prone than a disk duplicator. Imagine if you accidentally enter in a command (or commands are issued due to a bug) that destroys the filesystem or critical files. With RAID, the destruction will immediately be replicated to both copies, leaving you with nothing. With a duplicator, you still have the other copy. Aug 2, 2013 at 0:00
  • @DavidSchwartz You are not wrong. Aug 21, 2013 at 11:06
8

This idea you have that RAID arrays are failure prone is not valid. Certain kinds of RAID (like RAID 0) are risky, and you can have issues with a controller or if you have more disks fail than the array supports, but correctly implemented fault-tolerant volumes, with replaceable controllers, are still the safest option available.

Let me put it simply: Fault-tolerant RAID arrays do not just fail. Period. Hard drives fail, or controllers fail, but you should build your array to allow for the replacement of either.

You mention concern about rebuilding a RAID 1 mirror if a disk fails. Your alternative is manually copying to different disks. What do you think happens under your plan if a disk fails and you want to replace it? You have to make a whole new copy. This is more work than RAID, not less, because the RAID volume will do it automatically, rather than requiring you to manually make your copy.

Finally, I want to make one more point — mirrored data alone, with or without RAID, is not a backup. This only protects you against one kind of failure: a broken hard disk. A real backup also protects against local disaster (the building burns down) and accidental modification or deletion. This is accomplished by physically separating the backup from the live data, and by keeping multiple versions of your copied data.


With all that in mind, here's what I think you really want to do:

  1. Suck it up and get a simple two-disk RAID 1 volume. Remember that this will only be part of the solution. Use a replaceable add-on card if you're that concerned about it.
  2. Get a few (at least 3) extra disks with external enclosures.
  3. Periodically (where the period depends on the rate of change for your data — daily or weekly should work) use a software package to copy your RAID 1 volume to just one (not all) of your external disks. Software options include Windows Backup, Microsoft SyncToy, or other third-party option.
  4. Rotate which external disk you use each time, and make sure you take your most recent backup to a location some distance away.

This will be far superior to your plan to put a bunch of copies on disks in a server, because your sever will still allow all your disks to be destroyed in single fire and will faithfully copy accidental changes or deletions to all of your disks. If you truly value your data, you will not continue with your current plan.

1
  • Excellent points, and thank you for pointing out the often overlooked objectives of data preservation.
    – tdk2fe
    Feb 15, 2014 at 6:58
3

ZFS can do it either way - either triple (or quadruple, or more) mirroring, or setting the copies option to 3:

Controls the number of copies of data stored for this dataset. These copies are in addition to any redundancy provided by the pool, for example, mirroring or raid-z. The copies are stored on different disks, if possible. The space used by multiple copies is charged to the associated file and dataset, changing the "used" pro- perty and counting against quotas and reservations.

ZFS also works best without a RAID controller, so you can just move the disks to a new system if the old one breaks, and it has checksums of all your data so you know not a single bit has changed.

Yes, it's still a single pool that could break, so to be really paranoid you could have three ZFS pools (ideally consisting of two drives each for redundancy and error-recovery) that you then have a script that automatically send snapshots from the master pool to the other two.

1
  • Shame it doesn't exist on Windows...
    – RomanSt
    Sep 11, 2011 at 13:15
3

You won't be able to add comments until your reputation is over 100. For instance, I do not have 100 reputation so I'm making a new answer.

what I don't like about any RAID, even RAID 1 is that you are reliant on an array. If that array fails, you're screwed. That's why i like the manual copy x drive to x drives route. Rather than relying on a raid to rebuild which if I'm not incorrect after researching, RAID 1 would still have to do.

An array failure is a complete failure. For RAID other than RAID0, the array will not fail if a single disk fails. Yes, you will need to rebuild the array. Some RAID controllers allow you to rebuild from within Windows (see nVidia's NVRAID). Rebuilds can be transparent, but performance will be reduces while the new drive is syncing up.

Bear in mind this is only if the drive fails. We're talking maybe once every 5 years for good hard drives under reasonable load. Check the mean time to failure on your hard drives.

Note that controller failure can be recovered from by swapping in another card.

Lets put it this way, I'd much rather be using Norton Ghost or some other software solution and manually do a disk copy to 1 or more hard drives than risk losing my entire array.

If you want a poor man's solution then put some 1TB drives into your PC as I: J: and K: and then use Robocopy http://en.wikipedia.org/wiki/Robocopy with a combination of the /MON and /MOT and /XC to copy every so often. You would need to find a way to run Robocopy in the background - see Srvany http://support.microsoft.com/default.aspx/kb/137890

1
  • I can add comments. I guess the rule changed. good answer.
    – at.
    Jan 31, 2010 at 0:04
2

On linux you can do exactly that, a RAID 1 with more than 2 disks, (i have done this with four sata disks) About drivers or card you don't need any you just install disk drives as JBOD (just bunch of drives) and linux takes care about the rest.

I don't care about the speed performance but, if you have N disks there N-1 disks could fail and even then there would not be loosing data.

1

I think all the good answers were given.

I like ZFS because cryptographic verified reads/writes, multi-drive mirror, free with OpenSolaris, FreeBSD, Linux and appliances OSs like FreeNAS (which is FreeBSD).

I like RAID 1 controller because the drives will be identical regardless of OS. If you must, you can have two controller cards, each with two drives and have the OS (like Windows) make a software mirror across the two RAID 1 controllers. So all four drive should be the same.

Lastly, I like the Robocopy script and mirror between separate drives.

If you are as concerned about data integrity as it sounds, you would love ZFS. It is my choice and I will be building a server with a RAIDZ2 or I hear they will have RAIDZ3 & 4 coming out soon. I think RAIDZ3 means you can lose 3 of 10 drives.

Also, if you are going this far, you may want to replicate between machines because if the power supply in the machines smokes you could fry all the hard drives in that machine.

Of course none of this is the same as a backup. Because if all this works right, and a user, glitch or virus comes along and somehow deletes or modifies a file you have no way to revert (except ZFS which could take snapshots if configured, or if the RAID controller does snapshots). But as you said you don't want backup strategies. I assume this information need high availability to keep so many online copies.

You have some great answers to help you choose you data storage strategy and I hope your data remains safe.

0

Interesting question; if you're doing RAID 1 mirroring to two drives, it does seem logical to allow three drives.. or even (n) drives do the same thing. I found one reference:

If you are having challenges finding examples of multiple drive (more than two) RAID 1 implementations, try searching on "triple mirror RAID". It has been a host operating system capability for 15-17 years. Storage system vendors and even application vendors like Oracle also support triple mirror RAID. Another implementation of multiple drive RAID 1 would be a hybrid RAID 0+1 or 1+0 also known as RAID 10. This implementation works well when, for example, six disk drives are used with three each in the two unique RAID groups that are then striped or mirrored together (this depends on whether you're using 0+1 or 1+0).

Another way of finding specific implementations is to look for vendors that implement a combination of horizontal and vertical RAID. This is where one RAID group is created horizontally across a group or shelf of disk drives and the other RAID group, using perhaps a different RAID level, is created vertically across the horizontal RAID groups usually more for performance than availability.

although this "triple mirroring and beyond" doesn't seem to be supported by many controllers:

Its possible to make a 8 drive RAID 1 if you want, provided your controller supports it. But what a waste of space. It is not recommended to be swapping drives out of a raid 1 for backup. Thats adds undue stress on the whole system.

I thought the same thing until i did research as no Dell controller, or lsi logic or adaptec controller supports that.

0

To me it sounds like you want to be looking at a higher level than RAID controller for your backups. It look at using OS and/or Software to create the copies of your files across multiple drivers.

If you're building a backup box it would be straightforward to use something like rsync and cron if your running Linux to copy files between different drives. Or you could use Windows Home Server which will create redundant copies of files across drives out of the box.

For me the two advantages this software level copying are:

  • The drives can be vanilla formatted so you can pull one out and pop it in different machine and it will just work. You won't be tied to a particular RAID controller.
  • The drivers don't have to be exactly the same size. This makes it much easier to re-use all those HDDs you have lying around the house.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .