14

Let's say I have two 2TB HDDs and I want to start my first ZFS zpool. Is it possible to create a RAIDZ with just those two discs, giving me 2TB of usable storage (if I understand it right) and then later add another 2TB HDD bringing the total to 4TB of usable storage. Am I correct or does there need to be three HDDs to start with?

The reason I ask is I already have one 2TB drive I'm using that's full of files. I want to transition to a zpool but I'd rather only buy two more 2TB drives if I can. From what I understand, RAIDZ behaves similarly to RAID5 (with some major differences, I know, but in terms of capacity). However, RAID5 requires 3+ drives. I was wondering if RAIDZ has the same requirement.

If I have to, I can buy the three drives and just start there, later adding the fourth, but if I could start with two and move to three that would save me $80.

5 Answers 5

15

A method some people use, but that is not recommended:

  1. Create a pool with 2 drives and a sparse file (of the right, virtual size to match the other drives), and then immediately offline the sparse file. This will create a degraded RAID-Z1 pool with two drives worth of capacity and no redundancy.
  2. Copy data to the degraded pool.
  3. Replace the offlined sparse-file with the third disk and resilver.

The number of "devices" that a vdev is created with, is permanent after creation. Therefore, you must have three "disks" at the time of creation. This method gives a way to create a degraded RAID-Z1 pool and later restore the absent redundancy.

This method is not recommended, because while copying and while the 3rd drive is absent, there is no redundancy/parity. But it can work.

2
  • 1
    Thanks for this answer. We're not always looking for the recommended solution. Sometimes we're looking for ANY solution. A link to how to do what you explain would be nice
    – jsaddwater
    Sep 15, 2018 at 21:31
  • Unfortunately I can't create a 1TB sparse file on a 250GB drive, so when creating a RAIDZ array with 2 1TB drives + one sparse file (10GB) the array becomes 3x10GB. After removing the sparse file I couldn't grow the array because I couldn't copy the data from the third drive to a small array of 3x10GB and the array can't be grown in degraded state. Catch 22.
    – jsaddwater
    Sep 20, 2018 at 16:12
6

Unfortunately, at the moment it is not possible:

It is not possible to add a disk as a column to a RAID-Z, RAID-Z2, or RAID-Z3 vdev. This feature depends on the block pointer rewrite functionality due to be added soon. One can however create a new RAID-Z vdev and add it to the zpool.

RAID-Z1 might work with just 2 drives, but clearly this is not very useful if extra drives cannot be added later. RAID-Z (like RAID 5) becomes more efficient, in terms of usable storage space, the more drives that are used.

3
  • 4
    Here we are three years later, and the Wikipedia page still says "due to be added soon" ... :(
    – Icydog
    May 22, 2014 at 0:19
  • 2
    @Icydog real soon now ;-)
    – jlliagre
    Apr 27, 2016 at 9:49
  • what about now? :) Jan 1, 2019 at 11:09
3

RAID-Z1 can be used with only two disks but there is no advantage at all compared to mirroring - unless your two disks have different sector sizes, so that they can't be mirrored (e.g.: a new 4K sector disk and an older 512 byte disk).

As already stated, adding a device to a RAID-Z isn't supported. Should you want to add a disk to an existing RAID-Z, the only way is to backup your data elsewhere, destroy the the pool and recreate it with the extra disk added then restore your data.

3

Don't have enough "reputation" to comment, so putting this in as a separate answer

RE: Holger G suggestion

while I agree that this is NOT safe... I found this thread looking for a way to go from RAIDZ1 to RAIDZ2 at a later date when extra drive can be acquired.

So Holgers suggestion works great for me - create a RAIDZ2 with one dev as sparse file, offline the file device and have effectively RAIDZ1 for a while and then add a real device at some point to double the redundancy

Also works for the situation where the one drive holds the data that is to be copied to the new raid, as the OP has.

3
  • RAIDZ2 are silly -- the only situation, when the configuration might be useful, is a correlated failure of two drives. Instead of using two disks to guard against that, protect yourself against such failures by using same-capacity disks from different manufacturers and/or of different models.
    – Mikhail T.
    Nov 29, 2015 at 23:18
  • @MikhailT. - While using different manufacturers is good advice, it is terrible advice to say that RaidZ2 is silly; a rebuild is very likely to kill another drive.
    – antiduh
    Aug 10, 2017 at 19:13
  • rebuild is very likely to kill another drive - only if all drives are identical to the already failed one and thus likely to suffer a correlated failure. And in this case, having two drives will not help you either -- they are likely to all fail at once. If the expected drive-failures are independent, then sacrificing multiple disks to redundancy buys no perceptible improvement in reliability.
    – Mikhail T.
    Aug 10, 2017 at 19:17
0

Just a point to make some light:

  • If one disk can fail without prior knowing when will it fail... why two can not fail at the same time? or three or four or five, etc.?

But, think on this other situation:

  1. One disk has failed
  2. You put a new one and let it 're-build' to be safe
  3. While such 're-build' is been done (maybe some hours) another disk fails

You lost all!!! That is the main reason to tolerate a simultaneous two disk fail.

But if you are really paranoid (like me) you will use only Stripping, no other RAID level.

Am i mad? No, of course... i do not mind one disk fail and loose all data! I will really not loose anything just because i am really paranoid.

My paranoid scheme:

  • ZFS with stripping (2x, 3x, 4x ... or 10x disks), just to fit the max bandwidth of the bus (sata III, usb 3.1 gen 2, etc)
  • And other similar five ZFS (without power, so can not get damaged)

How to use all of such ZFS:

  • Have only two of them at the same time (just to sync changes)... like a master with five slaves

Paranoid point of view:

  • If one ZFS fails... like been hit by a storm ligth, burned, stolen, etc... i still have five copies more inside different places (bank, bunker, internet cloud, friends house, vacation house, etc)

To sync some of them... i must carry with me one ZFS and do the sync on the other physical place / country / etc... so i use one of thoose slaves.

Think when i talk about HDDs on one ZFS, on another ZFS can be 3.1 Gen2 64GiB USB sticks! No need to be identical size, type, etc.

Really paranoid point of view.

For thoose not so paranoid, and if they can support to loose all... there is RaidZ levels... i prefer to 'mirror' by my self.

Another advantage of do manual 'syncs' is your own fails:

  • You have a hypotetic perfect never ever fail ZFS system, but you decide to delete a file... some days after you discovered you missed the file you wanted to delete and delete one that you did not want to (by human error)... you are out of lucky unless you have a BackUp

So... again... if you need a BackUp, why not speed up the ZFS at max by only striping?... you will still be needing to do BackUp after all.

Well, the answer can be:

  • System partition for a24h online system

For data partitions i do not see the need for RaidZ levels versus having a external USB ZFS clone ready to be connected as soon as main ZFS in use fails... just unmount ZFS main point and mount the other good ZFS in place... then with another machine re-create the off-line ZFS that got damaged.

If you have two computers, you have what you need to 7x24h with just only a few seconds (boot time) of down-time.

For home users that want their photos stay safe... use more than two ZFS and have at least one or two of them never ever connected at the same time as the rest. Paranoid? Not at all.

It is just i learn the hard way how things can go really wrong!

Just to clarify... what happen...

  1. I was using the PC normally
  2. Data was getting to fail
  3. I took out the HDDs and place the ones that was OK
  4. Data on them also got damaged in less than ten seconds
  5. I lost all my data (not fully since i was a little paranoid and have BackUp on DVD media)

The problem was really in the the controller (integrated on the mainboard), not the disks... it was altering data on read/write operations.

Since then... my data is not on internal sata disks anymore... only on external USB 3.1 Gen 2 enclosures i can plug and un-plug (each disk in its own enclosure)... and when one fails... i test all things prior to connect to that PC anything with real data... RAM, processor, USB port, etc... with a dummy USB enclosure to test patterns... if that is OK... then i test the failed disk... if it works on another enclosure or not, on another PC or not, etc... to really know who is causing the fail... really paranoid, i know!

But learning from experience:

  • Never ever trust a disk fail is caused by the disk it self... test it... it could be the controller, the cables, the power suppy, etc.
  • Or worst, it could be the RAM

Talking about RAM, i had an not so old PC (BIOS based 64Bits processor) that if (think on three identical RAM Modules, call them A, B and C):

  • i put A on the PC works perfect (no matter on what of the four slots)
  • i put B on the PC works perfect (no matter on what of the four slots)
  • i put C on the PC works perfect (but only on one of the four slots, on the other tree it causes a lot of fails, BSODs, etc)
  • i put A & B it causes a lot of fails, BSODs, etc
  • i put A & C on the PC works perfect (no matter on what of the four slots)
  • i put B & C on the PC works perfect (no matter on what of the four slots)
  • i put A & B & C on the PC it causes a lot of fails, BSODs, etc

If i test such modules on another PC, all of them works really great, no problem, no defect, etc.

Things can go really mad... and if you do not test all combinations you can not know what is failing... yes really paranoid to test all combinations of three ram modules on four slots... but what i had discoverd let me blind... the failing alone module works perfect if another one is present but not if another two... yes, i also tested with another three and fail!

Oh, yes, all same brand, kind, etc... identical... only way to difference them is a sticker i placed on them for such! And yes, buyed at the same time, they came on the same manufacture locked plastic case.

Sometimes a ZFS fail can be caused by RAM... then no matter how many levels of RaidZ you have... you can loose all.

Oh, near forget... use ECC memory, else ZFS can not be trusted! And do not use any HDD with internal cache, internat RAID, etc. unless it is designed to do writes on the order sent (or do all of them in a power cut).

I only use HDDs with enought internal energy capacity to write all its cache to the plates after an abrupt power lost!

All this points / weakness let me with one conlcusion:

  • Best speed up things and do your own BackUPs
  • Have some 100% copies Off-Line and without power
  • Have something ready to recieve power and replace the failed part

That is why i have >2 PCs with >10 USB HDDs (5 ZFS copies *3 HDDs each in stripped)... i got the best of both... safe and speed! But i get the worst part... neeed to sync all of them periodically... but data safety is the most important part! also i do not want to wait a lot for such syncs (aka, striping comes in work).

And if a 3*HDDs stripe fail... i can test which one had fail of such three for some hours/days/weeks... while i have another ZFS copie On-Line and working... after be sure the other two are OK i buy a new one (maybe 30 to 45 days to arrive) and re-create such 3*HDDs ZFS and put data on it to have that fifth copy again.

Not cheap at all! But not as a main-data center price, of course... and no electricity bill (most HDDs are without power, only 3 or 6 at the same time).

Hope some one finds the idea useful (i use it daily).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy