2

I've noticed that guides on zfs+redundancy+performance tend to emphasise mirroring as a good option, but zfs administration information tends to provide relatively little information on managing a mirrored setup compared to one based on raidz.

I'm planning my zfs pool migrating from mirrored disks under Windows. My preferred setup has always been purely mirrored and having looked into zfs I'd like to keep this, and use pure mirroring for redundancy rather than RAIDZ. Ideally each drive remains logically readable in isolation, ie for 6 disks data is effectively stored as (1+2+3 mirror) + (4+5+6 mirror) rather than ((1+2 stripe) x 3 mirror). I appreciate this wouldn't be very efficient and would involve having multiple smaller pools/datasets but having lost data before, it has advantages I value over efficiency. It's also probably much faster at reading and resilvering, and far more flexible physically.

If I want to run this kind of setup, what are the implications of mirroring within a vdev vs mirroring vdevs, and if the storage is extended in future by adding a set of 3 new disks, is it better to add them as a separate pool or a separate vdev (or 3 separate vdevs)?

Improve this question

1 Answer 1

Reset to default
1

In ZFS you always have redundancy of any kind at the vdev level, you cannot have it at the pool level.

A vdev can consist of a single disk (1 disk), mirrored disks (2+ disks), Z1 stripe (2+ disks), Z2 stripe (3+ disks) or Z3 stripe (4+ disks). You can then build your pools, where each pool is backed by 1+ vdevs, and you can have as many pools as you like. Each pool uses all backing vdevs without any redundancy (that is managed at the vdev level) and losing a single vdev (not disk) means your whole pool is lost.

Therefore, if you split up your pools, you basically have several RAID1 arrays next to each other that are completely separate - loss of one pool does not affect the others, but you get lower performance because they are all RAID1 instead of RAID10.

Expansion also has this trade-off - if you grow your single pool, it will get faster (at least for newly written data, there is no automatic rebalance in place), but your chance of failure also increases.

Aside from that there is the question if you want to have 2-way or 3-way mirrors for your vdevs - 3-way is much safer (to t he point that 4-way is almost useless), but more costly (needing more disks or having less space). 3-way is also a bit faster for reads, but write speed does not change.

So it is a trade-off between cost, performance, safety and ease of use.

Improve this answer
3
  • That's a pretty cogent response - thank you. Two questions 1) can you clarify "expansion [in the way asked in the question or in some other way, it isn't clear?] also has this tradeoff". 2) I gather from your answer that there's no way to tell zfs to keep data as two distinct pools but then present them logically as one pool for file shares? Or can that be done by including within one pool a symlink to the root of the other pool, so that to the user they look like they have a common root and one filing system?
    – Stilez
    May 20, 2016 at 12:06
  • Re 1: It was a response to your last question with the 3 new disks as either an addition to the old pool or as creation of a separate new pool. Expansion is almost the same as initial creation, consider you have 3+3 disks at first and want to add 3 later on vs. you have 3+3+3 disks at first - in both cases, your layout depends on your needs.
    – user121391
    May 20, 2016 at 12:17
  • Re 2: That is correct, if you split pools, you also have to use separate file systems and can share them only separately from each other. In my view this is the biggest downside of using split pools beside the obvious performance loss. On the plus side, different pools can have different properties, like autoreplace for example. A useful distinction is to think of the pools as your hardware (controllers, cables, disks) and of the filesystems as your software (data, directories, shares, permissions). This way the right layout comes pretty naturally.
    – user121391
    May 20, 2016 at 12:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .