How would I grow a RAID10 array using mdadm?
Obviously I would need to add 2 drives at a time, or would it have to be 4 drives?
I have a feeling its just a bad idea.
Or would be just wiser to just opt for RAID5?
How would I grow a RAID10 array using mdadm?
Obviously I would need to add 2 drives at a time, or would it have to be 4 drives?
I have a feeling its just a bad idea.
Or would be just wiser to just opt for RAID5?
Last time I checked, mdadm won't let you --grow raid10. I glanced over mdadm's manpage now and it still says: Currently supported growth options including changing the active size of component devices and changing the number of active devices in RAID levels 1/4/5/6, changing the RAID level between 1, 5, and 6, changing the chunk size and layout for RAID5 and RAID5, as well as adding or removing a write-intent bitmap.
To grow RAID 10 you need mdadm in version min. 3.3 and kernel version min 3.5. You need also an even number of disks - unpaired ones can only work as a spare or, eventually, to grow into degradated mode (not tested).
Here goes the example of growing RAID 10 from 4 drives to 6 using mdadm 3.3-2ubuntu2 @ Linux 4.2.0-10-generic. Tested with ext4 data on it, filesystem was unmounted, ext4 was extended after the RAID grow without any issue.
~$ cat /proc/mdstat
md126 : active raid10 sdd1[1] sdc1[0] sdf1[3] sde1[2]
976428032 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
bitmap: 0/8 pages [0KB], 65536KB chunk
~$ sudo mdadm /dev/md126 --add /dev/sdi1 /dev/sdj1
mdadm: added /dev/sdi1
mdadm: added /dev/sdj1
~$ sudo mdadm --grow /dev/md126 --raid-devices=6
~$ cat /proc/mdstat
md126 : active raid10 sdj1[5] sdi1[4] sdd1[1] sdc1[0] sdf1[3] sde1[2]
1464642048 blocks super 1.2 512K chunks 2 near-copies [6/6] [UUUUUU]
bitmap: 0/6 pages [0KB], 131072KB chunk
I realize this is over a year old but someone might find this helpful...
You can expand a raid 10 array, but not how you are hoping. You would have to nest multiple levels of raid. This can be done with mdadm on 2 drives in raid 10, which quite nice performance depending on the layout, but you would have to make multiple 2 disk raid 10 arrays, then attach them to logical node. Then to expand add a few more, and stripe across that. If that is your use case (needing to expand a lot) then you would be wise to use a parity array, which can be grown.
These are the limitations you get with raid 10, while maintaining better read/write performance overall. And a clarification, raid 5/6 absolutely does not "In general, provide better write performance...". Raid 5/6 has their own respective pros/cons just as raid 10, but write performance is not a pro for raid 5/6.
Also, you didnt specify the size of your drives but beware of raid 5 on new large drives. Though if you are careful, you can recover from an unrecoverable read error, you risk downtime and the possibility of not being able to recover at all.
--edit to add info-- Use tools like hdparm (hdparm -i) and lshw to get the serial numbers along with the device name (/dev/sda) when you have a failure. This will ensure you remove the correct device when replacing. Up-arrow on Travis' comment as it is very correct and a nice layout, but as usual, weight the pros and cons of every solution.
Some great news from the release announcement for mdadm 3.3:
This is a major new release so don't be too surprised if there are a few issues...
Some highlights are:
...
- RAID10 arrays can be reshaped to change the number of devices, change the chunk size, or change the layout between 'near' and 'offset'. This will always change data_offset, and will fail if there is no room for data_offset to be moved.
...
According to this answer on U&L, you will need at least linux 3.5 as well.
I know it's more work and could get confusing, but you can always stripe multiple mirrors.
For example I just setup a 4 drive raid 10 array and latter want to add another 4 drive raid 10 array. Just use mdadm to create a new raid 10 array on the new drives. You could then either create another raid 0 array using the two existing raid devices. However I would use the features of lvm to create the stripe, thus keeping mdadm configs and /dev/md devices in an easy to understand state. Either method would work, and there are probably more but that's what I could do off the top of my head.
You cannot grow a RAID10-array, this is true. But you can start with two drives (at the cost of no redundancy at this point) and then later, add two more. I have used this method to switch from a RAID5 array with four disc drives to RAID10 with four disc drives.
I degraded RAID5 and removed one disc out of the array. This gave me two discs that were free to use. Then I created a RAID10-array with something along the lines of 'mdadm --create --level 10 /dev/md0 /dev/sda1 missing /dev/sdc1 missing'. This was enough to start the array.
When I had moved the data from the degraded RAID5-array to the RAID10-array, I added two of the RAID5-discs to the RAID10-array and added the extra one as a spare.
It is probably a good idea if someone who is more knowledgable than I can talk about any performance impact this may have had. But the primary reason for my post is to show that it is possible to start off with RAID10 and two drives.
Note: do read the following forum post that asks and answers whether the disk order in mdadm matters.
There was always and is a manual solution to this. Suppose you have A,B,C,D drives in raid 10. I usually use a R0(R1,R1) setting but lets suppose we have far offset of mdadm s hybrid raid 10. Remove 1 drive (A) and create new degragaded raid 10 array with new drives A,E,F force it to start and then copy your data. Afterwards wipe B,C,D and add it to degragaded array and sync.
If you had R0(R1(A,B),R1(C,D)) you could have removed two drives A and C and do the same copying
R0(R1(,B),(,D)) --> R0(R1(A,),R1(E,),R1(F,))
then adding C,B,D to degragaded R1 arrays and resyncing so that you will have
R0(R1(A,B),R1(E,C),R1(F,D))
I did not experimented on former, but I remember achieving latter. I also remember that R0(R1,R1,R1,...) works faster than hybrid R10 in fio benchmarks.
If you dont have backup it is risky. Don't do it.. However I am not sure if it is riskier than losing power when mdadm is growing the raid 10 array. You may argue that if copying is involved restoring from backup is easier however if you have many disks attached to a sas expander and raid card, restoring from backup especially over network maybe a lot slower. Since you can always dd old raid to new raid and expand the partition so that you can bypass slow read and write speed problems of many small files.