Need to find which drives are mirrored within RAID-10 array

Question

I have a home media server running on openSUSE 12.2. I'm using eight 2TB drives in a RAID-10 configuration. I deliberately bought two different types of drives: four Seagate Barracuda Green and four Western Digital Red. My goal is to configure the RAID such that each mirrored pair within the array will consist of dissimilar drives (ie, one Seagate drive and one WD drive). YaST2 Partitioner unfortunately did not give me this level of choice in the structure of the array, so now I'm trying to find out what the default RAID10 structure looks like.

I do know the following:

• sdc, sdd, sde, and sdf are all WD drives
• sdg, sdh, sdi, and sdj are all Seagate drives

I chose the default 'n2' layout when creating the RAID. My guess based upon info from these two sources is that adjacent drives are mirrored (ie, sdc==sdd, sde==sdf, etc), but I want to know for sure:

• http://en.wikipedia.org/wiki/Non-standard_RAID_levels#Linux_MD_RAID_10
• https://serverfault.com/questions/200725/mirrored-and-stripped-hd-drives-in-raid10

Here is the output of 'mdadm --detail /dev/md0':

/dev/md0:
        Version : 1.0
  Creation Time : Sat Mar 16 12:55:11 2013
     Raid Level : raid10
     Array Size : 7814045696 (7452.05 GiB 8001.58 GB)
  Used Dev Size : 1953511424 (1863.01 GiB 2000.40 GB)
   Raid Devices : 8
  Total Devices : 8
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Sat Mar 16 13:09:37 2013
          State : active, resyncing
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 2048K

  Resync Status : 1% complete

           Name : aldaris:0  (local to host aldaris)
           UUID : c6cc3943:97394500:b77d44cd:f02ed876
         Events : 149

    Number   Major   Minor   RaidDevice State
       0       8       33        0      active sync   /dev/sdc1
       1       8       49        1      active sync   /dev/sdd1
       2       8       65        2      active sync   /dev/sde1
       3       8       81        3      active sync   /dev/sdf1
       4       8       97        4      active sync   /dev/sdg1
       5       8      113        5      active sync   /dev/sdh1
       6       8      129        6      active sync   /dev/sdi1
       7       8      145        7      active sync   /dev/sdj1

And here are the contents of /proc/mdstat:

Personalities : [raid10] md0 : active raid10 sdj1[7] sdi1[6] sdh1[5] sdg1[4] sdf1[3] sde1[2] sdd1[1] sdc1[0]
      7814045696 blocks super 1.0 2048K chunks 2 near-copies [8/8] [UUUUUUUU]
      [>....................]  resync =  4.8% (375163456/7814045696) finish=1206.5min speed=102751K/sec
      bitmap: 57/59 pages [228KB], 65536KB chunk

unused devices: <none>

So my questions are:

1. How do I tell which drives are mirrors of each other?
2. Is there a way to change this, or should I just swap the wires around (since that will swap the drive letters) and then rebuild the RAID?

Thanks in advance.

---

Tangential note, for anyone wants to know my reasoning for doing this is: Drives of the same model and batch, operated under similar usage loads, uptime, and temperature have little systematic variation, and differences in time to failure between drives will be primarily driven by random variation in the manufacturing process. This increases the risk of multiple drives dying at once. By purchasing drives not just from different batches but completely different manufacturers, I am introducing systematic variation into my array, thus influencing which drives will fail at similar times.

Answer 1

Recent versions of mdadm show this right in the details of the array. Example from mdadm v3.3 - 3rd September 2013

 $ mdadm --detail /dev/md1

/dev/md1:
        Version : 1.1
  Creation Time : Tue Aug 23 11:45:41 2016
     Raid Level : raid10
     Array Size : 3864803328 (3685.76 GiB 3957.56 GB)
  Used Dev Size : 1932401664 (1842.88 GiB 1978.78 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Fri Aug 26 09:39:28 2016
          State : active
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 512K

           Name : px4-300r-THXOAP:1  (local to host px4-300r-THXOAP)
           UUID : 5ee06437:83dfdb64:808feaa2:5d57b1e6
         Events : 620

    Number   Major   Minor   RaidDevice State
       4       8       50        0      active sync set-A   /dev/sdd2
       1       8       34        1      active sync set-B   /dev/sdc2
       2       8       18        2      active sync set-A   /dev/sdb2
       3       8        2        3      active sync set-B   /dev/sda2

Note the denotation set-A or set-B. In the above case, sdd and sdb can fail together without data loss. It is possible this data is not available while the array is rebuilding though.

Answer 2

I had the same issue and after googling a while I didn't find a reliable answer. After giving it some thoughts, I figured that the mirrors have the same data and so we could compare some part of it.

NOTE: BE CAREFUL, IF YOU HAVE MORE THAN 2 DRIVES WITH THE SAME CHECKSUM YOU ARE PROBABLY COMPARING EMPTY DISKSPACE, CHOOSE ANOTHER OFFSET (skip option).

With this few commands, you can figure it out:

for disk in sda sdb sdc sdd
do
  echo -n "$disk = ";
  dd if=/dev/$disk skip=1M bs=1M count=1 2>/dev/null | md5sum;
done

This will output something like:

sda = 7c4ef0f3e0143b35e044d5d65908a3a2  -
sdb = 7c4ef0f3e0143b35e044d5d65908a3a2  -
sdc = e02f7d61ad3791bd691da5b7516928a5  -
sdd = e02f7d61ad3791bd691da5b7516928a5  -

Now we know that sda/sdb is one mirror and sdc/sdd another one. One of each must stay to avoid data loss.

The "dd" command is reading one time (count=1) one Megabyte (bs=1M) at one Megabyte offset from the disk start (skip=1M). Don't skip=0, because the begining of the disk contains different information. The data usually begins after 1MB.

Answer 3

You can always verify which are mirrors by a crude manner of comparing the data, for example:

# dd if=/dev/sda1 bs=1M skip=10 count=50 2> /dev/null | md5sum -
7c01afa434fc74aeddc8ec0546e4c332  -
# dd if=/dev/sdb1 bs=1M skip=10 count=50 2> /dev/null | md5sum -
1f0f8166857710d555cb35db4a23891a  -
# dd if=/dev/sdg1 bs=1M skip=10 count=50 2> /dev/null | md5sum -
7c01afa434fc74aeddc8ec0546e4c332  -
# dd if=/dev/sdf1 bs=1M skip=10 count=50 2> /dev/null | md5sum -
1f0f8166857710d555cb35db4a23891a  -

(if you don't get any matches, you may need to increase skip=, as you're not skipping over RAID superblocksl; and if you get same md5sum for more than 2 disks, you may need to increase count=, as you're probably reading and m5summing zeros - to prevent that you should put some data on the raid first, otherwise they might be full of zeroes)

As for the swapping wires around, you don't need to do that - mdadm should create raid with devices as specified on command line in mdadm --create, so you would just specify drives in different order on command line.

Answer 4

I think you are talking about a real raid 10 array (1+0 striped mirroring)

sdc/sdg = md0 raid1 2TB |
sdd/sdh = md1 raid1 2TB |_  md4 raid0 8TB
sde/sdi = md2 raid1 2TB |
sdf/sdj = md3 raid1 2TB |

1. Create your 4 raid1 arrays:

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/{sdc,sdg}
mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/{sdd,sdh}
mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/{sde,sdi}
mdadm --create /dev/md3 --level=1 --raid-devices=2 /dev/{sdf,sdj}

At this time you have 4 raid 1 arrays with 2TB space on each,

2. Let's assemble them!

mdadm --create /dev/md4 --run --level=0 --raid-devices=4 /dev/md{0,1,2,3}

--run options is usefull because some components are active in another array

3. Adapt your mdadm.conf file

You may need (depending of your configuration) to adapt this file to reflect changes about our new array (/dev/md4).

4. Enjoy.. Your new DIY raid10 array!

Answer 5

Run "mdadm --examine device" on each component device (i.e. /dev/sda1, /dev/sdb1, etc.). The information there should help you determine which components are each others' mirror.