I have a weird problem with my NAS, a Zyxel 540 with 4 × 2TB drives in RAID 5 configuration. Everything was working smoothly until I decided to replace one drive with a new WD Red, same size. As everyone expected, NAS would find out a new disk has inserted and starts to rebuild the RAID, meanwhile data are still safe... I already did this operation and worked, so... no problem!
In my dreams...
After I replaced the drive, NAS said the volume was inaccessible.
I panicked, so I mounted back the old drive...nothing happened, still problems...but data were accessible in the NAS manager, but not via LAN, and impossible to copy if not via terminal.
I tried (just to be sure) a partial recover on one drive with PhotoRec. The data are still there, so the problem must be on headers or whatever.
I tried some commands by ssh to check status of the RAID, like:
mdadm --create etc mdadm --examine /dev/md2
and moreover, find out drives order is gone, so I started all combinations like:
mdadm --create --assume-clean --level=5 --raid-devices=4 --metadata=1.2 --chunk=64K --layout=left-symmetric /dev/md2 /dev/sdd3 /dev/sdb3 /dev/sdc3 /dev/sda3
I expected to make NAS work again with the correct combination, until I submitted this combination: C / A / D / B
Now I can't change combination, I'm stuck with this message:
mdadm --stop /dev/md2 it responds with this:
mdadm: Cannot get exclusive access to /dev/md2:Perhaps a running process, mounted filesystem or active volume group?
I also tried
cat /proc/mounts, volume is not mounted :(
lsof | grep /dev/md2, nothing appears
# e2fsck -n /dev/md2 e2fsck 1.42.12 (29-Aug-2014) Warning! /dev/md2 is in use. ext2fs_open2: Bad magic number in super-block e2fsck: Superblock invalid, trying backup blocks... e2fsck: Bad magic number in super-block while trying to open /dev/md2 The superblock could not be read or does not describe a valid ext2/ext3/ext4 filesystem. If the device is valid and it really contains an ext2/ext3/ext4 filesystem (and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate superblock: e2fsck -b 8193 <device> or e2fsck -b 32768 <device>
PV VG Fmt Attr PSize PFree /dev/md2 vg_ca74d470 lvm2 a-- 5.45t 0
VG #PV #LV #SN Attr VSize VFree vg_ca74d470 1 2 0 wz--n- 5.45t 0
LV VG Attr LSize [snip] lv_be37a89a vg_ca74d470 -wi-a----- 5.45t vg_info_area vg_ca74d470 -wi-a----- 100.00m
Software RAID configuration:
# cat /proc/mdstat Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] md2 : active raid5 sda3 sdd3 sdb3 5848150464 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/3] [_UUU] md1 : active raid1 sdb2 sdd2 sdc2 sda2 1998784 blocks super 1.2 [4/4] [UUUU] md0 : active raid1 sdb1 sdd1 sdc1 sda1 1997760 blocks super 1.2 [4/4] [UUUU] unused devices: <none>
I'm out of options, guys... What should I do?