I have a software raid5 setup existing of 4 disks.
sda, sdb, sdc, sdd
Since everything was up and running I wanted to test the array. To do this I pulled the plug on one of the disks (sdd).
Everything went fine so I tried re-adding the disk with
mdadm --add /dev/md0 /dev/sdd, mdadm was rebuilding the array so after a couple of hours I checked and something went wrong, sdc failed.
Long story short: sdc has some bad sectors, when mdadm reads from these sectors it fails.
In /var/log/messages I can see:
read error not correctable (sector 753682864 on sdc).
I can not --assemble the array as it sees sdc and sdd as faulty.
I can however
mdadm /dev/md0 --create --assume-clean -l5 -n4 /dev/sd[abc] missing. After this I tried to add sdd but without success.
When mdadm fails and it reports the sector in /var/log/messages I can verify this by executing
hdparm --read-sector [sector] /dev/sdc.
When I overwrite the sector using
hdparm --write-sector it gets replaced and then it functions again. But this is a very time consuming process.
Are there any ways how I can recover from this mess? Would mirroring sdc to sdd (with dd) and then adding sdd as the missing drive help?