For the past 8 hours my mdadm
re-shape process has been stuck. The output is:
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md0 : active raid6 sdo[11] sdn[12] sdm[13] sdl[14] sdk[15] sdj[16] sdi[17] sdh[20](F) sdg[19] sdr1[1] sds1[2] sdt1[3] sdu1[4] sdp1[10] sdq1[6] sde1[0] sdd1[7] sdb1[9] sdc1[8] sdf1[5]
17581607424 blocks super 0.91 level 6, 64k chunk, algorithm 2 [20/19] [UUUUUUUUUUUUUUUUUU_U]
[====>................] reshape = 23.2% (453532992/1953511936) finish=6484322.5min speed=3K/sec
The count has been at exactly 453532992
the whole time, with the speed decreasing toward 0 (not reaching it, since it's an average speed and the speed was fine for a long time before). I can see that one of the drives has been marked as failed, which has me a little worried. I can re-build onto that drive (I know it's not a hardware failure) once the re-shape is done, but at this rate that's not happening.
Does anybody have any suggestions? Is there something I can do to get mdadm
going again?
Edit: To add to the fun, it looks like something else has gone wrong on the system. I'm getting I/O errors when I try to read anything on the root file system. The RAIDed file system is still up and running, but some processes are stuck that won't let me un-mount it. And the errors on the root file system seem to be preventing a clean reboot.
So I suppose it's possible that all of the problems are on the root device and nothing to do with the RAID, it just can't continue to re-shape because it can't read something it needs? I'm not sure. But if I can't do a clean reboot, what would happen to the re-shape if I had to hard-power-reset the box? What if the root drive is physically dead and I need to install the OS on a new drive? What would happen to the array then?