For the past 8 hours my
mdadm re-shape process has been stuck. The output is:
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath] md0 : active raid6 sdo sdn sdm sdl sdk sdj sdi sdh(F) sdg sdr1 sds1 sdt1 sdu1 sdp1 sdq1 sde1 sdd1 sdb1 sdc1 sdf1 17581607424 blocks super 0.91 level 6, 64k chunk, algorithm 2 [20/19] [UUUUUUUUUUUUUUUUUU_U] [====>................] reshape = 23.2% (453532992/1953511936) finish=6484322.5min speed=3K/sec
The count has been at exactly
453532992 the whole time, with the speed decreasing toward 0 (not reaching it, since it's an average speed and the speed was fine for a long time before). I can see that one of the drives has been marked as failed, which has me a little worried. I can re-build onto that drive (I know it's not a hardware failure) once the re-shape is done, but at this rate that's not happening.
Does anybody have any suggestions? Is there something I can do to get
mdadm going again?
Edit: To add to the fun, it looks like something else has gone wrong on the system. I'm getting I/O errors when I try to read anything on the root file system. The RAIDed file system is still up and running, but some processes are stuck that won't let me un-mount it. And the errors on the root file system seem to be preventing a clean reboot.
So I suppose it's possible that all of the problems are on the root device and nothing to do with the RAID, it just can't continue to re-shape because it can't read something it needs? I'm not sure. But if I can't do a clean reboot, what would happen to the re-shape if I had to hard-power-reset the box? What if the root drive is physically dead and I need to install the OS on a new drive? What would happen to the array then?