0

I have here a serious problem on my home server: I have a LVM on a RAID5 array. As memory ran short, I decided to enlarge the RAID array (and later the PV+VG+LVs). So far so good.

I stopped the system and installed the additional hard disk physically. It was detected and I closed the server box and went away (I do not reside in the place where the server is located; it's about 200 km distance). Now I logged (from far away) via SSH into the server and did the following: I created the partition label and a matching partition on the new device and added the partition with mdadm /dev/md1 --add /dev/sde1 to the md1 array as a hot spare. In a second step I wanted to enlarge the array with mdadm --grow /dev/md1 -n 4 (from 3 to 4 devices). The array started to rebuild.

After a quite short time I got the problem: The array went down. In fact I see in the output of dmesg some lines mentioning that the sync process was stopped and that the device (/dev/md1) was not responding and blocked for >120 sec. Since then every access to the array (via mounted LVs) just blocks. After some research in the net I found out that this might happen due to a too small stripe_cache_size on rebuild (actual value is 256).

The main problem is now that the PV contains a LV that is my root file system! Thus I cannot do anything as soon as the RAID blocks. I even cannot open any editor or use cat /proc/mdstat as these commands need access to the root file system where the programs reside. Now I look for a possibility to repair the system. I see only two solutions:

  • I go there personally again and restart from a start CD, ensure a large cache and let the rebuild process run till finished. Then I restart the original system that should start up as normal. The disadvantage is that I have to stay next to the machine till the rebuild is done completely.
  • I restart the system directly and avoid the resync process to start (1). Then I change the cache size and let the system do the rebuild actually. This might be possible without me having to drive 400 km for a simple restart as I might be able to tell someone up there what to do.

Now the question (1) is: Can I somehow avoid/prevent the automatic rebuild of the array?

I thought of disabling the newly added device (sde1) but that would (I think) not avoid the reshape from 3 to 4 disks with one missing disk.

Maybe you have a better alternative. In this case please let me know.

Thanks a lot!

1 Answer 1

0

For those who run in similar problems:

I ended up by booting up a rescue system and rebuilding the array "offline" (in the rescue system).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .