1

I had a RAID5 array consisting of 3x5TB disks assembled using mdadm. On top of that, I created a LUKS/dmcrypt encryption layer and formatted the encrypted device with ext4. I want the disks to spin down in case of inactivity.

Everything worked fine for several months, the disks were spinning down after a minute of inactivity. Now, I added a fourth 5TB disk of the same kind to the array by sudo mdadm --add /dev/md0 /dev/sdb1, then I grew the array onto that disk (mdadm --grow /dev/md0 --raid-devices=4), and finally grew the filesystem with sudo fsck -f /dev/mapper/raid5 and sudo resize2fs /dev/mapper/raid5.

No errors occured, and the array is now 5TB larger. But the disks are not spinning down anymore. The machine runs 24/7, is not using the disks, but despite waiting a couple of days, the disks are still running all the time.

iotop shows frequent occurences of ext4lazyinit, which pops up for less then a second every few seconds. I didn't notice that before growing the filesystem. So probably it's that tasks keeping the disks awake? But how can I force ext4lazyinit to complete it's task?

1

ext4lazyinit is doing exactly what it says - it's initializing the rest of the filesystem in a lazy way. It does this to give the apperance of quickly producing a filesystem. As you have noticed, it's going to try to impact the performance of your system as little as possible, and this means it's going to take a long time to complete.

The first option is to wait it out - it'll stop eventually, and your disks should return to idle.


Another option is to unmount the filesystem, and temorarily mount it with -o init_itable=0, forcing the lazy init to be more proactive, but performance will suffer. The default value is 10, so if performance is important in the meantime, perhaps try values in between. (ref)

init_itable=n    The lazy itable init code will wait n times the number of milliseconds
                 it took to zero out the previous block group's inode table. This
                 minimizes the impact on the system performance while file system's
                 inode table is being initialized.

A third option is do disable the itable initialization - though in my opinion, this is a bad option, especially for a filesystem that is in production and presumably has important data on it (that's why you're using a RAID, right?)

You can do this with the noinit_itable option:

noinit_itable     Do not initialize any uninitialized inode table blocks in the
                  background. This feature may be used by installation CD's so that the
                  install process can complete as quickly as possible; the inode table 
                  initialization process would then be deferred until the next time the
                  file system is unmounted.

Edit: Estimations on duration.

Bear in mind that disks have a write performance of ~110-120MB/s... You're running an array, so in an ideal world will see an improvement to ~330-360MB/s (110 * (n - 1)). I've also seen RAID5 run much slower (~40MB/s over 8 disks with a RAID controller - painful).

With an estimated 110MB/s you're looking at around 12 hours for a full one-disk 5TB init.

  • As I said, the array is idle most of the time, so performance is not an issue. But data integrity is, as you already guessed. So I tried with init_itable=0, which at least causes ext4lazyinit to be active with 100% disk usage most of the time. So I'll see if it finishes it's job by tomorrow. – LukeLR Mar 27 '18 at 18:48
  • 1
    good stuff - I've updated to include an estimate for how long you could expect this to go on for. – Attie Mar 27 '18 at 19:18
  • 1
    Thanks a lot for the estimations - it's already finished! The disks just spun down and everything is quiet again! I'm so glad that helped. Now I'll remount without init_itable=0 and see if it stays that way. I guess it was much faster than your estimations because the filesystem is 60% used. Although I got much slower speeds, around 4MB/s (probably random reads/writes?) – LukeLR Mar 27 '18 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.