5

It has already been mentioned in other questions that Red Hat recommends against using mdadm RAID 1 on SSD.

Red Hat also warns that software RAID levels 1, 4, 5, and 6 are not recommended for use on SSDs. During the initialization stage of these RAID levels, some RAID management utilities (such as mdadm) write to all of the blocks on the storage device to ensure that checksums operate properly. This will cause the performance of the SSD to degrade quickly.

I understand the reasoning behind this. However, I suspect this was written before the arrival of mdtrim, which is designed specifically for mdadm RAID 1. Does that sidestep the issue? If my understand of TRIM is correct then I believe it does but I'm not sure, hence I'm asking.

TRIM may not be right for me though. I need this for a production system and mdtrim looks experimental at best. More importantly, I require strong encryption and research has shown that TRIM reveals a little too much by highlighting which parts of the drive are actually in use. Is there any way to avoid the performance issue and still have strong encryption? I was wondering if it was possible to do a partial TRIM, freeing up some of the blocks for performance but not so many as to give too much away.

One suggestion I saw was to only use around 80% of each disk so that after mdadm does its initial check, there are still a handful of blocks left unused. But wouldn't these blocks be the first to be used on subsequent use of the disk? They would still be used up fairly quickly and then I'd be no better off, right?

1
  • There are drives on the market which do something similar to an auto-trim when they are idle. If your drives just need to be fast some of the time then those might be a solution for you.
    – Hennes
    Oct 27, 2014 at 17:57

2 Answers 2

1

Sure, you can do partial trim with mdtrim (See --reserve option) to always leave some amount of free space untrimmed. Or you could simply create some big file with dd(1) on your encrypted FS to use up some space, which will then never be trimmed (nor used by you). I guess trimming all but ~30% of unused space will give you plenty of performance benefits, without compromising security much.

you can (as you suggest) do overprovisioning instead of trimming instead (on fresh or ATA-securely-erased SSD) creating partition wih only 80% of space, and use that instead. It won't "be used up fairly quickly and then you'd be no better off". Here is why:

Assume (for simplicity) your disk has 10000 sectors (LBAs). When you partition so only half of it is used (again for simplicity), you'll be only using LBA 0-4999, while LBA 5000-9999 will never be touched. Now, level wearing firmware in drive has two ways of knowing which sectors are unused - those your OS specified by TRIM, and those that are being REwritten to. So, if you write to LBA 100 first time, it will be used (for example on physical block 123). When you writ to LBA 100 second time, the SSD will write it to new location (for example, physical block 124) and then mark OLD version of LBA 100 (which is physical block 123) as unused (TRIMed), so at later time when SSD is idle it can do garbage collection and (if all other physical blocks in that erase block are also unused), erase whole erase block (which is much bigger than physical write block - for example 512KB vs 4KB)

So by reducing range of LBA's used to half, you've increased number of overprovisioned physical sectors the drive can use. They won't get "used up", but you need to have enough of them so fragmentation on them (partly used, and partly unused physical blocks in same erase block) will disappear before you run out of free space (otherwise, SSD firmware will need to copy used blocks to other location before erasing whole erase block, thus leading to write amplification, low performance, and shortened life of the SSD)

TRIM command is still useful as it speeds that process up (and without losing so much space) by marking sectors unused before they need to be written to again (and thus also avoiding extra write and prolonging life of SSD).

1
  • Excellent answer, thanks. I've since stopped using SSDs and I suspect I wouldn't be able to easily corroborate what you've just said in any case but you certainly sound like you know what you're talking about and the level of detail is appreciated. You've confirmed some of the things I wasn't sure about and this knowledge will hopefully serve me well in future.
    – Chewi
    Dec 5, 2014 at 21:01
1

To add to the antiquated answer, I’ve recently replaced a failed HDD on my RAID1 with an SSD, from the experimenting and research I’ve conducted, I’ve found the following:

  • Linux md passes through TRIM commands to the constituent drives since ~2.6.39, but only if all of them supports the TRIM command.
    • For my HDD+SSD RAID1, I had to fail & remove the HDD, execute TRIM, then --re-add the HDD.
  • TRIM can be done on a hot block device using fstrim, Linux md will then relay this to the SSD.
  • The initial RAID1 recovery will write to the entire SSD.
  • The mdtrim script is still experimental and has not been updated for years.
  • By using TRIM on the excessively written blocks, space can be freed up in the eyes of the SSD firmware and improve performance, reduce write amplification, and more.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .