2

I have set up a software-raid 5 with mdadm on a 1.3 GHz AMD Neo 36L dual core machine using 3 1.5 TB Seagate Barracude Green drives (4k sectors). The chunk size of the raid is 512 KB. On top of the raid there is one lvm volume formatted with ext4 (Block size=4096, Stride=128 blocks, Stripe width=256 block). Both the raid and the lvm are aligned properly. When running bonnie++ I get the following results:

VVersion  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
               Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
                4G   362  97 69738  15 52826  16  1732  94 248671  31 539.0  32
Latency             92293us    3276ms    4274ms   22450us   38749us     111ms

Can anybody tell me if this performance is what I can expect from my setup? Especially the sequential output and the per-chr-access seems a bit too slow to me.

2 Answers 2

1

Yup. You can see from the bonnie output that its CPU bound (on what are relatively slow cores), as one would expect with software raid. The calculation of where to store data in a RAID 5 array is something better left to a dedicated XOR ASIC not a CPU. More info on that here:

http://www.dell.com/content/topics/global.aspx/power/en/ps2q03_luse?c=us&l=en

If performance is a concern you'd be way better off throwing down the cash for a 4th drive and doing RAID 10.

4
  • Thanks for your answer. I see that performance is an issue in the Per Chr-tests. But how do you explain the bad performance in the section Block of the sequential output? Output speed is 69738 K/sec at 15% cpu load while input speed is 248671 K/sec at 31% cpu load.
    – mupro
    Sep 18, 2011 at 8:53
  • Well since you are using RAID5 you are stuck at the speed of the slowest spindle. Those numbers seem normal for the disks you mentioned. If you are skeptical, take one disk out and benchmark it alone and see what you get. The results should be very similar(but probably not identical).
    – polynomial
    Sep 18, 2011 at 18:15
  • Thanks again for your help! I have just one more question concerning RAID5. On the website you mentioned in your answer it sais "Meanwhile, write performance of the RAID-5 array is nearly as fast as a single disk.". Could you explain why things are like that? Shouldn't RAID5 also lead to an increased write speed when the other Hardware (ASIC or CPU) is fast enough?
    – mupro
    Sep 18, 2011 at 21:09
  • @mupro due to the parity write. You can get more info on parity here: en.wikipedia.org/wiki/RAID
    – polynomial
    Sep 18, 2011 at 21:29
0

What block size I/O are you optimising for? RAID5 is very slow for partial writes, so you need to make sure you optimise it for the block size your application will write most of the data in. I wrote an article on optimising storage stack alignment which you may find useful. Note that LVM can throw out alignment due to odd length headers.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .