3

After much investigation and the purchase of PCMark7, I believe I have identified the root cause of my PC system's performance issues -- my RAID5 disk array. The PCMark7 scores are listed below. My assumption is that the poor performance results from the use of my motherboards on-board RAID controller.

My questions are:

  1. Am I correct that these performance scores are low?
  2. If yes, What solutions would improve the performance? (i.e., modifications to system)

NOTE: I am using RAID 5 partially for fun and partially to protect against disk failure. Note that I do also back up my data to an external drive, so no need to educate me on the fact that RAID is not replacement for a good backup strategy.

System Storage Score: 1308
System Storage - importing pictures 5.18 MB/s
System Storage - adding music 0.7 MB/s
System Storage - video editng  15.00 MB/s

Other Key System Specs:

  • Intel Core i7-950 Processor
  • 64-bit Windows 7 (6.1.7601)
  • 8,192 MB Memory(Corsair DDR3 @ 667 MHz)
  • 1,500 GB Volume0 Array

Full Results: http://www.3dmark.com/pcm7/584260

4 Answers 4

6

Onboard RAID5 usually performs very poorly. The Intel ICH10R (the onboard SATA controller that is usually paired with the x58 chipset) does not do the parity calculations, they get offloaded to your CPU.

If you want to continue using RAID with the onboard controller, I recommend RAID1 or RAID10. Both lack parity (RAID1 is a simple mirror, RAID10 is a striping across mirrors) and will provide much better performance.

If you want to continue to use RAID5, ditch your onboard SATA controller and get a proper RAID card. You will see much better performance (even faster than a single-disk in most cases).

5

First off, to confirm your side note, RAID (in any sort of configuration) is not a backup solution. Some RAID configurations are a redundant solution, but not backup. Redundancy means that you can sustain data in the event of a hardware failure. Backup means that you can restore data. If you accidentally delete a file from a RAID1 setup, it will be "deleted" from both drives.

To give you realistic examples of my RAID5 configuration, I have 20 Virtual Machines connected to the RAID5. The load on my server is not taxed really hard, and the IO delay is respectively not too bad considering 20 virtual machines are fighting for 4 7200 RPM drives. Under a heavier load, I have seen my IO Delay spike to 60% when all of the virtual machines are booting back up, but this would be expected. One of the servers does a nightly backup of remote computers and will usually see a 10-15% IO Delay. This is still under normal expectations. I have about 10x web servers and 2x phone servers constantly running and taxing the server. All in all, it handles very nicely.

enter image description here

4 x 7200RPM 1.5TB Hard Drives.

enter image description here

I get an average of 128MB/s with this setup with all of the Virtual Machines running (not going to shut them down for the purpose of this answer. hehe)

enter image description here

My Raid Controller is this one 3ware 9650SE-4LPML and has served nicely for many years. I do not have the BackUp Battery option for this Raid Controller, but do have the system on a UPS.

enter image description here

However, I do also have a SSD RAID 1 that I use for mission critical applications. Namely database servers and servers requiring a high IOPS. Even though my RAID5 does compete with the SSD RAID 1 in terms of throughput, the SSD configuration blows the RAID5 out of the water when you take Seek Time into account.

enter image description here

2

Raid5 is SLOW inherently when it comes to writes. Reads is where it excels, although it still lags behind other raid options such as RAID10 (o + 1). A dedicated raid card such as the Dell Perc 5/i (can be had on ebay for under $60) is a great option as this will certainly increase the performance you are seeing. Also check into RAID10, you will need 4 drives of the same size, but this will yield the greatest performance while also allowing up to 2 drive failure.

Raid10 is often used on production server systems that need I/O performance while raid5 is used where storage is more critical but performance is secondary.

EDIT: To expand, RAID10 will give you half of the drives in your array storeage-wise. So if you have 4 x 500GB drives in a raid10, you will have 1TB of usable space. In Raid5, you get n-1 drives. So the same 4 x 500GB drives in raid5 will yield 1.5TB array while still tolerating a single drive failure. THis is the reason that raid5 gets used because it has failure tolerance but still provides decent storage (a compromise). Since Raid5 must write parity to one of the disks on every write it does, this slows the process down quite a bit. Raid10 writes to the stripped disks (increased performance) and then the raid card will "copy" (mirror) the data to the redundant array. This does not take away from the performance since its mirroring after the write completed.

2
  • So, as a followup question -- i imagine to migrate to a new RAID controller it wouldn't be as simple as just plugging in the disk to the new card. I assume i'll need to create a disk image and restore it to the new array (i.e., Clonezilla?)
    – Aquadisco
    Mar 14, 2013 at 4:31
  • @Aquadisco Correct.
    – Rain
    Mar 14, 2013 at 4:41
1

Theoretical reliability RAID 0, 5, 6, 10.

RAID5/RAID6 - shows how much safer RAID6 RAID5.

RAID5/RAID10 - shows how much safer RAID10 RAID5.

p - the probability of failure HDD; (HDD failure probability in% divided by 100%)

q = 1 - p reliable HDD.

Red shows the probability of failure of RAID as a percentage.

Other colors are shown the reliability of other types of RAID to RAID5.

M3 - the first three months of the disk, the probability of failure,  3%; 
Y/2 - half a year, the probability of failure, HDD                    2%; 
1Y - the first year of HDD, the probability of failure, HDD           1,7%; 
2Y - second year of the HDD, the probability of failure HDD           8%; 
3Y - the third year of the HDD, the probability of failure, HDD       8,7%; 
4Y - the fourth year of the HDD, the probability of failure HDD       6%; 
5Y - the fifth year of the HDD, HDD failure probability of            7%; 

Information on the reliability disk of the source is taken: PDF! Failure Trends in a Large Disk Drive Population

Probability of failure Raid Arrays

Probability of failure Raid Arrays Year 4-5

probability of failure of the RAID1

M3: 0,09%; Y/2: 0,04%; 1Y: 0,0289%; 2Y: 0,64%; 3Y: 0,7569%; 4Y: 0,36%; 5Y: 0,49%

RAID1 easiest and most reliable array. Backing up can be seen as a sort of RAID1, if you do it once. The slight difference due to the different reliability.

probability of failure of the original and a copy for backup

If you want more reliability, you can create multiple backup copies on different devices. Possible percentage of data loss is reduced to:

probability of failure of the original and a (R-1)-number copy for backup

RAID0 is the easiest and unreliable array. Needed where the data is not that important at the time of such fleeting experiments. This array is used when speed is needed, and the data can be completely lost.

probability of failure of the RAID0

RAID5 long ago, when the HDD were small and expensive, and the trees were large and reliable due to its smaller size and the speed of recovery were acceptable at the time worked for this type of RAID. If you suddenly came a great idea use just such an array, look at the table.

When there will be a problem with one of the HDD, then RAID5 you will be able to lose the whole array. For example, if it is built from the HDD size 1Tb in the recovery rate will be up to 1-2 days, and the failure rate is about 10-12%, when the array is degraded and will not work. In this case it is better to restore the data to an external storage array.

In general, it was inexpensive storage websites.

To date would not advise use RAID5.

probability of failure of the RAID5

RAID6 - ode reliability. RAID5 to RAID6 slower 20-15-10%, depending on the controller. Recall that RAID5 is not very fast. As the number of drive reliability RAID6 catastrophic falls. 8-10 HDD in array to a reasonable limit.

probability of failure of the RAID6

N - are positive even numbers.

positive even numbers

RAID10 hard to find a good ground. Primarily due to the ease of recovery. Even if your controller is burned. Among other things it RAID10 reliability increases with the number of disks. For example, I saw running RAID10 c 12 disks, which of them do not work 4. Strangely enough, this situation is not unusual, since likely to remain in working order at the controller 48.5%. For two or three discs the situation is much better.

probability of failure of RAID 10 is designed in two ways, the first only with the failure of two HDD, the second - the total probability of failure. In total probability includes all possible number of failed HDD in which RAID will continue.

Failure of the two HDD in RAID 10:

probability of failure 2 HDD of the RAID10

The total probability of failure of RAID 10:

The total probability of failure of the RAID10

The bottom three tables represent a following items:

The first table contains the number of combinations in which RAID will continue. Left column - number of HDD in RAID. Top title - number of failed HDD.

Example: from 12 HDD in RAID10 out of order 3 HDD. The number of combinations in which the array will continue to work at failure of three HDD - 160.

The second table shows the number of unique combinations for a certain number of HDD in RAID for a set number HDD. Left column - number of HDD in RAID. Top title - number of HDD which counts the number of unique combinations.

Example: The number of unique combinations for 3 HDD in the array of 12 HDD - 220.

The third table shows the percentages of combinations in which RAID will continue to work and the total number of combinations. Left column - number of HDD in RAID. The top title - the probability of failure of RAID if its reliability depends only on the combination of HDD and such a test would be carried out.

Example: from 12 HDD in RAID10 out of order 3 HDD.

The probability of a good combination in which RAID array will continue to 72.7%.

These figures should be taken into account, but also keep in mind that we are dealing with the product of the probabilities. So the resulting probability will be less if this will come down three HDD.

The main purpose of these tables is to give an understanding of why there is a growth reliability RAID10 with the number of disks relatively strong fall reliability RAID6 and RAID5.

How can I check the table with a successful combination? Recall that the total number of successful combinations at failure of HDD is (3^m) -1, where m=N/2. Sum up the line with RAID 10 is like us, even if it is an array of 24 disk: 24+264+ ... + 4096 and compare with (3^12) -1.

Combination table

probability of failure of the RAID5 recovery

LSI 24 SAS port controller

Discussion about RAID risk

Joerg H. • Hot spares only help you if the array aggressively fails drives before they completely die. Then the array can copy the "failed" disk to the hot spare rather than rebuild the data from the parity. The problem with this is that it's a bit hit and miss, unless the array is VERY aggressive and even then there isn't a 100% guarantee that every failure will be detected before it's a hard failure. So, at the very least you have to plan on a rebuild from parity no matter what.

In regards to RAID-6 vs other RAID types and disk size. It really is as simple as that. When you consider the MTTDL (Mean Time to Data Loss) in a system with 20 x 1TB drives, a 50MB/sec rebuild rate, and about 15TB of actual data stored on the system you get the following RAID failure rates:

Year 1: RAID-5 - 13.76% RAID-10 - 0.078% RAID-6 - 0.516%

Year 2: RAID-5 - 25.6% RAID-10 - 0.156% RAID-6 - 1.03%

Year 3: RAID-5 - 36.86% RAID-10 - 0.23% RAID-6 - 1.54

Year 5: RAID-5 - 53.30% RAID-10 - 0.38% RAID-6 - 2.56%

So, as you can see, I stand by my previous suggestion that the rule of thumb is actually pretty good. With any drive size 1TB or greater, you want to use something other than RAID-5, period. You can't rely on hot spares to save you, so you're only options are RAID-10 or RAID-6 in your typical array. Pick your poison, or spin the chamber, it's up to you.

Oh, one more point, for a Netapp system RAID-DP numbers are almost identical to the RAID-10 numbers in terms of MTTDL. Obviously if your rebuild times are better than 50MB/sec, then your odds get better, and if it's worse, like on some SATA drives, then your odds get worse, but 50MB/sec are a pretty good average number to work with.

Finally, here are some who would argue that MTTDL isn't a good measure of reliability. Unfortunately it's the only one we have that in common usage. For more info on that topic, take a look at the paper "Mean time to meaningless: MTTDL, Markov models, and storage system reliability" by Greenan, Plank, and Wylie.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .