3

I've got two internal 135GB SAS drives in a RAID 0 array making a 270GB logical volume talking to my Dell Precission T5500 motherboard via a Dell supplied SAS 6iR controller.

Writing Speed

I SaveAs an 480MB Excel 2003 file into a new filename.

Using Performance Monitor I can see that the peak Disk Write is 31MB/sec. Mmmm... looks a bit slow...

SaveAs the same file to an external 2TB 7.2krpm SATA drive (SATA III drive inside a SATA II caddy) talking to the MB through its eSATA port and I get peak Disk Write of 100MB/s making the operation considerably quicker.

That's right two SAS 15krpm RAID 0 drives write at 31MB/s and a single 7.2rpm SATA (SATA II effectively) external drive writes at 100MB/s. WTF!!!

Does this indicate:

  • a problem with my SAS drives
  • a very poor SAS 6iR controller
  • Something else?

Admitedly the RAID 0 volume only has 59GB free out of its 270GB Total, and although the disk is defraged once a week maybe the free space is all in the inner sectors, whereas the external drive is pretty much empty and maybe it's writing to its outer sectors, but still this can't account for the massive difference, or can it?

Reading Speed

The story isn't as bad here

loading up said 480MB file into Excel 2003.

Peak Disk Read of the SAS RAID 0 array is 220MB/sec.

Loading from the external SATA drive i get a peak of 96MB/sec

So assuming RAID 0 should read about twice as fast as a single drive, I am getting 220/(96*2)-1 = 14% performance improvement from SAS 15krpm drives vs SATA II 7.2krpm drives. This is disapointing to say the least.

Is there something possibly wrong in my setup?

Edit1

It is possibly note worthy that the SATA drive reads and writes at about the same speed of 100MB/sec, whereas the SAS RAID 0 array reads at 220MB/s but writes at 31MB/s. That's a write speed of 14% of the read speed...

Edit2

The problem seems to be with the SAS drives. I have swaped the 15k rpm SAS drives for two new SATA III 7.2k rpm drives, used them to build a new RAID 0 array off the same controller, dumped my Vista 64 OS onto it from an image I just made and now I am getting 220MB/s for both read and write. This is a x2.2 improvement over what I was getting with one of these SATA drives connected to the eSATA port which is what I was expecting on an internal SATA RAID 0.

I don't like downloading software from the net (security) but from the picture on the HDTune web page it looks like the drive on that test performs twice as fast on the outer sectors than on the inner sectors. Is that pretty much standard for all HDDs?

If so it is plausible that the 15k rpm SAS drives, being 78% full, are performing at 75% of their full speed (just reading the blue plot on the HDTune webpage), and this still gives 220MB/s reads, implying a 290MB/sec top speed, which would make them faster than the SATA drives and possibly the world makes sense again.

However the question remains why would I be getting a 31MB/s write speed if I can read at 220MB/s?

Edit3 : 2 April 2013 15:53

Believe it or not I still haven't received my two new matching SAS drives so I can re test (received two unmatched drives so had to send back and waiting...). When I have the new setup up and running I'll post to share the results

6
  • 2
    Write caching and writing to the outer sections could account for the differences Mar 13, 2013 at 15:07
  • Is that possible if both drives have write caching enabled? Also would appreciate if you had and could point me to any papers/web pages documenting the speed difference to expect (or how to back of envelope calculate it) when writing/reading to/from the inner and outer sectors
    – OldSchool
    Mar 13, 2013 at 16:07
  • 1
    Are the numbers you're seeing close to the manufacturers' claimed specs?
    – rob
    Mar 13, 2013 at 17:12
  • @OldSchool I was thinking of different sizes of cache memory might account for differences in write speed. rob's answer about using HDTune is a good one. As for web pages I usually use tech sites like storagereview.com and techreport.com for information. Mar 13, 2013 at 17:49
  • Are you using a very small stripe size? 30MB/s seems way too low for a setup like that. Mar 13, 2013 at 22:19

3 Answers 3

3

You don't mention your OS, but HDTune (Windows-only) can print a performance graph across all tracks. This would help answer your question about how much difference there is between writing inner vs. outer tracks.

If you really want a fair comparison, you should backup and wipe both volumes, and run your test several times. If the numbers aren't close to the manufacturers' advertised specs, it may be helpful to use a synthetic benchmark in addition to your real-world test.

You should also make sure the OS, swapfile, or some other program is not trying to access either volume during your tests.

In addition to the inner/outer tracks theory that you mentioned, here are some other factors to consider:

  • areal density
  • number of platters
  • block size (filesystem)
  • RAID stripe size
  • write caching (check in both the RAID controller and OS)
  • disk contention
  • reallocated sectors
  • size of disk cache
  • differences in caching algorithms
  • use-case (e.g., random vs. sequential writes)

Your SAS 6/iR doesn't have onboard volatile cache, so BBU is a moot point, but this might be helpful for someone else, so I'll leave it in. Generally, if your RAID controller has onboard volatile cache, the controller's write cache cannot be enabled unless you install a BBU (battery back-up) for the cache. Newer RAID controllers use flash-backed cache, eliminating the need for BBU.

5
  • Thanks Rob, is there a way to tell if I have a BBU on the RAID controller?
    – OldSchool
    Mar 13, 2013 at 18:31
  • Sorry, I should have checked this before, but it looks like the SAS 6/iR doesn't have cache on the controller, making the BBU issue moot. (And it looks like newer models use flash-backed cache, eliminating the need for BBU.) So the drives' built-in cache would be the only cache you need to enable. dell.com/Learn/us/en/555/campaigns/dell-raid-controllers
    – rob
    Mar 13, 2013 at 18:40
  • Thanks Rob, so does this mean that neither the eSATA externally connected SATA drive nor the SAS6iR internally connected SAS drives use write caching even though the software option is enabled? Also check out my new Edit in the main question. It looks like the SAS drives are the problem somehow, and mostly when writing, not reading
    – OldSchool
    Mar 13, 2013 at 18:58
  • The RAID controller's cache would be secondary to the disks' built-in cache; you should still be able to use the disks' cache but may have to enable it for each disk in from the SAS 6/iR's firmware. The size of the cache and improvements in the write caching algorithms could also help explain the performance discrepancy.
    – rob
    Mar 13, 2013 at 22:47
  • @OldSchool how does the SAS RAID perform after replacing the defective disk?
    – rob
    Mar 28, 2013 at 18:59
2

Drives with higher density have higher burst (sequential write) speeds. Your benchmark is a burst write. Try writing and reading 120 000 4kb files and you will see a complete reversal in the performance.

It boils down to identifying what kind of reads and writes a particular application or system is doing most of and putting it on a HDD or RAID that is well suited to handle that kind of use. I'd wager that two 135gb 15K disks in RAID 0 are overall much better choice for putting your OS onto, even if terabyte 7K drives have higher bursts for files in the 0.1-0.5 gb range.

Try running something like Crystal Disk Mark (BSD license) to get a sense of how your drives are performing.

4
  • Thanks, I often work with large Excel files so that's why I did that test but you are right I should benchmark the OS too. In spite of that could any of this explain the large difference in read (220MB/sec) and write (31MB/sec) performance? I think something must be wrong, but looking for confirmation from someone with HDD experience (I'm a newb at this stuff) I've done a full Dell diagnostic and one of the SAS drives fails the "Confidence" test with a "Bad Block" error. chkdsk also highlights a bad block but sais everything is ok. Could that have something to do with it? (Grasping here...)
    – OldSchool
    Mar 13, 2013 at 23:24
  • @OldSchool, if chkdsk found a bad block, that means the drive has so many bad sectors that it ran out of spare sectors and it could not remap any more. As I mentioned in my answer earlier, a disk with reallocated sectors will have degraded performance. Although this may not be the only cause of your performance woes, you need to replace the disk because it will only continue to get worse (both in terms of performance and data corruption).
    – rob
    Mar 14, 2013 at 2:44
  • @rob ok, Dell are sending a replacement and I'll give it a whirl when it gets here. I was using the terms block and sector interchangably as per en.wikipedia.org/wiki/Disk_sector. Are they not the same thing? The Dell diagnostic test reffers to an error in Block XYZ and chkdsk to reffers to bad sectors, I assume they were talking about the same thing using a different word. Am i wrong?
    – OldSchool
    Mar 15, 2013 at 14:05
  • @OldSchool you're right, I was also using the terms interchangeably. I usually refer to the physical allocation unit as a sector, but disk utilities often refer to it as a block.
    – rob
    Mar 15, 2013 at 21:35
0

Ok I have just set up two brand new matching 15k rpm 300GB SAS drives in a RAID 0 array using the exact same setup as for the 7.2 rpm 2TB SATA drives.

I now get a peak read and write of 200MB/s

Conclusions

There was clearly a problem with my previous SAS drives which could only reach 30MB/s RAID-0 write speeds, possibly related to the bad sector error I was getting on one of the drives. So if you are getting write speeds that are much slower than read speeds on a configuration similar to mine you may also have a hardware problem and may benefit from replacing your drive(s).

However, the peak preformance of these (relaively) expensive 15k rpm SAS drives is no better than that of the significantly cheaper 7.2rpm SATA drives (half the price for more than 6 times the storage capacity) in this setup. Both SAS and SATA RAID arrays exhibit around 200MB/s peak read and write speeds and both boot from power off to Vista Business 64bit in around 120 seconds (The SATA drives actually seem to boot 5 seconds faster on average in the few tests I've run).

The SAS drives are also significantly noisier (You hardly hear the SATA drives).

Is this an indictement of SAS drives? Does it indicate that they are not worth the extra cash if they are not going to have long request queues (they don't in my tests nor my day to day usage)?

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .