1

I have a ProLiant DL380 Gen9 server with a HP Smart Array P440ar controller and HP MO0400JEFPA 400GB SAS SSD drives.

The server is meant to host a write intensive database so I'm particularly interested in the write speed.

We have 4 arrays: 3 raid-1 arrays with 2 drives each and a raid-10 array with 16 drives that will be used to store the data.

I've been measuring the write speed by using dd:

time sh -c "dd if=/dev/zero of=test.tmp bs=8k count=200000000 conv=fdatasync"

On raid-1 arrays I get around 600MB/s ( close to the theoretical 700MBPS of the drive ). But on the raid-10 array I only get 1.5GB/s. I understand that with a 16 drive raid-10 array I should theoretically be getting 8x write speed. And I'm getting just 3x. Should I expect more from it? Or is real performance this far away from theory? (on read speed I also get a 3x instead of the theoretical 16x).

This is the configuration of the array I get from hpssacli:

 Smart Array P440ar in Slot 0 (Embedded)
 Bus Interface: PCI
 Slot: 0
 Serial Number: xxxxxxxxxxxx
 Cache Serial Number: xxxxxxxxxxxx
 RAID 6 (ADG) Status: Enabled
 Controller Status: OK
 Hardware Revision: B
 Firmware Version: 4.52
 Rebuild Priority: High
 Expand Priority: Medium
 Surface Scan Delay: 3 secs
 Surface Scan Mode: Idle
 Parallel Surface Scan Supported: Yes
 Current Parallel Surface Scan Count: 1
 Max Parallel Surface Scan Count: 16
 Queue Depth: Automatic
 Monitor and Performance Delay: 60  min
 Elevator Sort: Enabled
 Degraded Performance Optimization: Disabled
 Inconsistency Repair Policy: Disabled
 Wait for Cache Room: Disabled
 Surface Analysis Inconsistency Notification: Disabled
 Post Prompt Timeout: 15 secs
 Cache Board Present: True
 Cache Status: OK
 Cache Ratio: 0% Read / 100% Write
 Drive Write Cache: Enabled
 Total Cache Size: 2.0 GB
 Total Cache Memory Available: 1.8 GB
 No-Battery Write Cache: Disabled
 SSD Caching RAID5 WriteBack Enabled: True
 SSD Caching Version: 2
 Cache Backup Power Source: Batteries
 Battery/Capacitor Count: 1
 Battery/Capacitor Status: OK
 SATA NCQ Supported: True
 Spare Activation Mode: Activate on physical drive failure (default)
 Controller Temperature (C): 55
 Cache Module Temperature (C): 45
 Number of Ports: 2 Internal only
 Encryption: Disabled
 Express Local Encryption: False
 Driver Name: hpsa
 Driver Version: 3.4.14
 Driver Supports HPE SSD Smart Path: True
 PCI Address (Domain:Bus:Device.Function): 0000:03:00.0
 Negotiated PCIe Data Rate: PCIe 3.0 x8 (7880 MB/s)
 Controller Mode: RAID
 Controller Mode Reboot: Not Required
 Latency Scheduler Setting: Disabled
 Current Power Mode: MaxPerformance
 Host Serial Number: xxxxxxxxxxx
 Sanitize Erase Supported: True
 Primary Boot Volume: None
 Secondary Boot Volume: None

And for the raid-10 array I get:

   Array: D
  Interface Type: Solid State SAS
  Unused Space: 0  MB (0.0%)
  Used Space: 5.8 TB (100.0%)
  Status: OK
  MultiDomain Status: OK
  Array Type: Data       Spare Type: dedicated
  HPE SSD Smart Path: disable

  Logical Drive: 4
     Size: 2.9 TB
     Fault Tolerance: 1+0
     Heads: 255
     Sectors Per Track: 32
     Cylinders: 65535
     Strip Size: 256 KB
     Full Stripe Size: 2048 KB
     Status: OK
     MultiDomain Status: OK
     Caching:  Disabled
     Unique Identifier: xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
     Disk Name: /dev/sdd
     Mount Points: /data 2.9 TB Partition Number 2
     OS Status: LOCKED
     Logical Drive Label: xxxxxxxxxxxxxxxxxxxxxxxxxxx
     Mirror Group 1:
        physicaldrive 1I:1:7 (port 1I:box 1:bay 7, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:8 (port 1I:box 1:bay 8, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:9 (port 1I:box 1:bay 9, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:10 (port 1I:box 1:bay 10, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:11 (port 1I:box 1:bay 11, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:12 (port 1I:box 1:bay 12, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:13 (port 1I:box 1:bay 13, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:14 (port 1I:box 1:bay 14, Solid State SAS, 400 GB, OK)
     Mirror Group 2:
        physicaldrive 1I:1:15 (port 1I:box 1:bay 15, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:16 (port 1I:box 1:bay 16, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:17 (port 1I:box 1:bay 17, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:18 (port 1I:box 1:bay 18, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:19 (port 1I:box 1:bay 19, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:20 (port 1I:box 1:bay 20, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:21 (port 1I:box 1:bay 21, Solid State SAS, 400 GB, OK)
        physicaldrive 1I:1:22 (port 1I:box 1:bay 22, Solid State SAS, 400 GB, OK)
     Drive Type: Data
     LD Acceleration Method: All disabled

  physicaldrive 1I:1:7
     Port: 1I
     Box: 1
     Bay: 7
     Status: OK
     Drive Type: Data Drive
     Interface Type: Solid State SAS
     Size: 400 GB
     Drive exposed to OS: False
     Native Block Size: 4096
     Firmware Revision: HPD2
     Serial Number: xxxxxxxxxx
     Model: HP      MO0400JEFPA
     Current Temperature (C): 26
     Maximum Temperature (C): 36
     Usage remaining: 99.74%
     Power On Hours: 6982
     Estimated Life Remaining based on workload to date: 111600 days
     SSD Smart Trip Wearout: False
     PHY Count: 2
     PHY Transfer Rate: 12.0Gbps, Unknown
     Drive Authentication Status: OK
     Carrier Application Version: 11
     Carrier Bootloader Version: 6
     Sanitize Erase Supported: False

(all drives are configured exactly the same way).

Is there any problem with the raid controller? Anything miss-configured? Should I be happy with a 3x write speed? Or should I look for the bottleneck somewhere else?

2 Answers 2

1

For the SSD raid performance "Linus Tech Tips" a channel on youtube made a video showing the performance of ssd's in raid-0 not sure if it would also go for you but its definetly worth checking out.

The conclusion of the video showed that the performance increased for 50% with each SSD added. e.x

  • 1 SSD had default speed.
  • 2 SSD increased that speed by 50% total.
  • 3 SSD increased the speed by 25% total.
  • 4 SSD increased the speed by 12,5% total.

If you want really high performance you are better off using a PCIe-SSD or an M.2 SSD.

M.2 SSD start at 3200MB/s Read and 1500MB/s Write (Samsung 960 EVO 250GB). But I mean really good performance is going to costs alot.

2
  • M.2 SSD speeds are only in theory. I have 2 of them and they are outperformed by an ancient OCZ 60GB.
    – Overmind
    Feb 28, 2017 at 12:53
  • Two notes: 1: Be careful just which M.2 (NVME) SSDs are used. Those Samsungs you mentioned are quite fast, but there are also NVME SSds which are barely faster than a SATA based SSD. 2: Ware of thermal throttling for some M.2 SSD's
    – Hennes
    Feb 28, 2017 at 12:53
0

I would try and change these:

Cache Ratio: 0% Read / 100% Write
SSD Caching RAID5 WriteBack Enabled: True
1
  • 3
    This is more a comment than a full answer...
    – Hennes
    Jul 16, 2017 at 16:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .