2

I'm trying to expand my ZFS raid setup and I have used up all the available Sata-ports on my motherboard. I have read about SAS Expanders online and I think i have understood what they do. But one detail seems to be skipped everywhere I looked. I know that SAS Expanders are supposed to be connected (through sff-8087) to a dedicated RAID card. The controller I have found has a 8x pcie slot. Why does a SAS expander that is supposed to transfer the data through the RAID Controller need such a big bus? wouldn't a 1x pcie slot be enough for transfering configuration (or whatever) to the SAS-expander? Or does this imply that I can transfer the actual data though the PCIe bus and skipp the RAID card if I don't want hardware RAID and just want to attatch disks to be passed through to the os (as JBOD)?

SAS-Expander: "HP 24-Bay 6G SAS Expander Server Card - 8x SFF-8087, 1x SFF-8088, PCI-E - 487738-001 / 468406-B21"

  • 1
    pcie x1 isn't enough because you could connect a 100+ hard drives, and even with a small array of SSD total bandwidth can saturated. 2 SSD is 1gb/s read. ALL data has to go through the pcie bus not just the configuration data. – cybernard Nov 20 '17 at 0:00
  • 1
    It may be that long just to provide enough “grip”. The connected cables may exert quite some force on the PCIe connector, it’s like a lever. – Daniel B Feb 11 '18 at 15:57
  • @cybernard expanders typically use the PCIe connector purely for power and physical attachment; data is not usually (ever?) sent through through the PCIe slot. The data is provided by the 4 lane 6gbs External SAS connector (24Gbs aggregate bandwidth). The HBA connecting to it would typically be x4 or x8 PCIe. – Damon Feb 11 '18 at 17:09
0

I know that SAS Expanders are supposed to be connected (through sff-8087) to a dedicated RAID card.

Expander must be connected to a RAID controller or a HBA (HBA can work in IT passthru or RAID mode, both will work).

The controller i have found has a 8x pcie slot.

Long story short: Integrated Intel chipset on motherboard (e.g. 8 sata connectors) drops speed proportionally to number of drives connected to chipset. 8 drives have pathetic benchmark when tested simultaneously. That is NOT the case with HBA/RAID. 20 drives will still have e.g. 400MB read and write thruput. Now you understand why 8x.

why does a SAS expander that is supposed to transfer the Data through the Raid Controller need such a big bus?

Expander chip consumes a lots of power, you can see it by temperature it generates, 12G expander heats to +90 celsius. Data does not travel thru the PCIe bus between Controller/HBA and Expander, it uses one/two uplink cables usually port 0 and 1 on expander. (If wrong ports are used, you will lose hierarchy, but will still work, read the manual of expnder for correct ports).

or does this imply that i can transfer the actual Data though the PCIe bus and skipp the RAID card if i dont want hardware RAID and just want to attatch disks to be passed through to the os (as JBOD)?

Expander won't do a thing if it's not connected to either HBA or RAID. Try to see Expander as some kind of multiplexer not as a controller itself.

SAS-Expander: "HP 24-Bay 6G SAS Expander Server Card - 8x SFF-8087, 1x SFF-8088, PCI-E - 487738-001 / 468406-B21"

It's highly compatible expander and will work with just anything you connect to it. Unlike its' sibling 12G expander which is damn picky (had a nightmare with 6 different models of IBM controllers/HBAs I had).

Have I answered all the questions you posted in such small article?

2

SAS expander requires an HBA (Host Bus Adapter).

You can find SAS expanders that do not require a PCI-e interface, and those that do, simply use the PCI-e interface for power and a place to mount the hardware. all data communication is via the SAS cable from the HBA to the expander.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.