ZFS NAS: PCI-e x1 SATA or built-in ICH10R?

Question

I am planning to install Nexenta (or NexentaStor, I've not decided yet) to build a ZFS NAS out from the following machine:

• MB: Intel DG45FC
• Disks: 4x 2TB WD caviar green
• RAM: 2x 2GB of non-ECC Kingston (I can't afford ECC)
• System disk: 2x 500GB 2.5'' WD to host a mirrored pool for the system and maybe another mirrored pool for high-priority data

Problem: The MB has only 5 SATA (4 internals + 1 eSATA) ports, whereas I need 4 + 2 SATA ports. I foresee two options:

1) Attach the system disks on the MB controller and buy a 2-port PCI-e x1 SATA card. Issue: Does anyone know the throughput of the ICH10R with Solaris ZFS in RAID5 mode?

2) Buy a 4-ports PCI-e x1 SATA card. Because I am not sure the ICH10R in AHCI mode is very fast, I am also thinking of buying a supported card with decent speed. Issue: I can't find a PCI-e x1 card that is supported and has 4 ports. This comprehensive list http://blog.zorinaq.com/?e=10 doesn't seem to have any that are compatible with my settings.

What do you recommend?

Answer 1

ICH10R should be fast enough for 4x2 TB WD green and I would recommend to connect the 4x2TB disks to the motherboard and the system disks to an add-on SATA controller.

Another issue to consider is to use data disks optimized for RAID operation instead of WD green (such as WD RAID edition).

Answer 2

You don't necessarily need to get an add-in PCIe x1 controller as your ICH10R supports port multipliers and thus isn't limited to just 1 device per sata channel. Buy something like an Addonics AD5SAPM 5x1 Port Multiplier which sits in a empty PCI bay and will let up to five devices share a single SATA channel. Performance of these devices will be capped at single channel SATA2 max speed (300MB/sec) but that's still better than what you'd get in aggregate with any PCIe x1 device (250MB/sec).

Be aware that the Caviar 2TB Caviar Green disks have 4K sectors, but lie and report 512byte sectors to the OS. At pool creation time make sure to specify ashift=12 so that ZFS can align everything to 4K boundaries otherwise your performance will be terrible.