0

A while back, a friend of mine who's a filmmaker, asked if I had any ideas on how to do fairly big storage without a huge initial cost. Performance is not a big issue, but reasonable redundancy is.

So I started thinking about using Linux software raid to build a RAID 60 of 2.5" disks over USB3. USB3 hubs are cheap, USB3 SATA controllers are cheap and 2.5" SATA disks are cheap.

Is there any reason why I couldn't add a hundred USB disks, perhaps in groups of five in md RAID6 and then striping them to get a very big RAID60?

  • You cannot connect hundreds of USB hdds a to single PC, there is a maximum, specified in the USB standard – Ramhound May 3 '17 at 22:36
  • 1
    I'd be surprised if USB Sata controllers are fast enough to be usable in this use case. – djsmiley2kStaysInside May 3 '17 at 22:36
  • @Ramhound isn't the maximum... 128? – djsmiley2kStaysInside May 3 '17 at 22:36
  • @djsmiley2k I thought it was 127 for some reason, still not hundreds, even if its 128 though. Better solutions than USB drives...this or but RAID60 has some serious problems I suggest gluster instead – Ramhound May 3 '17 at 22:41
  • 1
    I didn't write "hundreds". I wrote "a hundred". – Jo-Erlend Schinstad May 3 '17 at 23:05
3

While technically doable, I don't think its a very good idea - You will have difficulties tracking which disks are failing, the system will be cumbersome and prone to issues. 2.5" disks are not good value (per gig) compared with 3.5" disks. You will also likely have issues with stability of the RAID - both because of vibration and IO limits.

If I were tackling the problem I'd consider looking at a second hand NAS (with new drives), or Backblaze pods are not excessively priced and probably do what you want without a heavy premium.

IMPORTANT ADDENDUM

DO NOT USE RAID6 (or RAID5). These not scale well - the statistical chance of additional disks failing during a rebuild is surprisingly high with large disks (not to mention performance difficulties). Go for RAID10

Alternatively, I'd look for cheap computers with good 80+ PSU's, lots of hard drive bays and then use a distributed file system for redundancy and scaling - something like moosefs or gluster.

  • I wouldn't use big RAID5 or RAID6es. They would be small groups striped together as RAID50 or RAID60. Vibration shouldn't be a big deal, as USB would allow for some distance. Could you elaborate on the points about difficulty tracking failing disks? – Jo-Erlend Schinstad May 4 '17 at 1:58
  • Well, when a disk fails, how do you know which device number is going to map to which of your 60 disks without a lot of fluffing arround programming UDEVS each time a disk gets knocked offline ? RAID50 is a bad idea in any event, and I question if the CPU and additional risk and complexity (especially in a homebrew solution) of RAID60 outweighs the simplicity of a RAID10 solution (I'd actually do a RAID1, then use LVM to create an effective RAID10) – davidgo May 4 '17 at 2:21
  • Before going to RAID60 - you should read storagemojo.com/2010/02/27/does-raid-6-stops-working-in-2019 - It certainly does not say not to do it, but the math and assumptions about RAID6 drives are quite useful. – davidgo May 4 '17 at 2:38
  • 1
    "You will have difficulties tracking which disks are failing." Possibly. But if the enclosures support smartctl, you can get the serial number from the Linux device node, which should also be printed on the disk itself. (If it's not printed there, you can always label them, too.) If you're mounting them in some kind of rack/case, you could sort them by their serial numbers so that failed disks are easier to locate. Or keep a spreadsheet that maps drive serial numbers to their physical position. Not ideal, but feasible. – cdhowie Aug 7 '19 at 1:30
  • Were I asked this question today, I would suggest looking at something like AWS Glacier or equivalent - meets the low upfront cost and reasonable redundancy requirements better then a roll-your-own solution. (Which is not to say it is cheaper long term) – davidgo Aug 27 '19 at 21:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.