Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

My client has an archive of roughly 5,000 independent films, which vary in file size from 1GB to 5GB. All of these films are currently on disk, however, and the client wants a machine capable of archiving all of them and making them available to a network, for viewing.

My initial response was to utilize AWS / S3. However, this is too costly, and the client is dead-set on hosting the archive "in-house" (despite all the pains that it brings).

Can anybody recommend the appropriate type of hardware to achieve this level of storage and still meet the distribution / file access requirements my client wants?

As a bonus criteria, the client wanted to make all of the films available to the web, which added considerable complexity to the task, but if there is a hardware configuration suited to this task, I'm all ears.

The budget is up to £10,000.

share|improve this question

closed as not constructive by allquixotic, Dave M, TFM, Scott, Tanner Feb 20 '13 at 23:53

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. If this question can be reworded to fit the rules in the help center, please edit the question.

    
How many people will be accessing the films at the same time? That will set lower limits on the IO subsystem. (Read: Can you get away with cheap SATA drives despite all their disadvantages). –  Hennes Feb 20 '13 at 16:55
    
Internally, no more than 5 people. –  Harry Thomas Feb 20 '13 at 16:56

1 Answer 1

I fear this question may be closed as 'too localised' or Do my job for meTM.

However, it piqued my interest, but there is no way to answer it without much more information.

For starters:

  1. How much data needs to be stored (Indirect information 5000 × ( (1+5) / 2 ) GB).
  2. How much will the data grow in the future?
  3. How many people will access the data at the same time? (From your comment: 5 people internally, but what we really need is how much bandwidth gets used. Both on the wire and reading from disk).
  4. How much redundancy? ( 5000 films at about 3 GB is 15 TB of data. Using cheap SATA drives you might get away with a ten drive RAID 6 array with 3 TB drives ) That assumes those drives will be able to handle the IO from 5+ people at the same time. If not, consider 12 drives in RAID 10.
  5. How much is the upload to the Internet? Not the download speed which is often larger in asymmetrical setups. How many people will watch at the same time from the web site?
  6. Security.
  7. Backups.
  8. Server hardware. (Dual PSU's., UPS, ... or is some downtime fine?)

I guess I could come up with a few more questions. And this really should be in a comment but it is way too long for it.

Once you have the answers to all of those questions, come up with a working plan. Then if you get stuck with a specific part of that plan you can ask a specific, answerable question here.

share|improve this answer
    
I'm aware of how it looks, though it is genuinely a hardware related concern. I could easily scramble together a rudimentary system to do it, but wanted the community expertise. 1. Roughly 10 TB. 2. 800GB-10TB per year in singular increments. 3. I don't know how to answer this. The only usage will be file access over a network. 4. At least one layer of redundancy for failure. 5. This is where it gets way too out of control. The upload would be the same as the bulk file size: 10TB (thus thinking S3). 6, 7: Considered. 8. If it was only internal, it would only need to run from 9AM to 11PM. –  Harry Thomas Feb 20 '13 at 17:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.