Expert advice needed,technology for network to disk recording, 1+ Gbps

Question

Is there any other internal storage technology than RAM that can achieve 1Gigabit or more write performance coupled with concurrent reads?

The use case is the following:
Many (100's) threads capturing IP input and writing to a filesystem , combined IP ingest being in the 1Gbps+ range.

This will be "continuous" write,so i doubt SSD could fit the requirement.

My analysis is that RAM is the only internal device that can cope with this requirement , however i'm not an expert in storage area.

Moreover,Is there any filesystem issues to be aware of ?

Answer 1

PCI-Express based ssds could do the trick.

Short of that, you could try adding ssds to a pool to get enough throughput.

Answer 2

Depending on how much space you need, if your motherboard can support more RAM in it (or an upgrade) you may want to look into maxing it out and then creating a RAM drive to store the active file(s). then just make sure it periodically copies to a HDD.

Answer 3

Most SAN's will easily fill a 1Gb/s connection. Many Direct Attach SCSI devices can do this as well, with a good quality raid card and multiple drives. You can do it, and its not hard, but its not going to be "cheap"

Actually, come to think of it.. Thats really only 125MB/s 8 SATA drives in a raid array should be able to do that.. Probably less, it would depend on the sustained write speed.

Answer 4

1 Gigabit/s isn't really all that fast. A single HD can almost handle that for sequential transfers, and a 4-disk RAID-10 or a mid-range SSD would definitely work. If you're talking about 1 Gigabyte/s, then that's a whole different story.

Unfortunately for you, you've got a lot more problems to address and questions to answer before you can arrive at a usable solution.

• How much data are you capturing?
• How are you storing the data?
• What are you doing with it?
• How long do you have to keep it?
• How is it being written to disc?
• What's your budget?

Just saying "throw a RamSan at it" doesn't do you much good if it's too small or too expensive.

Other things that floated through my head: If "IP data" is packet capture, and you're talking about 1 GB/s, how are you getting it all into a single system to capture? If you're multiplexing data from several systems, are you using some sort of shared storage? If not, what do you really need?