66

Recording at 1080p and 100,000 FPS for 1 second in the ProRes 422 codec uses up 73.2 gigabytes of storage. (SOURCE)

How do ultra-high-speed cameras write to SD cards at such quick speeds, like 580.56 gigabits per second? Before writing, the footage also needs to be encoded/processed! What's the technology behind ultra-high-frame-rate cameras being able to manage huge amounts of image data? Do they have gigantic memory buffers?

  • 15
    Top of the range PCs can write to RAM at 19.2 GB/s, and that's off-the-shelf general purpose hardware. So dump it to RAM unprocessed and then take your time saving it. – RomanSt Jul 15 '18 at 22:28
  • 2
    Should this be moved to photo.SE? – dalearn Jul 16 '18 at 0:15
  • 4
    @dalearn: If anything, electronics.SE. How can cache be that fast? (about memory / cache bandwidth benchmark results on a multi-core IvyBridge) was well-received there. – Peter Cordes Jul 16 '18 at 0:27
  • 3
    @RomanSt: That's for a single stick (i.e. one channel / one memory controller). Typical desktop/laptop systems are dual-channel, high-end desktops can be tripler or quad-channel. e.g. without overclocking, the theoretical max memory bandwidth of an Intel CPU with DDR4-2400 (like a Coffee Lake i3-8100) is 37.5 GB/s. Practical throughput is somewhat lower, and it takes multiple cores to saturate DRAM bandwidth. – Peter Cordes Jul 16 '18 at 0:36
  • 8
    GPU memory bandwidth is maybe a better example of what can be done with DRAM soldered onto a board, with lots of channels, optimized for throughput for sequential access patterns. Like a GTX1080 with 320GB/s of bandwidth from 8GB of GDDR5X, or 112GB/s for a budget GTX1050. And these days there's 3D-stacked DRAM, like HBM (en.wikipedia.org/wiki/High_Bandwidth_Memory) giving up to 256 GB/s per package. – Peter Cordes Jul 16 '18 at 0:39
75

What's the technology behind high-frame-rate cameras being able to manage huge amounts of image data?

Recording at 1080p and 100,000 FPS

Not at the same time:

If you’re shooting 1,000,000 fps, you’re shooting in monochrome at 128 x 32 resolution. For full-resolution of 1280 x 800, it can shoot at 25,600 fps.

These cameras use a combination of:

  • large amounts of RAM

    "You can shoot for 7.6 seconds at full resolution with the 288GB RAM option

  • specialized SSD cards

    "A CineMag can directly record about 1Gpixel/second"

Here is just one example of how it is done:

Let’s take a look at a real high-speed camera, rather than just speaking in the abstract. This is the Phantom v2512, which can shoot at resolutions up to 1280 x 800 and frame rates of up to 1,000,000 fps with a special option. However, not both at the same time. If you’re shooting 1,000,000 fps, you’re shooting in monochrome at 128 x 32 resolution. For full-resolution of 1280 x 800, it can shoot at 25,600 fps. They claim this is the fastest 1Mpixel video camera on the market.

So it can shoot at 25 Gpixels per second. That’s not sent to an SSD, however, that goes to RAM. You can order the camera with 72GB, 144GB, or 288GB of RAM. Dynamic RAM, of course, far, far faster than any SSD. You can shoot for 7.6 seconds at full resolution with the 288GB RAM option.

For longer, slower shooting, they offer an SSD custom-designed for high speed video, the CineMag IV. A CineMag can directly record about 1Gpixel/second, so that’s 1500MB/s. Of course, given that they’re using a custom SSD and not an off-the-shelf unit, there’s no reason to expect these to necessarily be limited to the speeds of those used in standard off-the-shelf SSDs for PCs.

Source How do high speed cameras process gigabytes of data in such short speeds? - Quora


Further Reading

  • 24
    They also include bespoke, ultra high-speed ASICs to do the video compression and encoding in real time. Think not hardware-accelerated hardware-accelerated video encoding on steroids, but instead, pure literal hardware video encoding. The video "processor" probably accounts for more than $20,000 of the cost of the camera alone. – qasdfdsaq Jul 16 '18 at 1:19
  • An example of a real-world setup and usage of those SSDs, you can look at Linus Tech Tips' setup. It (probably) isn't the best setup, but is an example. – Ismael Miguel Jul 16 '18 at 11:49
  • 1
    Just not sure in what world 1500mb/s is fast write speed for a high end ssd. PM1725a, Samsung, does WAY more than that and that is a "standard" datacenter SSD - I have a couple of them in my servers. – TomTom Jul 16 '18 at 17:57
  • 1
    The quote says 10,000 FPS but the original says 100,000 FPS (too small for an edit). – Zackary Jul 16 '18 at 18:30
4

That's not what happens at all.

As can be clearly seen in the video, the recording at 120k fps is done with 512*320 resolution, which is less than 8% of the bandwidth for 1080p. At 170k fps, the resolution is only 256*256, and the 340k recording is done at again half that resolution, which is only 3% of the bandwidth you're talking about.
The final video on youtube is 1080p, sure, but the source videos (at least the high-speed ones) are not, they're upscaled after the fact.

So, while it's still quite some, it's not nearly what you think it is.

The smallest model of these cameras has 72GB of RAM, and they go up to 288GB of RAM. Assuming the lowest of the resolutions they've used in the video, and the largest model of the camera, the camera would be able to store about 2 million full-chroma frames, or about 5.8 seconds of video in RAM just like that, no need to ever write something.
But it doesn't do that, it stores frames which are about 1/10 that size. So really, it can record almost one minute without ever writing out something. Add to that a fast storage medium (of which even the cheaper ones are capable of taking 100+MB/s nowadays, and top notch is more like 1GB/s), and there's little surprise how it could work.
Note that thanks to parallel writes, even with a slow-ish medium which can't quite keep up with sustaining the writes indefinitely, you can greatly increase the total recording capacity so it's virtually unlimited (you don't do 2-hour high-speed recordings anyway).

As for processing/compression, ProRes 422 is not comparable to typical video compression. It is a dead simple chroma subsampling followed by a DCT, strictly constrained to a single frame. No movement estimation, prediction, interframe-dependency, none of that kind. So basically it's a kind of sub-sampled single-frame JPEG compression, which is well-researched for 25+ years, an embarrassingly parallel problem, and for which dedicdated, cheap hardware that has the whole transform hard-wired is readily available. Doing this fast is no technical challenge. Storing compressed data is just as fast as storing uncompressed data.

1

As mentioned, high speed RAM. If that does not cut the mustard, you can:

  • interleave high speed RAMS. Eg you attach eight or sixteen RAM chips in parallel to the sensor output, and tell them to store one after the other, then the first again, round robin fashion. The RAMs only need to be one eight or one sixteenths as fast. If they cannot handle the data changing so fast in front of their inputs, add one extra fast register as an intermediary.

    An alternative way is to strobe all eight or sixteen or however many RAMs all at the same time - but stagger the length of the wires/traces leading to them, so data that was current at different times appears at the input of each RAM.

  • Use memory that is not addressable but instead works in a FIFO/queue fashion - this does not need handling addresses at all, you can just clock data into these devices as fast as they can take, until it literally comes out the other end - if you need more space you just attach more of these devices to that other end. Once time has come to read it out, you can do so at your preferred speed.

  • fifo is implemented either through big shift registers or an incrementing address register fed into an address decoder. Shift registers are very expensive in power and silicon space (in relative terms) compared to an address decoder. – ratchet freak Jul 16 '18 at 22:51

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.