I am trying to build a solution which streams the visual content of a windows application (built with Unity 2017) to browser clients. Those applications provide high quality content which needs to be streamed in a 1:1 manner in high quality with the lowest latency possible. In essence, it's a very similar use case to game streaming, because apart from streaming, a bidirectional communication channel between the client and the streamed application is established, so that the user can interact with the application running on the server.

The video is directly captured from within Unity and piped as raw video to a FFMPEG process spawned before. FFMPEG now converts the data from Unity into a suitable stream. In addition, I would like to use the NVIDIA Video Codec SDK to speed up the process.

Is H.264/H.265 a suitable format for that use case? If so, which server could be used to transmit the video stream and what can be used on the client side (e.g. BroadwayJS https://github.com/mbebenita/Broadway)? If not, what would you suggest?

Note: I already tried to implement a solution with JSMPEG (so using mpeg-1/mpeg-ts, ffmpeg, and a custom server), but this implementation has several problems like high bandwidth usage and high CPU usage. I hope that by using H.264/H.265 in combination with the NVIDIA Video Codec SDK, I can at least lower the hardware and bandwidth usage.

On the client side, Flash or Silverlight can't be used.

  • 1
    It already isn't realtime anymore. Even an NVIDIA GPU will take time reencode a signal. Did you try to just open the stream with your browser? – Seth Dec 29 '17 at 8:32
  • Ok, what format would you suggest? How can I open a stream without using a server which forwards it accordingly (note that I am a newbie regarding that topic)? – scriptify Dec 29 '17 at 8:37
  • I edited the question and tried to add more details and what exactly I'd like to achieve. – scriptify Dec 29 '17 at 9:27
  • ffserver from ffmpeg documentation and ffmpeg directly supports using your gpu. If you want high quality and low latency you will have high bandwith and resource usage. In almost all cases those things are contradictions. Either use high compression (which requires computing power) or high bandwidth (less compression). More computing requires more time so latency rises. You could try out OBS to get a feel for those variables. – Seth Dec 29 '17 at 9:39
  • Ok, so I'll have a look at ffserver. I understand, that seems pretty clear. I thought by using a more advanced format like HVEC I could reduce those numbers. Anyway, I'll now try to play around with ffserver (I thought the development of this project is discontinued, but it doesn't seem so looking on the git repo). Thank you for your explaination! – scriptify Dec 29 '17 at 9:47

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.