I am trying to build a solution which streams the visual content of a windows application (built with Unity 2017) to browser clients. Those applications provide high quality content which needs to be streamed in a 1:1 manner in high quality with the lowest latency possible. In essence, it's a very similar use case to game streaming, because apart from streaming, a bidirectional communication channel between the client and the streamed application is established, so that the user can interact with the application running on the server.
The video is directly captured from within Unity and piped as raw video to a FFMPEG process spawned before. FFMPEG now converts the data from Unity into a suitable stream. In addition, I would like to use the NVIDIA Video Codec SDK to speed up the process.
Is H.264/H.265 a suitable format for that use case? If so, which server could be used to transmit the video stream and what can be used on the client side (e.g. BroadwayJS https://github.com/mbebenita/Broadway)? If not, what would you suggest?
Note: I already tried to implement a solution with JSMPEG (so using mpeg-1/mpeg-ts, ffmpeg, and a custom server), but this implementation has several problems like high bandwidth usage and high CPU usage. I hope that by using H.264/H.265 in combination with the NVIDIA Video Codec SDK, I can at least lower the hardware and bandwidth usage.
On the client side, Flash or Silverlight can't be used.