After years of actual project combat and accumulation of R&D experience, I summarized how to achieve plug-in-free H5 display for H5 video visualization in video playback, especially the solution that fits the security industry;
Except for HTTP and WebSocket transmission protocols, others cannot be universally transmitted to the browser. Therefore, if you want to make a universal H5 video player, it is basically a video player with HTTP/WebSocket protocol. Video sources similar to RTMP and RTSP protocols are unavoidable and need to be converted by the server. Then we can summarize several rigid requirements for the visualized live broadcast of web pages:
- H5 playback only supports streaming media sources with HTTP and WebSocket protocols;
- Need to support H.264 and H.265 video encoding formats at the same time;
- Support low-latency real-time video and video playback video;
Feasible solution 1: WebSocket transparent transmission
Through the WebSocket proxy server, a transparent transmission channel is established to forward video streams of various protocols. WebSocket is similar to a pipeline, only forwarding data as it is, establishing a transmission channel wrapped by WebSocket between the source device and the H5 client. The specific protocol interaction process is still carried out in accordance with the original protocol, RTSP, SIP type of text protocol is in accordance with the text protocol, and RTMP type of byte stream protocol is in accordance with the byte stream protocol;
Difficulties of the scheme:
- Front-end compatibility development is difficult: the front-end has to develop various signaling protocols, byte stream protocols, as well as unpacking, buffering, time stamp synchronization, decoding, display, and playback. The requirements for front-end development are relatively high;
- H.265 playback technology: In security scenarios, most cameras use the H.265 encoding format. Therefore, if the front-end Web supports the H.265 format, the wasm technology needs to be introduced;
Advantages and disadvantages of the scheme:
- Advantages: simple back-end development and fast speed;
- Disadvantages: The back-end only works as a pipeline, and does not have the actual device information knowledge right. This is impossible for device management and control, for example, no snapshots, no video recording, and no specific error information of the device;
Feasible plan two: protocol conversion
Using a full protocol access scheme similar to EasyCVR, video sources of various protocol types (RTSP, RTMP push/pull, HTTP, UDP, etc.) and video files can be output to HTTP through standardized protocol conversion. -FLV (real-time streaming) and HLS (live streaming/on-demand streaming) output to the terminal, which achieves standardized, all-terminal, and all-platform output;
Difficulties of the scheme:
- Back-end compatibility: The back-end needs to access various protocols, de-encapsulate them, and encapsulate them into standard flv, ts and other formats, but fortunately, there has been a lot of accumulation;
- H.265 player: It also requires H.265 Web playback, including HTTP-FLV and HLS H.265 format;
Advantages and disadvantages of the scheme:
- Disadvantages: high technical threshold;
- Advantages: strong controllability, separation of front and rear ends, front and rear personnel can be flexibly customized;
H.265 web playback program
You can see that the above two solutions will have a difficulty in playing H.265 web pages. The main reason here is that the current browsers basically cannot support the underlying decoding of H.265, or hard decoding, which requires H.265. Combine the development technology of the native player and the development technology of the Web player, that is, wasm technology, encapsulate C/C++ into wasm, and be called by js, so that js can make full use of C/C++ like a C/C++ native player The computing power and expansion capability of the video can be used to realize the video decoding process.
The main advantages of wasm broadcast are in line with the actual needs of security. Although major manufacturers have begun to popularize WebRTC live broadcasts, due to the particularity of security, H.265 has taken the lead. Only by solving the front-end display of H.265 can the entire security be completely solved. The need for visualization is something that cannot be solved by RTC playback.
In summary, we have a few thoughts:
- If you want to completely control the entire video visualization process, it is best to do video processing and protocol conversion on the server side;
- The front-end support of H.265 is indispensable. The conversion of H.265 from the back-end to H.264 to support the front-end playback is a retrogression of technology;
- In the era when the visualization system is Chromium, WebRTC may be a very good choice;