At present, video surveillance applications have been widely used in various application scenarios, and after years of accumulated construction, the equipment used by manufacturers is also varied, such as cameras of various brands such as Hikvision, Dahua, Uniview, and Tiandi Weiye. .
At the same time, with the construction of various digital application systems, many units need to develop their own mobile intelligent applications. As an important data element, security video needs to be referenced in mobile APP. The development of mobile APP is mainly based on the H5 technical route, so that a set of code can run on multiple platforms, including the mainstream Android platform and iOS platform. Therefore, there is an urgent need in the industry for a method to quickly integrate security video signals in mobile APPs in the form of H5.
Technical route analysis:
In order to realize this application, there are mainly the following technical implementation methods:
1. Use the SDK provided by the monitoring manufacturer to realize the access of the camera signal in the APP .
Various monitoring equipment manufacturers provide SDK development interfaces in this area for free, but there are the following problems:
1) The interface provided is not in the H5 mode, and the manufacturer's own dynamic library file needs to be introduced when the APP is packaged;
2) The SDK only supports the manufacturer's own equipment, and is not compatible with other monitoring manufacturers' equipment. If there are multiple manufacturers' equipment in the same system, it needs to be connected with multiple monitoring manufacturers;
2. Publish by converting the security camera signal into a video stream of the HLS protocol.
This is the most common technical implementation currently used, and its advantages and disadvantages are as follows:
advantage:
1) This solution is universal, because the HLS protocol video stream output after conversion by the streaming media server can be compatible with the two major mobile platforms of Android and iOS;
2) Compatible with security cameras and NVR equipment from various manufacturers;
3) The mobile terminal connection is fully compatible with the HTML5 technical standard, which is simple and convenient to implement;
shortcoming:
1) The delay of the converted live stream increases, and the real-time performance of the picture is poor. The end-to-end delay is usually more than 3 seconds;
This is due to a flaw in the HLS protocol itself developed by Apple, and cannot be optimized.
3. Through the low-latency communication technology developed by us that supports H5 , the camera signal can be accessed in the APP .
The system architecture diagram is as follows:
The main components of this system are as follows:
1. Low-latency video transcoding workstation
It is used to realize the unified access of the security cameras of various front-end manufacturers, and realize the unified protocol and encoding format conversion, and push it to the low-latency live broadcast publishing server in low-latency mode.
2. Low-latency live broadcast publishing server
It is used to realize the low-latency forwarding of various network streams, publish in HTML5 for various terminal devices (PC, iOS devices, Android devices), and support one-to-many high-concurrency applications.
The solution can support the following device terminals:
| PC terminal |
Android terminal |
iOS terminal |
| OS type: Windows/Linux/MacOS Browser type: Chrome/Firefox/Safari/Edge |
Browser type: Chrome/Firefox WeChat, WeChat applets |
Browser type: Safari WeChat, WeChat applets |
Low latency technical indicators:
The end-to-end delay of the system mainly occurs in the following links:
1. Video acquisition and encoding delay;
This part of the delay appears on the camera side, and the delay is in the range of 20~50ms;
2. Video access and transcoding delay;
This part of the delay occurs at the low-latency video transcoding workstation. When performing protocol conversion and video encoding format conversion, the delay is in the range of 10~30ms;
3. Delay in live release service;
This part of the delay occurs on the server side of the low-latency live broadcast publishing server. When the server receives the network stream pushed by the low-latency video transcoding workstation, it needs to cache 2~3 frames of data locally to resist network bandwidth jitter. To avoid the impact of the picture freeze.
According to different network stream formats, this part of the delay is in the range of 40~100ms;
4. Client decoding playback delay:
When the client-side HTML5 player is playing the network stream, it needs to wait for a complete frame of data to be received before it can decode and output, and it also needs to cache 1~2 frames of data based on the impact of fighting network jitter, so this part of the delay is in 20~80ms range;
End-to-end overall latency indicators:
To sum up, the delay time of the entire end-to-end system is usually in the range of 300~500ms, which is basically consistent with the browser plug-in mode of the monitoring manufacturer.
Advantages and disadvantages of this program:
advantage:
- The solution supports multi-terminal H5 integration;
- Support access to surveillance cameras of various brands;
- The end-to-end delay is extremely low, and the end-to-end delay does not exceed 300ms in a private network environment;
- It can be adapted to various playback terminals, including Windows system PC terminals, Linux system PC terminals, localized system PC terminals, Android system terminals, iOS system terminals, and WeChat applets;
shortcoming:
The construction cost is relatively high.
The actual effect of the program:
You can visit the online test system on the public network to see the actual effect intuitively.
Mobile access address:
Player interface 
http://www.shunjingtech.com/xmms/mobile.html
(It can be accessed directly in the mobile browser of WeChat or Chromium core)
PC -side access address:
Player interface http://www.shunjingtech.com/xmms/base.html
(It can be directly accessed in chromium kernel browsers such as Chrome, Firefox, Edge, etc.)