Chapter 1 Program Introduction
1.1 Industry Demand
In application scenarios such as emergency rescue and disaster relief, fire fighting, mobile law enforcement, power inspection, photovoltaic inspection, and environmental monitoring in the wild, the use of drones has greatly improved work efficiency. When encountering some major emergencies, drones can be sent to the scene of the incident for real-time picture transmission, and the pictures can be displayed live on the 3D visualization platform of the B/S structure and provided to the command center for scientific research and judgment At the same time, it can broadcast live through the Internet and provide various terminals (PC, Android, iOS) to access through HTML5 browsers without installing any playback controls. The end-to-end delay of the video is required to be within 500ms, so as to meet the needs of emergency communication and live broadcast.
1.2 Solution Introduction
As a professional developer of audio, video and streaming media technology, our company has a deep understanding of the application needs of industry users. Therefore, based on our own technology research and development for many years, we have developed a set of low-latency, multi-terminal compatible The HTML5 unified video access solution, which has been highly recognized by industry partners and end users for its excellent performance and good user experience since it was launched in the market at the end of 2020, is an important industry Technological innovation has brought real value to end users and partners.
The main features of the program are as follows:
- It is fully compatible with HTML5 technical standards and facilitates the integration of various third-party application systems. Including 3D visualization platform, GIS geographic information system, emergency communication command system, leadership cockpit system, map navigation system, digital twin system, remote unmanned driving system, robot remote control system, interactive live broadcast system, etc.
- The end-to-end delay is the lowest in the industry. The end-to-end delay is less than 500ms in the public Internet environment, and the end-to-end delay is less than 300ms in the private network or LAN environment.
- Compatible with multiple terminals. Support PC, Android mobile, and iOS direct access through browsers, and also support direct access in WeChat public accounts and WeChat applets, without installing any controls.
- Compatible with various mainstream front-end devices, including drone cameras, robot cameras, security surveillance cameras, broadcast cameras, smart phone terminals, etc.
- The solution is mature and stable, and has been widely used in many industries, and the maintenance cost of the platform is very low.
Chapter 2 Solution Implementation
2.1 Technical Architecture of the Solution
First of all, the solution is implemented based on multi-protocol video transcoding technology and low-latency live streaming publishing technology. The architecture of the whole solution is as follows:
2.2 Technology realization path
For this application, we are mainly based on three core technologies for low-latency playback and multi-terminal H5 adaptation.
1. Real-time transcoding technology
Mainly through the 4G low-latency live broadcast workstation, the HDMI high-definition signal output by the remote control terminal of the front-end drone is transcoded in real time, so that various back-end devices can be directly watched through the H5 method. The entire encoding and live stream push delay does not exceed 100ms.
2. Low-latency live broadcast release technology
Receive the program stream pushed by the transcoding server in real time through the low-latency streaming media server, and then use the low-latency streaming media publishing technology to buffer 1 to 2 frames of image data (40 to 80ms) and release it to multiple terminals with the UDP protocol;
3. Low-latency HTML5 player technology
Through the low-latency player developed by us that conforms to the HTML5 standard, hardware decoding and low-latency playback of video streams can be realized on browsers on Windows, Linux, Android, iOS and other platforms. The player only buffers 1 frame of data. Therefore, the delay at the decoding end is about 40ms, so as to ensure the real-time performance of video transmission.
2.3 Functional module composition
The solution is mainly composed of three parts:
1. 4G real-time return device for UAV images.
It can collect high-definition signals output by the wireless image transmission device at the remote control end, and after collection and encoding processing, push them to the low-latency live broadcast server through the 4G network.
2. Low-latency streaming media server platform
It is used to receive the video and audio signals pushed up by the video transcoding workstation, and then publish them in the form of streaming media.
The platform is implemented by a general-purpose hardware server + low-latency streaming media platform software, which can effectively save investment for users.
The streaming media content publishing platform can realize the following functions:
1) Support concurrent access of thousands of network cameras;
2) Support PC, smart phone, set-top box and other multi-terminal devices to play directly through the browser;
3) A single server supports more than 2000 Users watch concurrently;
4) Support multi-server load balancing technology and server disaster recovery backup;
5) Support synchronous recording function of live programs, and automatically generate on-demand content after recording;
6) Support server performance and status monitoring, including CPU And memory usage, number of real-time connections, network bandwidth usage, etc.
7) Multi-terminal automatic adaptation technology.
For the current mainstream client devices, we have developed a terminal adaptive playback function, which can automatically adapt to mainstream browsers on the PC side (Fifrfox, Chrome, Safari), HTML5 browsers on the Android side, HTML5 browsers on the iOS side, WeChat applets, etc. .
Screenshots of streaming media server software functions (Windows version):
3. After the real-time images captured by drones in the content publishing management system
are released through low-latency streaming media, the published network streams need to be displayed as web pages on the content management platform. Released and displayed in a manner so that end users can access and watch through a browser.
Content Publishing Management System
This is a key link to control the delay of live broadcast, because even if the delay of many previous links is controlled very low, once a large data cache is generated in this link, it will seriously affect the efficiency of the entire system. Therefore, in this terminal playback link, we developed a self-implemented video player to effectively control the decoding delay of the terminal.
Here, the core technologies we use include:
1) Ultra-low data buffer technology
After the player receives the data stream, it only buffers a few key frames before sending the audio and video data to the decoder;
2) Real-time decoding technology
decoder After receiving the audio and video data, start the decoding function immediately to decode the audio and video;
3) Multi-terminal adaptation technology
For the current mainstream client devices, we have developed a terminal adaptive playback plug-in, which is compatible with mainstream browsers on the PC side (Fifrfox , Chrome, Safari), HTML5 browsers on Android, HTML5 browsers on iOS, and WeChat applets.
Chapter 3 Platform Installation and Deployment
The platform is based on a modular design concept, which allows users to install and use it easily while ensuring the overall stability of the platform.
3.1 Construction of Video Transcoding Workstation
According to different application scenarios, video transcoding workstations are mainly divided into two types:
the first type, the basic version (only supports protocol format conversion); the
second type, the professional version (supports both protocol conversion and video encoding format conversion)
transcoding The specific operation steps of the server are as follows:
Step 1: Prepare the IP monitoring signal source.
The system supports IP cameras, NVR and other network monitoring equipment;
Step 2: Connect the equipment and connect the monitoring signal source and the video
transcoding workstation to the same local area network;
Step: Configure the Transcoding Workstation
- Log in to the transcoding server through a browser, as shown in the following figure:
- Enter "Channel Settings" and configure the transcoding information of each channel, as shown in the figure below:
According to the actual situation of the project, configure the front-end monitoring signal source to each channel of the transcoding workstation as shown in the figure above.
Video source address : Enter the program streaming address of the IP camera or NVR;
RTMP streaming address : Obtain the program streaming address of the streaming media server through the video surveillance management platform;
after the parameters are set, click the "Settings" button at the bottom of the page. Can be configured.
3.2 Streaming server construction
Deploy the streaming media server in the Internet IDC computer room or the central computer room of the unit LAN to ensure the interconnection between the server and the front-end network monitoring equipment.
The server can be a server of a general brand, such as DELL, HP, IBM, Lenovo, Inspur, etc.,
or you can rent a network server or cloud server, such as renting a cloud server from Amazon, Alibaba Cloud, or Tencent Cloud.
Streaming media distribution server configuration requirements:
CPU: Intel EXON E5-2650
Memory: >32GB
Hard disk: >1TB
Operating system: CentOS 7.2 x64 or above or Windows Server 2016 x64 or above Network
card: Gigabit Ethernet
Application software: Shunjing Technology Windows version of the streaming media service system
【Remarks】:
There is also a Linux version of the streaming media server software, without a graphical control interface.
[Concurrent hosting plan of streaming media server]
Usually, streaming media server is designed to support 200 signal sources/1000 concurrent users to watch.
The main reasons are as follows:
First, the resolution of the main stream of HD cameras is usually 1920x1080, and the average bit rate is set at about 4Mb/s when the H.264 encoding format is adopted. The total bandwidth required by 200 HD cameras is about 800Mb/s.
Considering that the backbone bandwidth of most networks is Gigabit, and the bandwidth utilization rate is usually about 85%, the actual available bandwidth of Gigabit networks is about 850Mb/s, so servers connected to Gigabit backbone networks can stably carry 200 HD camera signal input.
[Multi-server load balancing]
When more front-end signal sources need to be connected, multiple streaming media servers need to be deployed to carry them. At this time, multi-server load balancing technology is required to achieve.
The schematic diagram of multi-server load balancing is as follows:
[Streaming Media Service System Deployment]:
First download and install Shunjing Technology Streaming Media Server System, as shown in the figure below: Step
1 , click "online registration software" in the lower right corner On the web page, enter the necessary registration information and click "Start Registration"
The third step is to contact technical support to activate the test authorization, and the contact information is as shown in the figure above.
The fourth step , after the license is obtained successfully, refresh the license certificate
The fifth step is to configure the relevant parameters of the server, among which the following information needs to be paid attention to:
- The default TCP port numbers of each protocol should not be occupied by other applications, mainly including:
HTTP service port: 8080
RTMP service port: 1935, 8935
VOD service port: 8088 - Ensure that the save path of the HLS fragment file exists and is accessible
D:\iAVCastMedias\hls - Make sure that the DASH segment file save path exists and is accessible
D:\iAVCastMedias\dash
Step 6 : Start the service
3.3 WEB publishing of video programs
After the streaming media platform is built, it is necessary to publish the monitoring content through the website in the form of HTML5, so that the client can browse and watch through a variety of terminals. This function mainly depends on the "digital new media resource management platform". As shown in the figure below:
Step 1 , log in to the management background, open "Network Live Management→Manage Live Channel" in the left menu, as shown in the figure below; step 2, click
" Add Live Channel", and click "Play Address" at the bottom Enter the 4 program addresses obtained in Chapter 3.4, and then submit.
Chapter 4 Implementation Effect of the Platform
4.1 Live broadcast delay index
In the private network environment, the end-to-end live broadcast delay is usually within 300ms;
in the Internet environment, the end-to-end live broadcast delay is usually around 500ms;
4.2 Supported Terminal Types
Existing solutions can support the following device terminals:
4.3 Concurrency performance indicators
After actual testing, the concurrent performance indicators of our low-latency live server software system are as follows:
Server hardware configuration environment:
CPU: Intel E5-2650
Memory: 16GB
hard disk: 120GB solid-state disk
Network card: Intel Gigabit network card x 4-port
server operating system :
CentOS x64 7.6
Live stream: 2Mb/s
Image resolution: 1280x720
Video encoding format: H.264
Concurrent performance index: 2000 concurrent live reception
Peak CPU usage: 42%
Average CPU usage: 35%
Average memory usage: 56%
4.4 Multi-terminal playback effect
The following are the playback effects on various terminals:
PC single screen playback page
On the Jiugongge play page, 
play on the Android phone browser, play
on WeChat on the mobile phone, play on
the iPhone browser, and play on
the PC side:
on the mobile phone side:
Chapter 5 Platform Equipment List
According to the project situation, the system integrator can refer to the following equipment specifications for solution configuration.
Here is a list of equipment needed:
Remarks:
1 video source is 1 channel.
Chapter 6 Solution Value
Adopting this set of industry-leading solutions will bring the following values to system integrators and users:
For users:
1. Greatly improve the user application experience, and achieve the same low-latency playback effect as the manufacturer's player plug-in without installing any browser plug-ins; 2
. Support PC, Android, and iOS multiple platforms without plug-in playback;
3. Support various brands and models of UAV equipment;
4. Support privatization deployment, convenient for users to operate independently;
5. Support localized operating environment to ensure information and data security;
for system integrators:
1. Bring a unique solution for system integrators, and can achieve control of bids through technical capabilities, and firmly control the initiative of the project;
2. Old projects can be upgraded and transformed to create more project opportunities for integrators.
3. Due to the uniqueness of the technical solution, the project profit of the integrator can be guaranteed to the maximum extent.
4. Product OEM services can be provided to protect the interests of integrators.
Chapter 7 Contact Information
7.1 Cooperation and Exchange
