How to build a set of video streaming edge computing system EasyNVR video platform based on RTSP protocol through privatization deployment to realize real-time live broadcast of webcams?

RTSP protocol video platform

The EasyNVR video edge gateway service can connect high-definition network cameras IP Camera, NVR and other high-definition network cameras in the traditional surveillance industry with RTSP protocol output devices to EasyNVR through simple network camera channel configuration. EasyNVR can connect the audio and video data of these video sources. Pull, convert to RTMP/HLS, perform full-platform terminal H5 live broadcast (Web, Android, iOS), and EasyNVR can connect the live broadcast data of the video source to a third-party CDN network to realize Internet-level live broadcast distribution, the main construction of EasyNVR The goal is to uniformly access and manage a variety of security cameras and NVRs, and control the on-demand output of the stream, and provide a unified standard H5 stream output for all platform terminals:

EasyNVR live broadcast without plug-in

At the beginning of the design of EasyNVR, we built according to the goal of the video capability platform, connecting the hardware of various manufacturers at the bottom and various video scene applications at the top:

EasyNVR live broadcast without plug-in

EasyNVR technology implementation process

EasyOnvifClient performs device discovery and device live streaming address acquisition;
EasyStreamClient performs RTSP/RTMP/HTTP/HLS/file recording and streaming;
EasyAACEncoder transcodes security audio formats such as G.711/G.726 into AAC format;
EasyRTMP corrects the audio and video streams and pushes them to the EasyDSS streaming media server;
EasyDSS streaming media server performs RTMP/HTTP-FLV/HLS synchronous output, compatible with all terminals;
Peripheral controls the rhythm of pulling stream -> pushing stream -> forwarding according to the user's interface request;
Synchronization of each transfer node will do some snapshots, error code recording and external output;

How to build an EasyNVR streaming media edge video system

Step 1: Selection of device docking protocol

Whether it uses RTSP or Onvif protocol to fetch from the device, or the national standard GB/T28181 to fetch the stream to the device, it is based on the protocol supported by the device to obtain the audio and video data of the device callback through the protocol process, such as Haikang's NetSDK, national A/B interface of power grid video platform, etc.;

The choice of the streaming protocol depends on your own application requirements. For example, if your device is only Hikvision, then you can use Hikvision’s SDK to get the stream, but if your device is available from various manufacturers, and Not fixed, it is recommended to use RTSP, an international standard streaming method.

So, taking RTSP streaming as an example, we have many options in open source or commercial fields, such as ffmpeg and EasyRTSPClient ;

Step 2: Data processing and analysis

After fetching the audio and video stream from the first step, we need to unify the stream Demux into the ES stream for further processing. For example, most of the streams output by the Haikang SDK are PS streams. We need to parse the PS into ES audio and video data. Based on the audio and video data of ES, various operations such as snapshots, video information, transcoding, and video analysis are performed.

Step 3: Push and distribute

After the preliminary data processing in the second step, we need to push ES audio and video data to nginx-rtmp or EasyDSS streaming media server through librtmp or EasyRTMP for high-performance distribution and storage, and provide a series of external management interfaces;

Step 4: Interface processing

After completing the above 3 steps, it can only be regarded as running through the entire data process. We also need to control the entire process. For example, when a client requests to watch, we start streaming, transcoding, streaming, and distribution. In the process, when the user stops watching or the server does not keep alive within a period of time, the server stops the entire circulation process.

At the same time, we need to output the process of taking the stream and taking the video to a certain device in the form of an interface (refer to the realization of EasyNVR), so that a set of bottom layers can be provided to multiple sites and multiple projects.

Principles of service construction

Looking back at the development process of EasyDarwin->EasyDSS->EasyNVR, we basically maintain the development mode of first partial system, first component and then platform, step by step to consolidate every detail and function point, from the most basic and most compatible audio and video data Pull stream acquisition, to efficient and fully compatible data push, and then to the high-performance distribution of audio and video data by the streaming media server, and the entire platform of audio and video playback. This entire audio and video data transmission link is formed, It is enough to meet the needs of a variety of ever-changing audio and video applications.

Especially in the security field, its main characteristics are more equipment, fewer users, and high requirements for intelligence. We need to access equipment with multiple different protocols under multiple network environments, such as camera IPC, network hard disk video recorder NVR, hard disk video recorder DVR , GB/T28181 national standard equipment, individual equipment, SIP protocol equipment, etc., which will involve more data transmission and control protocols, such as: RTSP, RTP, Onvif, GB/T28181, SIP, State Grid A/B interface And so on, and the data encapsulation format of these transmission protocols will be various: ES, TS, PS, PES, what we need to do is to make all these protocols compatible, and provide a unified interface and Compatible audio and video output protocols, build a set of high cohesion, low coupling, easy expansion, and interface-friendly video capability platform, allowing application system developers to develop various value-added services based on the video capability platform Application platform!

Technical points of security video surveillance system

In the audio and video applications of traditional security combined with the Internet, it can basically be divided into the following points:

1. Protocol compatible

From the device fetching and data forwarding protocol, we need to involve:

RTSP fetches the stream;
Onvif protocol control;
GB/T28181 protocol control;
State Grid A/B interface control;
RTMP streaming;

2. Processing of stream encapsulation

After obtaining the audio and video data through the streaming protocol, we need to perform secondary processing on the packaging format and encoding format of various audio and video data, and output it uniformly:

RTP data analysis;
PS data analysis;
RTMP/FLV data package;

3. Distribution and storage of streaming data

Streaming media service is the core part of all video applications. It mainly requires stable, high-performance, full-platform distribution, and can provide a series of peripheral management services, including recording, retrieval, playback, authentication, etc.:

High-performance distribution;
RTMP/HLS/HTTP-FLV synchronous output;
Recording, retrieval and playback;

4. Audio and video codec

Video snapshot
G.711/G.726/PCM transcoding AAC;
Audio and video decoding and playback

✈ More video solution resource summary

Video Cloud Service
© EasyDSS Internet Video Cloud Service
© EasyCVR Security Video Cloud Service
© EasyGBS National Standard Video Cloud Service
© EasyRTC Online Video Conference Platform