At present, live video broadcasting, especially mobile video live broadcasting, has become extremely popular. Basically, major Internet companies have their own live broadcasting products, so some basic knowledge and main technical points of live broadcasting must be understood. This sharing Let me introduce to you the whole process of live broadcast and some technical points, and implement a simple demo.
Recording of live video
For H5 video recording, you can use the powerful webRTC (Web Real-Time Communication), which is a technology that supports web browsers for real-time voice conversations or video conversations. The disadvantage is that it only supports better on PC Chrome, and the mobile terminal does not support much. ideal.
The basic process of recording video using webRTC is:
Call window.navigator.webkitGetUserMedia() to obtain the video data of the user's PC camera;
convert the obtained video stream data into window.webkitRTCPeerConnection (a video stream data format);
use websocket to transmit the video stream data to the server.
Since many methods need to be prefixed with the browser, many mobile browsers do not yet support webRTC, so the real video recording still depends on the client (iOS, Android), and the effect will be better.
How to play live video in HTML5
For video playback, you can use the HLS (HTTP Live Streaming) protocol to play live streams. Both iOS and Android support this protocol naturally. The configuration is simple, just use the video tag directly. Instant messaging chat software app development can add Wei Keyun's v: weikeyun24 consultation
What is the HLS protocol
To put it simply, it divides the whole stream into small HTTP-based files for downloading, and only downloads some each time. As mentioned earlier, a .m3u8 file is introduced for H5 to play live video. This file is based on the HLS protocol. A file that stores video stream metadata.
Each .m3u8 file corresponds to several ts files. These ts files are the real storage video data. The m3u8 file only stores the configuration information and related paths of some ts files. When the video is played, .m3u8 changes dynamically , the video tag will parse this file and find the corresponding ts file to play, so generally in order to speed up, .m3u8 is placed on the web server, and the ts file is placed on the CDN.
The .m3u8 file is actually an m3u file encoded in UTF-8. This file itself cannot be played, but a text file that stores playback information.
HLS request flow
The process is roughly as follows:
HTTP requests the url of m3u8;
the server returns an m3u8 playlist, which is updated in real time, and generally gives urls of 5 segments of data at a time; the
client parses the m3u8 playlist, and then requests the urls of each segment in sequence, Get the ts data stream.
HLS Live Delay
We know that the hls protocol divides the live stream into small pieces of video to download and play, so assuming that the list contains 5 ts files, and each TS file contains 5 seconds of video content, then the overall delay is 25 seconds. Because when you see these videos, the anchor has already recorded the videos and uploaded them, so there is a delay like this. Of course, the length of the list and the size of a single ts file can be shortened to reduce the delay. Ultimately, the length of the list can be reduced to 1, and the duration of ts is 1s, but this will increase the number of requests and increase the pressure on the server. When the network speed is slow The time will cause more buffering, so the ts time recommended by Apple is 10s, so this will greatly change the delay of 30s. So the server receives the stream, transcodes, saves, slices, and then distributes it to the client. This is the root cause of the delay.
But H5 live video has some irreplaceable advantages:
It has good dissemination, which is convenient for sharing and other operations;
it can be released dynamically, which is conducive to real-time iteration of product requirements and rapid launch;
there is no need to install the app, just open the browser directly.
Taking iOS as an example to demonstrate the collection (recording) of audio and video data
Regarding audio and video capture and recording, first clarify the following concepts:
Video encoding: The so-called video encoding refers to the way of converting a video format file into another video format file through a specific compression technology. The video recorded by the iPhone we use must be encoded, uploaded, and decoded to It is actually played in the player on the client side.
Codec standard: The most important codec standards in video stream transmission include ITU's H.261, H.263, and H.264, among which the HLS protocol supports the encoding in H.264 format.
Audio encoding: Similar to video encoding, the original audio stream is encoded according to certain standards, uploaded, decoded, and played in the player at the same time. Of course, audio also has many encoding standards, such as PCM encoding, WMA encoding, AAC encoding, etc. Here, the audio encoding method supported by our HLS protocol is AAC encoding.
Using the camera on iOS to collect audio and video data is mainly divided into the following steps:
Audio and video capture, in iOS, use AVCaptureSession and AVCaptureDevice to capture the original audio and video data stream.
H264 encoding is performed on the video, AAC encoding is performed on the audio, and there are already packaged encoding libraries in iOS to implement the encoding of audio and video.
Assemble and package the encoded audio and video data;
establish an RTMP connection and push it to the server.
About RTMP
Real Time Messaging Protocol (RTMP for short) is a live video protocol developed by Macromedia and now belongs to Adobe. Like HLS, it can be applied to live video. The difference is that RTMP based on flash cannot be played in iOS browsers, but the real-time performance is better than HLS. Therefore, this protocol is generally used to upload video streams, that is, video streams are pushed to the server.
About streaming
The so-called streaming is to send the encoded audio and video data to the video streaming server. In iOS code, RTMP is commonly used to push streaming. You can use the third-party library librtmp-iOS to push streaming. librtmp encapsulates some The core API is for users to call. For example, streaming API, etc., after configuring the server address, the transcoded video stream can be pushed to the server.
So how to build a streaming server?
User Interaction in Live Video
The user interaction in the live broadcast can be roughly divided into:
Send a gift;
post a comment or barrage.
For gift giving, DOM and CSS3 can be used to implement gift giving logic and some special gift animations on the H5 side, and the technical difficulty is not too difficult.
For barrage, it is a little more complicated, and you may need to pay attention to the following points:
Real-time barrage, you can use websocket to send and receive new barrages in real time and render them;
for browsers that do not support webscoket, you can only downgrade to long polling or front-end timers to send requests to get real-time barrages;
Animation and collision detection when bullet chatting is rendered (that is, bullet chatting does not overlap), etc.