1. UDP/TCP
-
If you were asked to develop a real-time interactive live broadcast system by yourself, when choosing a network transmission protocol, would you choose to use the UDP protocol or the TCP protocol?
-
What if you use TCP? In extreme network conditions, TCP will repeatedly resend information for transmission reliability
-
In the TCP protocol, in order to avoid too many retransmissions, the timeout time of the timer will increase exponentially by 2, that is, if the timeout time set for the first time is 1 second, then the second time is 2 seconds, and the timeout time for the second time is 2 seconds. Three times is 4 seconds...the seventh time is 64 seconds. If it still times out after the seventh time, disconnect the TCP connection, and for such a long delay, the real-time interactive live broadcast system is simply unacceptable
-
So you must choose the UDP protocol when doing an online live broadcast system
Two, RTP protocol
-
When transmitting audio and video data streams in the real-time interactive live broadcast system, we do not directly send the audio and video data streams to UDP for transmission, but first add an RTP header to the audio and video data, and then hand it over to UDP for transmission
-
Because the amount of video data is too large during transmission, dozens of packets may be required to transmit one frame, and when the data is transmitted to the receiving end, these dozens of packets must be assembled to restore a complete image
-
The RTP protocol is for the purpose of ensuring that the order of the data is not disordered after the docking end assembles it. Think about it, if the order is disordered during assembly, is the assembled image still the transmitted image?
-
The RTP protocol is very simple, here is a brief introduction to RTP
-
sequence number: sequence number, used to record the order of the package
-
timestamp: Timestamp, the timestamps of different fragments of the same frame are the same. Timestamps for different frames are different
-
PT: Payload Type, the payload type of the data. The PT value of the audio stream is different from the PT value of the video, through which you can know what type of data is stored in this package
-
SSRC: The source of the shared media stream, which is globally unique, and different SSRCs identify different shared sources
-
CC: the number of CSRC
-
CSRC: Shared source, generally used for audio mixing or screen mixing
-
X: RTP extension header flag, if this position is 1, it means that this RTP packet has an extension header
-
M: Indicates the MARK bit, which is used to define the boundary of the video frame
-
P: padding bit
The benefits of this article, free C++ audio and video learning materials package, technical video/code, including (audio and video development, interview questions, FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, codec, push-pull stream, srs)↓↓↓ ↓↓↓See below↓↓Click at the bottom of the article to get it for free↓↓
3. RTP case
-
If you receive a set of audio and video data below on the network
-
Suppose PT=80 is video data, PT=100 is audio data
-
According to the above rules, is it easy to assemble the data?
{V=2,P=0,X=0,CC=0,M=0,PT:100,seq:14,ts:123456789,ssrc=888}, {V=2,P=0,X=0,CC=0,M=0,PT:80,seq:14,ts:123456789,ssrc=2345}, {V=2,P=0,X=0,CC=0,M=0,PT:100,seq:15,ts:123456789,ssrc=888}, {V=2,P=0,X=0,CC=0,M=0,PT:80,seq:15,ts:123456789,ssrc=2345}, {V=2,P=0,X=0,CC=0,M=0,PT:100,seq:16,ts:123456789,ssrc=888}, {V=2,P=0,X=0,CC=0,M=0,PT:80,seq:16,ts:123456789,ssrc=2345}
4. RTCP protocol
-
When using RTP packets to transmit data, problems such as packet loss, disorder, and jitter will inevitably occur
-
For example: high packet loss rate caused by network line quality problems, packet loss problems caused by transmitted data exceeding the bandwidth load, etc.
-
Before dealing with these problems, WebRTC must first let each end know what their own network quality is, which is the role of RTCP
-
RTCP has two most important messages :
RR（Reciever Report）andSR(Sender Report) -
Through the exchange of these two messages, each end knows its own network quality.
-
The message protocol is as shown in the figure below, where the meaning of the fields:
-
V=2: refers to the version of the message.
-
P: Indicates padding bit, if this bit is 1, there will be padding bytes at the end of the RTCP message
-
RC: full name Report Count, refers to the number of message blocks received in the RTCP message
-
PT=200: Payload Type, that is to say, the value of SR is 200
-
Header: partly used to identify the type of the message, such as SR or RR
-
Sender info: partly used to indicate how many packets have been sent as the sender
-
Report block: Partially indicates when the sender is the receiver, it receives packets from each SSRC
The benefits of this article, free C++ audio and video learning materials package, technical video/code, including (audio and video development, interview questions, FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, codec, push-pull stream, srs)↓↓↓ ↓↓↓See below↓↓Click at the bottom of the article to get it for free↓↓