Protocols UDP, TCP, RTP, RTCP in WebRTC of audio and video live broadcast system

1. UDP/TCP

• If you were asked to develop a real-time interactive live broadcast system by yourself, when choosing a network transmission protocol, would you choose to use UDP protocol or TCP protocol?

• What happens if TCP is used? In extreme network conditions, TCP will repeatedly resend information in order to ensure transmission reliability.

• In the TCP protocol, in order to avoid too many retransmissions, the timeout time of the timer will increase exponentially by 2. That is to say, assuming that the timeout time set for the first time is 1 second, then the timeout time set for the second time will be 2 seconds. The third time is 4 seconds...the seventh time is 64 seconds. If it still times out after the seventh time, the TCP connection will be disconnected. For such a long delay, the real-time interactive live broadcast system is simply unacceptable.

• Therefore, when building an online live broadcast system, you must choose the UDP protocol

2. RTP protocol

• When the real-time interactive live broadcast system transmits audio and video data streams, we do not directly hand over the audio and video data streams to UDP for transmission. Instead, we first add an RTP header to the audio and video data, and then hand it over to UDP for transmission.

• Because the amount of video data is too large when it is transmitted, dozens of packets may be needed to transmit one frame. When the data is transmitted to the receiving end, these dozens of packets must be assembled to restore it to a complete image.

• The RTP protocol exists so that the order will not be messed up after the docking end assembles the data. Think about it, if the order is messed up during assembly, will the assembled image still be the transmitted image?

• The RTP protocol is very simple. Here is a brief introduction to RTP.

• sequence number: sequence number, used to record the sequence of packets

• timestamp: timestamp. The timestamps of different fragments of the same frame are the same. The timestamps of different frames are different

• PT: Payload Type, the payload type of data. The PT value of the audio stream is different from the PT value of the video. Through it, you can know what type of data this packet stores.

• SSRC: The source of shared media streams. It is globally unique. Different SSRC identifies different shared sources.

• CC: Number of CSRCs

• CSRC: Shared source, generally used for mixing or screen mixing

• X: RTP extension header mark. If this position is 1, it means that this RTP packet also has an extension header.

• M: represents the MARK bit, used to define the video frame boundary

• P: padding bit

  3. RTP case

• If you receive a set of the following audio and video data on the network

• Assume that PT=80 is video data and PT=100 is audio data

• According to the above rules, is it easy to assemble the data?

• {V=2,P=0,X=0,CC=0,M=0,PT:100,seq:14,ts:123456789,ssrc=888},
  {V=2,P=0,X=0,CC=0,M=0,PT:80,seq:14,ts:123456789,ssrc=2345},
  {V=2,P=0,X=0,CC=0,M=0,PT:100,seq:15,ts:123456789,ssrc=888},
  {V=2,P=0,X=0,CC=0,M=0,PT:80,seq:15,ts:123456789,ssrc=2345},
  {V=2,P=0,X=0,CC=0,M=0,PT:100,seq:16,ts:123456789,ssrc=888},
  {V=2,P=0,X=0,CC=0,M=0,PT:80,seq:16,ts:123456789,ssrc=2345}

  4. RTCP protocol

• When using RTP packets to transmit data, problems such as packet loss, out-of-order, and jitter will inevitably occur.

• For example: high packet loss rate caused by network line quality problems, packet loss caused by the load of transmitted data exceeding the bandwidth, etc.

• Before dealing with these problems, WebRTC must first let each end know what their own network quality is. This is the role of RTCP.

• RTCP has two most important messages : RR（Reciever Report）andSR(Sender Report)

• Through the exchange of these two messages, each end knows the quality of its own network.

• The message protocol is as shown below, with the meaning of the fields:

• V=2: refers to the version of the message.

• P: indicates the padding bit. If this bit is 1, there will be padding bytes at the end of the RTCP message.

• RC: The full name is Report Count, which refers to the number of message blocks received in the RTCP message.

• PT=200: Payload Type, that is to say, the value of SR is 200

• Header: Part of it is used to identify the type of the message, such as SR or RR.

• Sender info: Part of it is used to indicate how many packets the sender has sent.

• Report block: Partially indicates that when the sender acts as a receiver, it receives packets from each SSRC.

•