Network infrastructure upgrades, audio and video transmission technology iterations, WebRTC open source and other factors have driven the gradual reduction of audio and video service delays, making real-time audio and video (RTC) technology a hot research direction. Real-time audio and video services are booming in the consumer Internet field, and are gradually penetrating into the industrial Internet field.

The benefits of this article, free C++ audio and video learning materials package, technical video/code, including (audio and video development, interview questions, FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, codec, push-pull stream, srs)↓↓↓ ↓↓↓See below↓↓Click at the bottom of the article to get it for free↓↓

1 Audio and video communication process

Audio and video communication technology relies on core audio and video codec, channel transmission, network scheduling and other capabilities to provide users with low-latency, high-quality audio and video content. The complete process of audio and video communication has the following links: pre- and post-processing, acquisition, encoding, transmission, decoding, buffering, rendering, etc.

Figure 1 Audio and video communication process

2 Protocols related to audio and video communication

The protocols involved in audio and video communication are: rtp, rtcp, rtmp, rtsp, http, hls, srt, sip. Among them, rtp and rtcp are transport layer protocols, which rely on the underlying udp protocol, and often cooperate with rts live broadcast; rtsp, rtmp, hls, and srt are application layer media transmission protocols, which are often used for live broadcast, interactive live broadcast, on-demand, etc.; sip relies on the underlying udp Protocol Signaling control protocol, used for media session initiation, termination, and modification control.

Table 1 Details of protocols related to audio and video communication

The computer network adopts a hierarchical design method. Layers are connected through interlayer interfaces. Each layer completes a function through a specific protocol, and multi-layer superposition completes the sending and receiving of the entire information. The relationship between the media communication protocol and the OSI protocol architecture is as follows:

3 Central signaling control service based on SIP

Cloud table SIP (Session Initiation Protocol) is an application layer session initiation protocol, which can be used for the initiation, termination, and modification of media sessions. It is an important protocol in NGN, that is, the next generation network. It is a multimedia communication protocol formulated by IETF. Participants in a session can communicate via multicast, mesh unicast, or a mixture of both. Signaling (Signaling) refers to the control instructions in the communication system, such as: status reporting, configuration parameter delivery, network resource scheduling, and is an important part of realizing audio and video communication. To achieve audio and video communication, information needs to be exchanged through the signaling server: media information, network information, specific services (for example: pull stream, pause, join the room, mute, etc.)

Figure 3 Signaling control service process

Media information: Before the two-terminal communication, use the SDP (SessionDescription Protocol) session description protocol to understand what capabilities both parties have, what codec is carried in the SDP media information, whether audio and video are supported, and what the encoding method is, etc.

Figure 4 Key attributes in the SDP session description protocol

Network information: When A communicates with B, it first transmits network-related information to the signaling server, and the server helps you switch to the peer end. After the peer end gets your information, if it is in the same local area network, it will be directly transmitted through P2P; If you are not there, first perform P2P traversal to see if you can get through.

4 SIPP performance test

Before the SIPP test process, we first understand the sip creation/end session process:

1. The calling party sends a SIP INVITE request message (carrying SDP) to the called party, requesting to establish a session

2. The called party returns a SIP 200 OK response message (carrying SDP) to accept the session establishment request

3. The calling party sends a SIP ACK request message to confirm the establishment of the session.

4. Any party in the session can send a SIP BYE request message to request to end the session;

5. The other party returns a SIP 200 OK response message to accept the session end request.

Figure 5 sip create/end session process

SIPP can be used to test many real SIP devices, such as SIP proxy, B2BUAS, SIP server, SIP/X gateway, SIPPBX, etc. It can imitate tens of millions of SIP proxies to call your SIP system.

❖ Take simulated UAC as an example, the test steps:

1. The user operates the management and control functions of the port device, such as streaming, pause, PTZ control and other functions

2.xshell connects to the server where the signaling is located, and executes the packet capture command tcpdump -i any -s 0 -w xx.pcap

3. Use xftp to synchronize the packet capture xx.pcap file to the local, and use wireshark to view it

4. Write a sipp script according to the interactive message of sip (as shown in Figure 1 on the right)

5. Put the uac script in the server directory deployed by sipp and execute the following command:

6.sipp -i 172.xx.xx.86 -p 5xxx5 -sf uac3.xml -inf data.csv 172.xxx.xxx.38:5xx0 -r 1 -l 10 -m 20

Figure 6 SIPP performance test script writing