RTMP is the core transmission protocol for various webcast applications at present, and it is also the most widely used protocol for interactive live broadcast. If the streaming media server (Server) is the backbone of webcasting, RTMP is the blood of webcasting. It can be said that without RTMP, there would be no such popular webcasting today.
The RTMP protocol is a protocol that works on the transport layer. When we play the RTMP stream, what happens the moment we click the play button? This article will unveil the mystery of the RTMP playback process for you.
As a benefit of this article, you can receive free C++ audio and video learning materials package, technical videos/codes, including (audio and video development, interview questions, FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, codec, push-pull streaming, srs)↓↓↓ ↓↓↓See below↓↓Click at the bottom of the article to get it for free↓↓
1. RTMP overview
RTMP connections all start with a handshake. ① The connection establishment phase is used to establish the "network connection" between the client and the server; ② The stream establishment phase is used to establish the "network flow" between the client and the server; ③ The playback phase is used to transmit video and audio data.
In this article, we use tcpdump to capture packets and wireshark to analyze the captured packets.
2. Capture RTMP packets
tcpdump packet capture:
tcpdump host IP and tcp port 1935 -w a.pcap
Whireshark analysis:
Open a.pcap with wireshark
3. Handshake
An RTMP connection starts with a handshake, let's look at the following picture first:
First of all, we need to be clear that the client IP is 192.168.1.102 (my computer), and 123.183.164.23 is the RTMP server.
The script should look like this:
- The RTMP protocol is the upper layer protocol of the TCP protocol, so a TCP connection must be established first, so I saw the 1-4 TCP three-way handshake packets.
- The client sends C0 chunks (chunks) to the server, indicating that it wants to shake hands with the server, and C0 contains the version number.
- After receiving C0, the server checks whether the version in C0 is supported, and if it supports, sends S0 as a response, otherwise it should terminate the connection.
- Both the client and the server wait for C1 and S1, respectively, for version confirmation.
- The client sends C2 after receiving S1, and the server sends S2 after receiving C1 (confirm sending, and the test handshake is completed.)
However, the actual implementation of the protocol is not according to the script (if it is done according to the script, the delay will be greatly increased), the actual implementation is as follows:
- The RTMP protocol is the upper layer protocol of the TCP protocol, so a TCP connection must be established first, so I saw the 1-4 TCP three-way handshake packets.
- The client sends the C0+C1 block, and directly tells the server the version I sent and I confirm it myself.
- The server was even more ruthless, and it was pulled back with a big mouth (send S0+S1+S2).
- After receiving it, the client sends C2 and the handshake is completed!
Attached is the flow chart in the RTMP protocol:
Some students stood up and asked questions. The No.13 package is C2, so what is the connect behind? This classmate, please sit down and look down.
4. Establish a network connection (NetConnection)
hint:
Network connections represent the underlying connectivity between server-side applications and clients.
Let’s look at the captured packets:
After the RTMP handshake is completed, the network connection needs to be established. Everyone knows what an ordinary standard rtmp stream looks like? Is this true for rtmp://IP:PORT/APP/Stream?
The actual script looks like this:
1. When the client sends C2, it also sends a connection request command to establish a network connection with the server application (live7). This is the Application in the RTMP URL. Soga, are you suddenly enlightened?
2. After receiving the connection request from the client, the server sends the following information:
It mainly tells the client to confirm the window size and set the node bandwidth, and then the server connects the "connection" to the specified application and returns the result, "Network connection successful". And returns the message at the beginning of the stream (Stream Begin 0).
3. After receiving the message from the server, the client returns the confirmation window size (No.18), and the network connection is created at this time.
Agreement flow chart:
5. Create a network stream (NetStream)
hint:
A network stream represents a channel for sending multimedia data.
Only one network connection can be established between the server and the client, and multiple network streams can reuse this network connection.
Then look at the packet capture:
Now that the tunnel is dug, all that's left is to lay the rails!
- The client sends to the server to set the Buffer Length to 0, 3000ms, and requests to create a stream (createStream).
- The client sends a control message to the server (go, I don't know what to do!)
- After receiving the request, the server sends _result() to the client to respond to the message to create the stream. At this point, NetStream creation is completed.
Agreement flow chart:
6. Play
hint:
Main function: transmit audio and video data
Look at the packet capture:
Everything is ready, all we need is the east wind.
1. The client sends a playback command to the server, requests to play the stream, and sets Buffer Length 1, 3000ms.
2. After receiving the request, the server sends a protocol message to set the block size to the client, and sends a bunch of other messages together:
Including Stream Begin (notifying the client that the stream ID is 0), NetStream.Play.Reset and NetStream.Play.Start (notifying the client that the playback is successful, and reset will adjust the stream ID to 1), etc.
3. The server sends a push notification to the client, along with metadata information (resolution, frame rate, audio sampling rate, audio bitrate, etc.) and video and audio data. At this point, the client can start playing the rtmp stream normally.
Agreement flow chart:
As a benefit of this article, you can receive free C++ audio and video learning materials package, technical videos/codes, including (audio and video development, interview questions, FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, codec, push-pull streaming, srs)↓↓↓ ↓↓↓See below↓↓Click at the bottom of the article to get it for free↓↓