This article comes from the paper "Principles and Applications of Streaming Media Technology Based on RTMP Protocol". The article studies the principles and applications of RTMP protocol used in streaming media systems based on Flash platform, and discusses the various transmission methods of real-time streaming media on the network. The advantages and disadvantages are analyzed. Then, it analyzes the characteristics of streaming media technology based on RTMP protocol, and introduces the format of RTMP protocol message. Finally, a live streaming system based on RTMP protocol is implemented.
For instant messaging developers, the relevant theories and ideas in the article are the same for the study of data transmission solutions in instant messaging real-time audio and video (video-time audio and video calls of IM chat applications) technology. Reference meaning, hope to bring you some inspiration.
Abstract
This article introduces the principle and application of RTMP protocol used in streaming media system based on Flash platform. First, the advantages and disadvantages of various transmission methods of streaming media on the network are analyzed. Then, it analyzes the characteristics of streaming media technology based on RTMP protocol, and introduces the format of RTMP protocol message. Finally, a live streaming system based on RTMP protocol is implemented.
1 Introduction
In recent years, with the increase of network bandwidth and the development of multimedia compression coding technology, streaming media technology has been widely used. The global streaming media market is developing at an extremely high speed and gradually replacing the traditional Internet based on text and pictures. According to Cisco's VisualNetworkingIndex (VNI) statistics, streaming media traffic accounted for only 5% of the total global Internet traffic in 2005, and by 2011 this proportion had increased to 40%, and it is expected that this proportion will further increase to 62% by 2015. %. At the same time, streaming media technology has broken through the limitations of computers and entered the fields of tablet computers and smart phones, and an era of Video Every Where is about to come [1].
In this environment of rapid development of streaming media, TV stations and video service providers in various places have started their own streaming media business. When building a business platform, how to choose a suitable streaming media platform becomes a crucial issue. China Internet TV, China Educational TV, Henan TV, Shenzhen TV and many other TV stations, as well as Liujianfang, Qiyi.com, Weishi.com and other streaming media service providers have chosen Adobe's Flash platform-based streaming media system. The RTMP protocol [2] used by the system to transmit data has therefore been widely used. This article will analyze its characteristics in detail and build a live streaming system based on RTMP protocol.
[Article benefits] The editor recommends my own linuxC/C++ language exchange group: 832218493. I have compiled some learning books, video materials and some source code that I think are better to share in it, and you can add them if you need them! ~
More excellent articles and videos in the public account
2 Streaming
The current streaming media services in the Internet can be roughly divided into two modes in terms of transmission methods: sequential streaming and real-time streaming.
1Sequential streaming
Sequential streaming uses a common HTTP server as a server for storing multimedia files. When the client initiates a connection and wants to watch multimedia resources, it directly downloads the file to the temporary folder of the client's local system through the HTTP protocol, and then uses the player to play the downloaded file. Its interaction with the server is shown in the figure below.
The essence of sequential streaming is to play local files. Sequential streaming has been widely used at present: YouTube, Youku, Tudou and other video service providers have adopted this method to provide multimedia services. The main advantage of sequential streaming is that it reduces the pressure on the server, that is, when the multimedia file is downloaded, the connection can be disconnected, thereby saving server resources and serving other clients. In addition, sequential streaming uses an ordinary HTTP server, and video service providers do not have to spend extra money to purchase streaming media servers, thus saving a sum of money.
Sequential streaming is shown in the figure below:
2 Real-time streaming
Real-time streaming uses a dedicated streaming media server to store multimedia files. When the client initiates a connection and wants to watch multimedia resources, the multimedia data located on the streaming media server is generally directly transmitted to the client's player through a proprietary real-time streaming protocol, and then played in real time. The flow of his interaction with the server is shown in the figure below.
The application of real-time streaming is currently still in the development stage, and is mainly used in live webcasts and on-demand genuine movies. When using real-time streaming to watch multimedia resources, since the files will not be downloaded locally, the content of the audiovisual provider can be prevented from being illegally copied, thereby protecting the copyright of the audiovisual content. In addition, when you use real-time streaming to watch multimedia resources, you can jump to any position of the video and audio at will, instead of only watching the downloaded part like sequential streaming, so it greatly increases the viewing time Degrees of freedom.
Real-time streaming is shown below:
3 Real-time streaming
The media data transmission in the streaming media system requires the support of the corresponding real-time streaming protocol. The real-time streaming protocol belongs to the application layer protocol in the Internet TCP/IP five-layer system structure. In the current Internet, many real-time streaming protocol standards are company-owned, so these protocol specifications are not public. There are currently several publicly standardized real-time streaming protocols:
1RTSP+RTP
RTSP was proposed by the IETF (Internet Engineering Task Force) [3]. The full name of the RTSP protocol is Real Time Streaming Protocol, that is, the real-time streaming protocol, which is an IETF RFC standard.
RTSP is used to control the transmission of streaming media, such as establishing a connection, playing, pausing, etc., but it does not transmit multimedia data itself. Multimedia data is usually transmitted using the RTP/RTCP protocol. The full name of RTP/RTCP protocol is Real-time Transport Protocol / Real-time Transport Control Protocol, which is also the IETF RFC standard, which is specially used to transmit multimedia data.
Although RTSP + RTP is a combination of international standards, it has not been able to "dominate the world" in the Internet world. This has a lot to do with the Internet environment.
RTP/RTCP is used as the network protocol for transmitting multimedia data. Generally, UDP protocol is used as the network protocol of its transport layer [3]. UDP is connectionless and does not provide reliable delivery. Therefore, when data is transmitted on the Internet (especially WAN), it is easy to cause packet loss, delay, jitter and other problems. Multimedia data has high requirements for packet loss, delay, and jitter, and a little problem will greatly affect the quality of user experience (QoE) [4]. Therefore, there are not many streaming media that use RTSP + RTP transmission on the Internet.
However, there are exceptions. Nowadays, the real-time audio and video chat in the mainstream instant messaging (ie IM chat application) uses this component more. The reason is that the real-time audio and video chat is different from the scene of movie playback, and the real-time is more than complete. Important, when the network is not good, you can load it for a while, and the user can wait, but this is a taboo in real-time audio and video chat. Fortunately, when the network is bad in real-time audio and video chat, the loss of image frames is not There will be a substantial impact, so for efficiency and real-time experience, the UDP method of RTP/RTCP is the opposite but the best.
Different from streaming media transmitted on the Internet, IPTV usually uses RTSP + RTP to transmit multimedia data [5]. Because IPTV usually uses private network transmission, the network condition is good, and there are few problems such as packet loss, delay, jitter, etc., and the simple protocol rules of UDP can greatly improve the transmission efficiency, so you can use RTSP + RTP "with confidence" Mode transmission.
2MMS
MMS was proposed by Microsoft Corporation. The full name of the MMS protocol is the Microsoft Media Server protocol, that is, the Microsoft Media Service Protocol, which is used to access the content on the Windows Media publishing point.
3HLS
HLS was proposed by Apple. The full name of HLS is HTTP Live Streaming, which is a real-time streaming protocol based on HTTP, which can realize live and on-demand streaming media. It is mainly used in the iOS system to provide audio and video live and on-demand solutions for iOS devices (such as iPhone and iPad).
4RTMP
RTMP was proposed by Adobe. The full name of RTMP protocol is Real Time Messaging Protocol, that is, real-time messaging protocol, which is used to transfer video, audio and data between Flash platforms. Different from the way that the RTSP + RTP combination provides streaming media services, the RTMP protocol itself can not only transmit multimedia data but also control multimedia playback.
The RTMP protocol uses the TCP protocol as the network protocol of its transport layer. TCP is connection-oriented [3] and provides a reliable delivery protocol, so there will be no packet loss when transmitting on the Internet, thus ensuring the user experience (QoE). However, the price of reliable delivery provided by the TCP protocol is that it adds some additional overhead and takes up some bandwidth and processor resources. With the increase of network bandwidth and the development of computer hardware, these expenses will become more and more insignificant. Therefore, the RTMP protocol has a good development prospect in the future.
4 Features of RTMP-based systems
Many Internet TV stations and streaming media service providers choose the RTMP protocol as their application layer protocol for streaming media services because it has the following characteristics: No client programs are required to ensure the quality of media transmission.
1 No need to install client program
There is no need to install client programs to watch streaming media provided by the RTMP protocol, which greatly simplifies the complexity of client operations. Generally, watching streaming media requires the support of the corresponding client software, and users must download the corresponding software (or plug-in) to watch streaming media. The streaming media client that supports the RTMP protocol can be made into an ordinary Flash file, and the web browser that has installed the Flash Player can automatically download the file and run it. And Flash Player is a necessary plug-in for Internet access. According to statistics, 98% of web browsers in the world have Flash Player installed. Therefore, ordinary users do not need any operation, as long as they open the playback page with a web browser, they can watch streaming media [6].
2 Guaranteed media transmission quality
The RTMP protocol effectively guarantees the quality of media transmission and enables users to watch high-quality multimedia. RTMP uses the TCP protocol as its transport layer protocol to avoid the loss of quality caused by the loss of multimedia data in the process of WAN transmission. In addition, the H.264 video encoding method supported by the FLV encapsulation format transmitted by the RTMP protocol can display good quality pictures at a very low bit rate, which is very suitable for watching streaming media when the network bandwidth is insufficient.
5 RTMP specification
1RTMP standard protocol format
The RTMP protocol is an application layer protocol in the Internet TCP/IP five-layer architecture. The basic data unit in the RTMP protocol is called a message. When the RTMP protocol transmits data on the Internet, the message will be split into smaller units called Chunk.
Message: A
message is the basic data unit in the RTMP protocol. Different types of messages contain different Message Type IDs, which represent different functions. A total of more than ten types of messages are specified in the RTMP protocol, each of which plays a different role. For example, messages with Message Type ID between 1-7 are used for protocol control. These messages are generally used for the management of the RTMP protocol itself, and users generally do not need to manipulate the data in them. Messages with Message Type ID 8 and 9 are used to transmit audio and video data respectively. Messages with Message Type ID 15-20 are used to send AMF encoding [8] commands, responsible for the interaction between the user and the server, such as play, pause, etc. The message header (Message Header) consists of four parts: Message Type ID to mark the message type, Payload Length to mark the length of the message, Timestamp to mark the timestamp, and Stream ID to mark the media stream to which the message belongs. The message structure of the message is shown in the figure below.
Message block:
When transmitting data on the network, the message needs to be split into smaller data blocks to be suitable for transmission on the corresponding network environment. It is stipulated in the RTMP protocol that a message is split into Chunks when it is transmitted on the network. The Chunk Header consists of three parts: Chunk Basic Header used to identify this block, Chunk Message Header used to identify the message to which this block load belongs, and Extended Timestamp that only appears when the timestamp overflows. The message structure of the message block is shown in the figure below. 
Message block:
In the process of dividing the message into several message blocks, the message body part is divided into fixed-size data blocks (the default is 128 bytes, and the last data block can be less than the fixed length), and in the header Add the Chunk Header to form the corresponding message block. The message block process is shown in the figure below. A message with a size of 307 bytes is divided into 128-byte message blocks (except for the last one).
In the process of RTMP transmitting media data, the sender first encapsulates the media data into messages, then divides the messages into message blocks, and finally sends the divided message blocks through the TCP protocol. After receiving the data through the TCP protocol, the receiver first reassembles the message block into a message, and then can recover the media data by decapsulating the message.
2 connection method
The RTMP protocol stipulates that two logical structures need to be created before publishing a media stream: the first step is to establish a network connection (Net-Connection); the second step is to establish a network stream (NetStream) based on the network connection. Among them, the network connection represents the basic connection between the server and the client; the network stream represents the connection of sending multimedia data, but many network streams can be created based on this connection. They are based on the channel. Only one network relationship can be established between the server and the client as shown in the following figure:
6 Construction of RTMP-based live broadcast system
1 system structure
This article will implement a live streaming system based on RTMP protocol. A complete streaming media live broadcast system includes the following parts: video source, streaming media server and client. The composition of the system is shown in the figure below. The video source publishes the video data to the streaming media server through the RTMP protocol; after the video is successfully published, the client connects to the streaming media server through RTMP, and the corresponding video can be played. Among them, the video source and the client are both Flash programs written in ActionScript language[10], and the streaming media server uses Ado-be's Flash Media Server software.
2 Video source
Video source is an application that provides video streaming. This system uses a Flash program written in ActionScript code as the video source. This program collects the camera data of the machine, compresses the data and uses the RTMP protocol to publish the data to the corresponding application (Application) of the streaming media server.
The following briefly introduces the meaning of important codes in the video source part:
// 建立一个 RTMP 网络连接
var nc: NetConnection = new NetConnection( ) ;
//连接到 IP 为 222. 31. 64. 249 的流媒体服务器上名字为 publishlive 的应用程序
nc.connect("rtmp://222.31.64.249/publish-live" ) ;
// 建立一个基于该连接的网络流
ns = new NetStream( nc) ;
// 调用本机的摄像头
cam = Camera. getCamera( ) ;
// 把摄像头添加到新建的流上
ns. attachCamera( cam) ;
//把一个多媒体流发布到服务器的应用程序 上,取名为“myCamera”
ns. publish( " myCamera" ," live" ) ;
3Streaming server
Streaming media server is software that stores (or receives) media streams and waits for client connections. This system uses Adobe's Flash Media Server as the streaming media server. The media stream must be published to the application that has been registered on the streaming media server. Create a new "publishlive" folder under the "Applica- tion" folder of the Flash Media Server installation directory, and you can register an application called "publishlive" without writing any code.
4 clients
The client is an application that plays video streams. This system uses a Flash program written in ActionScript language as the client for playing real-time streams. Use the RTMP protocol to obtain the video data from the streaming media server and display it for playback.
The following briefly introduces the meaning of some key codes of the client:
// 建立一个 RTMP 网络连接
var nc: NetConnection = new NetConnection( ) ;
//连接到 IP 为 222. 31. 64. 249 的流媒体服务 器上名字为 publishlive 的应用程序
nc.connect( "rtmp://222.31.64.249/publish- live" ) ;
// 建立一个基于该连接的多媒体流
nsPlayer = new NetStream( nc) ;
// 播放名为“myCamera”的多媒体流
nsPlayer. play( " myCamera" ) ;
// 新建一个 Video 对象用于显示视频
vidPlayer = new Video ( cam. width,cam.height) ;
// 将多媒体流添加到 Video 类上
vidPlayer. attachNetStream( nsPlayer) ;
// 在 Flash 舞台上显示 Video 对象
addChild( vidPlayer) ;
7 Summary and outlook
This article analyzes the two basic transmission methods of streaming media: the difference between sequential streaming and real-time streaming. And focused on analyzing the characteristics of several main real-time streaming protocols. Based on the RTMP protocol, it analyzed its characteristics and format, and finally realized a live streaming system based on the RTMP protocol. It is very helpful for a comprehensive understanding of the principles of the RTMP protocol, and at the same time can provide a reference for the design and implementation of more complex streaming media systems based on the RTMP protocol.
Since the beginning of this year, with the gradual rise of Internet TV (Over-The-Top TV), streaming media technology will usher in a major stage of development. It is foreseeable that streaming media supported by streaming media technology will occupy part of the traditional TV market and gain a considerable number of customer groups in the future. A streaming media system based on RTMP protocol with good audio and video quality without the need for users to install a client will also emerge in many streaming media systems and gain a large market share.