Introduction to streaming media protocols (rtp/rtcp/rtsp/rtmp/mms/hls)

The RTP
          reference document (RFC3550/RFC3551
         Real-time Transport Protocol) is a transport layer protocol for multimedia data streams on the Internet. The RTP protocol specifies the standard packet format for delivering audio and video over the Internet. RTP protocol is commonly used in streaming media system (with RTCP protocol), video conference and Push to Talk (with H.323 or SIP) system, making it the technical foundation of IP telephony industry. The RTP protocol is used together with the RTP control protocol RTCP, and it is based on the UDP protocol.

         RTP itself does not provide on-time delivery mechanisms or other quality of service (QoS) guarantees, it relies on low-level services to implement this process. RTP does not guarantee delivery or prevent out-of-order delivery, nor does it determine the reliability of the underlying network. RTP implements in-order delivery. The sequence number in RTP allows the receiver to reassemble the sender's packet sequence, and the sequence number can also be used to determine the appropriate packet location. For example, in video decoding, sequential decoding is not required.

      RTP consists of two closely linked parts: RTP - transmits data with real-time properties; RTP Control Protocol (RTCP) - monitors quality of service and transmits information about ongoing session participants.

RTCP
        (Real-time Transport Control Protocol or RTP Control Protocol or RTCP for short) is a sister protocol of Real-time Transport Protocol (RTP). RTCP provides out-of-band control for RTP media streams. RTCP itself does not transmit data, but works with RTP to package and send multimedia data. RTCP periodically transmits control data between participants of a streaming multimedia session. The main function of RTCP is to provide feedback for the Quality of Service provided by RTP.
        RTCP collects statistics about media connections, such as: bytes transmitted, packets transmitted, packets lost, jitter, one-way and two-way network latency, and more. Network applications can use the information provided by RTCP to try to improve the quality of service, such as limiting information flow or switching to less compressed codecs. RTCP itself does not provide data encryption or authentication. SRTCP can be used for such purposes.
SRTP & SRTCP
        reference document RFC3711
        Secure Real-time Transport Protocol (SRTP) is a protocol defined on the basis of Real-time Transport Protocol (Real-time Transport Protocol or RTP), designed for unicast and multicast The RTTP data in the application provides encryption, message authentication, integrity assurance, and replay protection. It was developed by David Oran (Cisco) and Rolf Blom (Ericsson) and was first published by the IETF as RFC3711 in March 2004.
        Because the RTTP is closely related to the Real-Time Transport Control Protocol (RTP Control Protocol or RTCP), which can be used to control the session of the RTTP, the Secure RTTP also has a companion protocol, which is called Secure Real-Time Transport Control protocol (Secure RTCP or SRTCP); Secure Real-time Transport Control Protocol provides similar security-related features for RT-PCR as those provided by Secure RT-TCP for RT-TLS.
        When using Real-time Transport Protocol or Real-time Transport Control Protocol, it is optional to not use Secure Real-time Transport Protocol or Secure Real-time Transport Control Protocol; but even when Secure Real-time Transport Protocol or Secure Real-time Transport Control Protocol is used, all the features provided (like encryption and authentication) are also optional, and these features can be used or disabled independently. The only exception is when using Secure Real-Time Transmission Control Protocol, which must use its message authentication feature.
RTSP
       The reference document RFC2326

        was jointly proposed by Real Networks and Netscape. This protocol defines how one-to-many applications can efficiently transmit multimedia data over an IP network. RTSP provides an extensible framework that enables controlled, on-demand delivery of real-time data such as audio and video. Data sources include live data and data stored in clips. The purpose of this protocol is to control multiple data transmission connections, to provide a way to select transmission channels such as UDP, multicast UDP and TCP, and to provide a method for selecting transmission mechanisms based on RTP.

        RTSP (Real Time Streaming Protocol) is a multimedia streaming protocol used to control sound or video, and allows multiple streaming requirements to be controlled at the same time. The network communication protocol used during transmission is not within the scope of its definition, and the server can choose by itself Use TCP or UDP to transmit streaming content. Its syntax and operation are similar to HTTP 1.1, but it does not particularly emphasize time synchronization, so it is more tolerant of network delays. The aforementioned Multicast allows multiple streams at the same time, which not only reduces the network usage on the server side, but also supports multi-party video conferences. Because it operates similarly to HTTP1.1, the caching function "Cache" of the proxy server "Proxy" is also applicable to RTSP, and because RTSP has a redirection function, the server providing the service can be switched according to the actual load situation, so that the Avoid excessive load concentration on the same server and cause delays.
The relationship between RTSP and RTP
       RTP is not like http and ftp, which can completely download the entire video file. It sends data on the network at a fixed data rate. The client also watches the video file at this speed. It cannot be played repeatedly unless data is requested from the server again.

      The biggest difference between RTSP and RTP is that RTSP is a bidirectional real-time data transmission protocol, which allows the client to send requests to the server, such as playback, fast forward, and reverse operations. Of course, RTSP can transmit data based on RTP, and can also choose TCP, UDP, multicast UDP and other channels to send data, which has good scalability. It is a network application layer protocol similar to the http protocol. An application encountered so far: the server collects, encodes and sends two-way video in real time, and the client receives and displays the two-way video. Since the client does not need to perform any operations such as playback and rewinding of the video data, it can be directly implemented by UDP+RTP+multicast.

Don't buy Hetian jade, unless you read this [click to enter]
the transfer from the old player of Hetian jade! Pieces of boutique! Unexpected opportunity to find leaks! Buy jade must see!
View



Don't buy Hetian jade, unless you read here [click to enter]
Transfer from old Hetian jade players! Pieces of boutique! Unexpected opportunity to find leaks! Buy jade must see!
View


RTP : Real-time Transport Protocol (Real-time Transport Protocol)
RTP/RTCP is the protocol that actually transmits data.
RTP transmits audio/video data. If it is PLAY, the server sends it to the client. If it is RECORD, it can be sent by the client to the server
. The RTP protocol consists of two closely related parts: RTP data protocol and RTP control protocol (ie RTCP)
RTSP: Real Time Streaming Protocol (RTSP)
RTSP requests mainly include DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, OPTIONS, etc., as the name implies, can be known to play a role in dialogue and control
During the RTSP dialogue, SETUP can determine the port used by RTP/RTCP, PLAY/PAUSE/TEARDOWN can start or stop the sending of RTP, etc.
RTCP:
RTP/RTCP is the protocol that actually transmits data.
RTCP includes Sender Report and Receiver Report. For audio/video synchronization and other purposes, it is a control protocol.
SDP
        Session Description Protocol (SDP) provides multimedia session descriptions for purposes such as session notification, session invitation, and other forms of multimedia session initialization.
       The session directory is used to assist in the announcement of multimedia conferences and to deliver relevant setup information to session participants. SDP is used to transmit this information to the receiver. SDP is entirely a session description format - it's not a transport protocol - it just uses different appropriate transport protocols including Session Notification Protocol (SAP), Session Initiation Protocol (SIP), Real Time Streaming Protocol (RTSP), MIME Extensions e-mail and Hypertext Transfer Protocol (HTTP).

        SDP is designed for versatility, it can be applied to a wide range of network environments and applications, not just limited to multicast session directories, but SDP does not support negotiation of session content or media encoding.
        In the Internet Multicast Backbone (Mbone), the session directory facility is used to advertise multimedia conferences and deliver the conference address and conference-specific facility information required by the participants to the participants, which is done by the SDP. After SDP is connected to the session, it sends enough information to the session participants. SDP messages are sent using the Session Notification Protocol (SAP), which periodically multicasts notification packets to known multicast addresses and ports. These messages are UDP packets, which contain SAP protocol headers and text payloads. The text payload here refers to the SDP session description. In addition, information can also be sent via e-mail or the WWW (World Wide Web).

SDP text information includes:

session name and intent;
session duration;
media that make up the session;
information about the received media (address, etc.).
Protocol Structure
SDP messages are text messages using the ISO 10646 character set in UTF-8 encoding. The SDP session is described as follows: (optional fields marked with a * symbol):
v = (protocol version)
o = (owner/creator and session identifier)
​​s = (session name)
i = * (session information)
u = * (URI description)
e = * (Email address)
p = * (phone number)
c = * (connection information - this field is not required if included in all media)
b = * (bandwidth information)

one or more Time description (see below):
z = * (time zone adjustment)
k = * (encryption key)
a = * (0 or more session attribute lines)
0 or more media description (see below)

time description
t = (session active time)
r = * (0 or more repetitions)

media description
m = (media name and transport address)
i = * (media title)
c = * (connection information - if included at the session layer then This field is optional)
b = * (bandwidth information)
k = * (encryption key)
a = * (0 or more session attribute lines)

RTMP/RTMPS
RTMP (Real Time Messaging Protocol) is Adobe Systems Inc. for Flash Player An open protocol developed for audio, video, and data transfer between servers and servers.
It has three variants:
1) A plaintext protocol that works on top of TCP, using port 1935;

2) RTMPT is encapsulated in HTTP requests and can pass through firewalls;

3) RTMPS is similar to RTMPT, but uses HTTPS connections;

      RTMP protocol ( Real Time Messaging Protocol) is used by Flash for the transmission of objects, video, and audio. This protocol is built on top of the TCP protocol or the polling HTTP protocol. The
      RTMP protocol is like a container for data packets, which can be either Data in AMF format, or video/audio data in FLV. A single connection can transmit multiple network streams through different channels. The packets in these channels are transmitted in fixed-size packets.

mms
        MMS (Microsoft Media Server Protocol), Chinese "Microsoft Media Server Protocol", a protocol used to access and stream .asf files in Windows Media Server. The MMS protocol is used to access unicast content on Windows Media publishing points. MMS is the default method of connecting to the Windows Media Unicast service. If viewers type a URL into Windows Media Player to connect to content, rather than accessing it via a hyperlink, they must use the MMS protocol to reference the stream. The default port (port) for MMS is 1755.
        When connecting to a publishing point using the MMS protocol, use protocol inversion for the best connection. A "protocol rollover" begins with an attempt to connect a client via MMSU. MMSU is the MMS protocol combined with UDP data transfer. If the MMSU connection is unsuccessful, the server attempts to use MMST. MMST is the MMS protocol combined with TCP data transfer.
If you connect to an indexed .asf file and want to fast forward, rewind, pause, start and stop the stream, you must use MMS. You cannot fast forward or rewind with a UNC path. If you are connecting to the publishing point from standalone Windows Media Player, you must specify the URL for unicast content. If the content is published on demand at the main publishing point, the URL consists of the server name and the name of the .asf file. For example: mms://windows_media_server/sample.asf. where windows_media_server is the Windows Media server name and sample.asf is the name of the .asf file you want to stream.
If you have live content to be published via broadcast unicast, the URL consists of the server name and the publishing point alias. For example: mms://windows_media_server/LiveEvents. Here windows_media_server is the name of the Windows Media server, and LiveEvents is the name of the release.

HLS
      HTTP Live Streaming (HLS) is an HTTP-based streaming media transmission protocol implemented by Apple Inc., which can realize live and on-demand streaming media. It is mainly used in The iOS system provides audio and video live and on-demand solutions for iOS devices (such as iPhone and iPad). HLS on-demand is basically the common segmented HTTP on-demand, the difference is that its segments are very small.
       Compared with common streaming media live broadcast protocols, such as RTMP protocol, RTSP protocol, MMS protocol, etc., the biggest difference of HLS live broadcast is that what the live broadcast client obtains is not a complete data stream. The HLS protocol stores the live data stream as continuous, short-duration media files (MPEG-TS format) on the server side, and the client side continuously downloads and plays these small files, because the server side always sends the latest live broadcast. The data generates new small files, so that as long as the client continuously plays the files obtained from the server in sequence, the live broadcast is realized. It can be seen that, basically, it can be considered that HLS realizes live broadcast by means of on-demand technology. Since the data is transmitted through the HTTP protocol, there is no need to consider the problem of firewall or proxy, and the duration of the segmented file is very short, and the client can quickly select and switch the bit rate to adapt to playback under different bandwidth conditions. However, this technical feature of HLS determines that its delay is generally higher than that of ordinary live streaming protocols.　

     According to the above understanding, to realize HTTP Live Streaming, it is necessary to study and realize the following key technical points:
Collect data of video source and audio source
Perform H264 encoding and AAC encoding on raw data
Video and audio data are encapsulated into MPEG-TS packets
HLS segmentation generation strategy and m3u8 index file
HTTP transport protocol