rtsp rtmp http live on demand

http://blog.chinaunix.net/uid-26000296-id-4932817.html

http://blog.chinaunix.net/uid-26000296-id-4932822.html

http://blog.csdn.net/zhangxinrun/ article/details/50739237 In the



live broadcast application, RTMP and HLS can basically cover all clients to watch,
HLS mainly has a relatively large delay, and RTMP has the main advantage of low delay.



1. The characteristics of RTMP are as follows:

1) Adobe supports it very well:
   RTMP is actually the industry standard protocol output by encoders, and basically all encoders (cameras and the like) support RTMP output.
   The reason is that the PC market is huge, and the PC is mainly Windows. Windows browsers basically support Flash, and
   Flash supports RTMP very well.
2) Suitable for long-time playback:
   Because RTMP supports very well, it can play RTMP stream continuously for a long time in flash.
   At that time, the test was 1 million seconds, that is, it can be played continuously for more than 10 days.
   For commercial streaming media applications, the stability of the client is of course a must, otherwise how can the end user play if they can't see it?
   I know of an education client who initially used the player to play http streams and needed to play different files, but there was always a problem.
   If the server side converts different files into RTMP streams, the client side can keep playing;
   After the client adopted the RTMP solution, it was distributed through the CDN, and there was no problem with the client.
3) Low latency:
   Compared with YY's UDP private protocol, RTMP has a large delay (1-3 seconds delay), and
   RTMP has a low delay compared to HTTP streaming delay (generally more than 10 seconds).
   For general live broadcast applications, RTMP delay is acceptable as long as it is not the kind of requirement for telephone conversations.
   In general video conferencing applications, RTMP delay is acceptable. The reason is that we are generally listening when others are talking.
   In fact, a 1-second delay does not matter. We also need to think about it. So fast).
4) There is a cumulative delay:
   The technology must know the weakness. One of the weaknesses of RTMP is the cumulative error, because RTMP is based on TCP and will not lose packets.
   Therefore, when the network status is poor, the server will cache the packets, resulting in accumulated delay; when
   the network status is good, they will be sent to the client together.
   The countermeasure for this is to disconnect and reconnect when the client's buffer is large.



After measurement, it is found that when the network is in good condition:
  . RTMP delay can be about 0.8 seconds.
  . Multi-level edge nodes will not affect the delay (the edge server of a CDN with the same origin as SRS can do it)
  . The delay of Nginx-Rtmp is a bit large. It is estimated that it is caused by cache processing and multi-process communication?
  . GOP is a hard indicator, but SRS can close the GOP cache to avoid this effect.
  . If the server performance is too low, it will also cause the delay to increase, and the server will not be able to send data in time.
  . The client's buffer length also affects latency.
    For example, if the NetStream.bufferTime of the flash client is set to 10 seconds, the delay will be at least 10 seconds.



4. GOP-Cache

What is GOP? It is the temporal distance between two I-frames in the video stream.
What is the impact of GOP?
Flash (decoder) can start decoding and playing only after getting GOP.
That is to say, the server generally gives an I frame to Flash first.
Unfortunately, the problem comes. Suppose the GOP is 10 seconds, that is, there are key frames every 10 seconds.
What if the user starts playing at the 5th second?
The first solution: wait for the next I frame,
that is, wait another 5 seconds before starting to send data to the client.
This way the latency is very low and the stream is always real-time.
The problem is: during the 5 seconds of waiting, the screen will be black. The phenomenon is that the player is stuck there and there is nothing.
Some users may think that they are dead and refresh the page.
In conclusion, some clients will consider waiting for keyframes an unforgivable mistake, so what does the delay matter?
I just want to be able to start and play the video quickly, preferably open and play!


The second option: start putting it right away, what to
put it on?
You must know, put the previous I frame.
That is to say, the server needs to always cache a gop,
so that the client can start playing from the previous I frame and start quickly.
The problem is: the delay is naturally large.


Is there a good plan?
have! There are at least two types: the
encoder reduces the GOP, such as 0.5 seconds a GOP, so the delay is also very low, and there is no need to wait.
The downside is that the encoder compression ratio will be lower and the image quality will not be as good.



5. Cumulative Latency

In addition to GOP-Cache, there is another relationship, which is cumulative latency.
The server can configure the length of the live broadcast queue, the server will put the data in the live broadcast queue,
if it exceeds this length, it will be emptied to the last I frame:


of course, this cannot be configured too small, for
example, GOP is 1 second, queue_length is 1 second, this will As a result, the data will be emptied after 1 second, which will cause jumps.


Is there a better way? some.
The delay is basically equal to the buffer length of the client. Because the delay is mostly due to the low network bandwidth, the
server caches and sends it to the client together. The phenomenon is that the buffer of the client becomes larger.
For example, NetStream.BufferLength=5 seconds, then the buffer There is at least 5 seconds of data in the .


The best way to deal with accumulated latency is for the client to detect that the buffer has a lot of data and reconnect to the server if it can.
Of course, if the network has been bad, there is no way.





1. Comparison of distribution methods There are two main distribution methods on the
Internet : HLS and RTMP.
When to use who depends entirely on the application scenario.
There are other distribution methods, these distribution methods are not common and common on the Internet, and do not compare them:
  . UDP:
    such as YY's real-time application, video conferencing, etc., or RTSP and the like.
    This type of application is characterized by a particularly high requirement for real-time performance, which is calculated in milliseconds.
    The TCP family of protocols simply cannot meet the requirements, so HTTP/TCP is not reliable.
    There is no general solution for such applications, and must implement distribution (server) and playback (client) by themselves.
  . P2P:
    such as RTMFP or their own protocols.
    This type of application is characterized by saving bandwidth.
    At present, RTMFP on PC/flash is relatively mature, P2P on Android belongs to the starting group and has different standards,
    and P2P on IOS should not be heard of.
  . RTSP:
    This is not the main application on the Internet, but is widely used in other fields such as security.


In addition, HTTP is also divided into several types:
  . HTTP progressive:
    When the early streaming media server distributed http files, it was distributed as ordinary http files. This is called progressive download,
    which means that if the file is large, such as 1 GB in 1 hour, It is impossible to start playback from the middle.
    But this method is outdated. Many http servers support the seek of http files, which is to start playing from the middle.
  . HTTP stream: HTTP stream that
    supports seek, such as the on-demand distribution method of various video websites.
    Or a little more complicated, such as cutting a large file into several segments and then distributing it.
    At present, the mainstream distribution of domestic on-demand on pc/flash is this way.
  . HLS:
    This is the most widely adapted way (except for flash, which requires additional as library support),
    If there is vlc on the PC, the Android/IOS native player supports playing HLS, and the url in HTML5 can write the HLS address.
    In short, the mobile terminal is mainly HLS.
  . HDS: Adobe's own HLS, a piece of shit.
  . DASH: The HLS proposed by various companies has not been widely used yet.


Compare the following streaming media distribution methods on the Internet:
  . HLS: Apple's HLS, which supports on-demand and live broadcast.
  . HTTP: HTTP stream, the http stream defined by each company, applied to domestic video-on-demand websites.
  . RTMP: Live broadcast application, which has certain requirements for real-time performance, mainly PC.



2. RTMP
1. RTMP is essentially a streaming protocol, and its main advantages are:
  . High real-time performance:
    RTMP's real-time performance is within 3 seconds, and after multi-layer CDN node distribution, the real-time performance is also about 3 seconds.
    In some applications that require real-time performance, RTMP is the main method.
  . Support encryption:
    RTMPE and RTMPS are encryption protocols.
    Although HLS also has encryption, flash support for RTMPE/RTMPS should be relatively good on PC platforms.
  . High stability:
    RTMP is the most stable way to play flash on PC platform.
    If you do CDN or medium and large cluster distribution, it is necessary to choose a protocol with high stability.
    HTTP is also very stable, but HTTP is stable on the protocol;
    Stability is not just a matter of the server. In terms of cluster distribution, server management, active/standby switching, and client support,
    RTMP still has advantages in PC distribution.
  . Encoder access:
    The encoder is output to the Internet (it can also be output to applications such as udp multicast), mainly RTMP.
    For example, professional encoders, or flash web encoders, or FMLE, or ffmpeg, or security cameras, all support RTMP output.
    If you need to access multiple devices, such as providing cloud services;
    or if you want the web page to directly capture the camera; or if you can switch between different encoders,
    then RTMP as the server's input protocol will be the best choice.
  . System fault tolerance:
    There are many levels of fault tolerance. When RTMP cluster is implemented, N upper layers can be specified. In case of errors, switching will not affect the lower layers or clients.
    In addition, RTMP streams are not identified, and streams switched to other servers can also continue. play.
    HLS hot standby switching is not so easy.
    If the fault tolerance requirements for live broadcast are high, such as reducing the probability of problems, RTMP will be a good choice.
  . Monitorable:
    From the perspective of a monitoring system or an operation and maintenance system, streaming protocols should be more suitable for monitoring.
    HTTP stream monitoring does not feel so perfect. This is not an absolute advantage, but it is more advantageous.


2. The disadvantages of RTMP are:
  . Complex protocol:
    RTMP protocol is much more complex than HTTP, resulting in low performance.
    The test found that when two servers are directly connected to a 100Gbps network, HTTP can run to 60Gbps, but RTMP can only run to 10Gbps, and the
    CPU usage rate of RTMP is much higher.
    Complex protocols make it less convenient to develop, expand, and maintain software systems than HTTP, so HTTP servers are now popular,
    apache/nginx/tomcat, N multiple HTTP servers;
    and RTMP protocol, although it has been published long ago, but it is really distributed on a large scale There is no good performance,
    adobe's own FMS often has problems in CDN.
  . Cache is troublesome:
    Streaming protocol is inconvenient for caching. For example, on-demand, if RTMP streaming protocol is used, it will be very troublesome to buffer RTMP at the edge.
    If it is HTTP, the cache is actually very troublesome, but the cache of the HTTP server has been done for a long time, so just use it.
    This is the reason why the on-demand video is all over HTTP.



3. HTTP
HTTP refers to HTTP streams, such as on-demand streams of major video websites.
HTTP is essentially file distribution. The main advantages are:
  . High performance:
    The performance of HTTP has nothing to say, the protocol is simple, and various HTTP high-performance servers are also perfect.
    If the distribution volume is particularly large, such as an on-demand video website, and there is no real-time requirement for live broadcast, the HTTP protocol is the best choice.
  . No fragmentation:
    HTTP has no fragmentation than HLS, and HTTP distribution of large files is much more convenient than small file distribution.
    Especially for storage, the performance of small files is extremely low, which is a flaw.
  . Through the wall:
    It is impossible for the Internet to not open the HTTP protocol, otherwise it will not be called the Internet. So
    any port is blocked, it will not cause the HTTP stream to be unreadable. (However, RTMP can also pass through walls, using the RTMPT protocol).


The disadvantages of HTTP are:
  . Poor real-time performance:
    There is basically no such thing as real-time performance.
  . Poor native support:
    As far as the flash on the PC supports HTTP streaming, it seems that only mp4 is available on Android/IOS. In short, the mobile terminal does not support HTTP very well.



Fourth, HLS
HLS is an open standard of adobe, and it is also natively supported in Android3? or above.
The main advantages of HLS are:
  . High performance: the same as HTTP.
  . Through walls: Same as HTTP.
  . Native support is good: perfect support on IOS. Support is poor on Android. There are also various as plugins now supporting HLS on PC/flash.


The main disadvantages of HLS are:
  . Poor real-time performance: Basically, the delay of HLS is more than 10 seconds.
  . File Fragmentation: If HLS is distributed, the bitrate is low and the slice size is small, and the distribution of small files is not very friendly.
    Especially for some storage-sensitive situations, such as the storage of the source station, the embedded SD card.



5. Application method The
recommended method is:
  . The encoder outputs RTMP protocol.
  . Streaming media system access using RTMP protocol.
  . The live distribution within the streaming system uses RTMP.
  . PC + live broadcast + high real-time requirements: use flash to play RTMP.
  . PC + live broadcast + no real-time requirements: RTMP or HLS can be used.
  . PC + VOD: use HTTP or HLS.
  . Apple IOS/OSX: Both use HLS (real-time requirements are so high that you can parse RTMP yourself, or use external libraries,
    such as https://www.vitamio.org)
  . Andorid: Same as IOS, but it is certain that you can develop your own RTMP is supported.



===================================================== =============================





RTSP, RTMP, HTTP common points, differences

Common points :

1: RTSP RTMP HTTP are in Application application layer.

2: In theory, RTSP RTMPHTTP can be used for live broadcast and VOD, but generally RTSP RTMP is used for live broadcast and HTTP is used for VOD. When doing video conferences, the SIP protocol was originally used, but now it is basically replaced by the RTMP protocol.



Differences:

1: HTTP: Hypertext Transfer Protocol (ftp is File Transfer Protocol).

HTTP: (Real Time Streaming Protocol), a real-time streaming protocol.

The full name of HTTP is Routing Table Maintenance Protocol.



2: HTTP handles all data as files. The http protocol is not a streaming protocol.

RTMP and RTSP protocols are streaming media protocols.



3: The RTMP protocol is Adobe's private protocol and is not fully disclosed. The RTSP protocol and the HTTP protocol are shared protocols, and are maintained by specialized agencies.



4: RTMP protocol generally transmits flv, f4v format streams, and RTSP protocol generally transmits ts, mp4 format streams. HTTP doesn't have a specific stream.



5: RTSP transmission generally requires 2-3 channels, and the command and data channels are separated. HTTP and RTMP generally transmit commands and data on one TCP channel.



RTSP, RTCP, RTP difference

1: RTSP real-time streaming protocol

As an application layer protocol, RTSP provides an extensible framework, and its significance lies in making real-time streaming media data controlled and on-demand possible. In general, RTSP is a streaming media presentation protocol, which is mainly used to control the transmission of data with real-time characteristics, but it does not transmit data itself, but must rely on some services provided by the underlying transmission protocol. RTSP can provide operations such as play, pause, fast forward, etc. for streaming media. It is responsible for defining specific control messages, operation methods, status codes, etc., and also describes the interaction with RTP (RFC2326).



2: RTCP control protocol

The RTCP control protocol needs to be used together with the RTP data protocol. When the application starts an RTP session, it will occupy two ports at the same time, which are used for RTP and RTCP respectively. RTP itself does not provide reliable guarantees for in-order transmission of data packets, nor does it provide flow control and congestion control, which are all done by RTCP. Usually, RTCP uses the same distribution mechanism as RTP to periodically send control information to all members in the session. By receiving these data, the application obtains relevant information of session participants, as well as feedback information such as network status and packet loss probability. , so that the quality of service can be controlled or the network status can be diagnosed.

The functions of the RTCP protocol are implemented through different RTCP datagrams, which mainly include the following types:

SR: Sender report, the so-called sender refers to the application or terminal that sends out the RTP datagram, and the sender can also be the receiver. (SERVER fixed time to send to CLIENT).

RR: Receiver report, the so-called receiver refers to an application or terminal that only receives but does not send RTP datagrams. (SERVER receives the response sent by the client).

SDES: source description, the main function is to serve as the carrier of the identification information of session members, such as user name, email address, phone number, etc., and also has the function of conveying session control information to session members.

BYE: Notify to leave, the main function is to indicate that one or several sources are no longer valid, that is, to notify other members of the session that they will exit the session.

APP: Defined by the application itself, it solves the scalability problem of RTCP and provides great flexibility for the implementer of the protocol.



3: RTP data protocol

The RTP data protocol is responsible for packetizing streaming media data and realizing real-time transmission of media streams. Each RTP datagram consists of two parts: header and payload, of which the first 12 are headers. The meaning of bytes is fixed, and the payload can be audio or video data.

The place where RTP is used is PLAY. The server transmits data to the client using the UDP protocol. RTP adds a 12-byte header (description information) in front of the transmitted data.

RTP payload encapsulation design The network transmission in this paper is based on the IP protocol, so the maximum transmission unit (MTU) is up to 1500 bytes. When using the IP/UDP/RTP protocol hierarchy, this includes at least 20 bytes of IP header , an 8-byte UDP header, and a 12-byte RTP header. In this way, the header information must occupy at least 40 bytes, so the maximum size of the RTP payload is 1460 bytes. Taking H264 as an example, if a frame of data is larger than 1460, it needs to be split and packaged, and then unpacked at the receiving end, combined into a frame of data, and decoded and played.