Article directory
Preface
This article mainly explains the knowledge related to streaming media and its live broadcast. The knowledge involved is relatively simple and is mainly for a simple understanding.
1. Streaming media
1. Basic concepts
Streaming media is also called streaming media. It means that merchants use a video transmission server (for example: vlc) to send programs (for example: xxxx.mp4) as data packets and transmit them to the network. After the user decompresses the data through a decompression device, the program will be displayed as it was before sending.
The so-called streaming media refers to the media format that is played on the Internet through streaming transmission. Such as: audio, video or multimedia files. Streaming media does not download the entire file before playing , but only stores the beginning part of the content in the memory. The data stream of streaming media is transmitted and played at any time, but there is some delay at the beginning. The key technology for streaming media implementation is streaming transmission. Achieve downloading and playing at the same time .
2. Streaming
The traditional way of transmitting multimedia information such as audio and video over the Internet is to download it completely and then play it. Downloading often takes several minutes or even hours.
Using streaming media technology , streaming transmission can be realized, and sound, images or animations are continuously and uninterruptedly transmitted from the server to the user's computer. The user does not have to wait for the entire file to be downloaded, but only needs a few seconds or more than ten seconds. start delay to watch . While the audio, video, etc. are played on the user's machine, the remainder of the file continues to be downloaded from the server.
Network protocols: http, rtmp, rtsp, rtp/rtcp, udp, tcp
There are two ways to implement streaming: Realtime streaming and progressive streaming.
- Sequential streaming : Sequential downloading. Users can watch online media while downloading files. At a given moment, users can only watch the downloaded part and cannot jump to the previous part that has not yet been downloaded. Sequential streaming does not Like real-time streaming, it adjusts during transmission based on the speed of the user's connection.
- Nginx, web server; mp4 file/flv/mov/mkv/ts/rmvb; http
- Real-time streaming : Ensure that the media signal bandwidth matches the network connection so that the media can be viewed in real time.
- RTP/rtcp
3. Principles of streaming media technology
The streaming implementation requires caching.
In the implementation of streaming transmission, HTTP/TCP is generally used to transmit control information, and RTP/UDP is used to transmit real-time sound data.
Usually the required capacity of the cache is not large (gop:ibbpbbpbbp...), because the cache uses a circular linked list structure to store data: by discarding the content that has been played, the stream can reuse the vacated cache space to cache subsequent unplayed content. content.
4. Streaming media transmission mode
The reason why streaming media can realize real-time playback of multimedia data is that it adopts a specialized network control protocol and data transmission mechanism. The server has a dedicated streaming media publishing system, while the client has a dedicated player. Both parts need to cache data through the data cache area.
5. H.264 streaming media transmission system framework
2. Live broadcast
1. Streaming media protocols used in live broadcasts
RTMP, acronym for Real Time Messaging Protocol. This protocol is based on
TCP and is a protocol suite, including RTMP basic protocol and RTMPT/RTMPS/RTMPE and other variants. RTMP is a network protocol designed for real-time data communication. It is mainly used for audio, video and data communication between the Flash/AIR platform and streaming media/interactive servers that support the RTMP protocol.
RTMP is based on flash and cannot be played in iOS browsers, but its real-time performance is better than HLS.
HLS, HTTP Live Streaming (HTTP live streaming technology), Apple's dynamic bit rate adaptive technology. Mainly used for audio and video services on PCs and Apple terminals. Including an m3u(8) index file, TS media fragment file and key encrypted string file.
2. Module division of live broadcast
- Video recording end : Generally, it is an audio and video input device on a computer or a camera or microphone on a mobile phone. Currently, mobile phone video is the main one. Technology: webRTC (purpose: H5 video recording)
- Video player : It can be the player on the computer, the Native player on the mobile phone, or the H5 video tag, etc. Currently, the Native player on the mobile phone is the main one. Technology: HLS protocol or RTMP protocol (purpose: video playback), ffmpeg (purpose: mobile video decoding when using RTMP protocol)
- Video server : usually an nginx/s rs/．．．The server is used to accept the video source provided by the video recording end and provide streaming services to the video playing end. Technology: RTMP protocol (purpose: upload video stream), nginx rtmp-module or SRS (simple-rtmp-server) (purpose: streaming server)
3. Video live broadcast process
The process of live video can be divided into the following steps:
collection -> processing -> encoding and encapsulation -> push to server -> server stream distribution -> player stream playback
①. Push the stream to the server
There are three main push protocols:
- RTSP (Real Time Streaming Protocol) : Real-time streaming protocol, a multimedia streaming protocol used to control sound or video;
- RTMP (Real Time Messaging Protocol)
: Real-time messaging protocol, an open protocol developed by Adobe for audio, video and data transmission between Flash players and servers ; - HLS (HTTP Live Streaming) : It is an HTTP-based streaming media transmission protocol implemented by Apple;
The RTMP protocol is based on TCP and is a network protocol designed for real-time data communication. It is mainly used for
audio, video and data communication between the flash/AIR platform and the streaming media/interactive server that supports the RTMP protocol. Software that supports this protocol includes Adobe Media Server/Ultrant Media
Server/red5, etc.
It comes in three variants:
- RTMP is a clear text protocol that works on top of TCP, using port 1935;
- RTMPT is encapsulated in HTTP requests and can traverse firewalls;
- RTMPS is similar to RTMPT, but uses an HTTPS connection;
RTMP is currently the mainstream streaming media transmission protocol and is widely used in the field of live broadcast. It can be said that the vast majority of live broadcast products on the market adopt this protocol.
②. Server stream distribution
The role of the streaming media server is to be responsible for the release and rebroadcast distribution functions of live streams.
But I chose Nginx, which is an excellent free web server that can expand the live broadcast function through plug-ins.
Nginx, srs, crtmpserver,
Live555, darwin,
webrtc
My qq: 2442391036, welcome to communicate!