Streaming video protocol is a network transport protocol.
basic concepts
Terms of video technology
-
A series of nouns: AVI, MPEG, RMVB, MP4, MOV
-
Noun Series II: H.261, H.262, H.263, H.264, H.265. Focusing on H.264
-
Series of three terms: MPEG-1, MPEG-2, MPEG-4, MPEG-7.
Video: a series of pictures.
Frame: a picture is one.
Frame rate: that is, how many pictures per second
Pixels: a pixel RGB, eight bits each. A picture of a good number of pixels.
Kinds of video per second, how much?
30 × 1024 × 768 × 24 = 566,231,040Bits = 70,778,880Bytes
Video every minute how much? 4,246,732,800Bytes, G 4 is already a.
Code: as few Bit save the video, is a compression process.
Video and pictures use some of the features of compression: spatial redundancy, temporal redundancy, visual redundancy, coding redundancy.
Essentially convey more information with less energy. This is somewhat similar to Shannon's information theory.
Video encoding two schools
-
ITU's VCEG
-
ISO's MPEG
-
ITU-T and MPEG jointly developed H.264 / MPEG-4 AVC
After encoding, the original one picture frame into binary data, the binary data into the file, save it in a certain format, is a series of nouns.
Then flow: receiving a video stream from the server where the anchor client
Transcoding: Switch from one encoding format to another encoding format.
Pull stream: the server from the client viewer pulls the video stream
Decoding: audience client binary stream, into a picture frame, consisting of video.
Broadcast network, transmission of data stream
We now live example of a network, it is about how to upload video shot from the anchor to the network, and how from the network to the fans on each computer screen.
1, the video camera will capture the anchor down, encoded video coding protocol, converted into a binary byte stream, via the HTTP protocol package them sent.
2, the server side which has the same video coding protocol, the TCP protocol parsed from the packet buffer or to make a transcoding, is converted from one format to another video encoding format. Because the playback device vary widely used by the client.
3, the client side requests a video stream from the server. This process is the test server, the server should at the same time as the huge number of client service. Then you need a video distribution network, video pre-loaded on the server where the client closer.
If people are watching the National Day parade, Liaoning viewers to see the video from the server Liaoning region is pulling the video stream. Guangzhou, the audience is watching a video stream on the Guangzhou region server.
4, after the computer viewer or display video stream pulled from the network, or binary bytes to be decoded to see.
The whole process live like this figure below:
Decomposition of the video transmission process
Encoding : the specific operation of the video into binary, to reduce the size of the video image, a video sequence is divided into three frames.
-
I-frames, keyframes are complete picture, we need to complete this frame data decoding.
-
P-frame, forward predictive coding to frames, which is represented by a difference, a difference between the key frame and the previous frame.
-
B-frames, bidirectionally predictive-coded frame interpolation. This frame represents the difference of the front and rear frames. To decode a B-frame, a picture of the front and rear of a picture desired by the screen data before and after the present frame and superimposed to obtain the final picture.
When transmitting, to facilitate spatial encoding, each frame is divided into many pieces, each lot is divided into macroblocks, each macroblock is divided into many small pieces.
These are divided into small pieces can not be issued directly to, or else chaos, the client would like to recover the picture is impossible. But how do you ensure this?
The video is divided into a lot of frames, a frame plate there are a lot, there is a macroblock, subblock. These sub-blocks constitute a sequence, then the need for a way to be able to these sub-blocks of the sequence. To ensure that they can be transmitted via the TCP protocol to the other end of the network, and can be restored to a video. The idea is to package the sheet frame NAL unit (NALUs) .
Each of NALU: Beginning identifier, NALU header and Payload thereof.
NALU head inside the main content type NAL Type.
-
The SPS, a sequence parameter set, comprising a sequence of images of all the information, such as image size, the video format.
-
PPS, a picture parameter set, include all relevant information pieces of a picture, comprising an image type, serial number and the like.
Decoding the video stream from a video, these two parameters are necessary. In order to ensure fault tolerance, in front of each I frame, I will pass over these two parameters.
The entire video format is this. Can be divided into a lot of video frames, one frame may be divided into many pieces, each piece are placed inside a NALU, are separated by a special start identifier NALU between, the first sheet in front of each I-frame, to be inserted SPS and PPS stored separately NALU, eventually forming a long sequence NALU.
Plug flow : package data stream transmitted to the peer
How binary streams packaged into a network packet sent needed here RTMP protocol.
RTMP protocol is a real-time information transmission protocol, Adobe company proposed to solve the multiplex transmission of multimedia data streams, and the subcontracting agreement. It is built on top of TCP protocol, it is primarily to ensure two things, one is the version number, a timestamp.
RTMP protocol is similar to the same need to connect TCP. Upon receipt of real-time information that come to find out the sender and receiver of the situation. Two of a smear can not come up on the message bar.
To transmit real-time messages, to determine in advance what to do? RTMP protocol to see how it is done, just send six messages to get. As shown below:
1, the first client sends C0, said his version, without waiting for other replies, direct recurrence C1, indicate their timestamp.
2, the server receives C0, to return S0, show their own version number, if no match is disconnected. After sending S0, the server continues to send S1, show their timestamp.
3, the client receives S1, the server back to a ack, timestamp show himself received the service side of the.
4. Similarly, the server receives C1, is sent back to the client a ack, timestamp show their receipt of the client.
This, the handshake is complete!
Just to complete the handshake connection is established, the two sides also need to pass some control information, such as the size of the Chunk block, the window size.
But also to create real time transmission of data a Stream, and then to push through this Stream flow (publish), it is to put NALU. Send Message, the Message also known as RTMP Packet package. Message format like this.
Large message block retransmission becomes small, can in the case of low bandwidth and reduce network congestion.
The transmission process of the data stream as follows:
The following is a process flow drawing viewers client:
Distribution network
Below that the distribution network server, which solves the problem caused by the pressure of a large audience to the server.
Center and edge delivery network is divided into two layers. Edge tier servers deployed throughout the country, and across major carriers in, and users are very close. Center is a cluster of streaming media services, is responsible for forwarding the content.
Intelligent load balancing system according to the user's location information, select the nearest edge server, to provide users with a push / pull service flow.
This mechanism will involve DNS, HTTPDNS, CDN and other technologies.
