In video compression, each frame represents a still image. In actual compression, various algorithms are used to reduce the data capacity, of which IPB is the most common. general speaking
I frame is a key frame, which belongs to intra-frame compression. It is the same as AVI compression. You can understand it as the complete preservation of this frame of picture; only this frame of data is needed for decoding (because it contains the complete picture)
P means forward search. P frame represents the difference between this frame and the previous key frame (or P frame). When decoding, it is necessary to superimpose the difference defined by this frame with the previously buffered picture to generate the final picture. (That is, the difference frame, P frame does not have complete picture data, only data that is different from the picture of the previous frame)
The B frame is a two-way difference frame, that is, the B frame records the difference between the current frame and the previous and next frames (the specifics are more complicated, there are 4 cases), in other words, to decode the B frame, you must not only obtain the previous buffered picture, but also decode After the screen, the final screen is obtained by superimposing the front and back screens with the current frame data. B frame compression rate is high, but the CPU will be more tired when decoding.
From the above explanation, we know that the decoding algorithm of I and P is relatively simple, and the resource occupancy is relatively small. I only need to complete it by myself. For P, the decoder only needs to buffer the previous picture. When encountering P, Just use the previously buffered picture. If the video stream only has I and P, the decoder can read and decode while going linearly regardless of the subsequent data. But many movies on the Internet use B-frames, because B-frames record the difference between the front and rear frames, which can save more space than P-frames, but in this way, the file size is small, and the decoder is troublesome. When decoding, you must not only use the previously buffered picture, but also know the next I or P picture (that is, pre-reading and pre-decoding), and B-frames cannot simply be discarded, because B-frames actually contain picture information , If you simply drop it, and simply repeat the previous picture, it will cause a picture card (in fact, it is a frame loss), and because the movie on the Internet often uses a lot of B frames to save space, B frames are used more, yes Players that don't support B-frames will cause more trouble and the picture will get stuck. Generally speaking, the compression rate of I is 7 (similar to JPG), P is 20, and B can reach 50. It can be seen that using B frames can save a lot of space, and the saved space can be used to save more I frames. Under the same bit rate, it can provide better picture quality.
Resolution: Affects the image size and is proportional to the image size: the higher the resolution, the larger the image; the lower the resolution, the smaller the image.
Video bit rate: (1: transmission bit rate; 2: encoding bit rate) bit rate: affects the volume, and is proportional to the volume: the larger the bit rate, the larger the volume; the smaller the bit rate, the smaller the volume.
1. Transmission code rate: The code rate is the number of data bits transmitted per unit time during data transmission. Generally, the unit we use is kbps, which is kilobits per second. That is, the sampling rate (not equivalent to the sampling rate, the unit of the sampling rate is Hz, which means the number of samples per second). The larger the sampling rate per unit time, the higher the accuracy, and the closer the processed file is to the original file. But the file volume is directly proportional to the sampling rate, so almost all encoding formats pay attention to how to use the lowest bit rate to achieve the least distortion. Around this core, cbr (fixed bit rate) and vbr (variable bit rate) are derived ), the "bit rate" is the degree of distortion, the higher the bit rate, the clearer, otherwise the picture is rough and more mosaic. 2. Encoding rate: the number of data bits per unit of time, (the following conclusion is that the encoding rate has nothing to do with the transmission rate) 1: In the case of a certain rate, the resolution is inversely proportional to the definition Relationship: The higher the resolution, the less clear the image, the lower the resolution, the clearer the image. (There is so much data in one frame of image, the higher the resolution (large area), the higher the bit rate, the less clear the relative image) 2: In the case of a certain resolution, the bit rate is proportional to the definition, the bit rate The higher the value, the clearer the image; the lower the bit rate, the less clear the image. Frame rate: It affects the smoothness of the picture and is directly proportional to the smoothness of the picture. The larger the frame rate, the smoother the picture; the lower the frame rate, the more dynamic the picture. If the bit rate is variable, the frame rate will also affect the volume. The higher the frame rate, the more pictures that pass through each second, the higher the required bit rate and the larger the volume. Frame rate: It is the number of frames of pictures transmitted in 1 second, which can also be understood as the graphics processor can refresh several times per second.
1. Audio code rate calculation formula: audio code rate = sampling rate x bit depth x channel = 44.1Khz x 16 bits x 2 channels = 1411.2 Kbps 2. File size = bit rate x duration (s) = file size = code Rate x duration (s) = (Kbps )= (Kbps )/ 1024 Kb = (MB) 2. Video file size calculation File size = bit rate x duration (s)/8 = (Kbps )= (Kbps )/ 1024 Kb = (MB)
Problem The principle of the audio and video synchronization problem of the codec brings about a problem: in the video stream, the first B frame cannot be decoded immediately, and it needs to wait for the subsequent I and P frames that it depends on to be decoded first, so that the playback time and decoding The time is inconsistent and the sequence is disrupted. How should these frames be played? At this time, we need to understand two other concepts: DTS and PTS. The concepts of DTS and PTS The concepts of DTS and PTS are as follows:
DTS (Decoding Time Stamp): Decoding time stamp. The meaning of this time stamp is to tell the player when to decode this frame of data. PTS (Presentation Time Stamp): Displays the time stamp. This time stamp is used to tell the player when to display the data of this frame. It should be noted that although DTS and PTS are used to guide the behavior of the player, they are generated by the encoder during encoding. When there is no B frame in the video stream, the order of DTS and PTS is usually the same. But if there are B frames, it comes back to the problem we mentioned earlier: the decoding order and the playback order are inconsistent. For example, in a video, the display order of the frames is: IBBP, now we need to know the information in the P frame when decoding the B frame, so the order of these frames in the video stream may be: IPBB, this time reflects that each frame is There are DTS and PTS. DTS tells us in what order to decode these frames, and PTS tells us in what order to display these frames. The sequence is roughly as follows: PTS:1423 DTS:1234Stream: IPBB audio and video synchronization The above mentioned the concept of video frame, DTS, PTS. We all know that in a media stream, in addition to video, it usually includes audio. Audio playback also has the concepts of DTS and PTS, but audio is not similar to the B frame in video and does not require bidirectional prediction, so the order of DTS and PTS of audio frames is the same. The audio and video are mixed and played together to present the generalized video that we often see. When the audio and video are played together, we usually need to face a problem: how to synchronize them to avoid the situation where the picture is not right. To achieve audio and video synchronization, it is usually necessary to select a reference clock. The time on the reference clock increases linearly. When encoding the audio and video stream, each frame of data is time stamped according to the time on the reference clock. When playing, read the time stamp on the data frame, and refer to the time on the current reference clock to schedule the playback. The timestamp mentioned here is the PTS we mentioned earlier. In practice, we can choose: sync video to audio, sync audio to video, sync audio and video to an external clock.