foreword
This column will continue to update the content of C++ audio and video development , including beginner, intermediate and advanced chapters , including but not limited to audio and video basics, FFmpeg combat, QT, streaming media client, streaming media server, WebRTC Actual combat, Android NDK , etc. It is the knowledge point in the course purchased by the blogger for nearly 5,000 yuan. It is also mixed with some of my personal understanding. I hope it can help you get started with audio and video development with me .
------This column refuses to charge!
[ Special note: This article belongs to the sub-content of video basics in the primary chapter. 】
The primary article catalog 
http://t.csdn.cn/z2pRy The rest of the chapters are being updated...
[C++ audio and video development] primary article | basic concepts of images
Video Basics
Image base elements
★ pixels
Pixel: The basic unit of a picture, pix is the abbreviation of the English word picture, and the English word "element" is added to get "pixel", abbreviated as px, so "pixel" means "image element".
For example: 2500X2000 photo refers to 2500 pixels horizontally and 2000 vertical pixels, a total of 2500X2000=5 million pixels, also commonly known as 5 million pixel photos.
When we continuously enlarge the eyes of the model above, and after zooming in to the pixel level, as shown in the lower right corner of the above figure, we can clearly see that the picture has become a rectangle composed of square blocks, which is the essence of the image , and these squares are pixels.
★ Resolution
Resolution: The size or dimensions of the image.
For example: 1920X1080. Common resolutions are: 360P (640X360), 720P (1280X720), 1080P (1920X1080), 4K (3840X2160), 8K (7680X4320).
Then there is a question - what is the difference between resolution and pixels? In fact, it is like this: 2500X2000 is the resolution, and 2500X2000=5 million is 5 million pixels.
- Resolution: It describes the number of pixels and the ratio of the number of horizontal pixels to the number of vertical pixels .
- Pixel: is a basic unit.
-------------------------------------------------------
The difference between different resolutions:
Often said 1080p and 720p is actually a single vertical pixel count . In addition, according to the ratio of 16:9 (width:height), the number of horizontal pixels needs to be calculated: the number of horizontal pixels of 720p is 720÷9×16=1280, and the total number of pixels is 921,600 pixels, which is about 920,000 pixels. 1080p has 1920 horizontal pixels, for a total of 2073600 pixels or about 2 million pixels, more than double that of 720p.
The higher the resolution of the image, the more pixels in the image, and the sharper the image, so 1080p is sharper than 720p.
From the above figure, we can clearly see that the more pixels, the greater the density of pixel distribution after zooming in, which means that more detailed textures are retained .
From the above picture, we can intuitively see that the 1080p image is indeed much clearer than the 720p image. However , it should be noted that the image clarity is not only affected by the resolution, that is to say, a high-resolution image does not necessarily look sharper than a low-resolution image , and it also depends on factors such as brightness .
★ Bit depth
Bit depth: When recording the color of a digital image, the computer actually expresses it in terms of the bit depth required for each pixel. For example, the red component uses 8bit.
The color pictures we see have three channels, namely red (R), green (G), and blue (B) channels. (and alpha component if transparency is needed)
Usually each channel is represented by 8bit, and 8bit can represent 2**8==256 colors, so 256*256*256=16,777,216=16.77 million colors can be formed. And the 8bit here is exactly the bit depth we want to talk about .
The greater the bit depth of each channel, the more color values it can represent. For example, the bit depth in high-end TVs is already 10bit color, that is, each channel is represented by 10bit, and each channel has 2** 10=1024 colors, and 1024*1024*1024 is about 1,073.74 million colors, which is 64 times as much as 8bit.
★ Frame rate
Frame rate: The number of frames of pictures transmitted in 1 second, which can also be understood as the image processor can refresh several times per second. For example: 25fps means that 25 pictures are refreshed in one second.
Frame rate is FPS, readers who often play games should be very familiar with this word, and we very much hope that the fps can be as high as possible, because this means that our game experience will be more smooth and comfortable, on the contrary, it is very stuck, not at all. Game experience. In the video, the same is true.
Due to the temporary stay of the visual image in the retina, the general video frame rate can reach 24 frames (fps=24), we think the video is smooth, continuous and dynamic.
Of course, the standards for fps compliance in each field are different, and they are listed below:
- The frame rate of the movie is generally: 24fps.
- TV series is generally: 25fps.
- Commonly used in surveillance industry: 25fps.
- Commonly used for audio and video calls: 15fps.
Similarly, although the higher the frame rate, the smoother the picture, but the higher the performance requirements of our device! Next, we will show how videos with low frame rates and those with higher frame rates play. :
Broken graphics card infer pedestrian detection model before TensorRT acceleration
Pedestrian detection video running on broken graphics card after TensorRT acceleration
The next two points are relatively simple, and I will briefly introduce them.
★ Bit rate
Bit rate: The data flow used by the video file in unit time. For example: 1Mbps
In most cases, for the same original image source and the same encoding algorithm: the higher the bit rate, the smaller the image distortion and the clearer the video picture.
However, the file size (bit rate) of the blurred video may also be large, and the video file with a small resolution may also be clearer than the video file with a large resolution. But it should be noted that the bit rate is not directly proportional, that is to say, a 1000Mbps video will not be ten times clearer than a 100Mbps video.
★ Stride
Sride: refers to the space occupied by each row of pixels in memory. In order to achieve memory alignment, the space occupied by each row of pixels in memory is not necessarily the width of the image.