-------------------Common image formats RAW, RGB, YUV
1、RAW
The raw data is the original data output by the sensor . Generally, there are raw8, raw10, raw12, etc., respectively indicating that a pixel has 8bit, 10bit, and 12bit data.
**It is the original record of the level when the sensor converts the optical signal into an electrical signal. **It is simply the image data without any processing, that is, the electrical signal directly obtained by the imaging element is obtained by digital processing.
The raw data is output in a certain order. There are four main types: GRBG, RGGB, BGGR, and GBRG. The following figure shows the BGGR format:
2、RGB
RGB format: that is, each pixel is composed of three primary colors R red, G green, and B blue. By mixing three colors, it is basically possible to mix all the colors that human vision can perceive.
Several common formats and descriptions of RGB:
-
Each pixel of RGB565 is represented by 16 bits, and each RGB component uses 5 bits, 6 bits, and 5 bits;
-
Each pixel of RGB555 is represented by 16 bits, and the RGB components use 5 bits (the remaining 1 bit is not used);
-
RGB24 Each pixel is represented by 24 bits, and each RGB component uses 8 bits;
-
RGB32 Each pixel is represented by 32 bits, and each RGB component uses 8 bits (the remaining 8 bits are not used);
-
Each pixel of ARGB32 is represented by 32 bits, and each RGB component uses 8 bits (the remaining 8 bits are used to represent the Alpha (transparency) channel value);
3. YUV
3.1. Definition of yuv:
is a color coding method adopted by the European television system. Among them, "Y" represents the brightness (Luma), which is the grayscale value, while "U" and "V" represent the chroma (Chroma).
Similar to the well-known RGB, YUV is also a color coding method; mainly used in television systems and analog video fields, it separates brightness information (Y) from color information (UV), and can display a complete image without UV information. It's just black and white,
Such a design well solves the compatibility problem between color TV and black and white TV. Moreover, YUV does not require three independent video signals to be transmitted simultaneously like RGB, so transmission in YUV mode takes up very little bandwidth.
Commonly used formats are: YUV444, YUV422, YUV420 (why is 4? Because this 4 actually expresses the largest unit of sharing! That is, up to 4 pixels are shared, so 4 is actually an implicit sampling set);
The most commonly used formats in cameras are YUV 422 format and YUYV format. Taking YUV422 8bit as an example, each pixel contains a brightness component (8bit) and one of the two UV color components (8bit).
Therefore, each pixel needs 16bit data. If you use YUV422 10bit, then each pixel needs 20bit to describe;
3.2. Sampling format:
From the aforementioned definition, it can be understood that the YUV space describes the pixel color of the RGB space by a "brightness" component and two "chroma" components. This coded representation is also more adaptable to the human eye. According to research, the human eye is more sensitive to brightness information than color information. The YUV down-sampling is to compress and sample the relatively insensitive color information of the human eye according to the characteristics of the human eye, and obtain a relatively small file for playback and transmission.
1) YUV 444
A family member is [yuv], each Y corresponds to a set of UV, and a YUV occupies 24bits 3 bytes;
2) YUV 422
A family member is [yu][yv], that is, two Ys share one UV, and one YUV occupies 2 sections of 8+4+4=16bits;
3) YUV 420
The meaning of yuv420 seems to be that on the basis of yuv422, two more v are removed, so there will be no v? In fact, the naming method of yuv420 is not very clever. The more accurate name is yuv420yuv402, that is, there are only two u in the first line, and only two v in the second line, as shown in the figure below;
For yuv420, the members of this family are shown in the figure below. Obviously, 4 Ys share a set of UVs, and each YUV occupies 8+2+2=12bits, 1.5 bytes;
4) The impact of different sampling formats of yuv on image quality
According to the aforementioned YUV sampling format analysis, here we analyze the impact on image quality. We take an original image as a red and blue pattern of 8*8 pixels, sample it according to different formats of YUV444, YUV422, and YUV420, and then restore the output.
Figure 5(a): We can see that there is no loss in the resolution of the chroma signal and the resolution of the luminance signal of YUV444, and we obtain a restored picture pattern that is consistent with the original pattern.
Figure 5(b): The restored pattern obtained by YUV422 has been lost in the horizontal direction. From the perspective of the green framed pixel, YUV422 has lost the color value of another pixel in the horizontal direction, so when the picture is restored It is just a simple copy reconstruction of the previous pixel value.
Figure 5©: The restored pattern obtained by YUV420 is lost in both the horizontal direction and the vertical direction, and the obtained restored image is greatly distorted from the original image.
From the intuitive observation shown in Figure 5, YUV444 is better than YUV422, and YUV422 is better than YUV420 in the image expression of high-frequency details of the image.
In terms of signal transmission bandwidth savings, YUV420 is more efficient than YUV444, and YUV422 is better than YUV444. Therefore, in common video codec algorithms, in order to save transmission bandwidth overhead, the sampling format of YUV420 or YUV422 is generally adopted.
3.3. Storage format:
- Planar planar format: refers to continuously store the Y component of all pixels, then store the U component, and finally the V component.
- packed Packing mode: It means that the Y, U, and V components of each pixel are stored continuously and alternately .
The following is a common YUV code stream storage method in the form of a diagram , and a method of sampling the YUV data of each pixel is attached after the storage method, where the meanings of Cb and Cr are equivalent to U and V.
1) Format based on YUV4:2:2 sampling
YUV 4:2:2 sampling stipulates that the Y and UV components are sampled at a ratio of 2: 1, and the two Y components share a set of UV components;
YUYV format
YUYV is stored in a packed format. Two adjacent Ys share their adjacent two Cb and Cr. Analysis shows that for pixel Y'00 and Y'01, the values of Cb and Cr are Cb00, Cr00, YUV values of other pixels and so on.
UYVY format
UYVY is also stored in a packed format, its order is opposite to YUYV, and the method of restoring the YUV value of each pixel is the same as above.
YUV422P format
YUV422P is also a kind of YUV422. It is a Plane mode, that is, a plane mode. It does not store YUV data interleaved, but stores all Y components first, then stores all U (Cb) components, and finally stores all V(Cr) component, as shown in the figure above.
The YUV value extraction method of each pixel is also the most basic extraction method following the YUV422 format, that is, two Ys share one UV. For example, for pixel points Y'00 and Y'01, the values of Cb and Cr are both Cb00 and Cr00.
2) Format based on YUV4:2:0 sampling
The formats based on YUV 4:2:0 sampling mainly include YUV 420P and YUV 420SP. Both YUV420P and YUV420SP are stored based on the Planar planar format. After storing all the Y components first ,
The YUV420P type will first store all U components or V components ,
And YUV420SP is stored according to the alternating sequence of UV or VU , see the following figure for details:
YUV420P ___ YU12
Also called I420 format under the android platform, first all Y values, then all U values, and finally all V values;
YUV420P___ YV12
The YV12 format is basically the same as YU12, first all Y values, then all V values, and finally all U values;
YUV420SP___NV21
The preview data collected by the android mobile phone from the camera is generally NV21 , the storage order is to store Y first, and then store VU alternately, and the storage order of NV21 is to store Y value first, and then store VU alternately;
YUV420SP ___ NV12
NV12 is similar to NV21, and also belongs to YUV420SP format. The storage order of NV12 is to store Y value first, and then store UV alternately;
The content is transferred from: https://zhuanlan.zhihu.com/p/538058910?utm_id=0
------------------- image format analysis, format conversion and picture viewing software
In the development of ISP's image algorithm, images in YUV, RAW and other formats are often involved.
For example, in the YUV domain, conversion between data formats such as I420, NV12, and P010 is often involved.
In the RAW domain, data viewing such as MIPI RAW is often involved.
Currently, YUV format analysis software includes open source YUView, and RAW format analysis software includes LibRaw, etc.
However, functions such as image conversion and picture viewing are often used in algorithm development, so it is more convenient to write a software by yourself.
According to usage habits, the software should be able to achieve the following functions:
- 1. First of all, it can analyze the formats commonly used in ISP, and can realize single and batch format conversion
- 2. Support drag and drop, convenient and quick view
- 3. It has a function similar to ImageView in Visual Studio, you can use the mouse wheel to move and zoom the picture, and view the specific pixel value
- 4. For RAW images, it has simple Black Level Subtraction, White Balance, Demosaic, Gamma and other functions to view color images
The content is transferred from: https://blog.csdn.net/FrankCai86/article/details/121549286