YUV data storage order
In the previous section, Data Format YUV Detailed Explanation, we focused on how the YUV format reduces the amount of data through subsampling when generating data , and how to combine the Y+UV values of the pixel based on the UV values of neighbors when applying data .
The order of data arrangement is also particularly important. For example, in YUV444 sampling mode, the value of pixel 1 can be combined into Y1+U1+V1, or Y1+V1+U1.
There are mainly three types of data combinations, taking YUV444 sampling as an example:
- Packed (or interleaved) means that the three data Y, U, and V are stored in interleaved order. Such as Y1+U1+V1+Y2+U2+V2+Y3+U3+V3.
- Planar (or fully-planar, usually ending with "p"), means that Y, U, and V data are stored in separate groups, such as Y1+Y2+Y3+U1+U2+U3+V1+V2+V3.
- Semi-planar (usually ending in "sp") means that Y is stored in a group and U+V data is stored in a group, for example, Y1+U1V1+Y2+U2V2+Y3+U3V3. Different storage orders lead
to The procedures for image processing are different, and the utilization of storage space and data processing efficiency are also different. There is no unified standard here, you can only try to adapt based on your experience. The definitions here in the Linux kernel can be used as a reference, but there are too many definitions and are too complicated to explain in detail.
When the order of U and V also becomes important
- Starting with NV is a Semi-Planar YUV format. NV12, that is, 12bits/pixel, is a type of YUV420, that is, 1 is in front, which means U first and then V, NV21 is 1 in the back, which means V is first and then U. Similar to this is NV16, which is 16bits/pixel, which is YUV422; NV24, which is 24bits/pixel, which is YUV444.
- YU is a Semi-planar format, that is, U comes first in the UV component.
- YV is a Semi-planar format, that is, in the UV component, V comes first.
Example: YUV420p:
Combined with the above definition of YUV420, for the Planar order, the storage format of Y, U, and V corresponding to its 24 pixels (only displaying some pixels) can be (blocks of the same color share UV vectors): In addition to the more
formal In addition to the definitions, some SDKs also give these formats aliases, such as YUV420p=I420 = IYUV = YV12, which is not surprising. These are not particularly formal definitions that are not listed here. If you encounter relevant definitions, please feel free to discuss them with me.
Several common output formats in projects
RAW, RGB, YUV, and Only Y are common organizational standards for image color data. In addition, in the video field, there are also ITU-R BT.601 and ITU-R BT.656 formats, which are standards developed by the Wireless Communications Sector (ITU-R) of the International Telecommunication Union and are a transmission interface standard. , in addition to specifying the formula for RGB-YUV conversion, it also defines a transmission protocol for the YUV format. Adding some frame headers and frame trailers to the YUV format data can be regarded as a professional video stream format definition.
For these data formats, common usage scenarios are:
- Screen projects, such as screens running LVGL, prefer RGB format data.
- For video streaming projects, data in YUV format is preferred.
- Image processing projects prefer RGB888 type data. Note that OpenCV uses BGR888 format data due to historical issues. In addition, some image recognition projects only focus on the outline of graphics, so they like Only Y, which is grayscale image data.
- Real-time games have limited bandwidth, so data in YUV420 format with smaller data volume is mainstream.
- AI-type projects are usually combined with images and videos. Image-type projects have both RGB and YUV, and projects combined with video-type projects are more in the YUV domain.
Summarize
1) The Camera data format article systematically introduces the definitions of RAW, YUV, RGB, and ARGB formats through the data format RAW, RGB (1) , the detailed explanation of the data format YUV (2) , and the content of this article.
2) This section follows the introduction of the YUV format in the previous section , and outlines that YUV mainly has three data storage methods: Packed, Planar, and Semi-planar. Different methods store the Y, U, and V components in different orders.
3) Some YUV formats named after bit numbers, such as NV16 and NV24, are actually a type of YUV422 and YUV444.