Table of contents
10. Common YUV format storage arrangement
1、YUV420P
YUV420P is also called plane plane mode, Y, U, V are in different planes, that is, there are three planes, it is YUV standard format 4:2:0, mainly divided into: YU12 and YV12
1.1、YU12 I420
The image examples on this wiki are very clear, the first image identifies the real pixel representation of the image. It can be seen that the four physically similar pixels Y1, Y2, Y7, and Y8 share the same U1 and V1, and the similar Y3, Y4, Y9, and Y10 use U2 and V2. The different colors here portray this feature very vividly, which is clear at a glance. Impressive! ! I have always felt that education, or basic education, must be explained in this way so that we can understand why. The kind of textbook that blindly copied a foreign textbook or combined several broken books is not to teach people, but to destroy people. Enough talking, let's get down to business. What I said before is the representation of physical pixels, and the distribution in memory is an array, and the order is YUV. So if you know the size of the pleasing to be expressed, it is easy to find the position of the specific YUV in the array.
1.2、YV12
The only difference from YU12 is that the V is in the front and the U is in the back. Just swap the 5th and 6th lines in the above picture.
2、YUV420SP
The image array in YUV420SP format, first of all Y values, and then UV or VU alternate storage, NV12 and NV21 belong to YUV420SP format, which is a two-plane mode, that is, Y and UV are divided into two planes, but UV (CbCr) For interleaved storage, instead of being divided into three planes like the YUV420 format. Mainly divided into: NV21 and NV12.
NV12 and YV12 are slightly different, the difference lies in the position of UV. Just look at the picture again and it will be clear!
2.1、NV12
The storage order is to store the Y value first, and then store the UV alternately: YYYYUVUVUV, taking a 4 × 4 picture as an example, the occupied memory is 4 × 4 × 3 / 2 = 24 bytes
In YV12, U and V are arranged continuously, while in NV12, U and V are arranged alternately. It can be seen that the layout in the memory is very clear. First, it is Y, and then it is a staggered arrangement of U1V1U2V2. The effect is the same for pixel compression. But it may be hardware accelerated support or not.
2.2、NV21
The storage order is to store the Y value first, and then store the VU alternately: YYYYVUVUVU, taking a 4 × 4 picture as an example, the occupied memory is 4 × 4 × 3 / 2 = 24 bytes
The only difference from NV12 is that V is in the front and U is in the back. Just swap the U and V in the 5th and 6th lines in the above picture.
3. YUV and RGB conversion
Y =(0.257* R)+(0.504* G)+(0.098* B)+16
U =(0.439* R)-(0.368* G)-(0.071* B)+128
V =-(0.148* R)-(0.291* G)+(0.439* B)+128
R =1.164*(Y -16)+2.018*(U -128)
G =1.164*(Y -16)+0.813*(V -128)-0.391*(U -128)
B =1.164*(Y -16)+1.596*(V -128)4. Separate YUV components
I420: YYYYYYYY UU VV =>YUV420P
YV12: YYYYYYYY VV UU =>YUV420P
NV12: YYYYYYYY UV UV =>YUV420SP
NV21: YYYYYYYY VU VU =>YUV420SP
YV12 ： 亮度（行×列） ＋ U（行×列/4) + V（行×列/4）
I420 ： 亮度（行×列） ＋ V（行×列/4) + U（行×列/4）5. YUV sampling
The chroma channel can be sampled at a lower rate than the luma channel without significant loss of perceived quality.
4:4:4 means full sampling.
4:2:2 means 2:1 horizontal sampling and vertical full sampling.
4:2:0 means 2:1 horizontal sampling and 2:1 vertical sampling.
4:1:1 means 4:1 horizontal sampling and vertical full sampling.
The most commonly used Y:UV recording ratio is usually 1:1 or 2:1. Video is recorded in YUV4:2:0, which is what we commonly call I420. YUV4:2:0 does not mean that there is only U (ie Cb) , V (that is, Cr) must be 0, but it means that U: V refer to each other, sometimes hidden, that is to say, for each row, there is only one U or V component. If a row is 4:2:0, the next One line is 4:0:2, the next line is 4:2:0...and so on. As for other common YUV formats, there are YUY2, YUYV, YVYU, UYVY, AYUV, Y41P, Y411, Y211, IF09, IYUV, YV12, YVU9, YUV411, YUV420, etc.
YUV 4:4:4
Fully sampled, each pixel has independent Y/U/V values
For the convenience of comparison, we selected 6x5 pixels of the horizontal gradient color block at the bottom of the screen and enlarged it by 40 times to see the YUV value of the pixel (other samples also take the same position).
As shown in the figure below, it can be seen that the Y/U/V values of adjacent pixels are not equal. It can also be seen that each pixel has an independent U/V value.
Hebrews 4:2:2
Every two Y share a pair of U/V values
422 samples, every two pixels in the horizontal direction share a pair of U/V values, as shown in the figure below, a box marked by a green box (U channel) and a blue box (V channel) has only one value in memory.
YUV 4:2:0
Every four Y share a pair of U/V values
420 sampling, because 4 adjacent pixels share a pair of U/V, so it will appear as shown in the figure below, and 4 identical UV values will appear in every 4 adjacent pixels.
YUV 4:1:1
Similar to YUV 4:2:0, all four Ys share a pair of U/V values. The difference is that 4:1:1 is that 4 Ys in the horizontal direction share a pair of U/V values, as shown in the figure below , the 1st-4th Y uses the same pair of U/V values, the 5th-8th Y uses the same pair of U/V values, and so on. . .
Here is no screenshot example, because 411 sampling is not commonly used so that general tools do not support playback, there are actually more examples with 422, 411 is equivalent to downsampling in the horizontal direction on the basis of 422, That is, every 4 consecutive pixels in the horizontal direction share a pair of UVs.
6、Planar Format
planar format
Packed (or Interleaved)
Put the prime values of the three YUV components in the same array and store them as a single array macropixel
Continuously interleave Y/U/V of each pixel
Taking UYVY and YUYV as an example, a macro pixel is composed of two pixels of Y/U/V, arranged in sequence
UYVY
U0Y0V0Y1 U2Y2V2Y3 U4Y4V4Y5 U6Y6V6Y7 …
YUYV
Y0U0Y1V0 Y2U2Y3V2 Y4U4Y5V4 Y6U6Y7V6 …
Planar
Each component is stored as a separate array and the final image is the integration of three separate planes
Taking I420 as an example, the Y/U/V arrangement is as follows:
width: width of image resolution height: resolution height n = width * height m = width * height / 4 Y0Y1Y2Y3Y4Y5…Yn U0U1U2U3…Um V0V1V2V3…Vm
Semi-Planar
Between Pakced and Planar, that is, the Y component is an array, and UV cross storage
NV12
width: width of image resolution height: resolution height n = width * height m = width * height / 4 Y0Y1Y2Y3Y4Y5…Yn U0V0U1V1U2V2U3V3…UmVm
Note: The name of the format ending with "p" is generally planar format, such as the yuvj420p format in ffmpeg, the name of the format ending with "sp" is generally semi-planar format
7. Relationship diagram
8. Calculate
Since the YUV format is uncompressed and only stores the data of the three channels of Y/U/V, playback or encoding must know the resolution and the specific format of YUV (such as nv12/i420/yuvj420ple), so when naming the YUV file The suggested name contains the YUV format name and resolution information
Here, the sample rate and bit depth in calculating the file size or calculating the number of frames can be obtained from the YUV format
YUV calculation file size
Known YUV sampling rate, resolution, and frame number
File size = width * height * sampling rate * round up (bit depth/8) * number of frames
file size(byte) = width * height * rate * Ceil(bitdepth/8) * frames
Take the most commonly used 420 with 8bit bit depth such as nv12/i420 as an example
File size = width * height * 1.5 * number of frames
YUV calculation frame number
The calculation of YUV frame number is similar to the calculation of file size
Requires known file size, sample rate, resolution
Number of YUV frames = file size/(width * height * sampling rate * rounded up (bit depth/8))
YUV frames = file size(byte) / (width * height * rate * Ceil(bitdepth/8))
sample map
| sample |
rate |
| 4:4:4 |
3 |
| 4:2:2 |
2 |
| 4:2:0 |
1.5 |
| 4:1:1 |
1.5 |
Big and small endian (byte order)
When the bit depth exceeds 8bit, there are big and small endian modes
big-edian (big-endian): the low address stores the high bits
little-endian (little-endian): the low address stores the low bits
The format name uses le as the little-endian sequence, and the format name uses be as the little-endian
sequence The YUV arrangement of the two formats is exactly the same, for example, yuv420p10le and yuv420p10be
9. Some common formats
YUV format relationship
There are many YUV format names, and different names in different places may represent the same format, such as yuyv422 in ffmpeg and yuy2 in gstreamer, and other format names YUY2/YUYV/YUNV, all represent the same format A YUV format.
| ffmpeg pix_fmt |
gstreamer format |
Duplicate formats |
number of components |
Bits per pixel |
remark |
| yuvj420p |
i420 |
IYUV/YU12 |
3 |
12 |
full range |
| yuv420p |
- |
3 |
12 |
limited range, the arrangement is exactly the same as yuvj420p, but the pixel color range is [16,235], 16 means black, 235 means white; UV range [16,240] |
|
| nv12 |
nv12 |
3 |
12 |
- |
|
| nv21 |
nv21 |
3 |
12 |
- |
|
| yuyv422 |
yuy2 |
YUY2/YUYV/YUNV |
3 |
16 |
YUYV 4:2:2 |
| uyvy422 |
uyvy |
UYVY/IUYV/HDYC/ UYNV/Y422 |
3 |
16 |
UYVY 4:2:2 |
| yuv420p10le |
i420-10le |
3 |
15 |
10 bit yuv |
|
| p010le |
p010-10le |
3 |
15 |
10 bit yuv |
|
| yv12 |
- |
3 |
12 |
YV12 (3-planar) ——These formats are identical to YU12 except that the U and V plane order is reversed. |
10. Common YUV format storage arrangement
Define:
s = width * height
n = width * height // 4
m = width * height // 2
I420/IYUV/YU12: yuv420 planar 的一种， 排列方式： y1y2y3…ysu1u2u3…unv1v2v3…vn
nv12: 8bit YUV420 Semi-Planar的一种格式, 排列方式： y1y2y3…ysu1v1u2v2u3v3…unvn
nv21: 8bit YUV420 Semi-Planar的一种格式, 排列方式： y1y2y3…ysv1u1v2u2…vnun
i420-10le： 10bit YUV420 Planar的一种格式, 排列方式与i420一样, 只是每个值都用10bit低字节序来存储
p010-10le： 10bit YUV420 Semi-Planar的一种格式, 排列方式与nv12一样, 只是每个值都用高10bit低字节序来存储
YUY2/YUYV/YUNV: 8bit YUV 4:2:2格式， 排列方式: y1u1y2v1 y3u2y4v2 y5u3y6v3…yn-1umynvm
UYVY/IUYV/HDYC/UYNV/Y422: 8bit YUV 4:2:2格式， 排列方式: u1y1v1Y2 u2y3v2y4 u3y5v3y6…umyn-1vmyn
yv12: YUV420 Planar的一种，UV的顺序与i420相反， 排列方式： y1y2y3y…ynv1v2v3…vnu1u2u3…un
YUV422P/I422: y1y2y3…ysu1u2u3…umv1v2v3…vm
nv16: yuv422 Semi-Planar的一种： y1y2y3…ys u1v1u2v2u3v3…umvm
nv61: 与nv16类似，只是u/v的位置互换（V前U后）
yv16: yuv422 Planar的一种： y1y2y3…ysu1u2u3…umv1v2v3…vm
yuvj444p: Planar的一种： y1y2y3y…ys u1u2u3…us v1v2v3…vsOther 4:1:1 sampling formats are generally less common
Y41P: 是一种4:1:1packed格式，其中U和V每四个像素水平采样一次。每个宏像素在三个字节中包含8个像素，并具有以下字节布局：U0 Y0 V0 Y1 U4 Y2 V4 Y3 Y4 Y5 Y6 Y7
Y41T与Y41P相同，除了每个Y样本的最低有效位指定色度键（0 =透明，1 =不透明）。
AI44是一种已标准化的YUV格式，每个样本8位。每个样本在4个最高有效位（MSB）中包含一个索引，在4个最低有效位（LSB）中包含一个alpha值。索引引用YUV调色板条目的数组，必须在该格式的媒体类型中定义。此格式主要用于子图片图像。