In terms of video display, I can't hide the relevant knowledge of YUV and RGB formats. This time I found a good documentation on the conversion of YUV and RGB formats.
The YUV format has the characteristics of separation of brightness information and color information, but most image processing operations are based on the RGB format.
Therefore, when the image is to be post-processed and displayed, it is necessary to convert the YUV format to the RGB format.
The conversion formula of RGB and YUV is as follows:
YUV (256 levels) can be calculated directly from 8-bit RGB:
Y = 0.299 R + 0.587 G + 0.114 B
U = - 0.1687 R - 0.3313 G + 0.5 B + 128
V = 0.5 R - 0.4187 G - 0.0813 B + 128
Conversely, RGB can also be calculated directly from YUV (256 levels):
R = Y + 1.402 (Cr-128)
G = Y - 0.34414 (Cb-128) - 0.71414 (Cr-128)
B = Y + 1.772 (Cb-128)
There are many YUV formats. The following takes the YV12 format as an example to illustrate the method of converting YV12 format into RGB24 format.
The basic idea is to perform pixel-by-pixel calculations according to the transformation formula of RGB and YUV, but in the actual implementation process, optimization methods and techniques affect the final conversion efficiency.
Note: In order to view the converted results conveniently, the BGR24 format is used instead of the RGB24 format during the implementation process, and the conversion process remains unchanged.
1. Basic implementation
According to the transformation formula of YUV and RGB, the values of Y, U, and V components are accessed pixel by pixel and converted to RGB.
bool YV12ToBGR24_Native(unsigned char* pYUV,unsigned char* pBGR24,int width,int height)
{
if (width < 1 || height < 1 || pYUV == NULL || pBGR24 == NULL)
return false;
const long len = width * height;
unsigned char* yData = pYUV;
unsigned char* vData = &yData[len];
unsigned char* uData = &vData[len >> 2];
int bgr[3];
int yIdx,uIdx,vIdx,idx;
for (int i = 0;i < height;i++){
for (int j = 0;j < width;j++){
yIdx = i * width + j;
vIdx = (i/2) * (width/2) + (j/2);
uIdx = vIdx;
bgr[0] = (int)(yData[yIdx] + 1.732446 * (uData[vIdx] - 128)); // b分量
bgr[1] = (int)(yData[yIdx] - 0.698001 * (uData[uIdx] - 128) - 0.703125 * (vData[vIdx] - 128)); // g分量
bgr[2] = (int)(yData[yIdx] + 1.370705 * (vData[uIdx] - 128)); // r分量
for (int k = 0;k < 3;k++){
idx = (i * width + j) * 3 + k;
if(bgr[k] >= 0 && bgr[k] <= 255)
pBGR24[idx] = bgr[k];
else
pBGR24[idx] = (bgr[k] < 0)?0:255;
}
}
}
return true;
}2. Implementation based on OpenCV
Use the conversion function provided by OpenCV to realize the conversion from YUV to RGB, which is simple and convenient. In the implementation process, it is only necessary to reasonably construct a Mat containing YUV data . The specific implementation method is as follows.
bool YV12ToBGR24_OpenCV(unsigned char* pYUV,unsigned char* pBGR24,int width,int height)
{
if (width < 1 || height < 1 || pYUV == NULL || pBGR24 == NULL)
return false;
Mat dst(height,width,CV_8UC3,pBGR24);
Mat src(height + height/2,width,CV_8UC1,pYUV);
cvtColor(src,dst,CV_YUV2BGR_YV12);
return true;
}3. Implementation based on FFmpeg
Use swscale in FFmpeg to realize the conversion from YUV to RGB. During the implementation process, you need to construct the AVPicture structure . The specific implementation method is as follows.
bool YV12ToBGR24_FFmpeg(unsigned char* pYUV,unsigned char* pBGR24,int width,int height)
{
if (width < 1 || height < 1 || pYUV == NULL || pBGR24 == NULL)
return false;
//int srcNumBytes,dstNumBytes;
//uint8_t *pSrc,*pDst;
AVPicture pFrameYUV,pFrameBGR;
//pFrameYUV = avpicture_alloc();
//srcNumBytes = avpicture_get_size(PIX_FMT_YUV420P,width,height);
//pSrc = (uint8_t *)malloc(sizeof(uint8_t) * srcNumBytes);
avpicture_fill(&pFrameYUV,pYUV,PIX_FMT_YUV420P,width,height);
//U,V互换
uint8_t * ptmp=pFrameYUV.data[1];
pFrameYUV.data[1]=pFrameYUV.data[2];
pFrameYUV.data [2]=ptmp;
//pFrameBGR = avcodec_alloc_frame();
//dstNumBytes = avpicture_get_size(PIX_FMT_BGR24,width,height);
//pDst = (uint8_t *)malloc(sizeof(uint8_t) * dstNumBytes);
avpicture_fill(&pFrameBGR,pBGR24,PIX_FMT_BGR24,width,height);
struct SwsContext* imgCtx = NULL;
imgCtx = sws_getContext(width,height,PIX_FMT_YUV420P,width,height,PIX_FMT_BGR24,SWS_BILINEAR,0,0,0);
if (imgCtx != NULL){
sws_scale(imgCtx,pFrameYUV.data,pFrameYUV.linesize,0,height,pFrameBGR.data,pFrameBGR.linesize);
if(imgCtx){
sws_freeContext(imgCtx);
imgCtx = NULL;
}
return true;
}
else{
sws_freeContext(imgCtx);
imgCtx = NULL;
return false;
}
}4. Implementation based on Pinknoise
Reference: http://wss.co.uk/pinknoise/yuv2rgb/
Download the yuv2rgb code provided by the above website, and use the yuv420_2_rgb888 function to realize the conversion from YUV to RGB.
bool YV12ToBGR24_Pinknoise(unsigned char* pYUV,unsigned char* pBGR24,int width,int height)
{
if (width < 1 || height < 1 || pYUV == NULL || pBGR24 == NULL)
return false;
unsigned char *yData = pYUV;
unsigned char *vData = &pYUV[width * height];
unsigned char *uData = &vData[width * height >> 2];
yuv420_2_rgb888(pBGR24,yData,uData,vData,width,height,width,width>>1,width*3,yuv2rgb565_table,0);
return true;
}Conversion Efficiency Analysis
Test sequence: 1920*1080
Test environment: OpenCV2.4.8, FFmpeg2.0, YUV2RGB v0.03
| Method |
Time(ms) |
| YV12ToBGR24_Native |
83.7263 |
| YV12ToBGR24_Table |
54.2376 |
| YV12ToBGR24_OpenCV |
26.0529 |
| YV12ToBGR24_FFmpeg |
3.41499 |
| YV12ToBGR24_Pinknoise |
14.1215 |
It can be seen from the above table that the format conversion efficiency based on FFmpeg is the highest.