Hass Multimedia (MPP) Development (5) - Regional Management (REGION & OSD character display)

(A) understand the basic concepts

(1) Overlay overlay

    Video overlay area, loading area where the support bitmap update etc. background color, transparency can be provided that is simple to understand, the following Alpha value is

(2) Cover occlusion

    Video shelter area, an area where support pure color blocking, and Overlay overlay different is that it can not load images, you can not set transparency

(3) Alpha Channel

 If the 32-bit bus having a graphics card, additional 8-bit signal is used to house a transparency signal is not visible to facilitate processing, this is the Alpha channel. Alpha white opaque pixels to define the color pixel, while the black pixels to define alpha transparent pixel gray scale between black and white pixels to define semi-transparent.

•     When VPSS OVERLAY, Alpha in the range [0, 255]. The smaller the value, the more transparent.
•     When VPSS VENC, the Alpha in the range [0, 127]. The smaller the value, the more transparent.

(4) Stride image span 

Image Stride (memory image lines span) when the video images are stored in the memory, the end of the image of each line may contain a number of extended content, these extended content affects only how the image is stored in memory, but does not affect how the image is displayed; Stride is the name of the content of these extensions, also referred to Stride Pitch, if the end of each line of pixels of the image content has expanded, the value must be greater than the width of the Stride value of the image, as shown below:

Two video frame buffers contain the same size (width and height), Stride not necessarily have the same value, if you deal with a video frame, when you have the Stride calculation into account;

In doing OSD watermark when, stride value superimposed image is larger than the width of the region of the canvas, the image is added to the canvas fail.

(B) pixel format:

• OVERLAY VENC types supported: Argb1555, Argb4444
• OVERLAY VPSS types supported: Argb1555, Argb4444, Argb8888

ARGB --- Alpha, Red, Green, Blue. One color model, RGB color model is attached on Alpha (transparency) channel, 32-bit storage structure is common in the bitmap.

(A) the type of format

ALPHA_8: number is 8, the graphics parameters should be represented by one byte should be an 8-bit bitmap, the color format common:

•     ARGB_4444: 4 + 4 + 4 + 4 = 16, the pattern parameter should be represented by two bytes, should be a 16-bit bitmap.
•     ARGB_8888: 8 + 8 + 8 + 8 = 32, the pattern parameter should be represented by four bytes, should be a 32-bit bitmap.
•     RGB_565: 5 + 6 + 5 = 16, the pattern parameter should be represented by two bytes, should be a 16-bit bitmap.

    Argb1555 15 represents transparency is used and 5 represent respectively R, G, B, to form a 32-bit bitmap, wherein the two bits are not used. (b) color format:
    color table can be viewed: https:? //tool.oschina.net/commons type = 3
    is also a RGB color, but the color format there are many, so the color look-up table with a color display It is not corresponding to the required format conversion or direct correspondence to find.

(C) a color conversion

Three kinds of RGB format representation:

• RGB555: R-5bit，G-5bit，B-5bit
• RGB565: R-5bit，G-6bit，B-5bit
• RGB888: R-8bit，G-8bit，B-8bit

转RGB555 RGB888
RGB888: R7 R6 R5 R4 R3 R2 R1 R0 G7 G6 G5 G4 G3 G2 G1 G0 B7 B6 B5 B4 B3 B2 B1 B0
RGB555: 0  R7 R6 R5 R4 R3 G7 G6 G5 G4 G3 B7 B6 B5 B4 B3

Other formats similar thereto.

(C) real-time image refresh OSD

    Hass official in the region sample provided, they use ready-made picture, which is ready to take pictures directly loaded into the video stream to go, so there is a problem is if I need to change the content in real time the OSD, this is not good deal. For example, add time watermark in the video.

    Time watermark, for example, will add time to achieve a watermark to the video stream to stream, only two major processes:

• Generating an image of the time band
• The time image is loaded into the preset region in the canvas to

(1) Bitmap generation time:

   It should be use to freetype, SDL, SDl_ttf three libraries.

• FreeType2 is a simple cross-platform font rendering engine
• SDL (the Simple DirectMedia Layer) is a cross-platform multimedia development library open source, written in C language. SDL provides several control images, sounds, input and output functions, allowing developers to use as long as the same or similar code can be developed across multiple platforms (Linux, Windows, Mac OS X, etc.) applications.
• SDL_ttf TrueType font rendering library, using SDL library almost as portable. It depends FreeType2 processing TrueType font data. It allows programmers to use TrueType fonts without having to code multiple font rendering program itself. With outline fonts and anti-aliasing power, high-quality text output can effortlessly get.

    Need these three libraries ported to go Hass device, cross-compiling migration process is not described here, there are many online introduction.
    Here is a simple test program:

/************************************************************
*Copyright (C),lcb0281at163.com lcb0281atgmail.com
*BlogAddr: caibiao-lee.blog.csdn.net
*FileName: debug_font_osd.c
*Description:测试生成带时间字符的图像
*Date:     2020-02-03
*Author:   Caibiao Lee
*Version:  V1.0
*Others:
*History:
***********************************************************/
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "SDL/SDL.h"
#include "SDL/SDL_ttf.h"

#define FONT_PATH       "./font/hisi_osd.ttf"

int string_to_bmp(char *pu8Str)
{
    SDL_PixelFormat *fmt;
    TTF_Font *font;  
    SDL_Surface *text, *temp;  

    if (TTF_Init() < 0 ) 
    {  
        fprintf(stderr, "Couldn't initialize TTF: %s\n",SDL_GetError());  
        SDL_Quit();
    }  

    font = TTF_OpenFont(FONT_PATH, 80); 
    if ( font == NULL ) 
    {  
        fprintf(stderr, "Couldn't load %d pt font from %s: %s\n",18,"ptsize", SDL_GetError());  
    }  

    SDL_Color forecol = { 0xff, 0xff, 0xff, 0xff };  
    text = TTF_RenderUTF8_Solid(font, pu8Str, forecol);

    fmt = (SDL_PixelFormat*)malloc(sizeof(SDL_PixelFormat));
    memset(fmt,0,sizeof(SDL_PixelFormat));
    fmt->BitsPerPixel = 16;
    fmt->BytesPerPixel = 2;
    fmt->colorkey = 0xffffffff;
    fmt->alpha = 0xff;

    temp = SDL_ConvertSurface(text,fmt,0);
    SDL_SaveBMP(temp, "save.bmp"); 

    SDL_FreeSurface(text);  
    SDL_FreeSurface(temp);
    TTF_CloseFont(font);  
    TTF_Quit();  

    return 0;
}

These projects include the following elements:

biao@ubuntu:~/test/github/hisi_sdk_develop/freetype_SDL_Dl_ttf_debug$ tree -L 2
.
├── bin
│   └── objs
├── debug_font_osd.c
├── debug_font_osd.h
├── font
│   ├── hisi_osd.ttf
│   └── hisi_osd.ttf_df
├── inc
│   ├── freetype2
│   ├── ft2build.h
│   └── SDL
├── lib
│   ├── libfreetype.a
│   ├── libfreetype.so
│   ├── libfreetype.so.6
│   ├── libSDL-1.2.so.0
│   ├── libSDL.a
│   ├── libSDLmain.a
│   ├── libSDL.so
│   ├── libSDL_ttf-2.0.so.0
│   ├── libSDL_ttf.a
│   ├── libSDL_ttf.so
│   └── pkgconfig
├── Makefile
├── save.bmp
└── test

8 directories, 18 files
biao@ubuntu:~/test/github/hisi_sdk_develop/freetype_SDL_Dl_ttf_debug$ 

Save.bmp saved image generated as follows:

(2) to add a watermark to the video stream of characters:

    According to official here is modified from the sample, the main processes are:

• The decoder and encoder bound, channel bonding area of ​​the channel encoding, data stream read from the file h264, input into the decoder, the decoder flows from the encoder, the encoder produces finally stored into the data file .

    Watermark regions with the image after encoding, there may be provided VENC VDEC and the actual resolution.

/************************************************* 
Function:    BIAO_RGN_AddOsdToVenc  
Description: 将视频文件添加时间水印
Input:  none
OutPut: none
Return: 0: success，none 0:error
Others: 解码器输入的分辨率与编码器的输出分辨率可以不相同，
    比如将1080P图像解码后，可以再编码成720P图像。
Author: Caibiao Lee
Date:   2020-03-08
*************************************************/
HI_S32 BIAO_RGN_AddOsdToVenc(HI_VOID)
{
    HI_S32 s32Ret = HI_SUCCESS;
    RGN_HANDLE OverlayHandle;
    HI_S32 u32OverlayRgnNum;
    MPP_CHN_S stSrcChn, stDesChn;
    RGN_ATTR_S stRgnAttrSet;
    RGN_CANVAS_INFO_S stCanvasInfo;
    BITMAP_S stBitmap;
    VENC_CHN VencChn;
    VDEC_CHN VdecChn;
    VDEC_SENDPARAM_S stVdesSendPram;
    VENC_PTHREAD_INFO_S stVencGetPram;
    SIZE_S stSize;
    FILE * pastream = NULL;
    HI_U32 i;
    int l_s32CanvasHandle = 0;

    /**分配缓存**/
    s32Ret = BIAO_RGN_SYS_Init(); 
    if(HI_SUCCESS != s32Ret)
    {
        printf("SAMPLE_RGN_SYS_Init failed! s32Ret: 0x%x.\n", s32Ret);
        goto END_O_VENC0;
    }
    
    /**创建区域，并将它添加到编码通道**/
    OverlayHandle    = 0;
    u32OverlayRgnNum = 1;
    s32Ret = BIAO_RGN_CreateOverlayForVenc(OverlayHandle, u32OverlayRgnNum);
    if(HI_SUCCESS != s32Ret)
    {
        printf("SAMPLE_RGN_CreateOverlayForVenc failed! s32Ret: 0x%x.\n", s32Ret);
        goto END_O_VENC1;
    }
    
    /**开启解码通道**/
    VdecChn = 0;
    s32Ret = BIAO_RGN_StartVdec(VdecChn);
    if(HI_SUCCESS != s32Ret)
    {
        printf("SAMPLE_RGN_StartVdec failed! s32Ret: 0x%x.\n", s32Ret);
        goto END_O_VENC2;
    }
    
    /**开启编码通道**/
    VencChn = 0;
    s32Ret = BIAO_RGN_StartVenc(VencChn);
    if(HI_SUCCESS != s32Ret)
    {
        printf("SAMPLE_RGN_StartVenc failed! s32Ret: 0x%x.\n", s32Ret);
        goto END_O_VENC3;
    }
    
    /**将解码通道绑定到编码通道**/
    stSrcChn.enModId  = HI_ID_VDEC;
    stSrcChn.s32DevId = 0;
    stSrcChn.s32ChnId = 0;

    stDesChn.enModId  = HI_ID_VENC;
    stDesChn.s32DevId = 0;
    stDesChn.s32ChnId = 0;

    s32Ret = HI_MPI_SYS_Bind(&stSrcChn, &stDesChn);
    if(HI_SUCCESS != s32Ret)
    {
        printf("HI_MPI_SYS_Bind failed! s32Ret: 0x%x.\n", s32Ret);
        goto END_O_VENC4;
    }

    /**创建一个线程用来从h264文件中读取数据，模拟h264数据流**/   
    stSize.u32Width  = DECODE_VIDEO_W;
    stSize.u32Height = DECODE_VIDEO_H;
    
    stVdesSendPram.bRun          = HI_TRUE;
    stVdesSendPram.VdChn         = VdecChn;
    stVdesSendPram.enPayload     = PT_H264;
    stVdesSendPram.enVideoMode   = VIDEO_MODE_FRAME;
    stVdesSendPram.s32MinBufSize = stSize.u32Height * stSize.u32Width / 2;
    pthread_create(&g_stVdecThread, NULL, BIAO_RGN_VdecSendStream, (HI_VOID*)&stVdesSendPram);


    /**更新OSD内容**/
    l_s32CanvasHandle = 0;
    pthread_create(&g_stRgnOsdThread, NULL, BIAO_UpdateCanvas, (HI_VOID*)&l_s32CanvasHandle);


    /**创建一个线程，将编码器输出的数据存成文件**/
    char pfilename[64]; 
    sprintf(pfilename, ENCODE_H264_FILE);
    pastream = fopen(pfilename, "wb");  
    HI_ASSERT( NULL != pastream);

    stVencGetPram.pstream   = pastream;
    stVencGetPram.VeChnId   = VencChn;
    stVencGetPram.s32FrmCnt = 0;
    pthread_create(&g_stVencThread, 0, BIAO_RGN_VencGetStream, (HI_VOID *)&stVencGetPram);

    printf("\n#############Sample start ok! Press Enter to switch!#############\n");

    
    /*************************************************
    step 8: stop thread and release all the resource
    *************************************************/

    /**延时之后推出编解码**/
    sleep(10);
    bExit = HI_TRUE;
    pthread_join(g_stVdecThread, 0);

    pthread_join(g_stVencThread, 0);

    pthread_join(g_stRgnOsdThread, 0);
    
    bExit = HI_FALSE;
    
END_O_VENC4:
    HI_MPI_SYS_UnBind(&stSrcChn, &stDesChn);

END_O_VENC3:
    BIAO_RGN_StopVenc(VencChn);
    
END_O_VENC2:
    BIAO_RGN_StopVdec(VdecChn);

END_O_VENC1:    
    BIAO_RGN_DestroyRegion(OverlayHandle, u32OverlayRgnNum);       

END_O_VENC0:
    SAMPLE_COMM_SYS_Exit();
    
    return s32Ret;
}

The Hass official video after video effects to add a watermark as follows:

The first generation of engineering time image can be obtained from the following:

GitHub： freetype_SDL_Dl_ttf_debug 

CSDN :  freetype_SDL_Dl_ttf_debug.tar.gz

The second watermark can be superimposed on the video test project from "catalog preface," the address provided to acquire

 

The first article in this column "catalog preface," lists the complete directory column, read by directory order to help your understanding.