ArcSoft recognition 3.0 - Introduction image data structure (C ++)

Enterprise 2019-12-13 09:00:33 views: null

Open the 2.0 version of the SDK from ArcSoft since, because of free, off-line use of the characteristics of our company in the face recognition access control applications, ArcSoft SDK, recognition results were pretty good, so dynamic more concerned about ArcSoft official SDK. Recently on the line ArcFace 3.0 SDK version, really done a relatively large update. Previous introduces improvements on the Android platform algorithm herein, this will tell us about the update on the Windows platform algorithm.

  • Feature matching support than for model selection, there are photos of life than on the model and witnesses than the model

  • Recognition rate, significantly improved effect of preventing ***

  • Eigenvalues ​​update, upgrade later to be re-registered face database

  • Face detection supports both single-wide angle and angle
  • A new image data into the way

In version V3.0 access process, we found the new image data structure still has a certain degree of difficulty, this paper describes the following points from the image data structure and use of the

  1. SDK Interface changes

  2. Image data structure

  3. Step size effect

  4. OpenCV image data structure is converted into image data structure ArcSoft

A, SDK interfaces change

When the access ArcFace 3.0 SDK, found added ASFDetectFacesEx, ASFFaceFeatureExtractEx, ASFProcessEx, ASFProcessEx_IR a set of interfaces, the interface using the set of LPASF_ImageDatapointers to structures way incoming image data to the face detection interface as an example, the following specific interfaces than :

Raw interface:

MRESULT ASFDetectFaces(
        MHandle             hEngine,                            // [in] 引擎handle
        MInt32              width,                              // [in] 图片宽度
        MInt32              height,                             // [in] 图片高度
        MInt32              format,                             // [in] 颜色空间格式
        MUInt8*             imgData,                            // [in] 图片数据
        LPASF_MultiFaceInfo detectedFaces,                      // [out]检测到的人脸信息 
        ASF_DetectModel     detectModel = ASF_DETECT_MODEL_RGB  // [in] 预留字段，当前版本使用默认参数即可
        );

New Interface:

MRESULT ASFDetectFacesEx(
        MHandle             hEngine,                            // [in] 引擎handle
        LPASF_ImageData     imgData,                            // [in] 图片数据
        LPASF_MultiFaceInfo detectedFaces,                      // [out] 检测到的人脸信息
        ASF_DetectModel     detectModel = ASF_DETECT_MODEL_RGB  // [in] 预留字段，当前版本使用默认参数即可
        );

With respect to the original interface, a new incoming interface via LPASF_ImageDataalternative embodiment of the original incoming image data interface image data structure pointer.

Second, the image data structure

The new image data structure introduced stepsize pi32Pitchconcept.

Defined steps: the number of bytes of image alignment line.

2.1 ArcSoft image data structure

The image structure definition:

typedef LPASVLOFFSCREEN LPASF_ImageData;

typedef struct __tag_ASVL_OFFSCREEN
{
    MUInt32 u32PixelArrayFormat;
    MInt32  i32Width;
    MInt32  i32Height;
    MUInt8* ppu8Plane[4];
    MInt32  pi32Pitch[4];
}ASVLOFFSCREEN, *LPASVLOFFSCREEN;

ArcSoft official introduction to the document image data structure:

Types of variable name description
MUInt32 u32PixelArrayFormat Color format
MInt32 i32Width Image width
MInt32 i32Height Image height
MUInt8* ppu8Plane Image data
MInt32 pi32Pitch Image step

2.2 OpenCV image data structure

OpenCV provided IplImageand Mattwo kinds of commonly used data structures in the image.

IplImage image data structure

typedef struct _IplImage
{
    int  width;             /* Image width in pixels.                           */
    int  height;            /* Image height in pixels.                          */
    char *imageData;        /* Pointer to aligned image data.         */
    int  widthStep;         /* Size of aligned image row in bytes.    */
    ...  //其他字段这里不做展示，感兴趣的小伙伴可以查看下opencv中的头文件
}
IplImage;

Mat image data structure

Attributes Explanation
cols The number of columns of the matrix (image width)
rows The number of rows of the matrix (image height)
data uchar pointer type. Mat class divided into two parts: header and pointers pointing matrix matrix data portion, data is a pointer to the data matrix.
step After the number of bytes of image alignment line

Third, the step size effect

We see from the above description and ArcSoft OpenCV library for image data structure algorithms have introduced the concept of image step, where we look at the image in steps.

  • Picture stories do OpenCV image alignment

    Below, a size 998x520of the image, the read image data using OpenCV, still image size 998x520, color format BGR24, the image is not a step 998 * 3, but rather 1000 * 3, the two pixels to the right is filled, the image made of four OpenCV byte aligned, then ArcSoft SDK internal algorithm to calculate the width of the image by passing step variation occurs, the image data is garbled, substantially impossible to detect the human face.

High byte alignment

  • The importance of the step size
    is only a few pixels of this difference, why can not detect the cause face it? Over said before, the step size will be appreciated that the number of bytes after the line image alignment. Reading a first row of pixels if the deviation, a pixel that subsequent reading will be affected.

The following is the size of a 1000x554picture, so as not to synchronize result of analyzing long:

To 1000 in steps of parsing To 996 in steps of parsing
To 1000 in steps of parsing To 996 in steps of parsing

It can be seen for a picture, if you use the steps to resolve the error, we may not be able to see the correct image content.

Conclusion: By introducing an image step can effectively prevent the high-byte aligned.

Four, OpenCV structure of the image data is converted into image data structure ArcSoft

The current C / C ++ developer for image codec processing generally uses OpenCV library, where we explain how to convert OpenCV to ArcSoft image data structure. ArcSoft official documentation described in seven colors format support, we list seven colors format conversion method.

  • OpenCV image reading generally came BGR24format, the image data structure can be converted using the following method.

  • If the original infrared image, the image needs to be converted to the ASVL_PAF_GRAYformat (the official document also example), and then converted using the following method.

IplImage turn ASVLOFFSCREEN


int ColorSpaceConversion(MInt32 format, IplImage* img, ASVLOFFSCREEN& offscreen)
{
    switch (format)     //原始图像颜色格式
    {
    case ASVL_PAF_I420:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img->width;
        offscreen.i32Height = img->height;
        offscreen.pi32Pitch[0] = img->widthStep;
        offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0] >> 1;
        offscreen.pi32Pitch[2] = offscreen.pi32Pitch[0] >> 1;
        offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
        offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0];
        offscreen.ppu8Plane[2] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0] * 5 / 4;
        break;
    case ASVL_PAF_YUYV:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img->width;
        offscreen.i32Height = img->height;
        offscreen.pi32Pitch[0] = img->widthStep;
        offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
        break;
    case ASVL_PAF_NV12:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img->width;
        offscreen.i32Height = img->height;
        offscreen.pi32Pitch[0] = img->widthStep;
        offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0];
        offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
        offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height;
        break;
    case ASVL_PAF_NV21:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img->width;
        offscreen.i32Height = img->height;
        offscreen.pi32Pitch[0] = img->widthStep;
        offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0];
        offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
        offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height;
        break;
    case ASVL_PAF_RGB24_B8G8R8:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img->width;
        offscreen.i32Height = img->height;
        offscreen.pi32Pitch[0] = img->widthStep;
        offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
        break;
    case ASVL_PAF_DEPTH_U16:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img->width;
        offscreen.i32Height = img->height;
        offscreen.pi32Pitch[0] = img->widthStep;
        offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
        break;
    case ASVL_PAF_GRAY:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img->width;
        offscreen.i32Height = img->height;
        offscreen.pi32Pitch[0] = img->widthStep;
        offscreen.ppu8Plane[0] = (MUInt8*)img->imageData;
        break;
    default:
        return 0;
    }
    return 1;
}

Mat turn ASVLOFFSCREEN


int ColorSpaceConversion(MInt32 format, cv::Mat img, ASVLOFFSCREEN& offscreen)
{
    switch (format)   //原始图像颜色格式
    {
    case ASVL_PAF_I420:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img.cols;
        offscreen.i32Height = img.rows;
        offscreen.pi32Pitch[0] = img.step;
        offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0] >> 1;
        offscreen.pi32Pitch[2] = offscreen.pi32Pitch[0] >> 1;
        offscreen.ppu8Plane[0] = img.data;
        offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0];
        offscreen.ppu8Plane[2] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0] * 5 / 4;
        break;
    case ASVL_PAF_YUYV:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img.cols;
        offscreen.i32Height = img.rows;
        offscreen.pi32Pitch[0] = img.step;
        offscreen.ppu8Plane[0] = img.data;;
        break;
    case ASVL_PAF_NV12:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img.cols;
        offscreen.i32Height = img.rows;
        offscreen.pi32Pitch[0] = img.step;
        offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0];
        offscreen.ppu8Plane[0] = img.data;
        offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height;
        break;
    case ASVL_PAF_NV21:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img.cols;
        offscreen.i32Height = img.rows;
        offscreen.pi32Pitch[0] = img.step;
        offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0];
        offscreen.ppu8Plane[0] = img.data;
        offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height;
        break;
    case ASVL_PAF_RGB24_B8G8R8:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img.cols;
        offscreen.i32Height = img.rows;
        offscreen.pi32Pitch[0] = img.step;
        offscreen.ppu8Plane[0] = img.data;
        break;
    case ASVL_PAF_DEPTH_U16:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img.cols;
        offscreen.i32Height = img.rows;
        offscreen.pi32Pitch[0] = img.step;
        offscreen.ppu8Plane[0] = img.data;
        break;
    case ASVL_PAF_GRAY:
        offscreen.u32PixelArrayFormat = (unsigned int)format;
        offscreen.i32Width = img.cols;
        offscreen.i32Height = img.rows;
        offscreen.pi32Pitch[0] = img.step;
        offscreen.ppu8Plane[0] = img.data;
        break;
    default:
        return 0;
    }
    return 1;
}

for example

Examples cited herein ArcSoft official website of the document, but the use of the above-described image format conversion method.


//opencv方式裁剪图片
void CutIplImage(IplImage* src, IplImage* dst, int x, int y)
{
    CvSize size = cvSize(dst->width, dst->height);//区域大小
    cvSetImageROI(src, cvRect(x, y, size.width, size.height));//设置源图像ROI
    cvCopy(src, dst); //复制图像
    cvResetImageROI(src);//源图像用完后，清空ROI
}


IplImage* originalImg = cvLoadImage("1280 x 720.jpg");  

//图像裁剪，宽度做四字节对齐，若能保证图像是四字节对齐这步可以不用做
IplImage* img = cvCreateImage(cvSize(originalImg->width - originalImg->width % 4, originalImg->height), IPL_DEPTH_8U, originalImg->nChannels);
CutIplImage(originalImg, img, 0, 0);

//图像数据以结构体形式传入，对更高精度的图像兼容性更好
ASF_MultiFaceInfo detectedFaces = { 0 };
ASVLOFFSCREEN offscreen = { 0 };
//IplImage 转 ASVLOFFSCREEN
ColorSpaceConversion(ASVL_PAF_RGB24_B8G8R8, img, offscreen);
if (img)
{
    MRESULT res = ASFDetectFacesEx(handle, &offscreen, &detectedFaces);
    if (MOK != res)
    {
        printf("ASFDetectFacesEx failed: %d\n", res);
    }
    else
    {
        // 打印人脸检测结果
        for (int i = 0; i < detectedFaces.faceNum; i++)
        {
            printf("Face Id: %d\n", detectedFaces.faceID[i]);
            printf("Face Orient: %d\n", detectedFaces.faceOrient[i]);
            printf("Face Rect: (%d %d %d %d)\n", 
                detectedFaces.faceRect[i].left, detectedFaces.faceRect[i].top, 
                detectedFaces.faceRect[i].right, detectedFaces.faceRect[i].bottom);
        }
    }

    //释放图像内存，这里只是做人脸检测，若还需要做特征提取等处理，图像数据没必要释放这么早
    cvReleaseImage(&img);
}
cvReleaseImage(&originalImg);

Personal Summary: The study found V3.0 SDK version using the old interface is also normal use, a new interface for higher-byte aligned images more compatible.

Demo can ArcSoft Face Recognition open platform download

Guess you like

Origin blog.51cto.com/14633836/2458136
Recommended
Ranking
[Nodejs] Native nodejs routing, getting parameters, static directory
Introduction and use of the datetime module in python - used to process date and time, calculate time intervals, etc.
Java bytecode decompiler in IntelliJIDEA for Scala
Resolved local variable 'str' referenced before assignment
Transformer Vision (2) || ViT-B/16 network structure
Translation: The most impressive YouTube channel that allows you to learn AI, machine learning and data science
gRPC- interceptor simple to use
Fortunately, three times faster investment has been lost how to do? There Baoying tutor you fly 69,937,609
Using Django to create a new project in the Windows10 environment gives an error [django-admin.py: The "django-admin.py" item cannot be recognized as the name of a cmdlet, function, script file, or executable program. 】
What should I do if I need to prepare the product standard
Daily
More
2024-06-05(0)
2024-06-04(0)
2024-06-03(1)
2024-06-02(0)
2024-06-01(1)
2024-05-31(1)
2024-05-30(0)
2024-05-29(1)
2024-05-28(1)
2024-05-27(1)

Code World

© 2018 codetd.com