RK3399 PRO rapid development - the license plate recognition

Introduction to
Rock-X SDK is based on RK3399Pro / RK1808 platform set of AI component library. Developers API interface provided by Rock-X SDK to quickly build AI applications. Rock-X Download: https://pan.baidu.com/s/1brKNqxBYDmElm-A56DLu4Q extraction code: ji14 Rock-X SDK major features

category	Features
Target Detection	Head detection, vehicle detection
human face	Face key points, face attribute analysis, face recognition
License Plate	Detecting license plate, license plate recognition
The key point of the human body	Human skeleton key, key point fingers

Benpian is carried out using Rock-X SDK license plate recognition process of rapid development, the use of custom or other models, please use rknn-toolkit and rknn-api development (link: http://wiki.t-firefly.com/ zh_CN / 3399pro_npu / ).

Performance indicators
license plate recognition performance

data set	Performance
CCPD	83.31%(8331/10000)

Notes:
1. CCPD (Chinese City Parking Dataset) is the license plate data set, from which 10,000 randomly selected for testing.
2. Support the identification of domestic blue, green and yellow license plates.
3. recognizable character plate shown in the following table.

Character classes	Recognizable character
Province Chinese characters	Shanghai Jin Yu Ji Jin Mongolia Liaoning, Jilin and Heilongjiang Jiangsu, Zhejiang and Fujian, Jiangxi, Hubei and Hunan and Guangdong Lu Yu Gui Qiongchuan your new cloud possession of Shaanxi Ganqing
Numbers and letters	0 1 2 3 4 5 6 7 8 9 A B C D E F G H J K L M N P Q R S T U V W X Y Z
车牌用途中文字符	港学使警澳挂军北南广沈兰成济海民航空

调用过程

硬件准备
AIO-3399ProC + 摄像头

相关代码
以下为Android的相关代码，完整代码在/RockX_SDK_V1.1.0_20191115/demo/rk3399pro_android_demo/rockx-android-demo-carplate_recognition.zip。

此demo功能是从摄像头获取图像数据，检测图像中的车牌，然后进行车牌对齐和识别，最后显示识别的结果。

详细的函数定义和相关数据结构请参考：/RockX_SDK_V1.1.0_20191115/doc/rockx_api_doc/html/index.html

1. 初始化

public void create() {
    mModelPath = installRockxData(mContext);
    mRockXPlateDetectionModule = native_create_rockx_module(mModelPath, ROCKX_MODULE_CARPLATE_DETECTION);
    mRockXCarplateOnetModule = native_create_rockx_module(mModelPath, ROCKX_MODULE_CARPLATE_ALIGN);
    mRockXCarplateRecogModule = native_create_rockx_module(mModelPath, ROCKX_MODULE_CARPLATE_RECOG);
}

程序会调用如上实现的RockX.create()方法创建车牌检测、车牌矫正对齐和车牌识别模块。其中RockX.native_create_rockx_module()实现为Java_com_rockchip_gpadc_demo_rockx_RockX_native_1create_1rockx_1module(),其调用Rock-X的API rockx_create()生成对应模块的实例。

2. 检测车牌
程序在java层调用RockX.detectCarplate()进行检测车牌,RockX.detectCarplate()实际是RockX.native_plate_detect()的封装,实际是RockX.native_plate_detect()部分实现如下:

extern "C"
JNIEXPORT jint JNICALL Java_com_rockchip_gpadc_demo_rockx_RockX_native_1plate_1detect
        (JNIEnv *env, jobject obj, jlong handle, jbyteArray inData, jint inWidth, jint inHeight, jint inPixelFmt,
         jobject detectObjectList) {

    .. // 省略部分代码
    rockx_ret_t ret = rockx_carplate_detect((rockx_handle_t)handle, &input_image, &object_array, nullptr);
    if (ret != ROCKX_RET_SUCCESS) {
        LOGE("rockx_face_detect error %d\n", ret);
        return -1;
    }

    object_array_c2j(env, &object_array, detectObjectList, inWidth, inHeight);

    env->ReleaseByteArrayElements(inData, in_data, JNI_ABORT);

    return 0;
}

程序调用的是Rock-X的API rockx_carplate_detect()进行检测车牌,省略的部分是数据结构的初始化和填充，返回的结果为object_array，通过object_array_c2j()赋值给detectObjectList传回java层。

3. 车牌对齐和识别
车牌对齐和识别分别调用的是Rock-X的rockx_carplate_align()和rockx_carplate_recognize()进行的。在java层，程序调用RockX.recogCarplate()将检测到的车牌进行对齐和识别。 RockX.recogCarplate()的部分实现：

public Map<String,Float> recogCarplate(byte[] inData, int width, int height, int inPixelFmt,
                                           int left, int right, int top, int bottom ,
                                           byte[] recog_result  , float trans_alignconfidence, float a_t, float a_num, float r_t, float r_num) {

        ... // 省略部分代码

        starttime = System.currentTimeMillis();
        alignconfidence = native_get_aligned_plate(mRockXCarplateOnetModule, inData,width, height, inPixelFmt,
                x1, x2, y1, y2, alignconfidence , alignimg);
        endtime = System.currentTimeMillis();
        a_t+=(endtime-starttime)/1000F;
        a_num++;

        System.gc();

        Map<String,Float> result=new HashMap<>();
        if (alignconfidence < 0) {
            alignimg = null;
            System.gc();
            result.put("alignconfidence",alignconfidence);
            return result;
        }

        if(alignconfidence>0.5) {       //filter the case of single character
            starttime = System.currentTimeMillis();
            float ret = native_get_recog_result(mRockXCarplateRecogModule, alignimg, inPixelFmt, recog_result);
            endtime = System.currentTimeMillis();
            r_t+=(endtime-starttime)/1000F;
            r_num++;
            Log.d(TAG, "current_ average recog time = " + r_t/r_num);

            if (ret < 0) {
                alignimg = null;
                System.gc();
                result.put("alignconfidence",alignconfidence);
                return result;
            }
        }

        ... // 省略部分代码
    }

可以看到车牌对齐调用的是RockX.native_get_aligned_plate(),车牌识别调用的是RockX.native_get_recog_result()这两个都是native函数，其实现其它的native函数实现形式差不多，都是通过调用Rock-X的API实现相关功能的。

代码测试

RK3399 PRO rapid development - the license plate recognition

Guess you like