Preface

Questionnaire survey collection is one of the most common social survey methods, usually in market surveys and information collection. But after the huge questionnaire survey is collected, how to enter such a large amount of data and convert the content into electronic documents? Today, I will introduce you to easily realize the form entry function by integrating the form recognition service of Huawei HMS ML Kit.

Application scenario

Huawei HMS ML Kit table recognition service uses AI technology to recognize and return table structure information (including row and column information and coordinate information of cells) and text information in the table (including text content in cells) from the input pictures. Support Recognize Chinese and English text messages and punctuation marks. The form recognition service is widely used in daily work scenarios. For example, after collecting a large number of paper form questionnaires, the identified questionnaire content can be converted into electronic documents through this service, reducing manual entry costs and greatly improving work efficiency.

Precautions

Support the identification of forms and questionnaires with form characteristics.
It does not support the recognition of multiple tables in the picture, and does not support obtaining the header and footer information.
The best recognition effect can be achieved when the following conditions are met (as shown in the figure below):

Filming angle	The plane tilt angle is less than 5 degrees
Form completeness	No missing corners, no bending table lines, table lines are continuous and uninterrupted
Table content	The text in the table is horizontal. The background color (ground color) and the color of the table line in the table are required to have obvious contrast. Only pure printed text is supported. The recognition of pictures, formulas, handwriting, seals, and seals in the table are not supported temporarily. Watermark etc.
Picture specifications	The ratio of the long side to the short side of the picture must be less than 3:1 (inclusive), the resolution must be greater than 960*960px, the direction of the table in the picture must be positive, and the table occupies more than 60% of the picture area and is in the middle of the picture

Development steps

1. Development Preparation

For detailed preparation steps, please refer to Huawei Developer Alliance:
https://developer.huawei.com/consumer/cn/doc/development/HMSCore-Guides-V5/dev-process-0000001050038076-V5?ha_source=hms1

Here are the key development steps.

1.1 Configure Maven warehouse address in project-level gradle

buildscript {
    repositories {
        google()
        jcenter()
        maven {url 'https://developer.huawei.com/repo/'}
    }
    dependencies {
        ...
        classpath 'com.huawei.agconnect:agcp:1.4.1.300'
    }
}

allprojects {
    repositories {
        google()
        jcenter()
        maven {url 'https://developer.huawei.com/repo/'}
    }
}

1.2 File header increase configuration

After integrating the SDK, add configuration to the file header

apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'

1.3 Configure SDK dependencies in application-level gradle

dependencies{
    // 引入基础SDK
    implementation 'com.huawei.hms:ml-computer-vision-formrecognition:2.0.4.300'
    // 引入表格识别模型包
    implementation 'com.huawei.hms:ml-computer-vision-formrecognition-model:2.0.4.300'
}

1.4 Add the following statement to the AndroidManifest.xml file to automatically update the machine learning model

 <meta-data
        android:name="com.huawei.hms.ml.DEPENDENCY"
        android:value= "fr"/>

1.5 Apply for camera permission

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" />

2. Code Development

2.1 Create a form recognition analyzer.

MLFormRecognitionAnalyzerSetting setting = new MLFormRecognitionAnalyzerSetting.Factory().create();
MLFormRecognitionAnalyzer analyzer = MLFormRecognitionAnalyzerFactory.getInstance().getFormRecognitionAnalyzer(setting);

2.2 Create an MLFrame object through android.graphics.Bitmap for the analyzer to recognize the table. The supported image formats include: jpg/jpeg/png. The recommended image size is not less than 960 960 pixels and not more than 1920 1920 pixels.

MLFrame mlFrame = MLFrame.fromBitmap(bitmap);

2.3 Invoke the asyncAnalyseFrame asynchronous method or analyzeFrame synchronous method to identify the table (for the data structure definition of the JsonObject object, please refer to JsonObject).

// asyncAnalyseFrame异步调用。
Task<JsonObject> recognizeTask = analyzer.asyncAnalyseFrame(mlFrame);
recognizeTask.addOnSuccessListener(new OnSuccessListener<JsonObject>() {
    @Override
    public void onSuccess(JsonObject recognizeResult) {
        // 识别成功。
    }
}).addOnFailureListener(new OnFailureListener() {
    @Override
    public void onFailure(Exception e) {
        // 识别失败。
    }
});

// analyseFrame同步调用。
SparseArray<JsonObject> recognizeResult = analyzer.analyseFrame(mlFrame);
if (recognizeResult != null && recognizeResult.get(0).get("retCode").getAsInt() == MLFormRecognitionConstant.SUCCESS) {
    // 识别成功。
} else {
    // 识别失败。
}

2.4 After the detection is complete, stop the analyzer and release the detection resources.

if (analyzer != null) {
    analyzer.stop();
}

Insert picture description here

to sum up

Huawei's machine learning form recognition service provides developers with services for recognizing forms in pictures, which can be widely used in application scenarios such as questionnaire survey data collection, instead of manual entry and reduce costs.

For more details, please refer to:

Official website of Huawei Developer Alliance:https://developer.huawei.com/consumer/cn/hms?ha_source=hms1

Obtain development guidance documents:https://developer.huawei.com/consumer/cn/doc/development?ha_source=hms1

To participate in developer discussions, please go to the Reddit community:https://www.reddit.com/r/HMSCore/

To download the demo and sample code, please go to Github:https://github.com/HMS-Core

To solve integration problems, please go to Stack Overflow:https://stackoverflow.com/questions/tagged/huawei-mobile-services?tab=Newest

If you have any questions or need help, please contact us: [email protected]

Original link:https://developer.huawei.com/consumer/cn/forum/topic/0204429136296960028?fid=18

Author: timer

One article easily integrates Huawei HMS ML Kit form recognition service