From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

This article is shared from Huawei Cloud Community " Target Recognition of Vehicle License Plate Numbers Based on ModelArts ", author: Yushan Island.

Preface

The vehicle license plate is the unique identity of the vehicle. It can provide information such as the type, color, and location of the vehicle. It has important application value in areas such as traffic management, safety monitoring, and smart travel. With the development of computer vision technology, automatic detection and recognition of vehicle license plates has become a popular research topic, and there are many mature commercial products and open source projects.

I am very interested in the technical principles and application scenarios of vehicle license plate detection and recognition. I want to improve my image processing and machine learning capabilities through practice. I also want to explore some new methods and ideas to solve some practical problems. Therefore, I chose Huawei Cloud's ModelArts platform and used the full-process support it provides such as data processing, model training, model management, and model deployment to implement a target detection model for vehicle license plates.

In this blog, I will introduce in detail my process and experience using the ModelArts platform, including the preparation of data sets, model selection and modification, training parameters and results, deployment methods and effects, etc., as well as some problems encountered and solution. I hope to share and communicate with interested readers through this blog, and I welcome your valuable comments and suggestions.

Preliminary work

First, make sure you have completed the activation and authentication of a Huawei Cloud account, and added the user's access key to facilitate the use of ModelArts.

Data set preparation

CCPD (Chinese City Parking Dataset), a large-scale domestic parking lot license plate data set open sourced by the University of Science and Technology of China in 2018, makes up for the shortcomings of the above license plate data. The license plate data set pictures are collected from various street roadside parking lots in Hefei City, Anhui Province. Each collector works from 07:30 in the morning to 22:00 in the evening regardless of the weather on the streets they are responsible for. Whenever a parking bill is issued, they need to hold special equipment to take pictures of the front and rear license plates as required and annotate the license plate information, so as to The diversity of license plate data obtained is guaranteed under the constraints of unconfirmable shooting location, shooting angle, shooting lighting, shooting weather, shooting background, shooting street, etc.

I obtained the 2020 new version of this data from github

Creation of OBS buckets and objects

OBS is an object storage service provided by ModelArts for the purpose of data storage, model backup and snapshots. Before creating the training data set in this experiment, you need to create an OBS bucket, and then create a folder in the OBS bucket to store the training files.

Creation of OBS bucket

Enter the service list, find the storage service item, and click on the object storage service OBS

Log in to the OBS management console and click "Create Bucket" in the upper right corner of the bucket list page to create an OBS bucket.

Fill in the parameters:

• "Region" : North China-Beijing IV, subsequent regions need to be selected consistent with the OBS bucket.

• "Bucket name" : Create a bucket name of "carlic-detect". Note that you can add a suffix if the name is the same.

• “Data redundant storage strategy” : multi-AZ storage

• "Default storage type" : standard storage

Use the default values for other values. After confirming that everything is correct, click "Create Now" in the lower right corner.

Creation of objects in OBS bucket

On the bucket list page, click the bucket name to enter the bucket overview page. Click "Object" in the left navigation of the overview page, click New Folder on the "Object" page, and create an OBS folder. We create two folders here, one for the input of the data set and one for the output of the data set.

Dataset input folder "carlic-input"

Data set output folder "carlic-output"

ModelArts automatic learning project creation

Enter the ModelArts automatic learning version and create an object detection project

Create a data set for the object detection project

At this point we discovered that we do not have a data set, so we need to create a data set. Now click Create Dataset

After clicking, the following interface will appear, in which the parameters are:

• "dataset name" : carlicense

• "Data type" : Picture

• “Data source” : OBS

• "Import path" : Select the carlic-input folder created when creating the OBS bucket before

• “Data annotation status“ : Not annotated

• "Data output location" : Select the carlic-output folder created when creating OBS before.

Then click Submit in the lower right corner, and the data set is created successfully.

At this time, fill in the various parameter information of the object detection project

• “Name” : carlicense-detect

• “Dataset” : Select the data set “carlicense” just created

• "Output path" : Select the data output folder "carlic-output" in the created OBS bucket

• “Training specifications” : Select the specifications that are free for a limited time

Then click Create Project in the lower right corner

After creating the project, you get the following interface:

Data upload and annotation

This interface shows that we need to label the instance. Click to enter the instance details.

Data upload

Since the data set is empty, the page shows that no images were found. At this time, we click Add Data.

The import data parameters are as follows:

• "Data source" : local upload (because the downloaded data is all local and not uploaded to OBS)

• "Upload data storage path" : Select the data input folder "carlic-input" in the OBS bucket created previously

• “Data labeling status“ : Unlabeled (all data in the data set are unlabeled)

Then click File Upload and enter the data in the OBS bucket created by uploading the local file into the folder.

Click Add File to upload the local training data file

After the upload is completed, we click on OBS storage and find that the input folder at this time already contains the used data.

However, there is no data information in the strength details at this time. At this time, we need to click Data Source Synchronization to synchronize all the data that has been uploaded to the OBS bucket to the data set.

The results are as follows

Data annotation

Next, we need to annotate the data in the data set.

Perform manual annotation first

After annotating more than 20 examples, we can use the tools provided by ModelArts: intelligent annotation, which means to automatically learn from the annotated examples to achieve automated annotation.

At this point we click to start smart annotation

Parameter selection is as follows:

• "Intelligent annotation type" : active learning (the system automatically uses semi-supervised learning, difficult example screening and other methods to perform intelligent annotation, reducing the amount of manual annotation and helping users find difficult examples)

• "Algorithm type" : precise

• "Calculation node specifications" : You can choose any

Click Submit and get the following interface

After the smart annotation is completed, we need to confirm whether the smart annotation information is correct. The following is the interface that needs to be confirmed after the smart annotation.

Then we click to enter the specific confirmation interface

Zoom in to view the smart annotation of this case

The intelligent annotation of license plates in this picture is very accurate, but this does not mean that every picture is very accurate. After confirming the intelligent annotation of multiple pictures, I found some common phenomena that are prone to incorrect annotation. Large car logos, rectangular advertising stickers on the car body similar to the license plate, some rectangular numbers on the ground that are very similar to the license plate, etc.

Too big car logo

Small rectangular advertisement on the car body-

The rectangular number plate on the ground is very similar to the license plate

After manual confirmation and correction, the final labeled data set is obtained

Then perform a quick review of the annotation data, click Quick Review, and get the following interface. We found that there is some annotation information that is obviously not a car logo.

Click one of them to get the following image annotation

It means that there are errors in the process of confirming the annotations, and then you only need to modify or delete these erroneous annotations.

Enter workflow

After successfully labeling the data, we can enter the workflow flow and start working.

The first is the release of the dataset version

Then there is data verification

At the same time, we can view the work records in the background and follow up on the progress of the workflow.

Then there is the training of the model

Model training

Next is the model registration work

Model deployment

Finally, the model deployment work

The parameters for model deployment are selected as follows:

• “AI Application Source” : My AI Application

• "Resource Pool" : Public resource pool

• "Whether to stop automatically" : Yes, choose to stop running after one hour

Keep other parameters as default

During the deployment process, we can view some basic information and track deployment events:

After successful deployment, we can view the successfully deployed AI applications. The deployed AI application information is as follows

At the same time, after the AI application status changes to running, we can start the practice of our vehicle license plate target recognition.

Result detection

Upload the image to the AI application for target detection, and the results are as follows

It was found that the approximate location of the detection was correct, but the accuracy was not high, so I uploaded the picture to detect the license plate several times:

First of all, the detection results of this model are generally correct, but not accurate enough. This shows that the accuracy of the model is not high enough and needs further improvement.

As for the reason for this problem, I guess there may be the following reasons:

1. Data set problem

2. Data processing algorithm design and implementation issues

3. Algorithm design and implementation issues

4. Super parameter setting problem

5. Environmental issues

……

In subsequent experiments, the model will be further optimized and new model versions and application versions will be launched.

Resource release

At this point, the practice has been completed. Next, we need to release the various resources applied for.

Delete a created online service

Delete the created automatic learning project

Delete objects in the OBS bucket used to store services

The release of resources ends here, and this practice is completed.

Practice summary

In this article, I introduce the practice of using Huawei cloud technology ModelArts to realize target recognition of vehicle license plates, and the process of deploying online services. I first introduced the background and significance of license plate target detection, and then introduced the characteristics and advantages of ModelArts, and how to use ModelArts for data set management, model training, evaluation and deployment. I used the CCPD dataset, which is a dataset containing about 300,000 Chinese license plate images, which is divided into eight categories according to the complexity of the images. I used YOLOv5 as the target detection model, which is a lightweight and efficient target detection framework based on PyTorch. I trained and evaluated the model on ModelArts and achieved high precision and recall. I also showed how to use the online service function of ModelArts to deploy the model as a callable API to achieve real-time detection of license plates.

Through this practice, I deeply realized the power and convenience of ModelArts. ModelArts provides me with a one-stop end-to-end machine learning platform that allows me to complete the entire process of data set management, model training, evaluation, and deployment on a unified interface. ModelArts also provides a wealth of preset algorithms and models, as well as high-performance computing resources and storage space, allowing me to quickly build and run my models, saving time and costs. The online service function of ModelArts also allows me to easily convert my model into a usable service, realizing the application scenario of license plate target detection. I think ModelArts is a platform that is very suitable for machine learning enthusiasts and developers. It can help us realize more machine learning innovation and value.

Click to follow and learn about Huawei Cloud’s new technologies as soon as possible~

Say goodbye to tedious work, ModelArts solves the problem of license plate number recognition with one click