Computer Vision: Using opencv to implement license plate recognition

1 Introduction

License Plate Recognition is a common application in daily life, especially in intelligent transportation systems, where license plate recognition plays a huge role. The automatic recognition technology of car license plates connects the image processing method with computer software technology, with the purpose of accurately identifying the characters of the license plate, and transmits the identified data to the real-time traffic management system to ultimately realize traffic supervision. function. In the automatic license plate recognition system, the process from car image acquisition to license plate character processing is a complex process, which is mainly divided into four stages: image acquisition, license plate positioning, character segmentation and character recognition. There are currently many algorithms for license plate recognition. This article builds the entire process of license plate recognition based on opencv for your reference.

1 Overview of license plate recognition

1.1 Introduction to opencv

The full name of OpenCV is: Open Source Computer Vision Library. OpenCV is a cross-platform computer vision library based on open source distribution and can run on Linux, Windows and Mac OS operating systems. It is lightweight and efficient - it consists of a series of C functions and a small number of C++ classes. It also provides interfaces in Python, Ruby, MATLAB and other languages, and implements many common algorithms in image processing and computer vision.

1.2 Decomposition of license plate recognition

The entire process of license plate recognition can be broken down into the following three steps:

• License plate positioning: The first step is to detect the location of the license plate from the car. This article will use rectangular frame detection in OpenCV to find the license plate position.
• Character cutting: After detecting the license plate, use opencv to crop it and save it as a new picture for subsequent recognition.
• Character recognition: Use optical character recognition (OCR) technology in new pictures to extract text, characters, and numbers in the picture.

2 Implementation of license plate recognition

2.1 License plate positioning

my country's car license plates generally consist of seven characters and a dot. The height and width of the license plate characters are fixed, 90mm and 45mm respectively. The distance between the seven characters is also fixed at 12mm. The diameter of the dot separator is 10mm. Differences between characters may cause the distance between characters to change.

In civil license plates, the arrangement and position of characters follows the following rules:

• The first character is usually the abbreviation of each province or region in my country, expressed in Chinese characters;
• The second character is usually the code number of the issuing authority. The last five characters are a combination of English letters and numbers. The letters are a combination of twenty-four capital letters (excluding the two letters I and O), and the numbers are represented by " Represented by a number between 0-9".

From an image processing perspective, car license plates have the following characteristics:

• The first feature is the geometric feature of the license plate, that is, the shape of the license plate is unified into a rectangle with fixed length, width and height;
• The second feature is that the grayscale distribution of the license plate shows a continuous trough-peak-trough distribution. This is because my country's license plates have a single color and the characters are arranged in a straight line;
• The third feature is that the license plate histogram exhibits a bimodal shape, that is, double peaks can be seen in the license plate histogram;
• The fourth feature is that the license plate has strong edge information. This is because the characters of the license plate are relatively concentrated in the center of the license plate, but there are no characters on the edge of the license plate, so the edge information of the license plate is strong;
• The fifth feature is that the character color of the license plate contrasts sharply with the background color of the license plate. At present, my country's domestic license plates can be roughly divided into white characters on a blue background and black characters on a yellow background. Special vehicles use white characters with black characters or black characters with white characters, sometimes supplemented by red characters.

In order to simplify the process, only license plates with blue background and white characters are considered in this study.

2.1.1 Image loading and grayscale

import cv2

img = cv2.imread('../data/bmw01.jpg')

# 调整图片大小
img = cv2.resize(img, (1024, 800))

# 灰度图
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# 显示效果
cv2.imshow('gray', gray)
cv2.waitKey(0)
cv2.destroyAllWindows()

The displayed results are as follows:

2.1.2 Bilateral filtering to remove noise

# 双边滤波
blf = cv2.bilateralFilter(gray, 13, 15, 15)
show_image('bilateralFilter', blf)

The displayed results are as follows:

2.1.3 Edge detection

# 边缘检测
edged = cv2.Canny(blf, 30, 200)
show_image('canny', edged)

The displayed results are as follows:

2.1.4 Find license plate outline (quadrangle)

cv2.findContours description:

• opencv3.x

image, contours, hierarchy = cv.findContours(image, mode, method[, contours[, hierarchy[, offset]]])

• opencv2.x and 4.x

contours, hierarchy = cv.findContours(image, mode, method[, contours[, hierarchy[, offset]]])

HSV space color comparison table in OpenCV

Extract the color of an image area

def reg_area_color(image):
    """找到原图像最多的颜色，当该颜色为红色或蓝色时返回该颜色的名称"""
    kernel = np.ones((35, 35), np.uint8)
    hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
    # 以上为图像处理
    Open = cv2.morphologyEx(hsv, cv2.MORPH_OPEN, kernel)
    # 对Open图像的H通道进行直方图统计
    hist = cv2.calcHist([Open], [0], None, [180], [0, 180])
    # 找到直方图hist中列方向最大的点hist_max
    hist_max = np.where(hist == np.max(hist))

    # hist_max[0]为hist_max的行方向的值，即H的值，H在0~10为红色
    if 0 < hist_max[0] < 10:
        res_color = 'red'
    elif 100 < hist_max[0] < 124:  # H在100~124为蓝色
        res_color = 'blue'
    else:
        # H不在前两者之间跳出函数
        res_color = 'unknow'
    return res_color

Find the license plate outline:

# 寻找轮廓（图像矩阵，输出模式，近似方法）
contours, _ = cv2.findContours(edged.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# 根据区域大小排序取前十
contours = sorted(contours, key=cv2.contourArea, reverse=True)[:10]
screenCnt = None
# 遍历轮廓，找到车牌轮廓
for c in contours:
    if cv2.contourArea(c) > 1024 * 768 * 0.05:
        continue

    # 计算轮廓周长（轮廓，是否闭合）
    peri = cv2.arcLength(c, True)
    # 折线化（轮廓，阈值（越小越接近曲线），是否闭合）返回折线顶点坐标
    approx = cv2.approxPolyDP(c, 0.018 * peri, True)
    # 获取四个顶点（即四边形, 左下/右下/右上/左上
    if len(approx) == 4:
        # [参数]左上角纵坐标:左下角纵坐标,左上角横坐标:右上角横坐标
        crop_image = img[approx[3][0][1]:approx[0][0][1], approx[3][0][0]:approx[2][0][0]]
        show_image('crop', crop_image)
        if 'blue' == reg_area_color(crop_image):
            screenCnt = approx
            break
# 如果找到了四边形
if screenCnt is not None:
    # 根据四个顶点坐标对img画线(图像矩阵，轮廓坐标集，轮廓索引，颜色，线条粗细)
    cv2.drawContours(img, [screenCnt], -1, (0, 0, 255), 3)

show_image('contour', img)

The running results show:

2.1.5 Image bit operation for masking

"""遮罩"""
# 创建一个灰度图一样大小的图像矩阵
mask = np.zeros(gray.shape, np.uint8)
# 将创建的图像矩阵的车牌区域画成白色
cv2.drawContours(mask, [screenCnt], 0, 255, -1, )
# 图像位运算进行遮罩
mask_image = cv2.bitwise_and(img, img, mask=mask)
show_image('mask_image', mask_image)

The running results show:

2.1.6 Image cropping

"""图像剪裁"""
# 获取车牌区域的所有坐标点
(x, y) = np.where(mask == 255)
# 获取底部顶点坐标
(topx, topy) = (np.min(x), np.min(y))
# 获取底部坐标
(bottomx, bottomy,) = (np.max(x), np.max(y))
# 剪裁
Cropped = gray[topx:bottomx, topy:bottomy]

The running results show:

2.1.7 OCR character recognition

paddleocr is a lightweight character recognition tool library that supports multi-language recognition, pip installation and custom training.

• Installation of tools under conda

pip install paddleocr -i https://mirror.baidu.com/pypi/simple 
pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple 

Code:

"""OCR识别"""

# 使用CPU预加载，不用GPU
ocr = PaddleOCR(use_angle_cls=True, use_gpu=False, ocr_version='PP-OCRv3')
text = ocr.ocr(cropped, cls=True)
for t in text:
    print(t[0][1])

The running results are shown as follows:

[2023/11/15 20:57:43] ppocr DEBUG: dt_boxes num : 1, elapsed : 0.016942501068115234
[2023/11/15 20:57:43] ppocr DEBUG: cls num  : 1, elapsed : 0.013955354690551758
[2023/11/15 20:57:43] ppocr DEBUG: rec_res num  : 1, elapsed : 0.12021970748901367
('苏A·0MR20', 0.8559348583221436)

2.2 Complete code implementation

import cv2
import numpy as np
from paddleocr import PaddleOCR


def show_image(desc, image):
    cv2.imshow(desc, image)
    cv2.waitKey(0)
    cv2.destroyAllWindows()


def reg_area_color(image):
    """找到原图像最多的颜色，当该颜色为红色或蓝色时返回该颜色的名称"""
    kernel = np.ones((35, 35), np.uint8)
    hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
    # 以上为图像处理
    Open = cv2.morphologyEx(hsv, cv2.MORPH_OPEN, kernel)
    # 对Open图像的H通道进行直方图统计
    hist = cv2.calcHist([Open], [0], None, [180], [0, 180])
    # 找到直方图hist中列方向最大的点hist_max
    hist_max = np.where(hist == np.max(hist))

    # hist_max[0]为hist_max的行方向的值，即H的值，H在0~10为红色
    if 0 < hist_max[0] < 10:
        res_color = 'red'
    elif 100 < hist_max[0] < 124:  # H在100~124为蓝色
        res_color = 'blue'
    else:
        # H不在前两者之间跳出函数
        res_color = 'unknow'
    return res_color


img = cv2.imread('../data/bmw01.jpg')

# 调整图片大小
img = cv2.resize(img, (1024, 768))

# 灰度图
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
show_image('gray', gray)

# 双边滤波
blf = cv2.bilateralFilter(gray, 13, 15, 15)
show_image('bilateralFilter', blf)

# 边缘检测
edged = cv2.Canny(blf, 30, 200)
show_image('canny', edged)

# 寻找轮廓（图像矩阵，输出模式，近似方法）
contours, _ = cv2.findContours(edged.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# 根据区域大小排序取前十
contours = sorted(contours, key=cv2.contourArea, reverse=True)[:10]
screenCnt = None
# 遍历轮廓，找到车牌轮廓
for c in contours:
    if cv2.contourArea(c) > 1024 * 768 * 0.05:
        continue

    # 计算轮廓周长（轮廓，是否闭合）
    peri = cv2.arcLength(c, True)
    # 折线化（轮廓，阈值（越小越接近曲线），是否闭合）返回折线顶点坐标
    approx = cv2.approxPolyDP(c, 0.018 * peri, True)
    # 获取四个顶点（即四边形, 左下/右下/右上/左上
    if len(approx) == 4:
        # [参数]左上角纵坐标:左下角纵坐标,左上角横坐标:右上角横坐标
        crop_image = img[approx[3][0][1]:approx[0][0][1], approx[3][0][0]:approx[2][0][0]]
        show_image('crop', crop_image)
        if 'blue' == reg_area_color(crop_image):
            screenCnt = approx
            break
# 如果找到了四边形
if screenCnt is not None:
    # 根据四个顶点坐标对img画线(图像矩阵，轮廓坐标集，轮廓索引，颜色，线条粗细)
    cv2.drawContours(img, [screenCnt], -1, (0, 0, 255), 3)
    show_image('contour', img)

"""遮罩"""
# 创建一个灰度图一样大小的图像矩阵
mask = np.zeros(gray.shape, np.uint8)
# 将创建的图像矩阵的车牌区域画成白色
cv2.drawContours(mask, [screenCnt], 0, 255, -1, )
# 图像位运算进行遮罩
mask_image = cv2.bitwise_and(img, img, mask=mask)
show_image('mask_image', mask_image)

"""图像剪裁"""
# 获取车牌区域的所有坐标点
(x, y) = np.where(mask == 255)
# 获取底部顶点坐标
(topx, topy) = (np.min(x), np.min(y))
# 获取底部坐标
(bottomx, bottomy,) = (np.max(x), np.max(y))
# 剪裁
cropped = gray[topx:bottomx, topy:bottomy]
show_image('cropped', cropped)

"""OCR识别"""
# 使用CPU预加载，不用GPU
ocr = PaddleOCR(use_angle_cls=True, use_gpu=False, ocr_version='PP-OCRv3')
text = ocr.ocr(cropped, cls=True)
for t in text:
    print(t[0][1])