Table of contents
1. Morphological license plate extraction (simple: single scenario)
1. Read the picture and convert it to grayscale
2. Extract the contour (the Sobel operator extracts the edge in the y direction)
4. Closing operation processing, closing and kneading the image, making the image regionalized
5. Corrosion/expansion denoising to obtain the license plate area
5-1. Lateral corrosion and expansion
5-2. Longitudinal corrosion and expansion
6-2. Draw the outline and display it
7. Intercept and obtain the license plate
2. Morphological license plate extraction (optimization: multiple scenarios)
3. Sobel operator extracts edges in Y direction
5. Open operation segmentation (vertical segmentation)
7. Corrosion/expansion in horizontal/vertical direction
7-1. Longitudinal direction: corrosion + expansion
7-2. Horizontal direction: expansion + corrosion
8. Corrosion expansion: denoising
10. Find the outline of the license plate according to the features of the license plate
1. Morphological license plate extraction (simple: single scenario)
(Single picture is not bad, but multiple pictures are not accurate)
1. Read the picture and convert it to grayscale
# 1、读取图片，转灰度图
img = cv.imread('Resource/car.jpg')
gray = cv.cvtColor(img, cv.COLOR_RGB2GRAY)
cv.imshow('gray', gray)
2. Extract the contour (the Sobel operator extracts the edge in the y direction)
Why take the edge for the Y direction: to make the image "thin" and make it easier to crumple the license plate.
# 2、提取轮廓（Sobel算子提取y方向）
y = cv.Sobel(gray, cv.CV_16S, 1, 0)
# 注：对x/y微分和得到x/y方向图像相反 要得到x/y方向边缘，就要求y/x方向的微分。
absY = cv.convertScaleAbs(y)
cv.imshow('Y', absY) 
3. Adaptive binarization
Note: The threshold function returns two parameters, the second parameter is the binary image! ! !
# 3、自适应二值化
ret, binary = cv.threshold(absY, 0, 255, cv.THRESH_OTSU)
cv.imshow('binary', binary)
4. Closing operation processing, closing and kneading the image, making the image regionalized
Closed operation processing, image regionalization, easy to find the license plate area, and then get the outline
After testing multiple groups of pictures, it was found that the convolution kernel of (17, 5) is better and can achieve the goal.
# 4、闭运算处理，把图像闭合、揉团，使图像区域化，便于找到车牌区域，进而得到轮廓
kernel = cv.getStructuringElement(cv.MORPH_RECT, (17,5))
print('kernel= \n', kernel)
close = cv.morphologyEx(binary, cv.MORPH_CLOSE, kernel)
cv.imshow('close', close)(Here, a rectangular convolution kernel is used.)
5. Corrosion/expansion denoising to obtain the license plate area
Although the binary image is obtained above and the approximate area is obtained, there are still too many noises (impurities), and the noise is denoised by erosion and expansion below.
5-1. Lateral corrosion and expansion
# 5-1、水平方向腐蚀/膨胀
erode = cv.morphologyEx(close, cv.MORPH_ERODE, kernel_x)
cv.imshow('erode_x', erode)
dilate = cv.morphologyEx(erode, cv.MORPH_DILATE, kernel_x)
cv.imshow('dilate_x', dilate)
After the horizontal processing is done, continue with the vertical processing.
5-2. Longitudinal corrosion and expansion
# 5-2、竖直方向腐蚀/膨胀
erode = cv.morphologyEx(dilate, cv.MORPH_ERODE, kernel_y)
cv.imshow('erode_y', erode)
dilate = cv.morphologyEx(erode, cv.MORPH_DILATE, kernel_y)
cv.imshow('dilate_y', dilate)
As shown in the figure above, the desired result is obtained , basically only a few outlines are left, and the license plate area is well preserved, which does not affect the positioning of the license plate.
6. Get the outline
Here I want to display the outer contour including the outline of the license plate first.
6-1. Get the outline
# 6-1、得到轮廓
contours, hierarchy = cv.findContours(dilate_y, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)6-2. Draw the outline and display it
# 6-2、画出轮廓
cv.drawContours(img_copy, contours, -1, (255,0,255), 2)
cv.imshow('Contours', img_copy)Obtain the outline code and effect:
# 6、获取外轮廓
img_copy = img.copy()
# 6-1、得到轮廓
contours, hierarchy = cv.findContours(dilate_y, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
# 6-2、画出轮廓并显示
cv.drawContours(img_copy, contours, -1, (255,0,255), 2)
cv.imshow('Contours', img_copy)
7. Intercept and obtain the license plate
First judge the characteristics of the license plate (such as width: height is generally 3~4), and then judge according to this feature, and only keep the pictures (license plates) that meet the characteristics.
# 7、遍历所有轮廓，找到车牌轮廓
for contour in contours:
# 7-1、得到矩形区域：左顶点坐标、宽和高
rect = cv.boundingRect(contour)
# 7-2、判断宽高比例是否符合车牌标准，截取符合图片
if rect[2]>rect[3]*3 and rect[2]<rect[3]*5:
# 截取车牌并显示
img = img[rect[1]:(rect[1]+rect[3]), rect[0]:(rect[0]+rect[2])]
cv.imshow('license plate', img)
2. Morphological license plate extraction (optimization: multiple scenarios)
1. Convert grayscale image
gray = cv.cvtColor(img, cv.COLOR_RGB2GRAY)
cv.imshow('gray', gray)2. Top hat operation
Extract the brighter part to make the license plate area more obvious.
# 2、顶帽运算
# gray = cv.equalizeHist(gray)
kernel = cv.getStructuringElement(cv.MORPH_RECT, (17,17))
tophat = cv.morphologyEx(gray, cv.MORPH_TOPHAT, kernel)
cv.imshow('tophat', tophat)
3. Sobel operator extracts edges in Y direction
Verticalize the license plate area to make the license plate more obvious .
# 3、Sobel算子提取y方向边缘（揉成一坨）
y = cv.Sobel(tophat, cv.CV_16S, 1, 0)
absY = cv.convertScaleAbs(y)
cv.imshow('absY', absY)
4. Binarized image
Select the threshold according to your actual situation. The larger the threshold, the more thresholds become white (255), and the less white part. (The threshold here is relatively small, but there are many advantages and disadvantages when the threshold is large)
# 4、自适应二值化（阈值自己可调）
ret, binary = cv.threshold(absY, 75, 255, cv.THRESH_BINARY)
cv.imshow('binary', binary)5. Open operation segmentation (vertical segmentation)
In order to prevent the subsequent closing operation from connecting to external impurities by mistake, it is necessary to separate them by opening operation.
# 5、开运算处理（纵向去噪，分隔）
kernel = cv.getStructuringElement(cv.MORPH_RECT, (1, 15))
Open = cv.morphologyEx(binary, cv.MORPH_OPEN, kernel)
cv.imshow('Open', Open)Before splitting :
It can be seen that these impurities have been completely integrated into the license plate, and it is very difficult to separate them later.
After splitting :
Although there are a small number of connections, they are not the previous large connections and are relatively easy to remove.
6. Closing operation merge
# 6、闭运算合并，把图像闭合、揉团，使图像区域化，便于找到车牌区域，进而得到轮廓
kernel = cv.getStructuringElement(cv.MORPH_RECT, (41, 15))
close = cv.morphologyEx(Open, cv.MORPH_CLOSE, kernel)
cv.imshow('close', close)The effect is above.
7. Corrosion/expansion in horizontal/vertical direction
# 7、膨胀/腐蚀（去噪得到车牌区域）
# 中远距离车牌识别
kernel_x = cv.getStructuringElement(cv.MORPH_RECT, (25, 7))
kernel_y = cv.getStructuringElement(cv.MORPH_RECT, (1, 11))
# 近距离车牌识别
# kernel_x = cv.getStructuringElement(cv.MORPH_RECT, (91, 31))
# kernel_y = cv.getStructuringElement(cv.MORPH_RECT, (1, 31))
# 7-1、腐蚀、膨胀（去噪）
erode_y = cv.morphologyEx(close, cv.MORPH_ERODE, kernel_y)
cv.imshow('erode_y', erode_y)
dilate_y = cv.morphologyEx(erode_y, cv.MORPH_DILATE, kernel_y)
cv.imshow('dilate_y', dilate_y)
# 7-1、膨胀、腐蚀（连接）（二次缝合）
dilate_x = cv.morphologyEx(dilate_y, cv.MORPH_DILATE, kernel_x)
cv.imshow('dilate_x', dilate_x)
erode_x = cv.morphologyEx(dilate_x, cv.MORPH_ERODE, kernel_x)
cv.imshow('erode_x', erode_x)7-1. Longitudinal direction: corrosion + expansion
Segment objects that are relatively close up and down.
7-2. Horizontal direction: expansion + corrosion
Further connect horizontal objects (following the above closed operation, connect the content of the license plate again)
8. Corrosion expansion: denoising
# 8、腐蚀、膨胀：去噪
kernel_e = cv.getStructuringElement(cv.MORPH_RECT, (25, 11))
erode = cv.morphologyEx(erode_x, cv.MORPH_ERODE, kernel_e)
cv.imshow('erode', erode)
kernel_d = cv.getStructuringElement(cv.MORPH_RECT, (25, 13))
dilate = cv.morphologyEx(erode, cv.MORPH_DILATE, kernel_d)
cv.imshow('dilate', dilate)
9. Get the outline
# 9、获取外轮廓
img_copy = img.copy()
# 9-1、得到轮廓
contours, hierarchy = cv.findContours(dilate, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
# 9-2、画出轮廓并显示
cv.drawContours(img_copy, contours, -1, (255, 0, 255), 2)
cv.imshow('Contours', img_copy) 
10. Find the outline of the license plate according to the features of the license plate
The license plate aspect ratio is generally between 3 and 4 , but I have a lot of pictures here first, and the impurities attached to some pictures have not been cleaned, so I set it between 3 and 6.5.
# 10、遍历所有轮廓，找到车牌轮廓
i = 0
for contour in contours:
# 10-1、得到矩形区域：左顶点坐标、宽和高
rect = cv.boundingRect(contour)
# 10-2、判断宽高比例是否符合车牌标准，截取符合图片
if rect[2]>rect[3]*3 and rect[2]<rect[3]*6.5:
# 截取车牌并显示
print(rect)
img = img[(rect[1]-5):(rect[1]+rect[3]+5), (rect[0]-5):(rect[0]+rect[2]+5)] #高，宽
try:
cv.imshow('license plate%d-%d' % (count, i), img)
# cv.imwrite('img%d-%d.jpg'%(count, i), img)
i += 1
except:
pass
cv.waitKey(0)Medium and long distance :
Normal license plate success rate: 100% (tested 11 cases, all successful)
Slanted license plate success rate: 67% (Test 2 cases, 3 license plates, the smallest license plate recognition failure) (but the processing is not very good)
Success rate of short-distance license plate: 0% (Test 2 cases, all failed)
close range :
(Success rate at close range is 50%, because only 2 were done)
the code
# 车牌识别
import cv2 as cv
import numpy as np
import os
# 提取车牌（形态学）
def Morph_Distinguish(img):
# 1、转灰度图
gray = cv.cvtColor(img, cv.COLOR_RGB2GRAY)
cv.imshow('gray', gray)
# 2、顶帽运算
# gray = cv.equalizeHist(gray)
kernel = cv.getStructuringElement(cv.MORPH_RECT, (17,17))
tophat = cv.morphologyEx(gray, cv.MORPH_TOPHAT, kernel)
cv.imshow('tophat', tophat)
# 3、Sobel算子提取y方向边缘（揉成一坨）
y = cv.Sobel(tophat, cv.CV_16S, 1, 0)
absY = cv.convertScaleAbs(y)
cv.imshow('absY', absY)
# 4、自适应二值化（阈值自己可调）
ret, binary = cv.threshold(absY, 75, 255, cv.THRESH_BINARY)
cv.imshow('binary', binary)
# 5、开运算分割（纵向去噪，分隔）
kernel = cv.getStructuringElement(cv.MORPH_RECT, (1, 15))
Open = cv.morphologyEx(binary, cv.MORPH_OPEN, kernel)
cv.imshow('Open', Open)
# 6、闭运算合并，把图像闭合、揉团，使图像区域化，便于找到车牌区域，进而得到轮廓
kernel = cv.getStructuringElement(cv.MORPH_RECT, (41, 15))
close = cv.morphologyEx(Open, cv.MORPH_CLOSE, kernel)
cv.imshow('close', close)
# 7、膨胀/腐蚀（去噪得到车牌区域）
# 中远距离车牌识别
kernel_x = cv.getStructuringElement(cv.MORPH_RECT, (25, 7))
kernel_y = cv.getStructuringElement(cv.MORPH_RECT, (1, 11))
# 近距离车牌识别
# kernel_x = cv.getStructuringElement(cv.MORPH_RECT, (79, 15))
# kernel_y = cv.getStructuringElement(cv.MORPH_RECT, (1, 31))
# 7-1、腐蚀、膨胀（去噪）
erode_y = cv.morphologyEx(close, cv.MORPH_ERODE, kernel_y)
cv.imshow('erode_y', erode_y)
dilate_y = cv.morphologyEx(erode_y, cv.MORPH_DILATE, kernel_y)
cv.imshow('dilate_y', dilate_y)
# 7-1、膨胀、腐蚀（连接）（二次缝合）
dilate_x = cv.morphologyEx(dilate_y, cv.MORPH_DILATE, kernel_x)
cv.imshow('dilate_x', dilate_x)
erode_x = cv.morphologyEx(dilate_x, cv.MORPH_ERODE, kernel_x)
cv.imshow('erode_x', erode_x)
# 8、腐蚀、膨胀：去噪
kernel_e = cv.getStructuringElement(cv.MORPH_RECT, (25, 9))
erode = cv.morphologyEx(erode_x, cv.MORPH_ERODE, kernel_e)
cv.imshow('erode', erode)
kernel_d = cv.getStructuringElement(cv.MORPH_RECT, (25, 11))
dilate = cv.morphologyEx(erode, cv.MORPH_DILATE, kernel_d)
cv.imshow('dilate', dilate)
# 9、获取外轮廓
img_copy = img.copy()
# 9-1、得到轮廓
contours, hierarchy = cv.findContours(dilate, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
# 9-2、画出轮廓并显示
cv.drawContours(img_copy, contours, -1, (255, 0, 255), 2)
cv.imshow('Contours', img_copy)
# 10、遍历所有轮廓，找到车牌轮廓
i = 0
for contour in contours:
# 10-1、得到矩形区域：左顶点坐标、宽和高
rect = cv.boundingRect(contour)
# 10-2、判断宽高比例是否符合车牌标准，截取符合图片
if rect[2]>rect[3]*3 and rect[2]<rect[3]*7:
# 截取车牌并显示
print(rect)
img = img[(rect[1]-5):(rect[1]+rect[3]+5), (rect[0]-5):(rect[0]+rect[2]+5)] #高，宽
try:
cv.imshow('license plate%d-%d' % (count, i), img)
cv.imwrite('car_licenses/img%d-%d.jpg'%(count, i), img)
i += 1
except:
pass
cv.waitKey(0)
if __name__ == '__main__':
global count
count=0
# 遍历文件夹中的每张图片（车）
for car in os.listdir('cars'):
# 1、获取路径
path = 'cars/'+'car'+str(count)+'.jpg'
# 2、获取图片
img = cv.imread(path)
# 3、定位车牌
Morph_Distinguish(img) #形态学提取车牌
count += 1
cv.waitKey(0)The proxy purchase is only for reference. At present, it can only extract the license plate of the car at a certain distance. It is too far or too close. It is not suitable for tilting, and it is difficult to apply it to real life. The situation in real life is much more complicated. What is there Suggestions or criticisms are free to speak out. (Slant processing may be added here later)
References
[Project combat] License plate recognition project based on OpenCV template matching_哔哩哔哩_bilibili