1. Introduction to License Plate Recognition
With the development of science and technology, artificial intelligence technology is more and more widely used in our lives. Many scenarios such as face recognition, license plate recognition, and target recognition have been applied, which has greatly improved our quality of life. The efficiency is greatly improved, and a large amount of labor is also saved.
Today we will talk about the task of license plate recognition. After years of development, license plate recognition technology has diversified technical routes. Let us introduce two of them:
The first, single character recognition
First of all, there is a sensor sensing area on the ground. When a vehicle passes by, it will automatically take a photo, and then preprocess the photo to turn it into a grayscale image, remove noise, etc., remove some interference factors, and reduce the size of the image. It is convenient for edge extraction in the later stage;
Secondly, use edge extraction techniques, such as canny operator and sobel operator, to extract the outline of the image, and remove most of the unsuspected ones according to the aspect ratio of the license plate, which is about 1:2.5; then, use the gray value of the image in the The projection in the horizontal direction and the vertical direction cuts out the characters;
Finally, use template matching or a trained deep learning model for recognition, recognize each character individually and concatenate them to obtain the entire license plate number.
The bottleneck of this method is how to accurately cut out each character, which involves many image processing techniques, such as Gaussian blur, image sharpening, image expansion, gamma change, affine transformation, etc. Currently, for English characters The recognition accuracy of numbers and numbers is about 96%, and the accuracy of Chinese character recognition is about 95%.
The second method, the entire license plate recognition
First of all, like the first type of license plate recognition, the sensor is used for vehicle sensing, and the entire vehicle is photographed;
Secondly, use the trained license plate detection algorithm, such as the YOLO algorithm or other target detection algorithms to detect the license plate. In order to improve the accuracy of license plate recognition, image preprocessing technology can be used to process the image and reduce interference items;
Finally, use the trained deep learning model to recognize the entire license plate. Such as LPRNet, Darknet, etc.
2. Realization of license plate recognition technology
## The first method - use pytesseract for identification
pytesseract is an OCR recognition tool. The specific pytesseract installation can be installed on Baidu. The accuracy of recognition obtained by directly using this tool is still a bit low, and the desired effect cannot be achieved. It needs to be trained with license plate data before it can be used.
import cv2 as cv
from PIL import Image
import pytesseract as tess
def recoginse_text(image):
"""
步骤：
1、灰度，二值化处理
2、形态学操作去噪
3、识别
:param image:
:return:
"""
# 灰度 二值化
gray = cv.cvtColor(image,cv.COLOR_BGR2GRAY)
# 如果是白底黑字 建议 _INV
ret,binary = cv.threshold(gray,0,255,cv.THRESH_BINARY_INV| cv.THRESH_OTSU)
# 形态学操作 (根据需要设置参数（1，2）)
kernel = cv.getStructuringElement(cv.MORPH_RECT,(1,2)) #去除横向细线
morph1 = cv.morphologyEx(binary,cv.MORPH_OPEN,kernel)
kernel = cv.getStructuringElement(cv.MORPH_RECT, (2, 1)) #去除纵向细线
morph2 = cv.morphologyEx(morph1,cv.MORPH_OPEN,kernel)
cv.imshow("Morph",morph2)
# 黑底白字取非，变为白底黑字（便于pytesseract 识别）
cv.bitwise_not(morph2,morph2)
textImage = Image.fromarray(morph2)
# 图片转文字
text=tess.image_to_string(textImage)
print("识别结果：%s"%text)
def main():
# 读取需要识别的数字字母图片，并显示读到的原图
src = cv.imread(r'C:\Users\lenovo\Desktop\00A8CX87_5.jpg')
cv.imshow("src",src)
# 识别
recoginse_text(src)
cv.waitKey(0)
cv.destroyAllWindows()
if __name__=="__main__":
main()
## The second identification method - PaddleOCR
PaddleOCR is an OCR module under Baidu paddlepaddle. You can use it to recognize license plates. You can directly download the pre-trained model for recognition. It has a good accuracy rate for numbers and mother-in-law, but the effect for Chinese recognition is not ideal. You need your own data set Do retraining.
To use this model to train license plate recognition, a large amount of license plate data is required. Manual collection not only takes a long time, but also requires a relatively large cost to involve the license plates of each province.
Another way is to use image processing technology for data generation, plus image enhancement technology to make data sets, which will be mentioned in our third chapter of recognition methods.
from paddleocr import PaddleOCR
import os
# ocr = PaddleOCR(use_gpu=False, use_angle_cls=True, lang="ch")
ocr = PaddleOCR(use_gpu=True, use_angle_cls=True, lang="ch")
path=r'E:\车牌数据'
results=[]
for file in os.listdir(path):
img_path =os.path.join(path,file)
result = ocr.ocr(img_path, cls=True)
if len(result)>0:
results.append([img_path,result[0][1][0]])
else:
results.append([img_path])
Export the results to an Excel folder
import pandas as pd
resultss=pd.DataFrame(results)
resultss.to_excel(r'C:\Users\lenovo\Desktop\11.xlsx')
## The third method - LPRNet algorithm
Since it takes a lot of time to collect data that satisfies deep learning, one way is to generate a large number of simulated data sets through data generation.
Next, we can use the LPRNet algorithm for license plate recognition. There are two versions of tensorflow and torch downloaded from GitHub.
The first step is to generate the blue license plate data set
. The second step is to modify the image name according to the algorithm model requirements and put it in the designated folder.
The third step is to train and test the model.
## (1) Blue license plate generation
import os
import cv2 as cv
import numpy as np
from math import *
from PIL import ImageFont
from PIL import Image
from PIL import ImageDraw
index = {"京": 0, "沪": 1, "津": 2, "渝": 3, "冀": 4, "晋": 5, "蒙": 6, "辽": 7, "吉": 8, "黑": 9,
"苏": 10, "浙": 11, "皖": 12, "闽": 13, "赣": 14, "鲁": 15, "豫": 16, "鄂": 17, "湘": 18, "粤": 19,
"桂": 20, "琼": 21, "川": 22, "贵": 23, "云": 24, "藏": 25, "陕": 26, "甘": 27, "青": 28, "宁": 29,
"新": 30, "0": 31, "1": 32, "2": 33, "3": 34, "4": 35, "5": 36, "6": 37, "7": 38, "8": 39,
"9": 40, "A": 41, "B": 42, "C": 43, "D": 44, "E": 45, "F": 46, "G": 47, "H": 48, "J": 49,
"K": 50, "L": 51, "M": 52, "N": 53, "P": 54, "Q": 55, "R": 56, "S": 57, "T": 58, "U": 59,
"V": 60, "W": 61, "X": 62, "Y": 63, "Z": 64}
chars = ["京", "沪", "津", "渝", "冀", "晋", "蒙", "辽", "吉", "黑",
"苏", "浙", "皖", "闽", "赣", "鲁", "豫", "鄂", "湘", "粤",
"桂", "琼", "川", "贵", "云", "藏", "陕", "甘", "青", "宁",
"新", "0", "1", "2", "3", "4", "5", "6", "7", "8",
"9", "A", "B", "C", "D", "E", "F", "G", "H", "J",
"K", "L", "M", "N", "P", "Q", "R", "S", "T", "U",
"V", "W", "X", "Y", "Z"]
def AddSmudginess(img, Smu):
"""
模糊处理
:param img: 输入图像
:param Smu: 模糊图像
:return: 添加模糊后的图像
"""
rows = r(Smu.shape[0] - 50)
cols = r(Smu.shape[1] - 50)
adder = Smu[rows:rows + 50, cols:cols + 50]
adder = cv.resize(adder, (50, 50))
img = cv.resize(img,(50,50))
img = cv.bitwise_not(img)
img = cv.bitwise_and(adder, img)
img = cv.bitwise_not(img)
return img
def rot(img, angel, shape, max_angel):
"""
添加透视畸变
"""
size_o = [shape[1], shape[0]]
size = (shape[1]+ int(shape[0] * cos((float(max_angel ) / 180) * 3.14)), shape[0])
interval = abs(int(sin((float(angel) / 180) * 3.14) * shape[0]))
pts1 = np.float32([[0, 0], [0, size_o[1]], [size_o[0], 0], [size_o[0], size_o[1]]])
if angel > 0:
pts2 = np.float32([[interval, 0], [0, size[1]], [size[0], 0], [size[0] - interval, size_o[1]]])
else:
pts2 = np.float32([[0, 0], [interval, size[1]], [size[0] - interval, 0], [size[0], size_o[1]]])
M = cv.getPerspectiveTransform(pts1, pts2)
dst = cv.warpPerspective(img, M, size)
return dst
def rotRandrom(img, factor, size):
"""
添加放射畸变
:param img: 输入图像
:param factor: 畸变的参数
:param size: 图片目标尺寸
:return: 放射畸变后的图像
"""
shape = size
pts1 = np.float32([[0, 0], [0, shape[0]], [shape[1], 0], [shape[1], shape[0]]])
pts2 = np.float32([[r(factor), r(factor)], [r(factor), shape[0] - r(factor)], [shape[1] - r(factor), r(factor)],
[shape[1] - r(factor), shape[0] - r(factor)]])
M = cv.getPerspectiveTransform(pts1, pts2)
dst = cv.warpPerspective(img, M, size)
return dst
def tfactor(img):
"""
添加饱和度光照的噪声
"""
hsv = cv.cvtColor(img,cv.COLOR_BGR2HSV)
hsv[:, :, 0] = hsv[:, :, 0] * (0.8 + np.random.random() * 0.2)
hsv[:, :, 1] = hsv[:, :, 1] * (0.3 + np.random.random() * 0.7)
hsv[:, :, 2] = hsv[:, :, 2] * (0.2 + np.random.random() * 0.8)
img = cv.cvtColor(hsv, cv.COLOR_HSV2BGR)
return img
def random_envirment(img, noplate_bg):
"""
添加自然环境的噪声, noplate_bg为不含车牌的背景图
"""
bg_index = r(len(noplate_bg))
env = cv.imread(noplate_bg[bg_index])
env = cv.resize(env, (img.shape[1], img.shape[0]))
bak = (img == 0)
bak = bak.astype(np.uint8) * 255
inv = cv.bitwise_and(bak, env)
img = cv.bitwise_or(inv, img)
return img
def GenCh(f, val):
"""
生成中文字符
"""
img = Image.new("RGB", (45, 70), (255, 255, 255)) #白色
# img = Image.new("RGB", (45, 70), (0, 0, 0)) #黑色
draw = ImageDraw.Draw(img)
# draw.text((0, 3), val, (0, 0, 0), font=f)
draw.text((0, 3), val, (0, 0, 0), font=f)
img = img.resize((23, 70))
A = np.array(img)
return A
def GenCh1(f, val):
"""
生成英文字符
"""
img =Image.new("RGB", (23, 70), (255,255,255))#白色
# img =Image.new("RGB", (23, 70), (0, 0, 0))#黑色
draw = ImageDraw.Draw(img)
draw.text((0, 2), val, (0,125,125), font=f) # val.decode('utf-8')
A = np.array(img)
return A
def AddGauss(img, level):
"""
添加高斯模糊
"""
return cv.blur(img, (level * 2 + 1, level * 2 + 1))
def r(val):
return int(np.random.random() * val)
def AddNoiseSingleChannel(single):
"""
添加高斯噪声
"""
diff = 255 - single.max()
noise = np.random.normal(0, 1 + r(6), single.shape)
noise = (noise - noise.min()) / (noise.max() - noise.min())
noise *= diff
# noise= noise.astype(np.uint8)
dst = single + noise
return dst
def addNoise(img): # sdev = 0.5,avg=10
img[:, :, 0] = AddNoiseSingleChannel(img[:, :, 0])
img[:, :, 1] = AddNoiseSingleChannel(img[:, :, 1])
img[:, :, 2] = AddNoiseSingleChannel(img[:, :, 2])
return img
class GenPlate:
def __init__(self, fontCh, fontEng, NoPlates):
self.fontC = ImageFont.truetype(fontCh, 43, 0)
self.fontE = ImageFont.truetype(fontEng, 60, 0)
self.img = np.array(Image.new("RGB", (226, 70),(255, 255, 255)))
# self.img = np.array(Image.new("RGB", (226, 70),(0, 0, 0)))
self.bg = cv.resize(cv.imread(r"C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\generate_picture\images\blue.bmp"), (226, 70))
# template.bmp:车牌背景图
self.smu = cv.imread(r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\images\\smu2.jpg")
# smu2.jpg:模糊图像
self.noplates_path = []
for parent, parent_folder, filenames in os.walk(NoPlates):
for filename in filenames:
path = parent + "\\" + filename
self.noplates_path.append(path)
def draw(self, val):
offset = 2
self.img[0:70, offset+8:offset+8+23] = GenCh(self.fontC, val[0])
self.img[0:70, offset+8+23+6:offset+8+23+6+23] = GenCh1(self.fontE, val[1])
for i in range(5):
base = offset + 8 + 23 + 6 + 23 + 17 + i * 23 + i * 6
self.img[0:70, base:base+23] = GenCh1(self.fontE, val[i+2])
return self.img
def generate(self, text):
if len(text) == 7:
fg = self.draw(text) # decode(encoding="utf-8")
fg = cv.bitwise_not(fg)
com = cv.bitwise_or(fg, self.bg)
com = rot(com, r(30)-10, com.shape,30)
com = rotRandrom(com, 5, (com.shape[1], com.shape[0]))
com = tfactor(com)
com = random_envirment(com, self.noplates_path)
com = AddGauss(com, 1+r(4))
com = addNoise(com)
return com
@staticmethod
def genPlateString(pos, val):
"""
生成车牌string，存为图片
生成车牌list，存为label
"""
plateStr = ""
plateList=[]
box = [0, 0, 0, 0, 0, 0, 0]
if pos != -1:
box[pos] = 1
for unit, cpos in zip(box, range(len(box))):
if unit == 1:
plateStr += val
plateList.append(val)
else:
if cpos == 0:
plateStr += chars[r(31)]
plateList.append(plateStr)
elif cpos == 1:
plateStr += chars[41 + r(24)]
plateList.append(plateStr)
else:
plateStr += chars[31 + r(34)]
plateList.append(plateStr)
plate = [plateList[0]]
b = [plateList[i][-1] for i in range(len(plateList))]
plate.extend(b[1:7])
return plateStr, plate
@staticmethod
def genBatch(batchsize, outputPath, size):
"""
将生成的车牌图片写入文件夹，对应的label写入label.txt
:param batchsize: 批次大小
:param outputPath: 输出图像的保存路径
:param size: 输出图像的尺寸
:return: None
"""
if not os.path.exists(outputPath):
os.mkdir(outputPath)
outfile = open(r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\label_blue.txt',
'w', encoding='utf-8')
for i in range(batchsize):
plateStr, plate = G.genPlateString(-1, -1)
img = G.generate(plateStr)
img = cv.resize(img, size)
content = "".join(plate)
cv.imwrite(outputPath + "\\" + str(i).zfill(2) + ".jpg", img)
outfile.write("./train/"+str(i).zfill(2)+".jpg"+'\t'+content +"\n")
G = GenPlate(r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\font\platech.ttf",
r'E:\AI_projects\vehicle license plate recognition\car_recognition\data\font\platechar.ttf',
r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\NoPlates")
G.genBatch(1000, r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test_in_blue', (272, 72))
How many license plate data need to be generated can be changed by modifying the number of the last line of code
## (2) Produce yellow license plates
The step of generating the license plate is to write the characters on the pure background image. The blue license plate is to write the white characters on the blue license plate. You can directly write the characters on the base plate, but the yellow license plate cannot be simply written. Write black characters, and the result of writing is that the characters are hidden and need special treatment.
The correct approach is:
Step 1, construct a pure black license plate the size of a license plate, and write white characters on it
Step 2, reverse the black and white, reverse the characters and background of the
card A yellow card with no characters is combined with the reversed license plate, and then various random combinations such as Gaussian blur, light change, and rotation angle are used to simulate the image taken and processed in real life.
#黄色车牌生成
import os
import cv2 as cv
import numpy as np
from math import *
from PIL import ImageFont
from PIL import Image
from PIL import ImageDraw
index = {"京": 0, "沪": 1, "津": 2, "渝": 3, "冀": 4, "晋": 5, "蒙": 6, "辽": 7, "吉": 8, "黑": 9,
"苏": 10, "浙": 11, "皖": 12, "闽": 13, "赣": 14, "鲁": 15, "豫": 16, "鄂": 17, "湘": 18, "粤": 19,
"桂": 20, "琼": 21, "川": 22, "贵": 23, "云": 24, "藏": 25, "陕": 26, "甘": 27, "青": 28, "宁": 29,
"新": 30, "0": 31, "1": 32, "2": 33, "3": 34, "4": 35, "5": 36, "6": 37, "7": 38, "8": 39,
"9": 40, "A": 41, "B": 42, "C": 43, "D": 44, "E": 45, "F": 46, "G": 47, "H": 48, "J": 49,
"K": 50, "L": 51, "M": 52, "N": 53, "P": 54, "Q": 55, "R": 56, "S": 57, "T": 58, "U": 59,
"V": 60, "W": 61, "X": 62, "Y": 63, "Z": 64}
chars = ["京", "沪", "津", "渝", "冀", "晋", "蒙", "辽", "吉", "黑",
"苏", "浙", "皖", "闽", "赣", "鲁", "豫", "鄂", "湘", "粤",
"桂", "琼", "川", "贵", "云", "藏", "陕", "甘", "青", "宁",
"新", "0", "1", "2", "3", "4", "5", "6", "7", "8",
"9", "A", "B", "C", "D", "E", "F", "G", "H", "J",
"K", "L", "M", "N", "P", "Q", "R", "S", "T", "U",
"V", "W", "X", "Y", "Z"]
def AddSmudginess(img, Smu):
"""
模糊处理
:param img: 输入图像
:param Smu: 模糊图像
:return: 添加模糊后的图像
"""
rows = r(Smu.shape[0] - 50)
cols = r(Smu.shape[1] - 50)
adder = Smu[rows:rows + 50, cols:cols + 50]
adder = cv.resize(adder, (50, 50))
img = cv.resize(img,(50,50))
img = cv.bitwise_not(img)
img = cv.bitwise_and(adder, img)
img = cv.bitwise_not(img)
return img
def rot(img, angel, shape, max_angel):
"""
添加透视畸变
"""
size_o = [shape[1], shape[0]]
size = (shape[1]+ int(shape[0] * cos((float(max_angel ) / 180) * 3.14)), shape[0])
interval = abs(int(sin((float(angel) / 180) * 3.14) * shape[0]))
pts1 = np.float32([[0, 0], [0, size_o[1]], [size_o[0], 0], [size_o[0], size_o[1]]])
if angel > 0:
pts2 = np.float32([[interval, 0], [0, size[1]], [size[0], 0], [size[0] - interval, size_o[1]]])
else:
pts2 = np.float32([[0, 0], [interval, size[1]], [size[0] - interval, 0], [size[0], size_o[1]]])
M = cv.getPerspectiveTransform(pts1, pts2)
dst = cv.warpPerspective(img, M, size)
return dst
def rotRandrom(img, factor, size):
"""
添加放射畸变
:param img: 输入图像
:param factor: 畸变的参数
:param size: 图片目标尺寸
:return: 放射畸变后的图像
"""
shape = size
pts1 = np.float32([[0, 0], [0, shape[0]], [shape[1], 0], [shape[1], shape[0]]])
pts2 = np.float32([[r(factor), r(factor)], [r(factor), shape[0] - r(factor)], [shape[1] - r(factor), r(factor)],
[shape[1] - r(factor), shape[0] - r(factor)]])
M = cv.getPerspectiveTransform(pts1, pts2)
dst = cv.warpPerspective(img, M, size)
return dst
def tfactor(img):
"""
添加饱和度光照的噪声
"""
hsv = cv.cvtColor(img,cv.COLOR_BGR2HSV)
hsv[:, :, 0] = hsv[:, :, 0] * (0.8 + np.random.random() * 0.2)
hsv[:, :, 1] = hsv[:, :, 1] * (0.3 + np.random.random() * 0.7)
hsv[:, :, 2] = hsv[:, :, 2] * (0.2 + np.random.random() * 0.8)
img = cv.cvtColor(hsv, cv.COLOR_HSV2BGR)
return img
def random_envirment(img, noplate_bg):
"""
添加自然环境的噪声, noplate_bg为不含车牌的背景图
"""
bg_index = r(len(noplate_bg))
env = cv.imread(noplate_bg[bg_index])
env = cv.resize(env, (img.shape[1], img.shape[0]))
bak = (img == 0)
bak = bak.astype(np.uint8) * 255
inv = cv.bitwise_and(bak, env)
img = cv.bitwise_or(inv, img)
return img
def GenCh(f, val):
"""
生成中文字符
"""
img = Image.new("RGB", (45, 70), (255, 255, 255)) #白色
# img = Image.new("RGB", (45, 70), (0, 0, 0)) #黑色
draw = ImageDraw.Draw(img)
# draw.text((0, 3), val, (0, 0, 0), font=f)
draw.text((0, 3), val, (0, 0, 0), font=f)
img = img.resize((23, 70))
A = np.array(img)
return A
def GenCh1(f, val):
"""
生成英文字符
"""
img =Image.new("RGB", (23, 70), (255,255,255))#白色
# img =Image.new("RGB", (23, 70), (0, 0, 0))#黑色
draw = ImageDraw.Draw(img)
draw.text((0, 2), val, (0,0,0), font=f) # val.decode('utf-8')
A = np.array(img)
return A
def AddGauss(img, level):
"""
添加高斯模糊
"""
return cv.blur(img, (level * 2 + 1, level * 2 + 1))
def r(val):
return int(np.random.random() * val)
def AddNoiseSingleChannel(single):
"""
添加高斯噪声
"""
diff = 255 - single.max()
noise = np.random.normal(0, 1 + r(6), single.shape)
noise = (noise - noise.min()) / (noise.max() - noise.min())
noise *= diff
# noise= noise.astype(np.uint8)
dst = single + noise
return dst
def addNoise(img): # sdev = 0.5,avg=10
img[:, :, 0] = AddNoiseSingleChannel(img[:, :, 0])
img[:, :, 1] = AddNoiseSingleChannel(img[:, :, 1])
img[:, :, 2] = AddNoiseSingleChannel(img[:, :, 2])
return img
class GenPlate:
def __init__(self, fontCh, fontEng, NoPlates):
self.fontC = ImageFont.truetype(fontCh, 43, 0)
self.fontE = ImageFont.truetype(fontEng, 60, 0)
self.img = np.array(Image.new("RGB", (226, 70),(255, 255, 255)))
# self.img = np.array(Image.new("RGB", (226, 70),(0, 0, 0)))
self.bg = cv.resize(cv.imread(r"C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\generate_picture\images\black.jpg"), (226, 70))
# template.bmp:车牌背景图
self.smu = cv.imread(r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\images\\smu2.jpg")
# smu2.jpg:模糊图像
self.noplates_path = []
for parent, parent_folder, filenames in os.walk(NoPlates):
for filename in filenames:
path = parent + "\\" + filename
self.noplates_path.append(path)
def draw(self, val):
offset = 2
self.img[0:70, offset+8:offset+8+23] = GenCh(self.fontC, val[0])
self.img[0:70, offset+8+23+6:offset+8+23+6+23] = GenCh1(self.fontE, val[1])
for i in range(5):
base = offset + 8 + 23 + 6 + 23 + 17 + i * 23 + i * 6
self.img[0:70, base:base+23] = GenCh1(self.fontE, val[i+2])
return self.img
def generate(self, text):
if len(text) == 7:
fg = self.draw(text) # decode(encoding="utf-8")
fg = cv.bitwise_not(fg)
com = cv.bitwise_or(fg, self.bg)
# com = rot(com, r(60)-30, com.shape,30)
# com = rotRandrom(com, 10, (com.shape[1], com.shape[0]))
# com = tfactor(com)
# com = random_envirment(com, self.noplates_path)
# com = AddGauss(com, 1+r(4))
# com = addNoise(com)
return com
@staticmethod
def genPlateString(pos, val):
"""
生成车牌string，存为图片
生成车牌list，存为label
"""
plateStr = ""
plateList=[]
box = [0, 0, 0, 0, 0, 0, 0]
if pos != -1:
box[pos] = 1
for unit, cpos in zip(box, range(len(box))):
if unit == 1:
plateStr += val
plateList.append(val)
else:
if cpos == 0:
plateStr += chars[r(31)]
plateList.append(plateStr)
elif cpos == 1:
plateStr += chars[41 + r(24)]
plateList.append(plateStr)
else:
plateStr += chars[31 + r(34)]
plateList.append(plateStr)
plate = [plateList[0]]
b = [plateList[i][-1] for i in range(len(plateList))]
plate.extend(b[1:7])
return plateStr, plate
@staticmethod
def genBatch(batchsize, outputPath, size):
"""
将生成的车牌图片写入文件夹，对应的label写入label.txt
:param batchsize: 批次大小
:param outputPath: 输出图像的保存路径
:param size: 输出图像的尺寸
:return: None
"""
if not os.path.exists(outputPath):
os.mkdir(outputPath)
outfile = open(r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\label_yellow.txt',
'w', encoding='utf-8')
for i in range(batchsize):
plateStr, plate = G.genPlateString(-1, -1)
img = G.generate(plateStr)
img = cv.resize(img, size)
content = "".join(plate)
cv.imwrite(outputPath + "\\" + str(i).zfill(2) + ".jpg", img)
outfile.write("./train/"+str(i).zfill(2)+".jpg"+'\t'+content +"\n")
G = GenPlate(r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\font\platech.ttf",
r'E:\AI_projects\vehicle license plate recognition\car_recognition\data\font\platechar.ttf',
r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\NoPlates")
G.genBatch(1000, r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test_in_yellow', (272, 72))
character and background inversion
#将车牌黑色反转
import cv2
import random
def addNoise(img): # sdev = 0.5,avg=10
img[:, :, 0] = AddNoiseSingleChannel(img[:, :, 0])
img[:, :, 1] = AddNoiseSingleChannel(img[:, :, 1])
img[:, :, 2] = AddNoiseSingleChannel(img[:, :, 2])
return img
sample_cp=cv.resize(cv2.imread(r"C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\generate_picture\images\yellow.bmp"), (272, 72))
path=r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test_in_yellow'
for file in os.listdir(path):
if file.endswith('.jpg'):
image=os.path.join(path,file)
img=cv2.imread(image)
for i in range(72):
for j in range(272):
if img[i][j][0]>50:
img[i][j][0]=0 #黑色
img[i][j][1]=0
img[i][j][2]=0
else:
img[i][j][0]=255 #白色
img[i][j][1]=255
img[i][j][2]=255
for i in range(72):
for j in range(272):
if img[i][j][0]<50 and img[i][j][1]<50 and img[i][j][2]<50:
img[i][j][0]=img[i][j][0]
img[i][j][1]=img[i][j][1]
img[i][j][2]=img[i][j][2]
else:
img[i][j][0]=sample_cp[i][j][0] #+random.randint(0,9)
img[i][j][1]=sample_cp[i][j][1] #+random.randint(0,5)
img[i][j][2]=sample_cp[i][j][2] #+random.randint(0,10)
addNoise(img)
images=os.path.join(path,file)
cv2.imwrite(images,img)
Combining characters and bottom plates, and performing various methods such as distortion, blurring, and rotation, a variety of license plate images are constructed.
import cv2
import numpy as np
from math import cos,sin
import os
def rot(img, angel, shape, max_angel):
"""
添加透视畸变
"""
size_o = [shape[1], shape[0]]
size = (shape[1]+ int(shape[0] * cos((float(max_angel ) / 180) * 3.14)), shape[0])
interval = abs(int(sin((float(angel) / 180) * 3.14) * shape[0]))
pts1 = np.float32([[0, 0], [0, size_o[1]], [size_o[0], 0], [size_o[0], size_o[1]]])
if angel > 0:
pts2 = np.float32([[interval, 0], [0, size[1]], [size[0], 0], [size[0] - interval, size_o[1]]])
else:
pts2 = np.float32([[0, 0], [interval, size[1]], [size[0] - interval, 0], [size[0], size_o[1]]])
M = cv2.getPerspectiveTransform(pts1, pts2)
dst = cv2.warpPerspective(img, M, size)
return dst
def rotRandrom(img, factor, size):
"""
添加放射畸变
:param img: 输入图像
:param factor: 畸变的参数
:param size: 图片目标尺寸
:return: 放射畸变后的图像
"""
shape = size
pts1 = np.float32([[0, 0], [0, shape[0]], [shape[1], 0], [shape[1], shape[0]]])
pts2 = np.float32([[r(factor), r(factor)], [r(factor), shape[0] - r(factor)], [shape[1] - r(factor), r(factor)],
[shape[1] - r(factor), shape[0] - r(factor)]])
M = cv2.getPerspectiveTransform(pts1, pts2)
dst = cv2.warpPerspective(img, M, size)
return dst
def r(val):
return int(np.random.random() * val)
def tfactor(img):
"""
添加饱和度光照的噪声
"""
hsv = cv2.cvtColor(img,cv2.COLOR_BGR2HSV)
hsv[:, :, 0] = hsv[:, :, 0] * (0.8 + np.random.random() * 0.2)
hsv[:, :, 1] = hsv[:, :, 1] * (0.3 + np.random.random() * 0.7)
hsv[:, :, 2] = hsv[:, :, 2] * (0.2 + np.random.random() * 0.8)
img = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)
return img
def random_envirment(img, noplate_bg):
"""
添加自然环境的噪声, noplate_bg为不含车牌的背景图
"""
bg_index = r(len(noplate_bg))
env = cv2.imread(noplate_bg[bg_index])
env = cv2.resize(env, (img.shape[1], img.shape[0]))
bak = (img == 0)
bak = bak.astype(np.uint8) * 255
inv = cv2.bitwise_and(bak, env)
img = cv2.bitwise_or(inv, img)
return img
def AddGauss(img, level):
return cv2.blur(img, (level * 2 + 1, level * 2 + 1))
def AddNoiseSingleChannel(single):
"""
添加高斯噪声
"""
diff = 255 - single.max()
noise = np.random.normal(0, 1 + r(6), single.shape)
noise = (noise - noise.min()) / (noise.max() - noise.min())
noise *= diff
# noise= noise.astype(np.uint8)
dst = single + noise
return dst
def addNoise(img): # sdev = 0.5,avg=10
img[:, :, 0] = AddNoiseSingleChannel(img[:, :, 0])
img[:, :, 1] = AddNoiseSingleChannel(img[:, :, 1])
img[:, :, 2] = AddNoiseSingleChannel(img[:, :, 2])
return img
path0=r"C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\generate_picture\NoPlates"
noplates_path=[]
for file in os.listdir(path0):
noplates_path.append(os.path.join(path0,file))
# bg=cv2.resize(cv2.imread(r"C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\generate_picture\images\new.jpg"), (272, 72))
# fg=cv2.imread(r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\picture_new\00.jpg')
# com=fg
# com = rot(com, r(30)-10, com.shape,30)
# com = rotRandrom(com, 5, (com.shape[1], com.shape[0]))
# com = tfactor(com)
# # com = random_envirment(com,noplates_path)
# # com = AddGauss(com, 1+r(4))
# com = addNoise(com)
# cv2.imshow('com',com)
# cv2.waitKey(0)
# cv2.destroyAllWindows()
# fg = cv2.bitwise_not(fg)
# com = cv2.bitwise_or(fg, bg)
path=r"C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test_in_yellow"
path1=r"C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test_in_yellow"
for file in os.listdir(path):
if file.endswith('.jpg'):
fg=cv2.imread(os.path.join(path,file))
com=fg
com = rot(com, r(30)-10, com.shape,30)
com = rotRandrom(com, 5, (com.shape[1], com.shape[0]))
com = tfactor(com)
com = random_envirment(com,noplates_path)
com = AddGauss(com, 1+r(4))
com = addNoise(com)
images=os.path.join(path1,file)
cv2.imwrite(images,com)
## (3) Production of new energy license plates
The generation method of the new energy license plate is the same as that of the yellow license plate, but the length of the new energy license plate is one bit longer than that of the blue license plate and the yellow license plate, and the third character is designated as D or F, but we do not consider it when generating. Just modify the code to let the program generate license plates of the same length.
#黄色车牌生成
import os
import cv2 as cv
import numpy as np
from math import *
from PIL import ImageFont
from PIL import Image
from PIL import ImageDraw
index = {"京": 0, "沪": 1, "津": 2, "渝": 3, "冀": 4, "晋": 5, "蒙": 6, "辽": 7, "吉": 8, "黑": 9,
"苏": 10, "浙": 11, "皖": 12, "闽": 13, "赣": 14, "鲁": 15, "豫": 16, "鄂": 17, "湘": 18, "粤": 19,
"桂": 20, "琼": 21, "川": 22, "贵": 23, "云": 24, "藏": 25, "陕": 26, "甘": 27, "青": 28, "宁": 29,
"新": 30, "0": 31, "1": 32, "2": 33, "3": 34, "4": 35, "5": 36, "6": 37, "7": 38, "8": 39,
"9": 40, "A": 41, "B": 42, "C": 43, "D": 44, "E": 45, "F": 46, "G": 47, "H": 48, "J": 49,
"K": 50, "L": 51, "M": 52, "N": 53, "P": 54, "Q": 55, "R": 56, "S": 57, "T": 58, "U": 59,
"V": 60, "W": 61, "X": 62, "Y": 63, "Z": 64}
chars = ["京", "沪", "津", "渝", "冀", "晋", "蒙", "辽", "吉", "黑",
"苏", "浙", "皖", "闽", "赣", "鲁", "豫", "鄂", "湘", "粤",
"桂", "琼", "川", "贵", "云", "藏", "陕", "甘", "青", "宁",
"新", "0", "1", "2", "3", "4", "5", "6", "7", "8",
"9", "A", "B", "C", "D", "E", "F", "G", "H", "J",
"K", "L", "M", "N", "P", "Q", "R", "S", "T", "U",
"V", "W", "X", "Y", "Z"]
def AddSmudginess(img, Smu):
"""
模糊处理
:param img: 输入图像
:param Smu: 模糊图像
:return: 添加模糊后的图像
"""
rows = r(Smu.shape[0] - 20) #50
cols = r(Smu.shape[1] - 20)
adder = Smu[rows:rows + 20, cols:cols + 20]
adder = cv.resize(adder, (20, 20))
img = cv.resize(img,(20,20))
img = cv.bitwise_not(img)
img = cv.bitwise_and(adder, img)
img = cv.bitwise_not(img)
return img
def rot(img, angel, shape, max_angel):
"""
添加透视畸变
"""
size_o = [shape[1], shape[0]]
size = (shape[1]+ int(shape[0] * cos((float(max_angel ) / 180) * 3.14)), shape[0])
interval = abs(int(sin((float(angel) / 180) * 3.14) * shape[0]))
pts1 = np.float32([[0, 0], [0, size_o[1]], [size_o[0], 0], [size_o[0], size_o[1]]])
if angel > 0:
pts2 = np.float32([[interval, 0], [0, size[1]], [size[0], 0], [size[0] - interval, size_o[1]]])
else:
pts2 = np.float32([[0, 0], [interval, size[1]], [size[0] - interval, 0], [size[0], size_o[1]]])
M = cv.getPerspectiveTransform(pts1, pts2)
dst = cv.warpPerspective(img, M, size)
return dst
def rotRandrom(img, factor, size):
"""
添加放射畸变
:param img: 输入图像
:param factor: 畸变的参数
:param size: 图片目标尺寸
:return: 放射畸变后的图像
"""
shape = size
pts1 = np.float32([[0, 0], [0, shape[0]], [shape[1], 0], [shape[1], shape[0]]])
pts2 = np.float32([[r(factor), r(factor)], [r(factor), shape[0] - r(factor)], [shape[1] - r(factor), r(factor)],
[shape[1] - r(factor), shape[0] - r(factor)]])
M = cv.getPerspectiveTransform(pts1, pts2)
dst = cv.warpPerspective(img, M, size)
return dst
def tfactor(img):
"""
添加饱和度光照的噪声
"""
hsv = cv.cvtColor(img,cv.COLOR_BGR2HSV)
hsv[:, :, 0] = hsv[:, :, 0] * (0.8 + np.random.random() * 0.2)
hsv[:, :, 1] = hsv[:, :, 1] * (0.3 + np.random.random() * 0.7)
hsv[:, :, 2] = hsv[:, :, 2] * (0.2 + np.random.random() * 0.8)
img = cv.cvtColor(hsv, cv.COLOR_HSV2BGR)
return img
def random_envirment(img, noplate_bg):
"""
添加自然环境的噪声, noplate_bg为不含车牌的背景图
"""
bg_index = r(len(noplate_bg))
env = cv.imread(noplate_bg[bg_index])
env = cv.resize(env, (img.shape[1], img.shape[0]))
bak = (img == 0)
bak = bak.astype(np.uint8) * 255
inv = cv.bitwise_and(bak, env)
img = cv.bitwise_or(inv, img)
return img
def GenCh(f, val):
"""
生成中文字符
"""
img = Image.new("RGB", (45, 70), (255, 255, 255)) #白色
# img = Image.new("RGB", (45, 70), (0, 0, 0)) #黑色
draw = ImageDraw.Draw(img)
# draw.text((0, 3), val, (0, 0, 0), font=f)
draw.text((0, 2), val, (0, 0, 0), font=f)
img = img.resize((19, 70))
A = np.array(img)
return A
def GenCh1(f, val):
"""
生成英文字符
"""
img =Image.new("RGB", (19, 70), (255,255,255))#白色
# img =Image.new("RGB", (23, 70), (0, 0, 0))#黑色
draw = ImageDraw.Draw(img)
draw.text((0, 2), val, (0,0,0), font=f) # val.decode('utf-8')
A = np.array(img)
return A
def AddGauss(img, level):
"""
添加高斯模糊
"""
return cv.blur(img, (level * 2 + 1, level * 2 + 1))
def r(val):
return int(np.random.random() * val)
def AddNoiseSingleChannel(single):
"""
添加高斯噪声
"""
diff = 255 - single.max()
noise = np.random.normal(0, 1 + r(6), single.shape)
noise = (noise - noise.min()) / (noise.max() - noise.min())
noise *= diff
# noise= noise.astype(np.uint8)
dst = single + noise
return dst
def addNoise(img): # sdev = 0.5,avg=10
img[:, :, 0] = AddNoiseSingleChannel(img[:, :, 0])
img[:, :, 1] = AddNoiseSingleChannel(img[:, :, 1])
img[:, :, 2] = AddNoiseSingleChannel(img[:, :, 2])
return img
class GenPlate:
def __init__(self, fontCh, fontEng, NoPlates):
self.fontC = ImageFont.truetype(fontCh, 43, 0)
self.fontE = ImageFont.truetype(fontEng, 52, 0) #60--52
self.img = np.array(Image.new("RGB", (226, 70),(255, 255, 255)))
# self.img = np.array(Image.new("RGB", (226, 70),(0, 0, 0)))
self.bg = cv.resize(cv.imread(r"C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\generate_picture\images\black.jpg"), (226, 70))
# template.bmp:车牌背景图
self.smu = cv.imread(r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\images\\smu2.jpg")
# smu2.jpg:模糊图像
self.noplates_path = []
for parent, parent_folder, filenames in os.walk(NoPlates):
for filename in filenames:
path = parent + "\\" + filename
self.noplates_path.append(path)
def draw(self, val):
offset = 2
self.img[0:70, offset+6:offset+6+19] = GenCh(self.fontC, val[0])
self.img[0:70, offset+6+19+7:offset+6+19+7+19] = GenCh1(self.fontE, val[1])
for i in range(6):
base = offset + 6 + 19 + 7 + 19 + 17 + i * 19 + i * 7
self.img[0:70, base:base+19] = GenCh1(self.fontE, val[i+2]) #23--19 6--7 8--6
return self.img
def generate(self, text):
if len(text) == 8:
fg = self.draw(text) # decode(encoding="utf-8")
fg = cv.bitwise_not(fg)
com = cv.bitwise_or(fg, self.bg)
# com = rot(com, r(60)-30, com.shape,30)
# com = rotRandrom(com, 10, (com.shape[1], com.shape[0]))
# com = tfactor(com)
# com = random_envirment(com, self.noplates_path)
# com = AddGauss(com, 1+r(4))
# com = addNoise(com)
return com
@staticmethod
def genPlateString(pos, val):
"""
生成车牌string，存为图片
生成车牌list，存为label
"""
plateStr = ""
plateList=[]
box = [0, 0, 0, 0, 0, 0, 0,0]
if pos != -1:
box[pos] = 1
for unit, cpos in zip(box, range(len(box))):
if unit == 1:
plateStr += val
plateList.append(val)
else:
if cpos == 0:
plateStr += chars[r(31)] #31
plateList.append(plateStr)
elif cpos == 1:
plateStr += chars[41 + r(24)] #41 24
plateList.append(plateStr)
else:
plateStr += chars[31 + r(34)] #31 34
plateList.append(plateStr)
plate = [plateList[0]]
b = [plateList[i][-1] for i in range(len(plateList))]
plate.extend(b[1:8])
return plateStr, plate
@staticmethod
def genBatch(batchsize, outputPath, size):
"""
将生成的车牌图片写入文件夹，对应的label写入label.txt
:param batchsize: 批次大小
:param outputPath: 输出图像的保存路径
:param size: 输出图像的尺寸
:return: None
"""
if not os.path.exists(outputPath):
os.mkdir(outputPath)
outfile = open(r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\label_new.txt',
'w', encoding='utf-8')
for i in range(batchsize):
plateStr, plate = G.genPlateString(-1, -1)
img = G.generate(plateStr)
img = cv.resize(img, size)
content = "".join(plate)
cv.imwrite(outputPath + "\\" + str(i).zfill(2) + ".jpg", img)
outfile.write("./train/"+str(i).zfill(2)+".jpg"+'\t'+content +"\n")
G = GenPlate(r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\font\platech.ttf",
r'E:\AI_projects\vehicle license plate recognition\car_recognition\data\font\platechar.ttf',
r"E:\AI_projects\vehicle license plate recognition\car_recognition\data\NoPlates")
G.genBatch(1000, r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test_in_new', (272, 72))
The above generates part of the code for the license plate, and other codes refer to the previous part
(4) Rename various license plate names to make it suitable for training note generation requirements
#将以数字命名的图片转换为省份编号+车牌命名
import os
import pandas as pd
path=r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\label_yellow.txt'
num=[]
with open(path,'rb') as f:
lines=f.readlines()
for line in lines:
data=line.decode('utf-8')#.encode('gb2312')
data1=data.split('\t')[0]
data2=data.split('\t')[1].split('\r')[0]
num.append([data1,data2])
number=pd.DataFrame(num,columns=['p','j'])
CHARS = ['京', '沪', '津', '渝', '冀', '晋', '蒙', '辽', '吉', '黑',
'苏', '浙', '皖', '闽', '赣', '鲁', '豫', '鄂', '湘', '粤',
'桂', '琼', '川', '贵', '云', '藏', '陕', '甘', '青', '宁',
'新',
'0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'J', 'K',
'L', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'U', 'V',
'W', 'X', 'Y', 'Z','_'
]
dicts = {'A01':'京','A02':'津','A03':'沪','B02':'蒙',
'S01':'皖','S02':'闽','S03':'粤','S04':'甘',
'S05': '贵', 'S06': '鄂', 'S07': '冀', 'S08': '黑', 'S09': '湘',
'S10': '豫', 'S12': '吉', 'S13': '苏', 'S14': '赣', 'S15': '辽',
'S17': '川', 'S18': '鲁', 'S22': '浙',
'S30':'渝', 'S31':'晋', 'S32':'桂', 'S33':'琼', 'S34':'云', 'S35':'藏',
'S36':'陕','S37':'青', 'S38':'宁', 'S39':'新'}
path_root=r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test_in_yellow'
for k in range(len(number)):
for t in range(len(CHARS)):
pp=number['p'][k].split('/train/')[1]
if number['j'][k][0]==CHARS[t]:
original_name=os.path.join(path_root,pp)
d=list(dicts.keys())[t]
pd=number['j'][k]
dd=d+'_'+pd[1:]+'_0'+'.jpg'
post_name=os.path.join(path_root,dd)
if os.path.exists(original_name):
if not os.path.exists(post_name):
os.rename(original_name,post_name)
(5) Count the number of license plates in each province
Mainly to confirm the balance of the number of Chinese characters
import os
path=r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\picture_real'
S=[]
for file in os.listdir(path):
a=file.split('_')[0]
S.append(a)
Set=set(S)
Dict={}
for item in Set:
Dict.update({item:S.count(item)})
print(Dict)
(6) Move the license plate to the training and testing folders
import os
import shutil
path_from=r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test_in_new'
path_to=r'C:\Users\lenovo\Desktop\Plate_Recognition-LPRnet-master\Plate_Recognition-LPRnet-master\test'
for file in os.listdir(path_from):
image=os.path.join(path_from,file)
# if not os.path.join(path_to,file):
shutil.move(image,path_to)
(7) Model training and testing
Next comes model training. . . (slightly)
Summarize
Using the third method LPRNet to carry out license plate recognition is a very good project, which involves various image-related knowledge such as image processing and deep learning. After completing the training of this project, the project engineering ability has been greatly improved. It is of great help in dealing with work and problems encountered in work.
Friends who can see this seriously, please give a thumbs up!