This report uses MATLAB functions to design and implement a license plate recognition system. The basic principle of the license plate recognition system is: input the image containing the vehicle license plate captured by the mobile phone into the computer for preprocessing, then search, detect, and locate the license plate, and segment the rectangular area containing the license plate characters, and then compare the license plate characters Perform binarization and split it into individual characters, and then match them with the created character template one by one, and output the number of the license plate number after the match is successful. The workflow of license plate recognition is as follows:
1. Image preprocessing
The specific operations of preprocessing are size adjustment, noise filtering, and adjustment to a uniform size to facilitate parameter setting for subsequent processing, improving positioning accuracy and recognition accuracy. The regular size function is imresize(I,[row,col])
followed by image smoothing and filtering. The smoothing filter of RGB image needs to extract the three color channels of R, G, and B respectively, and filter them separately. Here, a 3x3 median filter operator is used to filter the three color channels separately, and then the cat function is used to integrate the three color channels.
the code
%% 加载图片
I=imread('Lisence.jpg');
figure(1),imshow(I);title('original image');%显示车牌原图
%% RGB 转 Gray
I1=rgb2gray(I);
figure(2),subplot(121),imshow(I1);title('gray image');
figure(2),subplot(122),imhist(I1);title('the histogram of the picture');
%% 使用 roberts 算子进行边缘检测
I2=edge(I1,'roberts',0.18,'both');% select the threshold=0.18
figure(3),imshow(I2);title('roberts operator edge detection image');
%% 腐蚀操作
se=[1;1;1];
I3=imerode(I2,se);%对图像进行腐蚀操作，即膨胀的反操作
figure(4),imshow(I3);title('corrosion image');
%% 平滑图片
se = strel('rectangle',[25,25]);
I4 = imclose(I3,se);%图形聚类、填充图形
figure(5),imshow(I4);title('smoothing image');
%% pick out the small objects
I5=bwareaopen(I4,2000);%remove the part smaller than 2000
figure(6),imshow(I5);title('remove the small objects');
2. License plate positioning License
plate positioning is based on the characteristics of the blue background color of the license plate, that is, the color distinction method. Therefore, it is very important to determine the blue RGB value range of the license plate background color. First open a picture of the license plate and check the RGB value of the background color of the license plate. The background color of the license plate is blue, so the B value is higher, and the R and G values are smaller. The preliminary consideration of the RGB range of the license plate background color should be: R<=RT, G<=GT, B>=BT (RT, GT, BT They are the color thresholds of the RGB three-color channels respectively) Query the corresponding information from the Internet, and actually take points to check the RGB color values, and finally determine the judgment threshold.
the code
%% 车牌定位
[y,x,z]=size(I5);
myI=double(I5);
tic % begin timing
Blue_y = zeros(y,1); % generate a zero matrix of y*1
for i=1:y
for j=1:x
if(myI(i,j,1)==1)
Blue_y(i,1)=Blue_y(i,1)+1;
end
end
end
[temp, MaxY]=max(Blue_y);
PY1=MaxY;
while(Blue_y(PY1,1)>=5&&(PY1>1))
PY1 = PY1-1;
end
PY2 = MaxY;
while(Blue_y(PY2,1)>=5&&(PY2<y))
PY2 = PY2+1;
end
IY=I(PY1:PY2,:,:);
%行方向车牌区域确定
%X方向
Blue_x=zeros(1,x);
for j=1:x
for i=PY1:PY2
if(myI(i,j,1)==1)
Blue_x(1,j)=Blue_x(1,j)+1;
end
end
end
PX1 = 1;
while(Blue_x(1,PX1)<3&&(PX1<x))
PX1=PX1+1;
end
PX2=x;
while((Blue_x(1,PX2)<3)&&(PX2>PX1))
PX2=PX2-1;
end
PX1=PX1-1;
PX2=PX2+1;
dw=I(PY1:PY2-8,PX1:PX2,:);
t=toc;
figure(7),subplot(121),imshow(IY),title('Line direction areas');
figure(7),subplot(122),imshow(dw),title('positioning color images');
3. License plate area processing
Perform grayscale conversion, binarization, mean value filtering, erosion and expansion, and character segmentation on the segmented color license plate image to separate the individual character images that make up the license plate number from the license plate image, and process the segmented characters Preprocessing (binarization, normalization), then analysis and extraction, and recognition of the segmented character image to give the license plate number in text form.
4. Character Segmentation
In the process of automatic license plate recognition, character segmentation plays a role in linking the past and the future. It performs character segmentation on the basis of the previous license plate location, and then uses the segmentation results for character recognition. There are many algorithms for character recognition. Because the interval between characters is large, there will be no character sticking. Therefore, the method used in this processing is to find a block with continuous text. If the length is greater than a certain threshold, it is considered that the block consists of two characters. , need to split. Generally, the segmented text needs to be further processed to meet the needs of character recognition in the next step. But for the recognition of the license plate, the purpose of correct recognition can be achieved without too much processing. Here only normalization is done, followed by post-processing.
the code
%% 字符分割与识别
% 车牌的进一步处理
imwrite(dw, 'dw.jpg');
a = imread('dw.jpg');%读取
b = rgb2gray(a);
imwrite(b,'gray_license_plate.jpg');
figure(8);subplot(321),imshow(b),title(车牌灰度图像);
g_max=double(max(max(b)));
g_min=double(min(min(b)));
T=round(g_max-(g_max-gmin)/3); %T为二值化的阈值
[m,n]=size(b);
d=(double(b)>=T); %d=二值图像
imwrite(d,'binary_license_plate.jpg');
subplot(322),imshow(d),title('before filtering binary license plate');
% 均值滤波前
% 滤波
h=fspecial('average',3);
% 建立预定义的滤波算子，average为均值滤波，模板尺寸为3x3
d=im2bw(round(filter2(h,d))); %使用指定的滤波器h对h进行d即均值滤波
imwrite(d,'after_average_license_plate.jpg');
% 对图像进行操作
% 膨胀或腐蚀
se = strel('square',3); % 使用一个3x3的正方形结果元素对象创建的图像进行膨胀
se = eye(2); %eye(n) returns the n-by-n identity matrix 单位矩阵
[m,n] = size(d);%返回矩阵b的尺寸信息，并存储在m,n中
if bwarea(d)/m/n >= 0.365 %计算二值图像中对象的总面积与整个面积相比是否大于0.365
d=imerode(d,se); %如果大于0.365则图像进行腐蚀
elseif bwarea(d)/m/n <= 0.235
d = imdilate(d,se); %如果小于则实现膨胀操作
end
imwrite(d,'expansion_or_corrosion_the_license_plate.jpg');
subplot(324),imshow(d),title('expansion or corrosion the license plate');
% 字符分割
% 寻找连续有文字的块，若长度大于某阈值，则认为该块有两个字符，需要分割
% 首先创建子函数qiege和getword，然后调用子程序，将车牌的字符分割并且进行归一化
d = qiege(d);
[m,n]=size(d);
subplot(325),imshow(d),title(n);
k1=1;k2=1;s=sum(d);j=1;
while j~=n
while s(j)==0
j=j+1;
end
k1=j;
while s(j)~=0 && j<=n-1
j=j+1;
end
k2=j-1;
if k2-k1>=round(n/6.5)
[val,num]=min(sum(d(:,[k1+5:k2-5])));
d(:,k1+num+5)=0; % 分割
end
end
% 再分割
d=qiege(d);
% 切割出7个字符
y1=10;y2=0.25;flag=0;word1[];
while flag==0
[m,n]=size(d);
left=1;wide=0;
while sum(d(:,wide+1))~=0
wide=wide+1;
end
if wide<y1 %认为是左侧干扰
d(:,[1:wide])=0;
d=qiege(d);
else
temp=qiege(imcrop(d,[1,1,wide,m]));
[m,n]=size(temp);
all=sum(sum(temp));
two_thirds=sum(sum(temp([round(m/3):2*round(m/3)],:)));
if two_thirds/all>y2
flag=1;word1=temp; % 第一个字符
end
d(:,[1:wide])=0;d=qiege(d);
end
end
[word2,d]=getword(d); % 分割第二个字符
[word3,d]=getword(d); % 分割第三个字符
[word4,d]=getword(d); % 分割第四个字符
[word5,d]=getword(d); % 分割第五个字符
[word6,d]=getword(d); % 分割第六个字符
[word7,d]=getword(d); % 分割第七个字符
figure(9);
subplot(271),imshow(word1),title('1');
subplot(272),imshow(word2),title('2');
subplot(273),imshow(word3),title('3');
subplot(274),imshow(word4),title('4');
subplot(275),imshow(word5),title('5');
subplot(276),imshow(word6),title('6');
subplot(277),imshow(word7),title('7');
[m,n]=size(word1);
% 归一化大小为40x20
word1=imresize(word1,[40,20]);
word2=imresize(word2,[40,20]);
word3=imresize(word3,[40,20]);
word4=imresize(word4,[40,20]);
word5=imresize(word5,[40,20]);
word6=imresize(word6,[40,20]);
word7=imresize(word7,[40,20]);
subplot(278),imshow(word1),title('1');
subplot(279),imshow(word2),title('2');
subplot(2,7,10),imshow(word3),title('3');
subplot(2,7,11),imshow(word4),title('4');
subplot(2,7,12),imshow(word5),title('5');
subplot(2,7,13),imshow(word6),title('6');
subplot(2,7,14),imshow(word7),title('7');
%% 车牌识别
liccode=char(['0':'9', 'A':'Z', '苏豫陕鲁京辽浙']); %建立自动识别字符表
SubBw2=zeros(40,20);
l=1;
for I=1:7
ii=int2str(I);
t=imread([ii,'.jpg']);
SegBw2=imresize(t,[40,20],'nearest');
SegBw2=double(SegBw2)>20;
if I==1 % 第一个汉字识别
kmin=37;
kmax=43;
elseif I==2 %第二位A~Z字母识别
kmin=11;
kmax=36;
else l>=3 % 第三位以后是字母或数字识别
kmin=1;
kmax=36;
end
for k2=kmin:kmax
fname=strcat('字符模板\', liccode(k2), '.jpg');
SamBw2=imread(fname);
SamBw2=double(SamBw2)>1;
for i=1:40
for j=1:20
SubBw2(i,jP)=SegBw2(i,j)-SamBw2(i,j);
end
end
% 以上相当于两幅图相减得到第三幅图
Dmax=0;
for k1=1:40
for l1=1:20
if(SubBw2(k1,l1)>0 | SubBw2(k1,l1)<0)
Dmax=Dmax+1;
end
end
end
Error(k2)=Dmax;
end
Error1=Error(kmin:kmax);
MinError=min(Error1);
findc=find(Error1==MinError);
Code(l*2-1)=liccode(findc(1)+kmin-1);
Code(l*2)='';
l=l+1;
end
figure(10),imshwo(dw),title(['车牌号码', Code], 'Color', 'b');
5. Character recognition
Template matching is one of the most representative basic methods in image recognition methods. It combines several feature quantities extracted from the image or image area f(I, j) to be recognized with the template T(I, j ) to compare the corresponding feature quantities one by one, and calculate the normalized correlation between them. The one with the largest correlation indicates the highest degree of similarity. The image can be classified into this category, and the distance between the image and the template feature can also be calculated. , using the minimum distance method to determine the class it belongs to.
Here, the method of subtraction is used to find out which character is most similar to the character in the template, and then find the output with the largest similarity. There are generally seven characters in a car license plate. The first character of most license plates is a Chinese character, which represents the province to which the vehicle belongs, followed by letters and numbers. The difference between license plate character recognition and general text recognition is that it has a limited number of characters. Chinese characters A total of about 50, including 26 uppercase English letters and 10 numbers. For the convenience of the experiment, combined with the characteristics of the license plate selected in this design, only a template with 7 numbers, 26 letters and 10 numbers was established. Other templates are designed in a similar way.
Firstly, the character template is taken, and then the character to be recognized is matched with the template in turn, and subtracted from the template character, the more 0s are obtained, the more the match will be. Save the number of 0 values of each subtracted image, which is the recognized result.
Sixth, optimization
can consider using the HSV color space system, optimize the code, add the picture brightness detection link, determine the lighting conditions of the picture scene, match different color judgment systems according to the light intensity, and then optimize the positioning judgment; introduce the edge detection link, The boundary of the blue area detected by the lock is roughly rectangular and judged as the license plate area to distinguish other blue areas in the image; during morphological processing, the character area and border size, border ratio, interval size, and other parameters of the standard license plate are further added. Optimize the string region extraction link; in the character recognition and matching link, you can add multiple sets of character templates, add character templates from different angles, match multiple times, and improve the recognition accuracy rate, etc.
Reference: Digital Image Processing - License Plate Recognition Based on Matlab
Reference: Digital Image Processing License Plate Recognition Course Design