Preface
The purpose of license plate recognition is a traffic license recognition system, which is used in traffic application scenarios such as taking pictures of traffic violations and parking tolls. The process of license plate recognition can be divided into two parts: license plate positioning and character recognition. Traditional license plate positioning algorithms often use binarization Segmentation plus edge extraction, and then select the target. The connection method and the aspect ratio of the license plate are often used to determine the real license plate. After locating the license plate position, OCR character segmentation is performed. For high-precision license plate positioning and character recognition, currently introduced Deep learning algorithm for precise identification and processing. However, what this article is going to describe is a license plate recognition implementation based on stm32 single-chip microcomputer. It is unrealistic to use the above-mentioned advanced algorithms and complex calculations. Therefore, a set of feasible license plate positioning and character segmentation and recognition algorithms can be summarized. For your reference and study.
1. Basic principle of license plate recognition
This article describes the principle of license plate recognition based on stm32 single-chip microcomputer, including license plate image positioning, character segmentation, character normalization processing, and template matching. Of course, if the chip processing performance is sufficient, rotation correction can also be added. Multi-license plate processing, etc., this paper tries to use the most simple and easy-to-understand method for image segmentation processing, and at the same time speed up the processing.
First of all, let me talk about the hardware and software implementation of most stm32F103 license plate recognition on the network. The hardware uses stm32f103rct6+OV7670&FIFO+16bit parallel LCD screen; 72M main frequency, does not support floating-point operations and other operations; the software implementation process is roughly as follows
:
( 1) OV7670 has a FIFO camera image acquisition, uses GPIO to simulate the camera timing, and directly displays the image on the LCD screen by reading the FIFO output value (2) The
LCD screen is equivalent to an image cache, and it is also displayed at the same time, by reading the LCD screen (3) License plate positioning processing
, license plate positioning commonly used binarization segmentation, corrosion expansion processing, connected domain calculation and other operations, obviously these algorithms are very difficult to implement on stm32f1, and the processing speed is too slow , therefore, use RGB to HSV color space transformation and threshold value selection to locate the license plate, and then binarize the license plate location area, the part that is not the blue license plate is the character area;
(4) License plate character segmentation processing. Character segmentation first adopts the method of row statistics and column statistics to determine the effective pixel sum of each row and column, and further determine the character area; then perform horizontal statistical segmentation, and judge through the pixels and thresholds of each column The boundary line and number of characters;
(5) License plate normalization processing, the normalization processing first extracts each character, then performs horizontal and vertical compression according to the pixel value, and finally processes it into a character with the same size as the template; and Save character data on the LCD screen;
(6) Template matching, compare the normalized characters with the characters in the template through pixel values, and determine that the character with the highest similarity is the target value; the above method is simple and effective,
but For blurred images, more noise and different lighting conditions, the recognition effect will be poor.
2. License plate recognition algorithm based on stm32
2.1 Basic hardware
The hardware adopts stm32f407vet6 (168M main frequency, 192KB RAM) + OV7670 without fifo + SPI-LCD (using F407 and above main chip with DCMI digital camera interface); processing speed and memory are both It is much stronger than F103, so it does not need to read the pixel value, even without LCD display, it can also carry out identification processing.
2.2 Recognition Algorithm Flow
The license plate recognition is divided into four steps, which are double HSV color threshold positioning, license plate rotation correction, character segmentation and extraction, and final license plate character recognition. All of them use traditional image processing algorithms, which are suitable for single-chip operation.
(1) License plate location based on double-threshold HSV color space
First, our license plate area has dual-threshold characteristics. Blue plate vehicles have the characteristics of white characters on a blue background, and green license plates have the characteristics of black characters on a green background. This feature makes It is very convenient to perform positioning segmentation based on color space (although the threshold may not be accurate due to the influence of ambient light, this method is the simplest and most effective). After segmentation by color threshold, the real license plate target is determined according to the aspect ratio of the license plate. positioning.
(2) License plate correction based on Hough transform
For the license plate after positioning, binarization processing and boundary calculation can be performed to obtain the boundary line of the license plate, and the longest line segment on the license plate boundary can be found according to the Hough transform, and the deflection of the line segment can be calculated The angle is corrected for license plate rotation, and the corrected license plate area is then segmented to increase accuracy. After the correction, it is recommended to normalize the license plate image first, that is, to crop and scale the license plate area to a fixed size for subsequent processing.
(3) Character segmentation and extraction
First filter the binarized license plate to remove some impurities, and then calculate the length of the pixel line segment projected in the horizontal and vertical directions of the license plate according to the line segment method, which can segment the area where each character is located. However, due to the influence of the license plate edge fixture and the license plate stain, it is necessary to remove the non-character area at this time. First remove the area affected by the license plate fixture, calculate the upper and lower boundaries of the license plate through the horizontal projection line, and exclude the outer edge of the license plate, and then Carry out local edge filtering, remove stains, and finally segment the license plate part, and extract it for the next step of processing;
commonly used simple filtering methods include median filtering, local mean square error filtering, Gaussian filtering, box filtering, Laplacian edge enhancement, etc. , edge extraction algorithms include sobel operator, pewrit operator, canny operator, etc., which can be comprehensively used in the process of license plate character area processing
(4) character recognition
After extracting part of the license plate characters, first normalize the characters, use the image scaling algorithm to adjust the character size to a uniform size, and then perform character fingerprint conversion (convert 8bit to 1bit, which is convenient for character fingerprint comparison), and finally The template matching algorithm is used to calculate the similarity between the target character and the template. Here, the pixel-by-pixel comparison method can be used, or the cosine distance calculation can be used. Finally, the template with the highest similarity is marked, and the final result can be obtained after all the characters of the license plate are processed. .
2.3 Image processing algorithm
The following describes several algorithms that can be used for license plate image processing, among which LAB color threshold processing can be more accurate than HSV color threshold, which can be used as a reference (1)
RGB to LAB color space
direct table lookup method is the fastest and easiest , here is a direct reference to the algorithm in openmv, which can be viewed on the github of openmv.
#define COLOR_RGB565_TO_R8(pixel) \
({ \
__typeof__ (pixel) __pixel = (pixel); \
__pixel = (__pixel >> 8) & 0xF8; \
__pixel | (__pixel >> 5); \
})
#define COLOR_RGB565_TO_G8(pixel) \
({ \
__typeof__ (pixel) __pixel = (pixel); \
__pixel = (__pixel >> 3) & 0xFC; \
__pixel | (__pixel >> 6); \
})
#define COLOR_RGB565_TO_B8(pixel) \
({ \
__typeof__ (pixel) __pixel = (pixel); \
__pixel = (__pixel << 3) & 0xF8; \
__pixel | (__pixel >> 5); \
})
int8_t imlib_rgb565_to_l(uint16_t pixel)
{
float r_lin = xyz_table[COLOR_RGB565_TO_R8(pixel)];
float g_lin = xyz_table[COLOR_RGB565_TO_G8(pixel)];
float b_lin = xyz_table[COLOR_RGB565_TO_B8(pixel)];
float y = ((r_lin * 0.2126f) + (g_lin * 0.7152f) + (b_lin * 0.0722f)) * (1.0f / 100.000f);
y = (y>0.008856f) ? fast_cbrtf(y) : ((y * 7.787037f) + 0.137931f);
return fast_floorf(116 * y) - 16;
}
int8_t imlib_rgb565_to_a(uint16_t pixel)
{
float r_lin = xyz_table[COLOR_RGB565_TO_R8(pixel)];
float g_lin = xyz_table[COLOR_RGB565_TO_G8(pixel)];
float b_lin = xyz_table[COLOR_RGB565_TO_B8(pixel)];
float x = ((r_lin * 0.4124f) + (g_lin * 0.3576f) + (b_lin * 0.1805f)) * (1.0f / 095.047f); float y = ((r_lin * 0.2126f) + (g_lin * 0.7152f) + (b_lin * 0.0722f)) * (1.0f / 100.000f);
x = (x>0.008856f) ? fast_cbrtf(x) : ((x * 7.787037f) + 0.137931f); y = (y>0.008856f) ? fast_cbrtf(y) : ((y * 7.787037f) + 0.137931f);
return fast_floorf(500 * (x-y));
}
int8_t imlib_rgb565_to_b(uint16_t pixel)
{
float r_lin = xyz_table[COLOR_RGB565_TO_R8(pixel)];
float g_lin = xyz_table[COLOR_RGB565_TO_G8(pixel)];
float b_lin = xyz_table[COLOR_RGB565_TO_B8(pixel)];
float y = ((r_lin * 0.2126f) + (g_lin * 0.7152f) + (b_lin * 0.0722f)) * (1.0f / 100.000f); float z = ((r_lin * 0.0193f) + (g_lin * 0.1192f) + (b_lin * 0.9505f)) * (1.0f / 108.883f);
y = (y>0.008856f) ? fast_cbrtf(y) : ((y * 7.787037f) + 0.137931f); z = (z>0.008856f) ? fast_cbrtf(z) : ((z * 7.787037f) + 0.137931f);
return fast_floorf(200 * (y-z));
}
3）领域插值法图像缩放
void PicZoom_y8(unsigned char* Dst_y8,unsigned short Dst_width,unsigned short Dst_height, unsigned char*Src_y8,unsigned short Src_width,unsigned short Src_height)
{
unsigned short y=0;
unsigned short x=0;
if ((0Dst_width)||(0Dst_height)||(0Src_width)||(0Src_height)) return;
unsigned short xrIntFloat_16=(Src_width<<8)/Dst_width+1; //扩大倍数
unsigned short yrIntFloat_16=(Src_height<<8)/Dst_height+1;
unsigned char* pDstLine=Dst_y8;
unsigned short srcy_16=0;
for (y=0;y<Dst_height;++y)
{
unsigned char* pSrcLine=((unsigned char*)((unsigned char*)Src_y8+Src_width*(srcy_16>>8)));
unsigned short srcx_16=0;
for (x=0;x<Dst_width;++x)
{
pDstLine[x]=pSrcLine[srcx_16>>8];
srcx_16+=xrIntFloat_16;
}
srcy_16+=yrIntFloat_16;
pDstLine+=Dst_width;
}
}
(4) Template matching algorithm
/**
* @function 欧几里得距离计算，用于图片相似度计算
* @param[in] src1和src2，必须是相同大小灰度图片
* @param[out] 欧几里得距离
* @retval ERROR -1 错误
* @par 2021年5月28日 zhengmf
*/
float Euclidean_Distance(unsigned char *Src1,unsigned char *Src2,int length,float Euclideandis)
{
if(Src1==NULL||Src2==NULL)
{
return -1;
}
int sum=0;
int i=0;
for(i=0;i<length;i++)
{
sum+=(int)pow((*Src1-*Src2),2);
Src1++;
Src2++;
}
Euclideandis=(float)sqrt(sum);
return Euclideandis;
}
/**
* @function 余弦相似度计算，用于图片相似度计算
* @param[in] src1和src2，必须是相同大小灰度图片
* @param[out] 余弦相似度
* @retval 0 相似度小于0
* @retval ERROR -1 错误
* @retval CosineSimilar
* @par 2021年5月28日 zhengmf
*/
float Cosine_Similarity(unsigned char *Src1,unsigned char *Src2,int length,float CosineSimilar)
{
if(Src1==NULL||Src2==NULL)
{
return -1;
}
int sum=0,sum1=0,sum2=0;
float temp0=0,temp1=0;
int i=0;
for(i=0;i<length;i++)
{
sum+=(int)(*Src1)*(*Src2);
sum1+=(int)pow((*Src1),2);
sum2+=(int)pow((*Src2),2);
Src1++;
Src2++;
}
if(sum<=0)
{
return 0;
}
temp0=(float)(sqrt(sum1));
temp1=(float)(sqrt(sum2));
CosineSimilar=(float)((sum/temp0)/temp1);
return CosineSimilar;
}
/**
* @function 皮尔逊相似度计算，用于图片相似度计算
* @param[in] src1和src2，必须是相同大小灰度图片
* @param[out] 皮尔逊相似度
* @retval ERROR -1 错误
* @par 2021年5月28日 zhengmf
*/
float Pearson_Correlation(unsigned char *Src1,unsigned char *Src2,int length,float PearsonSimilar)
{
if(Src1==NULL||Src2==NULL)
{
return -1;
}
unsigned char aver1=0,aver2=0;
int sum=0,sum1=0,sum2=0;
float temp0=0,temp1=0;
int i=0;
for(i=0;i<length;i++)
{
sum1+=*Src1;
sum2+=*Src2;
Src1++;
Src2++;
}
aver1=(unsigned char)(sum1/length);
aver2=(unsigned char)(sum2/length);
sum1=0;
sum2=0;
for(i=0;i<length;i++)
{
sum+=(int)(*Src1-aver1)*(*Src2-aver2);
sum1+=(int)pow((*Src1-aver1),2);
sum2+=(int)pow((*Src2-aver2),2);
Src1++;
Src2++;
}
if(sum<=0)
{
return 0;
}
temp0=(float)(sqrt(sum1));
temp1=(float)(sqrt(sum2));
PearsonSimilar=(float)((sum/temp0)/temp1);
return PearsonSimilar;
}