The original text is here , refer to this for improvement
I feel that I have learned a lot, so I will make notes here.
Effect:
Table of contents
2. Morphological opening operation and morphological closing operation
2.1 The first perspective: eliminating smaller connected areas vs bridging smaller connected areas
2.2 Second angle: background noise removal vs foreground noise removal
6. The putText function prints Chinese
2. License plate identification code
1. Knowledge point learning:
1. fstream
Role: input and output files;
example:
fstream fin;
fin.open(filename, ios::in);
if (!fin.is_open())
{
cout << "can not open the file!" << endl;
return false;
}
string s;
while (std::getline(fin, s))
{
string str = s;
data_name.push_back(str);
}
fin.close();
The open function used above is introduced in detail:
void open ( const char * filename,
ios_base::openmode mode = ios_base::in | ios_base::out ); filename operation file name
mode The way to open the file, commonly used are the following two
ios::in： //文件以输入方式打开（文件数据输入到内存）
ios::out： //文件以输出方式打开（内存数据输出到文件）2. Morphological opening operation and morphological closing operation
I have never understood these two, and today I just call this opportunity to learn.
2.1 The first perspective: eliminating smaller connected areas vs bridging smaller connected areas
The functions of the morphological opening operation are as follows:
- Eliminate isolated points whose value is higher than that of neighboring points to achieve the effect of removing noise in the image;
- Eliminate smaller connected domains and retain larger connected domains;
- Disconnect the narrower neck, which separates two objects at their slender junction ;
- Smooth the boundary and outline of the connected domain without significantly changing the area of the larger connected domain;
The functions of the morphological closing operation are as follows:
- Eliminate isolated points whose value is lower than that of neighboring points to achieve the effect of removing noise in the image;
- connect two adjacent connected domains;
- Bridging narrower discontinuities and elongated gullies ;
- Remove small holes in the connected domain ;
- Like the open operation, it can also smooth the outline of the object;
2.2 Second angle: background noise removal vs foreground noise removal
On Operation: Cancels Background Noise
Closing operation: filling small holes in foreground objects, or small black spots on foreground objects
3. The approPolyDp function
The role of the function: to perform polygon fitting on the contour points of the image
The calling form of the function:
void approxPolyDP( InputArray curve,
OutputArray approxCurve,
double epsilon,
bool closed );Detailed parameter explanation:
InputArray curve: generally a point set consisting of the contour points of the image
OutputArray approxCurve: represents the output polygon point set
double epsilon: mainly indicates the accuracy of the output, which is the maximum distance between another contour point, 5,6,7,,8,,,,,
bool closed: Indicates whether the output polygon is closed
4. Bubble sort
Here is an intuitive animation display: animation
Here bubble sorting is used to sort the Rect of license plate characters:
for (size_t i =0; i< Character_ROI.size(); i++)
{
for (size_t j=0; j< Character_ROI.size() -1 -i; j++)
{
if (Character_ROI[j].rect.x > Character_ROI[j+1].rect.x)
{
License temp = Character_ROI[j];
Character_ROI[j] = Character_ROI[j+1];
Character_ROI[j+1] = temp;
}
}
}Assuming there are 5 characters, the X coordinates of their Rect are 4 1 3 0 2, now use bubble sort to sort:
5. Match the target
Here, the OpenCV absdiff function is used to calculate the pixel difference between two images to determine the similarity of the images.
In addition to template matching, other methods are based on Hu moment contour matching, and I will learn it in another blog for space reasons.
6. The putText function prints Chinese
I use OpenCV4.5.5, Ubuntun20.04, just import the header file directly.
Font file path (windows system):
/Windows/Fonts/Then copy it to a directory under the Ubuntu system.
Example:
#include <iostream>
#include <opencv2/freetype.hpp>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat src = imread("/home/jason/work/01-img/dog.png");
string text = "中华田园犬";
Ptr<cv::freetype::FreeType2> ft2;
ft2 = cv::freetype::createFreeType2();
ft2->loadFontData("/usr/share/fonts/winFonts/SIMYOU.TTF",0);
ft2->putText(src, text, Point(300, 200), 30 , Scalar(0, 0,255), 2, 8, true);
imshow("src", src);
waitKey();
return 0;
}
7. Text files, label files
I sent a private message to the blogger, but there was no reply, so I made one myself.
7.1 Text files
Pictures exported by wps word:
Extract the characters:
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
void Get_character(Mat & src, Mat & result)
{
Mat gray;
cvtColor(src, gray, COLOR_BGR2GRAY);
// 黑色的点
vector<Point> locations;
for (int x=0; x< src.cols; x++)
for (int y=0; y< src.rows; y++)
{
if(gray.at<uchar>(y, x) < 255)
{
locations.push_back(Point(x,y));
}
}
// 字符左上角 右下角
double xmin, ymin, xmax, ymax;
vector<int> xs, ys;
for(size_t i=0; i<locations.size(); i++)
{
xs.push_back(locations[i].x);
ys.push_back(locations[i].y);
}
Mat tempX(xs);
Mat tempY(ys);
Point p1;
minMaxLoc(tempX, &xmin, &xmax,0,0);
minMaxLoc(tempY, &ymin, &ymax, 0,0);
// 画框
Rect roi;
Mat temp = src.clone();
roi.x = xmin - 30;
roi.y = ymin - 30;
roi.width = xmax - xmin + 60;
roi.height = ymax - ymin + 60;
rectangle(src,roi, Scalar(255, 0,0), 1, 8);
// 扣出来
Mat ROI = temp(roi);
imshow("ROI", ROI);
result = ROI.clone();
imshow("src", src);
waitKey(10);
}
int main()
{
string tail = ".png";
string head;
string path, outpath;
string outpath_head = "/home/jason/work/01-img/car/car_roi/";
for (int i=0; i<= 69; i++)
{
if (i<10)
{
head = "/home/jason/work/01-img/car/car/0";
}
else
{
head = "/home/jason/work/01-img/car/car/";
}
path = head + to_string(i) + tail;
outpath = outpath_head + to_string(i) + tail;
Mat src = imread(path);
Mat result;
Get_character(src, result);
imwrite(outpath,result);
}
return 0;
}
7.2 Label file
2. License plate identification code
In the process of identifying Russia, I found that the font of my letters does not correspond to the font of the license plate, and there may be deviations.
what to do? I simply deduct the characters of the license plate and save it as a template!
locate.hpp
#include <iostream>
#include <opencv2/opencv.hpp>
#include<opencv2/freetype.hpp>
#include <fstream>
using namespace cv;
using namespace std;
using namespace cv;
using namespace std;
// 自定义车牌结构体
struct License
{
Mat mat; // ROI图片
Rect rect; // ROI所在矩形
};
class Locate
{
private:
// 车牌字符模板图片
vector<Mat> Dataset;
// 车牌字符名
vector<string> Data_name;
// 字体文件路径
string Font_Path;
// 车牌字符扣出来另存路径
string Character_Out_Path;
bool Read_Data(string filename, vector<Mat>& dataset);
bool Read_Data(string filename, vector<string>&data_name);
void Image_Preprocessing(Mat& gray, Mat& result);
void Morphological_Process(Mat& preprocess, Mat& result);
void Character_ROI_Preprocessing(vector<License>& License_ROI);
void Get_License_ROI(Mat &morpho, Mat &src,
vector<License>& License_ROI);
void Remove_vertial_Border(Mat& car_bord, Mat& result);
void Remove_Horizon_Border(Mat& car_bord, Mat & result);
public:
bool Set_Input(string label_Path, string template_Path,
string font_Path, string character_out_path);
void Get_License_ROI(Mat& src, vector<License>& License_ROI);
void Get_Character_ROI(vector<License>& License_ROI,
vector<vector<License>>&Character_ROI,
Mat &src, bool character_save);
int pixCount(Mat image);
void License_Recognition(vector<vector<License>>&Character_ROI,
vector<vector<int>>&result_index);
void Draw_Result(Mat &src,
vector<License>& License_ROI,
vector<vector<License>>&Character_ROI,
vector<vector<int>>&result_index);
};
locate.cpp
#include "Locate_License.h"
// 读取文件 图片
bool Locate::Read_Data(string filename, vector<Mat>& dataset)
{
vector<String> imagePathList;
glob(filename, imagePathList); // 遍历文件夹下所有文件
if (imagePathList.empty()) return false;
for (size_t i=0; i<imagePathList.size(); i++)
{
cout << imagePathList[i] << endl;
Mat image = imread(imagePathList[i]);
resize(image, image, Size(50, 100), 1, 1, INTER_LINEAR);
cvtColor(image, image, COLOR_BGR2GRAY);
threshold(image, image, 0, 255, THRESH_BINARY_INV|THRESH_OTSU); // 字符需要是白色
Mat kernel = getStructuringElement(MORPH_RECT, Size(3,3));
dilate(image, image,kernel,Point(-1,-1),1);
// imshow(to_string(i), image);
dataset.push_back(image);
}
this->Dataset = dataset;
return true;
}
//读取文件 标签
bool Locate::Read_Data(string filename, vector<string>&data_name)
{
fstream fin;
fin.open(filename, ios::in);
if(!fin.is_open())
{
cout << "can not open the file!" << endl;
return false;
}
string s;
while (getline(fin, s))
{
string str = s;
data_name.push_back(str);
}
fin.close();
this->Data_name = data_name;
return true;
}
bool Locate::Set_Input(string label_Path,
string template_Path,
string font_Path="/usr/share/fonts/winFonts/SIMYOU.TTF",
string character_out_path = "/home/jason/work/01-img/car/out")
{
this->Font_Path = font_Path;
printf("字体路径设置为: %s, 请检查该目录是否正确\n",font_Path.c_str());
this->Character_Out_Path = character_out_path;
printf("车牌字符输出路径设置为: %s, 请检查该目录是否正确\n",character_out_path.c_str());
if (Read_Data(label_Path, this->Data_name) &&
Read_Data(template_Path, this->Dataset))
{
printf("***** 成功读取模板图片、标签数据\n");
return true;
}
else
{
printf("***** err:读取模板图片、标签数据\n");
return false;
}
}
// 突出字符
void Locate::Image_Preprocessing(Mat& gray, Mat& result)
{
// 开操作,平滑作用，断开较窄的狭颈和消除细的突出物
Mat kernel = getStructuringElement(MORPH_RECT, Size(25,25));
Mat gray_blur;
morphologyEx(gray, gray_blur, MORPH_OPEN, kernel);
imshow("open1", gray_blur);
// 灰度图-开操作图，突显字符等部分
Mat rst;
subtract(gray, gray_blur, rst, Mat());
imshow("rst", rst);
// Canny算子进行边缘检测
Mat canny_Image;
Canny(rst, canny_Image, 400, 200, 3);
imshow("canny_Image", canny_Image);
result=canny_Image.clone();
}
// 通过膨胀连接相近的图像区域，
// 利用腐蚀去除孤立细小的色块，从而将所有的车牌上所有的字符都连通起来
void Locate::Morphological_Process(Mat& preprocess, Mat& result)
{
// 图片膨胀处理
Mat dilate_image, erode_image;
//自定义核：进行 x 方向的膨胀腐蚀
Mat elementX = getStructuringElement(MORPH_RECT, Size(19, 1));
Mat elementY = getStructuringElement(MORPH_RECT, Size(1, 19));
Point point(-1, -1);
dilate(preprocess, dilate_image, elementX, point, 2);
imshow("dilate1", dilate_image);
// // 闭操作，避免车牌与 其他区域联通在一起
// Mat kernel = getStructuringElement(MORPH_RECT, Size(10, 10));
// morphologyEx(dilate_image, dilate_image, MORPH_OPEN,
// kernel, Point(-1,-1),2);
// imshow("MORPH_OPEN", dilate_image);
erode(dilate_image, erode_image, elementX, point, 3);
imshow("erode1", erode_image);
dilate(erode_image, dilate_image, elementX, point, 2);
imshow("dialte2", dilate_image);
//自定义核：进行 Y 方向的膨胀腐蚀
erode(dilate_image, erode_image, elementY, point, 1);
imshow("yerode", erode_image);
dilate(erode_image, dilate_image, elementY, point, 2);
imshow("Ydilate", erode_image);
// 平滑处理
Mat median_Image;
medianBlur(dilate_image, median_Image, 15);
imshow("median1",median_Image);
medianBlur(median_Image, median_Image, 15);
imshow("median2", median_Image);
result = median_Image.clone();
}
// 扣出车牌
void Locate::Get_License_ROI(Mat &morpho, Mat &src, vector<License>& License_ROI)
{
vector<vector<Point>> contours;
findContours(morpho, contours, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE);
//
Mat temp =src.clone();
drawContours(temp, contours, -1, Scalar(255,0,0), 4);
//
double area;
for (size_t i=0; i< contours.size(); i++)
{
// 轮廓 --》 rect
Rect rect = boundingRect(contours[i]);
// 车牌的宽高比大约为3.3
double width_height = (double)rect.width/ (double)rect.height;
printf("height_width:%.2f\n", width_height);
if (width_height>2.5 && width_height < 4.0)
{
rectangle(temp, rect, Scalar(0,0, 255), 4, 8);
License temp_license = {src(rect), rect};
License_ROI.push_back(temp_license);
}
}
imshow("标出车牌",temp);
if (License_ROI.size() > 0)
{
printf("****** 共提取到 %d 块车牌\n",(int)License_ROI.size());
for (size_t i = 0; i< License_ROI.size(); i++)
{
string tempName = "第" + to_string(i) + "块车牌";
imshow(tempName, License_ROI[i].mat);
}
}
else
{
printf("****** 没有发现车牌\n");
}
}
// 从图片中扣出车牌
void Locate::Get_License_ROI(Mat& src, vector<License>& License_ROI)
{
// 灰度图
Mat gray;
cvtColor(src, gray, COLOR_BGR2GRAY);
imshow("gray", gray);
// 均衡化
equalizeHist(gray, gray);
// 突出字符，并获得canny边缘
Mat preprocess_result;
Image_Preprocessing(gray, preprocess_result);
// 将车牌字符形成一个整体
Mat morpho_image;
Morphological_Process(preprocess_result, morpho_image);
// 扣出整块车牌
Get_License_ROI(morpho_image, src, License_ROI);
}
void Locate::Remove_vertial_Border(Mat& car_bord, Mat & result)
{
Mat vline = getStructuringElement(MORPH_RECT, Size(1,car_bord.rows));
Mat dst1, temp1;
erode(car_bord, temp1, vline);
// imshow("V-erode",temp1);
dilate(temp1, dst1, vline);
// imshow("V-dilate",dst1);
subtract(car_bord, dst1, result, Mat());
// imshow("V-result",result);
}
void Locate::Remove_Horizon_Border(Mat& car_bord, Mat & result)
{
Mat hline = getStructuringElement(MORPH_RECT, Size(car_bord.rows,1));
Mat dst1, temp1;
erode(car_bord, temp1, hline);
// imshow("H-erode",temp1);
dilate(temp1, dst1, hline);
// imshow("H-dilate",dst1);
subtract(car_bord, dst1, result, Mat());
// imshow("H-result",result);
}
// 对整块车牌进行预处理，
void Locate::Character_ROI_Preprocessing(vector<License>& License_ROI)
{
for (size_t i=0; i<License_ROI.size(); i++)
{
// 灰度化
Mat gray;
cvtColor(License_ROI[i].mat, gray, COLOR_BGR2GRAY);
imshow("gray--", gray);
// // 均衡化 这里不需要用，用了方而效果不好，因为车牌中车牌字符本身就很显眼，不需要用均衡
// equalizeHist(gray, gray);
// 大津阈值化
Mat thresh;
threshold(gray, thresh, 0, 255, THRESH_BINARY|THRESH_OTSU ); // 字是白色的的
imshow("thres", thresh);
Mat hori;
Remove_Horizon_Border(thresh, hori);
Mat vert;
Remove_vertial_Border(hori,vert);
imshow("H V", vert);
Mat open;
Mat kernel = getStructuringElement(MORPH_RECT, Size(2,2));
morphologyEx(vert, open,MORPH_CLOSE, kernel, Point(-1,-1),1);
imshow("连接汉字两边", open);
License_ROI[i].mat = open.clone();
}
}
//
void Locate::Get_Character_ROI(vector<License>& License_ROI,
vector<vector<License>>&Character_ROI,
Mat &src,bool character_save=true)
{
Character_ROI_Preprocessing(License_ROI);
Mat temp = src.clone();
for (size_t j=0; j<License_ROI.size(); j++)
{
Mat temp_carbod = License_ROI[j].mat.clone();
Character_ROI.push_back({}); // 必须先添加一个空项进去
vector<vector<Point>> contours;
findContours(License_ROI[j].mat, contours, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE);
drawContours(temp, contours, -1, Scalar(255,0,0), 2, 8);
imshow("Get_Character_ROI", temp);
for (size_t i = 0; i<contours.size(); i++)
{
double area = contourArea(contours[i]);
//由于我们筛选出来的轮廓是无序的，故后续我们需要将字符重新排序
if (area > 100)
{
Rect rect = boundingRect(contours[i]);
// 计算外接矩形框高比
double ratio = double(rect.height)/ double(rect.width);
if (ratio > 1)
{
// 字符扣出来
Mat roi = License_ROI[j].mat(rect);
resize(roi, roi, Size(50, 100), 1, 1, INTER_LINEAR);
Character_ROI[j].push_back({roi, rect}); // 前面不添加一个空项进去，这就就报错
// 字符在原图画框
rectangle(temp_carbod ,rect, Scalar(255, 0, 0), 2, 8);
imshow("字符框",temp_carbod);
// 字符另外为
if (character_save)
{
threshold(roi,roi,0, 255, THRESH_BINARY_INV|THRESH_OTSU);
string outpath = this->Character_Out_Path + "/" + to_string(i) + ".png";
imwrite(outpath,roi);
}
}
}
}
//将筛选出来的字符轮廓 按照其左上角点坐标从左到右依次顺序排列
// 冒泡排序 ； 你查一下，用41302自己排下序就懂了
for (size_t k =0; k<Character_ROI.size(); k++)
{
for (size_t ii =0; ii< Character_ROI[k].size(); ii++)
{
for (size_t jj=0; jj< Character_ROI[k].size() -1 -ii; jj++)
{
if (Character_ROI[k][jj].rect.x > Character_ROI[k][jj+1].rect.x)
{
License temp = Character_ROI[k][jj];
Character_ROI[k][jj] = Character_ROI[k][jj+1];
Character_ROI[k][jj+1] = temp;
}
}
}
}
}
if (Character_ROI.size() > 0)
{
for (size_t k =0; k<Character_ROI.size(); k++)
{
printf("******* 第 %d 块车牌共扣出： %d 个字符\n", (int)k,(int)Character_ROI[k].size());
}
}
else
{
printf("***** err :第车牌没有扣出字符！\n");
}
}
int Locate::pixCount(Mat image)
{
int count =0;
if (image.channels() == 1)
{
for (int i=0; i<image.rows; i++)
{
for (int j=0; j<image.cols; j++)
{
if (image.at<uchar>(i, j) == 255) // 数的是白色像素
{
count++;
}
}
}
return count;
}
else
{
return -1;
}
}
// 识别车牌字符
// 使用OpenCV absdiff函数计算两张图像的像素差,以此来判断图像的相似程度
// 进行字符匹配的方法还有：模板匹配，基于Hu矩轮廓匹配
void Locate::License_Recognition(vector<vector<License>>&Character_ROI,
vector<vector<int>>& result_inedx)
{
for (size_t k =0; k<Character_ROI.size(); k++)
{
result_inedx.push_back({});
for (int i=0; i<Character_ROI[k].size(); i++)
{
// 车牌单个字符预处理
Mat roi_thresh;
threshold(Character_ROI[k][i].mat, roi_thresh, 0, 255, THRESH_BINARY_INV); // 车牌字符需是白色
string car = "car" + to_string(i);
imshow(car,roi_thresh);
int minCount = 1000000000;
int index = 0;
for (int j=0; j < this->Dataset.size(); j++)
{
// 计算车牌字符与模板的像素差，以此判断两张图片是否相同
Mat templa = this->Dataset[j];
Mat dst;
absdiff(roi_thresh, templa, dst);
// 白字黑底，两图像素相减，白色像素越少，两图越接近
int count = pixCount(dst);
if (count< minCount)
{
minCount = count;
index = j;
}
// imshow(to_string(j),dst);
}
string p = "templ" + to_string(i);
imshow(p, this->Dataset[index]);
result_inedx[k].push_back(index);
}
}
printf("*****共对 %d 块车牌的字符完成字符匹配\n",(int)Character_ROI.size());
}
// 显示最终效果
void Locate::Draw_Result(Mat &src, vector<License> &License_ROI,
vector<vector<License>>&Character_ROI,
vector<vector<int>>&result_index)
{
Ptr<cv::freetype::FreeType2> ft2;
ft2 = cv::freetype::createFreeType2();
ft2->loadFontData(this->Font_Path,0);
for (size_t k=0; k<License_ROI.size(); k++)
{
// 原图上框出车牌
rectangle(src, License_ROI[k].rect, Scalar(0, 255, 0), 2);
// 在原图车牌框上方上打印车牌字符
for (size_t i=0; i< Character_ROI[k].size(); i++)
{
// cout << data_name[result_index[i]] << " ";
string str = this->Data_name[result_index[k][i]];
ft2->putText(src, str,
Point(License_ROI[k].rect.x + Character_ROI[k][i].rect.x,
License_ROI[k].rect.y - Character_ROI[k][i].rect.y),
30,Scalar(255, 0, 0), 1, 8, true);
}
// cout << endl;
}
}
main.cpp
#include "Locate_License.h"
int main()
{
Mat src = imread("/home/jason/work/01-img/car.png");
if (src.empty())
{
cout << "No image!" << endl;
system("pause");
return -1;
}
Locate locate;
locate.Set_Input("/home/jason/work/01-img/car/car.txt",
"/home/jason/work/01-img/car/template",
"/usr/share/fonts/winFonts/SIMYOU.TTF",
"/home/jason/work/01-img/car/out");
vector<License> License_ROI;
locate.Get_License_ROI(src, License_ROI);
vector<vector<License>> Character_ROI;
locate.Get_Character_ROI(License_ROI, Character_ROI,
src, true);
vector<vector<int>> result_index;
locate.License_Recognition(Character_ROI,result_index);
locate.Draw_Result(src, License_ROI,
Character_ROI, result_index);
imshow("车牌识别结果", src);
waitKey();
return 0;
}
If you want template image files and label files, you can leave a message in the comment area or private message me. You must be a VIP to upload it on CSDN before you can download it.
3. Project summary
Code ideas:
- Get the whole license plate (this part involves preprocessing is interesting)
- Cut the license plate to get 7 characters
- Match the obtained license plate characters with the template
Insufficient items:
- This item is only useful for license plates with white characters
- No rotation correction [ p1 , p2 ] and perspective correction have been made on the license plate . These two factors have a great influence, and I will add it later when I have time
ps: I have been working on this project for several days, the cpp file has reached 500 lines, and the original text is only 300 lines, adding nearly half of the code, I don’t want to change it in a short time