table of Contents
3.1 Overview of face detection and face recognition
3.2 Face Characterization Detection (Facial Sense Location)
3.3 Introduction to the dlib library
0 Preface
The new type of coronavirus has shown a trend of globalization and spread, which reminds us that the prevention and control of infectious diseases will remain the focus of disease prevention and control for a long time in the future. Therefore, wearing a mask is a necessary part of daily life. For the safety of others and for their own safety, everyone needs to wear a mask. The goal of this course design is to realize the "masking" work of human faces based on face recognition.
- 1. Complete the face plus mask test on the given picture.
- 2. According to the designed model, give corresponding suggestions.
- 3. Give the simulation process and results.
The source code download link of this article: https://download.csdn.net/download/weixin_43442778/12503099
1 Design purpose
The new type of coronavirus has shown a trend of globalization and spread, which reminds us that the prevention and control of infectious diseases will remain the focus of disease prevention and control for a long time in the future. Therefore, wearing a mask is a necessary part of daily life. For the safety of others and for their own safety, everyone needs to wear a mask. The goal of this course design is to realize the "masking" work of human faces based on face recognition.
2 Tasks and requirements
1. Complete the face plus mask test on the given picture.
2. According to the designed model, give corresponding suggestions.
3. Give the simulation process and results.
3 design principles
The realization principle of this system is to use the Landmark face detection library of the Dlib module to identify facial features data, according to these data, determine the position data of the lips (48 points to 67 points), according to the detected mouth The size and direction of the mask can be adjusted with the help of the PLL module to achieve the proper position of the mask in the image.
3.1 Overview of face detection and face recognition
Face detection is mainly used in the preprocessing of face recognition in practice, that is, to accurately calibrate the position and size of the face in the image. The pattern features contained in face images are very rich, such as histogram features, color features, template features, structural features, and Haar features. The image preprocessing of the face is based on the result of face detection [2], the image is processed and finally serves the process of feature extraction. Face detection is to pick out the useful information and use these features to realize face detection.
Face recognition [1] is a kind of biometric recognition technology based on human facial feature information. It is to compare the face features to be recognized with the obtained face feature templates, and judge the identity information of the face according to the degree of similarity. The features that can be used in a face recognition system are usually divided into visual features, pixel statistical features, face image transformation coefficient features, face image algebra features, and so on. A series of related technologies that use a camera or camera to collect images or video streams containing human faces, and automatically detect and track human faces in the images, and then perform facial recognition on the detected faces, usually also called face recognition and facial recognition . The face recognition system mainly includes four components, namely: face image acquisition and detection, face image preprocessing, face image feature extraction, and matching and recognition.
3.2 Face Characterization Detection (Facial Sense Location)
Generally, face applications use two technologies, “face detection” and “face comparison”, such as face punching, automatic face mosaic, face unlocking and other applications. However, due to the complexity of this experiment, only face detection and comparison are not enough. Therefore, we not only need to judge the position of the face (rectangular box), but also need to detect the position of the facial features, such as the shape of the eyebrows, the eye area, the position of the nose, the position of the mouth, etc. After having the position data of the facial features, we can be based on These data are used in this experiment. The following figure shows the result of face characterization detection:
Figure 3.1 Face characterization detection diagram
The above figure shows 68 points of a face represented by machine learning. The parts of the face and the corresponding points are shown in the table:
Table 3.1 Correspondence table of face characterization detection
We can quickly obtain the corresponding coordinate point through the subscript (slicing can be used in python), which represents the actual position (pixel unit) of the face representation in the original input image.
3.3 Introduction to the dlib library
Dlib is a modern C++ toolbox that contains machine learning algorithms and tools for creating complex software in C++ to solve practical problems. It is widely used in industry and academia, including robotics, embedded devices, mobile phones and large-scale high-performance computing environments. In this experiment, the face feature extractor that comes with dlib is used, which is very convenient.
3.4 Introduction to Tkinter
Tkinter[3] is a module for window design using python. The Tkinter module ("Tk interface") is an interface to Python's standard Tk GUI toolkit. As a specific GUI interface for python, it is an image window. Tkinter is a GUI interface that comes with python and can be edited. We can use GUI to achieve many intuitive functions.
Python's Tkinter interface library is very simple. There are many GUI libraries for python. Choose Tkinter, one is the easiest, the other is its own library, no need to download and install, you can use it at any time, and the third is to start from the demand. As a scripting language, Python can be used as a flexible Tools, under this demand, Tkinter can do it!
For Tkinter programming, you can start with the following two aspects:
The first one, painting, corresponds to tkinter programming. The display screen is the easel that is supported. The root window is the drawing board. In tkinter it is the Toplevel. The canvas is the container (Frame) in tkinter. Many canvases can be placed on the drawing board ( Convas), the container in tkinter can also contain many containers. The composition layout in the painting is the layout manager (geometry manager) in tkinter. The content of the painting is the small components in tkinter. Many elements constitute, and our GUI interface is composed of individual components, which are widgets.
Second, the components of tkinter can also be seen as building blocks. The shapes may be different, but the essence is the same. It is a building block. No matter what it looks like, it is always a building block! These small components have many commonalities, the most important of which is the use of the layout manager.
4 system establishment
This part includes two parts in total, namely interface design and organ recognition.
4.1 interface design
Based on the tkinter module to implement GUI design, you can load character images, select four types of masks (the masks here are processed pictures), show the effect of wearing the mask, and exit the system after the operation is completed. The effect is shown in the figure below, see the specific code appendix.
Figure 4.1 Interface design
4.2 Facial Recognition
After the page function is realized, it is to rely on the Dlib library to realize the recognition of the key points of the face organs, analyze the position and size of the mouth, and display the key points of the character's face.
Figure 4.2 Comparison of facial features recognition
The Landmark person Dlib module face 68 keypoint detection library identify the face features data to determine position data (the lip portion 48 o'clock to 67 o'clock ), the detection of the size and direction of the mouth, by means of PLL module adjusts the mask The size and direction enable the mask to be placed in the proper position of the image.
Figure 4.3 Mask part
5 System operation results
5.1 Schematic diagram of mask
The picture below shows the mask that we can wear. It should be noted that the white background of the mask is not filled with white, but transparent white (PS software can be used for specific processing).
Figure 5.1 Mask style
5.2 System result display
Tested on the three given pictures, all got good results, see Figure 5.2, 5.3, 5.4.
Figure 5.2 Picture rendering of test 1
Figure 5.3 Picture renderings of test 2
Figure 5.4 Test 3 picture renderings
6 recommendations
Wearing masks scientifically has a preventive effect on respiratory infectious diseases such as new coronary pneumonia and influenza. It not only protects oneself, but also benefits public health. At present, in the fight against the new crown pneumonia epidemic situation, Chinese citizens wear masks scientifically, so that the epidemic can be effectively prevented and controlled. This measure is worth learning from other countries, especially the United States.
7 References
- Face recognition: https://baike.baidu.com/item/%E4%BA%BA%E8%84%B8%E8%AF%86%E5%88%AB/4463435?fr=aladdin
- https://blog.csdn.net/weixin_42346564/article/details/82500454
- Tkinter introduction: https://www.cnblogs.com/shwee/p/9427975.html#B
- Correct installation of Python dlib: https://ai-word.blog.csdn.net/article/details/88713658
- Quickly install the python package: https://blog.csdn.net/weixin_43442778/article/details/103095442
- Face application: https://www.cnblogs.com/xiaozhi_5638/p/12697035.html
- Key points detection of facial features: https://blog.csdn.net/zhr1030635594/article/details/104411165
- Automatic face mask wearing system based on Python: https://handsome-man.blog.csdn.net/article/details/104174562
Annex recorded
Source code download link that can be run directly: https://download.csdn.net/download/weixin_43442778/12503099
Five senses positioning code
| #coding=utf-8 #Picture Detection-Dlib version import cv2 import dlib import time t=time.time() path = "./pic/test1.jpeg" img = cv2.imread(path) gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#Face classifier detector = dlib.get_frontal_face_detector() # Get face detector predictor = dlib.shape_predictor( "./shape_predictor_68_face_landmarks.dat" )
dets = detector (gray, 1) for face in dets: shape = predictor(img, face) # Find 68 calibration points of the face # Traverse all points, print out their coordinates, and circle them for pt in shape.parts(): pt_pos = (pt.x, pt.y) cv2.circle(img, pt_pos, 1, (0, 255, 0), 2) cv2.imshow("image", img) print('The time used is {)'.format(time.time()-t)) cv2.waitKey(0) #cv2.destroyAllWindows() time.sleep(5) |
System build code
| from PIL import Image, ImageTk from tkinter.filedialog import askopenfilename import cv2 import tkinter as tk import GDP import dlib
class AddMask(object): def __init__(self): self.root = tk.Tk() self.root.title('Pyhon-based automatic face masking system') self.root.geometry('1200x500')
self.path1_ = None # self.path2_ = None self.seg_img_path = None self.mask = None self.label_Img_seg = None
decoration = PIL.Image.open('./pic/bg.png').resize((1200, 500)) render = ImageTk.PhotoImage(decoration) img = tk.Label(image=render) img.image = render img.place(x=0, y=0)
# 原图1的展示 tk.Button(self.root, text="打开头像", command=self.show_original1_pic).place(x=50, y=120) tk.Button(self.root, text="退出软件", command=quit).place(x=900, y=40)
tk.Label(self.root, text="头像", font=10).place(x=280, y=120) self.cv_orinial1 = tk.Canvas(self.root, bg='white', width=270, height=270) self.cv_orinial1.create_rectangle(8, 8, 260, 260, width=1, outline='red') self.cv_orinial1.place(x=180, y=150) self.label_Img_original1 = tk.Label(self.root) self.label_Img_original1.place(x=180, y=150)
tk.Label(self.root,text="选择口罩",font=10).place(x=600,y=120)
first_pic = Image.open("./pic/Mask1.png") first_pic = first_pic.resize((60, 60), Image.ANTIALIAS) first_pic = ImageTk.PhotoImage(first_pic) self.first = tk.Label(self.root, image=first_pic) self.first.place(x=600,y=160, width=60, height=60) self.first.bind("<Button-1>", self.mask1)
second_pic = Image.open("./pic/Mask2.png") second_pic = second_pic.resize((60, 60), Image.ANTIALIAS) second_pic = ImageTk.PhotoImage(second_pic) self.second_pic = tk.Label(self.root, image=second_pic) self.second_pic.place(x=600, y=230, width=60, height=60) self.second_pic.bind("<Button-1>", self.mask2)
third_pic = Image.open("./pic/Mask3.png") third_pic = third_pic.resize((60, 60), Image.ANTIALIAS) third_pic = ImageTk.PhotoImage(third_pic) self.third_pic = tk.Label(self.root, image=third_pic) self.third_pic.place(x=600, y=300, width=60, height=60) self.third_pic.bind("<Button-1>", self.mask3)
forth_pic = Image.open("./pic/Mask4.png") forth_pic = forth_pic.resize((60, 60), Image.ANTIALIAS) forth_pic = ImageTk.PhotoImage(forth_pic) self.forth_pic = tk.Label(self.root, image=forth_pic) self.forth_pic.place(x=600, y=370, width=60, height=60) self.forth_pic.bind("<Button-1>", self.mask4)
tk.Label(self.root, text="佩戴效果", font=10).place(x=920, y=120) self.cv_seg = tk.Canvas(self.root, bg='white', width=270, height=270) self.cv_seg.create_rectangle(8, 8, 260, 260, width=1, outline='red') self.cv_seg.place(x=820, y=150) self.label_Img_seg = tk.Label(self.root) self.label_Img_seg.place(x=820, y=150)
self.root.mainloop()
# 原图1展示 def show_original1_pic(self): self.path1_ = askopenfilename(title='选择文件') print(self.path1_) self.Img = PIL.Image.open(r'{}'.format(self.path1_)) Img = self.Img.resize((270,270),PIL.Image.ANTIALIAS) # 调整图片大小至256x256 img_png_original = ImageTk.PhotoImage(Img) self.label_Img_original1.config(image=img_png_original) self.label_Img_original1.image = img_png_original # keep a reference self.cv_orinial1.create_image(5, 5,anchor='nw', image=img_png_original)
# 人脸戴口罩效果展示 def show_morpher_pic(self): img1 = cv2.imread(self.path1_) x_min, x_max, y_min, y_max, size = self.get_mouth(img1) adding = self.mask.resize(size) im = Image.fromarray(img1[:, :, ::-1]) # 切换RGB格式 # 在合适位置添加头发图片 im.paste(adding, (int(x_min), int(y_min)), adding) # im.show() save_path = self.path1_.split('.')[0]+'_result.jpg' im.save(save_path) Img = im.resize((270, 270), PIL.Image.ANTIALIAS) # 调整图片大小至270x270 img_png_seg = ImageTk.PhotoImage(Img) self.label_Img_seg.config(image=img_png_seg) self.label_Img_seg.image = img_png_seg # keep a reference
def mask1(self, event): self.mask = Image.open('pic/Mask1.png') self.show_morpher_pic()
def mask2(self, event): self.mask = Image.open('pic/Mask2.png') self.show_morpher_pic()
def mask3(self, event): self.mask = Image.open('pic/Mask3.png') self.show_morpher_pic()
def mask4(self, event): self.mask = Image.open('pic/Mask4.png') self.show_morpher_pic()
def get_mouth(self, img): img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # img_gray = cv2.gray2bgr(img, cv2.COLOR_BGR2GRAY)
detector = dlib.get_frontal_face_detector() predictor = dlib.shape_predictor('./shape_predictor_68_face_landmarks.dat') faces = detector(img_gray, 0) for k, d in enumerate(faces): x = [] y = [] # 人脸大小的高度 height = d.bottom() - d.top() # 人脸大小的宽度 width = d.right() - d.left() shape = predictor(img_gray, d) # 48-67 为嘴唇部分 for i in range(48, 68): x.append(shape.part(i).x) y.append(shape.part(i).y) # 根据人脸的大小扩大嘴唇对应口罩的区域 y_max = (int)(max(y) + height / 3) y_min = (int)(min(y) - height / 3) x_max = (int)(max(x) + width / 3) x_min = (int)(min(x) - width / 3) size = ((x_max - x_min), (y_max - y_min)) return x_min, x_max, y_min, y_max, size
def quit(self): self.root.destroy()
if __name__ == '__main__': AddMask() |