A preliminary study of face detection and recognition in python

Types of face detection

What are the types of face recognition? It can be divided into two types.
The first is static face recognition : it refers to the person being recognized, in a static state or in a cooperative state, collecting the face image of the person to perform image recognition. This kind of cooperation requires the cooperation of the person, and it usually takes 1-2 seconds for the person to temporarily. Part of it is suitable for places that are not sensitive to time or that are not averse to collecting faces, such as face access control, identity reading, attendance check-in, ID card verification in Internet cafes, visitor registration, etc.;

The second is dynamic face recognition : when the person being recognized is in a moving state or walking and other non-cooperative situations, the video of his face is collected to perform face recognition. This type does not require the cooperation of the parties, so it refers to the non-sensitive places of the parties, such as : At stations, airports, wharfs and other places where people are concentrated, the key face control can be achieved to prevent problems before they happen.

Application of face detection in life

Face recognition is more and more widely used in our lives.

You can scan your face to unlock your phone, scan your face to pay, you can scan your face to open the door, and your face becomes a very important biometric information of a person. Nowadays, the development of face recognition technology is becoming more and more mature, and the application fields are becoming more and more widespread. For example, the Yunmai face recognition system can be applied in the following scenarios:

1. Enterprise and residential security and management: such as face recognition access control and attendance systems, face recognition anti-theft doors, etc.
2. E-passport and ID card: China’s e-passport program, Ministry of Public Security, is stepping up its planning and implementation.
3. Public security, justice, and criminal investigation: such as using facial recognition systems and networks to search for fugitives across the country.
4. Self-service: such as self-service real-name authentication, self-service remote account opening, etc.
5. Information security: such as computer login, e-government and e-commerce.

Face detection and face recognition

Before you start explaining face detection, to help you resolve the two concepts are often confused: 人脸检测 vs 人脸识别. note! ! Concepts that everyone often confuses.
Distinguish the difference between these two concepts. Simply put, their core differences are:

1. Where is the face? Face detection (the process of locating the face area in the image)

2. Who is the face? Face recognition (classification to detect who this face is)

One is to compare the face area obtained by face detection with a known face database ;

The other is to extract feature values ​​for facial image features . The feature value array is used to identify face information.

The latter is more practical.

In terms of sequence, face detection is performed first, and then face recognition is performed .

Then we will first talk about how to use OpenCV for face detection today.

How to use OpenCV for face detection

About HaarCascade

Face recognition in OpenCV is realized by the cascaded classifier of Haar features. There are not too many explanations of the underlying algorithm principles here.

In fact, there are many pre-trained in OpenCV HaarCascade模型(XML文件), such as face detection, eye detection, whole body detection, lower body detection, etc.

There are two code repositories in OpenCV.

Data 1 : opencv/data/cascades

Data 2 : opencv_contrib/modules/face/data/cascades

How to use the cascade classifier

By FaceCascade模型entering the picture, we can get the rectangular position of the area where the face is.

The method of using the model is relatively simple. First, load the corresponding HaarCascadefile. The file format is xml. For your convenience, I have downloaded the file to the haar folder and can reference it by relative path.

├── FaceDetection-v1.py
├── FaceDetection-v2.py
├── face.mp4
├── face.png
├── haar
    ├── haarcascade_eye.xml
    └── haarcascade_frontalface_default.xml

We CascadeClassifierpass in the corresponding HaarCascadefile in

# 载入人脸检测的Cascade模型
FaceCascade = cv2.CascadeClassifier('./haar/haarcascade_frontalface_default.xml')

Input

The next step is to pass the grayscale image of the picture into this FaceCascade模型for face detection.

# 检测画面中的人脸
faces = FaceCascade.detectMultiScale(gray)

The faces returned by Output is the ROI array of the region where the face is located, for example:

[(x1, y1, w1, h1), (x2, y2, w2, h2)]

Of course, you can set different parameters, such as setting the scaling factor and setting the minimum neighbor threshold.

# 检测画面中的人脸
faces = FaceCascade.detectMultiScale(
    gray,
    scaleFactor=1.1,
    minNeighbors=5
) 

For the specific meaning and usage of the parameters, please refer to the official API documentation of OpenCV:

OpenCV official document: The full name of cascadeclassifier-detectmultiscale

ROI is Region Of Interest, used to indicate the sub-area of ​​the screen. The origin （0,0）of the whole picture is in the upper left corner of the whole picture. ROIIt is essentially Tuple类型data, which （x,y）represents the coordinates of the upper left corner of the rectangular area where the face is, wrepresents the width hof the rectangle , and represents the height of the rectangle.

Demonstration of the effect of face detection

Face detection-V1

Read the picture from the picture, recognize the face, and identify the rectangular area where the face is located in the picture.

Face detection-V2

In the demonstration of the effect of face recognition v2, two HaarCascade models are used, one is to recognize faces and the other is to recognize eyes .

Program flow explanation

There are three core modules in the code of face recognition v1: HighGUI upper computer part , Draw drawing module , CascadeClassifier cascade classifier .

The general flow of the program is as follows:

1. HighGUI reads in pictures and converts them to grayscale
2. CascadeClassifier loads the cascade model (xml) file for face detection
3. CascadeClassifier uses the model to detect faces in grayscale images and returns faces
4. Draw traverses faces and draws the rectangular area of ​​the face on the color image
5. HighGUI creates a window called Face
6. HighGUI displays the image in the window Face (after drawing the rectangular area of ​​the face)
7. HighGUI waits for any key press
8. HighGUI exits the program and closes all windows

FaceDetection-v1.py

# -*- coding:utf-8 -*-
'''
人脸识别FaceDetection
通过HaarCascade模型，进行人脸识别与眼睛识别，在视频流中绘制矩形，标识人脸
'''
import cv2

# 设置图片路径
img_path = 'face.png'
# 载入带有人脸的图片
img = cv2.imread(img_path)
if img is None:
    # 判断图片是否读入正确
    print("ERROR：请检查图片路径")
    exit(1)
# 将彩色图片转换为灰度图
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)


# 载入人脸检测的Cascade模型
FaceCascade = cv2.CascadeClassifier('./haar/haarcascade_frontalface_default.xml')

# 检测画面中的人脸
faces = FaceCascade.detectMultiScale(
    gray,
    scaleFactor=1.1,
    minNeighbors=5
)

# 遍历返回的face数组
for face in faces:
    # 解析tuple类型的face位置数据
    # (x, y): 左上角坐标值
    # w: 人脸矩形区域的宽度
    # h: 人脸矩形区域的高度
    (x, y, w, h) = face
    # 在原彩图上绘制矩形
    cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), 4)

# 创建一个窗口 名字叫做Face
cv2.namedWindow('Face',flags=cv2.WINDOW_NORMAL | cv2.WINDOW_KEEPRATIO | cv2.WINDOW_GUI_EXPANDED)

# 在窗口Face上面展示图片img
cv2.imshow('Face', img)
# 等待任意按键按下
cv2.waitKey(0)
# 关闭所有的窗口
cv2.destroyAllWindows()

How to improve the speed of face recognition?

**

1. Narrow the search range and look around the rectangle where the face of the previous frame is located.
2. Zoom, search at low resolution, and then gradually pinpoint the position.