Preface: Because I want to do an elective course about AI, the whole code is actually excerpted from the python face recognition source code of csdn blogger hwtl0703598, but I also found some codes of the blogger during the debugging process Problems and improvements, I will point out the corrections at the back end of the entire article, basically it can be used directly after the environment is configured.
What is LBPH in brief overview?
LBPH (Local Binary Patterns Histograms) local binary encoding histogram, the basic idea of the face recognition method based on LBPH is as follows: first, center on each pixel, judge the relationship with the gray value of the surrounding pixels, and perform binary encoding on it , so as to obtain the LBP coded image of the whole image; then divide the LBP image into regions, obtain the LBP coded histogram of each region, and then obtain the LBP coded histogram of the whole image, by comparing the LBP coded histogram of different face images The image achieves the purpose of face recognition, and its advantage is that it will not be affected by illumination, scaling, rotation and translation.
PS: OpenCV provides three face recognition methods, namely the LBPH method, the EigenFishfaces method, and the Fisherfaces method.
Why use LBPH instead of others?
LBPH: Divide the detected face into small units and compare them with the corresponding units in the model, generating a histogram of the matching values for each region. Due to the flexibility of this method, LBPH is the only face recognition algorithm that allows the model sample face and the detected face to be different in shape and size .
The steps of face recognition
- step1: Enter the face
- step2: Use the saved face for training
- step3: Use the trained data to identify special people
Preparation:
- step1 . Use python, any compiler, eclipse or pycharm
- step2 . Prepare opencv, OpenCV is an open source computer vision library, used for image recognition and camera calls in this project
- step3 .numpy(Numerical Python): An extended program library of the python language, which supports a large number of dimensional arrays and matrix operations, and also provides a large number of mathematical function libraries for arrays.
- step4 .PIL:Python Imaging Library, the standard library for image processing on the Python platform
Face registration process:
- step1: Use opencv's VideoCapture function to open the camera and obtain the video stream.
- Step2: Obtain the image of each frame of the video stream, convert the image into a grayscale image, use the opencv face detector to detect the face on the image, draw a frame on the face, and save the face in the frame Save about 200 images in the image folder you selected. In fact, the more training you do, the higher the success rate of recognition will be.
- step3: After saving 200 images, the program will end automatically, release the camera cache and close the window.
PS: Each face needs to be converted into a grayscale image. The purpose of image grayscale is to simplify the matrix and improve the calculation speed. Here is to improve the recognition efficiency.
Complete code for face entry
import cv2
cap = cv2.VideoCapture(0)
face_detector = cv2.CascadeClassifier('D:/opencv\sources/data/haarcascades/haarcascade_frontalface_default.xml')
face_id = input('User data input,Look at the camera and wait ...')
count = 0
while cap.isOpened():
ret, frame = cap.read()
if ret is True:
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
else:
break
faces = face_detector.detectMultiScale(gray, 1.3, 5)
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x + w, y + w), (255, 0, 0))
count += 1
cv2.imwrite("D:/opencv_test/" + str(face_id) + '.' + str(count) + '.jpg', gray[y:y + h, x:x + w])
cv2.imshow('image', frame)
k = cv2.waitKey(1)
if k == 27:
break
elif count >= 200:
break
cap.release()
cv2.destroyAllWindows()Code analysis:
- cap = cv2.VideoCapture(0): call the camera, the parameter 0 means the first built-in camera of the notebook by default
- cap.isOpened(): Determine whether the video object is successfully read, and return True if the video object is successfully read, which is used as the condition for the loop to be executed all the time.
- ret,frame = cap.read(): read the video frame by frame, the return value ret is Boolean, return True if it is read correctly, and return False if it fails to read or read the end of the video; frame is the image of each frame .
- faces = face_detector.detectMultiScale(gray, 1.3, 5): #The first parameter is the grayscale image, and the second parameter is the scale transformation, which is how many times the original up or down each time, here is 1.02 times, the first The three parameters are the number of face detections. The higher the setting, the lower the false detection rate. However, for blurred pictures, the higher the setting, the harder it is to detect, and it should be reduced appropriately.
- key = cv2.waitKey(1): Waiting for keyboard input, parameter 1 means switching to the next frame with a delay of 1ms, parameter 0 means displaying the current frame, which is equivalent to pause, let it be equal to 27, 27 means Esc to exit on the computer.
- Here, according to the value of count, 200 sheets can be entered. In fact, more can be entered, but more is not better.
- The saved pictures here are fixed in a face image set, and the picture inside is named user.count.jpg
- The running code will stop at User data input, Look at the camera and wait ..., here is to input, because the name array needs to be used to display the name later, so the user name here is the subscript of 0, 1, 2 , the input is 0,1,2...
Effect display 1
Effect display 2
Face dataset training process
- step1 : Give the path of the saved face image folder, traverse all the pictures under the path, and import the name and image of the picture into the two list arrays of face_samples and ids through the os function, and transfer the picture through the array function in numpy into arrays.
- Step2: Call the LBPH face recognizer, train the image array and the corresponding image name, and save the result to the file trainer/trainer.yml.
Complete code for face training
import os
import cv2
import numpy as np
from PIL import Image
path = 'D:/opencv_test/'
recognizer = cv2.face.LBPHFaceRecognizer_create()
detector = cv2.CascadeClassifier('D:/opencv\sources/data/haarcascades/haarcascade_frontalface_default.xml')
def get_images_and_labels(path):
image_paths = [os.path.join(path, f) for f in os.listdir(path)]
face_samples = []
ids = []
for image_path in image_paths:
img = Image.open(image_path).convert('L')
img_np = np.array(img, 'uint8')
if os.path.split(image_path)[-1].split(".")[-1] != 'jpg':
continue
id = int(os.path.split(image_path)[-1].split(".")[0])
faces = detector.detectMultiScale(img_np)
for (x, y, w, h) in faces:
face_samples.append(img_np[y:y + h, x:x + w])
ids.append(id)
return face_samples, ids
faces, ids = get_images_and_labels(path)
recognizer.train(faces, np.array(ids))
recognizer.save('trainer/trainer.yml')
Code analysis:
- recognizer = cv2.face.LBPHFaceRecognizer_create(): Generate LBPH recognizer instance model
- cv2.face_FaceRecognizer.train(): Calculate LBPH for each reference image to get a vector, each face is a point in the entire vector set
- detector = cv2.CascadeClassifier('sources/data/haarcascades/haarcascade_frontalface_default.xml'): call Opencv's own trained face detector (default)
- The face detector that comes with OpenCV is in the sources/data/ directory. According to the directory installed by each person's OpenCV, the absolute address is also used here
- The id obtained here is user instead of count, which is why I improved the code problem of the blogger, trained all the face pictures under the corresponding path according to the corresponding user name, and saved the trained data set.
Face recognition process:
- Step1: Call the camera, obtain the image frame of the video stream, frame the face, use the trained face image set to match the face in the video, and use the prediction function predict() to obtain the confidence score.
- step2: If the LBPH recognition confidence score is above 80, it is considered unqualified, so judge the predicted confidence score. If the face in the video stream is not in the training set, it will be expressed as unknown.
- Step3: Frame the video stream image frame, frame the face, display the confidence score and the predicted face label, and use the label to specify which person in the image set the output is.
- step4: Release the camera resources and close the window.
Face recognition complete code
import cv2
recognizer = cv2.face.LBPHFaceRecognizer_create()
recognizer.read('trainer/trainer.yml')
face_cascade = cv2.CascadeClassifier("D:/opencv\sources/data/haarcascades/haarcascade_frontalface_default.xml")
font = cv2.FONT_HERSHEY_SIMPLEX
idnum = 0
cam = cv2.VideoCapture(0)
cam.set(6, cv2.VideoWriter.fourcc('M', 'J', 'P', 'G'))
minW = 0.1 * cam.get(3)
minH = 0.1 * cam.get(4)
names = ['linluocheng','zhupengcheng']
while True:
ret, img = cam.read()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(
gray,
scaleFactor=1.2,
minNeighbors=5,
minSize=(int(minW), int(minH))
)
for (x, y, w, h) in faces:
cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
idnum, confidence = recognizer.predict(gray[y:y + h, x:x + w])
if confidence < 80:
idum = names[idnum]
confidence = "{0}%".format(round(100 - confidence))
else:
idum = "unknown"
confidence = "{0}%".format(round(100 - confidence))
cv2.putText(img, str(idum), (x + 5, y - 5), font, 1, (0, 0, 255), 1)
cv2.putText(img, str(confidence), (x + 5, y + h - 5), font, 1, (0, 0, 0), 1)
cv2.imshow('camera', img)
k = cv2.waitKey(10)
if k == 27:
break
cam.release()
cv2.destroyAllWindows()
Code analysis:
- recognizer.predict(): A prediction function that obtains the label of the image and the similarity between the image and the training set, also known as the confidence score.
- cv2.putText(): Parameters (text position, font size, font color, font thickness added to the picture), display the text on the image, and Chinese will display garbled characters.
- format Formatting function: the basic syntax is to replace the previous % by {} and:, which is a bit similar to the format character of C language
- The round() method returns the rounded value of the floating point number x
- Convert it into a grayscale image and frame the face on it, and compare the face with the trained one. When the confidence score reaches a certain level, the match is successful, and the face and confidence score are displayed.
- There is still a mistake made by the blogger here, that is, the format function and the previous "" are separated by . instead of a comma.
Effect display picture 1
Effect display picture 2
In view of the various problems that arise when using the source code, make a summary:
1. The code here is slightly modified by me through other people's code. If you encounter problems and seek other people's opinions to modify them, there will still be problems. If there is a problem that the function is not found or does not exist, it is opencv The version problem, various functions will be modified accordingly after the update iteration.
2. Some people have various problems with the code of this blog. The reason is that the training process is not very clear. The key lies in the input name when the face is entered. Here, the following process is given for the running program.
step1 (note that the input here is 0, 1, 2, it is best to enter in order):
step2:
Face training can be run directly.
step3 (in the face recognition code, there will be a similar names list, this is our custom name, the serial number when entering the face, which corresponds to the subscript here, and then get the corresponding face label):
step3:
Then the recognition is carried out, and finally the recognition is completed.