I have nothing to do recently and want to write an article about face recognition based on deep learning. I mainly use two different neural networks for implementation, namely a simple three-layer convolutional neural network and a complex VGG16 neural network, and compare the recognition effects of the models trained by the two networks. Judging from the final results, as expected, the effect of VGG16, which has a more complex structure, is even better. Let me introduce its implementation process in detail below.
(In view of the fact that many friends have privately messaged me that I am interested in this project, I have attached the code link in the comment area. If you need it, please help yourself. If you find it useful, remember to like it and post Q~~~)
Next, I will explain it in the following order:
Environment configuration
Face detection part
Training model part
Face recognition part
1. Environment configuration
By the way, when installing these libraries, if you use the pip install method, it is recommended to use this method. Otherwise, you will get an error when installing using different methods (a piece of advice from a scammer haha). The configuration is about That’s it. Some of them may not be written down, and you may need to go through some configuration pitfalls. (Are you going to get slapped in the face here?)
2. Face detection
This part mainly uses opencv to call the camera and perform image processing, and then uses a method based on cascade classifier + haar feature for face detection . Opencv is very convenient to use. There are very few functions used here, which are just ordinary reading of images, grayscale conversion, image display, and simple image editing.
1) Read the picture
Just give the path of the image to be operated on.
import cv2
image = cv2.imread(imagepath)
2) Grayscale conversion
The purpose of grayscale conversion is to reduce the computational intensity of images converted to grayscale. Because the current color pictures are all three-channel data, without any processing, the amount of data will be very large, which cannot be held by the machines used by our students.
import cv2
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
The gray picture probably looks like this.
3) Draw pictures
One manifestation of the power of opencv is that it can edit and process images arbitrarily.
The last parameter of the function below specifies the size of the brush. In fact, it is to trace the detected face border.
import cv2
cv2.rectangle(image,(x,y),(x+w,y+w),(0,255,0),2)4) Display image
The edited image is either displayed directly or saved to a physical storage medium.
import cv2
cv2.imshow("Image Title",image)
5) Obtain face recognition training data
It seems complicated, but it is actually some description of facial features. In this way, after reading the data, opencv can perceive the features of the read pictures based on the sample data in training, and then perform face recognition on the pictures.
import cv2
face_cascade = cv2.CascadeClassifier(r'./haarcascade_frontalface_default.xml')
The xml file inside is very critical. It can be said to be the core of this model, and it is relied on to obtain face data. It is universally trained data shared by opencv on GitHub. We can use it directly.
6) Detect faces
To put it bluntly, it is the process of identifying new pictures based on training data.
import cv2
# 探测图片中的人脸
faces = face_cascade.detectMultiScale(
gray,
scaleFactor = 1.15,
minNeighbors = 5,
minSize = (5,5),
flags = cv2.cv.CV_HAAR_SCALE_IMAGE
)
We can arbitrarily specify the values of the parameters inside to achieve recognition at different precisions. The return value is the reflection of the detection result of the image by opencv.
Processing the results of face detection
After finishing the face detection just now, we can get the return value for further processing. But that doesn’t mean it’s going to be complicated, it’s just adding point eigenvalues.
import cv2
print "发现{0}个人脸!".format(len(faces))
for(x,y,w,h) in faces:
cv2.rectangle(image,(x,y),(x+w,y+w),(0,255,0),2)
The above has introduced some necessary functions, including functions for face detection. Then we are going to talk about this model. It is actually easy to talk about the model. First, obtain the training data, then write the model training, and finally detect the effect. In this order, let’s first talk about how to collect face data.
We only need to collect pictures from two people. Considering everyone's laptop configuration, each person only needs to collect a few hundred pictures. The file name is get_face.py, and the code is as follows:
def CatchPICFromVideo(window_name, camera_idx, catch_pic_num, path_name):
cv2.namedWindow(window_name)
# 视频来源，可以来自一段已存好的视频，也可以直接来自USB摄像头
cap = cv2.VideoCapture(camera_idx)
# 告诉OpenCV使用人脸识别分类器
data_path = "haarcascade_frontalface_default.xml"
classfier = cv2.CascadeClassifier(data_path)
# 识别出人脸后要画的边框的颜色，RGB格式
color = (0, 255, 0)
num = 0
while cap.isOpened():
ok, frame = cap.read() # 读取一帧数据
if not ok:
break
grey = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # 将当前桢图像转换成灰度图像
# 人脸检测，1.2和2分别为图片缩放比例和需要检测的有效点数
faceRects = classfier.detectMultiScale(grey, scaleFactor=1.2, minNeighbors=3, minSize=(32, 32))
if len(faceRects) > 0: # 大于0则检测到人脸
for faceRect in faceRects: # 单独框出每一张人脸
x, y, w, h = faceRect
# 将当前帧保存为图片
img_name = '%s/%d.jpg ' %(path_name, num)
image = frame[y - 10: y + h + 10, x - 10: x + w + 10]
cv2.imwrite(img_name, image)
num += 1
if num > catch_pic_num: # 如果超过指定最大保存数量退出循环
break
# 画出矩形框
cv2.rectangle(frame, (x - 10, y - 10), (x + w + 10, y + h + 10), color, 2)
# 显示当前捕捉到了多少人脸图片
font = cv2.FONT_HERSHEY_SIMPLEX
cv2.putText(frame ,'num:%d' % (num) ,(x + 30, y + 30), font, 1, (255 ,0 ,255) ,4)
# 超过指定最大保存数量结束程序
if num > catch_pic_num:
break
# 显示图像
cv2.imshow(window_name, frame)
c = cv2.waitKey(10)
if c & 0xFF == ord('q'):
break
# 释放摄像头并销毁所有窗口
cap.release()
cv2.destroyAllWindows()
3. Training model
After getting the data, the next step is to write the model training. I used two models respectively. The following model is implemented using 3 convolution operations + 1 full connection operation. If it is VGG16, just change the following parts. The other parts of the code are the same.
The model code is as follows:
def build_model(self, dataset, nb_classes=2):
# 构建一个空的网络模型，它是一个线性堆叠模型，各神经网络层会被顺序添加，专业名称为序贯模型或线性堆叠模型
self.model = Sequential()
# 以下代码将顺序添加CNN网络需要的各层，一个add就是一个网络层
self.model.add(Convolution2D(32, 3, 3, border_mode='same',
input_shape=dataset.input_shape)) # 1 2维卷积层
self.model.add(Activation('relu')) # 2 激活函数层
self.model.add(MaxPooling2D(pool_size=(2, 2))) # 5 池化层
self.model.add(Dropout(0.25)) # 6 Dropout层
self.model.add(Convolution2D(64, 3, 3, border_mode='same')) # 7 2维卷积层
self.model.add(Activation('relu')) # 8 激活函数层
self.model.add(MaxPooling2D(pool_size=(2, 2))) # 11 池化层
self.model.add(Dropout(0.25)) # 12 Dropout层
self.model.add(Flatten()) # 13 Flatten层
self.model.add(Dense(512)) # 14 Dense层,又被称作全连接层
self.model.add(Activation('relu')) # 15 激活函数层
self.model.add(Dropout(0.5)) # 16 Dropout层
self.model.add(Dense(nb_classes)) # 17 Dense层
self.model.add(Activation('softmax')) # 18 分类层，输出最终结果
# 输出模型概况
self.model.summary()
因为是用keras写的，所以看起来比较简洁。
训练模型的函数也很简洁
sgd = SGD(lr=0.01, decay=1e-6,
momentum=0.9, nesterov=True) # 采用SGD+momentum的优化器进行训练，首先生成一个优化器对象
self.model.compile(loss='categorical_crossentropy',
optimizer=sgd,
metrics=['accuracy']) # 完成实际的模型配置工作
One more thing here, we know that if we want to determine who two people are, we must classify the photos of the two people during training. For example, A is marked as 0 and B is marked as 1. This model is also trained in this way. There is a line of code in the load_dataset() function in load_face.py. The code is as follows:
# 标注数据，'LDY'文件夹下都是我的脸部图像，全部指定为0，另外一个文件夹下是同学的，全部指定为1
labels = np.array([0 if label.endswith('LDY') else 1 for label in labels])
This part of the file also needs to be modified, that is, change "LDY" to the name of your own folder.
If you want to identify multiple people, you also need to do tricks here. I have marked 0 and 1 here, so everyone will naturally know that I am doing two-person identification. If you want to identify more people, just Just make more marks.
Finally, there is one place that needs to be modified, which is the main function part of the train.py file:
# 此处文件地址是你收集的图片的文件夹地址
dataset = Dataset('D:\PyCharm-Community\Workplace\Face_Recognition\\face_data')
dataset.load()
model = Model()
model.build_model(dataset)
model.train(dataset)
# 此处地址是你保存训练好模型的地址
model.save_model(file_path='D:\PyCharm-Community\Workplace\Face_Recognition\\face_data\model\ldy_face_model.h5')
model.evaluate(dataset)
4. Face recognition
After the model is trained, you can finally test it with photos.
The file name is: Face_recognition.py, the code is as follows:
if __name__ == '__main__':
if len(sys.argv) != 1:
print("Usage:%s camera_id\r\n" % (sys.argv[0]))
sys.exit(0)
# 加载模型
model = Model()
model.load_model(file_path='D:\PyCharm-Community\Workplace\Face_Recognition\model\jianxin_face_model.h5')
# 框住人脸的矩形边框颜色
color = (0, 255, 0)
# 捕获指定摄像头的实时视频流
cap = cv2.VideoCapture(0)
# 人脸识别分类器本地存储路径
cascade_path = "D:\opencv\\build\etc\haarcascades\haarcascade_frontalface_default.xml"
# 循环检测识别人脸
while True:
ret, frame = cap.read() # 读取一帧视频
if ret is True:
# 图像灰化，降低计算复杂度
frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
else:
continue
# 使用人脸识别分类器，读入分类器
cascade = cv2.CascadeClassifier(cascade_path)
# 利用分类器识别出哪个区域为人脸
faceRects = cascade.detectMultiScale(frame_gray, scaleFactor=1.2, minNeighbors=3, minSize=(32, 32))
if len(faceRects) > 0:
for faceRect in faceRects:
x, y, w, h = faceRect
# 截取脸部图像提交给模型识别这是谁
image = frame[y - 10: y + h + 10, x - 10: x + w + 10]
faceID = model.face_predict(image)
print("faceID", faceID)
# 如果是“我”
if faceID == 0:
cv2.rectangle(frame, (x - 10, y - 10), (x + w + 10, y + h + 10), color, thickness=2)
# 文字提示是谁
cv2.putText(frame, 'Chengjianxin',
(x + 30, y + 30), # 坐标
cv2.FONT_HERSHEY_SIMPLEX, # 字体
1, # 字号
(255, 0, 255), # 颜色
2) # 字的线宽
else:
cv2.rectangle(frame, (x - 10, y - 10), (x + w + 10, y + h + 10), color, thickness=2)
cv2.putText(frame, 'Nobody',
(x + 30, y + 30), # 坐标
cv2.FONT_HERSHEY_SIMPLEX, # 字体
2, # 字号
(255, 0, 0), # 颜色
2) # 字的线宽
pass
cv2.imshow("Face Recognition", frame)
# 等待10毫秒看是否有按键输入
k = cv2.waitKey(10)
# 如果输入q则退出循环
if k & 0xFF == ord('q'):
break
# 释放摄像头并销毁所有窗口
cap.release()
cv2.destroyAllWindows()
其中有3处地方需要修改
# 加载模型
model = Model()
model.load_model(file_path='D:\PyCharm-Community\Workplace\Face_Recognition\model\jianxin_face_model.h5')
模型地址改成你自己的。
# 人脸识别分类器本地存储路径
cascade_path = "D:\opencv\\build\etc\haarcascades\haarcascade_frontalface_default.xml"
这个xml文件地址也改成你自己的，前面的get_face.py也是这样。
# 如果是“我”
if faceID == 0:
cv2.rectangle(frame, (x - 10, y - 10), (x + w + 10, y + h + 10), color, thickness=2)
# 文字提示是谁
cv2.putText(frame, 'Chengjianxin',
(x + 30, y + 30), # 坐标
cv2.FONT_HERSHEY_SIMPLEX, # 字体
1, # 字号
(255, 0, 255), # 颜色
2) # 字的线宽
else:
cv2.rectangle(frame, (x - 10, y - 10), (x + w + 10, y + h + 10), color, thickness=2)
cv2.putText(frame, 'Nobody',
(x + 30, y + 30), # 坐标
cv2.FONT_HERSHEY_SIMPLEX, # 字体
2, # 字号
(255, 0, 0), # 颜色
2) # 字的线宽
Just change the text prompts here to your own.