Face and facial expression recognition system based on deep learning [including Python source code + PyqtUI interface + detailed explanation of the principle]

Demo

Abstract: 面部表情识别（Facial Expression Recognition）It is a technology to recognize facial expressions in human images through technical means. This article introduces the technical principle of its implementation in detail, and at the same time gives the complete Pythonimplementation code, a well-trained deep learning model, and PyQTrealizes the UI interface, which makes it easier to display the functions. The expression recognition system can carry out expression recognition through 图片, 视频, and 摄像头3 ways. This article provides complete Python code and tutorials for reference and study by interested partners. See the end of the article for how to obtain complete code resource files.

Click to jump to the end of the article "Complete related documents and source code" to obtain

---

foreword

Changes in human facial expressions can convey changes in their inner emotions, and expressions are a true portrayal of human inner world. At present, the most common one is to define human expressions as 7, namely: 悲伤, 害怕, 厌恶, 高兴, 生气, 惊讶and 中立. These also constitute the seven basic facial expressions in today's research on facial expression recognition.

Because different facial expressions can reflect people's emotional changes and psychological changes in different situations, the recognition of facial expressions has very important research significance and practical application value for the study of human behavior and psychological activities. Nowadays, facial expression recognition mainly uses computers to analyze and recognize human facial expressions, so as to analyze and recognize emotional changes, which is of great significance in human-computer interaction, social network analysis, telemedicine, and criminal investigation monitoring.

According to the deep learning network model, the blogger has developed a simple facial expression recognition system, which can recognize facial expressions in 3 ways 图片, , 视频, and display the recognition results. 摄像头It can recognize 悲伤, 害怕, 厌恶, 高兴, 生气, 惊讶and 中立these 7 common expressions. Interested friends can try it by themselves.

觉得不错的小伙伴，感谢点赞、关注加收藏！如果大家有任何建议或意见，欢迎在评论区留言交流！

The interface is as follows:

1. Software core function introduction and effect demonstration

(1) Image facial expression recognition
Click 打开图片the button and select the image to be recognized. The operation demonstration is as follows:

(2) Video facial expression recognition
Click 打开视频the button and select the video to be recognized. The operation demonstration is as follows:

(3) Camera facial expression recognition
Click 打开摄像头the button to turn on the camera, click the button again to turn off the camera, the operation demonstration is as follows:

2. The basic principle of facial expression recognition

1. Basic principles

Face expression recognition can usually be divided into four steps: including 图像获取, 人脸检测, 面部图像预处理and 表情分类. Among them, face detection, facial image preprocessing (facial feature extraction) and facial expression classification are the three key links of facial expression recognition. The basic process of facial expression recognition is shown in the figure below:

We can first use the more commonly used face detection library face_recognitionto detect the face area and draw it on the picture. The core code is as follows:

image_path = 'TestImages/2.jpg'
image = cv2.imread(image_path)
face_locations = face_recognition.face_locations(image)
num = len(face_locations)
face = []
if num:
    for face_location in face_locations:
        top, right, bottom, left = face_location
        face.append(image[top:bottom, left:right])
        image = cv2.rectangle(image,(left, top), (right, bottom), (50, 50, 250),3)

After the above-mentioned face area is detected, the face area is extracted and converted into the 48*48 size required by the face detection model. Pass in the model to detect facial expressions, take the classification with the highest probability, and draw the expression recognition results on the picture. The core code is as follows:

# 加载训练好的DenseNet121深度学习模型
inputs = keras.Input(shape=(48, 48, 1), batch_size=64)
x = create_dense_net(7, inputs, include_top=True, depth=121, nb_dense_block=4, growth_rate=16, nb_filter=-1,
                     nb_layers_per_block=[6, 12, 32, 32], bottleneck=True, reduction=0.5, dropout_rate=0.2,
                     activation='softmax')
model = tf.keras.Model(inputs, x, name='densenet121')
filepath = 'models/DenseNet121.h5'
model.load_weights(filepath)

# 进行人脸表情识别
image = cv2.imread(image_path)
frame, faces, locations = face_detect(image)
if faces is not None:
    for i in range(len(faces)):
        top, right, bottom, left = locations[i]
        face = cv2.cvtColor(faces[i], cv2.COLOR_BGR2GRAY)
        face = cv2.resize(face, (48, 48))
        face = face / 255.0
        num = np.argmax(model.predict(np.reshape(face, (-1, 48, 48, 1))))
        label = labeldict[num]
        frame = cv2.putText(frame, label, (left, top-10), cv2.FONT_ITALIC, 0.8, (0, 0, 250), 2,
                            cv2.LINE_AA)
cv2.imshow('frame',frame)
cv2.waitKey(0)

Based on the above content, the blogger has python+ pyqt5developed a facial expression recognition system with a simple interface. That is the content introduced in the second part.
The complete source code, UI interface code and other related files related to the facial expression recognition system have been packaged and uploaded, and interested partners can obtain them by themselves through the download link.

---

【method of obtaining】

Follow the business card GZH below: [Axu Algorithm and Machine Learning], reply [Expression Recognition] to get the download method

All the complete program files involved in this article: including python source code, UI files, etc. (see the picture below), see the end of the article for how to obtain them:

Note: The code is developed with Pycharm+Python3.8, the main program of the running interface is MainProgram.py, the picture test script can run imgTest.py, and the camera test script can run cameraTest.py. In order to ensure the smooth operation of the program, please follow 程序环境配置说明.txtthe configuration software to run the required environment.

Follow the business card below GZH: [Axu Algorithm and Machine Learning], reply [Expression Recognition] to get the download method

---

conclusion

The above is the entire content of the facial expression recognition system developed by the blogger. Due to the limited ability of the blogger, it is inevitable that there are omissions. I hope that the friends can criticize and correct me. If you have any suggestions or opinions on this article, please
comment District message exchange!

Friends who feel good, thank you for your likes, attention and collection!