1. Project Background

Facial expression recognition is a very important but very complex subject in pattern recognition. Firstly, the research background and development process of computer facial expression recognition technology are briefly reviewed. Then, a classification review of recent methods for facial expression recognition is given. Through the analysis and comparison of various recognition methods, several aspects that need to be considered in the practical application of facial expression recognition technology are proposed, and then the development direction of facial expression recognition technology in the future is prospected.

2. Recognition effect display

3. Identify video presentations

Python based OpenCV facial expression recognition system [source code & deployment tutorial]

4. Implementation method analysis

Face expression recognition needs to use face detection technology. After the face is recognized, the expression image is preprocessed (color image grayscale, image geometric normalization and illumination preprocessing), and then the expression features are extracted. analysis, so as to realize the recognition of expressions. There has been a lot of research on facial expression recognition at home and abroad in recent years, and many algorithms have emerged, but the accuracy of facial expression recognition still needs to be improved.

5. Implementation method analysis

5. Algorithm flow chart

6. Introduction of fer2013 facial expression dataset

The Fer2013 facial expression dataset consists of 35,886 facial expression pictures, including 28,708 test pictures (Training), 3,589 public verification pictures (PublicTest) and 3589 private verification pictures (PrivateTest). It is composed of 48×48 grayscale images, and there are 7 kinds of expressions, which correspond to the digital labels 0-6 respectively. The labels corresponding to the specific expressions are as follows: 0 anger angry; 1 disgust disgust; 2 fear fear; 3 happy happy; 4 sad sad; 5 surprised surprised; 6 normal neutral.

However, the data set does not directly give pictures, but saves the data of expressions, picture data, and purposes in a csv file, as shown in the following figure, as shown in the

above figure, the first picture is the beginning of the csv file, the first The row is the header, explaining the meaning of each column of data, the first column is the emoji tag, the second column is the picture data, here is the original picture data, and the last column is the purpose.

7. Expression picture extraction

After knowing the data structure, it is easy to do, use pandas to parse the csv file, (the simple usage of pandas can be viewed in this blog: https://blog.csdn.net/rookie_wei/article/details/82974277 ), and then the original image The data is saved as a jpg file, and is classified according to the purpose and label, and saved to the corresponding folder. The code is relatively simple, and detailed remarks are made. The complete code is directly as follows

Code

#encoding:utf-8
import pandas as pd
import numpy as np
import scipy.misc as sm
import os
 
emotions = {
    '0':'anger', #生气
    '1':'disgust', #厌恶
    '2':'fear', #恐惧
    '3':'happy', #开心
    '4':'sad', #伤心
    '5':'surprised', #惊讶
    '6':'normal', #中性
}
 
#创建文件夹
def createDir(dir):
    if os.path.exists(dir) is False:
        os.makedirs(dir)
 
def saveImageFromFer2013(file):
 
 
    #读取csv文件
    faces_data = pd.read_csv(file)
    imageCount = 0
    #遍历csv文件内容，并将图片数据按分类保存
    for index in range(len(faces_data)):
        #解析每一行csv文件内容
        emotion_data = faces_data.loc[index][0]
        image_data = faces_data.loc[index][1]
        usage_data = faces_data.loc[index][2]
        #将图片数据转换成48*48
        data_array = list(map(float, image_data.split()))
        data_array = np.asarray(data_array)
        image = data_array.reshape(48, 48)
 
        #选择分类，并创建文件名
        dirName = usage_data
        emotionName = emotions[str(emotion_data)]
 
        #图片要保存的文件夹
        imagePath = os.path.join(dirName, emotionName)
 
        # 创建“用途文件夹”和“表情”文件夹
        createDir(dirName)
        createDir(imagePath)
 
        #图片文件名
        imageName = os.path.join(imagePath, str(index) + '.jpg')
 
        sm.toimage(image).save(imageName)
        imageCount = index
    print('总共有' + str(imageCount) + '张图片')
 
 
if __name__ == '__main__':
    saveImageFromFer2013('fer2013.csv')

After running the above code, you will get 3 folders, there are subfolders with corresponding expressions under the

files, and there are corresponding pictures under the subfolders

8. System Integration

9. Complete source code & Ministry of Environment summer video tutorial & dataset & custom UI interface

Baidu Bread can download the source code by searching for the title name

10. References

[1] Synthesis of Facial Expression Animation Driven by Artificial Mental Model [J]. Gu Xuejing, Xu Guangbin, Xiong Wei, Wang Zhiliang. Computer Engineering and Applications. 2008(08)
[2] CNN-based face recognition decompression and sleep aid WeChat applet [J]. Hu Wenhan, Li Jianzhe, Zeng Tianlun, Shen Qing. Computer Programming Skills and Maintenance. 2022(08)
[3] Application of wavelet transform in facial expression recognition research [J]. Cui Jingxia. Journal of Changchun University of Science and Technology (Natural Science Edition). 2011(03)
[4] Analysis and Recognition of Facial Expression Based on Active Appearance Model [J]. Zuo Kunlong, Liu Wenyao. Optoelectronics·Laser. 2004(07)
[5] Overview of facial expression recognition methods [J]. Chen Jun. China Collective Economy. 2009(16)
[6] A new method of facial expression recognition based on semantic attributes [J]. Yu Yongbin, Liu Qingyi, Mao Qirong, Zhan Yongzhao. Small Microcomputer System. 2016(02)
[7] Research progress of facial expression feature extraction [J]. Qiao Ming, Chen Yajun. Journal of Chongqing Institute of Technology (Natural Science Edition). 2008(06)
[8] Research progress of automatic facial expression recognition technology [J]. Zhang Jiashu, Chen Hui, Li Defang, Luo Xiaobin, Xia Xiaodong. Journal of Southwest Jiaotong University. 2005(03)
[9] Facial Expression Recognition Based on Multi-Group Genetic Algorithm [J]. Zhu Yani, Wu Guohua, Du Jiayou. Journal of Hangzhou Dianzi University. 2013(01)
[10] Facial expression data enhancement method based on generative adversarial network [J]. Sun Xiao, Ding Xiaolong. Computer Engineering and Applications. 2020(04)