Face recognition algorithms are a fundamental component of any face detection and recognition system or software. Experts divide these algorithms into two core methods: geometric methods focus on distinguishing features, and photometric methods are used to extract values from images. These values are then compared to the template to remove discrepancies. These algorithms can also be divided into two more general categories - feature-based models and ensemble models. The former focuses on facial landmarks and analyzes their spatial parameters and correlations with other features, while the holistic approach considers the face as a whole.
Artificial neural networks are the most popular and successful approach in image recognition. The facial recognition algorithm is based on mathematical calculations, and neural networks perform a lot of mathematical operations at the same time.
These algorithms perform three main tasks: detecting faces in images, videos, or live streams; computing a mathematical model of a face; and comparing the model to a training set or database to identify or verify a person.
This article covers the most famous face recognition algorithms and key features. Since each method has its advantages for specific tasks, researchers are actively trying to combine methods and develop new techniques.
Convolutional Neural Networks (CNNs)
Convolutional neural network (CNN) is one of the breakthroughs in the development of artificial neural network (ANN) and artificial intelligence. It is one of the most popular algorithms in deep learning, a type of machine learning in which models learn to perform classification tasks directly on images, video, text, or sound. The model has shown impressive results in several domains: computer vision, natural language processing (NLP), and the largest image classification dataset (ImageNet). A CNN is an ordinary neural network with new layers - convolutional and pooling layers. CNNs can have tens and hundreds of such layers, each of which learns to detect a different imaging feature.
Eigenface
Eigenfaces is a face detection and recognition method for determining the variance of faces in image datasets. It uses these differences to encode and decode faces through machine learning. A set of eigenfaces is a collection of "normalized face components" determined through statistical analysis of a large number of face images. Facial features are assigned mathematical values because this method does not use digital pictures but a statistical database. Any face is a combination of these values in various percentages.
Fisherfaces
Fisherfaces is one of the most popular facial recognition algorithms; it is considered superior to many of its alternatives. As an improvement of the Eeigenfaces algorithm, it is often compared with Eigenfaces and is considered to be more successful in class discrimination during training. The main strength of the algorithm is its ability to interpolate and extrapolate lighting and facial expression changes. It has been reported that the Fisherfaces algorithm has an accuracy of 93% when combined with the PCA method in the preprocessing stage.
Kernel methods: PCA and SVM
Principal Component Analysis (PCA) is a general statistical method with many practical applications. When used during face recognition, PCA aims to reduce the size of the source data while retaining the most relevant information. It generates a set of weighted eigenvectors that in turn build eigenfaces—a large number of different images of faces. A linear combination of eigenfaces represents each image in the training set. PCA is used to receive these eigenvectors from the covariance matrix of the training image set. For each image, calculate its principal components (from 5 to 200). Other components encode subtle differences between faces and noise. The identification process consists of comparing the principal components of the unknown image with those of all other images.
A Support Vector Machine (SVM) is a machine learning algorithm that uses two sets of classification principles to distinguish between human faces and "non-human faces". For each class, the SVM model receives a labeled training dataset to classify new test data. The researchers applied linear and nonlinear SVM training models to face recognition. Recent results show that nonlinear training machines have larger headroom and better recognition and classification results.
Her cascade
Haar Cascade is an object detection method for locating objects on images. The algorithm learns from large numbers of positive and negative samples—the former containing objects of interest, while the latter contain anything but the object you're looking for. After training, the classifier can find objects of interest on new images. The method combines local binary pattern algorithm for face recognition, which is used for criminal identification. The Haar cascade classifier uses 200 (out of 6000) features to ensure 85-95% recognition rate even with expression changes.
3D recognition
The basic idea of 3D face recognition technology is the unique structure of human skull. Each individual's skull structure is unique and can be described by dozens of parameters. This method of facial recognition is based on comparing 3D facial scans to database patterns. It has an important advantage - make-up, facial hair, glasses and similar factors do not affect the detection and recognition process. The latest research uses techniques for mapping 3D geometric information onto regular 2D grids. It allows combining the descriptiveness of 3D data with the computational efficiency of 2D data and shows the highest performance reported by FRGC v2 (Face Recognition Grand Challenge 3D Face Database).
Skin Texture Analysis
Skin recognition technology has many applications - face detection algorithms, bad image filtering, gesture analysis, etc. It usually uses high-resolution images. The special case of skin texture analysis uses different unique parameters such as moles, skin color, skin color, etc. Recent studies based on combinations of texture features and skin tones have shown interesting results. Researchers use neural networks to develop and test skin recognition systems. The feed-forward neural network used in the project classifies input texture images as "skin" and "non-skin" and demonstrates impressive performance.
thermal imager
A thermal imager is a device used to monitor the temperature distribution of the surface to be inspected. The temperature distribution is displayed as a color picture with different colors corresponding to the temperature. The technology already has several practical applications adapted to global change — smartphone-based immunity certificates, remote fever detection, and thermal facial recognition. The thermal camera face recognition model is based on the unique temperature patterns of the human face. The consistent temperature "signature" of humans is measured with the imaginary part of thermal infrared (IR). There is an undeniable benefit to using thermal methods in facial recognition — make-up, beards, hats, and glasses don’t affect its accuracy. Also, it can distinguish twin siblings.
ANFIS
Adaptive Neuro-Fuzzy Interference System (ANFIS) is an artificial neural network. The method combines the principles of neural networks with the principles of fuzzy logic, combining their advantages in a single structure. ANFIS is used in the preprocessing stage to classify the image features extracted from the dataset. Data scientists combine this approach with various feature extraction algorithms. As a result, several studies report an incredible 97.1% classification accuracy for ANFIS after feature extraction using 2D principal component analysis.
Local Binary Pattern Histogram (LBPH)
The method uses Local Binary Patterns (LBP), a simple and effective texture operator in computer vision, which labels pixels in an image by thresholding each pixel's neighborhood and treating the result as a binary number. During the learning phase, the LBPH algorithm creates a histogram for each labeled and classified image. Each histogram represents each image in the training set. Thus, the actual recognition process implies comparing the histograms of any two images.
FaceNet
The face recognition system FaceNet developed by Google researchers in 2015 is based on the face recognition benchmark dataset. Available pre-trained models and various open-source third-party implementations make this system very extensive. Compared to other algorithms developed earlier, FaceNet has shown excellent results in terms of research investigations, test performance, and accuracy. FaceNet accurately extracts face embeddings, and high-quality features are used to train the face recognition system later.
NEC
A solution developed by Japanese technology company NEC can identify people with a high degree of accuracy while recognizing age changes. The solution uses adaptive region mix-matching, a model that focuses on highly similar segments for mapping. The NEC technique divides the input and registered images into small segments, and only focuses on the segments with large similarities. It allows the system to show higher recognition accuracy, even when the face is wearing a mask or glasses. As its underlying algorithm, the NEC solution uses Generalized Learning Vector Quantization (GLVQ).
Megvii (FACE++)
The Megvii algorithm is based on image detection and blurry image search techniques. The technical solution uses MegEngine, the company's proprietary deep learning framework built on big data. The company's technology successfully performs face information extraction, including several key functions: face and human body detection and tracking, face recognition and clustering, key point detection, face attribute estimation, and face search engine.
Facial recognition: combining different technologies
Each facial recognition technology has its own characteristics that make it effective. However, recent research proves that the best results are achieved through a combination of different algorithms and methods. These combinations are designed to account for many of the usual problems in the facial recognition process—variance in facial expressions, poses, lighting conditions, image noise, and more. The latest experiments combine the LBP algorithm with advanced image processing techniques: bilateral filtering, histogram equalization, contrast adjustment, and image blending. Such techniques show significant improvements over LBP codes and look very promising for further research.
Summarize
There are various face recognition algorithms and methods. Although they all have one main goal, they can be task- and problem-specific. Depending on the purpose of use and implementation, they range from neural networks and mathematical models to technical solutions of private companies.
This article covers the broadest range of algorithms and methods. However, more research and scientific experiments show that there are undeniable benefits of combining different algorithms in the facial recognition process for better results. It leads to the emergence of new technologies and specific-purpose methods.
Frequently Asked Questions about Face Recognition Algorithms
How does the face detection algorithm work?
Typically, a specific neural network is trained to detect facial landmarks and distinguish faces from other objects in an image. Landmarks are generic human facial features such as eyes, nose, mouth, eyebrows, etc. How this is actually implemented varies from algorithm to algorithm.
What is a face recognition algorithm?
A face recognition algorithm is a method of building a biometric face model for further analysis and face recognition process.
How to do face detection in OpenCV?
The Open Source Computer Vision Library (OpenCV) is a popular open source general algorithm library for computer vision algorithms, image processing, and numerical values. Using OpenCV, the face recognition process can be done in three steps:
- Face Detection
- Feature Extraction Using Face Embeddings
- face recognition
How to train the face recognition algorithm?
Before facial recognition algorithms are ready to perform the necessary tasks, they need to process massive amounts of data—accurately labeled sets of images. These collections are used to develop machine learning models.
Extended reading
Face recognition technology is a high-precision, easy-to-use, high-stability, and difficult-to-counterfeit biometric technology, which has extremely broad market application prospects. There is a wide range of needs in industries and departments such as public security, national defense, customs, transportation, finance, social security, medical care and other civil security controls.
The R&D personnel of our TSINGSEE Qingxi video are also actively developing AI technologies such as face detection, face recognition, people flow statistics, and helmet detection recently, and actively integrating them into the existing video platform. A typical example is EasyCVR video fusion cloud service, which has the capabilities of AI face recognition, license plate recognition, voice intercom, PTZ control, sound and light alarm, monitoring video analysis and data aggregation, and is widely used in intelligent access control of residential areas and buildings. Suspicious people wandering detection around the perimeter, people flow statistics in scenic spots and other scenarios.