Face recognition is a biometric technology for identification based on human facial feature information . A series of related technologies that use a video camera or camera to collect images or video streams containing human faces, automatically detect and track human faces in the images , and then perform facial recognition on the detected faces, usually also called portrait recognition and facial recognition. .
Development History
The research on the face recognition system began in the 1960s. After the 1980s, it was improved with the development of computer technology and optical imaging technology, and it really entered the primary application stage in the late 1990s. Technology implementation is the main focus; the key to the success of the face recognition system is whether it has a cutting-edge core algorithm, and the recognition result has a practical recognition rate and recognition speed ; the "face recognition system" integrates artificial intelligence, machine recognition, machine learning , model theory, expert system, video image processing and other professional technologies, and at the same time need to combine the theory and implementation of intermediate value processing. transformation.
technical features
The traditional face recognition technology is mainly based on face recognition of visible light images , which is also a familiar recognition method and has a research and development history of more than 30 years. However, this method has insurmountable defects, especially when the ambient light changes, the recognition effect will drop sharply, which cannot meet the needs of the actual system. Solutions to lighting problems include 3D image face recognition and thermal imaging face recognition . But these two technologies are still far from mature, and the recognition effect is not satisfactory.
One solution that is rapidly developing is the multi-light source face recognition technology based on active near-infrared images. It can overcome the influence of light changes and has achieved excellent recognition performance. The overall system performance in terms of accuracy, stability and speed exceeds that of 3D image face recognition. This technology has developed rapidly in the past two or three years, making face recognition technology gradually practical.
The human face is born with the same biological characteristics (fingerprint, iris, etc.) Has the following characteristics:
Non-mandatory: Users do not need to cooperate with face acquisition equipment, and can acquire face images almost unconsciously. This sampling method is not "mandatory" ;
Non-contact: users can obtain face images without direct contact with the device ;
Concurrency: Sorting, judgment and recognition of multiple faces can be performed in practical application scenarios;
In addition, it also conforms to the visual characteristics: the characteristics of "knowing people by appearance", as well as the characteristics of simple operation, intuitive results, and good concealment.
technical process
The face recognition system mainly includes four components: face image acquisition and detection, face image preprocessing, face image feature extraction, and matching and recognition .
Face image acquisition and detection
Face image collection: Different face images can be collected through the camera lens, such as static images, dynamic images, different positions, different expressions, etc. can be well collected. When the user is within the shooting range of the collection device, the collection device will automatically search for and capture the user's face image.
Face detection: Face detection is mainly used for preprocessing of face recognition in practice, that is, to accurately mark the position and size of the face in the image . The pattern features contained in the face image are very rich, such as histogram features, color features, template features, structural features, and Haar features . Face detection is to pick out the useful information and use these features to realize face detection.
The mainstream face detection method uses the Adaboost learning algorithm based on the above features. The Adaboost algorithm is a method for classification. It combines some weaker classification methods to form a new and strong classification method .
In the face detection process, the Adaboost algorithm is used to select some rectangular features (weak classifiers) that best represent the face, and the weak classifiers are constructed into a strong classifier according to weighted voting, and then several strong classifiers obtained from training are A cascaded classifier is formed in series to effectively improve the detection speed of the classifier.
Face Image Preprocessing
Face image preprocessing: Image preprocessing for faces is the process of processing images based on face detection results and finally serving for feature extraction. The original image acquired by the system cannot be used directly due to various conditions and random interference, and image preprocessing such as grayscale correction and noise filtering must be performed on it in the early stage of image processing. For face images, the preprocessing process mainly includes light compensation, grayscale transformation, histogram equalization, normalization, geometric correction, filtering and sharpening of face images .
Feature Extraction of Face Image
Face image feature extraction: The features that can be used by the face recognition system are usually divided into visual features, pixel statistical features, face image transformation coefficient features, face image algebraic features, etc. Face feature extraction is carried out for certain features of the face. Face feature extraction, also known as face representation, is the process of modeling the features of a face. The methods of face feature extraction can be summarized into two categories: one is the knowledge-based representation method; the other is the representation method based on algebraic features or statistical learning.
The knowledge-based representation method is mainly based on the shape description of facial organs and the distance characteristics between them to obtain feature data that is helpful for face classification, and its feature components usually include Euclidean distance, curvature, and angle between feature points, etc. . The human face is composed of parts such as eyes, nose, mouth, and chin. The geometric description of these parts and the structural relationship between them can be used as important features for recognizing faces. These features are called geometric features. Knowledge-based face representation mainly includes methods based on geometric features and template matching.
Face Image Matching and Recognition
Face image matching and recognition: The extracted feature data of the face image is searched and matched with the feature template stored in the database. By setting a threshold, when the similarity exceeds this threshold, the matching result is output . Face recognition is to compare the face features to be recognized with the obtained face feature templates, and judge the identity information of the faces according to the degree of similarity. This process is further divided into two categories: one is confirmation, which is a process of one-to-one image comparison, and the other is identification, which is a process of one-to-many image matching and comparison.
recognition algorithm
face recognition
Generally speaking, a face recognition system includes image capture, face positioning, image preprocessing, and face recognition (identity confirmation or identity search). The input of the system is generally one or a series of face images with undetermined identities, as well as several face images with known identities in the face database or corresponding codes, and its output is a series of similarity scores, indicating that The identity of the face to be recognized.
Classification of face recognition algorithms
Feature-based recognition algorithms based on facial feature points .
Appearance-based recognition algorithms based on the entire face image .
Template-based recognition algorithms .
Recognition algorithms using neural network .
Based on the lighting estimation model theory
A lighting preprocessing method based on Gamma grayscale correction is proposed, and on the basis of the lighting estimation model, the corresponding lighting compensation and lighting balance strategies are carried out.
Optimal Deformation Statistical Correction Theory
Based on the correction theory of statistical deformation, optimize the face posture; strengthen the iterative theory
The enhanced iteration theory is an effective extension of the DLFA face detection algorithm;
Original real-time feature recognition theory
This theory focuses on the intermediate value processing of real-time face data, so that the best matching effect can be achieved between the recognition rate and recognition performance
identification data
Face recognition needs to accumulate a large amount of face image-related data collected to verify the algorithm and continuously improve the recognition accuracy . These data, such as A Neural Network Face Recognition Assignment (neural network face recognition data), orl face database, The face recognition database of MIT Biological and Computing Learning Center, the face recognition data of School of Computer and Electronic Engineering, University of Essex, etc.
Cooperation degree
The existing face recognition system can achieve satisfactory results when the user cooperates and the acquisition conditions are ideal. However, when the user does not cooperate and the acquisition conditions are not ideal, the recognition rate of the existing system will drop sharply. For example, when a face is compared, there is a discrepancy with the face stored in the system, such as shaving the beard, changing the hairstyle, adding more glasses, or changing the expression, which may cause the comparison to fail.
advantage difficulty
Advantage
The advantage of face recognition lies in its naturalness and the characteristics of not being noticed by the individual being tested.
The so-called naturalness means that the identification method is the same as the biological characteristics used by human beings (or even other creatures) to identify individuals. For example, face recognition, human beings also distinguish and confirm identities by observing and comparing faces. In addition, natural recognition includes speech recognition, body shape recognition, etc., while fingerprint recognition, iris recognition, etc. are not natural, because humans or others Creatures do not distinguish individuals by such biometrics.
The unobtrusive feature is also important to a method of identification, making it less objectionable and less likely to be deceived because it is less noticeable . Face recognition has this feature. It uses visible light to obtain face image information completely. Unlike fingerprint recognition or iris recognition, it needs to use electronic pressure sensors to collect fingerprints, or use infrared rays to collect iris images. These special collection methods are very easy. be detected and thus more likely to be deceived by a disguise.
difficulty
Face recognition is considered to be one of the most difficult research topics in the field of biometrics and even in the field of artificial intelligence. The difficulty of face recognition is mainly caused by the characteristics of the face as a biological feature.
similarity
face similarity
There is little difference between different individuals, all the structures of the faces are similar, and even the structures and shapes of the facial organs are very similar. Such a feature is beneficial for using human faces for positioning, but it is unfavorable for using human faces to distinguish human individuals .
variability
The shape of the face is very unstable. People can produce many expressions through changes in the face, and the visual images of the face are also very different at different viewing angles. In addition, face recognition is also affected by lighting conditions (such as day and night, Indoor and outdoor, etc.), many face coverings (such as masks, sunglasses, hair, beards, etc.), age and other factors.
In face recognition, the first type of change should be amplified as a criterion for distinguishing individuals, while the second type of change should be eliminated because they can represent the same individual. The first type of change is usually called inter-class difference, while the second type of change is called intra-class difference. For faces, intra-class variation is often greater than inter-class variation, making it extremely difficult to distinguish individuals using inter-class variation when interfered by intra-class variation.
The main purpose
Face recognition is mainly used for identification
Face recognition is mainly used for identification. Due to the rapid popularization of video surveillance, many video surveillance applications urgently need a long-distance and rapid identification technology in the non-cooperative state of the user, in order to quickly confirm the identity of personnel at a long distance and realize intelligent early warning. Face recognition technology is undoubtedly the best choice. The use of fast face detection technology can find faces in real time from surveillance video images and compare them with face databases in real time to achieve rapid identity recognition.
Application prospect
Biometric technology has been widely used in government, military, banking, social welfare, e-commerce, security and defense and other fields. For example, a depositor walks into a bank and withdraws money without bringing a bank card or recalling a password. When he withdraws money at an ATM, a camera scans the user's eyes, and then quickly and accurately Successfully completed the user identification and completed the business. This is a real scene that happened in a business department of Union Bank of Texas. The sales department uses the "iris recognition system" in modern biometric technology. In addition, after the "9.11" incident in the United States, anti-terrorist activities have become the consensus of governments of all countries, and it is very important to strengthen the security and defense of airports. The facial recognition technology of American company Visag has shown its prowess at two airports in the United States. It can pick out a certain face in a crowded crowd and judge whether it is a wanted criminal.
Cases of burglary, robbery, and wounding frequently occur in the current society. In view of this reason, anti-theft doors have begun to enter thousands of households and bring peace to the family; however, with the development of society and the advancement of technology , the acceleration of the pace of life, the improvement of consumption levels, people's expectations for home furnishing are getting higher and higher, and the requirements for convenience are becoming more and more urgent. Anti-theft doors based on traditional purely mechanical designs are difficult to quickly To meet these emerging needs: convenience, door opening record and other functions. Face recognition technology has been widely recognized, but its application threshold is still high: high technical threshold (long development cycle), high economic threshold (high price).
Face recognition products have been widely used in finance, justice, military, public security, border inspection, government, aerospace, electric power, factories, education, medical care and many enterprises and institutions. With the further maturity of technology and the improvement of social recognition, face recognition technology will be applied in more fields.
1. Enterprise and residential security and management. Such as face recognition access control and attendance system, face recognition anti-theft door, etc.
2. Electronic passport and ID card. China's e-passport program is being intensively planned and implemented by the Ministry of Public Security.
3. Public security, justice and criminal investigation. For example, use the face recognition system and the Internet to search for fugitives across the country.
4. Self-service.
5. Information security. Such as computer login, e-government and e-commerce. All transactions in e-commerce are completed online, and many approval processes in e-government have also been moved online. At present, the authorization of transactions or approvals is realized by passwords. If the passwords are stolen, security cannot be guaranteed. However, with the use of biometrics, the digital and real identities of the parties on the Internet can be unified, thereby greatly increasing the reliability of e-commerce and e-government systems.
main products
digital camera
Face AF and Smile Shutter Technology: The first is face capture. It judges according to the parts of the human head. Firstly, the head is determined, and then the head features such as eyes and mouth are judged. Through the comparison of the feature database, it is confirmed that it is a human face, and the face capture is completed. Then focus automatically on the face, which can greatly improve the clarity of the photos taken.
Smile shutter technology is based on face recognition, completes facial capture, and then starts to judge the degree of upward bending of the mouth and downward bending of the eyes to judge whether it is a smile. All the above captures and comparisons are done in the context of comparing the feature library, so the feature library is the basis, which contains various typical face and smile feature data.
access control system
Secured areas can identify whoever tries to enter through facial recognition. Facial recognition systems can be used in businesses, residential security and management. Such as face recognition access control and attendance system, face recognition anti-theft door, etc.
Face recognition access control
Face recognition access control is a safe and practical access control product based on advanced face recognition technology combined with mature ID card and fingerprint recognition technology. The product adopts a split design, and the collection of face, fingerprint and ID card information is separated from the inside and outside of biometric information identification and access control, which is highly practical, safe and reliable. The system adopts network information encrypted transmission, supports remote control and management, and can be widely used in access control security control in key areas such as banks, military, public security, and intelligent buildings.
identification
Such as electronic passports and ID cards. This may be the future scale application. The International Civil Aviation Organization has determined that from April 1, 2010, its 118 member countries and regions must use machine-readable passports. Face recognition technology is the first recognition mode , and this regulation has become an international standard. The United States has required countries with which it has a visa-free agreement to use an electronic passport system that combines biometric features such as face fingerprints before October 26, 2006. By the end of 2006, more than 50 countries had implemented such a system. The Transportation Security Administration plans to roll out a biometric-based domestic universal travel document across the United States. Many countries in Europe are also planning or implementing similar plans to use biometric documents to identify and manage passengers. China's e-passport program is being intensively planned and implemented by the Ministry of Public Security.
The crowd can be monitored in public places such as airports, stadiums, and supermarkets, such as installing surveillance systems at airports to prevent terrorists from boarding planes. Such as the bank's automatic teller machine, if the user's card and password are stolen, others will steal cash. Applying face recognition at the same time will avoid this from happening. By querying the target portrait data to find out whether there is basic information about key populations in the database. For example, installing systems at airports or stations to catch fugitives.
Internet application
Use face recognition to assist credit card network payment to prevent non-credit card owners from using credit cards. Such as computer login, e-government and e-commerce. All transactions in e-commerce are completed online, and many approval processes in e-government have also been moved online. At present, the authorization of transactions or approvals is achieved by passwords. If the password is stolen, there is no guarantee of security. If biometrics are used, the online digital and real identities of the parties can be unified. Thereby greatly increasing the reliability of e-commerce and e-government system.
entertainment application
Face recognition technology is widely used in daily life, such as camera shooting, picture comparison, etc. Especially in the past two years, dating shows have been in full swing, and Zhejiang TV's "Love Lianliankan" best couple portrait link uses face comparison technology To test the similarity of the faces of the hero and heroine.
With the rise of the mobile Internet, some face recognition technology developers have applied this technology to the entertainment field, such as the application of happy star faces, etc., according to the contours, skin color, texture, texture, color, lighting and other characteristics of the face to identify Calculate the similarity between the main character and the star in the photo.
Application example
On April 13, 2012, the face recognition system project in the Beijing-Shanghai high-speed railway security check area began to invite bids. The security check areas of Shanghai Hongqiao Station, Tianjin West Station and Jinan West Station will be installed with a high-tech security check system for identity recognition - face Recognition system to assist the public security department in arresting fugitives, a high-tech innovative enterprise of face recognition products and system solutions. The core technology research and development team is composed of experts in this field, focusing on face recognition technology as the core, product design and research and development projects covering multiple fields such as attendance, access control and security. Today's face recognition products have been widely used in finance, justice, military, public security, border inspection, political law, aerospace, electric power, factories, education, medical care, and many enterprises and institutions.
On September 5, 2013, the facial recognition payment system was unveiled at the China International Finance Exhibition. The face-swiping payment system is based on the biometric cloud financial platform independently developed by Tiancheng Shengye. It integrates the military-level face recognition algorithm with independent intellectual property rights with the existing payment system, and connects the payment, transfer, and settlement in our lives. and transactional aspects. When paying, people no longer need bank cards, passbooks, passwords, or even mobile phones. They only need to nod to the camera and show a smiling face. The face-swiping payment system will complete identity confirmation, account reading, and transfer payment within seconds. , transaction confirmation and other one-stop payment links to create a better payment experience for users.
From August 2014, Japan will restart the experiment of the face recognition system at the entry and exit inspection (border inspection) of some airports. The first experiment implemented in 2012 was suspended due to frequent errors, but the Ministry of Justice decided that the border inspection speed needed to be improved in order to prepare for the 2020 Tokyo Olympics, so it decided to restart the experiment. The experiment will be carried out for about 5 weeks from August 2014, and the subjects will be Japanese who boarded at Haneda Airport and Narita Airport. The company responsible for the experiment will be finalized in the near future. The Japanese government has installed automatic border inspection gates at various airports that can only be passed through fingerprint recognition, but the utilization rate of passengers is not high because fingerprints need to be registered in advance. Face recognition does not require prior registration.
On March 15, 2015, the Hannover IT Expo (CeBIT) opened in Germany. Alibaba founder Jack Ma, as the only entrepreneur representative invited, gave a keynote speech at the opening ceremony. After his speech, Jack Ma also demonstrated Ant Financial's Smile to Pay face-scanning technology for German Chancellor Angela Merkel and Chinese Vice Premier Ma Kai, and bought gifts for the guests on the spot with his own face. The gift Ma Yun chose was a 1948 Hannover commemorative stamp on Taobao. He logs in to Taobao with his mobile phone, first selects the product; the second step enters the payment system, and the face-scanning page appears after confirming the payment; then scans the face (after taking a photo) for background authentication; then it shows that the payment is successful. Ma Yun presented a special gift to German Chancellor Angela Merkel on the spot: a commemorative edition of the German calendar page, which happened to be the date of birth of the female chancellor.
laws and regulations
On July 28, 2021, with the rapid development of information technology, face recognition has gradually penetrated into all aspects of people's lives. While face recognition technology plays a huge role in many fields, it is also being abused. The Supreme People's Court issued a judicial interpretation to regulate face recognition.
On July 28, 2021, the "Provisions of the Supreme People's Court on Several Issues Concerning the Application of Law in the Trial of Civil Cases Related to the Use of Face Recognition Technology to Handle Personal Information" was officially released. The "Regulations" clarify: "Property service companies or other building managers use face recognition as the only verification method for owners or property users to enter and exit the property service area. If the owner or property user who disagrees requests it to provide other reasonable verification methods , the people’s court shall support it according to law.” According to this regulation, when the community property uses the face recognition access control system to enter face information, it shall obtain the consent of the owner or property user. The property shall provide alternative verification methods, and shall not infringe on the personal rights and other legitimate rights and interests of the owner or property user.
In August 2021, as face recognition technology is widely used in real life, people's concerns about its abuse are also increasing, and calls for strengthening the protection of face information are growing louder. The Supreme Law issued the "Regulations on Several Issues Concerning the Application of Law in the Trial of Civil Cases Related to the Use of Face Recognition Technology to Handle Personal Information", which clearly defines the collection of face information without the separate consent of natural persons or their guardians as "infringement" . It not only provides a legal basis for the judiciary to handle such disputes, but also strengthens citizens' legal confidence in the protection of their personal rights and interests.
On August 20, 2021, the 30th meeting of the Standing Committee of the Thirteenth National People's Congress voted to pass the "Personal Information Protection Law of the People's Republic of China", which will come into force on November 1, 2021. In response to the abuse of face recognition technology, this law requires that when image collection and personal identification equipment are installed in public places, prominent reminder signs should be set up; personal images and identification information collected can only be used for the purpose of maintaining public safety.
On November 14, 2021, the Cyberspace Administration of China issued the "Regulations on the Administration of Network Data Security (Draft for Comment)" and publicly solicited opinions from the public. The draft for comments proposes that if data processors use biometrics for personal identity authentication, they should conduct a risk assessment of necessity and security, and must not use biometrics such as face, gait, fingerprint, iris, and voiceprint as the only personal identity Authentication methods to enforce individual consent to the collection of their personal biometric information.
Reposted from: Face Recognition_Baidu Encyclopedia