Overview of Fingerprint Recognition (2): Fingerprint Sensors

This article is mainly based on the content of Chapter 2 " Fingerprint Sensing " of the third edition of " Handbook of Fingerprint Recognition " . This article will be updated from time to time to reflect some new progress and thinking.

1 Introduction

The fingerprint identification system uses sensors, image processing, and pattern recognition technologies to automatically identify whether two fingerprints are consistent. The fingerprint identification system mainly has three modules, which are fingerprint collection module, feature extraction module and matching module. First, the fingerprint image is obtained by the sensor, and then some salient features are extracted from the image (these features are more suitable for identification tasks), and finally the fingerprint features are matched to obtain a matching score. As the first module, the fingerprint collector is obviously crucial to the whole system.

Basic process of fingerprint identification system

Why do we need a dedicated fingerprint collector? Because the dedicated fingerprint collector can acquire fingerprint images with high contrast, which is beneficial to feature extraction and identification. However, the fingerprint images captured by ordinary cameras (similar to the fingerprints seen by the naked eye) have very low contrast, which is not conducive to fingerprint recognition algorithms. See the image below for a comparison of the two images.

For the same finger, the image obtained by the fingerprint collector (left image) has clearer contrast than the photo taken by the mobile phone camera (right image), which is more conducive to the recognition algorithm.

The earliest fingerprint collection technology is the ink method, which has a history of at least hundreds of years. In the 1990s, live fingerprint collectors emerged, using various sensor technologies (such as optical, capacitive, and ultrasonic) to obtain digital fingerprint images directly from the user's finger. After 30 years of development, fingerprint collectors with multiple technologies and various forms have emerged continuously, which has promoted the application of fingerprint recognition technology in many fields such as public security, consumer electronics, and e-commerce.

Development of fingerprint collectors (Jain et al., 2016)

Fingerprint collector and sensor are two different concepts, and the latter is an internal module of the former. Collectors with very different shapes may be based on similar sensor technologies. For example, the following pictures are fingerprint collectors based on optical sensing technology, which can collect four-finger, rolling fingerprints, flat fingerprints, and partial flat fingerprints. Collectors with similar shapes may be based on completely different sensor technologies. For example, fingerprint sensors under the screen of mobile phones based on ultrasound and optics, although the technical principles are completely different, ordinary users may not feel the difference between them.

Various forms of fingerprint collectors based on optical sensing technology collect four-finger, rolling fingerprints, flat fingerprints, and partial flat fingerprints.

The following section first introduces off-line fingerprint collection and various living fingerprint collection technologies, and then discusses fingerprint image quality issues.

2. Offline collection

2.1 Ink method

The ink method is to dip the finger in ink, press the finger or roll the finger on a special pointing card. The upper two rows of the fingerprint card in the figure below are the fingerprints collected by rolling, which can collect as much information as possible; the lower row is the plane collection of left and right hands. Although the area of ​​the rolling fingerprint usually covers the flat fingerprint, the image of the flat fingerprint is clearer and less deformed. Collecting ten planar fingerprints has another use, which can help detect rolling fingerprint misalignment. Because in the actual collection process, it is inevitable that there will be misalignment. And only from a single rolling fingerprint, it is difficult to judge the left and right hands and the specific finger position. Once a misalignment occurs, it may lead to misidentification in the future.

Fingerprints are collected on fingerprint cards using the ink method.

For serious cases, the police will even comprehensively collect the skin texture of the hands, including the sides of the joints of the fingers, the palm, and the side of the palm. Comprehensive collection of skin texture can improve the probability of recognition, but the collection process is time-consuming.

For serious cases, the police will comprehensively collect the skin texture of the hands, including the sides of the joints of the fingers, the palm, and the side of the palm.

Although living fingerprint collection technology has long been popularized, the police fingerprint database has accumulated ink fingerprints (scanned into electronic versions) for many years, and the police fingerprint recognition system still needs to be compatible with ink fingerprints.

2.2 On-site fingerprint extraction technology

A special kind of fingerprint image, called latent or scene fingerprint, is very valuable in identifying criminal suspects. When a finger touches an object (such as glass, paper), the moisture and oil on the skin are transferred from the finger to the object and leave marks on the object. On-site fingerprints are usually invisible to the naked eye and require some chemical and physical means to develop. Common methods include brushing, ninhydrin, iodine fumigation, and silver nitrate (Lee and Gaensslen, 2012). Compared with fingerprint images obtained by other acquisition techniques, the quality of on-site fingerprint images is often poor, manifested by strong background noise, unclear ridges, small effective area of ​​fingerprints, and large skin deformation.

High, medium and low quality live fingerprints (from NIST SD27)

The figure below shows the process of extracting live fingerprints by powder brushing method. First, the fingerprint is revealed by brushing powder; then a ruler is placed next to the fingerprint, and both are photographed at the same time, the ruler will be used to adjust the fingerprint image to a standard resolution (for example, 500 ppi); finally, the fingerprint is extracted with adhesive tape, as Storage of physical evidence.

The process of extracting on-site fingerprints by brushing powder method

3. Living body collection

3.1 Optical sensor

3.1.1 Frustrated Total Internal Reflection (FTIR)

FTIR (Hase & Shimisu, 1984) is the oldest live fingerprint sensing technology and is still widely used in police and government applications. The figure below is the schematic diagram of FTIR. When a finger touches the top of the prism, the ridges make contact with the prism surface, while the valleys do not. Light entering the prism from the bottom left is totally reflected at the valleys and scattered at the ridges. Light exits the right side of the prism and is focused through the lens onto the image sensor. In the formed fingerprint image, the ridges appear very dark, while the valleys appear very bright, black and white, easy to identify.

Schematic of FTIR

The original FTIR technique had one obvious drawback: it was too bulky. Such a large fingerprint collector is used in police stations and entry-exit gates, so the size is not a problem. For many personal applications, fingerprint collectors need to be embedded in various devices to provide authentication functions, so such a large collector is obviously not suitable.

There are two reasons for the large size of the original FTIR collector: the long optical path and the large prism. One way to reduce the volume is to use multiple mirrors bouncing back and forth to maintain the optical path length while reducing the overall volume. In order to shrink the prism, Zhou et al. (1998) invented the sheet prism (combination of a group of small prisms), which reduced the cost while reducing the size.

Schematic of a sheet prism (Xia and O'Gorman, 2003)

3.1.2 Optical imaging without prisms

The package size of the harvester can be significantly reduced by completely removing the prisms and lenses and closely fitting the photodetector array to the inside of the collection surface. To this end, two problems need to be solved: (1) The photons reflected by the finger need to be guided to the photodetector to avoid crosstalk between adjacent pixels; (2) Since the magnification effect of the lens cannot be used, the photodetector array must As large as the entire acquisition area, this would result in high cost if using CMOS imaging.

The first problem can be solved by using fiber optic layers or light collimators (see figure below).

Fiber-based sensors. The residual light emitted by the finger is transmitted through the micro-optical guide plate to the photodetector array forming the CMOS or TFT backplane.

Another approach is to illuminate the finger by using a cone of light with an acute angle of incidence (Bae et al., 2018).

Lensless optical sensor based on TFT technology. In the scheme proposed by Bae et al. (2018), the LED light source is placed under the TFT glass substrate and partially covered by the pixel array. A cone of light with the proper angle of incidence reaches the surface of the coverslip and is reflected back to the photodetector with no crosstalk between pixels.

The second problem can be solved by replacing CMOS with TFT sensors. The TFT process is a mature and inexpensive technology for making large LCD panels, which creates transistors by depositing thin films of amorphous silicon on a glass substrate. The transistors cover only a small portion of each pixel's area, with the rest of the film etched away to allow light to pass through. To develop a TFT optical sensor, each pixel of the array consists of a photodiode and a readout transistor. Photodiodes can be constructed from amorphous silicon (Bae et al., 2018) or printed organic materials (Tordera et al., 2019). TFT technology allows the design of large-area panels (Liao et al., 2015) and high-resolution devices (Huang et al., 2015). Transparent materials and flexible substrates can be used to embed TFT optical sensors into the displays of portable devices, or to wrap them around curved surfaces.

3.1.3 Electroluminescence

Electro-luminescent panels contain a polymer that, when polarized with an appropriate voltage, emits light depending on the potential applied to one side. Because the ridges touch the polymer and the valleys don't, when a finger is placed on it, the potential is not the same across the surface, and the amount of light emitted varies, thus creating the luminescent pattern of a fingerprint (see Integrated Biometrics' LES Technology, 2019). The emission pattern can be converted to a digital image using a conventional CMOS camera or an array of TFT photodiodes.

Photoelectric Fingerprint Sensor

3.1.4 Non-contact optical acquisition

In the case of good lighting conditions, accurate focus, and appropriate distance, images taken directly by a high-resolution camera or a mobile phone camera on a fingertip can also be used for fingerprint recognition. The advantages of non-contact acquisition over contact acquisition include no elastic deformation of the skin, more hygiene, and no residual fingerprints on the sensor surface. However, the contrast of non-contact fingerprint images is low, and a recent technical evaluation N2N by NIST in the United States shows that the recognition accuracy of non-contact fingerprints is worse than that of contact (Libert et al., 2019). You should know that all the NIST tests are dedicated non-contact fingerprint collectors, and their performance is not as good as that of contact fingerprint collectors. In 2019, some media hyped that taking photos of scissorhands would reveal fingerprints, which is an anxiety about selling. Despite the poor performance of non-contact fingerprint recognition, this direction has attracted more attention from academia and industry since the COVID-19 pandemic. In scenarios where many users share the same collection device (for example, immigration identity verification, enterprise attendance), contactless collection is very attractive.

In addition to improving the recognition accuracy of the non-contact fingerprint itself, improving the cross-matching accuracy with the contact fingerprint is also very important for the promotion and application of the non-contact fingerprint. In practical applications, it is common to use different fingerprint collectors for the enrollment and identification phases. For example, there are many manufacturers of fingerprint collectors in the field of ID card fingerprint recognition. It is very important to ensure that the fingerprint recognition algorithm is compatible with different collectors. As an emerging technology, non-contact acquisition is compatible with mature contact acquisition, which is a problem that must be solved. The difficulty of this problem lies in the huge difference in images of different acquisition modalities (see the comparison in the figure below), such as perspective distortion, low contrast of ridge lines, focusing problems unique to non-contact acquisition, elastic deformation of skin, dry and wet skin unique to contact acquisition .

Cross-matching of contactless and contact fingerprints requires resolution of modality differences (Grosz et al., 2022)

3.1.5 3D Fingerprint Collection

The finger itself is a three-dimensional object, and the three-dimensional fingerprint is the most original form of the fingerprint (see the figure below). The advantages of 3D fingerprints over 2D fingerprints include: (1) Avoid skin deformation; (2) Complete fingerprints can be collected at one time without rolling fingers; (3) 3D information has additional discrimination.

3D fingerprint obtained by structured light technology

Researchers have proposed a variety of 3D fingerprint acquisition techniques (see the figure below), including contour method (Parziale et al., 2006), multi-view (Liu and Zhang, 2014), shadow method (Kumar and Kwong, 2015; Kumar, 2018), Focusing method (Abramovich et al., 2013), structured light (Wang et al., 2010), lidar (Galbally et al., 2017), etc. However, due to the bulky size, high cost, and lack of obvious advantages in recognition performance, these three-dimensional fingerprint collection technologies have not yet achieved large-scale applications.

Several 3D Fingerprint Acquisition Technologies

Cui et al. (2023) proposed a technique for reconstructing 3D fingerprints from a non-contact fingerprint image, which only requires an ordinary camera to collect 3D fingerprints, significantly reducing hardware costs. Different from the previous 3D acquisition scheme that relied too much on hardware, this scheme uses machine learning technology to learn the 3D shape prior of fingers and the 3D information contained in 2D non-contact images from a large number of samples. Experiments show that the 3D fingerprint reconstructed by this technology is very close to the reconstruction result of the bulky and expensive structured light 3D imaging equipment.

Cui et al. (2023) proposed to reconstruct a 3D fingerprint from a non-contact fingerprint image

The identification of 3D fingerprints can be completed by a dedicated identification algorithm, but a more realistic solution is to obtain a 2D image similar to rolling fingerprints through 3D fingerprint expansion technology (Chen et al., 2006), and then use a mature 2D fingerprint identification algorithm. This not only leverages the traditional fingerprint recognition algorithm that has been optimized for many years, but also solves the compatibility problem with mainstream fingerprint collection technologies and existing fingerprint databases.

3.1.6 Optical Coherence Tomography (OCT)

Optical coherence tomography uses low-coherence light to capture deep images of biological tissue and is currently used primarily in the medical field (seeing skin, eyes, blood vessels, etc.). In recent years, researchers have begun to use OCT equipment to obtain fingerprint tomographic images (see the figure below).

OCT fingerprint volume data (Ding et al., 2021). For some reason, looking at the OCT fingerprint images, I feel the pain of my finger skin being torn off.

The main advantage of OCT imaging is that the dermis can be detected. Since the fingerprint patterns of the dermis and epidermis are originally consistent, when the skin on the surface of the finger is damaged, the dermis pattern can provide more reliable information (see the comparison in the figure below). In addition, features such as sweat glands under the skin can be used to identify pseudo-fingerprints. However, the current OCT equipment is still bulky, slow in imaging speed, expensive, and still far from commercial use.

Epidermis fingerprints (top) and dermis fingerprints (bottom) extracted from OCT volumetric data by Ding et al. (2021)

3.2 Capacitive sensor

Although various capacitive fingerprint sensor schemes have been proposed by researchers since the 1980s (Edwards, 1984; Tsikos, 1982), they were not commercially viable until the mid-1990s (Xia and O'Gorman, 2003). A capacitive sensor is an array of tiny capacitors embedded in the panel (see picture below), and the other side of each microcapacitor is the finger skin itself. When a finger is pressed against the chip, an electric charge is formed between the finger and each capacitor. The magnitude of these charges depends on the distance between the fingerprint and the capacitance (Tartagni & Guerrieri, 1998). Therefore, fingerprint ridges and valleys cause different capacitance values ​​on the board. To measure capacitance accurately, researchers have developed various methods to obtain enough sensitivity to distinguish ridges from valleys.

Capacitive Fingerprint Sensor Schematic

In the first generation of capacitive fingerprint sensors, microcapacitor arrays were embedded in silicon wafers fabricated in CMOS technology (Inglis et al., 1998; Lee et al., 1999; Morimura et al., 2000). According to such a scheme, in order to control the cost, the area of ​​the fingerprint sensor is generally small. In 2013, the Apple iPhone 5S mobile phone integrated a capacitive fingerprint sensor on the home button, leading the trend of mobile phone fingerprint recognition. Since then, other mobile phones have integrated capacitive sensors, and fingerprint unlocking has become a standard feature of many smartphones.

Since the capacitive fingerprint sensor area of ​​most mobile phones is very small, in order to improve the user experience (increase the correct acceptance rate while the false acceptance rate remains unchanged), the fingerprint recognition system relies on the template update technology to increase the correct acceptance rate. Due to the high frequency of users unlocking mobile phones, template updates are also conditional. In other fingerprint recognition applications (such as immigration, ID card), the frequency of users using fingerprint recognition is very low, the significance of template update is small, and there are not many opportunities. Frequent recognition is unique to mobile phone fingerprint recognition. Template updating is not unique to fingerprints, and other biometrics also use updating techniques to improve performance (Rattani et al., 2009; Pisani et al., 2019).

There was once a bug in the template update technique. Someone discovered that after his mobile phone's fingerprint sensor was cracked, other people's fingerprints could also unlock his mobile phone. Some technicians took it a step further and found that orange peels can also crack the fingerprint recognition of mobile phones . The reason is that if the surface of the sensor is not clean (there is a background texture caused by sensor cracking, fingerprint stickers, etc.), when the background texture + personal fingerprint is verified, it will be registered as a template. If the characteristics of the background texture account for a large proportion, and then other fingers, orange peels, or anything press the sensor, the sensor reads the pattern of anything + the background texture, and the verification is likely to pass. This bug was basically resolved later. There is a special direction poisoning attack (Biggio et al., 2013) to study the use of template update problems to crack the recognition system.

A newer generation of capacitive sensors is based on TFT technology with sensor arrays embedded on glass substrates (Hashido et al., 2003; Hwang et al., 2017; Jeon et al., 2019; Seo et al., 2018; Young et al., 1997). The TFT process has a cost advantage in fabricating large-area sensors and can be combined with displays (Jeon et al., 2016). TFT capacitive sensors are simpler to design than optical TFTs, which need to provide illumination and guide reflected photon beams to focus on pixels. However, the disadvantage of capacitive technology is that it is difficult to obtain high-quality fingerprint images through thicker glass.

In order to reduce the cost of the CMOS capacitive sensor, the researchers also invented a very narrow swipe fingerprint sensor. When the user slides the finger vertically across the sensor, the sensor obtains a series of small images, and the built-in fingerprint stitching algorithm reconstructs a complete fingerprint. This kind of sensor has been widely used in notebook computers.

The small-area sliding fingerprint sensor collects large-area fingerprints through image stitching algorithms

3.3 Ultrasonic sensors

Ultrasonic sensing is based on sending an acoustic signal directed at the fingertip and capturing the echo signal (see figure below), from which the ridge structure of the fingerprint is calculated. The two main components of an ultrasonic sensor are the transmitter and receiver. The former generates sound pulses, and the latter detects the response of these pulses as they bounce off the fingerprint (Schneider and Wobschall, 1991). The approach is to image the deeper layers of the skin of the fingers (even through thin gloves), so dirty, wet fingers are better imaged.

Ultrasonic Fingerprint Sensor Schematic

John K. Schneider studied ultrasonic fingerprint imaging during his Ph.D. and founded Ultra-Scan after graduation, aiming to make ultrasonic fingerprint collectors into products. Early ultrasound collectors were bulky, had mechanical parts, and were quite expensive (hundreds of dollars). Additionally, the acquisition process is slow due to the mechanical scanning required. Ultra-Scan mainly positions its products for police and military use. In 2013, Ultra-Scan was acquired by mobile chip giant Qualcomm. Subsequently, Qualcomm developed an ultrasonic fingerprint sensor for mobile phones, and also introduced a large-area ultrasonic fingerprint sensor (20*30 square millimeters in area) that can collect two fingers. Compared with the optical fingerprint sensor under the screen of the mobile phone, the ultrasonic solution does not need to be illuminated (people who swipe their mobile phones at night should be familiar with the strong light of the optical fingerprint sensor under the screen). share.

Qualcomm's large-area ultrasonic fingerprint sensor

3.4 Thermal sensor

Thermal sensors are made of pyroelectric materials that generate current differences depending on temperature (Mainguet et al., 1999; Han and Koshimoto, 2008; Miki and Tsuchitani, 2017). The fingerprint ridges are in contact with the sensor surface, while the valleys are some distance away from the sensor surface, resulting in different temperature differences. The sensor is kept at a high temperature by electrical heating to increase the temperature difference between the surface of the sensor and the finger. When the finger touches, the temperature difference creates an image, but this image quickly disappears. The reason is that thermal equilibrium is reached quickly, and the pixel temperature stabilizes. There are two ways to overcome this problem:

• Fingerprint images are acquired by swiping the finger (as described in Section 2.4), so that the contact points of the ridges are constantly changing.
• Active thermal sensing technology delivers pulses of heat that are imperceptible to the user through an array of detectors (see Next Biometrics' LTPS technology ).

Thermal sensing technology is less sensitive to static electricity than capacitive technology and can accept thicker protective coatings (since heat is easily transmitted through the coating). However, thermal sensors are typically more power-hungry and have slower acquisition speeds.

3.5 Pressure sensor

A pressure sensor generates an electrical signal when a force is applied. Early pressure sensors were based on piezoelectric materials. The sensor surface is made of a non-conductive dielectric material that generates an electrical charge when pressed by a finger (an effect called piezoelectricity). The amount of current generated depends on the pressure the finger exerts on the sensor surface. Since the ridges and valleys are at different distances from the sensor surface, they result in different amounts of current flow. Unfortunately, these materials are not sensitive enough to detect subtle differences in ridges and valleys; moreover, the protective coating also blurs the resulting image. Researchers have proposed many other designs (see Mainguet's web page ) to increase sensitivity and reduce cost. At present, the market share of pressure-based fingerprint sensors is relatively low.

4. Fingerprint image quality

Faced with a wide variety of fingerprint sensors, how to evaluate and choose? For a certain fingerprint sensor, how to optimize its parameters? The most important criterion is the image quality of the sensor. The main parameters of the fingerprint image will be briefly introduced below; then the subjective and objective reasons for the low quality of the fingerprint image will be analyzed; then how to automatically evaluate the quality of the fingerprint image will be discussed; finally, the method to improve the quality of the fingerprint image will be introduced.

4.1 Fingerprint Image Parameters

The main parameters of the fingerprint image include: resolution, area, number of pixels, geometric accuracy, gray scale range, contrast, signal-to-noise ratio, etc. The figure below shows the images obtained by different fingerprint collectors from the same finger.

Images obtained from the same finger by different fingerprint collectors

4.2 Low Quality Fingerprint Image

Many subjective and objective factors may lead to poor fingerprint image quality, which in turn leads to low recognition rate. Poor fingerprint quality is manifested in four aspects:

• Low signal-to-noise ratio: Many factors can cause low signal-to-noise ratio, for example, finger skin that is too dry or wet, finger pressure that is too light or Common background patterns of fingerprints, dirt on sensor acquisition surfaces, sensor noise.
• Small fingerprint area: the sensor is small, and the contact surface between the finger and the sensor is small.
• Large skin deformation: After pressing with fingers, a large shearing force and torsion force are applied.
• Improper posture or position: The angle of finger pressing is too biased (the roll angle and pitch angle are too large), and the pressing part is not the pulp (such as knuckles).

Various types of low-quality fingerprint images

4.3 Fingerprint Image Quality Evaluation

There are many dimensions to evaluate the quality of fingerprints, and it is very difficult to use a single number to comprehensively and objectively measure the quality of fingerprint images. However, it is useful to give a quality score to fingerprint images. For example, it is convenient for quality control in fingerprint collection, and a suitable fingerprint algorithm is called according to the quality of a specific image.

Grother and Tabassi (2007) argued that the quality of a fingerprint should be a quantitative predictor of its matching performance. If the library fingerprint of an input fingerprint can be ranked first in a very large database, it can be considered that the quality of the input fingerprint is very high. In order to estimate the quality of the input fingerprint only, it is necessary to train a model in the offline stage to extract features from a single image and map them into quality scores. The most influential technique in this regard is NFIQ (Tabassi et al., 2004, 2021).

4.4 Improve fingerprint image quality

The easiest and most effective way to improve the quality of a fingerprint image is to have the user take it again. If the fingerprint collection system can give intelligent prompts to guide the user to adjust the degree of dryness and wetness of the finger, and control the angle and strength of pressing the fingerprint, the quality of the fingerprint collected again by the user will often improve. In addition, methods to improve the quality of fingerprint images from the algorithm level include: using fingerprint enhancement to improve the signal-to-noise ratio (Hong et al., 1998), increasing the fingerprint area through image stitching (Cui et al., 2021), using deformation self-correction to minimize Deformation of fingerprint skin due to shear force, torsion force (Si et al., 2015).

4.4.1 Fingerprint Enhancement

Many fingerprint sensors use basic image enhancement algorithms (such as linear stretch contrast) to improve the image quality, and the output of the sensor is the enhanced image. The enhancement effect of the basic image enhancement algorithm is very limited, and it is difficult to remove strong noise. Image enhancement algorithms that make full use of fingerprint characteristics (such as Gabor filtering) have strong noise suppression capabilities, but they will also change and destroy tertiary features (such as sweat pores, immature ridges). A strong enhancement algorithm can be used in the fingerprint recognition algorithm, but cannot replace the original image of the sensor as output.

The Gabor filter fingerprint enhancement method proposed by Hong et al. (1998) can significantly improve the signal-to-noise ratio, but it will also destroy tertiary features such as sweat pores. The left is the original image, and the right is the enhanced image.

4.4.2 Fingerprint stitching

Fingerprint stitching technology can fuse multiple fingerprint images from different angles and different parts into a fingerprint with a larger area and clearer ridges, improving the effective area and signal-to-noise ratio of the fingerprint image. The main challenges of fingerprint stitching are skin elastic deformation, too small overlap and noise. The traditional fingerprint stitching method first finds the matching minutiae between two fingerprints, and then fits the deformation field between the fingerprints. However, when the deformation is large or the matching details are wrong, the fitted deformation field is inaccurate and cannot be aligned with the ridges of the two fingerprints, which will lead to discontinuity in the splicing and bring side effects.

The fingerprint dense registration technique proposed by Cui et al. (2021) can calculate the pixel-level deformation field between two fingerprint images. The author uses the dense registration technique to stitch the plane fingerprints from various angles into a complete fingerprint . Compared with the traditional rolling fingerprint, the full fingerprint has a larger area and more details, especially the fingertip area that the rolling fingerprint usually lacks, and the full fingerprint can also cover it. Many crime scene fingerprints come from fingertips. If the police collect such complete fingerprints, it will be very helpful for improving the identification rate of fingerprints at crime scenes.

Dense registration techniques stitch a series of planar fingerprints into a full fingerprint (Cui et al., 2021).

The area of ​​the fingerprint sensor of the mobile phone is very small. In order to improve the user experience and ensure that the user can pass through from different angles, the user needs to collect fingerprints many times as a template during the registration phase. Cui et al. (2021) used dense registration technology to stitch many small fingerprint images into a larger fingerprint template, so that a higher recognition rate can be achieved with only one fingerprint.

Dense registration techniques stitch a series of small fingerprint images into larger area fingerprints (Cui et al., 2021).

4.4.3 Fingerprint distortion self-correction

For distorted fingerprints, the traditional approach is to modify the fingerprint matching algorithm to tolerate the distortion. However, the disadvantage of this method is that different fingerprints will become more similar, which increases the false recognition rate; in addition, due to the large distortion tolerated, the matching speed will also slow down.

In addition to the matching process, the distortion can also be processed in the fingerprint collection process. Compared with the matching link, the advantage of processing distortion in the acquisition link is that it does not affect the efficiency of large database identification. But in the absence of a reference fingerprint, it is very difficult to estimate its deformation field only from a fingerprint image. The distortion self-correction technology proposed by Si et al. (2015) estimates the distortion caused by shear force and torsion from the input fingerprint image, and removes the distortion to obtain a normal fingerprint. For the corrected fingerprint, there is no difficulty in identifying the existing fingerprint identification technology.

Distortion self-correction technology removes the distortion caused by shear force and torsion in the input fingerprint to obtain a normal fingerprint (Si et al., 2015).

5. Summary

The popularization of fingerprint identification technology in many fields is inseparable from the innovation and progress of fingerprint sensing technology. Compared with other biometric identification technologies, there are many types and quantities of fingerprint sensors. In 2006, some technology media once selected AuthenTec’s cumulative shipment of 10 million fingerprint sensors as one of the top ten news in the field of biometric identification. In the mobile Internet era, the shipments of fingerprint sensors have doubled several times, and the market value of major fingerprint sensor manufacturers once exceeded 100 billion yuan . A chip factory executive once said that fingerprint sensors used to consume more silicon than CPUs.

It is difficult to predict the development of fingerprint sensors from a technical point of view, but it may be more reliable to predict from the point of view of user needs. Future fingerprint sensing technology should evolve towards thinner, softer, more durable, higher signal-to-noise ratio, faster, more energy-efficient, smarter, and cheaper.

references

1. Abramovich G, Harding K G, Hu Q, et al. (2013). Method and system for contactless fingerprint detection and verification: U.S. Patent 8,406,487.
2. Bae, S., Ling, Y., Lin, W., & Zhu, H. (2018). Optical fingerprint sensor based on a-Si:H TFT technology. Proceedings of SID Symposium Digest of Technical Papers, 49(1), 1017–1020.
3. Biggio, B., Didaci, L., Fumera, G., & Roli, F. (2013). Poisoning attacks to compromise face templates. In 2013 international conference on biometrics (ICB) (pp. 1-7).
4. Calmel, M. (2000). Fingerprint sensor device, US Patent 6128399.
5. Chen, Y., Parziale, G., Diaz-Santana, E., & Jain, A. K. (2006). 3D touchless fingerprints: Compatibility with legacy rolled images. In Proceddings of Biometric Symposium.
6. Cui, Z., Feng, J., & Zhou, J. (2021). Dense registration and mosaicking of fingerprints by training an end-to-end network. IEEE Transactions on Information Forensics and Security, 16, 627-642.
7. Cui, Z., Feng, J., & Zhou, J. (2023). Monocular 3D Fingerprint Reconstruction and Unwarping. IEEE transactions on pattern analysis and machine intelligence.
8. Ding, B., Wang, H., Chen, P., Zhang, Y., Guo, Z., Feng, J., & Liang, R. (2021). Surface and internal fingerprint reconstruction from optical coherence tomography through convolutional neural network. IEEE Transactions on Information Forensics and Security, 16, 685-700.
9. Fujieda, I., Ono, Y., & Sugama, S. (1995). Fingerprint image input device having an image sensor with openings, US Patent 5446290.
10. Galbally, J., Bostrom, G., & Beslay, L. (2017). Full 3D touchless fingerprint recognition: Sensor, database and baseline performance. In Proceedings of International Joint Conference on Biometrics (IJCB) (pp. 225–233).
11. Grosz, S. A., Engelsma, J. J., Liu, E., & Jain, A. K. (2022). C2cl: Contact to contactless fingerprint matching. IEEE Transactions on Information Forensics and Security, 17, 196-210.
12. Grother, P., & Tabassi, E. (2007). Performance of biometric quality measures. IEEE transactions on pattern analysis and machine intelligence, 29(4), 531-543.
13. Han, H., & Koshimoto, Y. (2008). Characteristics of thermal-type fingerprint sensor. In Proceedings of SPIE Conference on Biometric Technology for Human Identification V.
14. Hase, M., & Shimisu, A. (1984). Entry method of fingerprint image using a prism. Transactions of the Institute of Electronic and Communication Engineers of Japan, J67–D, 627–628.
15. Hashido, R., Suzuki, A., Iwata, A., Okamoto, T., Satoh, Y., & Inoue, M. (2003). A capacitive fingerprint sensor chip using low-temperature poly-Si TFTs on a glass substrate and a novel and unique sensing method. IEEE Journal of Solid-State Circuits, 38(2), 274–280.
16. Huang, S., Huang, Y., Yeh, C., Sugiura, N., You, J., & Peng, C. (2015). Design and modeling of 1000ppi fingerprint sensor. In Proceedings of IEEE Sensors Conference.
17. Hwang, H., Lee, H., Jang, B., Kim, H., Lee, T., & Chae, Y. (2017). A 500-dpi transparent on-glass capacitive fingerprint sensor. In Proceedings of SID Symposium Digest of Technical Papers.
18. Inglis, C., Manchanda, L., Comizzoll, R., Dickinson, A.,Martin, E., Mandis, S., Silveman, P.,Weber, G., Ackland, B., & O’Gorman, L. (1998). A robust, 1.8 V, 250 mW, direct contact 500 dpi fingerprint sensor. In Proceedings of IEEE Solid-State Circuits Conference.
19. Integrated Biometrics. (2019). LES film technology. https://integratedbiometrics.com/wp-content/uploads/2020/03/LES-Film-Technology-Whitepaper.pdf.
20. Jain, A. K., Nandakumar, K., & Ross, A. (2016). 50 years of biometric research: Accomplishments, challenges, and opportunities. Pattern recognition letters, 79, 80-105.
21. Jean-Francois Mainguet's website. https://biometrics.mainguet.org/types/fingerprint/fingerprint.htm
22. Jeon, Y. E., Lee, Y. J., Jang, M. K., Seo, B. M., Kang, I. H., Hong, M. T., Lee, J. M., Jacques, E., Mohammed-Brahim, T., & Bae, B. S. (2016). Capacitive sensor array for fingerprint recognition. In Proceedings of International Conference on Sensing Technology (ICST) (pp. 1–4).
23. Jeon, G., Lee, S., Lee, S. H., Shim, J., Ra, J., Park, K.W., Yeom, H., Nam, Y., Kwon, O.,&Park, S. K. (2019).Highly sensitive active-matrix driven self-capacitive fingerprint sensor based on oxide thin film transistor. Scientific Reports, 9(1), 3216.
24. Kumar, A. (2018). Contactless 3D fingerprint identification. Springer.
25. Kumar, A., & Kwong, C. (2015). Towards contactless, low-cost and accurate 3D fingerprint identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3), 681–696.
26. Lee, H. C., & Gaensslen, R. E. (2012). Advances in fingerprint technology (3rd ed.). CRC Press.
27. Lee, J. W., Min, D. J., Kim, J., & Kim, W. (1999). A 600 dpi capacitive fingerprint sensor chip and image synthesis technique. IEEE Journal of Solid-State Circuits, 34(4), 469–475.
28. Liao, Y., Chang, C., Lin, C., You, J., Hsieh, H., Chen, J., Cho, A., Liu, Y., Lai, Y., Tseng, J., Chiang, M., & Lin, Y. (2015). Flat panel fingerprint optical sensor using TFT technology. In Proceedings of IEEE Sensors Conference.
29. Libert, J., Grantham, J., Bandini, B., Ko, K., Orandi, S., & Watson, C. (2019). Interoperability assessment 2019: Contactless-to-contact fingerprint capture. NIST-IR 8307.
30. Liu, F., & Zhang, D. (2014). 3D fingerprint reconstruction system using feature correspondences and prior estimated finger model. Pattern Recognition, 47(1), 178-193.
31. Mainguet, J. G., Pegulu, M., & Harris, J. B. (1999). Fingerchip: Thermal imaging and finger sweeping in a silicon fingerprint sensor. In Proceedings of Workshop on Automatic Identification Advances Technologies (pp. 91–94).
32. Maltoni, D., Maio, D., Jain, A. K., & Feng, J. (2022). Fingerprint sensing. Handbook of Fingerprint Recognition, 3rd Edition. 63-114, Springer.
33. Miki, H., & Tsuchitani, S. (2017). Structural design points in arrayed micro thermal sensors (III) ~ polymer-based approach. International Journal of Engineering and Technical Research, 7(3), 24–32.
34. Morimura, H., Shigematsu, S., & Machida, K. (2000). A novel sensor cell architecture and sensing circuit scheme for capacitive fingerprint sensors. IEEE Journal of Solid-State Circuits, 37(10), 1300–1306.
35. Parziale, G., Diaz-Santana, E., & Hauke, R. (2006). The surround ImagerTM: A multi-camera touchless device to acquire 3D rolled-equivalent fingerprints. In Proceedings of International Conference on Biometrics. pp. 244–250.
36. Pisani , PH , Mhenni , A. , Giot , R. , Cherrier , E. , Poh , N. , ... & Amara , NEB (2019). Adaptive biometric systems: A review and perspectives. ACM Computing Surveys (CSUR), 52(5), 1-38.
37. Rattani, A., Freni, B., Marcialis, G. L., & Roli, F. (2009). Template update methods in adaptive biometric systems: A critical review. In International Conference on Biometrics (pp. 847-856). Springer, Berlin, Heidelberg.
38. Schneider, J., & Wobschall, D. (1991). Live scan fingerprint imagery using high resolution C–SCAN ultrasonography. In Proceedings of International Carnahan Conference on Security Technology (25th ed., pp. 88–95).
39. Seo, W., Pi, J., Cho, S. H., Kang, S., Ahn, S., Hwang, C., Jeon, H., Kim, J. & Lee, M. (2018). Transparent fingerprint sensor system for large flat panel display. Sensors, 18(1).
40. Si, X., Feng, J., Zhou, J., & Luo, Y. (2015). Detection and rectification of distorted fingerprints. IEEE transactions on pattern analysis and machine intelligence, 37(3), 555-568.
41. Tabassi, E., Wilson, C., & Watson, C. (2004). Fingerprint Image Quality. NIST-IR 7151.
42. Tabassi, E., Olsen, M., …, & Schwaiger, M. (2021). NFIQ 2.0—NIST Fingerprint Image Quality. NIST-IR 8382.
43. Tartagni, M., & Guerrieri, R. (1998). A fingerprint sensor based on the feedback capacitive sensing scheme. IEEE Journal of Solid-State Circuits, 33(1), 133–142.
44. Tordera, D., Peeters, B., Akkerman, H. B., van Breemen, A. J. J. M., Maas, J., Shanmugam, S., Kronemeijer, A. J., & Gelinck, G. H. (2019). A high resolution thin film fingerprint sensor using a printed organic photodetector. Advanced Material Technologies, 4(11).
45. Xia, X., & O’Gorman, L. (2003). Innovations in fingerprint capture devices. Pattern Recognition, 36(2), 361-369.
46. Young, N. D., Harkin, G., Bunn, R. M., McCulloch, D. J., Wilks, R. W., & Knapp, A. G. (1997). Novel fingerprint scanning arrays using polysilicon TFT’s on glass and polymer substrates. IEEE Electron Device Letters, 18(1), 19–20.
47. Wang, Y., Hassebrook, L. G., & Lau, D. L. (2010). Data acquisition and processing of 3-D fingerprints. IEEE Transactions on Information Forensics and Security, 5(4), 750–760.
48. Zhou G, Qiao Y, & Mok F. (1998). Fingerprint sensing system using a sheet prism: U.S. Patent 5,796,858.