A HIERARCHICAL FINGERPRINT MATCHING SYSTEM

Transcription

1 A HIERARCHICAL FINGERPRINT MATCHING SYSTEM A thesis submitted in partial fulfillment of the requirements for the degree of Bachelor-Master of Technology (Dual) by ABHISHEK RAWAT to the Department of Computer Science and Engineering Indian Institute of Technology Kanpur July 2009

2

3 Abstract Fingerprint Recognition is a widely popular but a complex pattern recognition problem. It is difficult to design accurate algorithms capable of extracting salient features and matching them in a robust way. The real challenge is matching fingerprints affected by: i) High displacement/or rotation which results in smaller overlap between template and query fingerprints, ii) Non-linear distortion caused by the finger plasticity, iii) Different pressure and skin condition and iv) Feature extraction errors which may result in spurious or missing features. The information contained in a fingerprint can be categorized into three different levels, namely, Level 1 (pattern), Level 2 (minutia points), and Level 3 (pores and ridge contours). Despite their discriminative power, the Level 3 features are barely used by the vast majority of contemporary automated fingerprint authentication systems (AFAS) which rely mostly on minutiae features. This is mainly because, most of these authentication systems are equipped with 500 ppi (FBI s standard of fingerprint resolution for AFAS) scanners, and reliably extracting fine and detailed Level 3 features require high resolution images. While this may have been the case with many older live-scan devices, the current devices are capable of detecting a reasonable amount of level 3 details even at the relatively limited 500 ppi resolution. In this thesis the above mentioned problems have been addressed and a new hierarchical matcher has been proposed. The hierarchical matcher utilizes Level 3 features (pores and ridge contour) in conjunction with Level 2 features (minutiae) for matching. The aim is to reduce the error rates, namely FAR (False Acceptance Rate) and FRR (False Rejection Rate) in the existing minutiae based systems. The hierarchical matcher has been tested on three diverse databases in public domain. The obtained results are promising and verify our claim.

4 2 Acknowledgments I would like to express my deep-felt gratitude to my advisor, Dr. Phalguni Gupta for giving me an opportunity to work with the Biometrics Group and for his advice, encouragement, constant support. I wish to thank him for extending me the greatest freedom in deciding the direction and scope of my research. It has been both a privilege and a rewarding experience working with him. I would also like to thank all my friends and colleagues here at IIT Kanpur for all the wonderful times I have had with them. Their valuable comments and suggestions have been vital to the completion of this work. I want to thank the faculty of Computer Science Department and the staff for providing me the means to complete my degree and prepare for a career as a computer scientist. And finally, I am grateful to my parents for their love, sacrifice, understanding, encouragement and support. Abhishek Rawat

5 i Contents List of Figures List of Tables iii v 1 INTRODUCTION Fingerprint as a Biometric History of Fingerprint Recognition Fingerprint Representation Motivation and Problem Definition Approach Thesis Outline BACKGROUND AND LITERATURE REVIEW Data Acquisition Image Preprocessing Fingerprint Image Enhancement Feature Extraction Minutiae Extraction Pores Extraction Ridge Contour Extraction Fingerprint Matching Correlation-based Techniques Minutiae-based Methods Global Matching Local Matching Ridge Feature-based Matching Techniques Texture Feature based Techniques Level 3 Features-based Techniques

6 ii 3 PROPOSED HIERARCHICAL MATCHER Preprocessing and Enhancement Feature Extraction Hierarchical Matching Minutia-based Matcher Local Structure Matching Consolidation step Level 3 features based Matcher Pore Matching RidgeContour Matching Fusion of Level 2 and Level 3 features EXPERIMENTAL RESULTS Database Performance Evaluation Experiment Experiment Experiment Timing Analysis CONCLUSION & FUTURE WORK Conclusion Future Work Bibliography 85

7 iii List of Figures 1.1 Fingerprint features at Level 1, Level 2 and Level 3 ([7, 54]) Open and closed pores ([29]) Characteristics of ridge contours and edges ([9]) Architecture of a Fingerprint Verification System Fingerprints at different resolutions a) 380 ppi, b) 500 ppi, c)1000 ppi ([29]) A typical minutiae extraction process ([30]) An example of two false matched local structures ([36]) The proposed hierarchical matcher Effect of differential finger pressure([36]) Ridge Contour extraction process in a image scanned with Cross Match Verifier 300 scanner at 500 dpi Pores extracted from a image scanned with Cross Match Verifier 300 scanner at 500 dpi The schematic description of a star with k = 6 neighbors. The neighbors are chosen from all the four quadrants Two impressions of same fingerprint at 500 dpi Schematic of square to circular transformation The basic LBP operator Sample images from Neurotechnology Database Sample images from FVC 2004, DB3 Database Sample images from FVC 2006, DB2 Database ROC curves for proposed minutia matcher and CBFS-Kplet based matcher, on Neurotechnology Database ROC curves for proposed minutia matcher and CBFS-Kplet based matcher, on FVC 2004, DB3 Database ROC curves for proposed minutia matcher and CBFS-Kplet based matcher, on FVC 2006, DB2 Database

8 4.7 ROC curves comparing the performance of LBP and ZM (Neurotechnology Database) ROC curves comparing the performance of LBP and ZM (FVC 2004, DB3 Database) ROC curves comparing the performance of LBP and ZM (FVC 2006, DB2 Database) ROC curves for the minutia matcher and hierarchical matcher on Neurotechnology Database ROC curves for the minutia matcher and hierarchical matcher on FVC 2004, DB3 Database ROC curves for the minutia matcher and hierarchical matcher on FVC 2006, DB2 Database Distribution of number of matched minutiae, for Genuine and Impostor cases, on Neurotechnology Database Distribution of number of matched minutiae, for Genuine and Impostor cases, on FVC 2004, DB3 Database Distribution of number of matched minutiae, for Genuine and Impostor cases, on FVC 2006, DB2 Database iv

9 v List of Tables 4.1 Equal Error Rate (EER) comparison between proposed minutia matcher and CBFS-Kplet based matcher Equal Error Rate (EER) comparison between proposed minutia matcher and proposed hierarchical matcher Comparison of matching time for Level 3 Features

10 1 Chapter 1 INTRODUCTION Biometric based recognition, or biometrics, is the science of identifying, or verifying the identity of, a person based on physiological and/or behavioral characteristics [14]. Physiological traits are related to the physiology of the body and mainly include fingerprint, face, DNA, ear, iris, retina, hand and palm geometry. Behavioral traits are related to behavior of a person and examples include signature, typing rhythm, gait, voice etc. Biometric recognition offers many advantages over traditional PIN number or password and token-based (e.g., ID cards) approaches. A biometric trait cannot be easily transferred, forgotten or lost, the rightful owner of the biometric template can be easily identified, and it is difficult to duplicate a biometric trait [20]. There are a number of desirable properties for any chosen biometric characteristic [14]. These include: 1. Universality : Every person should have the characteristic. 2. Uniqueness : No two persons should be the same in terms of the biometric characteristic.

11 2 3. Permanence : The biometric characteristics should not change, or change minimally, over time. 4. Collectability : The biometric characteristic should be measurable with some (practical) sensing device. 5. Acceptability : The user population and the public in general should have no (strong) objections to the measuring/collection of the biometric trait. A biometric system is essentially a pattern recognition system that operates by acquiring biometric data from an individual, extracting a feature set from the acquired data, and comparing this feature set against the template set in the database [32]. Depending on the application context, a biometric system may operate either in verification mode or identification mode: In the verification mode, an individual provides his/her biometric data and claims an identity, usually via a PIN (Personal Identification Number), a user name, a smart card, etc. The system then verifies the individual s identity by comparing the acquired biometric data with the individual s own biometric template(s) stored in system database. Such a system basically performs a oneto-one comparison to determine whether the claimed identity is true or not. In the identification mode, the system compares the given biometric data with the templates of all the users in the database. Therefore, the system conducts a one-to-many comparison to establish an individual s identity (or fails if the subject is not enrolled in the system database) without the subject having to claim an identity.

12 1.1. FINGERPRINT AS A BIOMETRIC 3 The effectiveness of a biometric system can be judged by following characteristics [35]: 1. Performance : This refers to the achievable recognition accuracy, speed, robustness, the resource requirements to achieve the desired recognition accuracy and speed, as well as operational (work environment of individual, e.g., manual workers may have a large number of cuts and bruises on their fingerprints) or environmental factors (humidity, illumination etc.) that affect the recognition accuracy and speed. 2. Scalability : This refers to the ability to encompass large number of individuals without a significant decrease in the performance. 3. Non-invasiveness : This refers to the ease with which the information can be captured from individuals, without damaging an individual s physical integrity and ideally without special preparations by/of an individual. 4. Circumvention : This refers to the degree to which the system is resistant to spoofs or attacks. A practical biometric system should meet the specified recognition accuracy, speed, and resource requirements, be harmless to the users, be accepted by the intended population, and be sufficiently robust to various fraudulent methods and attacks to the system. 1.1 Fingerprint as a Biometric A fingerprint is an impression of the friction ridges, from the surface of a fingertip. Fingerprints have been used for personal identification for many decades, more

13 1.1. FINGERPRINT AS A BIOMETRIC 4 recently becoming automated due to advancements in computing capabilities. Fingerprint recognition is nowadays one of the most important and popular biometric technologies mainly because of the inherent ease in acquisition, the numerous sources (ten fingers) available for collection, and the established use and collections by law enforcement agencies. Automatic fingerprint identification is one of the most reliable biometric technologies. This is because of the well known fingerprint distinctiveness, persistence, ease of acquisition and high matching accuracy rates. Fingerprints are unique to each individual and they do not change over time. Even identical twins (who share their DNA) do not carry identical fingerprints. The uniqueness can be attributed to the fact that the ridge patterns and the details in small areas of friction ridges are never repeated. These friction ridges develop on the fetus in their definitive form before birth and are known to be persistent throughout life except for permanent scarring. Scientific research in areas such as biology, embryology, anatomy and histology has supported these findings [4]. Also, the matching accuracy of fingerprint based authentication systems has been shown to be very high. Fingerprint-based authentication systems continue to dominate the biometrics market by accounting for almost 52% of authentication systems based on biometric traits [35] History of Fingerprint Recognition Fingerprints have been found on ancient artifacts recovered from excavation sites of various civilizations [42]. However fingerprints have been used for identification only from nineteenth century onwards. A time-line of important events that has established the foundation of the modern fingerprint based biometric technology can be found in [2]. Henry Fauld [21] has first scientifically suggested the individuality and uniqueness of fingerprints. Sir Francis Galton has published the well-known book

14 1.1. FINGERPRINT AS A BIOMETRIC 5 entitled Fingerprints [22], in which a detailed statistical model of fingerprint analysis and identification has been discussed. Galton has introduced Level 2 features by defining minutia points as either ridge endings or ridge bifurcations on a local ridge. An important advance in fingerprint identification has been made by Edward Henry, who has established a system known as Henry system for fingerprint classification [65]. In [44], Locard has introduced the science of poroscopy, the comparison of sweat pores for the purpose of personal identification. Locard has stated that like the ridge characteristics, the pores are also permanent, immutable, and unique, and are useful for establishing the identity, especially when a sufficient number of ridges is not available. Chatterjee has proposed the use of ridge edges in combination with other friction ridge formations to establish individualization, which is referred to as edgeoscopy [9]. Over the last few years, poroscopy and edgeoscopy have received growing attention and have been widely studied by latent fingerprint examiners [9]. It has been claimed that shapes and relative positions of sweat pores and shapes of ridge edges are as permanent and unique as traditional minutia points. And when understood, they add considerable weight to the conclusion of identification [9] Fingerprint Representation The types of information that can be collected from a fingerprint s friction ridge impression can be categorized as Level 1, Level 2, or Level 3 features as shown in Figure 1.1. At the global level, the fingerprint pattern exhibits one or more regions where the ridge lines assume distinctive shapes characterized by high curvature, frequent

15 1.1. FINGERPRINT AS A BIOMETRIC 6 Figure 1.1: Fingerprint features at Level 1, Level 2 and Level 3 ([7, 54]) termination, etc. These regions are broadly classified into arch, loop, and whorl. The arch, loop and whorl can further be classified into various subcategories. Level 1 features comprises these global patterns and morphological information. They alone do not contain sufficient information to uniquely identify fingerprints but are used for broad classification of fingerprints. Level 2 features or minutiae refers to the various ways that the ridges can be discontinuous. These are essentially Galton characteristics, namely ridge endings and ridge bifurcations. A ridge ending is defined as the ridge point where a ridge ends abruptly. A bifurcation is defined as the ridge point where a ridge bifurcates into two ridges. Minutiae are the most prominent features, generally stable and robust to fingerprint impression conditions. The distribution of minutiae in a fingerprint is considered unique and most of the automated matchers use this property to uniquely

16 1.1. FINGERPRINT AS A BIOMETRIC 7 identify fingerprints. Uniqueness of fingerprint based on minutia points has been quantified by Galton [22]. Statistical analysis has shown that Level 2 features, have sufficient discriminating power to establish the individuality of fingerprints [56]. Level 3 features are the extremely fine intra ridge details present in fingerprints [6]. These are essentially the sweat pores and ridge contours. Pores are the openings of the sweat glands and they are distributed along the ridges. Studies [9] have shown that density of pores on a ridge varies from 23 to 45 pores per inch and 20 to 40 pores should be sufficient to determine the identity of an individual. A pore can be either open or closed, based on its perspiration activity. A closed pore is entirely enclosed by a ridge, while an open pore intersects with the valley lying between two ridges as shown in Figure 1.2. The pore information (position, number and shape) are considered to be permanent, immutable and highly distinctive but very few automatic matching techniques use pores since their reliable extraction requires high resolution and good quality fingerprint images. Ridge contours contain valuable Level 3 information including ridge width and edge shape. Various shapes on the friction ridge edges can be classified into eight categories, namely, straight, convex, peak, table, pocket, concave, angle, and others as shown in Figure 1.3. The shapes and relative position of ridge edges are considered as permanent and unique. Figure 1.2: Open and closed pores ([29])

17 1.2. MOTIVATION AND PROBLEM DEFINITION 8 Figure 1.3: Characteristics of ridge contours and edges ([9]) 1.2 Motivation and Problem Definition Fingerprint recognition is a complex pattern recognition problem. It is difficult to design accurate algorithms capable of extracting salient features and matching them in a robust way, especially in poor quality fingerprint images and when low-cost acquisition devices with small area are adopted. There is a popular misconception that automatic fingerprint recognition is a fully solved problem since it was one of the first applications of machine pattern recognition. On the contrary, fingerprint recognition is still a challenging and important pattern recognition problem. The real challenge is matching fingerprints affected by: i) High displacement/or rotation which results in smaller overlap between template and query fingerprints (this case can be treated as similar to matching partial fingerprints), ii) Non-linear distortion caused by the finger plasticity, iii) Different pressure and skin condition and iv) Feature extraction errors which may result in spurious or missing features. The vast majority of contemporary automated fingerprint authentication systems (AFAS) are minutiae (level 2 features) based [46]. Minutiae-based systems generally rely on finding correspondences 1 between the minutia points present in query and reference fingerprint images. These systems normally perform well with high- 1 A minutiae in the query fingerprint and a minutiae in the reference fingerprint are said to be corresponding if they represent the identical minutiae scanned from the same finger

18 1.2. MOTIVATION AND PROBLEM DEFINITION 9 quality fingerprint images and a sufficient fingerprint surface area. These conditions, however, may not always be attainable. In many cases, only a small portion of the query fingerprint can be compared with the reference fingerprint as a result the number of minutiae correspondences might significantly decrease and the matching algorithm would not be able to make a decision with high certainty. This effect is even more marked on intrinsically poor quality fingers, where only a subset of the minutiae can be extracted and used with sufficient reliability. Although minutiae may carry most of the fingerprint s discriminatory information, they do not always constitute the best trade-off between accuracy and robustness. This has led the designers of fingerprint recognition techniques to search for other fingerprint distinguishing features, beyond minutiae, which may be used in conjunction with minutiae (and not as an alternative) to increase the system accuracy and robustness. It is a known fact that the presence of Level 3 features in fingerprints provides minute detail for matching and the potential for increased accuracy. The forensic experts in law enforcement often make use of Level 3 features, such as sweat pores and ridge contours, to compare fingerprint samples when insufficient minutia points are present in the fingerprint image or poor image quality hampers minutiae analysis. That is, experts take advantage of an extended feature set in order to conduct a more effective matching. Despite their discriminating property, level 3 features are barely utilized in the commercial automated fingerprint authentication systems (AFAS), as a result a large amount of fingerprint information is ignored by such systems. This is mainly because, most of these authentication systems are equipped with 500 ppi (pixels per inch) scanners, and reliably(or consistently) extracting fine and detailed Level 3 features require high resolution images. While this may have been the case with many of the older live-scan devices, the current devices are capable of detect-

19 1.3. APPROACH 10 ing a reasonable amount of level three detail even at the relatively limited 500 ppi resolution. Ray et al. [62] have presented a means of modeling and extracting pores (which are considered as highly distinctive Level 3 features) from 500 ppi fingerprint images. This study showed that while not every fingerprint image obtained with a 500 ppi scanner has evident pores, a substantial number of them do have. Thus, it is a natural step to try to extract Level 3 information, and use them in conjunction with minutiae to achieve robust matching decisions. In addition, the fine details of level 3 features could potentially be exploited in circumstances that require high-confidence matches. 1.3 Approach In this thesis an approach has been presented which addresses the various issues and challenges (discussed in previous section) in fingerprint matching. The aim is to reduce the error rates, namely False Acceptance Rate (FAR) and False Rejection Error (FRR) in the existing fingerprint matching algorithms. The proposed approach utilizes Level 3 features (pores and ridge contours) along with Level 2 features for matching fingerprints at 500 ppi, in a hierarchical manner. The first stage of hierarchical matcher is the minutia matching stage in which the Level 2 minutiae points from the query and reference fingerprints are matched. Based on the performance of the minutiae matcher the matching process either stops if a match is found or continues to next stage where the Level 3 features are used to decide match/non-match decision. The hierarchical matcher utilizes the fine details of Level 3 features to decide the match/non-match decision in circumstances where the decision can not be made solely on the basis of Level 2 features.

20 1.3. APPROACH 11 The proposed approach addresses the various challenges in fingerprint matching in following way: The plastic nature of finger skin results in non-linear distortion in successive acquisitions of the same finger. To deal with this problem the matching of all feature classes (Level 2 and Level 3) are done within a local region. The use of localized matching minimizes the effects of non-linear distortion. This is because the effects of distortion does not significantly alter the fingerprint pattern locally [35]. Due to high displacement and rotation which are introduced during fingerprint acquisition, different impressions of the same finger differ from each other. Most of the existing minutia matching algorithms first align fingerprint images and then find minutia correspondences. But in case of poor quality fingerprints, global registration (alignment) parameters do not exist and as a result it is not possible to get a correct alignment. The errors introduced during registration steps can introduce errors in the subsequent steps. The proposed approach does not use alignment at any stage and relies on rotationally invariant structures (in case of minutia and pore matching) and features (in case of ridge contours) The high displacement/rotation introduced during fingerprint acquisition results in small overlap between query and reference fingerprints. Also the noise introduced by several factors such as poor skin conditions, unclean scanner surface etcetera, results in a very small portion of the fingerprint which can actually be used for comparison. The use of Level 3 features in conjunction with Level 2 features takes care of such situations. Studies [40, 41] have shown that given a sufficiently high resolution fingerprint, the use of Level 3 features

21 1.4. THESIS OUTLINE 12 from fingerprint fragments results in same quantity of discriminative information that can be extracted when Level 2 features are considered and the entire image is used. Finally, due to the noise in the fingerprints the feature extraction techniques often introduce errors such as missing or spurious minutiae/pores. The matching technique should handle such cases. The proposed approach uses an elastic string matching algorithm for pores matching and minutiae matching, which can accommodate perturbations of minutiae/pores from their true locations and can tolerate spurious and missing minutiae/pores. Also, for matching ridge contours we have used statistical features based on Zernike moments and Local Binary Patterns (LBP) which are known for their tolerance to noise and gray-scale invariance, respectively. 1.4 Thesis Outline The outline of the thesis is as follows: Chapter 2 discusses the literature on fingerprint based verification systems. Chapter 3 presents the proposed hierarchical fingerprint matching system. Chapter 4 presents the results and evaluations of the proposed approach. Finally, in Chapter 5 we conclude and outline the future work.

22 13 Chapter 2 BACKGROUND AND LITERATURE REVIEW A fingerprint recognition system may operate either in verification mode or identification mode. In verification mode, the system verifies an individual s identity by comparing the input fingerprint with the individual s own template(s) stored in the database. In, the identification mode, the system identifies an individual by searching the templates of all the users in the database for a match. The fingerprint classification and indexing techniques are used to speed up the search in fingerprint based identification systems. The fingerprint feature extraction and matching algorithms are usually quite similar for both fingerprint verification and identification problems. In this thesis the focus is on fingerprint based verification systems. The various stages in a fingerprint verification system is shown in Figure 2.1. The first stage is the data acquisition stage in which a fingerprint image is obtained from an individual by using a sensor. The next stage is the pre-processing stage in which the input fingerprint is processed with some standard image processing

23 2.1. DATA ACQUISITION 14 algorithms for noise removal and smoothening. The pre-processed fingerprint image is then enhanced using specifically designed enhancement algorithms which exploit the periodic and directional nature of the ridges. The enhanced image is then used to extract salient features in the feature extraction stage. Finally, the extracted features are used for matching in the matching stage. This chapter discusses the current state of the art feature extraction techniques and gives a literature survey on the various fingerprint matching algorithms. Figure 2.1: Architecture of a Fingerprint Verification System. 2.1 Data Acquisition Traditionally, in law enforcement applications fingerprints were acquired off-line by transferring the inked impression on a paper. Nowadays, the automated fingerprint verification systems use live-scan digital images of fingerprints acquired from a fingerprint sensor. These sensors are based on optical, capacitance, ultrasonic, thermal and other imaging technologies.

24 2.1. DATA ACQUISITION 15 The optical sensors are most popular and are fairly inexpensive. These sensors are based on FTIR (Frustrated Total Internal Reflection) technique. When a finger touches the sensor surface (which actually is a side of a glass prism), one side of the prism is illuminated through a diffused light. While the fingerprint valleys that do not touch the sensor surface reflect the light, ridges that touch the surface absorb the light. The sensor exploits this differential property of light reflection to differentiate the ridges (which appear dark) from the valleys. The capacitive sensors utilize the principle associated with capacitance to form the fingerprint images. These sensors consists of a two-dimensional array of metal electrodes. Each metal electrode acts as one plate of a parallel-plate capacitor and the contacting finger acts as the other plate. When a finger is pressed on the sensor surface, it creates varying capacitance values which depends inversely on the distance between the sensing plate and the finger surface. The ridges thus have increased capacitance compared to valleys. This variation is then converted into an image of the fingerprint. The Ultrasound technology based sensors are the most accurate of the fingerprint sensing technologies. It uses ultrasound waves and measures the distance based on the impedance of the finger, the plate and air. The thermal sensors are made up of pyro-electric materials, which generate a temporary electrical potential when they are heated or cooled. When a finger is swiped across the sensor, there is differential conduction of heat between the ridges and valleys (as skin is a better conductor than the air in the valleys) which is measured by the sensor. One of the most essential characteristics of a digital fingerprint image is its resolution, which indicates the number of dots or pixels per inch (ppi). The minimum resolution that allows the feature extraction algorithms to locate minutiae is 250 to

25 2.2. IMAGE PREPROCESSING ppi. The FBI s standard for resolution of fingerprint sensors is 500 ppi. A large number of automated fingerprint verification systems accept 500 ppi fingerprints. Figure 2.2 shows the fingerprints captured at different resolutions. At 500 ppi pores are visible but in-order to extract pores reliably a significantly higher resolution (1000 ppi) of image is needed. Figure 2.2: Fingerprints at different resolutions a) 380 ppi, b) 500 ppi, c)1000 ppi ([29]). 2.2 Image Preprocessing The preprocessing steps try to compensate for the variations in lighting, contrast and other inconsistencies which are introduced by the sensor during the acquisition process. The following preprocessing steps are generally used: Gaussian Blur: A Gaussian blur is a convolution operation which is applied to the original fingerprint image to reduce image noise (introduced by sensor). The Gaussian kernel used for blurring is given by: G(x, y) = 1 x 2 +y 2 2πσ e 2σ 2 (2.1) where σ is the variance of the gaussian distribution, x is the distance from the

26 2.3. FINGERPRINT IMAGE ENHANCEMENT 17 origin along the horizontal axis, y is the distance from the origin along the vertical axis. Sliding-window Contrast Adjustment : Sliding-window contrast adjustment is used to compensate for any lighting inconsistencies within a fingerprint and to obtain contrast consistency among different fingerprints. A m m window is centered on each pixel of the Gaussian blurred image. The corresponding output pixel value is then calculated by finding the minimum and maximum intensity values within the window and by using: ( ) 255 O(i, j) = (I(i, j) min i,j ) (2.2) max i,j min i,j where I(i, j) and O(i, j) are the input and output pixel intensity values respectively, and min i,j and max i,j are the minimum and maximum pixel intensity values within the m m window centered at pixel (i, j). Histogram-based Intensity Level Adjustment : This final step is used to further enhance the ridges and valleys. The image s histogram is examined to determine two intensity values: a lower threshold L and an Upper threshold U. The intensity value (I(i, j)) of each pixel is processed using these thresholds to obtain the output pixel intensity (O(i, j)) which is given by the following equation : 255 if I(i, j) >U O(i, j) = 0 if I(i, j) <L (2.3) (I(i, j) L) ( ) 255 else U L 2.3 Fingerprint Image Enhancement The performance of fingerprint feature extraction and matching algorithms relies heavily on the quality of the input fingerprint images. Due to various factors such as

27 2.3. FINGERPRINT IMAGE ENHANCEMENT 18 skin conditions (e.g., wet, dry, cuts, scars and bruises), non-uniform finger pressure, noise introduced by sensor and inherently poor-quality fingers (e.g, manual workers, elderly people), a significant percentage of fingerprint images is of poor quality. Infact, a single fingerprint image may contain regions of good, medium, and poor quality. Thus an enhancement algorithm which can improve the quality of ridge structure is necessary. A survey on different enhancement techniques can be found in [35]. The most widely used technique for fingerprint image enhancement is based on contextual filters. The parameters of these filters change according to the local context i.e. local ridge frequency and orientation. Such a filter can capture the local information and can use them to efficiently remove the undesired noise (i.e. fill small ridge breaks, fill intra-ridge holes, and separate parallel touching ridges) and preserve the true ridge and valley structure. The filters themselves may be defined in spatial or in the Fourier domain. This section describes a popular enhancement algorithm by Sharat et al. [16], which uses contextual filtering in Fourier domain. The algorithm consists of two stages. The first stage consists of STFT (Short Time Fourier Transform) analysis and the second stage performs the contextual filtering. The STFT analysis stage yields the ridge orientation image (O(x, y)), the ridge frequency image (F (x, y)) and the region mask (R(x, y)). The orientation image represents the instantaneous ridge orientation at every point in the fingerprint image. The frequency image indicates the average inter ridge distance distance within a local region. The region mask indicates the foreground regions of the image where ridge structures are present. During STFT analysis, the image is divided into overlapping windows. This is done based on the assumption that the image has a consistent

28 2.3. FINGERPRINT IMAGE ENHANCEMENT 19 orientation and frequency within a small local region. This assumption however is not true for regions with singularities such as core, delta etc. The Fourier spectrum within each window is analyzed and a probabilistic approximation of the dominant ridge orientation and frequency within each window are obtained. An energy map E(x, y) is also obtained during STFT analysis where each value indicates the energy content of the corresponding block. This energy map can be used as a region mask to distinguish between the foreground and background regions (the background and noisy regions are characterized by very little energy content in the Fourier spectrum.). The orientation image (O(x, y)) is then used to compute the Coherence Image which contains coherence values of the various regions within the fingerprint. The coherence value is low in regions with points of singularities (core, delta etc.). The coherence image is then used to adapt the angular bandwidth of the directional filter. The resulting contextual information from STFT analysis is then used to filter each overlapping window (B) in the Fourier domain. The filter used is separable in radial (Equation 2.5) and angular (Equation 2.6) domain and is given by H(r, φ) = H r (r)h φ (φ) (2.4) [ ] (rr BW ) H r (r) = 2n (rr BW ) 2n + (r 2 rc) 2 2n cos 2 π (φ φ c) 2 φ H ( φ)(φ) = BW if φ < φ BW 0 otherwise (2.5) (2.6) where r BW and φ BW are the radial bandwidth and angular bandwidth, respectively; r c and φ c are the mean frequency and mean orientation, respectively. The enhanced block B is obtained by: F = F H(r, φ) (2.7)

29 2.4. FEATURE EXTRACTION 20 B = F F T 1 (F ) (2.8) Finally, the results of each analysis window is tiled to obtain the enhanced image. 2.4 Feature Extraction Chapter 1 introduced the fingerprint features at different Levels. The feature extraction technique for minutiae points (bifurcation and endings), pores and ridge contours is described in this section Minutiae Extraction Most of the existing minutia extraction techniques trace the fingerprint skeleton to find the different type of minutia points. The ridge bifurcation and ridge endings are the most prominent minutiae types and have been extensively used by matching algorithms. The flowchart of a typical minutia extraction algorithm is depicted in Figure 2.3. It consists of following stages: Orientation Estimation : A fingerprint image is a oriented texture pattern and the ridge orientation at a pixel (x, y) is the angle that the ridges within a small neighborhood centered at (x, y), form with the horizontal axis. The fingerprint image is first divided into a number of non-overlapping blocks. An analysis of grayscale gradients within a block is done to estimate the representative ridge orientation within that block. Different approaches such as optimization[61], averaging [39] or voting[47] could be used to determine the block orientation. Segmentation : During this stage the portions of fingerprint image depicting

30 2.4. FEATURE EXTRACTION 21 the finger (foreground) is segmented. This step is useful in order to avoid the extraction of spurious features from background and noisy regions within a fingerprint. The foreground and background can be differentiated by the presence of an oriented pattern in the foreground and of an isotropic pattern (i.e., one without a dominant orientation) in the background. The simplest approaches segment the foreground by global or local thresholding. Since the fingerprint background is not always uniform and lighter than foreground (due to the presence of noise such as dust, grease on the sensor surface), a simple approach based on local or global thresholding is not effective and more robust segmentation techniques [45, 10, 60] are required. Ridge Detection : An important property of the ridges in a fingerprint image is that the gray level values on ridges attain their local maxima along a direction normal to the local ridge orientation [30]. The ridge pixels are identified based on this property. The resulting ridge map often contains false ridges in the form of holes and speckles (due to presence of noise, breaks, and smudges, etc in the input image). The ridge map is cleaned (false ridges are removed) using a connected component algorithm [57]. Finally, the ridges are thinned using standard thinning algorithm [50]. Minutiae Detection : The minutia points are then extracted from the thinned ridge map by examining the 8-neighborhood of each ridge skeleton pixel. Let (x, y) denote a pixel on a thinned ridge, and N 0, N 1,..., N 7 denote its eight neighbors. A pixel (x, y) is a ridge ending if ( 7 i=0 N i) = 1 and a ridge bifurcation if ( 7 i=0 N i) > 2. The minutiae points thus obtained in the above step may contain many spurious minutiae. This may occur due to the presence of noise,

31 2.4. FEATURE EXTRACTION 22 ridge breaks (even after enhancement) and image processing artifacts. Postprocessing : A number of heuristics are used to remove spurious minutiae. False minutia points are generally obtained at the borders as the ridges ends abruptly. These false minutiae at the borders can be recognized by analyzing the number of foreground pixels in a region around minutia point. If number of foreground pixels is relatively small then the minutia point can be removed. Too many minutiae in a small regions, very close ridge ending (with orientations anti-parallel to each other) and two very closely located bifurcations sharing a common short ridge may indicate spurious minutiae and could be discarded [27] Pores Extraction Pores are extremely fine details which are lost after the enhancement stage. Therefore, for pore extraction the enhancement stage is omitted and pores are directly extracted from the preprocessed image. The pore extraction algorithm can be broadly classified into two classes: the first class of algorithms extract pores by tracing fingerprint skeletons, the second class of algorithms extract pores directly from gray scale image. Stosz et al. [40] and Kryszczuk et al. [67] have proposed skeletonization based approach for pore extraction. The skeletonization based approach is reliable for extracting pores in good quality (and high resolution) images. As the image resolution decreases or the skin condition is not favorable, the method does not give reliable results. In [29] Jain et al. have proposed a pore extraction technique directly from gray scale image. The majority of existing approaches for pore extraction consider only location information of pores for matching. The pores are distributed over

32 2.4. FEATURE EXTRACTION 23 Figure 2.3: A typical minutiae extraction process ([30])

33 2.4. FEATURE EXTRACTION 24 ridges and using orientation detail can provide additional information for matching. A recent study [23] by the International Biometric Group has proposed a new approach for pore extraction which utilizes orientation information of pores along with the location information. The approach proposed in [23] is presented in this section. The first step in pore extraction process is the estimation of ridge orientation. This data is utilized later in the representation of pores. The Local Ridge orientation is determined by the least square estimate method [25]. The fignerprint image is first divided into a number of non-overlapping blocks of dimension w w. For each pixel (i, j) in the preprocessed image, gradients in both the horizontal and vertical direction (Equation 2.9 and Equation 2.10) are calculated by using a Sobel operator. For each w w block, the gradient along both the x and y directions is summed (Equation 2.11 and Equation 2.12) and finally the arctangent is used to obtain the representative orientation (Equation 2.13). δ x = (I(i 1, j 1) + 2I(i 1, j) + I(i 1, j + 1)) (I(i + 1, j 1) + 2I(i + 1, j) + I(i + 1, j + 1)) (2.9) δ y = (I(i 1, j 1) + 2I(i, j 1) + I(i + 1, j 1)) (I(i 1, j + 1) + 2I(i, j + 1) + I(i + 1, j + 1)) (2.10) V x (i, j) = V y (i, j) = i+w/2 j+w/2 u=i w/2 v=j w/2 i+w/2 j+w/2 u=i w/2 v=j w/2 2δ x (u, v)δ y (u, v) (2.11) (δ 2 x(u, v) δ 2 y(u, v)) (2.12) θ(i, j) = 1 ( ) 2 arctan Vy (i, j) V x (i, j) (2.13)

34 2.4. FEATURE EXTRACTION 25 Next the pores are extracted from the preprocessed image. The pores are first enhanced by convolving the preprocessed fingerprint image with a Mexican hat kernel. The Mexican hat mother wavelet is defined in Equation 2.14 and the daughter wavelets are defined in Equation Ψ(i, j) = ( 1 x 2 y 2) e x2 +y 2 2 (2.14) Ψ a,b (λ) = 1 ( ) λ b Ψ a a (2.15) where a is the factor by which mother wavelet is scaled (dilated), b is the factor by which the mother wavelet is translated (or shifted) and λ specifies the center of daughter wavelet. After the convolution step, the fingerprint image is thresholded with a single-point threshold (T ). After this step the pores have an intensity of 255. The pores are then extracted by a blob detector which locates groups of connected pixels (pores) with an intensity of 255 and with size within a pre-determined range. Each pore thus extracted is represented by the coordinates of the central pixel and an orientation θ, which is the ridge orientation at that particular location Ridge Contour Extraction Ridge contours can be extracted by using classical edge detection algorithms. However, these algorithms are sensitive to creases, pores etc. and as a result the detected edge contours are often very noisy (especially in low resolution images). Jain et al. [29] have proposed an algorithm to extract the ridge contours which uses a simple filter to detect ridge contours. The algorithm can be described as follows: First, the image is enhanced by using Gabor filters [24]. Next, a wavelet transform

35 2.5. FINGERPRINT MATCHING 26 is applied to the original fingerprint image. The enhancement ridge contours are obtained by using a linear subtraction of wavelet response and gabor enhanced image. The resulting image is then binarized using a heuristically determined threshold. Finally, the binarized image is convolved with a filter H (Equation 2.16) and ridge contours are extracted. r(x, y) = I b (x, y)h(x n, y m) (2.16) n,m where filter H = (0, 1, 0; 1, 0, 1; 0, 1, 0) counts the number of neighborhood edge points for each pixel. A pixel (x, y) is classified as a ridge contour pixel if r(x, y) = 1 or Fingerprint Matching A variety of automatic fingerprint matching algorithms have been proposed in the pattern recognition literature. This chapter provides a survey of existing approaches for automatic fingerprint matching. Most of these algorithms have no difficulty in matching good quality fingerprint images, but matching low quality and partial fingerprints remains a challenging problem. The main approaches proposed in the literature for fingerprint matching can be roughly classsified into three categories and they are correlation-based matching, minutiae-based matching and ridge feature-based matching. In correlation-based fingerprint matching, the template and query fingerprint images are spatially correlated to estimate the degree of similarity between them. The minutia-based techniques essentially consists of finding the alignment between the query and template minutia points. The ridge feature-based techniques rely on various features of fingerprint ridge pattern such as ridge shape, texture information, local orientation and frequency. A very good literature survey on fingerprint recognition can be found in [35].

36 2.5. FINGERPRINT MATCHING Correlation-based Techniques Let T and Q denote the template and query fingerprint images, respectively. The sum of squared differences (SSD) between the intensities of corresponding pixels in the two images, can be used as a measure of the diversity between them. SSD(T, Q) = T Q 2 = (T Q) t (T Q) = T 2 + Q 2 2T t Q (2.17) In the above equation, the superscript t denotes the transpose operation on a vector. The cross-correlation (CC) between T and Q is given by, CC(T, Q) = T t Q. If the terms T 2 and Q 2 in above equation are constants, the diversity (SSD(T, Q)) becomes inversely proportional to cross-correlation, which constitutes the third term ( 2.CC(T, Q)) in Equation 3.1. The cross-correlation then becomes a measure of similarity. Two fingerprint impressions from same finger differ due to many factors, as discussed in the beginning of this chapter. Thus, additional steps are required before similarity between T and Q can be calculated. Let Q ( x, y,θ) represent a transformation of the query image, where x and y are translation parameters along x and y direction, and θ is the rotation parameter. Them a similarity measure can be given by: S(T, Q) = max x, y,θ CC(T, Q( x, y,θ) ) (2.18) S(T,Q) only considers the rotation and translation factors and thus is not a very accurate measure of similarity. This method fails if the images are highly distorted. The distortion is more pronounced in global fingerprint patterns, thus considering local regions can minimize distortion to some extent. [11, 51] presents some of the approaches to localized correlation based matching. Also variable finger pressure and skin condition cause image brightness, contrast and ridge thickness to vary across different acquisitions of same finger. The use of more sophisticated correlation measures

37 2.5. FINGERPRINT MATCHING 28 such as the normalized cross-correlation or the zero-mean normalized cross-correlation may compensate for contrast and brightness variations and applying a proper combination of enhancement, binarization, and thinning steps (performed on both T and Q) may limit the ridge thickness problem [35]. The computation of maximum correlation (S(T, Q)) in spatial domain is very expensive. The computational complexity can be reduced and translation invariance can be achieved by calculating correlation in fourier domain [18]. T Q = F 1 (F (T ) F (Q)), (2.19) where denotes the correlation in spatial domain, denotes the complex conjugate, denotes the point-by-point multiplication of two vectors, F (.) and F 1 (.) denote the Fourier transform and inverse Fourier transform, respectively. Rotation has to be dealt with separately in this technique. Fourier-Mellin transform [68] can be used to achieve both rotation and translation invariance Minutiae-based Methods Let T and Q be the feature vectors, representing minutiae points, from the template and query fingerprint, respectively. Each element of these feature vectors is a minutia point, which may be described by different attributes such as location, orientation, type, quality of the neighbourhood region, etc. The most common representation of a minutia is the triplet x, y, θ, where x, y is the minutia location and θ is the minutia angle. Let the number of minutiae in T and Q be m and n, respectively. T = m 1, m 2,..., m m, m i = x i, y i, θ i, i = 1...m Q = m 1, m 2,..., m n, m j = x j, y j, θ j, j = 1...n

38 2.5. FINGERPRINT MATCHING 29 A minutia m i in T and m j in Q are considered matching, if following conditions are satisfied: sd(m j, m i ) = ((x j x i) 2 + (y j y i) 2 ) r 0 (2.20) dd(m j, m i ) = min( θ j θ i, 360 θ j θ i ) θ 0 (2.21) Here, r 0 and θ 0 are the parameters of the tolerance window which is required to compensate for errors in feature extraction and distortions caused due to skin plasticity. The number of matching minutia points can be maximized, if a proper alignment (registration parameters) between query and template fingerprints can be found. Correctly aligning two fingerprints requires finding a complex geometrical transformation function (map()), that maps the two minutia sets (Q and T ). The desirable characteristics of map() function are: it should be tolerant to distortion, it should recover rotation, translation and scale 1 parameters correctly. Let match() be a function defined as: match(m j, m i ) = 1 if m j and m i satisfy (2.20), (2.21) 0 otherwise (2.22) where, map(m j) = m j. Thus, the minutia matching problem can be formulated as [35]: max P m match(map(m P (i)), m i ) (2.23) i=1 where P () is the minutia correspondence function that determines the pairing between the minutia points in Q and T. A minutiae-based matching algorithm thus attempts to find the appropriate mapping function (map()) and correspondence function (P ()), so that the number of matching minutia points, between T and Q, can be maximized. 1 scale is considered when the fingerprints are captured at different resolutions

39 2.5. FINGERPRINT MATCHING 30 If either of P () or map() is known then solving Equation 2.23 becomes a very trivial task. But, in practice, neither map() nor P () is known apriori. The errors in minutia extraction (spurious, missing minutia and measurement errors) further make the minutia matching problem very hard. A number of minutia matching algorithms have been proposed in literature. These algorithms can be broadly classfied as: Global Matching : This approach tries to simultaneously align all the minutia points. The alignment could be either implicit or explicit. The Implicit alignment technique tries to find the point correspondences and in the process optimal alignment is obtained. The explicit alignment technique on the other hand explicitly aligns the minutia sets first, before finding the point correspondences. Local Matching : This approach tries to match local minutia structures; local structures are characterized by attributes that are invariant with respect to global transformations. The local versus global matching is a trade-off among simplicity, low computational complexity, high distortion tolerance (local matching), and high distinctiveness (global matching). Local matching techniques are more robust to non-linear distortion and partial overlaps when compared to global approaches. Matching local minutiae structures relax global spatial relationships which are considered to be highly distinctive, and therefore reduce the amount of information available for discriminating fingerprints.

40 2.5. FINGERPRINT MATCHING Global Matching The minutia matching problem has been addressed as point pattern matching problem in literature. A number of approaches to point pattern matching exists in literature. Some of the approaches are discussed here. The relaxation approach [58] iteratively adjusts the confidence level of each corresponding pair based on its consistency with other pairs until a certain criterion is satisfied. Let p ij be the probability that point i corresponds to point j, and c(i, j; h, k) be a compatibility measure between the pairing (i, j) and (h, k). At each iteration r, p ij is incremented if it increases the compatibility of other points and is decremented otherwise. p (r+1) ij = 1 m m h=1 [ ] max c(i, j; h, k).p(r) ij, i = 1..m, j = 1..n (2.24) k=1..n At convergence, each point i is associated with a point j such that p ij = max s p is. The iterative nature of this approach makes it slow and unsuitable for automatic matching. Hough transform based approach [66] are quite popular for minutia matching. This approach converts point pattern matching to a problem of detecting the highest peak in the hough space of transformation parameters. The hough space of transformation parameters consists of all the possible values of parameters under the assumed distortion model. Ratha et al. [59] have used a transformation space consisting of quadruples ( x, y, θ, s), representing translation along x direction, translation along y direction, rotation and scale parameters, respectively. To avoid computation complexity the search space is discretized into a finite set of values: x { x 1, x 2,..., x a } y { y 1, y 2,..., y b }

41 2.5. FINGERPRINT MATCHING 32 θ {θ 1, θ 2,..., θ c } s {s 1, s 2,..., s d } A four dimensional accumulator A of size (a b c d) is maintained. Each cell A(i, j, k, l) represents the likelihood of the transformation parameters ( x i, y j, θ k, s l ). The following algorithm is used to accumulate evidence: for each m i in T for each m j in Q for each θ {θ 1, θ 2,..., θ c } if dd(θ j + θ, θ i ) θ 0 for each s {s 1, s 2,..., s d } { x y = x i y i s. cos θ sin θ sin θ cos θ x j y j } Let ( x +, y + ) be the quantization of ( x, y) to the nearest bin A [ x +, y +, θ, s] = A [ x +, y +, θ, s] + 1 At the end of the accumulation process, the best alignment transformation is obtained as ( x, y, θ, s ) = argmax ( x +, y +,θ,s)a [ x +, y +, θ, s ] A hierarchical Hough transform-based algorithm [38] may be used to reduce the size of the accumulator array by using a multi-resolution approach.

42 2.5. FINGERPRINT MATCHING 33 It can be shown that if a perfect pre-alignment could be achieved, the minutiae matching problem could be reduced to a simple pairing problem. Most minutia-based matchers first transform (register) the input and template fingerprint features into a common frame of reference. The parameters of alignment are typically estimated either by (i) superimposing singular points in the fingerprints, e.g., core and delta segments; (ii) correlating the orientation images; or (iii) by correlating ridge features (e.g., length and orientation of ridges). Jain et al [34] have proposed an alignment based minutia matching approach that uses ridge features for alignment and an adaptive elastic string matching algorithm [19] for matching the pre-aligned minutia sets. In their approach, each minutia is associated with the ridge on which it resides. The ridge is represented as a planar curve with origin coincident with the minutia location and x-axes along the minutia direction. The global transformation parameters ( x, y and θ) are caculated from a pair of matching ridges. Finding such a pair involves iteratively matching pairs of ridges, untill a pair is found whose matching degree exceeds a certain threshold. The pair found is then used for aligning the query and template minutia sets. The minutia corresponding to the matching ridges are used as reference minutia points during matching stage. Each minutia in Q and T is converted to a polar coordinate system with respect to the reference minutia in its set. Both Q and T are transformed into a string of minutia points, ordered in increasing order of radial angle. The strings are matched using a dynamic programming technique to find their edit distance. The use of an adaptive tolerance window for matching minutia points tolerates local distortion. And matching minutiae representations by their edit distance tolerates missing and spurious minutiae. However, an exhaustive reordering and matching is required to deal with the problem of order flips, which may be caused due to measurement errors introduced during feature extraction.

43 2.5. FINGERPRINT MATCHING 34 The alignment based approaches are not very accurate, as reliably extracting singular points from low quality images is difficult. They suffer when image is of poor quality or is highly distorted. In such cases global registration parameters does not exist and errors introduced during alignment step lead to errors in further steps Local Matching The local matching approaches rely on evidence accumulated from matching local structures in neighbourhood of minutia points. These local structures are generally characterized by properties that are invariant with respect to global transformation, and therefore are suitable for matching without any apriori alignment. However, local neighborhoods do not sufficiently capture the global structural relationships thereby making false accepts very common. Thus, it is possible that local minutia structures in two non-matching fingerprints might match. Figure 2.4 shows an example of two false matched local structures. They are similar at the local structures but conflict with each other in global context (at very different locations with respect to the core and delta points). To deal with this problem, the local matching algorithms use an additional consolidation step to check whether the locally matched minutia points match at global level or not. A large number of local matching techniques has been proposed in literature. An overview of some of the algorithms is discussed here. Jiang and Yau [37] use a local structure formed by a central minutia and its two nearest-neighbour minutiae; the feature vector v i associated with the minutia m i, whose nearest neighbors are m j and m k is : v i = [d ij, d ik, θ ij, θ ik, φ ij, φ ik, n ij, n ik, t i, t j, t k ] where d ab is the distance between minutiae m a and m b, φ ab is the direction difference

44 2.5. FINGERPRINT MATCHING 35 Figure 2.4: An example of two false matched local structures ([36]) between the angle θ a of m a and the direction of the edge connecting m a to m b, n ab is the ridge count between m a and m b, and t a is the minutia type of m a. For all minutia pairs (m i, m j), m i T and m j Q, a weighted distance between their vectors v mi and v m j is calculated. An additional consolidation step is used to enforce the result of local matching. The best matching pair (with least distance ) is used for registering the two minutia sets. The feature vectors of the remaining aligned pairs are matched and a final score is computed by taking into account contributions from first stage and consolidation stage. Ratha et al. [26] have proposed a star representation for capturing local minutia structure in the form of a minutia adjacency graph (MAG). The star associated with a minutia m i is the graph G i = (V i, E i ) consisting of the set of vertices V i containing all minutia points m j whose distance (d ij ) from m i is less than a predefined threshold, and the set of edges E i contining edges from m i to all vertices in V i. Each edge e ij E i is labelled with a 5-tuple (m i, m j, d ij, rc ij, φ ij ), where rc ij is the ridge count between

45 2.5. FINGERPRINT MATCHING 36 m i and m j and φ ij is the angle subtended by the edge with the x-axis. During local matching, each star in Q is matched with each star in T, the matching is performed by clockwise traversing the corresponding graphs in an increasing order of radial angle. At the end of local matching stage, a set T OP of best matched star pairs is returned and these pairs are further checked for consistency during the consolidation stage. A pair of stars is consistent if their spatial relationships (distance and ridge count) with a minimum fraction of the remaining stars in TOP are consistent. Sharath et al. [15] define a local structure representation called K plet that is invariant under translation and rotation. The K plet consists of a central minutiae m i and K other minutiae m 1, m 2...m K chosen from its local neighbourhood. Each neighbourhood minutiae m j is defined as a 3-tuple (φ ij, θ ij, r ij ), where r ij represents the Euclidean distance between m i and m j, θ ij is the relative orientation of m j with respect to the central minutiae m i, and φ ij represents the direction of the edge connecting m j and m i, with respect to the orientation of central minutia. A given fingerprint is represented as a directed graph G(V, E). V is the set of vertices containing all minutia points and E is the set of directed edges containing all possible (m i, m j ) pairs, where m i and m j are neighbouring minutia points. Each vertex v is denoted by a 4-tuple (x v, y v, θ v, t v ) representing the co-ordinate, orientation and type of minutiae. Each directed edge (u, v) is labelled with corresponding K plet co-ordinates (r uv, φ uv, θ uv ). Let G(V, E), H(V, E ) be the graphical representation for T and Q, respectively. The matching algorithm is based on matching a local neighbourhood (K-plets) and propagating the match to the K-plet of all the minutiae in the neighbourhood successively. The minutia points in K-plet are arranged in increasing order of radial distance r ij and a dynamic programming based string alignment algorithm [19] is used for local matching. The consolidation of the local matches is done by a

46 2.5. FINGERPRINT MATCHING 37 Coupled Breadth First Search (CBFS) algorithm that propagates the local matches simultaneously in both the fingerprints. The CBFS algorithm requires two vertices v i T and v j Q as the source nodes from which to begin the traversal. The CBFS algorithm is executed for all possible correspondence pairs (v i, v j ). The pair with maximum number of matches is used to compute the final matching score Ridge Feature-based Matching Techniques There are various reasons which induce designers of fingerprint recognition techniques to look for features, beyond minutiae. Minutiae based matching algorithms do not perform well in case of small fingerprints that have very few minutiae. Reliable extraction of minutiae from poor quality fingerprints is very difficult. Furthermore, registering minutiae representation is very challenging. The most commonly used alternative features are i) global and local texture information, and ii) Level 3 features Texture Feature based Techniques Global and local texture information are important alternatives to minutiae. Textures are defined by spatial repetition of basic elements, and are characterized by properties such as scale, orientation, frequency, symmetry, isotropy, and so on. Fingerprint ridge lines are mainly described by smooth ridge orientation and frequency, except at singular regions. These singular regions are discontinuities in a basically regular pattern and include the loop(s) and the delta(s) at a coarse resolution and the minutiae points at a high resolution. Global texture analysis fuses contributions from different characteristic regions into a global measurement and, as a result, most of the available spatial information is lost. Local texture analysis has proved to be more effective than global feature analysis; although most of the local texture information

47 2.5. FINGERPRINT MATCHING 38 is carried by the orientation and frequency images, most of the proposed approaches extract texture by using a specialized bank of filters. Jain et al. [31] have proposed a filter bank based local texture analysis technique. The fingerprint image is tesselated into 80 cells (5 bands and 16 cells) around the core point and a feature vector representing texture information is obtained from the tesselation. The feature vector consists of an ordered enumeration of the features extracted from the local information contained in each sector. Thus the feature elements capture the local texture information and the ordered enumeration of the tesselation captures the global relationship among the local contributions. A bank of 8 gabor filters (8 orientations and 1 scale = 1/10) is used to obtain texture informtion from each sector. Thus each fingerprint is represented by a 640 (80 8) fixed-size feature vector called the F ingercode. The generic element V ij of the vector (i = is the cell index, j = 1..8 is the filter index) denotes the energy revealed by the filter j in the cell i, and is computed as the average absolute deviation (AAD) from the mean of the responses of the filter j over all the pixels of the cell i. Matching two fingerprints is then performed by computing the Euclidean distance between their Finger-Codes. The disadvantage of this approach is that it uses a core points as a reference. When the core points cannot be reliably detected, or it is close to the border of the fingerprint area, the FingerCode of the input fingerprint may be incomplete or incompatible with the template. Also, fingercodes are found to be not as distinctive as minutiae. However, they carry complementary information which can be combined with minutiae to yield higher accuracy. In [28] and [64] a variant of above method has been proposed where the two fingerprints to be matched are first aligned using minutiae and tessellation is performed over a square mesh grid. Nanni et al.[52] have proposed a hybrid approach where fingerprints are pre-

48 2.5. FINGERPRINT MATCHING 39 aligned using minutiae, and then texture features are extracted by invariant local binary patterns (LBP) from the fingerprint image convolved with Gabor filters. The fingerprint image is first divided into several sub-windows, then a segmentation step is performed in order to discard the background, finally each sub-window is convolved with a bank of Gabor filters and LBP are calculated. The comparison between the unknown fingerprint and the stored template is performed by the average Euclidean distance calculated among the correspondent couples of foreground sub-windows Level 3 Features-based Techniques The use of Level 3 features in an automated fingerprint identification system has been studied by only a few researchers. Existing literature is exclusively focused on the extraction of pores in order to establish the viability of using pores in high resolution fingerprint images to assist in fingerprint identification. Stosz and Alyea [67] have presented a technique which utilizes pore information, extracted from high-resolution fingerprint images, to augment matching processes that use only Level 2 data. The images were taken by a custom built optical/electronic sensor in a controlled environment. An enrollment process is required in which singular points (used as origin), minutia points, and characteristic regions containing pores are selected manually and stored in template. Matching is initiated by determining the origin of the input fingerprint which is defined as the position of maximum correlation between the origin stored in the template and the input binary fingerprint. Next, the remaining binary image segments from the template are compared to the input fingerprint. Two pores are considered matched if they lie within a certain bounding box. Finally, experimental results based on a database of 258 fingerprints from 137 individuals showed that by combining minutia and pore information, a lower FRR

49 2.5. FINGERPRINT MATCHING 40 of 6.96 percent (compared to 31 percent for minutiae alone) can be achieved at a FAR of 0.04 percent [67]. Later, Roddy and Stosz [63] conducted a statistical analysis of pores and predicted the performance of a pore-based automated fingerprint system. The study demonstrated the efficacy of using pores, in addition to minutiae, for improving the recognition performance. Kryszczuk, et al. [40, 41] have conducted research to determine whether Level 3 features could compensate for decreased number of Level 2 features when attempting to match with fingerprint fragments, given sufficiently high-resolution data. The experiments are done on fingerprints acquired from a high resolution (2000 dpi) custombuilt scanner. The size of database used was small (12 genuine and 6 impostor comparisons). Pores and ridge structure are used in conjunction with Level 2 features for comparison of fragmentary fingerprint images and comparison has been done based on a geometric distance criterion. The authors have presented two hypotheses: i) as the size of the fingerprint fragment decreases, or the number of minutiae decreases, the usefulness of Level 3 data increases, and ii) given a sufficiently high resolution, the same quantity of discriminative information could be extracted from fingerprint fragments by using Level 3 data, as could be when considering Level 2 data extracted from the complete fingerprint image. Jain, et al. [29, 33] have proposed a hierarchical matching system that utilizes all the three levels extracted from 1000 ppi fingerprint images. The level 3 features (pores and ridge contours) are locally matched, in windows associated with matched minutiae points, using the Iterative Closest Point (ICP) [13] algorithm. There is a relative reduction of 20 percent in the EER when Level 3 features are employed in combination with Level 1 and 2 features. This significant performance gain was consistently observed across various quality fingerprint images. The iterative nature

50 2.5. FINGERPRINT MATCHING 41 of ICP algorithm makes the approach unsuitable for automated matching. Also, the Level 3 matching (ICP algorithm) relies on an initial alignment based on Level 2 features and in case of distorted fingerprints such an alignment is not reliable. In [70] Vatsa et al. have presented a score-level fusion technique that combines level-2 and level-3 match scores to provide high accuracy. The match scores obtained from Level 2 and Level 3 classifiers are first augmented with a quality score that is quantitatively determined by applying redundant discrete wavelet transform to a fingerprint image. The quality augmented match scores are then fused using Dezert- Smarandache theory. The proposed fusion method provided better results than other existing fusion techniques. The proposed algorithm is able to perform well even in the presence of imprecise, inconsistent, and incomplete fingerprint information.

51 42 Chapter 3 PROPOSED HIERARCHICAL MATCHER Fingerprint Matching is the most important stage in fingerprint recognition process. A fingerprint matching algorithm compares two sets of features originating from two fingerprints and determines whether or not they represent the same finger. Fingerprint matching is an extremely difficult problem, mainly due to the large intra-class variations that exists in different impressions of the same finger. The main factors responsible for these intra-class variations are [35]: i) Displacement and Rotation : The same finger may be placed at different locations and at different orientation on the sensor during different acquisitions resulting in a (global) translation and rotation of the fingerprint area, ii) Partial overlap : Finger displacement and rotation often cause part of the fingerprint area to fall outside the sensor s field of view, resulting in a smaller overlap between different impressions of the same finger, iii) Non-linear distortion : The act of sensing maps the three-dimensional shape of a finger onto the two-dimensional surface of the sensor. This mapping results in a non-linear distortion

52 43 in successive acquisitions of the same finger due to finger skin plasticity, iv) Pressure and skin condition: The ridge structure of a finger would be accurately captured if ridges of the part of the finger being imaged were in uniform contact with sensor surface. However, finger pressure, dryness of the skin, skin disease, sweat, dirt, grease, and humidity in the air all confound the situation, resulting in a non-uniform contact. As a result, the acquired fingerprint images are very noisy, and v) Feature extraction errors : The feature extraction algorithms are imperfect and often introduce measurement errors. For example, in low-quality fingerprint images, the minutiae extraction process may introduce a large number of spurious minutiae and may not be able to detect all the true minutiae. Chapter 2 has provided an overview of different fingerprint matching approaches. The correlation based method requires the complete image to be stored (large template sizes). The texture based methods are less accurate than minutiae based matchers since most regions in the fingerprint carry low textural content. Both types of methods requires accurate alignment of fingerprints. The minutia based techniques on the other hand are more accurate and they very closely resemble the manual approach as used by forensic experts. Studies [31], [70, 67, 29, 33] have shown that by combining additional information in the form of texture features, level 3 features with minutia based matcher, higher accuracy can be achieved. The minutia based approaches like other approaches cannot give a high confidence match when the images are of poor quality or when there is a very small overlap i.e. very few minutia points are available for matching. Level 3 features are known to carry discriminative information and forensic examiners often make use of Level 3 features when insufficient minutia points are present. A new hierarchical matching system which utilizes additional information in form of Level 3 features (pores and ridge contours) has

53 3.1. PREPROCESSING AND ENHANCEMENT 44 been proposed in this chapter. Figure (3.1) illustrates the architectural design of the proposed hierarchical system. A similar architecture (hierarchical) was first proposed by Jain et al. [29]. Figure 3.1: The proposed hierarchical matcher. 3.1 Preprocessing and Enhancement To compensate for the variations in lighting, contrast and other inconsistencies three preprocessing steps are used: Gaussian blur, sliding window contrast adjustment, and histogram based intensity level correction. Gaussian blurring is used to remove any noise introduced by the sensor. The lighting inconsistencies are adjusted by using sliding-window contrast adjustment on the Gaussian blurred image.

54 3.2. FEATURE EXTRACTION 45 To further enhance the ridges and valley a final intensity correction is made by using Histogram-based Intensity Level Adjustment. These techniques are described in Chapter 2. The preprocessed image is enhanced using a popular fingerprint enhancement technique by Sharat et al.[16] which uses contextual filtering in fourier domain. The technique is discussed in Chapter 2. The enhanced fingerprint image is not suitable for extracting pores as the pore information is lost during enhancement. Thus for pore extraction the preprocessed image is used. 3.2 Feature Extraction The minutiae points are extracted from the enhanced image by using NIST s 1 minutia extraction software (NFIS). The method first generates the image quality maps by checking the regions with high curvature, low flow and low contrast. And then, a binary representation of the fingerprint is constructed. Minutiae are generated by comparing each pixel neighborhood with a family of minutiae templates. Finally, spurious minutiae are removed by using a set of heuristic rules. The NFIS also counts the neighbor ridges and assigns each minutia point a quality (in the range 0 to 100) determined from image quality maps. The minutia representation generated by NFIS consists of the location information, orientation, minutia type (bifurcation or ending) and minutia quality. The proposed minutia matcher does not differentiate between different minutiae types. This is because the minutia types are difficult to distinguish when the applied finger pressure during acquisition varies. Figure 3.2 shows one such example where the same minutiae extracted from two different impressions appears 1 NIST stands for National Institute of Standards and Technology and is a federal technology agency that develops and promotes measurement, standards, and technology.

55 3.3. HIERARCHICAL MATCHING 46 as a bifurcation in one image and as a ridge ending in other. Figure 3.2: Effect of differential finger pressure([36]). The Level 3 features are extracted only when the minutiae based matching fails to match the query fingerprint image with the template image. The ridge contours are extracted from the enhanced fingerprint image by using the approach proposed by Jain et al. [29]. The extracted ridge contours by using this approach is shown in Figure 3.3. For extracting pores the technique proposed in [23] by International Biometric Group is used. Figure 3.2 show the extracted pores by using this approach. The pore information thus extracted contains the location information and orientation information of pores. The pore extraction and ridge contour extraction techniques are described in Chapter Hierarchical Matching Dealing with Non-Linear distortion is the major problem faced by fingerprint matching algorithms. The effect of non-linear distortion is more pronounced when global structures with global parameters are considered for matching. Localized matching can minimize the effect of non-linear distortion. Throughout our approach we have used localized matching in-order to tolerate the effects of non-linear distortion. Also we have used rotation invariant structures (in case of pores and minutiae)

56 3.3. HIERARCHICAL MATCHING 47 (a) Sample 500 dpi fingerprint (b) enhanced fingerprint (c) Traced Ridge Contours Figure 3.3: Ridge Contour extraction process in a image scanned with Cross Match Verifier 300 scanner at 500 dpi. (b) extracted pore (a) Sample 500 dpi fingerprint Figure 3.4: Pores extracted from a image scanned with Cross Match Verifier 300 scanner at 500 dpi.

57 3.3. HIERARCHICAL MATCHING 48 and features (in case of ridge contours) for matching, thus at any stage any type of alignment is not required. Let T and Q be the template and query images which are to be matched. At first the minutia-based matcher matches T and Q and returns the matching minutia pairs, which is a measure of degree of similarity between the two representations (fingerprints). Let S 2 be the number of matched minutia pairs returned by the minutia-based matcher. If S 2 > τ 2 (τ 2 is a threshold whose value is chosen as 12), the two fingerprints are considered matched and the matching terminates. The threshold τ 2 is set to be 12 because typically, a match consisting of 12 minutia points (the 12-point guidelines) is considered as sufficient evidence for making positive identification in many courts of law [56]. However, If S 2 τ 2 then the matching continues and Level 3 based matcher further matches the two fingerprints and returns the final match/nonmatch decision. The matched minutiae at Level 2 are further examined and the Level 3 features in their neighborhood are matched Minutia-based Matcher We propose a local structure based minutia matcher. The local structure ( star ) is similar to the kplet used by Sharat et al [15]. It consists of a central minutia point m i and its k neighboring minutia points (m i1, m i2,...m ik ) within a local neighborhood. The local neighborhood is defined by a region within distance d max from the central minutia m i. The k neighbors are chosen amongst all the minutiae within d m ax distance and from all four quadrants. Choosing neighboring minutia from all four quadrants captures local structure information accurately and during experiments we found that it is more effective than choosing k nearest neighbors of the central minutia. Figure 3.5 demonstrates a star with k = 6 neighbors. There exists a

58 3.3. HIERARCHICAL MATCHING 49 trade-off while choosing the size of local neighborhood (d max ). d max should be small enough to tolerate distortion, however keeping d max too small might not accumulate enough evidence and hence may lead to false matches. k represents the number of neighboring minutia points which forms the star and its value can vary between a predefined range bounded by k min and k max i.e. k min k k max. If for a given minutia k < k min then that minutia is not considered for matching. On the other hand, if k > k max then k max neighbors are considered. The matching is performed in two stages: Figure 3.5: The schematic description of a star with k = 6 neighbors. The neighbors are chosen from all the four quadrants 1. Local matching : the stars corresponding to central minutia points are matched. At the end of this stage pairs of matching minutia points are returned.

59 3.3. HIERARCHICAL MATCHING Consolidation step : During this step the matched minutia pairs are further checked for consistency at global level Local Structure Matching During this stage the stars in Q are matched with the stars in T. If two stars are found to be matching then the minutiae representing their centers are also considered to be matching. A star associated with a central minutia m and with k neighboring minutiae is represented by a k element feature vector v m. Each element v mi v m is a 3-tuple (r i, θ i, φ i ) representing the radial parameters of i th neighboring minutia point with respect to the central minutia m. v m = {(r 1, θ 1, φ 1 ),..., (r k, θ k, φ k )} where r i is the distance of i th neighboring minutia from m, θ i is the orientation of i th neighboring minutia with respect to m s orientation and φ i is the direction difference between m s orientation and the direction of the edge connecting m and m i. Throughout this chapter we shall refer to a star by its feature vector. The star structure is invariant to rotation and translation and thus any type of alignment is not required. The local structure matching problem thus reduces to matching two ordered sequences v m = (v m1,...v mk ) = {(r 1, θ 1, φ 1 ),.., (r k, θ k, φ k )} and v m = (v m 1,...v m k ) = {(r 1, θ 1, φ 1),.., (r k, θ k, φ k)} where v m represents a star in T and v m represents a star in Q. The edit distance is used as a similarity measure between two stars. Edit Distance between two sequences indicates a measure of similarity between them. The

60 3.3. HIERARCHICAL MATCHING 51 edit distance is capable of capturing the impression variations (deletion of genuine minutiae, insertion of spurious minutiae, and perturbation of minutia) associated with different impressions of same finger. The algorithm considers a query minutiae (in the query star ) and a template minutiae (in the template star ) to be a mismatch if their attributes (radius, radial angle and minutia direction with respect to the central minutiae) are not within a tolerance window. A penalty is associated with every match and mismatch (Equation 3.2). The penalty associated with a match is proportional to the disparity in the values of the attributes related to the two matched minutiae. On the other hand, a predefined huge penalty is associated with every mismatch. The sum of all penalties associated with match and mismatch between two sequences defines the edit distance. Among several possible sets of transformation of the query sequence into the template sequence, the string matching algorithm chooses the transform associated with the minimum cost (edit distance) based on dynamic programming. An unmatched star, v m Q is matched with all the unmatched stars in T and the star which returns the minimum edit distance is chosen as the best match. Given two stars, v m = (v m1,...v mk ) T and v m = (v m 1,...v m k ) Q, of lengths k and k, respectively, the edit distance, D(k, k ), is recursively defined with the following equations : 0 if i = 0, or j = 0 D(i 1, j) + Ω D(i, j) = min D(i, j 1) + Ω 0 < i k, and 0 < j k D(i 1, j 1) + w(i, j) (3.1)

61 3.3. HIERARCHICAL MATCHING 52 α r + β θ + γ φ w(i, j) = Ω if r < t r, θ < t θ and φ < t φ otherwise (3.2) where, r = ( r i r j /min(r i, r j)), θ = θ i θ j and φ = φ i φ j ; α, β and γ are weights associated with each component, respectively; t r, t θ and t φ are the parameters of the tolerance window ; and Ω is a pre-specified penalty for mismatch. Once the best match for v m is found, say v n = (v n1,...v nl ), then a dynamic programming based sequence alignment technique ([53]) is used to find the number of matching edges between v m and v n. Sequence alignment tries to find the best alignment between an entire sequence S1 and another entire sequence S2. Consider two sequences S1 = [GAAT T CAGT T A] and S2 = [GGAT CGA]. If required gaps( ) are inserted in S1 and T 1 and finally the aligned strings have same length. A penalty is associated with each gap and mismatch and a reward is associated with each match in the final alignment. Based on these rewards and penalty a final alignment cost is calculated. The optimal global alignment is one having the maximum cost. S1 = GAAT T CAGT T A S2 = GGA T C G A There are two phases in the alignment process: Forward phase: The optimal alignment cost for every substring is computed (by using a recurrence relation) and stored in C(a (k + 1) (l + 1) dimensional matrix). C[i, j] denotes the best cost for aligning two subsequences v m[1...j] (1 j k ) and v n [1...i] (1 i l).the base conditions are: C[0, 0] = 0, C[0, j] = C[0, j 1] + gap score

62 3.3. HIERARCHICAL MATCHING 53 and C[i, 0] = C[i 1, 0] + gap score The recurrence relation is: C[i 1, j 1] + score(v m[j], v n [i]), C[i, j] = max C[i 1, j] + gap score, C[i, j 1] + gap score match score score(v m[j], v n [i]) = nonmatch score (3.3) if r < t r, θ < t θ and φ < t φ otherwise where r = ( r i r j /min(r i, r j)), θ = θ i θ j and φ = φ i φ j ; match score is the reward when a match is found, nonmatch score and gap score are penalties when mismatch is found and gap is introduced, respectively; t r, t θ and t φ are the parameters of the bounding box. In other words if we have an optimal alignment up to v m [1..j] and v n [1..i] then there are only three possibilities of what could happen next: i) v m[j] and v n [i] match, ii) a gap is introduced in v n, and iii) a gap is introduced in v m. The use of gaps takes care of the cases when there are spurious and missing minutiae. Traceback : This step constructs the optimal alignment by tracing back in the matrix C any path from C[l, k ] to C[0, 0] that maximizes the cost. Let Q map be the Hash Map containing the mapping of minutiae m Q and their corresponding stars v m and T map be the Hash Map containing the mapping of minutiae m T and their corresponding stars v m. Q map = {(m 1, v m1 ),..., (m x, v mx )} T map = {(m 1, v m ),..., (m y, v 1 m )} y

63 3.3. HIERARCHICAL MATCHING 54 where x, y denote the number of minutia points which are considered for matching in Q, T respectively. The matching technique is summarized in algorithm 1. Algorithm 1 Minutia-based Matching Require: Q map and T map 1: let M be the Hash Map containing matched minutia pairs (initially empty). 2: for all (m, v m ) Q map do 3: for all (m, v m ) T map do 4: if m is already matched then 5: continue 6: end if 7: calculate edit distance between v m and v m 8: end for 9: choose v m having minimum edit distance with v m 10: find the number of matching edges (num match ) between v m and v m by using a sequence alignment technique. 11: if num match > min match then 12: Insert (m, m ) in M {min match is a predefined constant} 13: end if 14: end for 15: return M Consolidation step During this step the matched minutiae pairs are validated at global level. The matched minutiae should be consistent in terms of global characteristics like orientation. Thus all valid matches must have similar orientation differences if they come from same finger. We approximate the orientation difference between fingerprints by plotting a histogram with bins representing orientation differences and each bin 36 degree wide. The bin with maximum height and its immediate neighboring bins are considered. The matched minutiae pairs in other bins are removed from the matched list M. Let N 2 be the number of matched pairs in M, the Level 2 match score (S 2 )

64 3.3. HIERARCHICAL MATCHING 55 is calculated using a very famous approach: S 2 = 2 N 2 N Q + N T (3.4) where N Q and N T are the number of minutia points in Q and T, respectively. If N 2 12, the two fingerprints (Q and T ) are considered to be from same finger, otherwise the matching continues to Level Level 3 features based Matcher The matched minutiae pairs at Level 2 are further examined and Level 3 features in their local neighborhood are compared. Thus for a given pair of matched minutiae, we compare the level 3 features in their neighborhood and the minutia correspondence is recomputed based on the agreement of Level 3 features. The ridge contours and pores are separately compared in the neighborhood. While comparing the Level 3 features, we need to consider the fact that the detected features will vary in different impressions of the same finger, due to the degradation of image quality (because of noise, skin deformation etc.). Thus, a decision based OR rule is used to verify the minutiae correspondences, i.e if either of pores or ridge contours agree then the minutia correspondence is verified. Localized matching is done to tolerate the effect of non-linear distortion. In [40] it has been observed that given a sufficiently high resolution, the same quantity of discriminative information could be extracted from fingerprint fragments by using Level 3 data, as could be when considering Level 2 data extracted from the complete fingerprint image. Thus by using localized window sufficient information can be obtained.

65 3.3. HIERARCHICAL MATCHING Pore Matching Each pore is represented by a 3-tuple (x, y, θ), where x,y denote its location and θ is the direction of the ridge, at the location where it lies. [70], [67] and [29] have used only location information of pores for matching. Similar to minutiae, pores maintain the same relative orientation to other pores within a fingerprint between different impressions. In a fingerprint pores are distributed over ridges and associating direction information with pores provides an additional information for matching. We propose a novel approach for matching pores. Pores within a circular region around the matched minutia points, obtained from minutia-based matcher, are used for matching. Given a minutia m as the origin, the pores in a circular region C m around it are arranged in order of radial distance first and then pores with same radial distance are ordered by the increasing radial angle (the angle between the direction of line segment joining a pore with the central minutia and the central minutia s orientation). Suppose (m, m ) is a matching minutia pair, the pore information related to them can be represented by m p, m p respectively: m p = (p 1,..., p i ) = ((r 1, θ 1, φ 1 ),..., (r i, θ i, φ i )) m p = (p 1,..., p j) = ((r 1, θ 1, φ 1),..., (r j, θ j, φ j)) where i,j are the number of pores in C m, C m respectively; r k, θ k, φ k denotes the radial distance, orientation difference, radial angle of k th pore with the central minutia m p ; r k, θ k and φ k have similar relationship with m p. A dynamic programming based approach is used to find the edit distance, d pores between the ordered pore strings associated with two matching minutiae. The same approach is used for finding the edit distance which is used in case of minutiae. The edit distance gives a measure of similarity and is tolerant to spurious and missing

66 3.3. HIERARCHICAL MATCHING 57 pores RidgeContour Matching Ridge contour are observed to be more reliable than pores at low resolution (500 dpi). Figure (3.6) shows two fingerprints at 500 ppi having prominent ridge structure. However, pores are not found to be consistent. We concentrate on localized matching for tolerating non-linear distortion. Thus ridge contours are matched in a w w window around the matched minutiae pairs. Non-linear distortion is a major concern when matching smaller structures, such as points along ridges (ridge contours). Localized matching can only tolerate non-linear distortion to an extent, but to deal with the adverse effects of distortion (caused by skin plasticity) more efforts are needed. Infact, the matching technique should be robust to non-linear distortion. (a) (b) Figure 3.6: Two impressions of same fingerprint at 500 dpi. A novel approach for ridge contour matching is proposed which uses Zernike mo-

67 3.3. HIERARCHICAL MATCHING 58 ments as shape features and Local Binary Patterns (LBP) as texture features. The proposed approach combines the Zernike moments and LBP to form a set of features suitable for texture shape features. Shape Feature extraction using Zernike Moments Zernike moments costitute a powerful shape descriptor in terms of robustness and description capability and have been employed in a wide range of applications like character recognition, object recognition, palm-print recognition, iris recognition and face recognition [71, 8, 48, 17]. This shape descriptor has proved its superiority over other moment functions [12, 43], regarding to its description capability and robustness to noise or deformations. Zernike moments are based on a set of complex polynomials (Zernike polynomials) that form a complete orthogonal set over the interior of the unit circle [72]. The Zernike function of order p and repetition q are defined in the polar coordinate system (r, θ) as V pq (r, θ) = R pq (r).e iqθ (3.5) where i = 1 and R pq is the orthogonal real valued radial polynomial defined as R pq (r) = (p q )/2 n=0 where p q is even and q p ( 1) n (p n)! n!( p+ q n)!( p q n)! rp 2n 2 2 (3.6) Zernike moments of an image are the projections of the image function onto these orthogonal basis functions. The Zernike moments of order p and repetition q for an image intensity function f(r, θ) over the polar coordinate space is: Z pq = p + 1 Π 2Π V pq(r, θ)f(r, θ)rdrdθ (3.7) where V pq denotes the complex conjugate of V pq. If N is the number of pixels along

68 3.3. HIERARCHICAL MATCHING 59 each axis of the image, then Equation (3.7) can be written in the discrete form as p + 1 N N Z pq = V Π(N 1) pq(r, θ)f(x, y) (3.8) 2 x=1 y=1 where r = (x 2 + y 2 )/N, and θ = tan 1 (y/x). The polar form of Zernike moments suggests a square-to-circular image transformation [49], so that the Zernike polynomials need to be computed only once for all pixels mapped to the same circle. This transformation from square to circular region is shown in Figure 3.7. If the image coordinate system (x,y) is defined with the origin at the center of the square pixel grid, then the pixel coordinates of the transformed circular image can be represented by γ, ξ where γ denotes the radius of the circle and ξ the position index of the pixel on the circle. The normalized polar coordinates r, θ of the pixel (γ, ξ) are given by r = 2γ/N, θ = πξ/(4γ) (3.9) Figure 3.7: Schematic of square to circular transformation From Equation 3.7 and 3.5, Zernike moments of a pattern rotated by an angle a around its center of mass are given in polar coordinates as Z a pq = Z pq e iqa (3.10)

69 3.3. HIERARCHICAL MATCHING 60 From Equation 3.10 we have Z pq = Z pq e iqa, thus magnitude of Zernike moments are invariant to rotation. Due to the property of orthogonality the contribution of each moment is unique and independent. The Zernike moments are calculated in a w w window around the matched minutia pairs. The Zernike feature vector with moment order n and repetition m is given by: f Zernike = [ ZM 00, ZM 01,..., ZM (n 1)(m 1) ] (3.11) where ZM mn denotes Zernike moments of order n and repetition m. Texture Feature Extraction using Local Binary Patterns Local Binary Patterns (LBP) [55] is a very simple, yet efficient, multi-resolution approach to gray-scale and rotation invariant texture description and has been extensively used as a state of the art feature extractor in Face Recognition [69]. The basic LBP operator, introduced by Ojala et al. [55], labels the pixels of an image by thresholding the n n (n = 3) neighborhood of each pixel with the center value and considering the result as a binary number. Then the histogram of the labels can be used as a texture descriptor. The basic LBP operator has been illustrated in Figure 3.8. Rotation invariance can be achieved by selecting the minimum possible value of the binary pattern as the label. Figure 3.8: The basic LBP operator Given an image f(x,y), a histogram of the labeled image f l (x, y) can be defined as

70 3.3. HIERARCHICAL MATCHING 61 H i = x,y I{f l (x, y) = i}, i = 0,..., n 1 (3.12) in which n is the number of different labels produced by the LBP operator and 1 A is true, I{A} = (3.13) 0 A is false. This histogram contains information about the distribution of the local micropatterns, such as edges, spots and flat areas, over the whole image. The rotation invariant LBP histogram, f LBP is calculated from a w w symmetric (square) local region around both the matched minutia pairs. Fusion of LBP-Zernike features The LBP-Zernike feature vector is obtained by combining the LBP and Zernike features vectors as follows: Normalize the LBP feature and Zernike Feature respectively by z score normalization. LBP-Zernike feature is given by: f LBP Zernike = {f LBP } U {f zernike } (3.14) Finally the similarity between two minutia points m, m is the Euclidean distance (d ridges ) between their respective f LBP Zernike features Fusion of Level 2 and Level 3 features The distance measure obtained by matching pores and ridges in a local neighborhood around matched minutia pairs are used to reverify the minutia correspondence. Two heuristically determined thresholds τ pores and τ ridges are used for deciding

71 3.3. HIERARCHICAL MATCHING 62 whether the pores and ridge contours, respectively, from two localized regions, agree or not. Thus, a minutia pair m, m matched at Level 2, is considered matched at Level 3, if either of the following two conditions hold. d ridges < τ ridges d pores < τ pores Thus at Level 3 minutia correspondences are recomputed and the number of matched minutiae N 3 at level 3 is then used to calculate Level 3 match score S 3 = 2 N 3 N Q + N T (3.15) where N Q and N T are the number of minutia points in Q and T, respectively. If S 3 > τ 3 (τ 3 is heuristically determined threshold) then the fingerprints are considered to be matched other wise they are declared as non-matched fingerprints.

72 63 Chapter 4 EXPERIMENTAL RESULTS This chapter describes the various experiments conducted and discusses the obtained results. The proposed approach has been evaluated on three different databases. 4.1 Database The evaluations and testing of the proposed approach has been done on three diverse fingerprint databases: Neurotechnology Database [3], FVC2004 [5] DB3 Database and FVC2006 [1] DB2 Database. They are explained in detail below. Neurotechnology Database: The database consists of 51 different fingers with 8 impressions per finger resulting in 408 images. The fingerprint samples are scanned with an optical scanner (Cross Match Verifier 300) at 500 dpi. The sample images from this database are shown in Figure 4.1. FVC 2004 DB3 Database: The database consists of 100 different fingers with 8 impressions per finger resulting in 800 images. The fingerprints are scanned with a thermal sweeping sensor (FingerChip FCD4B14CB by Atmel) at 512 dpi.

73 4.1. DATABASE 64 Figure 4.1: Sample images from Neurotechnology Database The FVC 2004 databases are known to be difficult because of the perturbations which were deliberately introduced during database collection [5]. The sample images from this database are shown in Figure 4.2. Figure 4.2: Sample images from FVC 2004, DB3 Database FVC 2006 DB2 Database: The database is 140 fingers wide and 12 samples per

74 4.2. PERFORMANCE EVALUATION 65 finger in depth (total 1680 fingerprint). The fingerprints were scanned with an optical sensor at 569 dpi. A heterogeneous population which includes manual workers and elderly people was used to create the database [1]. The volunteers were simply asked to put their fingers naturally on the acquisition device, but no constraints were enforced to guarantee a minimum quality in the acquired images. Figure 4.3 displays the sample images from this database. Figure 4.3: Sample images from FVC 2006, DB2 Database 4.2 Performance Evaluation Each sample in a Database is matched against the remaining samples of the same finger to compute the False Rejection Rate(FRR). The FRR is the fraction of