Enhancing Optical Cross-Sensor Fingerprint Matching Using Local Textural Features

Transcription

1 Enhancing Optical Cross-Sensor Fingerprint Matching Using Local Textural Features Emanuela Marasco Center for Secure Information Systems George Mason University Fairfax, VA USA Alexander Feldman Computer Science Brandeis University Waltham, MA USA Keleigh Rachel Romine Computer Science Winthrop University Rock Hill, SC USA Abstract Fingerprint systems have been designed to typically operate on images acquired using the same sensor. Existing fingerprint systems are not able to accurately compare images collected using different sensors. In this paper, we propose a learning-based scheme for enhancing interoperability between optical fingerprint sensors by compensating the output of a traditional commercial matcher. Specifically, cross-sensor differences are captured by incorporating Local Binary Patterns (LBP) and Local Phase Quantization (LPQ), while dimensionality reduction is performed by using Reconstruction Independent Component Analysis (RICA). The evaluation is carried out on rolled fingerprints pertaining to 494 users collected at West Virginia University and acquired using multiple optical sensors and Ten Print cards. In cross-sensor at False Acceptance Rate of 0.01%, the proposed approach achieves a False Rejection Rate of 4.12%. Figure 1. Rolled fingerprint images obtained by the same finger but using four different optical sensors among those in the integrated Automated Fingerprint Identification System (IAFIS) certified product list provided by the Federal Bureau of Investigation (FBI). 1. Introduction As people become more electronically connected, the need for accurate automatic human identification in critical applications such as fraud prevention increases [1]. However, the operating efficiency of biometric recognition systems is still a challenge. Although features extracted from raw data are expected to be an invariant representation of a person, the sensor, the environment, improper userinteraction or changes of the biometric trait over time can compromise the required invariance [2]. The sensor output depends on its technical characteristics; subsequently, organizations which deploy biometric technologies might need to re-enroll all the subjects if the product they are currently using will not be provided in the future. Fingerprints are the most recognized biometric. For automatic fingerprint recognition systems deployed in several applications, the introduction of a different sensor may render the stored data unusable. Different sensors even if belonging the same technology produce different raw data 1. Variations are due to different arrangements of sensing elements, ergonomics of the acquisition system such as alignment and positioning which affect image quality, i.e., the clarity of the ridge pattern. Additionally, different manufacturers usually induce physical differences between units (i.e., lenses) [3]. Fig. 1 shows fingerprint images pertaining to the same finger acquired using different optical devices. Interoperability is the ability of a system to handle variations in the data due to device diversity which may negatively affect matching [4]. Fingerprint sensor interoperability has been addressed mainly as a problem pertaining to matching across different sensing technologies. Further- 1

2 more, existing approaches model general distortions and are not able to meet high security requirements of real-world applications. In this paper, we extend the preliminary studies presented in [5] and [6] by exploiting textural features which complement the traditional match score. Specifically, in order to isolate variations in the image due to device diversity, we extract Local Binary Patterns (LBP) which provides a gray-scale invariant texture representation and invariance against gray level changes. Additionally, we extract Local Phase Quantization (LPQ) which quantizes phase information of the local Fourier Transform to construct a local blur invariant representation. Dimensionality reduction is performed by using Reconstruction Independent Component Analysis (RICA). The defined features are combined with the match score to train a classifier. The proposed model is able to reduce the complexity of the classification algorithm compared to existing methods. The rest of the paper is organized as follows. In Section 2, we describe the state-of-the-art about fingerprint interoperability accommodation strategies. Evaluation is carried out on data pertaining to 494 users, collected at West Virginia University using four optical-based fingerprint sensors as well as ink rolled prints. Section 3 presents the proposed approach. Section 4 discusses experiments and results. Section 5 draws conclusions and future research directions. 2. Related Works Previous studies related to fingerprint sensor interoperability mainly pertain to matching involving different sensing technologies. In 2004, Jain and Ross carried out experiments using optical and capacitive sensors with same resolution 500 dpi, but different scanning areas [7]. The intra-sensor Equal Error Rate (EER) was 6.14% on data collected using the optical device and 10.39% using the capacitive one. Matching was more accurate at a larger scanning area. The reported cross-sensor EER is 23.13%. In 2006, Ross and Nadgir modeled relative distortions due to sensing diversity through Thin-Plate Splines [8]. Translation parameters were determined by aligning centroids of the two images, while rotation was estimated using the orthogonal matrix and the singular value decomposition (SVD). The deformation model is used to distort minutiae of one sensor before matching. The reduction in cross-sensor error rate was significant but still not close enough to realistic scenarios. Additionally, this method is not completely automated given that control points (i.e., minutiae points) are manually selected. In 2007, Alonso-Fernandez et al. performed an image quality assessment involving different sensing technologies (i.e., capacitive and optical) [9]. They demonstrated that quality measures based on grey level features are the most discriminative ones for optical sensors than capacitive sensors. In 2009, Modi et al. carried out a statistical analysis to examine sensor dependent variations in minutiae count, image quality and performance due to technology diversity (i.e, optical and capacitive) [10]. The best image quality was provided by the optical touch device while the lowest interoperability was achieved when matching optical touch versus swipe. Kukula et al. investigated the impact of force on error rates, minutiae count and image quality in both optical and capacitive sensors [11]. They showed that as the amount of force applied to the optical sensor surface increases, image quality becomes better. In 2010, Poh et al. designed a Bayesian Belief Network (BBN) to model the relationship between image quality, match score and device. The BBN estimates the posterior probability p(d q) of the device d given quality measures q [12]. Quality estimates are clustered and each cluster is associated to one of the device. This approach does not explicitly model the influence of the device. In [13], Xie et al. studied inter-sensor image quality assessment. In particular, they found a set of features applicable to various types of sensors. They are orientation certainty, consistency measure and local orientation quality. Experiments were carried out on optical, capacitive and thermal sensors. By combining the three defined features the accuracy is above 95% for any of the three sensors. In 2013, Lugini et al. statistically analyzed how match scores change across different optical devices [14]. The Kendall s rank correlation test pointed out non-symmetric differences in genuine scores between sensor pairs. In [5] Marasco et al. proposed a learning-based approach to enhance matching across different fingerprint optical devices. They exploited quality and intensity-based characteristics which were concatenated with the match score to train a pattern classifier. In [6] Marasco et al. considered Binarized Statistical Image Features (BSIF) and characteristics derived from the Discrete Wavelet Transform (DWT). Experiments are carried out on a data set consisting of fingerprints obtained from 494 users acquired using four different optical devices. Results show a significant reduction in error rates compared to the baseline as well as improved performance compared to previous research. 3. The Proposed Approach Given two fingerprints, the proposed system verifies if they pertain to the same identity by taking into account possible device diversity. The learning program uses multiple representations of the identity, i.e., features of different nature are combined through a classifier [15]. The architecture of the proposed scheme is illustrated in Fig. 2 and steps involved by the process are summarized below. 1. Two fingerprint images are selected from a training set collected using different devices. 2. A pre-defined set of interoperability features is ex-

3 Figure 2. Interoperability enhancement is performed in parallel to a typical biometric matching operation. Features are concatenated with the match score to train a pattern classifier which outputs a verification decision in cross-device domain. tracted from both images. 3. Features are fused with the typical match score. 4. Independent Components are extracted to reduce feature dimensionality. 5. Selected features are used for training a classifier. 6. In operational mode, a fingerprint probe is acquired using any of the available devices. 7. The probe is matched against the corresponding labeled gallery data and a match score is generated. For gallery and probe interoperability features are extracted and concatenated with the match score. 8. The feature vector generated in the previous step is used as input of the classifier to predict the crossdevice matching outcome The Exploited Features Device diversity impacts image quality, matching and textural properties of fingerprint images. We design a set of features in order to captures these variations and use them to build a better representation of the identity in cross-sensor scenarios. Selected features compose the input vector of a pattern classifier during training. Matching Features. Most fingerprint matchers use minutiae points for recognition. A minutiae point (i.e., ridge ending or ridge bifurcation) is represented as a triplet m = [x,y,θ] that indicates minutiae location co-ordinates and angle. Two fingerprints originated by the same finger may differ depending upon the placement on the sensor. This is addressed by the Alignment process which geometrically transforms two sets of minutiae points to the same coordinate system [16]. For intra-device alignment, a rigid transformation is generally sufficient while in cross-device acquisition, alignment may deal with non-linear deformations. Methods which can be used to align two fingerprints are: Generalized Hough Transform, local descriptors, energy minimization, etc. In this work we use the Hough Transform algorithm which takes in input two sets of minutiae m g and m p extracted from the gallery and probe images, respectively. Transformation parameters are computed according to the algorithm used in [5]. They capture differences pertaining to positioning then they used as interoperability features. After alignment, minutiae are paired based on predefined distance and angle thresholds. Match Score indicates the degree of fingerprint similarity computed as a function of the corresponding minutiae number. Image Quality Features. Image Quality indicates usefulness degree of a biometric sample for automated matching [17]. Quality measures can significantly vary across different sensors. The optical sensor technology is based on light reflection properties which directly impact grey level values. Subsequently, quality measures which deal with grey-level characteristics are sensitive their variability [13]. A fingerprint image of high quality usually presents clear ridges and easy to locate most of the minutiae [13, 4]. A minutiae-based matcher might not be accurate if only a few minutiae points can be detected. Minutiae Count is the number of minutiae extracted from a fingerprint image and it represents and indicator of quality. It may vary based on human-sensor interaction where optics design such as the physical layout of fingerprint sensors plays a relevant role [11]. The standard de facto for fingerprint image quality assessment is the tool NIST Fingerprint Image Quality (NFIQ) 2. The NFIQ estimate is an integer value ranging between 1 and 5, where 1 is assigned to the best image quality. Image Texture Features. Texture is characterized by repeating patterns of local variations in image intensity. Discriminative features from local regions are usually extracted to facilitate classification of such patterns. We seek for a texture descriptor which is invariant to monotonic transformations of grey-levels and robust to rotated textures, since we wish to recognize textures with changes due to the acquisition process (noise, scale,..). Fig. 3 shows four fingerprints and an enlarged patch in each. All fingerprints in a row are from the same subject and those within a column are from the same device. Local texture is robust to changes in device while highly sensitive to differences between subjects. The Local Binary Patterns (LBP) operator is known 2

4 Figure 3. Examples of Fingerprint images pertaining to the two subjects acquired using two different optical sensors. Figure 4. LBP applied to a fingerprint image. to offer these properties. LBP 3 is able to efficiently characterize the spatial configuration of local image texture by detecting spatial structures, referred to as local binary texture patterns. Texture is defined as the joint distribution of gray values of a circularly symmetric neighbor set of P image pixels on a circle of radius R (see Eqn. (1)), T = t(g c, g 0,..., g P 1 ) (1) where g c is the gray value of the center pixel of the local neighborhood and g 0,..., g P 1 are the gray values of P equally spaced pixels on the considered circular symmetric neighbor set. The gray value of the center pixel of the circularly symmetric neighbor set is subtracted from the gray values of the circularly symmetric neighborhood. Features (e.g., statistics) are extracted directly from LBP histograms, obtained as described in Eqn. (2) [18] [19], LBP = s(g p g c )2 p (2) p=0,...,p 1 where s is defined as: s(x) = 1 if x 0, else s(x) = 0. Fingerprints present different orientations, and the rotation invariant operator Local Phase Quantization (LPQ) 4 can point out differences in the phase spectrum of the image [20]. In particular, the approach measures differences in terms of information loss at high frequencies. A local Discrete Fourier Transform (DFT) is applied to the entire image by considering image patches in order to analyze the frequency content locally. The local FT G s (u, v), with s being the patch index, is obtained by applying a linear filtering technique based on a weighted sum of the frequency components of the image patch [20]. For every pixel of the image, I(x, y), the phase of the local DTF is computed in a M M neighborhood. Data reduction is based on the quantizer defined in Eqn. (3), 1, if g j 0 q j = (3) 0, otherwise where g j is the j th component of G. The quantizer results in a 2-bit integer representation for a single frequency component at every point x. The L quantized coefficients are presented as histograms according to the following binary coding: L b = q j 2 j 1, (4) j=1 which ranges between 0 and 2 L 1 and describes the local texture at location x. Fig. 5 illustrates the process explained above. Image Gradient measures the rate of change in grey levels throughout the image. The direction of the gradient informs about the direction of the maximum change, while its magnitude quantifies the amount of such change: [ ] Gx I = (5) G y where G x corresponds to I x, the differences in x (horizontal) direction. G y corresponds to I y, the differences in y (vertical) direction. Computing the gradient results in two matrices, a matrix for G x and another for G y, each with the same size as the original image. The magnitude of the image gradient is given by I = [G 2 x + G 2 x] 1/2 (6) The average of the gradient magnitude is used in the final model. 4. Experimental Results 4.1. Data set The data set used in this study consists of fingerprints pertaining to 494 users. They were collected using four livescan devices (D0-D3) and ink-based ten-print cards (D4), as illustrated in Table 1. For each live-scan device, users sequentially provided two sets of fingerprints, each made up of: rolled individual fingers on both hands, left slap, right slap, and thumbs slap. No quality check was applied during acquisition. Match scores were generated for right index fingers only, using the commercial matcher Identix Bio- Engine Software Development Kit. Impostor match scores were generated by dividing users in groups of 100 and

5 Figure 5. LPQ applied to a fingerprint image. Table 1. Characteristics of the Live-scan devices used for the fingerprint acquisition carried out in this study. Manufacturer Model Resolution (dpi) Image size (pixels) Capture area (mm) D0 Cross Match Guardian R x x 76 D1 i3 digid Mini x x 76 D2 L1 Identity Solutions TouchPrint x x 76 D3 Cross Match Seek II x x 38.1 D4 Ten Print Scans x matching the fingerprints within the same group. For this study we limited the matching to the point fingers from the right hand only Evaluation Procedure Experiments were carried out by extracting a reduced set of features initially designed in [5] and [6]. This set includes match scores, image quality, alignment parameters and difference in image gradient, referred to as (MQAG). Then, we incorporate features related to local descriptors LBP and LPQ by adding values obtained per each image to the set described above. We perform dimensionality reduction using Reconstruction Independent Component Analysis (RICA) [21]. Generally, it is difficult to scale ICA to overcomplete and high dimensional data. By replacing the orthonormalization constraint in ICA s projected gradient descent with a linear reconstruction penalty, RICA is able to use the L-BFGS optimizer instead. L-BFGS performs better on overcomplete data than projected gradient descent. This modification also makes RICA robust to approximate whitening which is desirable when examining high dimensional data. In order to maximize RICA s efficiency at retaining the local texture information of a fingerprint as it reduces the dimensionality of the data, the MQAG features were also considered. After the RICA transformation, our local texture information was reduced to 10 dimensions. We separately trained three distinct instances of the Random Forest classifier, one with the MQAG features, one with the MQAG and the reduced LBP features, and one with the MQAG and the reduced LPQ features. We randomly partitioned the data in 25% for training, 25% for validation and 50% for testing. In all the partitions, subjects are kept mutually exclusive. MATLAB R2017a was used for computation Results Our experimental results are shown in Table 2. When incorporating Local Phase Quantization, we achieved a false rejection rate (FRR) of 4.12% while restricting the false acceptance rate (FAR) to 0.01% which improves the baseline FRR by 1.67% and previous work in [6] by 0.12%. In addition, we found that separating MQAG from the other features in [6] improved the performance of that model. We achieved modestly improved performance while employing fewer features and fewer trees. LBP was not as effective in this application as LPQ, although it achieved results comparable to MQAG even when operating with only 20 trees. Fusion of LBP and LPQ was not as accurate as LPQ alone. Gradient difference is an informative predictor for fingerprint matching. Fig. 6 shows the distribution of gradient difference by device. Fig. 7 shows the distribution of gradient difference for matches and non-matches for all the possible device combinations. The median value for matches is 0.05 while the non-match median is There is less difference between devices when examining gradient in both intra-device and cross-device pairings. For all devices the cross-device median value is approximately 0.1 higher than the intra-device value, but all device medians lie between 0.10 and Fig. 8 shows an example of uniform LBP histograms of two fingerprint images pertaining to the same subject but acquired with two different devices. We depict the verification performance of the fingerprint recognition system in a Detection Error Tradeoff (DET) curve, shown in Fig. 9. DET curves are obtained for both intra-device and cross-device matching including all device pairs. While the most important results for high security

6 Normalized Gradient Difference Guardian x Guardian Guardian x Other digid x digid digid x Other TouchPrint x TouchPrint Gradient Difference by Device TouchPrint x Other Seek x Seek Seek x Other Figure 6. Differences in image gradient per device. Tenprint x Tenprint Tenprint x Other Gradient Difference Between Fingerprints 5.0% Detection Error Tradeoff (DET) Curves % 4.0% 3.5% Baseline (Only Match Score) BSIF+DWT (Biosig 2017) MQAG RICA (LBP) RICA (LPQ) False Rejection Rate 3.0% 2.5% 2.0% 1.5% 1.0% Match Non-match Figure 7. Differences in image gradient related to all the possible combinations of the devices considered in this study. 0.5% 0% 0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10% False Acceptance Rate Figure 9. DET curves which compare the proposed approach to the existing ones. lower error rates at most operating points. The proposed method which incorporates LPQ outperforms all others at most operating points which indicates that local texture may be a discriminating feature for less secure authentication as well. 5. Conclusions Figure 8. LBP histograms of two fingerprint images pertaining to the same subject but acquired with two different devices. applications are those at a very low false acceptance rates (0.01%), the DET curve shows the verification performance of a classifier at a wide range of operating points. At all operating points, learning based models greatly outperform match score alone. The MQAG model which uses fewer features and fewer trees than the BSIF+DWT model has Inspired by the urgent need for enhancing interoperability between optical fingerprint sensors, we defined a set of features able to capture device-specific characteristics. In addition to existing textural features applied in previous work, we extract Local Binary Patterns (LBP) which provides a gray-scale invariant texture representation and Local Phase Quantization (LPQ) which constructs a local blur invariant representation. We combined them with the match score output by a commercial matcher in order to render cross-device matching more accurate. Future investigation will regard scalability of proposed approach, i.e., when involving tens or hundreds fingerprint sensors, and possibly millions of users.

7 Table 2. Average and standard deviation of FRR % related to 10 runs are reported at FAR=0.01%. The classification algorithm is Random Forest. Feature Set Num. of Trees Avg FRR% Std FRR % Baseline (only Match Scores) MQAG MQAG + BSIF + DWT [6] RICA(MQAG + LBP) RICA(MQAG + LPQ) Acknowledgements Feldman and Romine contributed to this work during Summer 2017 funded by the Research Experiences for Undergraduates (REU) NSF Program at the University of North Carolina Charlotte (UNCC) mentored by Marasco. References [1] J. Wayman, A. Jain, D. Maltoni, and D. Maio, An Introduction to Biometric Authentication Systems, Biometric Systems, pp. 1 20, [2] F. Alonso-Fernandez, R. Veldhuis, A. Bazen, J. Fiérrez- Aguilar, and J. Ortega-Garcia, Sensor Interoperability and Fusion in Fingerprint Verification: A Case Study using Minutiae-and Ridge-based Matchers, IEEE International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1 6, [3] D. Maltoni, D. Maio, A. Jain, and S. Prabhakar, Handbook of Fingerprint Recognition. Springer, [4] A. Jain and A. Kumar, Biometrics of Next Generation: An Overview, Second Generation Biometrics, [5] E. Marasco, L. Lugini, and B. Cukic, Minimizing the Impact of Low Interoperability between Optical Fingerprint Sensors, Biometrics: Theory, Applications and Systems (BTAS), pp. 1 8, [6] E. Marasco, Z. Chapman, and B. Cukic, Improving Fingerprint Interoperability by Integrating Wavelet Entropy and Binarized Statistical Image Features, The 15th International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1 12, [7] A. Ross and A. Jain, Biometric Sensor Interoperability: A Case Study in Fingerprints, International ECCV Workshop on Biometric Authentication, pp , [8] A. Ross and R. Nadgir, A Calibration Model for Fingerprint Sensor Interoperability, SPIE, vol. 6202, [9] F. Alonso-Fernandez, F. Roli, G. Marcialis, J. Fierrez, and J. Ortega-Garcia, Comparison of Fingerprint Quality Measures using an Optical and a Capacitive Sensor, IEEE Biometrics: Theory, Applications, and Systems (BTAS), pp. 1 6, [10] S. Modi, S. Elliott, and H. Kim, Statistical Analysis of Fingerprint Sensor Interoperability Performance, IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems (BTAS), pp. 1 6, [11] E. Kukula, C. Blomeke, S. Modi, and S. Elliott, Effect of Human-Biometric Sensor Interaction on Fingerprint Matching Performance, Image Quality and Minutiae Count, International Journal of Computer Applications in Technology, vol. 34, no. 4, pp , [12] N. Poh, J. Kittler, and T. Bourlai, Quality-based Score Normalization with Device Qualitative Information for Multimodal Biometric Fusion, IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 40, no. 3, pp , [13] S. Xie, S. Yoon, J. Shin, and D. Park, Effective Fingerprint Quality Estimation for Diverse Capture Sensors, Sensors, vol. 10, no. 9, pp , [14] L. Lugini, E. Marasco, B. Cukic, and I. Gashi, Interoperability in Fingerprint Recognition: a Large-Scale Study, Workshop on Reliability and Security Data Analysis (RSDA), Budapest, pp. 1 6, June [15] T. Mitchell, The Discipline of Machine Learning. Carnegie Mellon University, School of Computer Science, Machine Learning Department, [16] A. Paulino, J. Feng, and A. Jain, Latent Fingerprint Matching using Descriptor-based Hough Transform, International Joint Conference on Biometrics (IJCB), pp. 1 7, [17] P. Grother and E. Tabassi, Performance of Biometric Quality Measures, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 4, pp , [18] T. Ojala, M. Pietikainen, and T. Maenpaa, Multiresolution Gray-scale and Rotation Invariant Texture Classification with Local Binary Patterns, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp , [19] T. Mäenpää, The Local Binary Pattern Approach to Texture Analysis: Extentions and Applications, PhD Thesis, Oulun yliopisto University, [20] V. Ojansivu and J. Heikkilä, Blur Insensitive Texture Classification using Local Phase Quantization, Image and Signal Processing, pp , [21] Q. Le, A. Karpenko, J. Ngiam, and A. Ng, ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning, Advances in Neural Information Processing Systems, pp , 2011.