Fingerprint verification by decision-level fusion of optical and capacitive sensors

Transcription

1 Fingerprint verification by decision-level fusion of optical and capacitive sensors Gian Luca Marcialis and Fabio Roli Department of Electrical and Electronic Engineering University of Cagliari Piazza d Armi Cagliari (Italy) {marcialis, roli}@diee.unica.it Abstract. Although some papers argued that multi-sensor fusion could improve performances and robustness of fingerprint verification systems, no previous work explicitly dealt with such topic, and no experimental evidence has been reported. In this paper, we show by experiments that a significant performance improvement can be obtained by decision-level fusion of two well-known fingerprint capture devices. As, to the best of our knowledge, this is the first work on multi-sensor fingerprint fusion, we believe that it can contribute to stimulate further researches on this promising topic. 1 Introduction Fingerprints [1] are one of the most widely used biometrics for personal verification, mainly due to their difficult reproducibility, and the impossibility to forget them. Personal verification is aimed to grant or deny the access to certain critical resources (e.g., computer, ATM, or a restricted area in a building). The person that would access to a certain resource submits to the automatic verification system her/his identity and fingerprint. The system matches the given fingerprint with the one stored in its database and associated to the claimed identity. A degree of similarity, named score, is computed. If such score is higher than a certain acceptance threshold, then the person is classified as a genuine (i.e., the claimed identity is accepted). Otherwise the person is classified as an impostor, and the access to the required resource is denied. Although many algorithms to match two fingerprints have been proposed so far, it is very difficult to obtain verification performances which satisfy the stringent requirements of many real fingerprint verification applications. In order to improve performances and robustness of fingerprint verification systems, various data fusion approaches have been proposed: fusion of multiple matching algorithms, multiple fingerprints, and multiple impressions of the same fingerprint [2-6]. In some works [6-7], it has been argued that fusion of different imaging sensors could improve performances and robustness of fingerprint verification systems. However, to the best of our knowledge, no previous work explicitly dealt with such topic, and no experimental evidence has been reported. In our opinion, the investigation of multi-sensor fingerprint fusion has been discouraged by the required increase of system s cost and user co-operation. These

2 issues, which also hold for the case of multi-modal systems [7], point out that a multisensor fingerprint verification system should realize a good trade-off between the performances improvement and the system s cost increase. In other words, the acceptability of a multi-sensor fingerprint verification system strictly depends on the increase of verification accuracy achievable with respect to that of a single-sensor system. With regard to this issue, it is worth noting that information fusion theory and results achieved in other application fields support the hypothesis that combining information coming from different physical sensors can substantially improve fingerprint verification performances [8]. Moreover, other reasons can be raised to justify multi-sensor fingerprint fusion: to allow a higher coverage of the user population (a certain fingerprint sensor can be suitable only for certain types of fingerprints), and to discourage fraudulent attempts to deceive the system (submission of fake fingers could require different kinds of fake fingers for each sensor). In this paper, we started to investigate the fusion of two well-known fingerprint sensors by adopting a simple fusion strategy. The selected capture devices are an optical and a capacitive sensor. The fusion strategy is performed at the so-called decision-level [8], that is, the matching scores separately provided by the two sensors are combined by score transformation (fusion rules) [3-4]. We show by experiments that the performance improvement theoretically and practically achievable by optical and capacitive sensor fusion can justify the system s cost and user co-operation increases. Results also show that such improvement can be obtained by simple fusion rules. The paper organisation is as follows. Section 2 describes the proposed fusion architecture. Section 3 reports experimental results. Section 4 concludes the paper. 2 The proposed fusion architecture Figure 1 shows the architecture of the proposed multi-sensor fingerprint verification system. Firstly, the image of the user finger is acquired by the optical and the capacitive sensors. Then, the images are processed and two sets of features per image are extracted, in terms of minutiae-points. A minutiae-based matching algorithm is separately applied to the optical and capacitive minutiae sets, so providing two matching scores. Such scores are fused by score transformation.

3 Optical image Pre-processing and minutiae extraction String Matcher Template Database Claimed Identity Pre-processing and minutiae extraction Score Fusion String Matcher s o s c s Decision Threshold Genuine/Impostor Capacitive image Template Database Fig. 1. Architecture of the proposed multi-sensor fingerprint verification system. 2.1 The selected fingerprint sensors In this first stage of our research, we selected an optical and a capacitive sensor because they are the most widely used, and their cost is low with respect to other fingerprint sensors [1, 9]. Moreover, their technology is considered mature. Finally, as it has been shown that fusion is effective when the information provided by different acquisition sources is used [8], their integration can be expected to work well. The optical sensors are characterised by a LED light source and a CCD placed on the side of a glass platen constituting a glass prism, on which the fingerprint to acquire is placed. The LED illuminates the fingerprint and the CCD captures the light reflected from the glass. According to the Snell law, the light is totally reflected in correspondence of the ridges, while it is totally absorbed in correspondence of the valleys. So, the CCD captures the light with high degree of luminance where a ridge is localised. The luminance degree decreases from the ridges to the valleys. Although the high image quality, the optical scanners cannot be easily integrated in PC or other electronic devices, because of their dimensions due to the prism inside such devices. Xia and O Gorman [9] report various technological innovations to improve the optical sensors, especially in order to reduce their dimension. As an example, a sheet prism is introduced, so reducing the dimension of the sensor. The core of the capacitive sensors is the sensing surface, which is made up of a two-dimensional array of silicon capacitor plates. The second plates are considered to be the finger skin. The capacitance is dependent on the distance between the finger skin, i.e. ridges and valleys, and the plates. The captured image is derived from the

4 capacitance measures from each array element. The main limitation of the capacitive sensors is that the sensing area dimension depends on the number of capacitive plates. The sensors cost strongly increases with such number. Consequently, such sensors have a small sensing area, with the strong limitation that they often cannot guarantee a sufficient number of details to reliably match two fingerprints. The different acquisition physical principles of such sensors are expected to be promising for a fusion algorithm [8]. Moreover, novel technological innovations [9] could allow to simplify the integration of such sensors in a unique acquisition module, in order to limit the overall system s cost and the user co-operation increase. 2.2 Fingerprint image processing Our pre-processing of fingerprint images provided by the optical and capacitive sensors is characterised by the following steps (Figure 2): fingerprint enhancement by FFT and rank transformation; fingerprint image binarisation and separation from the background (we considered as background the image regions too sharp or poorly contrasted); thinning and skeletonisation of the obtained fingerprint image in order to reduce each ridge line to 1 pixel of width; post-processing of the obtained fingerprint image in order to delete the spurious ridges and to merge the interrupted ridge lines. Finally, the minutiae-points are extracted. The minutiae-points [1] are defined as bifurcation and termination of each ridge line. The orientation of each minutia is also computed according to the dominating orientation of the ridge line which the minutia belongs to. Such features are widely used for fingerprint verification. They have shown to be the most appropriate features to this end [1-6]. The algorithm we selected to match the minutiae sets of two fingerprint is the socalled String algorithm [10]. The String algorithm can be summarised as follows. Let X be the template minutiae set. Let Y be the input minutiae set. For each minutia x X, the following algorithm is performed. For each y Y, x is aligned to y. After this alignment, x and y match perfectly. Let A(x, y) = {( xi, yi ), xi X, yi Y : aligned( xi, yi ) = true} be the set of other couples of aligned minutiae. x i and y i are considered as aligned on the basis of a pre-defined minutiae distance not exceeding a certain fixed threshold. At the end of such loops, the value max A( x, y) is converted in a matching score by the formula: x, y { } ( max{ A( x, y) }) 2 (1) score = X Y As usual, the obtained matching score is a real value between 0.0 and 1.0. It represents the degree of similarity between the compared fingerprint. The higher such value, the higher the degree of similarity. It is worth remarking that, in our system, the String algorithm is applied separately to the input-template minutiae extracted from the images provided by the optical and the capacitive sensors (Figure 1).

5 (a) (b) (c) (d) (e) (f) Fig. 2. Pre-processing of the fingerprint image. (a) Original fingerprint image acquired with an optical sensor. (b) Image processed by FFT. (c) Rank transformation. (d) Threshold-based binarisation. (e) Fingerprint/background segmentation. (f) Thinning, skeletonisation, and post processing: spurious ridges deletion, and interrupted ridges merge. 2.3 Decision-level fusion of optical and capacitive sensors Let s o and s c be the matching scores provided by the matching algorithm applied to the images acquired from the optical and the capacitive sensors, respectively (Figure 1): - Apply the following transformation to the above scores s o and s c to implement the fusion: s = f ( s o, s c ) (2) - Compare the obtained score value s with a threshold. The claimed identity is classified as genuine user if: s > threshold (3) otherwise it is classified as impostor. It is easy to see that the above methodology can be also used for the case of more than two matchers based on multiple sensors.

6 We investigated two kinds of score transformations according to eq. (2) to implement the fusion rule. The first fusion rule belongs to the so-called fixed fusion rules, because they require no parameter estimation. In particular, we investigated the mean of the scores (Mean): s o + s s = c 2 The second fusion rule belongs to the so-called trained fusion rules, because they require a training phase in order to set a certain number of parameters. In particular, we investigated the logistic transformation (Logistic): 1 (5) s = 1+ exp[ ( w0 + w1 so + w2s c )] The parameters of the logistic transformation (w 0, w 1, w 2 ) were computed by a gradient descent algorithm with a cross-entropy loss function [11]. (4) 3 Experimental results 3.1 The Data Set According to the FVC protocol [12], we created a data set containing 1,200 multisensor fingerprint images captured from 20 volunteers (Figures 3 and 4). We used the Biometrika FX2000 (312x372 pels images at 569 dpi) and the Precise Biometrics MC100 (250x250 pels images at 500 dpi) as optical and capacitive sensors. The forefingers, the ringfingers and the middle-fingers of both hands were acquired (six classes per person). According to the FVC protocol [12], the total number of classes (different fingerprints) is 6*20 = 120. Ten impressions of each finger were acquired, so creating the data base containing 1,200 multi-sensor fingerprint images. 3.2 Experimental protocol For each verification algorithm we computed two sets of scores. The first one is the so called genuine-matching scores set G, made up of all comparisons among fingerprints of the same identity. A score from the set G belongs to the genuine user class. The second one is the impostor matching scores set I, made up of all comparisons among fingerprints of different identities. A score from the set I belongs to the impostor class. The total number of genuine and impostor comparisons was 5,400 and 1,398,000, respectively. We randomly subdivided the above sets in two parts of the same size, so that: G=G1UG2, I=I1UI2. Sets G1 and G2, as well as I1 and I2, are disjoint. The training set Tr={G1, I1} was used to compute the weights of the logistic fusion rules. The test set Tx={G2, I2} was used to evaluate and compare the performances of algorithms.

7 Fig. 3. Examples of fingerprint images acquired from the optical sensor. Fig. 4. Examples of fingerprint images acquired from the capacitive sensor. Performances were assessed and compared in terms of: Equal Error Rate (EER) on the training set, correspondent to the error rate computed at the threshold value for which the percentage of genuine users wrongly rejected by the system (FRR) is equal to the percentage of impostors wrongly accepted by the system (FAR); Generalisation errors, i.e., we computed FAR and FRR on the test set using the EER threshold previously estimated. Total Error Rate (TER) [13], which is the overall generalisation error rate of the system computed at the EER threshold. We also investigated the complementarity among results provided by optical and capacitive sensors, in order to analyse at which extent their fusion can allow

8 recovering those fingerprints wrongly recognized by one sensor. Such analysis pointed out the potentialities of optical and capacitive sensor fusion. 3.3 Results Table 1 summarises our results in terms of EER on the training set (second column) and FAR and FRR on the test set computed with the EER threshold previously estimated (third and fourth columns). Table 1. Errors of the single and multi-sensor fingerprint verification systems. The EER is computed on the training set. FAR and FRR are computed on the test set using the EER threshold estimated from the training set. EER FAR FRR Optical 3.4% 3.2% 3.6% Capacitive 18.5% 18.2% 18.8% Fusion by Mean 2.9% 3.1% 2.8% Fusion by Logistic 2.3% 2.4% 2.3% As expected, the capacitive sensor performs notably worse than the optical one. This is mainly due to the reduced sensing surface (it is evident from Figures 3-4), which also reduces the number of extracted minutiae. This also causes that multiple impressions of the same fingerprints correspond to different parts of them, so the extracted minutiae do not match each others. Results about fusion point out the performance improvement in terms of EER. In particular, fusion by logistic allows to reduce the EER from 3.4% to 2.3%. Results in terms of test set errors also point out that fusion allows improving the robustness of the system: in fact, the deviation between the expected (training set) performance (Table 1, second column) and test set performance (Table 1, third and fourth columns) is reduced by fusion, especially when the logistic fusion rule is used (Table 1, fifth row). 3.4 Analysis of sensor complementarity With regard to results reported in section 3.3, it is worth noting that the performance difference between optical and capacitive sensors could limit the effectiveness of the fusion rules we used [8]. However, fusion provides performances better than the ones of the optical sensor (Table 1). This result suggests that optical and capacitive sensors are strongly complementary. The term complementarity refers to the overlapping degree among the sets of misclassified fingerprints of the optical and capacitive matchers [8]. The less such overlapping degree, the more the number of patterns theoretically recoverable (i.e., which can be correctly recognized) by a fusion rule. The maximum number of such patterns is theoretically recovered by the so-called oracle fusion rule [8]. The oracle is the ideal fusion rule able to select the matcher, if any, that correctly recognizes the input fingerprint. Accordingly, the oracle provides the minimum verification error achievable by fusion of optical and capacitive

9 matchers. In other words, the oracle points out the maximum performance achievable by the investigated multi-sensor fusion. In order to investigate such performance, we computed the Total Error Rate (TER) achieved by the individual matchers at the EER threshold, and the correspondent TER provided by the oracle. The TER value is given by the following formula: TER( s*) = N i FAR( s*) + N i g N + N g FRR( s*) (6) In eq. (6), s* is the acceptance threshold on which the TER is computed, N i and N g are the number of impostor and genuine matching scores in the data set. Results are reported in Table 2. Table 2. Results of the individual matchers and the oracle fuser in terms of Total Error Rate (TER) computed on the test set at the EER threshold estimated on the training set. Optical Capacitive Oracle TER 3.2% 18.5% 0.7% Reported results show that fusion is theoretically able to reduce the verification error rate close to zero. Obviously, this is just a theoretical lower bound for any fusion rules, as the oracle performance can be only approximated by fusion algorithms [14]. Nevertheless, this result points out that optical and capacitive matchers exhibit a very high degree of complementarity. In order to show at which extent the investigated fusion rules allow recovering those fingerprints wrongly recognized by one sensor, we report in Table 3 the percentage of patterns recovered by fusion and wrongly classified by one sensor. Reported results refer to the percentages of fingerprints misclassified by one of the considered sensor and recovered, i.e., correctly classified, by the different fusion rules. Table 3. Percentages of genuine and impostor fingerprints misclassified by one of the considered sensor and recovered, i.e., correctly classified, by the different fusion rules (the term recover rate is used for indicating these percentages). For each sensor, recover rates are computed with respect to the maximum number of wrongly classified genuine (second and fourth columns) and impostor patterns (third and fifth columns) in the test set. Recover Rate Optical Sensor Recover Rate Capacitive Sensor Genuine Impostor Genuine Impostor Fusion by Mean 87.1% 91.7% 93.6% 86.7% Fusion by Logistic 75.8% 70.7% 97.9% 94.3% Reported results are particularly relevant for the capacitive sensor. Although such sensor exhibits a performance considerably worse than that of the optical one, the information provided by the capacitive sensor is sufficient to recover many patterns wrongly classified by the optical sensor. For example, fusion by simple mean allows recovering 91.7% of impostor fingerprints which were wrongly classified by the

10 optical sensor. This result contributes to increase the interest about the potentialities of multi-sensor fusion. 4 Conclusions In this paper, a multi-sensor fingerprint verification system, based on the fusion of optical and capacitive sensors, has been presented. To the best of our knowledge, no work about such kind of fusion has been proposed so far, although it presents some substantial advantages, as discussed in section 1. Reported results showed that the proposed multi-sensor system can improve performances of the best individual sensor (the optical sensor). Moreover, the analysis of the complementarity between optical and capacitive matchers pointed out the potentialities of multi-sensor fusion, which can theoretically achieve very low verification errors, so justifying the increase of the system s cost and user cooperation. The capability of recovering many patterns misclassified by one sensor contributes to increase the interest in such kind of fusion. It is worth remarking that the same feature extraction process has been applied separately to the images produced by each capture device, and a simple fusion rule has been applied. Hence, the proposed system is very simple to implement. Reported results are preliminary. Further work will concern the investigation of other fingerprint sensors, and the application of different matching algorithms and fusion rules. Acknowledgments This work was partially supported by the Italian Ministry of University and Scientific Research (MIUR) in the framework of the research project on distributed systems for multisensor recognition with augmented perception for ambient security and customisation. References [1] D. Maltoni, D. Maio, A.K Jain, and S. Prabhakar, Handbook of Fingerprint Recognition, Springer Verlag, [2] A.K. Jain, S. Prabhakar, and S. Chen, Combining Multiple Matchers for a High Security Fingerprint Verification System. Pattern Recognition Letters, 20 (11-13) , [3] G.L. Marcialis and F. Roli, Experimental results on Fusion of Multiple Fingerprint Matchers. Proc. 4 th Int. Conf. on Audio and Video-Based Person Authentication AVBPA03, J. Kittler and M.S. Nixon Eds., Springer LNCS2688, pp , [4] G.L. Marcialis and F. Roli, Perceptron-based Fusion of Multiple Fingerprint Matchers. Proc. First Int. Work. on Artificial Neural Networks in Pattern Recognition ANNPR03, M. Gori and S. Marinai Eds., pp , 2003.

11 [5] A. Ross, A.K. Jain, and J. Reisman, A Hybrid Fingerprint Matcher. Pattern Recognition, 36 (7) , [6] S. Prabhakar and A.K. Jain, Decision-level Fusion in Fingerprint Verification. Pattern Recognition 35 (4) , [7] A.K. Jain, R. Bolle, and S. Pankanti (Eds.), BIOMETRIC Personal Identification in Networked Society. Kluwer Academic Publishers, Boston/Dordrecht/London, [8] T. Windeatt and F. Roli (Eds.), Multiple Classifier Systems, Springer Verlag, Lecture Notes in Computer Science, Vol [9] X. Xia and L. O Gorman, Innovations in fingerprint capture devices. Pattern Recognition, 36 (2) , [10] A.K. Jain, L. Hong, and R. Bolle, On-line Fingerprint Verification. IEEE Transactions on Pattern Analysis and Machine Intelligence 19 (4) , [11] C.M. Bishop, Neural Networks for Pattern Recognition. Oxford University Press, [12] D. Maio, D. Maltoni, R.Cappelli, J.L. Wayman, and A.K. Jain, FVC-2000: Fingerprint Verification Competition. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (3) , [13] P. Verlinde, P. Druyts, G. Chollet, and M. Acheroy, A multi-level data fusion approach for gradually upgrading the performances of identity verification systems, in B.Dasarathy Ed., Sensor Fusion: Architectures, Algorithms and Application III, vol.3719, pp.14-25, Orlando, FL, USA, SPIE Press, [14] G. Giacinto and F. Roli, Dynamic Classifier Selection Based on Multiple Classifier Behaviour. Pattern Recognition 34 (9) , 2001.