Fingerprint Enhancement and Identification by Adaptive Directional Filtering

Transcription

1 Fingerprint Enhancement and Identification by Adaptive Directional Filtering EE5359 MULTIMEDIA PROCESSING SPRING 2015 Under the guidance of Dr. K. R. Rao Presented by Vishwak R Tadisina ID: EE5359 Multimedia Processing - Spring

2 Acronyms 1D- One Dimension 2D- Two Dimension AFIS Automatic Fingerprint Identification System DC- Direct Current ECTI-CON - Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology Conference FBI Federal Bureau of Investigation FFT- Fast Fourier Transform ICBA- International Conference on Bioinformatics and its Applications ICPR International Conference on Pattern Recognition IEE Institution of Electrical Engineers IEEE- Institute of Electrical and Electronics Engineers ISCV International symposium on Computer Vision LCNS- Lecture Notes in Computer Science LPF- Low Pass Filter MATLAB Matrix Laboratory MTF Modulation Transfer Function WACV- Winter Conference on Applications of Computer Vision EE5359 Multimedia Processing - Spring

3 Introduction Identifying a person based on the biometrics has become important in current diverse businesses like law enforcement, information system security, finance physical access control etc. [4]. Fingerprint recognition is one of the most important biometric technologies which has drawn a substantial amount of attention recently [4]. The best aspect of fingerprint-based identification is that the fingerprints of a person are unique and does not alter with aging of an individual [1] A method to manually match fingerprint was developed by law enforcement agencies [4]. But this method is tedious and time taking. Automatic fingerprint identification system (AFIS) Input can be given by digitalizing the image take by ink or by using inkless scanners. EE5359 Multimedia Processing - Spring

4 Stages in AFIS Fig.1 Different stages involved in an Automatic fingerprint identification system [11]. EE5359 Multimedia Processing - Spring

5 Suitable features for representation of a fingerprint Keep back the uniqueness of each fingerprint in various levels of resolution. Distinct characteristics of a fingerprint can be estimated easily. Easy to apply automatic matching algorithms. Immune to noise distortions. Effective and simple representation [11]. EE5359 Multimedia Processing - Spring

6 Fingerprint Structure Fingerprint is the image of the surface of the skin of the fingertip. It consists of ridges and valleys as shown in Fig.2. The ridge pattern in a fingerprint can be described as an oriented texture pattern with fixed dominant spatial frequency and orientation in a local neighbourhood [2]. Orientation - flow pattern of the ridges [2]. Frequency - inter-ridge spacing [2]. The anomalies in a fingerprint are called minutiae (ex: ridge endings, bifurcations, crossovers, short ridges etc. as shown in Fig.2 Fig. 2 Bifurcations and short ridges in a fingerprint structure [11]. EE5359 Multimedia Processing - Spring

7 Fingerprint Enhancement Algorithm An ideal algorithm must increase the contrast between the ridges and valleys of a fingerprint for visual examination or automatic feature extraction [2]. In this algorithm [2] during the processing of each pixel a local neighbourhood of that pixel is considered and this can be explained using Fig. 3. As the ridges and valleys have well-defined frequency and orientation in the local area directional filters are used [2]. The filtering process is adaptive as the parameters of these directional filters depend on the local ridge frequency and orientation [2]. EE5359 Multimedia Processing - Spring

8 Determining minutiae based on neighbouring pixels Fig.3 In (a) the pixel with three neighbours is a ridge bifurcation and in (b), pixel with only one neighbour is a ridge ending [15]. EE5359 Multimedia Processing - Spring

9 Steps involved in the Fingerprint Enhancement Algorithm Normalization: To obtain a pre-specified mean and variance, an input fingerprint image is normalized [2] Local orientation and Frequency estimation: The normalized input fingerprint image is used for computing orientation and frequency images [2]. Region mask estimation: Each block in the normalized input fingerprint image are sorted out into a recoverable or an unrecoverable block to find a region mask estimate [2]. Filtering: A bank of Gabor filters or Butterworth filters that are tuned to local ridge orientation and ridge frequency are used [2]. EE5359 Multimedia Processing - Spring

10 Flowchart of a fingerprint enhancement algorithm Fig.4 Flowchart of a fingerprint enhancement algorithm [4]. EE5359 Multimedia Processing - Spring

11 Normalization Normalization reduces the variations in grey-level values along ridges and valleys [2] I(x, y) denote the grey-level value at pixel (x, y) M i and V i denote the estimated mean and variance of I M 0 and V 0 are the desired mean and variance values N i (x, y) denote the normalized grey-level value at pixel (x, y) (1) EE5359 Multimedia Processing - Spring

12 Fingerprint after normalization Fig. 5 The result of normalization. (a) Input image. (b) Normalized image [4]. EE5359 Multimedia Processing - Spring

13 Local Ridge Orientation Local ridge orientation is usually specified blockwise rather than at every pixel [4]. Least mean square orientation estimation based on gradient is used here [4]. Each fingerprint image in divided into equal blocks and gradients are calculated for each pixel in a block and average squared gradient for the block is calculated from this [4]. The average gradient ϕ direction and dominant local orientation O [1] for the block are given by: (2) Correction for 90 degrees is necessary since the angle of gradient is perpendicular to the ridge orientation [4]. Here blocks of size W W = 8 8 for orientation estimation and gradients g x and g y are used and calculated using Sobel operator [2]. (3) EE5359 Multimedia Processing - Spring

14 Local Ridge Orientation Additional smoothing (Low pass filtering) is required at distorted and noisy regions [4].It is done by converting orientation image into a continuous vector field as shown in the Fig. 8, defined as follows Where Ψ x i, j and Ψ y i, j are the x and y components of the continuous vector field respectively. (4) (5) Fig.6 A continuous vector field formed by a local orientation image with a block of size W x W and center O (i, j). EE5359 Multimedia Processing - Spring

15 Local Ridge Orientation The filter implementation [1] is given by, (6) (7) (8) where L is a 2D LPF and W Ψ W Ψ specifies the size of the filter Ψ x i, j and Ψ y i, j are the x and y components of the continuous vector field respectively after smoothing. EE5359 Multimedia Processing - Spring

16 Local Ridge Frequency Local ridge frequency is found by projecting the grey values of all the pixels located in each block along a direction orthogonal to the local orientation. 1D wave with the local extrema corresponding to the ridges and valleys of the fingerprint [4]; Let K(i, j) be the average number of pixels between two consecutive peaks in the 1D wave generated above. The frequency ω i, j [4] is computed as ω i, j =1/K(i, j) (4) In order to explain the above estimation a one dimensional (1D) modeled fingerprint image instead of the original raw fingerprint images can be used. A finite rectangular wave (as seen in Fig. 7) which is regarded as the simplification of the projection of all grey values of the pixels in a direction, normal to the local orientation of the block with local extrema corresponding to the ridges and valleys of the fingerprint. EE5359 Multimedia Processing - Spring

17 Finite rectangular wave as a modeled fingerprint Fig. 7 Finite rectangular wave as a modeled fingerprint [15]. EE5359 Multimedia Processing - Spring

18 Directional Filtering An ideal model of band pass directional filter [1] in Fourier domain can be expressed using polar coordinates (ρ, ϕ) as Hr (ρ) depends on local ridge spacing and Ha (ϕ) depends on local ridge orientation [3]. Instead EE5359 Multimedia Processing - Spring

19 Directional Filter in Fourier Domain Fig. 8 Filter in Fourier domain (a) band pass (radial) component, (b) directional (angular) component, (c) combination of previous two [1]. EE5359 Multimedia Processing - Spring

20 Filtering of input image Filtering [3] an input fingerprint image q is performed as follow: The FFT F of input fingerprint image q is computed, here u= 0, 1, 2,, 31 and v = 0, 1, 2,, 31. (10) Each directional filter P i is point-by-point multiplied by F, obtaining n filtered image transforms PF i, i = 1,..., n. Inverse FFT is computed for each PFi resulting in n filtered images p f i, i = 1,..., n (spatial domain) [3]. For x = 0, 1, 2 31 and y = 0, 1, The enhanced image is obtain in following manner: all pixels in one block of enhanced image take the value of pixels on the same position from the filtered image which emphasizes determined orientation for corresponding block [3]. (11) EE5359 Multimedia Processing - Spring

21 Block Diagram of a Fingerprint Enhancement Algorithm Fig. 9 Block diagram of a fingerprint enhancement algorithm [12]. EE5359 Multimedia Processing - Spring

22 Butterworth Filter The band pass Butterworth filter [3] for radial component H r (ρ) of order k (usually k = 2), having centre frequency ρ 0 and bandwidth ρ BW [3] is given as: and the directional component is given by (13) (12) (13) Where ϕ BW is the angular bandwidth, and ϕ c is the orientation of the filter [3]. EE5359 Multimedia Processing - Spring

23 Frequency response of Butterworth Filter Fig.10 Butterworth bandpass frequency response EE5359 Multimedia Processing - Spring

24 Gabor Filter Gabor filters are very useful both in frequency and spatial domain, due to their frequencyselective and orientation-selective properties [4]. By simple adjustment of mutually independent parameters, Gabor filters can be configured for different shapes, orientations, different width of band pass and different central frequencies [4, 6]. An even Symmetric Gabor filter general form [4] in the spatial domain [1] is given by (14) (15) (16) EE5359 Multimedia Processing - Spring

25 ϕ is the orientation of the Gabor filter, f is the frequency of the sinusoidal plane wave along the x-axis, σ x and σ y are the standard deviations of the Gaussian envelope along the x and y axes, respectively. The modulation transfer function (MTF) [4] of the Gabor filter can be represented as, (17) (18) (19) here σ u = 1/2πσ x and σ v = 1/2πσ y. The filter is more immune to noise, if σ x and σ y are significantly large, but is more likely to create unauthentic ridges and valleys. The filter is not effective in removing the noise, if standard deviations are too small (20) (21) EE5359 Multimedia Processing - Spring

26 An even symmetric Gabor filter and its MTF Fig. 11 An even-symmetric Gabor filter. (a) The Gabor filter with f = 10 and ϕ = 0. (b) The corresponding MTF [4]. EE5359 Multimedia Processing - Spring

27 Fingerprint Identification For fingerprint identification it is ideal to get representations of fingerprints which are invariant with reference to scale, translation and rotation [22]. The scale variance difficulty can be eliminated easily since most fingerprint images could be scaled as per the dpi specification of the sensors. To remove the other two variance problems a reference frame can be formed which is rotation and translation invariant [22]. The translation invariance is handled by establishing a single reference point (core point). This reference point is obtained based on the assumption that all the fingerprints are vertically oriented. But practically the fingerprint images may be oriented up to ± 45º away from actual assumed vertical orientation [22]. Cyclic rotation of the feature values in the Fingercode in the matching stage handles this image rotation partially. EE5359 Multimedia Processing - Spring

28 Reference Point Location The reference point or core point of a fingerprint is the point at which the curvature of the concave ridges is maximum as shown in Fig.12. Fig. 12 Concave and convex ridges in a fingerprint image when the finger is positioned upright [22]. After finding the smoothened orientation image in section 3.2. From (8) compute E, an image containing only the sine component of O [22]. Integrate pixel intensities R 1 and R 2 for each pixel (i, j) in E as shown in Fig. 13. Assign the value of their difference in corresponding pixels to A (A label image which indicates the reference point is initialized)[22]. (22) (23) EE5359 Multimedia Processing - Spring

29 Reference Point Location On a large database a reference point algorithm is used to empirically determine the regions R 1 and R 2 [22]. The maximum curvature in concave ridges can be captured making use of the geometry of regions R 1 and R 2 [22]. Find the maximum value in A [22] and assign its coordinate to the core, i.e., the reference point. Fig. 13 Regions for integrating E pixel intensities for A (i, j) [22]. EE5359 Multimedia Processing - Spring

30 Feature vector A fingerprint image can be sectored into a total of 16 5 = 80 sectors (S 0 through S 79 ) whose core point [22] is the center of these sectors as shown in Fig. 14. Fig. 14 Reference point (x), the region of interest, and 80 sectors [22]. Let F iϕ (x, y) be the ϕ - direction filtered image for sector S i. Now i ϵ {1,2,3, 79} and ϕ ϵ {0º, 22.5º, 45º, 67.5º, 90º, 112.5º, 135º, 157.5º}. The feature value V iϕ [22]is the average absolute deviation from mean defined as where n i is the number of pixels in S i and P i ϕ is the mean of pixel values in a sector. The average absolute deviation of each sector in each of the eight filtered images defines the components of the feature vector. (24) EE5359 Multimedia Processing - Spring

31 Fingerprint matching The Euclidean distance between the corresponding Fingercodes is used for fingerprint matching [22]. A Fingercode is a compact length code obtained by the filter-based bank algorithm in [22] which uses a bank of Gabor filters to capture both local and global details in a fingerprint. Reference point removes the translation variance problem [22]. To eliminate rotational variance the Fingercode is rotated cyclically [22]. Equations (25), (26) and (27) give single step cyclic rotation [22] of the features of the Fingercode. This corresponds to a feature vector which would be obtained if the image were rotated by 22.5º. A rotation by R steps corresponds to a rotation R 22.5º of the image. A positive and negative rotation implies clockwise and counterclockwise rotation respectively. The Fingercode [22] obtained after R steps of rotation is given by (25) (26) (27) EE5359 Multimedia Processing - Spring

32 Fingerprint matching where m is the number of sectors in a band, i ϵ {0,1, 2, 79} and ϕ ϵ {0º, 22.5º, 45º, 67.5º, 90º, 112.5º, 135º, 157.5º}. Five templates are stored corresponding to the following five rotations of the Fingercode: V iφ 2, V iφ 1, V iφ 0, V iφ 1 and V iφ 2 [22]. This Fingercode corresponds to 22.5º rotation. So to make the code more robust we need rotation corresponding to 11.25º [22]. The original image is rotated by 11.25º and the corresponding five templates are stored. Making a total of ten templates. These ten templates give ten Fingercodes. So in order to perform matching the Fingercodes of the input image is compared with the Fingercodes in the database [22]. And the Fingercodes with least Euclidian distance is matched. EE5359 Multimedia Processing - Spring

33 Implementation Description of the database: Used the database from FVC 2004 These images are greyscale images of size 640x480 and 96dpi spatial resolution. Normalization The parameters used for normalization M 0 =100 V 0 =100 EE5359 Multimedia Processing - Spring

34 (1) (c) (a) (3) (b) (2) (d) (4) Fig. 15 a, b, c and d: Original fingerprint images; 1, 2, 3 and 4: Corresponding normalized images. EE5359 Multimedia Processing - Spring

35 Ridge orientation and frequency Input fingerprint images are divided into non-overlapping blocks of size 8 8. Then the gradients g x (i, j) and g y (i, j) for each pixel (i, j) of the block, are calculated by Sobel edge-emphasizing filter. Average squared gradient and average gradient direction computed from the above values. These images are smoothened using a 2D-LPF filter and noise is eliminated. EE5359 Multimedia Processing - Spring

36 Fig. 16 (a) and (b) are original fingerprint images; (i) and (ii) are their respective Edge detected images; (1) and (2) are their respective Gradient images. EE5359 Multimedia Processing - Spring

37 Orientation images using quiver plots Fig. 17 (a), (b) and (c) are original fingerprint images; (1), (2) and (3) are their respective orientation image quiver plots. EE5359 Multimedia Processing - Spring

38 Directional filtering using Gabor filter bank The inter-ridge distance in the fingerprint image is the main factor in determining the parameters Ω, σ x and σ y, for optimal Gabor filter operation. If Ω is too large spurious ridges are created in the filtered image, whereas if Ω is too small nearby ridges are merged into one. We set parameters to be Ω = 1/5, and σ x = σ y = 4.0 [21]. Eight different directional Gabor filters are used. Eight different values for ϕ = iπ/8 (0º, 22.5º, 45º, 67.5º, 90º, 112.5º, 135º, 157.5º) with respect to the x-axis are used. A 0º oriented filter accentuates those ridges which are parallel to the x-axis and smoothens the ridges in the other directions [21]. EE5359 Multimedia Processing - Spring

39 Enhanced fingerprint images- Gabor filter Fig. 18 (a), (b) and (c) are original fingerprint images; (1), (2) and (3) are the enhanced images obtained by directional filtering using a series of Gabor filters. EE5359 Multimedia Processing - Spring

40 Scope of the Project The objective of this project is to apply the algorithm proposed in section 3 to smudged and corrupted fingerprints to obtain enhanced images. This is done by adaptive directional filtering in the frequency domain by using Butterworth [2] and Gabor filters [1] for fingerprint image enhancement and also for removing noise. MATLAB is used to normalize the corrupted fingerprints. Then the frequency and ridge orientation are computed for each fingerprint image. After that the image is filtered using directional filters. Here Butterworth and Gabor filters are used to obtain an enhanced image. The quality of the images obtained from both filters is compared visually. Fingerprint identification is done using MATLAB coding on the filtered enhanced image by detecting reference point and storing a feature vector in the form of a Fingercode in a data file. This data file is used as a database for fingerprint matching [21]. EE5359 Multimedia Processing - Spring

41 Future work Fingerprint enhancement using Butterworth filter Comparing the enhanced images from Gabor and Butterworth filters. Fingerprint matching using reference point position and Fingercode. EE5359 Multimedia Processing - Spring

42 References [1] A. M. Raiˇcevi c and B. M. Popovi c, An Effective and Robust Fingerprint Enhancement by Adaptive Filtering in Frequency Domain, Facta Universitatis (NIS) Ser.: Elec. Energ., vol. 22, no. 1, pp , April [2] J. E. Hoover, The Science of Fingerprints: Classification and Uses, Federal Bureau of Investigation, Washington, D.C., Aug [3] B. G. Sherlock, D. M. Monro and K. Millard, Fingerprint enhancement by directional Fourier filtering, IEE Proc. Vision Image Signal Process., vol. 141, no. 2, pp.87 94, April [4] L. Hong, Y. Wan and A. K. Jain, Fingerprint image enhancement: Algorithm and performance evaluation, IEEE Trans. Pattern Anal. Machine Intell. vol. 20, no. 8, pp , Aug [5] A. Willis and L. Myers, A cost-effective fingerprint recognition system for use with low-quality prints and damaged fingertips, Pattern Recognition, vol. 34, pp , Jan [6] J. Yang, L. Lin, T. Jiang and Y. Fan, A modified Gabor filter design method for fingerprint image enhancement, Pattern Recognition Letters, vol. 24, pp , Jan [7] L. Hong, A.K. Jain, S. Pankanti and R. Bolle, Fingerprint Enhancement, Proc. First IEEE WACV, pp , Sarasota, Fla., Dec EE5359 Multimedia Processing - Spring

43 References [8] T. Kamei and M. Mizoguchi, Image Filter Design for Fingerprint Enhancement, Proc. ISCV 95, pp , Coral Gables, Fla., Nov [9] K. Karu and A.K. Jain, Fingerprint Classification, Pattern Recognition, vol. 29, no. 3, pp , July [10] L. O Gorman and J.V. Nickerson, An Approach to Fingerprint Filter Design, Pattern Recognition, vol. 22, no. 1, pp , Jan [11] N. Ratha, S. Chen and A.K. Jain, Adaptive Flow Orientation Based Feature Extraction in Fingerprint Images, Pattern Recognition, vol. 28, no. 11, pp , March [12] Y. Wang, J. Hu and F. Han, Enhanced gradient-based algorithm for the estimation of fingerprint orientation fields, Applied Mathematics and Computation, vol. 185, no. 2, pp , Feb [13] D. L. Hartmann, Filtering of Time Series, [Online]. Available: [14] S. Chikkerur, A. N. Cartwright and V. Govindaraju, Fingerprint enhancement using STFT analysis, Pattern Recognition, vol. 40, pp , Jan [15] R. Iwai and H. Yoshimura, "A New Method for Improving Robustness of Registered Fingerprint Data Using the Fractional Fourier Transform", International Journal of Communications, Network and System Sciences, vol. 3, no. 9, pp , Sept EE5359 Multimedia Processing - Spring

44 References [16] A. Sherstinsky and R.W. Picard, Restoration and Enhancement of Fingerprint Images Using M-Lattice: A Novel Non-Linear Dynamical System, Proc. 12th ICPR-B, pp , Oct [17] E. Bezhani, D. Sun, J. Nagel and S. Carrato, Optimized filterbank fingerprint recognition, Proc. SPIE 5014, Image Processing: Algorithms and Systems, vol. 2, pp , May [18] Project Idea, EE5359-Multimedia Processing Course Website. [Online]. Available: [19] P. Salil, J. Anil and P. Sharath, Learning fingerprint minutiae location and type, Pattern Recognition, vol. 36, pp , Oct [20] MATLAB version , Release R2013a, The MathWorks, Inc., Natick, Massachusetts, United States, Feb [21] E. Zhu, J. Yin, G. Zhang and C. Hu, A Gabor Filter Based Fingerprint Enhancement Scheme Using average Frequency, International Journal of Pattern Recognition and Artificial Intelligence, vol. 20, no. 3, pp , May [22] A. K. Jain, S. Prabhakar, L. Hong and S. Pankanti, Filterbank-Based Fingerprint Matching, IEEE Transactions on Image Processing, vol. 9, no. 5, pp , May [23] A. K. Jain, S. Prabhakar and L. Hong, A multichannel approach to fingerprint classification, IEEE Trans. Pattern Anal. Machine Intell. vol. 21, no. 4, pp , Apr EE5359 Multimedia Processing - Spring

45 References [24] Fingerprint Verification Competition, The Biometric system lab, University of Bologna, Cesena-Italy, [Online]. Available: [25] FBI Fingerprint Database, Washington, D. C., United States. [Online]. Available: [26] K. R. Rao and S. Chakraborthy, Fingerprint Enhancement by Directional Filtering, ECTI-CON, Hua Hin, Thailand, May [27] K. R. Rao, D. N. Kim and J. J. Hwang, Fast Fourier Transform - Algorithms and Applications, Springer Science & Business Media, New York, [28] S. Chikkerur, C. Wu and V. Govindaraju, "A systematic approach for feature extraction in fingerprint images", ICBA, LCNS, vol. 3072, pp , July [29] K. Nilsson and J. Bigun, Localization of corresponding points in fingerprints by complex filtering, Pattern Recognition Letters, vol. 24, no. 13, pp , Sept [30] A. R. Rao, A Taxonomy for Texture Description and Identification, Springer Series in Perception Engineering, New York, [31] M. Kass and A. Witkin. "Analyzing oriented patterns", Computer vision, graphics and image processing, vol. 37, no. 3, pp , March [32] Image Processing Toolbox User s guide, The MathWorks, Inc., Natick, Massachusetts, United States, March [Online]. Available: [33] L. Rosa, MATLAB Fingerprint Recognition Functions Code, L'Aquila, Italy. [Online]. Available: EE5359 Multimedia Processing - Spring