Fingerprint Mosaicking by Rolling with Sliding

Transcription

1 Fingerprint Mosaicking by Rolling with Sliding Kyoungtaek Choi, Hunjae Park, Hee-seung Choi and Jaihie Kim Department of Electrical and Electronic Engineering,Yonsei University Biometrics Engineering Research Center, Seoul, Korea Abstract. 1 Introduction Numerous fingerprint-based verification systems have been widely adopted throughout the modern world, since these systems are convenient to use and relatively superior to other biometric systems in terms of price and performance. One advantage of small sensors (e.g., solid-state sensors) is that they can be used with many applications (e.g., laptops, cellular phones). However, information about the fingerprint is limited due to the small physical size of the sensing area, as shown in Fig.. (a) (b) Fig. 1. Fingerprint images captured by a large sensor and smaller one : (a) a fingerprint image captured by a large sensor, (b) a fingerprint image captured by a small sensor Therefore, the overlap between the template impression and the query impression produces inferior results, for example, a higher rate of false rejects. To overcome this problem, some researchers have explored the field of fingerprint fusion. The fingerprint fusion algorithm can be categorized largely into two types. The first type fuses feature sets from several fingerprint images; the second type produces mosaic fingerprint images. At the feature level, the fusion algorithm is very simple, and also, its recognition performance is better than that of a non-fusion system. However, it is difficult to apply it to existing systems which use features different from those used in the feature level fusion algorithm. It is

2 also difficult to add new features to the system. Several researchers have studied fusion systems at an image level. Jain and Lee captured several impressions by dabbing a finger on a small sensor and made a mosaic with a rigid transform [1],[2]. Ratha captured sequential impressions by rolling a finger on a large sensor. These kinds of sensors can cover a whole fingerprint and mosaic that print by stitching without calculating the transform among the fingerprints [3]. The multiple impressions captured by the dab approach (as used by Jain and Lee) are very hard to be mosaicked when the overlap between two images is very small, as shown in Fig. 2(a). In addition, the dab approach has little effect on two impressions obtained from a similar portion of a finger, as shown in Fig. 2(b). Therefore, it is difficult to acquire a whole fingerprint when using a small sensor. While the rolling approach (suggested by Ratha) is able to acquire a whole fingerprint, it requires large sensors, as shown in Fig. 2(c). It therefore cannot be applied to systems that use small sensors. (a) (b) (c) Fig. 2. Mosaicked images of previous algorithms : (a) small overlap area, (b) large overlap area, (c) rolled image with a large sensor To capture a whole fingerprint with a small sensor we present a new enrollment scheme and propose a new mosaicking algorithm which mosaics an image sequence captured by our enrollment scheme. Our paper is organized as follows. In Section 2, we describe our enrollment scheme and our system flow chart. In Section 3, we describe an image selection method that rejects a low-quality image from a sequence and a simple stitching method that works without calculating the transform parameter. In Section 4, we describe the image mosaicking process that calculates the local transform parameters between the images and warps one image to the other. The experimental results are shown in Section 5. Finally, conclusions appear in Section 6.

3 2 Fingerprint Enrollment and System Flow Charts To capture whole fingerprint images with small sensors, we present a new enrollment scheme as shown in Fig. 3. The user puts the left side of the finger on a small sensor, as shown in Fig. 3(a). The user rolls the finger on the sensor until the right part of the captured image contains the foreground region of a fingerprint, as shown in Fig. 3(b). The user then slides the finger on the sensor horizontally to acquire the part of the fingerprint that has not yet been captured, as shown in Fig. 3(c) and 3(d). Finally the user rolls the finger again to capture the right side of the finger, as shown in Fig. 3(e). Using this enrollment scheme, we were able to capture a sequence of images of a fingerprint and mosaic the sequence to produce a whole fingerprint. By using our enrollment scheme, we were able to obtain a wider area of a fingerprint with a small sensor than by using the dab approach. Fig. 3. New Enrollment Scheme We have explained how to acquire the horizontal region of a finger, as shown in Fig. 3. We can also acquire the vertical region of a finger by sliding a finger on the sensor vertically, as shown in Fig. 4. Furthermore, we can acquire a whole fingerprint to mosaic the horizontal region and the vertical region of a finger. The algorithm used to mosaic images that were captured by rolling and sliding a finger on the sensor horizontally is very similar to the algorithm that mosaics images that were captured by sliding the finger vertically. In this paper, we will explain only the former algorithm. Fig. 5 shows the flow chart of our algorithm that mosaics a fingerprint image sequence into a wide fingerprint image. In the preprocessing procedure, we segment foreground (fingerprint) and background areas in each frame and reject the motion blurred image. If the translation between the previous frame and the current frame doesn t occur, our algorithm stitches the current frame to the previous one (to expand the foreground region of the previous frame.) If the translation does occur, the algorithm executes the next procedure to mosaic the current frame to the previous one. To mosaic two images, our algorithm uses the global transform parameter to align one image to the other (roughly) and then uses the local transform parameters to align the local image blocks of one image to those of the other image. The corresponding points are considered to be the center points of each block. Finally, we warp one image to the other with

4 (a) (b) Fig. 4. The images captured by sliding vertically : (a) samples captured by sliding vertically, (b) the mosaicked image these corresponding points and mosaic the two images. Each procedure of our algorithm is explained in greater detail in the following sections. Preprocessing Estimation of a Global Parameter Hierarchical Block Matching Global Translation Vector Local Translation Vectors Image Warping Assignment of Gray Value Fig. 5. The system flow chart 3 Preprocessing In the preprocessing procedure, we first segment foreground and background areas in an image using the block variance of the image. This is because the background region contains a very low variance in the majority of the optical sensors. After the segmentation, we find the mean and variance of the foreground region and normalize the image, as suggested in [4]. In our enrollment scheme, it is possible that a few images become blurred. Blurring occurs when a user slides the finger on the sensor. These motion-blurred

5 images are rejected with the median value of the tenengrad of each image [5]. (The tenengrad refers to the magnitude of the gradient of an image.) The median value of the tenengrad is computed as T = med 1 N N N N G x (i, j) 2 +G y (i, j) 2 (1) i,j F b k The gradients G x and G y of an image are calculated by the sobel gradient algorithm and then the median value of the tenengrad is calculated in the foreground area blocks. In this case, the size of each block is 8*8 pixels. Fig. 6 shows the median tenengrad values of each frame. In Fig. 6 the tenengrads of motion-blurred images at frame number 17,79,92 are below the threshold (150) that is defined by using 2400 samples. The image at frame number 33 has a very high tenengrad value. After rejecting motion-blurred images, we check if the translation between the current frame and the previous one has occurred. If there has been no translation, we stitch the current frame to the previous one to expand a foreground area with the minimum method as proposed in [3]. To check the occurrence of the translation, we check whether the SAD (Sum of Absolute Difference) in the common foreground region between two frames is below the threshold or not. If the translation has occurred, we divide the current frame into several blocks and find the local translation vector of each block between two frames to mosaic them. In the following section, our image mosaicking procedure is explained in more detail. (17) (33) (79) (92) The Median Tenengrad of a image sequence! threshold Median Tenengrad Image Frame Number Fig. 6. The Median tenengrad of a image sequence

6 4 Image Mosaicking The image mosaicking procedure is divided into three parts. The first part involves searching for a global translation vector, the second part involves hierarchical block matching to find a local translation vector of each image block, and the third part involves the warping and assigning of gray-values to pixels which are found at the boundary between the two frames. 4.1 Searching for a Global Translation Vector We do not consider the rotation parameter between two frames because users slide their fingers horizontally on the sensor. We use the block-matching algorithm proposed by Chen et al to find global translation vectors and local ones [6]. Chen s algorithm allows the translation vector to be searched in a global minimum (like the full-search algorithm). Processing time can be reduced by about 1/10 when using Chen s algorithm. Even though many algorithms are faster than Chen s algorithm, most of them do not guarantee the global minimum solution. Furthermore, these searching algorithms are more likely to be trapped in a local minimum in fingerprint images than in other images, because the pattern of a fingerprint image is similar to a 2D-sinusoidal signal. 4.2 Hierarchical Block Matching After finding a global translation vector, we have to find the local translation vectors hierarchically, as shown in Fig. 7. When the user slides the finger on the sensor, plastic distortion caused by rubbing is inevitable. This makes it hard to align one image to the other exactly when using a global translation vector. To solve this problem, we divide an image into several blocks and find the local translation vector of each block. We then warp one image to the other with these local translation vectors. To find these local translation vectors, we align two frames (roughly) with a global translation vector and set the common foreground area between two frames. The common area is divided into four sub-blocks and each sub-block is divided into four high-level blocks until the smallest block size becomes 16*16 pixels, as shown in Fig. 7. The size of each block is a multiple of 2, so if the size of the common area is not a multiple of 2, the sub-blocks overlap. The smaller the sub-block size, the larger the probability of incorrect searching of the sub-block translation vector. This incorrect searching can be due to image noise, plastic distortion and simple patterns in a small area of a fingerprint image. To find the vectors correctly we implemented a regularization step on the Bayesian theory. The translation vector of the sub-block i in level l+1 is computed as ( ( )) t l+1,i = arg max t k l+1,i P t k /t l+1,i l,i/2 ( ( ) ) = arg max P t l,i/2 /t k P (t k ) t k l+1,i l+1,i l+1,i (2)

7 P (t k l+1,i) = ( ) P t l,i/2 /t k = l+1,i 1 e 2πσ N(S l+1 ) 1 tl,i/2 t k 2 l+1,i 2σ 2 (3) SAD(t k l+1,i ) SAD(t x l+1,i ) (4) t x l+1,i S l+1 We assume that the A-posterior PDF (Probability Density Function) of the translation vector is Gaussian and find the MAP (Maximum A-posterior Probability) solution for the local translation vector. In Eq. 3 t k is the kth translation vector of the block i in level l and the A-posterior probability in the l+1,i translation vector t l,i/2 of the parent block i/2 in level l, is computed as Eq.??. The prior probability of the translation vector t k l+1,i is P (tk l+1,i ) and the size of the searching area for the translation vector is N(S l+1 ). We find the translation vector of each sub-block in each level hierarchically through Eq. 2, as shown in Fig. 7. We define the plastic distortion of each block as the difference between the local translation vector and the global one. The center points of each smallest sub-block are the corresponding points between two images, which can be used in the image warping procedure. (0) f ( 0,0 = f ) golbal Level 0 (1) f 0,1 (1) f 0,0 (1) f 1,1 (1) f 1,0 Level 1 (2) (2) f 2,0 f 2,1 (2) (2) f 3,0 f 3,1 Level 2 Fig. 7. Hierarchical structure for searching the local translation vector 4.3 Warping and Gray-value Assignment After the block translation vectors are estimated, local distortions are compensated for using the point-based warping technique. Every center point of every block is considered a corresponding point when image Q is warped. That is, points derived from a global translation vector are utilized as destination points while source points are defined as translated points by the local translation vectors, as shown in Fig. 8. In the image warping procedure, we use the 2-pass meshwarping algorithm [7]. This algorithm includes Fant s resampling algorithm and

8 uses cublic spline as the interpolation method. The 2-pass mesh-warping algorithm is simple and is well-suited to our algorithm because the center points of the sub-blocks have a lattice structure. After performing the warping procedure, we stitch the image Q to the image P and make the boundary between image Q and P seamless. We define the transition region around the boundary (width is 5 pixels) and assign a gray value of each pixel in this transition region as the weighted sum of the gray values from Image Q and P in equation Eq. 5. In equation Eq.5, x is the distance from the boundary to the pixel position x of image Q. W is the width of the transition region, and t g is a global translation vector t used to align the warped image P to the image Q. Finally, we obtain a mosaicked image from images P and Q. We obtain a wide fingerprint image from an image sequence by applying our algorithm, as shown in Fig. 10. I t (x) = x W I q(x) + (1 x W ) I p(x + t g ) (5) Fig. 8. Illustration of the warping scheme Image Q boundary Transition Region Image P x Fig. 9. Gray value assignment of the boundary between two images 5 Experimental Results To capture fingerprint images We used 4 enrollment schemes as follows 1. A image for each finger enrolled by using the dab approach

9 Fig. 10. the mosaicked image with a fingerprint sequence Table 1. Mosaicking Success Rate # of images our algorithm Random Optimum A mosaicked image with several images enrolled by using the dab approach [2]. 3. A mosaicked image with a sequence of images enrolled by rolling a finger on a large sensor [3]. 4. A mosaicked image with a sequence of images enrolled by our enrollment scheme. We enrolled 100 fingers through 4 enrollment schemes. For 1st and 2nd enrollment sheme we capture 1000 images (10 images per a finger) and for third one we capture 100 rolled images (an image per a finger) and for the last one we capture 100 sequences (a sequence per a finger). The number of images belong to a sequence becomes different by each person. We use the ACCO 1394 sensor whose image size is with 500 dpi resolution. For 1st, 2nd and 4th enrollment shemes, we clipped the center region of an image whose size is Instead of using a small sensor, we used the ACCO 1394 sensor because its frame per second is enough to acquire a sequence enrolled by our enrollment scheme. We compare the mosaicking success rate of Lee s algorithm [2] and ours as shown in Table. 2. The success rate of Lee s algorithm varies according to the base image to which the algorithm align other images. Lee s algorithm has no method how to select the base image among several images, so we select an image among 10 images as a base image and align others to the image. We execute this procedure 1000times (10 times per a finger) by changing the base image. In Table. 2, # of images is the number of images successfully mosaicked and Random is the mosaicking success rate when we select a base image randomly and Optimum is the success rate when we select a base image to align others as well as possible. The success rate of our algorithm is independent of the base image, becase our algorithm aligns the other images to the first enrolled image from a sequence. To show that our enrollment scheme can acquire a wider area of a fingerprint than

10 Table 2. The size of foreground area and the number of minutiae enrollmentscheme our algorithm Random Optimum the dab approach, we measured the size of the foreground area and the number of minutiae from an image enrolled by the dab approach, the mosaicked image of Lee s algorithm, an rolled image and the mosaicked image of our algorithm as shown in Table. 6 Conclusions Acknowledgements This work was supported (in part) by the Biometrics Engineering Research Center, (KOSEF). References [1] A.K. Jain and A. Ross, Fingerprint mosaiking Proc. International Conference on Acoustic Speech and Signal Processing(ICASSP), vol. 4, pp , 2002 [2] Dongjae Lee, Sanghoon Lee, Kyoungtaek Choi and Jaihie Kim, Fingerprint fusion based on minutiae and ridge for enrollment LNCS on Audio-and Video-Based Biometric Person Authentication, vol.2688, pp , Jun [3] N.K. Ratha, J.H. Connell and R.M Bolle, Image mosaicing for rolled fingerprint construction Pattern Recognition, Proceedings. Fourteenth International Conference on, vol.2, pp , Aug [4] Lin Hong, Yifei Wan and A.K. Jain, Fingerprint Image Enhancement Algorithm and Performance Evaluation IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.20, No. 8, pp , Aug [5] NK Chern, PA Neow and MH Ang Jr, Practical issues in pixel-based autofocusing for machine vision Int. Conf. On Robotics and Automation, pp , 2001 [6] Yong-Sheng Chen, Yi-Ping Hung and Chiou-Shann Fuh, Fast block matching algorithm based on the winner-update strategy IEEE Transactions on Image Processing, vol. 10, No. 8, pp , Aug [7] George Wolberg Digital image warping IEEE Computer Society Press, 1988