MULTIMODAL FRAME WORK FOR FUSING FINGERNAILS AND FINGERPRINTS PATTERN

Transcription

1 MULTIMODAL FRAME WORK FOR FUSING FINGERNAILS AND FINGERPRINTS PATTERN Kumuda N S, Dinesh M S University of Mysore P E T Research Center, P E S College of Engineering Mandya, Karnataka, India Abstract Biometrics is the central source of information in most places to help person recognition. It can be included in a variety of technology to identify person's unique attributes and to categorize. We have designed a system to fuse information from multiple modalities such as fingernails and fingerprints to achieve higher recognition rate and to provide more security. The demand for more attribute combination and learning becomes necessary from time to time to establish strong identity. In this work, fusion of fingernails shapes along with fingerprint patterns is suggested as array of multi-modality for person identification. The proposed recognition system consists of three stages. In stage 1: Extract nail plate regions from multiple images of fingernails and extract ridge patterns from multiple scanned fingerprints. In stage 2: Identify nail plate shapes and identify fingerprint patterns for all ten fingers of individuals. In stage 3; The nail plate shapes features and ridge patterns features are combined to generate strings of patterns that belongs to individual and stored in system memory and used for recognition. While generating the string pattern we followed the sequence starting from left hand thumb finger through little finger and right hand thumb finger through little finger. The experiment is carried out on training set of 300 samples, test dataset of 100 samples and unseen dataset of 100 samples. The recognition analysis of test dataset resulted in 96% accuracy while recognition analysis of unseen dataset resulted of 94% accuracy. The results are encouraging to consider nails as multimodal in combination with fingerprint biometrics. Keywords Multimodal, fusion, fingernail shapes, fingerprints patterns, string generation, correlation similarity. I. INTRODUCTION Multiple biometric sources are popular because they are able to address the limitations like; user acceptance, nonuniversality, imaging noise, high intra-class variations and vulnerability to imitation which are very common in single biometric systems. The most popular biometrics used are person's fingerprint, iris, face, written signatures, etc., [23]. Multiple of them are used in different combinations as composite identification measures to improve system efficiency, stability and quality in person detection and recognition. The fusion methodology of emerging biometrics in different combinations such as; face and fingerprints with palmprint [2], hand shape, ear, knuckle, or iris [25], are available both in literature and in practice and are commercially in demand. The government of India has used face-fingerprints-iris-signature features in combination to issue Adhaar cards as Unique Identification since Sep but did not get implemented completely till The practical requirements in Iris Recognition and to combine multiple biometrics is given [7], which made strong influence on our proposal of fingernails shape and fingerprints pattern as multi-modality. Most of the expected automatic systems for fingerprint comparison are based on minutiae matching [1]. Minutiae characteristics are local discontinuities in the fingerprint pattern which represent terminations and bifurcations [13]. the majority of the minutiae detection methods proposed in the literature [21] are based on image binarization, while some others extract the minutiae directly from gray scale images [17]. A ridge termination is defined as the point where a ridge ends abruptly. The ridge structures in fingerprint images, sometimes appear as smeared. Therefore algorithm has to enhance necessarily ridge structures clarity [22]. The work [11] suggests some modification to Hong s Gabor-based technique. For a fast and direct grayscale fingerprint enhancement, an algorithm based on a unique anisotropic filter is suggested [9]. Morphological or binary ridge operations for filtering noise are considered as important procedure in literatures of fingerprints study, were are all referred and have influenced the research work [19]. Additional proposals [8, 9] is based on modified anisotropic filtering technique is implemented on fingerprints to segment the patterns for recognition. There is increase in the number of research works carried out on fingernails, since Latest literature survey indicate fingernail as biometric identifier in the following: finger nail surface is used for biometric authentication [26]. The finger knucle and fingerprints are used as multimodal biometric system [27]. The finger nails are used as transient biometrics [28]. The finger nail plates are proposed as new biometric identifier [20]. The geometric property of finger nails is suggested to find person match using fuzzy measures 246

2 [13], and the biometric authentication using finger nail plates is also proved [3]. The attempts made to study fingernails have proved that there is further scope for research. The fingernails extraction for identification are more inclined towards colour structure formation [24]. There are very few literatures found on study of extra growth of distal free nail (free edge of nail plate) that continues to grows in every 7-15 days time in mm as explained [29] and is generally trimmed off. The different nature and shape can be clipped off automatically as proposed", [16, 18] in "Human Fingernail Segmentation. The clipped nails were studied for composition and for blood metabolites presence [30]. The commercially available Biotech systems can capture multiple modalities in various combinations for person identification such as; (a) hand shapes with face, (b) iris with fingerprint, (c) face, fingerprint with iris, (d) Hand veins and fingerprint and are successfully put into practice. Additional reasons for multimodality are; to overcome hacking, to avoid spoofing or network related threats and to improve recognition from noisy images. The simplicity of fingerprints pattern are extensively used in different categories and therefore used as biometric identifiers in various applications like confirmation, validation and identity check of individuals. The multimodality biometric combinations are prepared at data acquisition level, feature level, as well as at match score level. Latest methods in multimodality bring in balance between maximizing the number of matches and minimizing number of features. To generate the absolute similarity score and distance measure between query and reference fingerprints to check minutiae match in the overlapping areas, a fully connected Neural Networks (NN) is proposed [21]. In this work, we propose a new fusion technique to combine fingernails and fingerprints as identifiers at feature level to create unique string pattern for every individual and measure correlation similarity for match score (maximum) identification. The suggested design is to extend fingerprint patterns with fingernails shape properties in an inclusive biometrics which can be put into use in the present day multimodality devices. Fingerprint patterns generally appear like array of biometric collection on palmer side, while the fingernails [32], appear in different shape patterns on distal side for equivalent fingers of human hands. Digital images are always associated with noise, and appear prominently at random locations in fingerprint images when captured (manual and sensor levels). Person character understanding and extraction algorithm relies heavily on the quality of input fingerprint images (no dirt or sweat). The contemporary sensors have units which attempts to determine whether a finger is live or synthetic. Detectors to measure body temperature, blood-oxygen level, pulse rate, blood flow, humidity, or skin conductivity can also be integrated. The, image enhancement is a essential preprocessing step for digital pattern study, experimentation [12], and for online feature extraction operations of the identifiers. In case of 3-D fingernails acquisition, shadow and glare effects appear as unwanted or noisy information. This nature of biometric objective insists that the data has to be captured with careful settings, e.g., the top view, front view, complete view, without shadow or glare effects of the, etc. As a result there is a need to develop much faster enhancement algorithms which can comparatively [10], improve the quality of biometric images The paper is organized as follows: (a) Creation of dataset of fingernails and fingerprints for ten people, all ten fingers having five samples each, a total 500 views, (b) Present an algorithm to extract of features from nail plate shapes and fingerprint patterns, (c) To generate string sequence separately for fingernail and fingerprints formations, (d) And finally calculate correlation similarity, variance measures between training and test/unseen samples for person recognition and its accuracy analysis. II. METHOD A. Mltimodal Biometric Multimodal biometrics are popular and have proven their importance by improving system performance. In order to make learning algorithm more simpler and acceptable than the existing ones, we propose multiple along with multiple fingerprints as a string combination of attributes. When the two features are practically combined and empirically evaluated would give favorable results because multiple fingernails have intra-class similarity with more correlation as compared to fingerprints need not necessarily have high correlation between the corresponding fingers. B. Dataset Creation The biometric data for experimentation is created from ten persons as images information. The multimodality data is collection as follows: ten people, five fingerprints samples were collected both on hard copy and from server connected to fingerprint scanner units as in Fig 1. (a,b). Similarly ten people fingernail images in five different views were captured from digital cameras Fig. 1 (c,d) and stored in folders. a) b) 247

3 c) d) Fig. 1. Existing biometric scanners [6]; a) Single fingerprint unit, b) Multiple fingerprints unit, c) Fingernail capturing system, d) Capture finger images from laboratory setup of (c) Table 2 : Dataset for Multimodal Fusion of Fingerprints and Fingernails Dataset Samples for Training, Testing and Unseen Created Dataset 10 persons x 10 fingers x 5 samples 500 The Dataset was Divided into Following for Experimentation Training Dataset 10 persons x 10 fingers x 3 samples 300 Test Dataset 10 persons x 10 fingers x 1 samples 100 Unseen Dataset 10 persons x 10 fingers x 1 samples 100 C. Ground Truth The ground truth information is extracted from fingerprints obtained. The pattern formations are examined by subject experts and observations are stored in memory as hands-on data. Similar procedure is followed for collecting nail plate shapes through boundary area selection as region of interest by expert's observation and knowledge. Ground truth of nail plate area is also stored as different hands on data. D. Fingerprint Patterns Fingerprint are rich in information like minutiae, ridges and details in valley arrangement [4] with different curvature turnings. There are nine recognized pattern formations of fingerprints, that are used as symbols/characteristics Fig. 2 (a, b), to identity a person if captured properly. a) b) Fig. 2. a) Parts of palm side of hand, b) Nine fingerprint patterns found on palm side of hands. [ Fingerprints are generally in binary format either scanned from light emitters or hard copies. Fingerprints are the most widely used biometrics; fingernails are just getting into the field of research as biometric identifiers. Fingerprints have following main types of features; (i) Global ridge and furrow structure are visible at the central region of the fingerprint and (ii) Minutiae details are associated with the local ridge and furrow structure. (iii) Fingerprint are proven to have strong texture character, ridge distance as attributes. The Fourier transform as a spectral analysis method is tested on fingernails spectrum, to find the possible thickness of the ridges along with separation between ridges as much as possible. E. Fingerprint Features The survey indicates fingerprints as most established research work which is empirical proven with higher percentage of recognition rate and preserves the reliability by circumventing the spoofing of information on networks. The fingerprints are establishes as popular biometric tools in recognition with proven features as listed in Table 3. Table 3 : Established Fingerprints Features for Recognition No. Established Fingerprint Properties 0 Isolated Point -0 [Bhanu B. X. Tan, 2003] 1 Ending Point -1 2 Connective Point -2 3 Bifurcation Point -3 4 Crossing Point Thickness of ridges -5 Reference Point Detection -6 [Jiang X.D.,et al, ; Liu M., 2005] Attributes 1-5 are proposed by many authors. Fingerprint information that gets stored in server connected to network of scanner machines were extracted for ridge analysis and patterns formations were labeled as specified in Table 4. The fingerprints patterns were collected for ten individuals are maintained in the memory as first set of biometric data. Fingerprints were captured both electronically and manually. The electronic dataset generated is in binary format while the ink impressions on white paper are pictorial patterns. From the electronic device we could register the fingerprints of all ten fingers with some amount of finger pressure as the device is manufactured for touch sensitivity. It registered all subjects correctly with first ten finger impressions from two hands. Live-scan fingerprints are in use in educational institutions to mark the attendance of both employees and students, which captures fingerprint images as similar to document scanning method (OCR). F. Fingernail Shapes The fingernails data is created for same group of ten people who presented fingerprint biometric section just explained. The camera systems can easily capture image objects from multiple direction, without human involvement. It can also address the memory requirements within the system by storing images in low resolution. With commercially available camera timer set, it can capture multiple view images of fingernails. Fingernails are segmented, extracted nail plate area is measured for shape estimation and features stored as labels. Nail plate colour as spectral information [24] and 248

4 region based measures area predictor were augmented for structural feature analysis (spatial). Fig. 3: Nine types of fingernails shapes that can appear on the different nail plates of hands [ The recognition analysis have proved that the experimental results were strong enough to identify individuals. Comparative analysis with ground truth for confirmation and high accuracy has established a possibility. The results of finger shape recognition of 87.5% for test dataset was reported [16] when tested for 200 image samples. When the same algorithm was extended and implemented with convex hull boundary corrections on 600 finger [Thumb-T, M-middle, Little-L] images for person recognition, the recognition had improved accuracy of 91%. G. Algorithm An algorithm proposed to fuse multiple biometric features is as follows: Step 1. Acquire multiple fingerprint samples from biometric scanner. Acquire multiple finger image samples from digital camera. Step 2. Extract fingerprints patterns at feature level, e.g. W-whorl, LL-left loop, etc. Apply proposed segmentation technique to extract nail plate shapes at feature level e.g., BC-broad circular, BR-broad rectangular, etc. Step 3: Combine fingerprint patterns and fingernails shapes to generate string patterns starting from left hand thumb to little finger and right hand thumb to little finger as shown in Table 4. In step 4. Compute similarity of the generated patterns from step-3, with set of individual's patterns stored in the memory. Maximum similarity pattern identified is used for person identification and verification III. EXPERIMENT AND RESULT Presently there is not much growth in fingernails feature extraction that can influence the subject methods. This clearly indicates scope for different feature extraction methods and application areas. There is a need to address implementation of fusion prospects of fingernails with fingerprints. The current examination of fusion of array of multiple features method is seen as testing procedure. The features are converted to string patterns and most algorithms consider speed and efficiency during string search. A. Fingernail and Fingerprint as Heterogeneous Features The purpose is to find the ridge patterns of all ten fingers to avoid any threat of damage to structures. Identification requires method to check query images/features for the available dataset in the system to find correct or best match. Manual checking is time consuming and more demanding even for moderate to large database sizes. There is a need to examine the patterns that were manually observed and patterns which are automatically identified, to search for minimum difference and maximum similarity for every view pattern captured, so that PN NP1 for small P1 << 1/N << 1 requirement. While searching in a stored string feature of length M of string length N. The identifiers need to be roughly estimated N times for more improved results to get equal chances (than one time automatic verification) against a false acceptance as shown in Table 4. Store the extracted features as array features in matrices. Make two tables/matrix of distinct feature sets; one for fingerprint patterns as string and another for fingernail shape as string. Create the combination of pattern string by alternatively recording the individual features from each set as given in Table 4. Assess the string combination for similarity accuracy for person identification using array of multi-modality dataset created. Table 4: Pattern String Generation Combining Fingerprint Patterns and Fingernail Shapes B. Problem The algorithm is developed to check string pattern similarity and to prove the following requirements: String with small similarity like small letters to capital letters should not identified as dissimilar. any sub-string overlap (similarity) and non-overlap (dissimilarity) should be identified and copy of it should be stored in a directory format for every pair of strings. the algorithm needs to be sensitive to sequence of characters. even one character difference should be considered dissimilar. Generalization of the algorithm to check any type of string formation. C. Similarity Testing The most common form of pattern matching involves sequence of characters checking in strings. String matching has received both theoretical and practical importance in pattern recognition. It is also an important topic in combinatorial optimization for matching as referred to in literatures. The programming language Matlab and its string functions were used for testing the pattern sequence. Additional syntax were used to check expressions generated like (-) to describe the fusion of features. However, the well 249

5 accepted string matches are more suitable for similarity check on multi-biometric pattern generated [5]. The combination rule stating that both condition under consideration has to agree for acceptance. Reference to John Daugman's "Combining Multiple Biometrics" [7, 31] we carry similar principle in conditional check for identification. If set of ten fingernail features (as rule A) are correctly identified and another set of ten fingerprint features (as rule B) as indicated in Table 4., are also correctly identified with "AND" rule, it proves True Accept without doubt as given by Equation 1. Alternatively if 80 percent (or above) of both the features in pattern strings are correctly identified, we can use combination of the two tests 1 and 2. The False Accept can only occur if both tests 1 and 2 produce a False Accept independently as given by Equation 2-3. Thus the combined possibility of a False Accept, P B (FA), is the product of its two possibility for the individual tests: (clearly a lower possibility than for either test alone). P B (FA) = P 1 (FA).P 2 (FA) (1) P B (FR) = 1-[1-P 1 (FR)][1-P 2 (FR)] (2) or P B (FR) = P 1 (FR) + P 2 (FR) - P 1 (FR)P 2 (FR) ( 3) But the possibility of a False Reject when using this Rule, can be expressed as the complement of the probability that neither test 1 nor 2 produces a False Reject, is higher than it is for either test alone: The patterns can be concatenated as shown in Table. 5. If we consider the nail plate formation features and fingerprint patterns separately it would look as a) and b) and we can use probability of any one of the conditional sequences in fourth row. But if we combine the pattern sequence can become more complex as explained in Equation 1. D. Result Analysis The recognition results are as tabulated below in Table 5. When test samples were compared with trained samples, 96 samples for combined string format were identified correctly. Similar experiment when conducted with unseen string pattern 94 samples were correctly. Table 5: Experiment Result Showing Performance Measures of Fingerprint and Fingernail Biometric Pattern No. 1 2 Dataset for training and testing 10 persons x 10 fingers x 1 samples 10 persons x 10 fingers x 1 samples No of String Features Correctly Identified Percentage Accuracy samples 96% samples 94% The recognition accuracy of test sample set and unseen sample set are 96% and 94% respectively, which is encouraging to have fusion of patterns in sequence of combination. of fingernail string features and fingerprint pattern string features. IV. DISSCUSSION AND CONCLUSION When fingerprints fade away with age or when there is a scratch, it fails to recognition the test. So combining multiple identifiers would limit false rejection and support higher recognition. The fingerprints are highly prone to noise. The residual noise that exists even after pre-processing in higher resolution images requires correction. There is scope for better noise models in multimodality systems. The following conditions are recommended to model the dataset creation setup: fixing the location, placing the object either vertically or horizontally without unnecessary orientation, and to avoid lighting effects from different directions which causes shadow or glare effects. As a future extension of this work we have come up with an idea where effort can be made to device a capturing unit that can accommodate camera for fingernail imaging and scanner for fingerprint patterns. This will improve system performance by simultaneous capture of two biometrics for a larger number of individuals [15] by reducing time involved. The fingernails grow only at distal free edge creating extra nail length. But the pink nail plate region remains unchanged in shape throughout unless damaged by external injury. The nail plates embedded on nail beds cannot be changed, easily. There is no literature findings on imitation techniques for fingernails. Therefore fingernails as multiple biometrics are highly recommended for person recognition [32]. The work presented in this paper on fusion of fingernails is more challenging [14] as an emerging subject, gaining momentum in research as a biometric. The initial experimental results are encouraging for person identification. There is no publicly available dataset for research and therefore it is required to create fingernail dataset and in combination with fingerprints as proposed in this paper. The requirement of larger dataset creation with fingernails and fingerprints combination will be considered in further research. V. REFERENCE [1] Afsar F Fingerprint Identification and Verification System using Minutiae File Format: Real- Time Imaging Vol. 8, Issue 3. [2] Aggithaya V., et al., "A Multimodal Biometric Authentication System Based on 2D and 3D Palmprint Features", In Proceedings of SPIE, Vol C

6 [3] Amioy K., et al., 2014, "Biometric Authentication Using Finger Nail Plates", In Expert Systems with Applications, Vol. 41, Issue. 2.1, pp [4] Baruch O., 1988, "Line Thinning by Line Following", Pattern Recognition Letters, Vol 8, No. 4, pp [5] Benny Porat, et al., 2008, "Pattern matching with pair correlation distance", Elsevier Theoretical Computer Science 407, [6] Brian M. et al., "A Novel Region Based Liveners Detection Approach using Fingerprint Scanners", [7] Daugman John, et al., 1998, "Recognizing Persons by their Iris Patterns, in Biometrics: Personal Identification", In Networked Society, Amsterdam, Kluwer, pp [8] Greenberg S., "Adaptive Anisotropic Filter Applied for Fingerprint Enhancement". Submitted for Publication. [9] Greenberg S "Fingerprint Image Enhancement Using Filtering Techniques", Real-Time Imaging Archive, Vol. 8, Issue 3, table of contents, pp , ISSN: [10] Hollingum J., "Automated Fingerprint Analysis Offers Fast Verification," Sensor Review, Vol. 12, No. 3, pp [11] Hong L. and Jain A., 1998, "Integrating Faces and fingerprints for Personal Identification " In Proc 3rd Asian Conference on Computer Vision, pages , Hong Kong, China. [12] Hong L., et al., "Fingerprint Image Enhancement: Algorithm and Performance Evaluation," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 20, No. 8, pp [13] Kavita Jaba Malar R. and Joseph Raj V., 2014, "Geometric Fingernail Matching Using Fuzzy Measures", International Journal of Innovative Technology and Exploring Engineering (IJITEE) ISSN: , Vol. 4, Issue. 4. [14] Kumar A., and Zhou Y., 2009, "Personal Identification using Finger Knuckle Orientation Features", Electronics Letters,vol. 45, no. 20. [15] Kumar A. and C. H. Ravikanth. 2009, "Personal Authentication Using Finger Knuckle Surface", IEEE Trans. Info. Forensics and Security, Vol. 4, No. 1, pp [16] Kumuda N S and Dinesh M S., 2015, "Human Fingernail Segmentation", In Proceedings of ICERECT Conference, IEEE Computing Society, Mandya, Karnataka, India, IEEE Explore, ISBN [17] Mario D. and Maltoni, D., "Direct Gray-Scale Minutiae Detection In Fingerprints," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 19, No. 1, pp , [18] Runne U., et al., The Human Nail: Structure, Growth and Pathological Changes Current Problems in Dermatology 9, Pub Med: pp [19] Sang Keun Oh, et al., "New Fingerprint Image Enhancement Using Directional Filter Bank ". Journal of WSCG, Vol. 11, No. 1., ISSN WSCG. [20] Shruti Garg.et al., 2014, "Finger Nail Plate: A New Biometric Identifier," Proc in International Journal of Computer Information Systems and Industrial Management Applications. ISSN Vol. 6 pp [21] Tsai-Yang J. and Venu Govindaraju. 2005, "A Minutiae- Based Partial Fingerprint Recognition System", Pattern Recognition. Vol. 38, Issue 10, pp [22] Yang G. Z., et al., "Structure Adaptive Anisotropic Image Filtering," Image and Vision Computing, No. 14, pp [23] Yang M. H., et al., "Detecting Human Faces in Colour Images". In Proceedings International Conference on Image Processing, Vol. 1, pp [24] Zarit B. D., et al., "Comparison of Five Colour Models in Skin Pixel Classification," In Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp [25] Zhu Y et al 1999 Biometric Personal Identification System Based on Iris Pattern Chinese Patent Application, No [26] Garg S. et al,. 2012" Biometric authentication using finger nail surface''. IEEE, 12th ISDA, pp [27] Karbhari V. Kale, et al., 2013, "Multimodal Biometric System Using Fingernail and Finger Knuckle", International Symposium on Computational and Business Intelligence. [28] Barbosa I. B. et al., 2013, "Transient Biometrics using Finger Nails", In IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems, (BTAS). [29] Gita Rani gupta, et al., 2005, "Human Nail Growth Pattern and Medicolegal Aspect", JIAFM,:27(2)ISSN [30] Eugene Kanabrocki, et al., 1968, "Neutron -Activation Studies of Trace Elements in Human Fingernail", Journal of Nuclear Medicine, pp [31] Daugman, 1993 [32] 251