CHAPTER - 2 LITERATURE REVIEW. In this section of literature survey, the following topics are discussed:

Transcription

1 15 CHAPTER - 2 LITERATURE REVIEW In this section of literature survey, the following topics are discussed: Biometrics, Biometric limitations, secured biometrics, biometric performance analysis, accurate biometric system, multiple modalities, multimodal biometrics, fusion levels, fusion strategies, fusion scenario, Palm as a biometric, palm feature, palm feature extractions methods, edge detection methods, Face as a biometric, face recognition system, face detection, face feature extraction, approaches, classifiers, multi resolution methods, feature level fusion, feature compatibility and approaches. Biometrics is a Greek term defining bios as life and metric as a measurement. Biometrics is basically a pattern recognition system using human characters recognition. History says that astrologers studied palm prints in predicting the future. Finger prints were used in olden days as a mark of authentication of a document. Behavioral characters like signature are used from olden days to identify a person. With the growing technology and rapid growth in applications of various fields like communication, networking, banking, and highly secure applications like criminal detection, forensic, military security, we can see tremendous growth in the field of biometrics to fulfill the demands of get reliable, cost effective, user friendly system. Anil K Jain [1] introduces the biometric recognition which works in verification mode or identification mode. Identity of a person is

2 16 established by comparing the input data with the stored template data. Author gives the basic biometric system operation modes as sensor level which captures the input data, feature level which extracts the features of collected data, match score level which estimates the degree of matching between input data and stored data, and a decision level which decides to either accept or reject the input. Author presents an overview of errors that could occur in the biometrics as false match and false mismatch. Biometric limitations like noisy sensor data, inter and intra user variations, spoof attacks, etc., are presented. A brief review on multimodal biometrics is given. The paper gives an overview of the biometrics. Anil Jain [10] presented biometrics as a promising frontier for the identification. He describes biometric can be knowledge based like pass words or token based like ID cards. A comparison of face, finger print, hand and iris based on universality, acceptability, permanence, uniqueness is presented as a case study and summarized that biometrics will likely be used in almost every transaction. Even years after the author s observation, biometrics has found its place in every transaction of a modern world. Publications related to biometrics are found in the literature from 1998 onwards and later within a decade a tremendous growth in research publications are seen. James Wayman [4] describes taxonomy of uses, various issues like performance of a biometric system. This paper gives applications,

3 17 taxonomy of uses, habituated vs non-habituated, public vs private, open vs closed, classification of applications, system model of biometric system. R. Brunelli [7] in his patent presented an integrated multisensory recognition system using acoustic and visual features for person identification. Integration of multiple information was a key issue in implementation of a reliable system. The work was carried out using acoustic and visual features of a person for identification. The speaker and face recognition systems are decomposed into two and three single feature classifier respectively. The resulting five classifiers produce non homogenous scores which are combined using different approaches. The speaker recognition is based on vector quantization of the acoustic parameter space. Face recognition is based on comparison of facial features at the pixel level. Integration of information, using multiple classifiers is considered as learning task. R.Michael McCabe [16] presented the standards for identifying biometric accuracy in 2003.In his work carried under NIST (National Institute of Standards and Technology),facial recognition vendors test, finger print verification etc., were conducted using image based biometrics. Standards were set and it is observed that not all subjects can easily finger printed and 2 to 3.5% have damaged friction ridges. This work gave a platform to indicate that a dual biometric system including more images may be needed to meet the existing system

4 18 requirement. Biometric performance analysis was also presented in the work of [11] [15] [17]. Marco Gamassi [19] presented a critical analysis of the measurement of accuracy and performance of a biometric system. He compared and criticized the current approaches stating that the performance under larger databases must be tested. He also stated that second source of uncertainty which effects the overall accuracy should be considered. It is also observed that accuracy depends on how information of the biometrics features is used, but not on the complexity of the design. The author also presents typical accuracy indices like symmetry, asymmetry, matching scores, false match rate, false non-match rate etc. P.J.Phillips [12] presented tools and techniques for biometric testing. The performance limitations that are nearly impossible to work around was analyzed and working towards the multiple biometrics for performance improvement was stated in his work. Li Hong [23] in his work presented the architecture of integration of face and finger prints where a case study was performed using score based recognition. The integration system retrieves first top five matches for face recognition. Finger print verification was applied to each of the resulting top five matches and final decision is made by decision fusion scheme. Experiments are conducted on a small database with 64 individuals and it is presented that an integration scenario gave better results compared to finger or face taken separately.

5 19 Biometric recognition system works in four levels namely sensor level, feature level, match score level, and decision level. U. Dieckman [33] proposed a person authentication system SESAM in which three different modalities namely sound of voice, lip motion, and fixed image of face are used as cues in identifying a person. The system uses three different sensors. Optical cue are recorded with grey level CCD camera (768x572 pixels). A second video recording is triggered by acoustic signal so that voice and lip movements are extracted. Each cue is recorded separately and preprocessed independently. The data is trained by three separate classifiers. The work presented includes multiple classifiers and multiple sensors. The drawback of the implementation was the large storage space of learning patterns. A.Ross [22] provides the information of fusion scenario of biometric system. Biometric system limitations such as noisy sensor data, spoof attacks, inter class similarity and intra- class variations can be reduced using fusion of information. A.Ross [22] presented the fusion in the context of biometric as single biometric multiple samples, multi biometric samples, multi classifiers and multiple approaches. A.Ross [22] presented that fusion in match score level increases the performance of the biometric system. Finger print and face data are obtained from 50 users with 5 samples each to generate 500 (50x10) genuine scores and are compared with imposters. Sum rule is used to find the weighted average of the final score.

6 20 Decision tree and linear discriminate methods were used to compare it with sum rule. User specific applications with widely acceptable biometric character selection were addressed. Ross and Jain [21] gave an overview of multimodal biometrics presenting levels of fusion, fusion scenario using multiple sensors, multiple classifiers and multiple approaches. Integration strategies such as feature level fusion, match score level fusion, rank level fusion and decision level fusion are presented. Multimodal biometrics addresses several limitations of unimodal system. The performance of unimodal can be improved with the integration of multiple source of information. Researchers presented various papers related to application and implementation of multimodal biometric system in their work [30] [31] Some of the fusion level implementations contributed for biometric system are: D.Kisku [60] proposed feature level fusion of face and ear. S. Prabhakar [29] presented decision level fusion in finger print verification, Kittler [35] presented combining classifiers for decision level fusion, P.Verlinde [32] on comparing decision fusion using KNN classifier, decision trees and logistic regression, E.Bigun [30] with multimodal system using Bayesian statistics and presented to increase the decision reliability. Researches worked on various levels of fusion and literature finds that match score level fusion was the most studied and

7 21 presented with different approaches. Feature level fusion is an understudied problem with limited publications. Feature level fusion involves the fusion of features extracted from the multiple biometrics. The features are to be represented in a feature vector space. The salient features of a biometric modality are extracted and fused. Fusion at feature level involves consolidation of features of multiple biometrics and representing in a common domain. Dimensionality of the fused vectors has to be reduced to suit the representation. Selecting the biometric traits to suit the requirement is a challenging task. As a result, feature level fusion is still an ongoing research issue for multimodal biometrics. Selection of biometric characters for feature level fusion is based on the condition that the fusion should satisfy the condition of resolution, localization, sampling directionality and anisotropy. Feature level fusion involves, identifying the region of interest, preprocessing, extracting the features of multiple biometrics, concatenating the multiple features in to single feature set, identifying the vector space to represent the larger feature set, dimension reduction of larger feature sets and finally storing the resultant feature as a template in the database. Stored templates are unique. Selecting biometric characters for feature level fusion involves two constraints Compatibility of feature set Dimensionality

8 22 Compatibility of a feature set is based on selection of biometric character to be fused. In the present thesis work, palm and face are chosen as biometric characters whose features are compatible and can be used for biometric fusion. Palm print based biometrics is one of the low cost biometric systems used. Palm print has rich information like principal lines, textures, ridges, minutiae and is a user friendly biometric character, which are easy to obtain from a low cost sensor. It is used in both unimodal and multimodal biometric system. Zhang et al and Han were the first research team developed CCD based palm print scanner for Hong Kong polytechnic university. CCD based scanners align the palm print and capture high quality image in a controlled environment. Palm image in a more hygienic environment was captured by contactless sensors and are used for person identification [145]. Digital cameras and video cameras were also used as palm image sensors but were found to have low quality image. X. Wu [41] presents a novel approach for palm line extraction for on line palm prints. It was presented in two stages, coarse level extraction and fine level extraction. Morphological operations are used to extract palm lines in different directions in first stage and for each extracted line a recursive process is used to extract further features. Palm lines form one of the most important features for palm print

9 23 recognition. Palm lines are bold and thick compared to other regions of palm surface. A. Kumar and D. Zhang [42] investigated the performance of bimodal biometric system using fusion of shape and texture. To improve the performance of the hand shape user authentication Discrete Cosine Transform coefficients for feature extraction of palm is used. Score level fusion was used to fuse the hand shape and palm features scores. Product rule is used for calculating the match scores of palm and hand shape. The results are compared with sum and max rules. Application of multiple approaches were used and compared against their performance. Application of various approaches used by researchers enhanced the performance of palm print recognition system. Approaches like line based, sub space based and statistic based were used to extract palm features. [37][38][39][40][41]. Researchers proposed combination of palm print for better recognition than a single palm print. A. Kumar and D.Zhang [45] proposed a new approach where simultaneous use of three approaches was used for better results. Gabor, line and appearance based palm print representations with score and decision level fusion was used. Integrating shape and texture was also proposed by the author using a common sensor palm and hand geometry features are acquired from the same image and at the same time. Features are examined for individual and combined performance.

10 24 Template security and database attacks was addressed by various researchers A.Kong [48] addresses the three relevant security issues like data attack, replay attack, template reuse attacks. The paper proposes random orientation filter bank as a feature extraction to generate noise like feature codes called competitive codes. Secret messages are hidden in template to prevent database attacks. Author proposed palm print recognition system based on competitive code for template security. The palm print images collected from the scanner are aligned to suit to the coordinate system and pre-processed for further feature extraction and template storage. Competitive codes are generated and hidden in the templates to prevent replay and database attacks. Authors in their work also addressed the secured measures for template storing [87], [28]. Multi resolution methods and its application in pattern recognition and image processing gave promising results. Multi resolution analysis tools, more popularly wavelets have been found quite useful in analyzing the information content of images. Zhang [145] used complex wavelets to decompose palm print images and proposed complex wavelet structural similarity (CW-SSIM) index for measuring the local similarity of two images. The overall similarity of two palm prints is estimated as the average of all local modified CW-SSIM.

11 25 Zhou [50] used wavelet to decompose palm prints and used support vector machine as a classifier. The input of the support vector machine is low sub band images. This approach lacks the information in middle frequency spectrum. Chen et al [52] proposed two dimensional dual tree complex transform on the pre-processed palm print image to resolve the weakness of traditional wavelet transform like shift invariant. Symbolic aggregate approximation is used to represent the features and minimum distance to compare the two feature vectors. The initial work of feature level fusion of palm print was proposed by Han [93]. Author extracted seven specified line profiles from a pre-processed palm prints and three fingers and used wavelets to compute low frequency information. A new feature vector is formed using the information and the dimensionality is reduced using PCA. A learning vector quantization and optimal positive Boolean function are used to make final decision. Statistical approaches, transform approaches were used to extract palm print features. Using either Fourier transform or wavelet transform the directional features are not represented accurately. Contour transforms, complex wavelet transform were used to represent the directional property but has the limitation of larger dimensionality. Curvelet transform overcomes the various limitations set by other transform methods and its application to image processing is

12 26 being introduced recently. The present thesis work is to use an approach which can be used for face and palm print feature extraction and later use for feature level fusion. Face recognition is the most referred topic in the computer vision literature area for the past 30 years. Face recognition is still an active research due to its application in various human activity involved applications like face tracking, face surveillance, criminal detection, etc. Face biometrics is most preferred biometric character as it is non-intrusive and can also be taken without user knowledge for identification and verification. Face recognition system stages involves face detection, pre-processing, feature extraction and face recognition. The main goal of face detection is to find presence or absence of a face from a picture, video in surveillance. Automatic face recognition is to identify a given face image from the stored database. The limitations that are encountered are variations of given image with respect to the stored database image of a genuine user. Variations can be due to aging, pose, illumination, clutter in the background, occlusions due to accessories etc. Various approaches were proposed by researchers to overcome the limitations. Viola and Jones [107] has made face detection practically feasible to real world applications. The Viola and Jones face detector contains three main ideas that make it possible to build a successful

13 27 face detector that can run in real time.i.e., integral image classifier learning with Ada Boost and a cascade structure. Face detection methods in literature survey shows that they are classified into two groups, knowledge based and image based. Knowledge based methods uses facial features (like shape of face, eyes, eyebrows) template matching, skin colour etc. Many detection algorithms are based on facial features. HI-fang [129] detected face and facial features by extraction of skin like region with YCbCr colour space and edges are detected in skin like region. Eyes, mouth are found with geometrical information. Researchers proposed various approaches using skin colour to detect the face features. [130] Human face detection using template matching was proposed by various researchers in the literature study. Chen et al [131] used half face template instead of full face template to reduce the computation time. The template based methods compute the correlation between a face and one or more model templates to find the face identity. Brunelli and Poggio [126] suggested that the optimal strategy for face recognition is holistic and corresponds to template matching. The author compared geometric feature based technique with template matching based system and achieved a very high accuracy. Principal component analysis (PCA), Linear discriminate analysis (LDA), Neural networks, kernel methods were used to construct suitable set of face templates [76] [77] [81] [85]

14 28 Geometric feature based methods analyze explicit local facial features and their geometric relationships. Yuille proposed a shape model for face image and later the extension of work can be seen by Cortes et al [110]. Wiskott [95] was first to develop elastic bunch graph model and application of PCA into local features was developed by Penev et al [96]. The work carried out by these researchers gave a basis for many applications and development in the face recognition and image processing. Phillips [84] presented FERET evaluation tests in his work and the main objective of the test was to assess the state of art, identify the future area of research and to measure the algorithm performance. Later FRVT 2000, FRVT 2002, FAT 2004, were conducted to address age, pose, illumination variations and standards were set for future research. Extraction of facial features is an integral process involved in face detection; face modeling, face recognition, animation and facial expression determination. Featured-based approaches first process the input image to identify, extract and measure distinctive facial features such as the eyes, mouth, nose, etc. and then compute the geometric relationships among those facial points. Standard statistical pattern recognition techniques are then employed to match faces using these measurements. H. Schneiderman and T. Kanade [127], work which extracted 16 facial parameters, used Euclidean distance measurement for

15 29 matching. The accuracy achieved was 75%. Brunelli and Poggio [121] extended the Kanades work with 35 geometric features with a 90% recognition rate. Li and Yin [140] introduced a system in which a face image is first decomposed with a wavelet transform to three levels. The Fisher faces method is then applied to each of the three low-frequency subimages. Then, the individual classifiers are fused using the RBF neural network. Melin et al. [141] divided the face into three regions (the eyes, the mouth, and the nose) and assigned each region to a module of the neural network. A fuzzy Sugeno integral was then used to combine the outputs of the three modules to make the final face recognition decision. They tested it on a small database of 20 people and reported that the modular network yielded better results than a monolithic one. Multiple classifiers were used and their information was fused to give more accurate performance. Lu et al. [138] fused the results of PCA, ICA and LDA using the sum rule. Bayes combination rule to integrate the weighted outcomes of three classifiers based on frontal and profile views of faces. Wan et al. [142] used SVM and HMM hybrid model, Kwak and Pedrycz [143] divided the face into three regions, applied the fisher faces method to the regions as well as to the whole face and then integrated the classification results using the fuzzy integral, Haddadnia et. al. [144] used PCA, the Pseudo Zernike Moment

16 30 Invariant (PZMI) and the Zernike Moment Invariant (ZMI) to extract feature vectors in parallel, which were then classified simultaneously by separate RBF neural networks and the outputs of these networks were then combined by a majority rule to determine the final identity of the individual in the input. Principal component analysis which used to represent Eigen faces with a lower dimension suffers from computational load and correlation of facial features. Image representation should satisfy the condition of multi resolution, localization, critical sampling, directionality and anisotropy. Multi resolution analysis tool wavelets, have been found quite useful in analyzing the information content of images. Wavelets have been successfully used in image processing and its ability to capture localized time-frequency information of image motivates its use for feature extraction. Wavelet-based methods focus on the sub bands that contain the most relevant information for better representation of the data. The wavelet transform can be interpreted as a multi scale differentiator or edge detector that represents the singularity of an image at multiple scales and three different orientations as horizontal, vertical, and diagonal. Zhang et al. [137] proposed a modular face recognition scheme by combining the techniques of wavelet sub band representations and kernel associative memories. By the wavelet transform, face images are decomposed and the computational complexity is substantially reduced by choosing a lower spatial-frequency sub band image. Then

17 31 a kernel associative memory (KAM) model is built up for each subject, with the corresponding prototypical images without any counter examples involved. Multi class face recognition is thus obtained by simply holding these associative memories. When a probe face is presented, the KAM model gives the likelihood that the probe is from the corresponding class by calculating the reconstruction errors or matching scores. Various researchers implemented wavelets as feature extraction method. [139] [140]. Wavelet transform has been found quite useful for analyzing the information content of an image, has the limitations of identifying only the point singularities in an image. The edges and curves are not well defined by the wavelet transform. Application of ridge transform, contour transform was able to give information of edges and curves but suffers with the dimensionality problem of representing the coefficients. Curvelets which is a multi-scale, multi-resolution transform overcomes the limitations of wavelets and provides optimal representation of objects with curve singularities. Curvelets requires relatively small number of coefficients to represents a line or a curve in a given image. Literature study shows that application of Curvelet transform was a recent introduction to biometric face recognition. Its application to multimodal fusion is still under research. Selecting a biometric character for multimodal fusion depends on various criteria. The level of fusion selected must be compatible for

18 32 integration of information from the different biometric traits. Multimodal fusion is categorized as sensor level, feature level, match score level and decision level. Feature level fusion is an understudied problem by researchers due to the following limitations Compatibility of feature sets of different biometric characters Large dimension of feature set of fused data The present research work was carried out by addressing the initial problem of selecting a biometric character, which are compatible in their feature representation. Palm print with its rich information and face as a universal biometric identifier are used for fusion. Literature survey shows that fusion of palm and face under feature level is a under studied problem. Fusion under match score level was used extensively by researchers with different approaches. D.kisku [61] proposed feature level fusion of palm and face by isomorphic graph based K-medoids partitioning. Partitioning the palm and face images with K invariant SIFT points, forming number of clusters. On each cluster an isomorphic graph is drawn. The most probable pair of graph is searched using iterative relaxation algorithm from all possible graph pair of face and palm print images. Graphs are fused into augmented groups using concatenation rule. The accuracy of the system is very high when compared with other level of fusion. [89][119]

19 33 D. Kisku [146] proposed feature level fusion of face and finger prints by extracting features independently from two modalities and making two point sets compatible for concatenation. Feature reduction techniques are employed prior and after feature set fusion. Comparative experiments were also conducted with match score level fusion. Face features were extracted using SIFT and fingerprint verification on minutiae matching and K means clustering was employed in feature reduction. The proposed technique was also tested on different databases. FAR, FRR and accuracy obtained was compared with unimodal, multimodal and after using reduction techniques. [67][86] Yucheng Wang [90] proposed feature level fusion of palm and face based on kernel fisher discriminate analysis (KFDA) and Linear discriminate analysis (LDA). The results showed that multimodal fusion had a higher accuracy when compared with unimodal. [64] Anil Jain and Ling Hong [71] used integration of face, finger print and speaker verification in making personal identification. The fusion level used was decision level. Face recognition was implemented using Eigen faces. The speaker recognition was based on text dependent speaker recognition. The minutiae of ridges were used for extraction and matching. The aim was to show that identity established in multimodal system is more reliable than using a single modality alone. The system was tested only on a small data set and its performance for a larger data set was not tested. [75]

20 34 Raghavendra R [113] Introduced Particle swam optimization for feature level fusion of face and palm print. Face and palm print features are represented using Log Gabor features which are then concatenated to form a fused feature vector. PSO is used to compute weights for each of the features qualitatively. Binary PSO is used to select most discriminate features from fused features. The feature level was compared with match score level. The proposed scheme outperformed the state of art scheme. By studying and analyzing the literature the main requirement of a biometric system is summarized as Selection of a biometric trait depends on the application. Biometric system performance is not optimum. Biometric system has limitations. Using multimodal biometrics the performance accuracy can be improved. Multiple biometric characters selected must be compatible in fusion level used. Performance and accuracy of a biometric system can be improved using multiple approaches. At the end of literature it was found that Application of multi scale edge detection methods were not used for palm print feature extraction. Application of Curvelet transform for feature level fusion of palm and face was not found in the literature.