International Journal of Advanced Research in Computer Science and Software Engineering

Volume 3, Issue 2, February 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Level of Fusion in Multimodal Biometrics: a Review Dapinder Kaur Research Fellow Sri Guru Granth Sahib World University, Fatehgarh Sahib,Punjab Gaganpreet Kaur Asst. Professor Sri Guru Granth Sahib World University, Fatehgarh Sahib,Punjab. Abstract User verification systems that use a single biometric indicator often have to contend with noisy sensor data, restricted degree of freedom, non-universality of the biometric trait and unacceptable error rates. So the need of using multimodal biometric system occurred. A multimodal biometric system combines the different biometric traits and provides better recognition performance as compared to the systems based on single biometric trait or modality. In this paper, studies of different modalities are discussed and also discuss the various techniques used in different level of fusion with the objective of improving performance & robustness at each level of fusion. Keywords Multimodal Biometrics, level of fusion, rule based methods, estimation based methods, classification based fusion methods. I. INTRODUCTION A wide variety of applications require reliable verification schemes to confirm the identity of an individual requesting their service. Examples of such applications include secure access to buildings, computer systems, laptops, cellular phones and ATMs. In the absence of robust verification schemes, these systems are vulnerable to the wiles of an impostor. Credit card fraud for example, costs the industry millions of dollars annually, primarily due to the lack of effective customer verification techniques. Traditionally, passwords (knowledge-based security) and ID cards (token-based security) have been used to restrict access to applications. However, security can be easily breached in these applications when a password is divulged to an unauthorized user or a badge is stolen by an impostor. The emergence of biometrics has addressed the problems that plague traditional verification methods. Biometrics refers to the automatic identification (or verification) of an individual (or a claimed identity) by using certain physiological or behavioural traits associated with the person (Figure 1). Biometric systems make use of fingerprints, hand geometry, iris, retina, face, hand vein, facial thermo grams, signature or voiceprint to verify a person's identity. They have an edge over traditional security methods in that they cannot be easily stolen or shared. Biometrics refers to an automatic authentication of a person based on his physiological and/or behavioural characteristics. The usage of biometrics as a reliable means of authentication is currently gaining momentum, thou the industry is still evolving and emerging. The unimodal biometric recognition systems have to contend with a variety of problems and thus presently the amount of applications employing unimodal biometric systems is quite limited. Some limitations of the unimodal biometric systems can be alleviated by using multimodal biometric systems, which integrate information at various levels to improve performance. Fig. 1 Examples of some biometric traits The Multibiometric systems can offer substantial improvement in the matching accuracy of a biometric system depending upon the information being combined and the fusion methodology adopted as follows; Multi sensor: Multiple sensors can be used to collect the same biometric. Multi-modal: Multiple biometric modalities can be collected from the same individual, e.g. fingerprint and face, which requires different sensors. Multi-sample: Multiple readings of the same biometric are collected during the enrolment and/or recognition phases, e.g. a number of fingerprint readings are taken from the same finger. Multiple algorithms: Multiple algorithms for feature extraction and matching are used on the 2013, IJARCSSE All Rights Reserved Page 242

same biometric sample. Multi-instance: -which means the use of the same type of raw biometric sample and processing on multiple instances of similar body parts, (such as two fingers, or two irises) also been referred to as multi-unit systems in the literature. II. FUSION IN MULTIMODAL BIOMETRICS A biometric system has four important modules. The sensor module acquires the biometric data from a user; the feature extraction module processes the acquired biometric data and extracts a feature set to represent it; the matching module compares the extracted feature set with the stored templates using a classifier or matching algorithm in order to generate matching scores; in the decision module the matching scores are used either to identify an enrolled user or verify a user s identity. Sanderson and Paliwal [1] have classified information fusion in biometric systems into two broad categories: pre-classification fusion and post-classification fusion (see Fig. 2). Pre-classification fusion refers to combining information prior to the application of any classifier or matching algorithm. In post-classification fusion, the information is combined after the decisions of the classifiers have been obtained. A. Pre-classification fusion Prior to classification/matching, integration of information can take place either at the sensor level or at the feature level. The raw data from the sensors are combined in sensor level fusion. For example, the face images obtained from several cameras can be combined to form a single face image. In sensor level fusion, the data obtained from the different sensors must be compatible, and this may not always be possible (e.g., it may not be possible to fuse face images obtained from cameras with different resolution). Feature level fusion refers to combining different feature vectors that are obtained by either using multiple sensors or employing multiple feature extraction algorithms on the same sensor data. When the feature vectors are homogeneous (e.g., multiple fingerprint impressions of a user s finger), a single resultant feature vector can be calculated as a weighted average of the individual feature vectors. When the feature vectors are non homogeneous (e.g., feature vectors obtained using different feature extraction techniques, or feature vectors of different biometric modalities like face and hand geometry), we can concatenate them to form a single feature vector. Concatenation is not possible when the feature sets are incompatible (e.g., fingerprint minutiae and Eigen face coefficients). Biometric systems that integrate information at an early stage of processing are believed to be more effective than those systems which perform integration at a later stage. Since the features contain richer information about the input biometric data than the matching score or the output decision of a classifier/matcher, integration at the feature level should provide better recognition results than other levels of integration. However, integration at the feature level is difficult to achieve in practice because of the following reasons: (i) the relationship between the feature spaces of different biometric systems may not be known. In the case where the relationship is known in advance, care needs to be taken to discard those features that are highly correlated. This requires the application of feature selection algorithms prior to classification. (ii) Concatenating two feature vectors may result in a feature vector with very large dimensionality leading to the curse of dimensionality problem. Although, this is a general problem in most pattern recognition applications, it is more severe in biometric applications because of the time, effort and cost involved in collecting large amounts of biometric data. (iii) Most commercial biometric systems do not provide access to the feature vectors which they use in their products. Hence, very few researchers have studied integration at the feature level And most of them generally prefer post-classification fusion schemes. B. Post-classification fusion Schemes for integration of information after the classification/ matcher stage can be divided into four categories: dynamic classifier selection, fusion at the abstract level, fusion at the rank level and fusion at the matching score level. A dynamic classifier selection scheme chooses the results of that classifier which is most likely to give the correct decision for the specific input pattern. This is also known as the winner-take-all approach and the device that performs this selection is known as an associative switch. Integration of information at the abstract or decision level can take place when each biometric matcher individually decides on the best match based on the input presented to it. Methods like majority voting, behaviour knowledge space, weighted voting based on the Dempster Shafer theory of evidence, AND rule and OR rule, etc. can be used to arrive at the final decision. When the output of each biometric matcher is a subset of possible matches sorted in decreasing order of confidence, the fusion can be done at the rank level. Ho et al. describe three methods to combine the ranks assigned by the different matchers. In the highest rank method, each possible match is assigned the highest (minimum) rank as computed by different matchers. Ties are broken randomly to arrive at a strict ranking order and the final decision is made based on the combined ranks. The Borda count method uses the sum of the ranks assigned by the individual matchers to calculate the combined ranks. The logistic regression method is a generalization of the Borda count method where the weighted sum of the individual ranks is calculated and the weights are determined by logistic regression. When the biometric matchers output a set of possible matches along with the quality of each match (matching score), integration can be done at the matching score level. This is also known as fusion at the measurement level or confidence level. Next to the feature vectors, the matching scores output by the matchers contain the richest information about the input pattern. Also, it is relatively easy to access and combine the scores generated by the different matchers. Consequently, integration of information at the matching score level is the most common approach in multimodal biometric systems. 2013, IJARCSSE All Rights Reserved Page 243

In the context of verification, there are two approaches for consolidating the scores obtained from different matchers. One approach is to formulate it as a classification problem, while the other approach is to treat it as a combination problem. In the classification approach, a feature vector is constructed using the matching scores output by the individual matchers; this feature vector is then classified into one of two classes: Accept (genuine user) or Reject (impostor). Generally, the classifier used for this purpose is capable of learning the decision boundary irrespective of how the feature vector is generated. Hence, the output scores of the different modalities can be non-homogeneous (distance or similarity metric, different numerical ranges, etc.) and no processing is required prior to feeding them into the classifier. In the combination approach, the individual matching scores are combined to generate a single scalar score which is then used to make the final decision. To ensure a meaningful combination of the scores from the different modalities, the scores must be first transformed to a common domain. Fig. 2 Approaches to information fusion III. METHODS FOR MULTIMODAL FUSION The fusion methods are divided into the following three categories: rule-based methods, classification based methods, and estimation-based methods. This categorization is based on the basic nature of these methods and it inherently means the classification of the problem space, such as, a problem of estimating parameters is solved by estimation-based methods. Similarly the problem of obtaining a decision based on certain observation can be solved by classificationbased or rule based methods. However, if the observation is obtained from different modalities, the method would require fusion of the observation scores before estimation or making a classification decision. A. Rule-based fusion methods The rule-based fusion method includes a variety of basic rules of combining multimodal information. These include statistical rule-based methods such as linear weighted fusion (sum and product), MAX, MIN, AND, OR, majority voting. There are custom-defined rules that are constructed for the specific application perspective. The rule-based schemes generally perform well if the quality of temporal alignment between different modalities is good. B. Classification-based fusion methods This category of methods includes a range of classification techniques that have been used to classify the multimodal observation into one of the pre-defined classes. The methods in this category are the support vector machine, Bayesian inference, Dempster Shafer theory, dynamic Bayesian networks, neural networks and maximum entropy model. Note that we can further classify these methods as generative and discriminative models from the machine learning perspective. For example, Bayesian inference and dynamic Bayesian networks are generative models, while support vector machine and neural networks are discriminative models. 2013, IJARCSSE All Rights Reserved Page 244

Fig. 3 A Categorization of the Fusion Methods C. Estimation-based fusion methods The estimation category includes the Kalman filter, extended Kalman filter and particle filter fusion methods. These methods have been primarily used to better estimate the state of a moving object based on multimodal data. For example, for the task of object tracking, multiple modalities such as audio and video are fused to estimate the position of the object. IV. CONCLUSION The domain of multi biometrics is a new and exciting area of information science research which is directed towards understanding of traits and methods for accurate and reliable personal information representation for subsequent decision making and matching. In the recent years there is a significant increase in research activity directed at understanding all aspects of biometric information system representation and utilization for decision-making support, for use by public and security services, and for understanding the complex processes behind biometric matching and recognition. Future works could go in the direction of using more robust modelling techniques against forgeries and fusion at feature extraction level can be used. More than two modalities can be used together to make forgeries more difficult. REFERENCES [1] C. Sanderson, K.K. Paliwal, Information fusion and person verification using speech and face information, Research Paper IDIAP-RR 02-33, IDIAP, September 2002. [2] A.K. Jain, A. Ross, S. Prabhakar, An introduction to biometric recognition, IEEE Trans. Circuits Systems Video Technol. 14 (1) (2004) 4 20 (special issue on image- and video-based biometrics). [3] Hassan Soliman et al. Feature Level Fusion of Palm Veins and Signature Biometrics International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: 12 No: 01 28 [4] Mohammad H. Mahoor, 2008, A Multimodal Approach for Face Modelling and Recognition, IEEE Transactions on Information Forensics and Security, Vol. 3, No. 3, September, pp: 431-440. [5] Harbi AlMahafzah, Mohammad Imran and H.S. Sheshadri Multibiometric: Feature level fusion IJCSI, Vol. 9, Issue 4, No 3, July 2012 [6] S. Perumal Sankar, Dinakardas C. N., Multimodal biometric Authentication System Based on High level feature fusion approach ISSN 1450-216X Vol. 84 No. 1 (2012), pp. 55-63. [7] Kevin W. Bowyer, K.I. Chang Multi-modal Biometrics [8] Pradeep K. Atrey, M. Anwar Hossain Multimodal fusion for multimedia analysis Multimedia systems (2010) 16:345-379 [9] P.D.Garje, Prof. S.S. Agrawal Multimodal Identification System (IOSRJECE) ISSN: 2278-2834 Volume 2, Issue 6 (Sep-Oct 2012) [10] Pradeep K. Atrey, M. Anwar Hossain, Abdulmotaleb El Saddik, Mohan S. Kankanhalli Multimodal fusion for Multimedia analysis: a survey Multimedia Systems (2010) 16:345-379. [11] R. BRUNELLI, D. FALAVIGNA, "Person identification using multiple cues," IEEE Transactions on Pattern Analysis and Machine Intelligence 1995. [12] J. KITTLER, R. P.W. DUIN, "The combining classifier: to train or not to train," in Proceedings of the International Conference on Pattern Recognition, vol. 16, no. 2, pp. 765 770, 2002. [13] L. HONG and A. K. JAIN, "Integrating faces and fingerprints for personal identification," IEEE Trans. PAMI, vol. 20, no. 12, pp. 1295-1307, 1998. [14] S. BEN-YACOUB, Y. ABDELJAOUED, and E. MAYORAZ, "Fusion of face and speech data for person identity verification," IEEE Trans. Neural Networks, vol. 10, no. 5, pp. 1065-1075, 1999. 2013, IJARCSSE All Rights Reserved Page 245