CHAPTER 5 FEASIBILITY STUDY ON 3D BIOMETRIC AUTHENTICATION MECHANISM

107 CHAPTER 5 FEASIBILITY STUDY ON 3D BIOMETRIC AUTHENTICATION MECHANISM 5.1 AUTHENTICATION MECHANISMS Authentication is the process of establishing whether a peer is who or what it claims to be in a particular context. The authentication mechanism is responsible for creating a credential from the details produced by the peer, and the created credential is a representation of a successfully authenticated peer. The private P2P trust management model s success relies on its tolerance to various kinds of network attacks. In order to identify the attacker or a malicious peer in private P2P networks, it is mandatory for the peers in most of the security models to authenticate with the centralized server of the P2P system before requesting any service. There are three types of authentication mechanisms used in real-time systems. The first type of authentication is accepting proof of identity given by a credible peer who has evidence on the said identity or on the originator. Currency, coins, tokens, tickets, cheques, and other financial documents use the first type of authentication. The second type of authentication is comparing the attributes of the object itself to what is known about objects of that origin. The existing methods of password, biometrics, Personal Identification Number (PIN), and challenge response use the second method of authentication. The third type of authentication relies on documentation or other external affirmations. The third party certificates for websites use this method.

108 There are different combinations of authentication mechanisms available within many security models depending upon the level of security. The ways in which peers in private P2P networks may be authenticated, specifically fall into three categories, based on the factors of authentication: Something peers know, Something peers have, or Something peers are. Each authentication factor covers a range of elements used to authenticate or verify a peer's identity prior to be granted access. The peers authenticate themselves using anyone of the factors like, ownership factor which is something the peer has or owns (ID card, security token, software token or cell phone), knowledge factor which is something the peer knows (password, pass phrase, PIN or challenge response) and inherence factor which is something the peer is or does (biometric traits). The problem with the ownership factor for the peer s authentication is that the material or item can be lost, stolen, or misplaced. Moreover, when the ownership of the item goes in the hand of an attacker or a malicious peer, it can be misused and can cause unrecoverable damages. The knowledge factor is prone to be forgotten, disclosed, or compromised. In contrast to the ownership or knowledge based verification techniques, the inherence based identification/verification offers the advantage of presenting the individual personality whose attributes are hard to steal or forge. Biometric based authentication systems are widely used for their reliable security compared to the other methods that have high risk during misplacement, theft, lending, or borrowing. Though static biometrics like fingerprint, iris scan are vulnerable to spoof during a person in an unconscious state, dynamic biometrics, such as signature and voice are strong in tolerance to such attacks. 5.2 BIOMETRIC AUTHENTICATION Biometric based personal identification is a reliable and widely accepted method for authentication. Biometrics is chosen for feasibility study in order to implement a powerful authentication mechanism in private P2P networks.

109 Biometrics refers to the method of recognizing individuals based on their physiological or behavioural traits. The physiological recognition is based on the biological individuality of users, like, fingerprint, face, hand geometry, vein patterns, retina, and iris. The behavioural biometric recognition considers voice, and handwritten signature. Biometric systems are widely used for authentication, identification, and verification of any individual. Any human physiological and/or behavioural characteristic can be used as a biometric characteristic as long as it satisfies requirements like universality, distinctiveness, permanence, collectability, and acceptability [155]. Although physiological biometric traits have consequently become more integrated into commercial products, behavioural biometric traits exhibit the quality of memory that makes them attractive for security applications. 5.2.1 3D Biometrics The biometric systems are frequently under forgery attacks by the imposters as some of the biometric traits are easy to replicate. Physiological characteristics like fingerprint, face, and iris are considered to be stable and secure compared to the behavioral characteristics like voice and signature. But still the 2D physiological characteristics are under attack by biometric synthesis through image based or statistics based approaches [156]. Recently, the use of 3D data as a source of distinguishing features for personal identification is increasingly noted by many researchers [157]. 3D physiological biometrics has already shown positive signs, but 3D behavioural biometrics is the least explored. 3D physiological biometrics like fingerprint, face recognition can be spoofed by a duplicate or when the person is in an unconscious state of mind [158], but 3D behavioral biometrics like handwritten signatures possess strong barriers for such spoofing even when the individual is in a medicated state. Many 2D biometric authentication systems use features extracted from intensity images which results in the recognition rate to depend on the individual s pose and changes in illumination. In combating the situation, 3D biometric data offers a richer source of information [159].

110 5.2.1.1 Physiological 3D Biometrics 3D information of face, finger surface, and ear geometry are already researched for their accuracy in identification and verification [160]. 3D face recognition is attractive compared to the other physiological traits due to common usage and low privacy of the information that is exhibited. 3D face recognition has the capability to overcome problems, like feature localization, illumination level, pose, and expression changes in the 2D image. As 2D fingerprint is an unique and sensitive information, it is illegally acquired and forged with the available database. Once 3D features like finger thickness, backside view, length along with the fingerprint is used; the security of the system is enhanced [161]. 3D ear geometry is reported to have a high degree of accuracy, but less emphasized compared to face and finger surface [162]. 5.2.1.2 Behavioural 3D Biometrics 3D information of signature or handwriting is the only reliable behavioural 3D biometrics [163]. As permanence nature is lacking in this biometric trait, the recognition algorithm needs to be updated often to improve the Equal Error Rate (EER). The fusion with different 3D physiological biometric traits is preferred to enhance the decision criteria for minor variations during the acquisition of handwritten signature biometric traits. 5.2.2 3D Handwritten Signature Biometrics Human handwritten signature is used as a traditional way of authentication in banking, business transactions, acknowledgment of goods/services received due to its acceptance in legal and social levels. The static (off-line) and dynamic (real-time) signature verification for the paper-based document is done by humans. The

111 challenges faced in that verification are: any signature can be learnt; it can be changed by the owner and has several versions of the signature depending on the level of importance or intent of the signer. Most humans are a relatively poor judge of handwritten signature authenticity, leading to the success of expert forgers. The 2D signature verification methods are vulnerable to spoof [144], [145]. The main reason for failure is due to the fact that signatures are verified in 2D. As handwritten signatures in 2D are easily forged, their impact in biometric security is quite low. A new model for reliable and accurate identification/verification of 3D handwritten signature by considering the depth parameter in different layers of z-axis in the signature pad is proposed in this thesis. Signature verification has a number of statistical features that can be derived from the basic set of data from the signature pad. The 3D handwritten signature records pressure information on the special signature pad during the signing process. The pressure information recorded on different layers of the signature pad provides distinct information for verification of the individuals based on their signatures. This unique pressure information raises the level of difficulty in the forgery of signatures. The model proposes a special signature pad with no need for any special type of pen to be used. As the individuals feel uneasy with different pen sizes, the model considers the use of all type of pens giving the freedom for their own choice of pen. The z-axis pressure variation is measured by non-linearly spaced layers of the signature pad as in Figure 5.1. Figure 5.1 Non-linearly spaced layers in the proposed signature pad.

112 Figure 5.2 Special signature pad with three axis variations. The three-axis variations are shown in Figure 5.2 as x-axis for the left to right direction, y-axis for the top-to-bottom direction and z-axis for the depth of the signature. Figure 5.3 demonstrates the different pressure points on each layer of the proposed five layer signature pad. Figure 5.3 Z-axis pressure variations in the layers of the signature pad with upper layer being layer 1 and lower layer being layer 5. The pressure applied on the signature pad during the process of signing leads to the formation of deep points in the different layers which are sampled at regular intervals. As the process of sampling is done on each layer for every interval of time, the locations of pressure points are obtained from the grids in each layer separately.

113 The non-linearity considered in the model for reasons of minute pressure variation in the z-axis remains with the upper layers. The spacing difference is small in the upper layers to record minor variations of the depth of the signature. The lower layers are widely spaced to record the details of heavy pressure points during the process of signing. This makes the signature to be three dimensional with depth. The signature pad has grid lines on its upper layer to help the individual in maintaining their usual angle of signing. 5.2.3 Multi-parameter Handwritten Signature Biometrics The handwritten recognition and signature verification studies often make use of nearly 50 features [138], [139]. The features considered in many of those work, like velocity, acceleration, pressure, direction, pen ups/downs, total time taken, length of the signature for the verification of signature is categorized as time based feature or global parameters [140], [141]. Local parameters concern features extracted from specific parts of the signature. The depth of the signature, process of curve fitting, surface fitting, calculation of solid angles are considered as local parameters. Global parameters are denoted as {G i } and local parameters are denoted as {L i } where, i=1,2,, k depending upon the number of parameters k considered. 5.2.3.1 Parameters of interest Global parameters considered in the signature verification using 2D values in time axis are commonly reported in the field as: (a) Velocity: The rate of change of displacement along the x-axis during the process of signing. (b) Acceleration: The rate of change of velocity occurring in the process of signing. (c) Pressure: The stress level applied normally by the individual on the signature pad by the pen while signing.

114 (d) (e) (f) (g) Direction: The pen movements in the x, y axis of the pad during the time interval of signing. Pen ups/downs: The total number of pen lifts during the process of signing. Total time taken: The time used between the initial point and the final point of the signature. Length: The full length of the signature is the same for an individual even if scaling is required. The local parameters which consider the 3D values of the handwritten signature are given below: (h) (i) (j) (k) Depth: This z-axis parameter is the third dimensional value as points over the planes. Curve fitting: 2D curve fitting is performed by using the distinguished pressure points of the layers with polynomial curves used for the exact fit. Surface fitting: 3D surface fitting is performed with points from different layers. The values of the best fit are used as the local parameter. Solid angle: The z-axis values combined with the surface fitting is used to calculate the different solid angles between points and 3D surfaces. 5.2.3.2 Proposed multi-parameter biometrics model The existing models of biometric authentication are weak if cryptanalysis is performed on the transmitted data [164], [165]. The way to overcome this problem is to use hidden parameters along with strong cryptographic algorithm. A new model to bridge the gap between biometric authentication and security is proposed. The process of authentication is performed with the global and local parameters. The 3D values of the signature are taken as the important local parameter of consideration in the proposed model given in Figure 5.4. Encryption function is employed on the combined global and local parameters making individual parameter cryptanalysis a difficult process.

Figure 5.4 Multi-parameter Biometric Authentication Model. 115

116 5.2.3.3 Signature data acquisition Data acquisition in the proposed dynamic signature verification system uses online acquisition by generating the electronic signal representative of the signature during the process of signing. The signature is sampled online for every interval of time to acquire required data and extract the features from it. The features extracted from the signature are denoted depending upon the global or local parameters as G 1, G 2, G 3, G 4, G 5, G 6 or L 1, L 2, L 3, L 4. In the process of sampling, for the sampling period t, the sampled signal value from each layer 1 l Lof the signature pad S l (n) at time nt of the signing process 0 n N is given as S n 0,1,2,..., ( ). l n N (5.1) (a) Velocity ( G1 ): The rate of change of displacement along the x -axis through the grid lines through sampling process is given as dx G 1 v. dt (5.2) (b) Acceleration ( G 2 ): It is the rate of change of velocity calculated from Equation (5.2), i.e.; G dv dt 2 a. (5.3) (c) Pressure ( G 3 ): The stress level applied on the pad by the pen, F being the force applied on the area A. Here, pixels are considered from the signature pad for area A, i.e.; df G3 P. (5.4) da

117 (d) Direction ( G 4 ): The pen movements in the x, y axis are recorded as binary values 0 or 1 depending upon the backward or forward movement, i.e.; G 4 x, y. (5.5) (e) Pen ups/downs ( G 5 ): The discrete signals x(t) and y(t) specify the location of the pen on the grid lines of the signature pad at time t, and the binary signal u(t) specifies whether the pen is up or down at time t, i.e.; G5 u(). t (5.6) (f) Length ( G 6 ): The Mahalanobis distance function, d(x,y), is used to determine the length of the signature as the function is scale-invariant with the covariance matrix S. G d x y x y S x y T 1 6 (, ) ( ) ( ). (5.7) The local parameters collected and calculated from the handwritten signature are given below: (g) Depth: This z axis parameter is collected individually as the pressure points of the signature that are registered in different layers. (h) Curve fitting: 2D curve fitting is performed on the distinguished pressure points of the lower layers (especially layer 4 and layer 5) with polynomial curves for the exact fit. (i) Surface fitting: 3D surface fitting is performed with points from different layers through moment theorem. The values of the best fit are used as a local parameter. (j) Solid angle: The z axis values combined with the surface fitting is used to calculate the different solid angles between points and 3D surfaces.

118 5.2.3.4 Curve fitting techniques Each layer in the signature pad is considered as a 2D plane. The distinguished pressure points are produced by different levels of pressure applied over the layer in the process of handwritten signature. Those pressure points in each layer are considered with the curve fitting equation for a best fit of curve. The linear equations, polynomials, rational, logarithmic, exponential functions, non-linear transition, non-linear power functions are used for curve fitting. Apart from these equations and functions, there exists large numbers of user defined functions giving a wide range of randomness. The distinguished points selected are matched by one of the way with the polynomial given in Equation (5.8) to get the best fit of the curve as 0 0 0 1 1 1 2 2 2 n n n f ( x, y, z) x y z x y z x y z... x y z. (5.8) Once the curve is fit, the degree of the polynomial used to obtain the best fit of the curve is the local parameter (L 2 ). The distances between pressure points P i and P j in each layer are calculated for signature verification. The distance between any two points x,y,z in different layers are given by Equation (5.9) as x,y,z i i i to j j j ΔP PP ( x x ) ( y y ) ( z z ). (5.9) 2 2 2 ij i j i j i j i j The effect of scaling due to the individual s signing condition is overcome by using the modulus ratio of point differences. The value, P ij, is normally equal and it is compared to that stored in the database during verification. This feature helps in effective verification of 3D signature of different sizes. The rate of change of x, y axis pressure point values, R t, with respect to time is shown in Equation (5.10) with k is the total time taken for signature as

119 R t k Pi. (5.10) P t 0 j This calculates acceleration between various selected points over the layers in the signature pad as in Equations (5.2) and (5.3). In order to check the validity of the proposed model, the signatures are collected from the paper based method which is normally used for financial and administrative purposes. The depth of the handwritten signature is collected from each layer of the signature registered in the paper to perform curve fitting techniques. Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 Genuine Signature Forged Signature Figure 5.5 A sample signature of a genuine user compared with the imposter shown for each layer.

120 Samples of 100 signatures are collected from 100 individuals given with five layers of paper to sign on layer 1 with their usual pressure applied on paper. Among them, 50 good and perfect signatures are selected by rejecting the remaining 50 signatures. Those selected 50 genuine signatures are forged by 20 imposters. From the sample of 50 genuine signers, 25 are selected to repeat their signature at different intervals of time in a 10 day period. A sample signature of a genuine user compared with the imposter for five layers is shown in Figure 5.5. The pressure information recorded on each layer shows a distinct variation in the signatures of the genuine user and the imposter. The selected 25 individuals are allowed to repeat their signatures at different times. A sample signature of a genuine user compared to his/her signatures at different times is shown in Figure 5.6 for layers 1 and 5. The initial investigation of the signatures of the same individual shows almost equal impressions on layer 5 for different attempts at different times. Layer 1 Layer 5 Genuine Signature Genuine Signature Figure 5.6 A sample signature of a genuine user compared with his signature at different times shown for layer 1 and layer 5. The obtained pressure points from the different layers are matched with the database using computer vision. In order to clearly differentiate the genuine user from the imposters, curve fitting equations are applied on the pressure points

121 obtained from layer 5 for verification purposes. The 2D curve fitting calculations on each layer leads to various best fits for individual signatures. The best fit for a selected number of points is obtained by the curve fitting algorithm. The sample signatures as shown in Figure 5.5 and Figure 5.6 are used for implanting curve fitting algorithms. Layer 5 images of Figure 5.6 of a genuine user and the imposter are curve fitted as shown in Figure 5.7 and Figure 5.8. Genuine Signature Forged Signature Figure 5.7 Rational function from degrees N=1, M=0 to N=8, M=10 is used in curve fitting of genuine and imposter signatures, respectively, with x and y axis in centimeters. Genuine Signature Forged Signature Figure 5.8 Best fit function with the combination of linear regression, Fourier, rational, and waveform functions of genuine and imposter signatures, respectively, with x and y axis in centimeters.

122 Rational functions used with the same parameters for both genuine and imposter signatures show clearly that there is a wide variation in the curves as shown in Figure 5.7. The best fit functions with the combination of linear regression like, Fourier series, rational funtion, and waveform functions of genuine and imposter signatures, exhibit a clear boundary of variations between the signatures, proving that the corresponding equations are unequal. Kalman smoothing method and cubic spline interpolation functions employed on the signatures of the same individual exhibit an approximate matching between them as shown in Figure 5.9 and Figure 5.10. Genuine Signature Genuine Signature Figure 5.9 Smoothing method is employed on the signature of the same individual using Kalman smoothing with 3 history points. Genuine Signature Genuine Signature Figure 5.10 Cubic Spline Interpolation function is employed on the signature of the same individual.

123 The authentication systems using 3D biometrics are tolerable for circumvention. The drawbacks of 3D physiological and behavioural biometric recognition include high cost, lack of powerful image processing algorithms, and low accuracy of acquisition devices. The preliminary results reveal that 3D handwritten signature has a unique feature that helps in improving the security of authentication systems. The large sets of samples prove that the genuine and imposter signatures look similar at the upper layers and have wide differences with the lower layer. The fitted curve is a unique parameter which is hidden even from the owner of the signature as the process of verification is done in the background operation. The minor variations of the individual s signatures at different times make curve fitting equations to deviate much. This matching gap can be filled by applying fusion with any one of the 3D physiological biometric traits for improvement of security in the existing authentication systems. 5.2.3.5 3D Surface analysis The pressure points from more than one layer are considered to fit a 3D surface. There exists many surface fits depending on the points considered in each layer. The optimum equation to describe the three dimensional empirical data is obtained from the best fit through the standard least squares minimization. The points between two, three, four, and five layers are used to perform the best 3D surface fit. Various points of x, y, z axis from all layers of the signature produce complex 3D surface. The fitted 3D surfaces are another unique local parameter (L 3 ) for verification. The 3D surface fitting utilizes linear equations, polynomials, rational, logarithmic, exponential functions, linear regression, logistic functions, Fourier approximation, B-splines, parametric curves least squares approximations and user defined functions for fitting of 3D surfaces. A 2D contour plot on the top and bottom of the surface fit graph is also a distinct value for the parameter, L 3. The points and surfaces in three-dimensional space produces the solid angle at every different point and location considered. This is a measure of scaling for the

124 signature. The angle θ will remain constant for different sizes of the signature, making the process of verification unchanged. Figure 5.11 exhibits the solid angle calculation in one of the layers. This hidden parameter (L 4 ) is important and effective data for 3D signature verification. Figure 5.11 Solid angle calculations in the layers of the signature. This solid angle θ leads to a curved-surface area of the signature that is quickly compared and interpreted for further analysis. This model relies on a unique feature of curved surface evaluation in three dimensions. 5.2.3.6 Enrolment phase The authentication algorithm for the proposed model has three phases, namely, enrolment phase, encryption/decryption phase, and verification phase. Enrollment phase works offline, and verification phase works online. During the enrollment phase, each legitimate individual is allowed to sign in the signature pad for registering the signature samples. Once the first sample is obtained from the signature pad, the features are extracted from it and the values are stored in the database. The features are extracted from all possible combinations of global G,G,G,G,G,G parameters as in Equations (5.11) and (5.12) of R i for 1 2 3 4 5 6 i=1, 2,, k. Here, k denotes the total number of features considered as

125 R G G G G G G, 1 1 2 3 4 5 6 (5.11) and R G G G G. 2 2 4 5 6 (5.12) The local parameter features are extracted from L, L, L L 1 2 3, 4 and stored in the database individually. When the second sample of the signature is received, the extracted features are verified offline with the previous sample for variations. The degree of variance is calculated and stored in the database. Once enrolment of the 3D signature is completed after collecting sufficient number of samples, the full degree of variance from the complete set of signatures is calculated in order to minimize the False Acceptance Rate (FAR) and False Rejection Rate (FRR). 5.2.3.7 Encryption and Decryption phase The attacks like Man-in-the-middle and cipher-text alone attacks, exploit security in signature verification [166]. The algorithm used for signature verification is time-dependent to avoid any attempt of cryptanalysis during the process of verification of signature. The selected parameters from the algorithm are combined with function, f ( G, L), and encryption is performed using XOR operation as given in Equation (5.13). The function varies the combination of the considered global and local parameters every time along with a different key, k, as given by f ( G, L) f ( G, L) k. (5.13) The encrypted data is sent for verification through the channel. The receiver decrypts the data as in (5.14) to get the features for verification from the database as f ( GL, ) f( GL, ) k. (5.14)

126 Each layer is encrypted with dynamic or varying symmetric keys and the whole signature is encrypted with asymmetric key to protect from spoofing and cryptanalysis. The database storing the detailed analysis of all the parameters is updated periodically and encrypted for higher protection. 5.2.3.8 Verification phase During the verification phase, the signature is acquired from the signature pad and the features are extracted. The 2D curve fitting, 3D surface fitting, and solid angle calculation leads to different unique values for an individual signature. The best fit for every selected point is obtained by the curve fitting algorithm. From the extracted features, the algorithm selects some combination of the global parameters along with the calculated local parameters. The selected parameters are sent through the channel after encryption for the process of verification. In order to combat from the Naïve, static and dynamic expert forgery, the local parameters of curve fitting, surface fitting, and solid angle calculations are varied in the verification phase every time by the defined algorithm. The comparison of the received signature with the database determines whether the values fall within a certain statistical range and accordingly acceptance/rejection is sent as the result of verification. The signature verification system is updated with the individual s more recent sample of the signature to avoid minor variations in the signature of the individual due to aging or other known factors. The updated information is analyzed to modify the authentication algorithm to be used. 5.2.4 Multimodal Biometric Fusion Model Most of the commercially available biometric recognition systems work with a single biometric identifier. These unimodal biometric systems use any one of the physiological or behavioural biometric identifiers. Unimodal systems contend with a

127 variety of problems, such as noise in sensed data, intra-class variations, inter-class similarities, non-universality, and spoof attacks [167]. The noise, intra-class variations, and non-universality can be overcome by selecting a high quality sensor with an appropriate biometric trait, whereas, the inter-class similarities, spoof attacks pose a danger for the system being compromised. The circumvention in unimodal biometric recognition systems exhibit the ways of deceiving the system through fraudulent methods, which are of main concern in security and privacy. The well trained imposters perform more attempts to forge a particular biometric trait in unimodal biometric systems. Improving the method of analysis and tightening the threshold for recognition, reduces the issue of circumvention, but instead, they will increase false rejection rate and failure to enroll rate. The appropriate solution for this problem is to use multi-biometric traits working in serial with suitable fusion method to decide upon the credentials of the individual under question. The multimodal biometric system is also forged by expert forgers but fusion of matching scores decides the result. 5.2.4.1 Multimodal biometrics Multimodal biometric systems use more than one physiological and/or behavioural biometric trait for recognition of individuals. The physiological biometric authentication methods like fingerprint, iris, voice, face recognition can be spoofed by a duplicate, or when the person is in an unconscious state of mind. The behavioural biometric authentication like voice, handwritten signature possess strong barrier for such spoofing even when the individual is in a medicated state due to the need for memory. The balance between no need for memory and need for memory is obtained by considering one biometric trait from each of physiological and behavioural biometric traits. The proposed model considers one biometric trait from physiological for identification (finger print, face recognition, iris recognition) and one from behavioural for verification (3D handwritten signature). The biometric identifiers considered individually have exhibited some drawbacks [155], whereas fusion exhibit some merits as shown in Table 5.1.

128 Human fingerprints are unique to each person and even the fingerprints of twins are not exactly the same [168]. Fingerprint is the pattern of ridges that make loops, arches or whorls. In each fingerprint, there are regions where changes in ridge are noticed, like, a ridge ends, splits into two ridges, join another ridge or create an island; these features are called minutia [169]. It is these features that are extracted and compared for determining a match. The comparison of two fingerprints is performed through feature-based/minutia-based matching methods [170]. Different fingerprint matching algorithms use different types of information extracted from the input fingerprints for matching. Automatic identification methods based on fingerprint provide positive identification with a very high accuracy [171], [172]. Face recognition measures analyzes the overall structure, shape and proportions of the face. The features extracted from the face images are used in comparison with face database for identification [173]. The commonly used features are distances between individual organs (like eye, nose, mouth) located on a face, length of the organs, area, angle made between two organs. Automated face recognition system is capable of capturing face images from a distance using camera, extract features, and compares with the database for recognizing individuals [174], [175]. The iris is the annular region of the eye bounded by the pupil and the sclera on either side. The complex iris texture carries very distinctive information for every individual [155]. Iris recognition is the process of recognizing the random pattern of the iris. It has higher consistency and uniqueness compared to fingerprint or face [176]. Automated algorithm for iris recognition is available which works by locating iris using landmark features. The landmark features and the distinct shape of the iris allow for imaging, feature extraction and identification [177]. Fusion combines multiple sources of information to form a single value for comparison. Fusion can be performed at different levels in the multimodal biometric systems, like fusion at feature level, match score level, or decision level [167], [178].

129 Feature level fusion is difficult as the features extracted from multiple biometric traits are of different types [179]. Decision level feature is like majority voting which depends on the winning results from different biometric traits that can be spoofed by imposters. The only viable way for fusion between different features of the various biometric traits is matching score fusion [180]. Table 5.1 Comparison of Unimodal and Multimodal Biometric Identifiers Based on the Perception of the Authors - High, Medium and Low are Denoted by H, M, and L, Respectively [155] Biometric Identifier Universality Distinctiveness Permanence Collectability Performance Acceptability Circumvention Fingerprint M H H M H M M Face H L M H L H H Iris H H H M H L L 3D Handwritten Signature L M L H M H L Fusion (Fingerprint + 3D Handwritten M H M H H H L Signature) Fusion (Face + 3D Handwritten Signature) M M M H M H M Fusion (Iris + 3D Handwritten Signature) M H M H H M L 5.2.4.2 Proposed biometrics fusion model The proposed multimodal biometric security model for efficient authentication uses fusion for protection against expert forgers. The model deals with three phases for identification, verification and decision. The first phase employs physiological biometric traits for identification. The second phase uses 3D handwritten signature for verification, and the third phase decides from the fusion of

130 the obtained matching scores compared with the threshold value. A simple block diagram of the model with direction flow is shown in Figure 5.12, and the detailed block diagram of the multimodal biometric authentication model with feedback is shown in Figure 5.13. Figure 5.12 Simple block diagram of the proposed multimodal biometric model. 5.2.4.3 Identification phase The physiological biometric traits like fingerprint, face or iris is used for identifying the individuals. These biometric traits can be used independently or in the combined mode depending on the applications for identification. Initially, the physiological biometric image acquisition is made for a clear and perfect image. Then the features are extracted from the obtained image. Once the required features are acquired, the physiological matching score (MS p ) is calculated by comparing the features with the available database as in (5.15); f match is the number of matched features, and f total is the total number of features considered. The calculated matching score is sent for identification and fusion. The matching score is calculated as MS f match. (5.15) f total

Figure 5.13 Block Diagram of the Multimodal Biometric Authentication Model. 131

132 In the identification process, 1:N matching is done by comparing the matching score with an identification threshold (T id ) to identify one individual from N individuals in the database. The identification threshold is chosen from the scale depending upon the application. Once the identification process is successful, the identified individual s database code is transferred to the database of the verification phase along with an indication to the next phase to proceed. When the identification process ends in failure without identifying any individual, an error message is sent along with an indication to repeat the identification phase. 5.2.4.4 Verification phase Once an individual is identified, the credential of the individual is verified through behavioural biometric traits in the verification phase. The behavioural biometric trait considered is 3D handwritten signature. Once the 3D signature is acquired through a special signature pad, dynamic features like velocity, acceleration, pressure, direction, pen ups/downs, and total time taken are extracted along with the pressure information from each layer. The dynamic feature and pressure information from each layer is given as f d and f layer. The obtained individual features are compared with the identified database of the individual for dynamic feature matching score (MS d ) and layer matching score (MS layer ) as in (5.15). The 3D signature weighted mean matching score (MS 3DSign ) is calculated by considering the dynamic and layer matching scores with weight factors, a and b respectively as in (5.16). Here, MS 3 DSign q r a MSdi bj MSlayerj i1 j1. r a bj j 1 (5.16)

133 The number of dynamic features (q) combined matching score is multiplied by a common weight factor a. In the consideration of layer matching scores, the weight factor, b, for each layer is assigned individually with more weightage to the lower layers as pressure information is distinct for individuals. The weight factor b is gradually increased starting from upper layer to lower layer with r being the total number of layers. The calculated weighted mean matching score is transferred to the decision phase for fusion and authentication. 5.2.4.5 Decision phase The identification phase acts as a user ID, verification phase acts as a password, and the decision phase provides the authentication results for user ID and password like, the general authentication method used in internet applications. In this model, individuals are safe from attacks as physiological biometric is used along with the 3D signature by fusion. Fusion is performed at the matching score level from identification phase, and verification phase matching scores as in (5.16) to calculate the weighted mean total score (WMTS) with α p, β as weight factors depending upon the applications, given by WMTS n p MSp MS 3 DSign p 1 n p 1 p. (5.17) When more than one physiological biometric trait is considered, the individual weight factor α p is assigned to each biometric trait with n being the total number of physiological biometric traits considered. The summation of the weight factors with their respective matching scores gives the total physiological biometric matching scores (MS p ). The 3D handwritten signature matching score is

134 emphasized by the weight age factor β. The calculated weighted mean total score is compared with the preset threshold of fusion (T fusion ) to decide upon the acceptance/rejection condition. The fusion threshold (T fusion ) is chosen from a scale depending upon the application. Authentication depends on the weight factors and the threshold of fusion for efficient results. 5.3 STATISTICAL RESULTS AND DISCUSSIONS Samples of 140 handwritten signatures are collected from 140 individuals. Among them, 100 genuine signatures are forged by 20 imposters. Individuals acting as genuine signers are provided with five layers of paper to sign on layer 1 with their usual pressure applied on the paper. The imposters are then trained and allowed to imitate genuine signatures with a reasonable time limit. From the sample of 100 genuine signers, 25 individuals are selected to repeat their signature at different intervals of time in a 10 day period. A sample signature of genuine user compared with an imposter at five layers is shown in Figure 5.14. The pressure information recorded on each layer shows a distinct variation in the signatures of the genuine user and the imposter. A total number of 25 individuals are selected from the categories of simple signatures, normal signatures, and difficult signatures. The selected 25 individuals are allowed to repeat their signatures at different times. A sample signature of a genuine user compared with his signature for layer 1 and layer 5 at different times is shown in Figure 5.15. A comparison of the signatures of the same individual shows equal impression on layer 5 for different attempts at different times. A sample signature of a genuine user compared with the imposter is shown in Figure 5.16 for layers 1 and 5. The signature of a genuine user and an imposter shows exact matching at layer 1 with less matching at layer 5.

135 5.3.1 3D Signature Samples Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 Genuine Signature Forged Signature Figure 5.14 A sample signature of a genuine user compared to the imposter shown for each layer. Layer 1 Layer 5 Genuine Signature Genuine Signature Figure 5.15 A sample signature of genuine user compared to his signature at different times shown for layer 1 and layer 5.

136 Layer 1 Layer 5 Genuine Signature Forged Signature Figure 5.16 A sample signature of a genuine user compared to the imposter shown for layer 1 and layer 5. 5.3.2 Signature Verification by Humans The collected signature samples are verified off-line by three human judges; out of them, two are experienced in signature verification and the third is an amateur signature verifier. The judges are given reasonable time limit to verify signature and award matching score, varying in the matching scale of 0 to 5, 0 being the point for no matching, and 5 being the point for exact matching. Points 1, 2, 3, and 4 denote 20%, 40%, 60% and 80% matching between two signatures under test. Table 5.2 Matching Scores from the Mean of Sample Genuine Signatures with Forged Signatures is Given in 0-5 Scale Denoting with 0 Being No matching and 5 Being Exactly Matching Layers Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 Judges Judge 1 3.42 2.80 2.05 0.77 0.01 Judge 2 3.81 3.38 2.51 1.07 0.09 Judge 3 3.40 3.06 2.31 1.06 0.14 Average 3.54 3.08 2.29 0.97 0.08

137 Figure 5.17 Matching scores between genuine users and imposters for the sample of 100 signatures given as the mean of the three judges for layer 1, layer 3, and layer 5. Figure 5.18 Matching scores between genuine users and imposters for the selected 25 signature samples given as the mean of the three judges for layer 1 and layer 5. Figure 5.19 Matching score between genuine users with their own signatures for the selected 25 individuals given as the mean of the three judges for layer 1 and layer 5.

138 The results of human judgment are presented in Table 5.2. The results show that an expert forger can replicate a genuine signature in the 2D surface easily at upper layers, namely, layer 1 and layer 2. When the hidden information on pressure is considered for the verification of genuine and forged signatures, it is observed that expert forgers are unable to replicate exact pressure that is applied by the genuine user as shown in layer 4 and layer 5 matching scores. The matching scores between the genuine and forged signatures decrease while considering the pressure of signatures from layer 2 to layer 5. The mean of the matching scores of the three judges for 100 signature samples compared with the imposter in layer 1, layer 3, and layer 5 is shown in Figure 5.17. Layer 5 matching scores between the genuine user and the imposter are low due to the different matching points of the signatures recorded. Figure 5.18 shows the mean of the matching scores between the genuine user and the imposter for 25 signature samples showing huge difference in the matching scores between layer 1 and layer 5. The results of signature matching score from the same user compared with his/her own signature has high values even at lower layers, like, layer 4 and layer 5. The matching score differences between layer 1 and layer 4 are quite low as shown in Figure 5.19. 5.3.3 FAR and FRR calculations False Acceptance Rate (FAR) denotes the percentage of acceptance of an imposter as a genuine user. False Rejection Rate (FRR) denotes the percentage of rejection of the genuine user deciding the user as an imposter. The percentages of FAR and FRR solely depends on the threshold value set for a particular application. Table 5.3 shows the threshold value (T) set for 3D signature verification matching points, and its effect on FAR and FRR. The mean value of the three judges are considered to decide upon FAR and FRR. When the threshold value is set at a high value, FAR is fully eliminated and FRR increases exponentially. FAR increases whereas leaving FRR to 0% for the less threshold value. The increasing values of FRR for high threshold are mainly due to image quality in layer 4 and layer 5 of the samples.

139 The FAR and FRR values are plotted against the threshold values as shown in Figure 5.20. The Equal Error Rate (EER) denotes the lowest point where the values of FAR and FRR are considered to be equal. EER helps in setting the optimum value of the threshold. As per the values obtained, the EER is at the threshold values 1 to 2. The image quality obtained down to layer 2, namely, layer 3 to layer 5 make some signatures to be difficult for judgment. The automated system with dedicated hardware for 3D signature acquisition can show better results. The calculated values of FAR, FRR, and ERR exhibit encouraging preliminary results for the deployment of 3D signature verification hardware. Table 5.3 Calculation of FAR and FRR Based on the Human Judges Perception for 3D Signature S. No. Threshold Value (T) FAR FRR 1 0.5 4% 0% 2 1 0% 0% 3 1.5 0% 0% 4 2 0% 0% 5 2.5 0% 4% 6 3 0% 12% 7 3.5 0% 72% 8 4 0% 88% Figure 5.20 FAR and FRR corresponding to the threshold values.

140 5.4 SIMULATION RESULTS AND DISCUSSIONS A prototype signature pad for the proposed model has been developed and used in the collection of sample signatures. This thesis work has used about 150 samples of handwritten signature collected from 150 individuals. A total number of 100 individuals are selected from the categories of simple signatures, normal signatures, and difficult signatures. Individuals acting as genuine signers are given with an electronic signature pad and layers of paper to sign with their usual pressure applied on the paper which is simultaneously recorded on the electronic signature pad. The recorded signatures are saved in the database as genuine signatures. The imposters are then trained and allowed to imitate the genuine signatures with reasonable time limit. Some of the selected individuals are allowed to repeat their own signatures at different timings for checking the consistency in their signatures. 5.4.1 3D Signature Acquisition The handwritten signatures are collected through an electronic signature pad (epad [35]) as a 2D image from each layer. The digital data of the signature is acquired through Visual Basic (VB) program written with the Graphical User Interface. The program controls the operation of the signature acquisition process through different forms created by the VBScript with multiple attempts of signing process. VBScript access the elements of the signature pad within which it is running. Initially, a set of specimen signatures are acquired from a genuine user with multiple attempts and the signature samples are stored as digital images for each layer in the database. Those collected signatures are then trained and forged by imposters. Samples of signature acquisition process for layer 1 to layer 5 and signatures are shown in Figures 5.21, 5.22, 5.23, and 5.24.

141 (a) (b) Figure 5.21 Screen shots of the signature acquisition process (a) Signature acquired in layer 1 (b) Layer 1 signature sample saved to the database.

142 (a) (b) Figure 5.22 Screen shots of signatures acquisition process (a) at different attempt for all layers (b) at single attempt with distinct pressure points in layer 5.

143 Figure 5.23 A sample signature of the genuine individual compared with an imposter shown for each layer. Figure 5.24 A sample signature of the genuine individual compared with his signature at different times shown for layer 1 and layer 5.

144 5.4.2 Signature Analysis through Pattern Recognition In the collected signature samples, it is observed that most of the individuals have enough pressure in their signatures to be recorded in layers 1, 2, and 3. But 5% and 22% of individuals signatures failed to register in layer 4 and layer 5, respectively, as shown in Table 5.4 due to lack of pressure applied during the signing process. Table 5.4 Signature registration in each layer Layer Signature Registration 1 100% 2 100% 3 100% 4 95% 5 78% The signatures are compared by normalized 2D cross-correlation technique using template matching [181]. Normalized form of cross correlation is preferred for feature matching applications. It is computed in the spatial domain. The correlation coefficient is calculated by normalizing the image and feature vectors to unit length, yielding a cosine-like correlation coefficient as in [181] is given by,,, xy, f x y fuv, t xu yv t uv,, 2 0.5 2 f x, y f,, xy, uv t xu yv t xy, (5.18) where is the mean of the feature, and, is the mean of, in the region under the feature.

145 Normalized cross-correlation is computed for the signature image under test and the signature image from the database [182]. Usually, the signature image from the database is larger than the one under test for the efficient check of crosscorrelation. The resulting values from the normalized cross-correlation coefficients range from -1.0 to 1.0. In the signature verification process, the brightness of the test signature and database signature vary due to different pen sizes used. These signature images are first normalized for further processing. This is typically done at every step by subtracting the mean and dividing by the standard deviation of a test signature with a database signature. The Normalized 2D cross correlation between genuine and forgery signatures show that matching in layer 1 is higher than that of layer 5 as shown in Table 5.5. But when genuine signature is compared with the same individual s different attempts, the matching curve displays that correlation is high in layer 5, increasing exponentially from layer 1 as shown in Figure 5.28. Table 5.5 Comparison of cross correlation values Analysis Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 Cross-correlation between genuine and forgery signatures Cross-correlation between genuine signatures of different attempts 0.152 0.148 0.161 0.158 0.144 0.213 0.215 0.224 0.259 0.398

146 Figure 5.25 Cross correlation values of a sample genuine signature with the forged signature. Figure 5.26 Cross correlation values of a sample genuine signature with his own signature.

Figure 5.27 Normalized 2D cross-correlation between (a) genuine and forgery signatures (b) genuine signature at different attempts. 147

148 0.45 Cross correlation values 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 1 2 3 4 5 Layers in the signature pad Crosscorrelation between genuine and forgery signatures Crosscorrelation between genuine signatures of different attempts Figure 5.28 Comparison of cross correlation values of a sample genuine signature with imposter and with his own signature. 5.4.3 FAR and FRR calculations In Figure 5.28, it is observed that though the imposters imitate the signatures exactly in layers 1, 2, and 3; they cannot replicate the exact pressure given by the genuine individual in layers 4 and 5. Comparing the cross correlation curves, the mean difference between the imposters and genuine users are less in layers 1 to 3 which allows False Acceptance Rate (FAR) to increase. When compared to layers 4 and 5, the cross-correlation difference is high enough to reject maximum FAR indeed, increasing False Rejection Rate (FRR) as shown in Table 5.6. The Equal Error Rate (EER) denotes the lowest point where the values of FAR and FRR are considered to be equal. EER helps in setting the optimum value of the threshold (T). As per the cross-correlation values, the obtained EER lies between the values 2 to 3 in layers 1, 2, and 3 whereas increases to 5 in layer 5.

Table 5.6 Comparison of FAR and FRR with different Threshold Values S.No. Threshold Value (T) FAR Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 FRR FAR FRR FAR FRR FAR FRR FAR FRR 1 0.10 100% 0% 98% 0% 98% 0% 93% 0% 68% 0% 2 0.15 55% 0% 59% 0% 66% 0% 68% 0% 59% 0% 3 0.20 9% 28% 7% 28% 18% 24% 18% 12% 55% 0% 4 0.25 5% 48% 2% 64% 2% 60% 7% 24% 18% 8% 5 0.30 2% 64% 2% 76% 2% 64% 5% 48% 9% 12% 6 0.35 0% 80% 0% 84% 0% 76% 0% 60% 0% 20% 7 0.40 0% 88% 0% 96% 0% 84% 0% 64% 0% 28% 8 0.45 0% 96% 0% 100% 0% 92% 0% 80% 0% 52% 9 0.50 0% 100% 0% 100% 0% 100% 0% 96% 0% 80% 149

150 5.5 APPLICATION DOMAINS The rapid growth of internet has led to numerous on-line business transactions and administrative works through computers. The need to ensure that only the right person gets access to the highly secured information, the requirement for reliable security methods to protect information transferred through insecure channel leads to various authentication mechanisms. Industrial Automation needs appropriate control mechanisms in order to perform well. Biometrics is one of those reliable and standard control mechanisms used in authentication and authorization. The industries are currently implementing biometric security systems in a large scale to solve most of the issues in authentication. Human handwritten signature is used as a traditional way of authentication in industries starting from work attendance to approval of security measures, decisions, and acknowledgements. The problem of signature forgery is common in industries having huge number of employees. The damages caused by such signature forgeries range from small level of attendance manipulation to the highest level of threat to lives, unrecoverable financial losses, and national security. Even though there are alternate mechanisms like key, token, ID cards for authentication, they have high risk during misplacement, theft, lending or borrowing. As the proof of the person using such mechanism is hard to verify, the possessor is always the victim for any mishandling. The flow diagram of the proposed 3D signature biometric authentication model is shown in Fig. 5.29. The authentication process starts initially by collecting the signature, extracting the features from the signature, and considering necessary parameters. Then the matching score is calculated by comparing the features of the collected signature with the database. When the matching score is greater than the threshold, the signature is accepted and authentication is successful. The general authentication model that is proposed can be used in most of the application domains discussed in this section.

151 Figure 5.29 Flow diagram of 3D Signature biometric authentication model. In industries, behavioural biometric authentication is most preferred in order to check the fitness of the employee in the job field than the routine duty of showing the fingerprint, face and iris. The proposed 3D signature biometric authentication model will increase the security in access control, attendance report, acknowledgement receipts in industries. The deployment of 3D signature verification can replace the present methods of authentications like key, ID card or physiological biometrics. The applications of the 3D signature biometric authentication model are in access control, contract/agreement execution, banking services, financial transactions, and acknowledgment of goods/services received. The process of authorization is distinct from that of authentication. Authentication is the process of verifying that "you are who you say you are" and authorization is the process of verifying that "you are permitted to do what you are trying to do". Authorization always presupposes authentication. The 3D signature biometrics can be used in both authentication and authorization process. One familiar use of authentication and authorization is access control. A system that is supposed to be used only by those authorized must attempt to detect and exclude the unauthorized. Access to it is therefore usually controlled by insisting on an authentication procedure to establish with some degree of confidence, the identity of the user, and granting privileges established for that identity.