Finger or Stylus: Their Impact on the Performance of Online Signature Verification Systems

Transcription

1 MACRo th International Conference on Recent Achievements in Mechatronics, Automation, Computer Science and Robotics Finger or Stylus: Their Impact on the Performance of Online Signature Verification Systems Margit ANTAL 1, András BANDI 2 1 Department of Mathematics-Informatics, Faculty of Technical and Human Sciences, Sapientia University, Tg. Mureş, manyi@ms.sapientia.ro 2 Student, Department of Electrical Engineering, Faculty of Technical and Human Sciences, Sapientia University, Tg. Mureş, andras_bandi1994@yahoo.com Manuscript received September 10, 2017, revised October 12, Abstract: The widespread use of smartphones and the ability of these devices to digitize signatures have made it possible to sign electronic documents in this way. In this paper we compared two on-line signature databases in terms of verification performance: the MCYT containing signatures drawn by stylus pen, and MOBISIG containing finger drawn signatures. Performance evaluations were performed using both local and global systems. In the case of global systems, we evaluated the performance of a novel information theory features set. Little improvement was achieved by this feature set. There were large differences between the two databases in terms of performance. Finger drawn signatures collected by mobile device were proved inferior to signatures collected by digitizing tablet and its stylus. Keywords: behavioral biometrics, on-line signature, touchscreen, informationtheory-based features, performance evaluation. 1. Introduction The touchscreen allows the acquisition of signatures, which can be drawn even with the finger. In this paper we study the performance of signature verification system using on-line signatures collected through mobile devices, where the signatures were drawn by finger. State-of-the art global (feature-based) and local (time functions based) verification systems were evaluated for two on-line signature databases: the MCYT and the MOBISIG. The MCYT database [1] was collected by using a high resolution digitizing tablet and its stylus. The MOBISIG database [2] was collected by using an Android tablet and the subjects finger. Huge performance degradation was found in the case of finger-drawn signatures collected on touchscreen compared to stylus-drawn signatures collected on digitizing tablet. The performance degradation characterized both global and /macro

2 12 M. Antal, A. Bandi local systems, but it was higher in the case of the local system. This type of performance degradation has already been reported by Tolosana et al. [3] for e- BioSign biometric database and Martinez et al. [4] for DooDB database. The rest of this paper is organized as follows. In section 2 we review researches which report performance evaluations on finger-drawn signatures. Section 3 presents the local and global systems used for signature verification. First, the local signature verification system with the used time functions is presented, then the new information theory related features are described, and finally the global system with the used features. In section 4 the experiments are described: the databases, the evaluation protocol, and the results in terms of EER (Equal Error Rate) and error curves. Finally, section 5 draws the conclusions. 2. Related work There are only a few studies reporting performance evaluations on finger drawn signatures. One of the first studies reporting performance evaluation results on such signatures is the semester thesis of Pascal Bissig [5]. Unfortunately, the study reports only random forgery type evaluation. The reported results cannot be reproduced because their dataset is not publicly available. The first publicly available database DooDB, containing finger drawn signatures was collected by the Biometric Recognition Group ATVS. Martinez et al. [4] report benchmark results using a local system based on Dynamic Time Warping (DTW) and several time series. Although the random forgery type evaluations resulted in low error rates, these errors are too high for skilled forgery evaluations. In a later study [6], these results were improved by using Gaussian Mixture Models (GMM). Finger drawn signatures collected on mobile devices were studied by Sae-Bae et al. [7]. However, they were only able to report only random forgery evaluation results because the database contains only genuine signatures. This database is also private. A recent study by Tolosana et al.[3] reports benchmark results on the new e- BioSign database, which contains signatures and handwriting information acquired by using several acquisition devices, both digitizing tablets and mobile devices. On two devices data were collected using both stylus and the finger. A significant performance gap was reported between stylus and finger drawn signatures. The performance deterioration is over 10% when using the finger instead of stylus.

3 Finger or Stylus: Their Impact on the Perf. of On-line Sign. Ver. Syst Methods In this section we describe the local and the global systems used for signature verification. 3.1 Local system Local systems work with time-functions extracted from the signatures. In this paper we used the following eight time-functions: x(t), y(t), x (t), y (t), x (t), y (t), p(t) and p (t). The x(t), y(t) are the coordinate time series, x (t), y (t) are the horizontal and vertical speed time series, x (t), y (t) are the horizontal and vertical acceleration time series, and p(t), p (t) are the pressure and its first derivative time series. Fig. 1. shows these time series for a genuine signature from the MOBISIG database. Signatures were compared using the Dynamic Time Warping (DTW)[8] algorithm, which is able to compare time sequences having unequal lengths. We created an eight-dimensional time series for each signature using the eight timefunctions. The distance returned by the DTW algorithm was converted to a 1 similarity score using the formula Similarity(S i, S j ) =, where S 1+DTW(S i,s j ) i and S j are the two 8-dimensional time sequences corresponding to the two compared signatures. Figure 1: Local features (time-functions) for a genuine signature from the MOBISIG database.

4 14 M. Antal, A. Bandi 3.2 Global system Information theory features Permutation entropy features have already been investigated by Rosso et al. [9] for signature verification. They proposed Shannon entropy, statistical complexity, and the Fisher information computed from the Bandt and Pope histogram. This type of histogram was computed from horizontal and vertical coordinates of signatures. We detected two major problems in this publication. The first is that they reported their results erroneously. We have already reported this problem in an earlier publication [10]. Another major problem is regarding the computation of permutation entropy, which omits repeated consecutive values. This deficiency was admitted by the authors in a later paper [11]. We propose to compute Shannon entropy and Fisher information from a new type of histogram which takes into consideration repeated consecutive values. In a time series we have three types of relation between two consecutive points: less, equal, and greater (x i+1 < x i, x i+1 = x i, x i+1 > x i ). For three consecutive points there are nine relations depicted in Fig. 2 (e.g. the 4. relation has the following meaning: x i = x i+1 and x i+1 > x i+2 ). Figure 2: Relations between three consecutive points. Let us consider the following time series X = {1,2,2,3,3,1,2,1,3,4}. In the first step we associate the N = 9 codes (see Fig. 1.) with the consecutive triplets resulting in the following eight coded triplets: {8,6,8,4,3,7,3,9} (1,2,2 8; 2,2,3 6; 2,3,3 8; 3,3,1 4; 3,1,2 3; 1,2,1 7; 2,1,3 3; 1,3,4 9). Then, we compute the normalized frequencies associated with these nine codes (see Eq. 1).

5 Finger or Stylus: Their Impact on the Perf. of On-line Sign. Ver. Syst. 15 f(1) = 0, f(2) = 0, f(3) = 2 8, f(4) = 1 8, f(5) = 0, f(6) = 1 8, f(7) = 1 8, f(8) = 2 8, f(9) = 1 8. (1) Shannon entropy is defined in Eq. 2. N S[P] = p i ln(p i ) (2) i=1 where P = {p i ; i = 1,, N} with i p i = 1 is a discrete probability distribution. The normalized Shannon entropy is defined in Eq. 3. H[P] = S[P], (3) S max where the entropy is maximal when the probabilities are equal, thus S max = ln N. For our time series X we have H[X] = 9 i=1 f(i) ln f(i)) = ln 9 Fisher information was computed using the formula (Eq. 4.) proposed by Rosso [9], but instead of the Bandt & Pope histogram we used our histogram s normalized frequencies. N 1 F[P] = 1 2 [ p i+1 p i ] 2 (4) i= Global features In our global system we used 15 state-of-the-art (see Table 1) and 12 information theory based features (see Table 2). Some of the state-of-the-art features were selected from the set of 100 global features proposed by Fierrez- Aguilar et al. [12] Table 1: State-of-the-art global features. No. Feature description No. Feature description 1 Signature total duration 9 Sign changes of y = dy/dt 2 Average velocity 10 Sign changes of y = dy /dt 3 Average pressure 11 Average v x 4 Max velocity (v max ) 12 Average v y 5 Average pointwise velocity 13 Sign changes of p = dp/dt 6 Average pointwise acceleration 14 Sign changes of p = dp /dt 7 Sign changes of x = dx/dt 15 Number of sampled points 8 Sign changes of x = dx /dt

6 16 M. Antal, A. Bandi Table 2: Information theory based features. No. Feature description No. Feature description 1 Entropy of x(t) 7 Entropy of x (t) 2 Entropy of y(t) 8 Entropy of y (t) 3 Entropy of p(t) 9 Entropy of p (t) 4 Fisher of x(t) 10 Fisher of x (t) 5 Fisher of y(t) 11 Fisher of y (t) 6 Fisher of p(t) 12 Fisher of p(t) In a global system, signatures are represented as feature vectors. Therefore, from each signature a fixed number (D) of features are extracted, which constitute a feature vector: S i = {f 1 i, f 2 i,, f D i }, i = 1.. N. Our global system works as follows: in the training phase we construct a template from a fixed number of N signatures. During template construction, we calculate the maximum and minimum for each feature: {min (f i j ), max (f i j )}, j = 1, D. These values are i=1,n i=1,n used for min-max normalization of the feature vectors belonging to the template. Furthermore, these values are stored together with the template and are used in the verification phase for normalizing the signature to be verified. The normalization was applied for each user separately, in this way we did not take into account other users data. For evaluation we used three types of anomaly detector: Euclidean, Manhattan and k-nearest neighbor (k-nn) detectors. We described these detectors in an earlier paper [13]. 4. Experimental work 4.1 Databases We used the publicly available MCYT-100 subset from the MCYT database, which contains signatures from 100 subjects captured with a digitizing tablet and its stylus [1]. The important details of this database are summarized in Table 3. The second database used in this study is our MOBISIG database [2] which contains signatures from 83 subjects captured with a Nexus 9 tablet with Android 6.0 operating system. There are two important differences between these two databases: (i) the sampling frequency is constant in the case of MCYT database (100 Hz), whereas it is event-based in the case of the MOBISIG database (approximately 60 Hz); (ii) signatures in MCYT database were drawn by using a

7 Finger or Stylus: Their Impact on the Perf. of On-line Sign. Ver. Syst. 17 stylus pen, whereas those in the MOBISIG database were drawn by the fingertip of the subjects. Table 3: Basic characteristics of the MCYT and MOBISIG databases MCYT MOBISIG Device WACOM, INTUOS A6 Nexus 9 tablet, Android 6.0 USB. Drawing Stylus (pen) Finger Frequency 100 Hz (periodic sampling) Approx. 60 Hz (event based sampling) Coordinates x: , y: x: , y: Used raw data x(t), y(t), p(t) x(t), y(t), p(t) Sessions NA 3 Subjects 100 (44 women, 56 men) 83 (34 women, 49 men) Samples 25 genuine, 25 forgery Sess1: 15 genuine Sess2: 15 genuine, 10 forgery Sess3: 15 genuine, 10 forgery 4.2 Evaluation protocol We used the same evaluation protocol for both databases. We evaluated separately the skilled forgery and random forgery cases. Skilled forgery evaluation used the collected skilled forgery samples, whereas random forgery evaluation used randomly selected samples from the dataset. The evaluation was repeated three times using the first 5, 10 and 15 genuine samples for training. The next 20, 15, and 10 genuine samples were used for positive score computations, and all the available skilled forgery samples for negative score computations (skilled forgery case). In the random forgery case for each subject we selected as negative samples the first sample from every other subject (N-1 samples, N is the number of subjects in a database). The same evaluation protocol was followed for both global and local system, except that in the case of the local system, on the MOBISIG database, we always used the 15 genuine samples from the second session for positive score computations. Two types of Equal Error Rates (EER) were computed. Global EER (EER g ) was computed using a common genuine and forgery score lists for all the subjects of a database. This type of EER is based on a global threshold. User specific EER was computed by using user specific threshold (EER u ). In this case both mean (μ) and standard deviation (σ) are reported.

8 18 M. Antal, A. Bandi 4.3 Results Table 4 summarizes the performances for the global system, whereas Table 5 shows the same performances for the local system. These tables present the evaluations in the case of using 15 samples for template creation. The rest of the results can be seen at Table 4: Global system performance results (EER in %) for different feature sets. 15 training samples, 10 positive test samples, 20 negative test samples, and Euclidean detector. Skilled Forgery MOBISIG MCYT EER g EER u : μ(σ) EER g EER u : μ(σ) (1) Inf.th (11.94) (9.17) (2) St. o.t. art (8.48) (6.40) (1)+(2) Both (8.71) (5.79) Random Forgery MOBISIG MCYT EER g EER u : μ(σ) EER g EER u : μ(σ) (1) Inf.th (6.18) (6.51) (2) St. o.t. art (4.74) (3.53) (1)+(2) Both (3.82) (3.45) Table 5: Local system performance results (EER in %) for different time series. 15 training samples. DTW algorithm. Skilled Forgery MOBISIG MCYT EER g EER u : μ(σ) EER g EER u : μ(σ) xy (12.63) (5.02) x y (10.26) (3.88) x y (17.06) (3.78) xyx y x y (11.37) (3.78) xyx y x y pp (9.10) (3.72) Random Forgery MOBISIG MCYT EER g EER u : μ(σ) EER g EER u : μ(σ) xy (1.00) (1.01) x y (0.00) (0.18) x y (3.36) (0.37)

9 Finger or Stylus: Their Impact on the Perf. of On-line Sign. Ver. Syst. 19 xyx y x y (0.09) (0.15) xyx y x y pp (0.63) (0.15) Detection Error Trade-off (DET) curves [14] were created using the Matlab package available at NIST 1. Fig. 3. shows the results obtained for skilled forgery type evaluation of the two databases. In order to show the discrimination ability of a feature set, separate curves are shown for using information theory features (12), state-of-the-art features (15), and both types of features together (27). Fig. 4. shows the same error curves for random forgery type evaluation. Figure 3: Verification performance results for the MCYT and MOBISIG databases. Skilled forgeries case. 1

10 20 M. Antal, A. Bandi Figure 4: Verification performance results for the MCYT and MOBISIG databases. Random Forgeries case. 5. Conclusions In this paper we have compared - in terms of verification performance - two on-line signature databases, the MCYT database, which was collected using a high quality digitizing tablet with a stylus for drawing the signatures, and MOBISIG, which used a mobile device (Nexus 9 tablet with Android 6.0 operating system) and finger for data collection. The performance gaps between the two databases are highly significant, especially for skilled forgery type evaluation. Error rates for finger-drawn signatures are almost twice as high as for stylus-drawn signatures in the case of global systems (skilled forgery case). The local features were more strongly influenced by the signature drawing modality. In this case the error rates for the MOBISIG database were five or six times higher than for the MCYT database. The discrepancy between EER g and EER u has been observed by other researchers [4]. However, in the case of local systems there are unusually large differences. Despite the fact that we have taken into account the successive equal values in the time series, information theoretical features have resulted in little improvement (around 1%) in the performance of the global system.

11 Finger or Stylus: Their Impact on the Perf. of On-line Sign. Ver. Syst. 21 Acknowledgements The work of András BANDI was supported by a Student Research Grant offered by Accenture Romania. The work of Margit ANTAL was partially supported by the Sapientia Foundation Institute for Scientific Research. References [1] J. Ortega-Garcia et al., MCYT baseline corpus: a bimodal biometric database, IEE Proc. - Vision, Image, Signal Process., vol. 150, no. 6, pp , [2] M. Antal, The MOBISIG on-line signature database. Tirgu Mures, [3] R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega- Garcia, Benchmarking desktop and mobile handwriting across COTS devices: The e-biosign biometric database., PLoS One, vol. 12, no. 5, [4] M. Martinez-Diaz, J. Fierrez, and J. Galbally, The DooDB Graphical Password Database: Data Analysis and Benchmark Results, Access, IEEE, vol. 1, pp , [5] P. Bissig, Signature Verification on Finger Operated Touchscreen Devices, ETH Zürich, Distributed Computer Group, [6] M. Martinez-Diaz, J. Fierrez, and J. Galbally, Graphical Password- Based User Authentication With Free-Form Doodles, IEEE Trans. Human-Machine Syst., vol. 46, no. 4, pp , Aug [7] S.-B. Napa and N. Memon, Online Signature Verification on Mobile Devices, Inf. Forensics Secur. IEEE Trans., vol. 9, no. 6, pp , Jun [8] DTW algorithm. [Online]. Available: [Accessed: 12- Oct-2017]. [9] O. A. Rosso, R. Ospina, and A. C. Frery, Classification and Verification of Handwritten Signatures with Time Causal Information Theory Quantifiers, PLoS One, vol. 11, no. 12, [10] M. Antal and L. Z. Szabo, Some remarks on a set of information theory features used for on-line signature verification, in th International Symposium on Digital Forensic and Security (ISDFS), 2017, pp [11] L. Zunino, F. Olivares, F. Scholkmann, and O. A. Rosso, Permutation entropy based time series analysis: Equalities in the input signal can lead

12 22 M. Antal, A. Bandi to false conclusions, Phys. Lett. A, vol. 381, no. 22, pp , Jun [12] J. Fierrez-Aguilar, L. Nanni, J. Lopez-Peñalba, J. Ortega-Garcia, and D. Maltoni, An On-line Signature Verification System Based on Fusion of Local and Global Information, in Proceedings of the 5th International Conference on Audio- and Video-Based Biometric Person Authentication, 2005, pp [13] M. Antal and L. Z. Szabo, On-line verification of finger drawn signatures, in 2016 IEEE 11th International Symposium on Applied Computational Intelligence and Informatics (SACI), 2016, pp [14] A. Martin, G. Doddington, T. Kamm, M. Ordowski, and M. Przybocki, The DET Curve in Assessment of Detection Task Performance, Proc. Eurospeech 97, pp , 1997.