Writer Authentication Based on the Analysis of Strokes

Transcription

1 Writer Authentication Based on the Analysis of Strokes Kun Yu, Yunhong Wang, Tieniu Tan * NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, P.R.China ABSTRACT This paper presents an on-line handwriting authentication system for text-independent Chinese handwriting. The proposed strategy is implemented on the stroke level, and the writing strokes and interstrokes are separated stepwise. The writing features are extracted from the dynamics of substrokes and interstrokes, including the writing velocity, the pressure, and the angle between the pen and the writing surface. To alleviate the effect of writing character number on the performance of the algorithm, we adopt the feature vectors of selected dimensions. In live experiments the authentication result is promising. KEYWORDS: On-line, stroke, substroke, interstroke, feature selection INTRODUCTION Every person has his own style of writing, and thus handwriting has long been considered bearing traits embodying the personal identity, and in many cases, we can decide the writer of certain handwriting just after a brief look. Handwriting analysis has long been used to assist in cases under police and judicature situations, especially in those concerning handwriting authentication. Unfortunately, because the problems are so complex that the correlative authentication work is usually finished by manual matching, which is time-costing and relies on the specific experience of the expert. Recently people have resorted to the computers for help to explore the essence of writing, and the pertinent research has been drastically boosted. This work is focused on on-line handwriting authentication. Writer authentication deals with problems concerning selecting the writer of a specific handwriting after comparison with all the candidates for reference. People have focused on this area of research for decades with different tools, but because of the complex mechanism and various situations of writing, still too many problems are left unsolved. Nowadays there is a consensus of opinions among researchers that handwriting is an art [][9], and they have switched their focus onto many specific problems with certain constraints, such as off-line or on-line situations, or just confined in signatures of some scripts. The strategy introduced in this paper is especially designed for the on-line text-independent case. Many people have made great work concerning text-dependent handwriting [2][9], and many methods have been applied to signatures with good performance. The prevalent methods include HMM [3] and DTW [4] etc. With the written text for reference, the problem is simplified to some extent, but in most cases we do not know the written text, and the methods * {kyu, wangyh, tnt}@nlpr.ia.ac.cn, phone ; fax ; nlpr-web.ia.ac.cns

2 applied to the text-dependent cases are no longer feasible. So we must find new methods to tackle this problem. Global methods for handwriting authentication have made promising achievements [5], but methods based on local analysis [6] still needs further exploration. As an expansion of our previous work in NLPR [7], our method starts from the writing strokes and substrokes of the handwriting, and tries to capture each aspect of the writing style of the writer. Different from our previous algorithm, we have further considered the interstrokes for feature extraction, and the matching strategy is modified accordingly. The flow chart in Fig. describes the procedure of the proposed algorithm. Substrokes Dynamic Features Training Samples Preprocessing Feature Extraction Feature Selection Feature Set Writer Feature Code Substrokes Dynamic Features Test Samples Preprocessing Feature Extraction Pattern Matching Conclusion Figure. The flow chart of the handwriting authentication system The data set we adopt is the NLPR handwriting data set, which includes the on-line writing samples of 55 subjects collected with the interval from two weeks to one month. The handwriting is sampled with a frequency of 200 dots per second, with the resolution of 2450 lines per inch. The remainder of this paper is organized as follows: the preprocessing steps are introduced in section 2, and the next part in section 3 describes the features extracted. Section 4 will introduce the classifier we employed, and in section 5, we described the experiments in detail. In section 6 we draw the conclusion. 2 PREPROCESSING On-line handwriting is different from off-line handwriting in that the whole writing course is captured as a sequence of dots, with the time, writing pressure and pen inclination information recorded. The information mentioned above is considered representative of the writing style of the writers. In the following steps we segment the writing sample into strokes [9] and substrokes by time, in order to get an explicit description of the writing attributes of the writers. 2. Segmentation of strokes The handwriting retrieved from the original sample dots is shown in Fig. 2. In order to organize the different handwriting into a uniform framework, we adopt the strokes in common use [6]. A stroke is the complete trajectory of the penpoint. From Fig. 2 we can see that some handwriting samples comprise cursive strokes while others compose

3 strokes of mainly straight lines. Fig. 2 Writing samples in NLPR data set Fig. 3 shows an original sample and the segmentation result. In the Fig. 3 (b), the grey level of the writing dots represents the writing pressure, and the dark rectangle dots are indicators of the starts and ends of the strokes respectively. The interstrokes are the trajectories of the penpoint from the end of one stroke to the start of the successive stroke. Both the strokes and the interstrokes constitute the complete trajectory of the penpoint. (a) (b) Fig. 3 Segmenting the original writing sample into strokes 2.2 Segmentation of substrokes and interstrokes The substrokes are segmented according to the following rule combining the effect of local positional maximum and pressure maximum: V w + = x x x x Pos D D D D sgn[( ( ) ( )) ( ( ) ( ))] t t t t w 2 sgn[( D ( y) ( )) ( ( ) ( ))] t D y y y t D t D t+ + ()

4 V w w = sgn[ D ( p ) D ( p )] + sgn[ D ( p ) D ( p )] (2) Pre p t t p2 t t+ V = Tol V + Pre V (3) Pos where D ( x), ( ) t D y and ( ) t D p is the attribute of the sample dot at time t. x, y, p are the coordinates and t the pressure of the dot D respectively. The weight coefficients, namely t w, w, 2 w and p w are p 2 selected experientially. In the system w and w play the key role, and 2 w and p p 2 w are auxiliary coefficients. V is the positional score of the point in consideration, and is the segmentation score concerning Pos Pre the writing pressure, here the formula is based on the assumption that pressure maximums usually occur at corners of the strokes, and in our experiments, we have found this assumption feasible. The judgment whether the dot in consideration is a segmentation point is decided by the total score V V V, which is the sum of Tol V and Pos Pre. The threshold for V Tol is fixed on. In Fig. 4 the black rectangle dots represent the start and end points of the substrokes. Fig. 4 Segment the original writing sample into substrokes. Similarly, the interstroke trajectories are segmented according to formula (), because the dots within an interstroke trajectory bear no information of pressure. The weight coefficients are magnified accordingly to ensure that the threshold for V Pos remains. Here a time threshold is necessary for the decision of an interstroke, because the time interval of moving the penpoint from one line to another is much longer than the time interval between successive strokes. 2.3 Substroke type classification After the strokes are segmented into substrokes, the substrokes are classified into 3 primary types using similar

5 strategy of [7]. The substrokes are shown in Tab. and the arrow beside each substroke indicates its writing direction. The structural proportion in Tab. means the statistical proportion of the respective substroke in Chinese characters [8]. The sample proportion is obtained based on the NLPR handwriting data set. When we consider the substroke type, the writing pressure and position of key points within the substroke is considered. The key points usually include the start point and the end point. But when the curves listed from 8 to 2 in the following table are considered, another point is necessary to discriminate the different curve types. All the substrokes, which are not reliably classified into the first 2 types, are collected in type 0, and discarded in the following steps. The substroke is matched in the same order as the serial number listed below, which helps to reduce the overall matching time. Tab. Primary substroke types We adopt similar but different strategy in processing the interstrokes, for interstrokes are not so complex compared with the substrokes. The interstroke types are listed below in Tab. 2. Tab. 2 Primary interstroke types The above arrows from No. to No. 6 represent the writing direction of the interstrokes. Interstroke No. 7 means the penpoint is suspending above the surface of the tablet, with little or no movement in position. Gesture information is extracted from this interstroke, which reflects the habit of the writer when grasping the pen for a short period. 3 FEATURE EXTRACTION The writing features are extracted from the substrokes and interstrokes utilizing statistical method. We adopt the Gaussian Models to describe the attributes of handwriting according to the histograms of the pressure, writing velocity, and azimuth, altitude of the pen (Fig. 5). Gaussian Mixture Models have also been in our consideration, but because of the drastic time cost in modeling, they are finally phased out.

6 Fig. 5 Histogram of pressure, velocity, azimuth and altitude of substroke type. For each type of substroke, four Gaussian Models are utilized for the pressure, the writing velocity, the azimuth and the altitude respectively. Concerning the first 6 types interstrokes, three Gaussian Models for the writing velocity, the azimuth and the altitude are adopted. The interstroke of type 7 bears no information, so we just utilize two models to describe the azimuth and altitude. The dimensions of the writing features are listed in Tab. 3. TYPE SUBSTROKE 4*2 4*2 4*2 4*2 4*2 4*2 4*2 4*2 4*2 4*2 4*2 4*2 INTERSTROKE 3*2 3*2 3*2 3*2 3*2 3*2 2*2 Tab. 3 Components of the feature vector 4 CLASSIFIER DESIGN The feature vector includes 36 components corresponding to the means and deviations of the 68 Gaussian Models. Although the dimensionality of the feature vector is not very high, it already leads to inconvenience in feature matching. Meanwhile, it could be argued that the number of some types of substrokes or interstrokes is not so sufficient as to reliably extract the parameters of the Gaussian Models. This problem is associated with the number of characters, which has been an important factor for almost all prevalent algorithms for handwriting analysis. To improve the efficiency of the algorithm and at the same time ensure the accuracy, we adopted the feature vectors of reduced dimensions to alleviate the effect of character shortage. Namely, the number of substrokes and interstrokes of each type decides the reliability of respective Gaussian Models. The matching score is evaluated using the following rule [0]: M( x) = ( ) σ 68 2 w i 2 µ x µ i i= i (4)

7 2 where µ, i σ are the parameters of the Gaussian models respectively, and i w is the weight coefficient i correlative with the reliability of the Gaussian Model. If there are too few segments allocated into some types of substroke or interstroke, their weight coefficients are assigned to 0, and other weight coefficients are magnified accordingly. 5 EXPERIMENTS The robustness and the accuracy of the system were tested on the NLPR handwriting data set. This data set includes the handwriting of 55 contributors, among which 6 writers are females. The writing document for template extraction includes nearly 600 Chinese characters written in 2 pages, which is enough for extraction of the feature templates. All of the test experiments are carried out in the same set-up with the written characters increased stepwise. The horizontal coordinates in the following graphs represent the writing lines and each writing line consists of nearly 20 Chinese characters, and the vertical coordinates represent the identification accuracy. The first group of experiments is carried out to evaluate the effect of the selected feature vectors. In Fig. 6, the dashed line represents the accuracy of the algorithm using feature vector from all the 68 Gaussian Models, and the real line using the Gaussian Models selected by the reliability coefficients. It is proved that the feature vectors with reduced dimensions get better performance than the fixed feature vectors, especially when the number of characters is small fixed adaptive Fig. 6 Performance of the method with feature selection The other group of experiments is focused on the performance of different writing features. The identification result is concerned with the substroke information, the interstroke information, and all the information synthesized. For the convenience of comparison, the results utilizing writing angle and writing velocity are shown in the same graph. As proved in our previous work [7], the method using writing angles alone, namely the information associated with the inclination of the pen outperforms all other features when the number of characters is small. As the number of character increases, the combination of features performs much better than any single aspect of writing information. Compared with our previous work [7], if we adopt the features extracted from the interstrokes alone, the performance of the

8 algorithm is far from satisfactory. Based on the algorithm using substroke information, there is tangible improvement in the identification performance when we phase in the interstroke features, especially when the number of characters is ensured, but the time cost of the algorithm almost doubles. Therefore, the proposed method should be adapted to work according to specific requirements angle velocity substroke interstroke all Fig. 7 Performance of different features Classification errors occur in the writing samples of Fig. 8. From the analysis of the feature vectors, we find that some factors do have negative effect on the proposed method. Firstly, the number of characters must be enough to ensure that the dimensionality of the feature vectors is ensured. This factor answers for the misclassification in Fig. 8(a). Moreover, the time interval between successive strokes in this sample is relative longer than in common cases, which is another disadvantage for the analysis of interstrokes. Secondly, the writing time have impact on the dynamic features of the writing procedure. After some time of writing, as the writer gets tired, the writing pressure decreases gradually, and the writing angles are influenced as well. The sample in Fig. 8(b) is collected after some period of writing, and it is misclassified because of the unstable inclination angles of the pen. The writing interface also has some impact on the writer, for those who do not get accustomed to the pen easily. The pertinent misclassified writing sample is shown in Fig. 8(c). (a) (b)

9 (c) Fig. 8. Misclassified samples. 6 CONCLUSION This paper has proposed and validated a strategy for text-independent handwriting authentication. The method introduced in this paper has shown promising performance both in live experiments and in the laboratory setup. Different from many traditional methods [][2], this method tries to break through the constraints of fixed written text, and thus the substrokes, together with interstrokes are selected as the primary unit for feature extraction. Feature selection further improves the accuracy and time cost of the system. Although the NLPR data set and the experiments are focused on Chinese characters, with minor modifications the proposed method could be extended to handwriting of other oriental characters, such as hiragana, katakana, where characters can also be separated into strokes and substrokes. 7 ACKNOWLEDGEMENTS This work is sponsored by the Natural Science Foundation of China under Grant No and Our thanks also goes to our colleagues in NLPR who have generously helped us with data collection, and those who have offered great support to us in the research arena. REFERENCES [] [2] Daigo Muramatsu and Takashi Matsumoto, An HMM On-line Signature Verifier Incorporating Signature Trajectories, IEEE ICDAR 03. [3] Edson J. R. Justino, Flavio Bortolozzi and Robert Sabourin, Off-line Signature verification using HMM for random, simple and skilled forgeries, IEEE ICDAR 0. [4] Diana Kalenova, Personal authentication using signature recognition, [5] H.E.S. Said, T.N. Tan and K.D. Baker, Personal Identification Based on Handwriting, Pattern Recognition 33 (2000), pp [6] In-Jung Kim and Jin-Hyung Kim, Statistical Character Structure Modeling and Its Application to Handwritten Chinese Character Recognition, IEEE Trans. on PAMI, Vol. 25, No., Nov [7] Kun Yu, Yunhong Wang and Tieniu Tan, Writer Identification Using Dynamic Features, ICBA 04.

10 [8] [9] Mi-Gyung Cho and Hwan-Gue Cho, A Script Matching Algorithm for Oriental Characters on PDAs, ICPR02, Vol. 2, pp [0] Merijin van Erp, Louis Vuurpijl and Lambert Schomaker, An Overview and Comparison of Voting Methods for Pattern Recognition, IEEE IWFHR 02. [] Vishvjit S. Nalwa, Automatic On-Line Signature Verification, Proceedings of the IEEE, Vol. 85, No. 2, Feb. 997, pp [2] A. Lawrence Spitz, Determination of the Script and Language Content of Document Images, IEEE Trans. on PAMI, Vol. 9, No. 3, Mar. 997.