Writer Identification from Gray Level Distribution

Transcription

1 Writer Identification from Gray Level Distribution M. WIROTIUS 1, A. SEROPIAN 2, N. VINCENT 1 1 Laboratoire d'informatique Université de Tours FRANCE vincent@univ-tours.fr 2 Laboratoire d'optique Appliquée Université de Toulon et du Var FRANCE Abstract When identifying a writer from a handwritten text, most often, either some characteristic patterns or some shape parameters are extracted. They are assumed to be specific of the writer. Here, we are to explore a different approach, we consider the distribution of the pixel gray levels within the line. It is linked to pressure and writing speed when text is realized. In the line, the direction that is perpendicular to the writing way of drawing is privileged. The curve associated with the gray levels in a stroke section is characterized by use of 4 shape parameters. More over the regular sections are selected and are grouped in section lots. The distributions of the sections and of the section lots are quantified. Thus 22 parameters are extracted. Three different classifiers are used with and without genetic selection of the most significant parameters for the classifier. Then the classifiers are combined and the results show the gray level distribution within the writing is characterizing the writer in a significant way. keywords : writer identification, gray-level image, gray level distribution, stroke section. 1. Introduction In spite of the development of electronic documents use, the paper documents are still more and more exchanged and the problems of identification and authentication of the writers take a more and more important place. At any time, studies have been performed to characterize the writings; they are achieved at different levels, according to the style [1] and more precisely to the writer [2, 3]. The writer identification process is also performed on legal papers by way of a signature. But, in some cases, it is also needed to identify a handwritten document without a signature, using only the handwritten text and in an independent way from the text content. Human experts, lawyers or forgers, have a profound knowledge of the nature of handwriting. It happens very difficult to do as well as they do in an automatic way. A better understanding of the handwriting formation process would be necessary. Nevertheless, it is commonly admitted that a good forgery cannot be realized without a simultaneous copy of the shape and of the drawing realization speed. May be this explains the identification systems relying on on-line data better perform than those relying on off-line data. In the case of on-line writing, three types of information are displayed: shape, speed and pressure at any point of the line. In this study we are focusing on off line handwriting that has been scanned. In order to work well, human expert needs the original writing rather than a copy. That is to say his judgment relies on all the details that are contained in the writing, the parts that seem quite regular but also those that present some irregularities. Besides, studies that rely on the image of the writing are most often performed on binary images and this could explain the low level of the results. The studies are privileging too much the patterns rather than the study of the line itself. Only the patterns are compared between a reference writing and the writing to be identified [2]. Then the benefit of the global aspect the initial text can have in the human perception is lost. This global vision concerns the perception of the exact contours as well as the perception of the pressure that has been exerted. It is perceived from the distribution of the gray levels within the line. It is quite sure that the use of gray level images would bring some supplementary information; it must allow a better handling of the problem and would be able to contribute to the writer identification for example. The results would be notably improved. Here, the aim of the study we are presenting is not to obtain a perfect identification of the writers but rather to show how line observation can lead to a partial result that could help when included in the more traditional studies. Here, the shape of the lines and the writing style are nearly omitted. In the first part, we intend to show how we are to use the line and we present how we have considered the distribution of the pixel gray levels. The pixels have been grouped according to the sections of the line. Then we present the parameters that we have extracted in order to

2 characterize the writing. Finally the identification of the writing will be performed by combining the results of several classifiers. The selection of the significant parameters by a genetic algorithm will be explained and some results will be presented. 2. Sections of the writing We consider three elements are complementary in the writing performance, shape, speed and pressure that are expressed during the realization of the writing. The physicians take into account the writing pressure when they have a neurology disease to diagnose. The measure is difficult. They look at the number of paper sheets that are impressed when writing. In [4] the link is established between contour smoothness, pressure and speed. In a more perceptive way, it can be said the darker the line is, the more important has been the pressure. Of course this has some consequence on the pixel gray levels in the writing image. For our purpose an average level computed on the whole image is not sufficient. Indeed, the line is far from being uniform. Some parts are more marked than others. Some ends of strokes are often thinner and are nearly not inked. More over the outline of the drawing is more or less dark; in some writings the darkest pixels figure a writing skeleton. In Figure 1 the evolution of the pixel gray levels along a section can be seen. Figure 1. Evolution of pixel gray levels along a section With respect to the pixel gray levels, a section of the writing line most often presents a minimum. The symmetry that could be expected from the section with respect to the minimum presents a large variability according to the writer; it is linked to the way the pen is placed on the paper. This aspect of the curve can be compared to the angular parameter that is at disposal in some dynamic acquisition systems. We are now to define in a more precise way what we call stroke or line section. In order to get as much information as possible about the distribution of the gray levels, we are working on handwritten text images that have been scanned with a 600ppi resolution. All the images were acquired with the same scanner Extraction of the sections The sections can be considered in different ways. They can be longitudinal, perpendicular to the stroke line or with a random direction. A section of the line along the graphical line gives some information about the regularity of the pressure during writing. But it can be noticed that large variations are linked to the written words. Here, the problem will be tackled in a general way. A statistical study is necessary in order to make the method independent from the text content. The use of longitudinal section, in the field of writer identification would need a longer text. Then, this type of section has been excluded from our study. In the same way, we have excluded the use of random direction supports in the text image. Thus we have chosen to consider the sections that are perpendicular to the stroke axis. A priori they are regularly spaced along the writing thread. Nevertheless, the data is an image of the text and not some on-line signal. Then the writing line axis has to be recovered, at least partially. Often, the skeleton is used, but the precision is not enough, the many barbells have no actual correspondence with the lines drawn by the writer. The outline of the drawing is much more significant with respect to the writer than the skeleton. We are going to use the contour and make the section line extraction from the contour. This makes implicit the hypothesis that at least locally, a line can be approximated by a rectangle. Then a section is modeled by the list of the pixel gray levels along a straight line perpendicular to the writing thread, that is to say perpendicular to the tangent line at a contour point. Our goal is to extract as much information as possible from the gray level data. Then, in order to find the writing contour, we cannot limit our investigation to a binarisation step that would give evident contours. More over the sections are characterized by two elements, on the one hand by the position of the straight line support and on the other hand by the gray level values along a segment. These two elements are successively determined. First the section direction is determined on a binary image of the text, and then the exact segment is precisely defined by a local study of the pixel gray levels. Contour extraction is realized from a binary image that is obtained using the scanner with same resolution as the gray level image has, that is to say 600dpi. In order to give sense to the tangent line at a writing contour point, a morphological smoothing is performed on the binary image to regularize the contours. At any point of the contour, the tangent line is computed as the regression straight line of the set of 11 contour pixels nearest to the studied contour point. This technique also minimizes the impact of irregular points on the contours. It can be seen in Figure 2 the supports of the section are coherent with a skeleton image of the writing.

3 Figure 2. Section location with respect to skeleton Thus the section supports are detected; anew we are now to take into account the gray level data to extract the line exact position. A priori the extremities of the sections are given by the binary image. They are considered as an approximation we want to improve with respect to local properties of the writing. The evolution of the gray levels along the section support is studied. Very often, the writing line is not entirely within the pattern extracted by the binary image. Then, we extend the investigation domain over to the points the gray levels of which are quite near the average gray level of the image background. In Figure 3 an example of the modification we can obtain is shown. A section issued from the binary image is in (a) and in (b) the section is extended from the gray level data. - Those the length of which is inferior or superior to a specific ratio of the average length of all the sections are not representative. Then it is possible to suppress the sections that are located in the part of the word where the writer has printed several times. These parts are not representative of a line drawing. An example can be seen in Figure 4. Figure 4. Example of a double line and sections to be suppressed - When a stroke is finishing, the contour, as in any other place, is closed, but the pixels cannot be associated with the extremity of a line section, according to our definition. Nevertheless these points are processed but the associated segments are not sections. Thus the sections for which the distance between any point and the contour is too low are suppressed. - As we have processed the entire writing contour and as a section touches the contour line at two points, we find some redundant sections, those that are identical. - Besides, some of the detected section supports are perpendicular to the writing line only at one of the extremities and then they do not have the length of the width stroke. They also have to be suppressed. An example is shown in Figure 5. These sections, nearly always, appear among the too long sections. (a) Figure 3. Profile of a section before and after gray level data process The section detection is achieved only every two pixels along the contour. As a consequence, on the one hand, the sections number is important and on the other hand, some of them are not coherent with the definition that has been first given. Indeed, for each section, only one contour extremity has been considered for the construction. Then a selection among these sections is needed. (b) Figure 5. Non significant sections example The final result of the extraction of selected sections is presented in Figure 6. These sections can be qualified as regular sections in the considered writing Significant sections selection Several criteria are used in order to select the sections that will be further used in the study. Figure 6. Set of the selected sections

4 2.3. Local grouping of sections Before we characterize the selected sections, we have defined a third observation level, the level of the sections "lots". A lot is a sub-set of sections that are spatially near in a transitive way. In this sense a lot corresponds to the natural vision of an homogeneous stroke. course the classes that are obtained for different writers have not always the same meaning. In order to compare between writers, we sort the classes in decreasing order according to their importance in the written text, that is to say their occurrence frequency. This description gives a vector with 16 components extracted from the distribution of the gray levels within the written line Section distribution Figure 7. The section lots are presented using different gray level colors From this selected and transformed data we are to define some parameters that are representative of the writing and more precisely of the writer. They are parameters that highlight some hidden phenomena that are not contained in the only shapes of the letters and of the whole writing. They can be said biometrics parameters. 3. The parameters We can classify our parameters within two large families. On the one hand those that are associated with the sections themselves, and on the other hand those that are associated with the disposition of the selected sections The regular sections The aim is to define some parameters that would describe the shape of the profile of the sections. That is the aspect of the curve we want to quantify. We have chosen to describe a section using 4 parameters, its length, the level of the darkest pixel along the section, the skewness parameter, and the asymmetry coefficient. We hope we characterize the writer by the whole set of selected sections. Then we have to compact the extracted information in a very limited number of parameters. According to the patterns involved in the letters, all parts of the strokes are not identical; they also vary according to the position in the letters themselves. The extremity of a stroke or the vertical parts of a stroke can be distinguished for example. Then we have considered the population of the sections was not an homogeneous set. In order to increase the homogeneity of the elements we want to describe, we have decided to consider 4 subsets among the set of sections. The different sets are obtained in an adaptive way according to the writer by use of the clustering k-means algorithm. For each of the 4 classes we consider the center of gravity of the class. Of Here we are to define some other parameters that allow to describe the distribution of the selected sections along the writing line thread. We have already defined the section lots after some sections have been suppressed from the study. Then we consider the percentage of the sections that have been suppressed OS. On the whole, the remaining sections act as separations in the line zone, some pixel connected components are defined. The average area of the connected components is chosen as one of the parameters AAW as well as the percentage of connected components with an area greater than 50 pixels AAW50. We also consider some parameters that indicate the variability at different observation levels. There are 3 of them: the density SR of the sections computed as the ratio between the number of selected sections and the length of the skeleton; the density LR of the lots computed as the ratio between the number of lots and the number of sections; the average number SL of section classes that are present in the lots. A normalization of the parameter values on the interval [0,1] is performed according to the values that have been observed in the learning process. 4. Writer identification To choose a classifier is a difficult task especially when we have no precise a priori information on the shape of the classes that are to be characterized, that is to say the set of points associated with texts written by an only writer. So we have been interested in the use of several classifiers relying on different hypotheses The classifiers We consider three classifiers. We have chosen two nearest neighbor classifiers, one relies on the Euclidean distance ED and the other relies on the Mahanalobis distance MD. The hypotheses on the shape of the classes are different. In the first case they are assumed to be isotropic in the representation space. Thanks to Mahanalobis distance the shapes of the classes are adapted to data. With these two classifiers, separation is not linear.

5 Then we have chosen a third classifier that is linear and deduced from a descriptive discriminant analysis DA. The results obtained with each of the classifiers are quite disappointing, 27, 14 and 51 per cent of good identification are obtained Selection of the best variables We have chosen different variables but we have no real information about their independence. We find it wise to try and minimize the number of variables. Of course, the significant variables would depend on the classifier we consider. The selection way is relying on genetic algorithms. The optimization is achieved by the set of right variables to be considered, and the identification rate is to be optimized on the learning set. To a set of considered variables is associated a Boolean vector with 22 dimensions. It will be used as a gene. The fitness function is the recognition rate obtained using the selected variables by a classifier relying on one of the three previous principles. In order to obtain an unbiaised measurement of classifier quality, the leave one out method is used. In each case the population of variables t- uples evolves along the generations according to the genetic algorithm and the classical crossing, mutation and selection operators. We could observe that, depending on the type of classifier, the parameters that have been selected were not the same. Nevertheless AAW, OS, SR and LR are always selected. As it is essentially a stochastic process, it has been repeated several times with a great stability in the results. In Table 1, for each type of classifier is indicated the number of selected parameters. A significant improvement of the results can also be observed, respectively 17, 39 and 20 per cent for each classifier. L base and T base denote respectively the learning base and the test base Fusion Obviously the classifiers have various behaviors according to the writer. Then we have investigated further each of the classifiers and analyzed the confusion matrix associated with the M writers. n denotes the number of elements of class i that are recognized with label j by the k th classifier in the learning set. This allows weighting the results with some confidence in the classifier according to the recognized writer. For a writing x, e k (x)=j k is the decision of classifier k. The final decision function is computed according to the method described in [5]. It can be expressed from the decision of each classifier. Knowing the decision of each of the K classifiers j 1,j 2, j K, the decision would be class i that gives next expression its maximum value. i,j M k= 1 n i,l l= 1 The results are indicated in Table 1. 3 n i,j k Table 1. Identification rates classifier ED MD DA Fusion Nvar L base T base Conclusion Whereas many approaches concerned with writer identification study the patterns and the style of the writing, we have here shown how the use of gray level distribution is a very important element that can help in discriminating the writers. Of course the study would have to be processed with a larger number of writers. Besides the results we have obtained are quite convincing, they could be used in order to improve those obtained by pattern study, either by fusion of the results or by considering some other parameters. 6. References [1] J.-P. Crettez, A set of handwriting families: style recognition, Inter. Conf. on Document Analysis and Recognition (ICDAR'95), Montreal (Canada), 1995, pp [2] A. Nosary, L. Heutte, T. Paquet and Y. Lecourtier, Defining writer's invariants to adapt the recognition task, International Conference on Document Analysis and Recognition (ICDAR'99), Bangalore (India), 1999, pp [3] A. Séropian, N. Vincent, Writers Authentication and Fractal Compression, 8 th Inter. Work. on Frontiers in Handwriting Recognition IWFHR VIII, Niagara on the Lake, IEEE, 2002, pp [4] K. Franke, T. Sy, O. Bünnemeyer, Ink texture analysis for writer identification, 8 th Inter. Work. on Frontiers in Handwriting Recognition, IWFHR VIII, Niagara on the Lake, IEEE Publisher, 2002, pp [5] L. Xu, A. Kryzak, C.Y. Suen, "Methods of combining multiple classifiers and their applications to handwriting recognition", IEEE Transactions on systems, man, and cybernetics, 1992, vol. 22, n 3, pp