LEKHAK [MAL]: A System for Online Recognition of Handwritten Malayalam Characters

Transcription

1 LEKHAK [MAL]: A System for Online Recognition of Handwritten Malayalam Characters Gowri Shankar, V. Anoop and V. S. Chakravarthy, Department of Electrical Engineering, Indian Institute of Technology, Madras, Chennai {schakra@ee.iitm.ernet.in} ABSTRACT A system for online recognition of handwritten Malayalam characters is presented. A handwritten character is constructed by executing a sequence of strokes. A structure- or shape-based representation of a stroke is used in which a stroke is represented as a string of shape features. Using this string representation, an unknown stroke is identified by comparing it with a database of strokes. A full character is recognized by identifying all the component strokes. Development of similar systems for other Indian scripts is contemplated. 1. INTRODUCTION Online handwriting recognition consists of recognizing a script as it is written using an electronic stylus or a pen on a tablet. Parameters related to the pen tip like position, velocity, acceleration and sometimes pressure (on the writing surface) are available to the data acquisition system [1]. Although early work looks back to the sixties, electronic pen devices have gained special attention in the recent times due to the increased demand for more human-like interfaces with the computer. Systems based on online handwriting recognition fall into three categories: 1) pen-based computers, 2) signature verifiers, and 3) developmental tools. The first refers to small hand-held computers that use pen-based inputs due to restrictions on keyboard size etc. The second refers to online signature verification to ascertain the identity of a person. The third class of systems used for educational and rehabilitation purposes, incorporate knowledge of neuromotor characteristics of handwriting [2,3]. Online handwriting recognition takes on a novel significance in the context of Indian languages. Presently, word processing in Indian languages can be a vexing experience, considering the restriction on use of the regular keyboard, designed for English. Elaborate keyboard mapping systems are normally used in case of Indian languages, which are not convenient to use. A comfortable solution would be to let the user write in a natural, normal fashion using a suitable pen-like device and let the computer do the rest. That transfers the burden of learning keyboard mappings from the user to the computer. This is the motivating idea of the present work. We have recently developed an online character recognition system for Tamil. Taking advantage of the similarities of Tamil and Malayalam characters we now present a similar system for Malayalam also. 2. THE MALAYALAM CHARACTER SET Malayalam script, as other Indian scripts, is generally written in non-cursive style, unlike English handwriting, which is normally written in cursive style rendering recognition difficult. However, Indian scripts pose a peculiar problem non-existent in European scripts the problem of composite characters. Unlike in English alphabet where a single character represents a consonant or a vowel, in Indian scripts a composite character represents all the sounds that comprise a complete syllable. For example, the word stree, a single syllable comprising three consonant sounds and a vowel, is represented by a single

2 composite character in which graphical elements corresponding to the 3 consonants and the vowel are combined to form a complex structure. For this reason, though the number of consonants and vowels is a small number, the number of composite characters may well run into thousands. However, the problem of composite characters is not so critical in Malayalam since the number of different ways in which the basic characters can form composite characters is quite restricted. Stroke Database Pen Input Preprocessing Feature Extraction Stroke Recognition The Malayalam script consists of 15 vowels, 36 consonants, 27 special or Grantha characters and 17 individual vowel modifiers. As a general rule when the vowel appears at the beginning of the word, the full vowel character is displayed. When a vowel appears inside a word, a vowel modifier is added to the preceding consonant creating a composite character. The only composite characters in Malayalam are of consonantvowel type, generating only a small number of combinations. Further only in a small number of cases do vowel modifications lead to new character shapes. Typically, vowel modification is obtained by prefixing and/or suffixing a consonant with specific structures. These simplifying features make Malayalam an ideal candidate for automated pen-based recognition. 3 THE ALGORITHM In our approach to recognition, a handwritten character is represented as a combination of strokes. A stroke is defined as the trajectory negotiated by the pen tip from the instant when it first made contact with the writing surface to the earliest moment when that contact is broken. Therefore, the first step in character recognition is to recognize all the component strokes in a character. Once the component strokes are identified, the character itself can be easily identified. The main modules in our Malayalam online recognition system are shown in figure 1. Figure 1: A schematic describing process flow of the online Malayalam character recognition system. 3.1 Pen Input Stroke-Char LUT We have used the Superpen TM, a product of UC Logic Inc. to generate online character data. The product, which comes with an electronic pen and a tablet, can also be used as a regular substitute for a mouse. Therefore, the pen trajectory can be recorded directly using the standard functions used for reading mouse coordinates. The pen also has provision for real-time monitoring of the pressure exerted by the pen tip on the tablet. However, pressure information is not used in our recognition algorithm. Therefore, data output by this stage consists of a sequence of strokes, where each stroke consists of the x- and y-coordinates of the pen tip, over a finite interval of time Preprocessing Character Recognition Display/Archive Preprocessing consists of normalization and smoothing of strokes. The x- and y- coordinates of each stroke are normalized such that the stroke fits tightly inside a unit

3 square. The time-varying coordinate functions, x(t) and y(t), are then smoothed independently along t-axis using a Gaussian filter of suitable standard deviation. represented as a string of shape features. Therefore, the function of feature extraction stage is to convert strokes into strings of shape features Feature extraction A feature-based approach, rather than a template-based one, is perhaps more appropriate for handwritten character recognition, considering the extent of distortion that handwritten characters undergo. In an earlier work [4], we have identified certain general features known as the shape features of handwritten characters, which are less susceptible to distortion introduced by writing. A set of 18 shape features used in the present system are summarized in Fig. 2. The 18 shape features are denoted by uppercase English alphabets from A to R. Figure 3: Representing Malayalam character R (as in Rishi) as a string of shape features Stroke Identification In this step, the shape feature string of an unknown stroke is compared with a database of such strings. This stroke database has shape feature strings of all the strokes that form handwritten Malayalam characters. A single stroke may have multiple variations. All the variations of a single stroke are given a common integer identity the stroke ID. The stroke identification step consists of mapping an unknown string of shape features onto a stroke ID. Figure 2: List of shape features. Since the shape features occur in a sequence along a stroke as it is drawn through time, a given stroke may be represented as a string of shape features. Fig. 3 shows how the Malayalam vowel character R (as in Rishi), written in a single stroke, can be Regular string matching techniques give disastrous results since, often, stray features are inserted or expected features absent in the unknown string. Therefore, the situation warrants use of soft-matching of strings. Our experience shows that the feature string of a typical stroke consists of stable feature sequences or feature words interspersed by noisy features. Corresponding words in both the database string and the unknown string are matched. Adding up all such match values give the total match between the two strings String matching algorithm Initialization: smallstr = unknown string;

4 largestr = database string; matchval = 0; Step 0: swap() Step 1: Look for smallstr(1) within largestr(1:win). If the search fails go back to step 1; If the search succeeds, clip largestr up to the point of match; swap(); go to step 2. Set i = 1; Step 2: Check if smallstr(i) = largestr(i). If true increment matchval by 1; i = i + 1; go to Step 2. If false go to Step 3. Step 3: Clip both strings up to point of last match. If the length of the smaller string is 0, STOP; else go to Step 1. (Note: The function swap() swaps largestr and smallstr if the length of the largestr happens to be smaller than the smallstr.) 3.5. Character Recognition Since a character is constructed by one or more strokes, recognizing a character consists of grouping stroke IDs into character IDs. This process is particularly simple in Malayalam, compared to other Indian scripts, since a majority of characters may be written using single strokes. Two kinds of character codes exist for any character set used in a word-processor program. One is a syntactic or a syllabic code, like the ASCII code for instance, and the other is a glyph code, which is the font code. For Indian scripts there is a syntactic or a syllabic code based on the sounds of the characters. Noted among them is the Indian Script Code for Information Interchange (ISCII), a government standard, which amazingly is a common code for all Indian script systems [5]. The other code used for display purposes is a font code, known in the present case as the Indian Standard FOnt Code (ISFOC), which varies with script. Malayalam script is written from left to right, like all other Indian script (barring perso-arabic scripts) using isolated graphical structures. Every such horizontally isolated structure has a unique ID in the ISFOC. In the present final stage of character recognition, the stroke IDs from the previous stage are mapped onto ISFOC IDs. 4. DISCUSSION We have presented a system for online recognition of Malayalam characters. In our approach, a character is considered as a combination of multiple strokes. A single stroke is represented as a string of shape features. Individual strokes are identified by comparing the unknown stroke with a database of strokes. Combinations of strokes are then mapped onto font codes of Malayalam characters. Figure 5 shows a screen dump of the userinterface of our system which consists 2 windows: the display window at the top and the client window at the bottom. The writer writes on the lines in the client window, and the recognized characters are displayed in the display window. Currently our stroke database consists of 86 strokes with a total of 216 variations. Stroke data is generated from 3 writers. Performance on 4 data sets generated by one of the above 3 writers is shown in Table 1. Future efforts towards improvement of this system will include populating both the stroke database and test sets using samples from a wider selection of writers. Test Sl.No. Recognized Strokes Total No. of Strokes % Recogniti on Table 1: Performance of LEKHAK on 4 test sets. The stroke database has 86 strokes with a total of 216 variations (i.e., about 3 variations for each stroke).

5 5. REFERENCES Figure 4: A sample of handwritten Malayalam text displayed in the client area of LEKHAK Figure 5: LEKHAK recognizes handwritten text shown in Figure 5. Only 1 mistake was made in the 4 th character of the last word, denoted by a question mark (? ). 1. R. Plamondon and S.N. Srihari, Online and Off-line Handwriting Recognition: A comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, No. 1, January, M. Eden, Handwriting and Pattern Recognition, IRE Transactions on Information Theory, vol. 8, R. Plamondon, D. Lopresti, L.R.B. Shomaker and R. Srihari, On-line Handwriting Recognition, Encyclopedia of Electrical and Electronics Eng., J.G. Webster, ed., vol. 15, pp , New York: Wiley, V.S. Chakravarthy and B. Kompella, The Shape of Handwritten Character, National Conference on Document Analysis and Character Recognition NCDAR2001, Mandya, India. July 12-13, Indian Script Code for Information Interchange, IS , Bureau of Indian Standards, Manak Bhavan, 9 Bahadur Shah Zafar Marg, New Delhi.