RECOGNITION OF HANDWRITTEN DEVANAGARI WORDS USING NEURAL NETWORK

Transcription

1 RECOGNITION OF HANDWRITTEN DEVANAGARI WORDS USING NEURAL NETWORK ABSTRACT: Sonali G. Oval and R. V. Dagade Department of Computer Engineering MMCOE Savitraibai Phule Pune University, India Word Recognition is an important problem of Pattern Recognition. Online recognition system for words is still in developing stage and becoming challenging due to the large complexity involvement. In India, more than 300 million people use script for documentation. There has been a significant improvement in the research related to the recognition of printed as well as text in the past few years. Though is the script for Hindi, which is the official language of India, its character and word recognition pose great challenges due to large variety of symbols and their proximity in appearance. We have developed an offline recognition system to recognize the Legal Amount words using neural network. First we have applied preprocessing techniques on scanned image, such as grayscale, binarization, and thinning techniques. For performing thinning, we used Stentiford algorithm. Then we have performed segmentation on thinned image. Finally we used recurrent neural network classifier as a recognition method. We have used database containing 26,720 legal amount words written in Hindi and Marathi languages i.e. ( Script) by different writers. Out of these, we have trained 500 words as a one dataset for our proposed system. Result graph was plot, these training dataset against recognized word by the system. Keywords: Words, offline handwriting recognition, Pre processing, Recurrent Neural Network, Segmentation. [1] INTRODUCTION Handwriting recognition can be divided into two categories: Offline and Online shown in [Figure 1] [1]. This division is done on the basis of representing data to the system. For the first kind of recognition system user's handwriting is digitized by a scanner or a camera at a later time and the data is presented to the system as an image while for the second kind, handwriting is digitized by a tablet and stylus at the time of writing and strokes are captured as they are being formed by sampling the pen's position at evenly spaced time intervals. A pressure sensitive switch on the tip of the pen detects pen-up/pen-down position and discriminates stroke segmentation. HMMs are able to segment and recognize at the same time, which is one reason for their popularity in handwriting recognition [1]. The idea of applying HMMs to handwriting recognition was originally motivated by their success in speech recognition, where a similar conflict exists between recognition and segmentation. Over the years, numerous refinements of the basic HMM approach has been proposed, such as the writer-independent system. HMMs have several well-known Sonali G Oval and R V Dagade 89

2 RECOGNITION OF HANDWRITTEN DEVANAGARI WORDS USING NEURAL NETWORK drawbacks. One of these is that they assume that the probability of each observation depends only on the current state, which makes contextual effects difficult to model. Another is that HMMs are generative, while discriminative models generally give better performance in labeling and classification tasks. Recurrent neural networks (RNNs) do not suffer from these limitations and would therefore seem a promising alternative to HMMs. Figure: 1. Types of Recognition. [1.1] Literature Survey Sr. Author s No. Name 1 S. Arora, D. Bhattacharjee, M. Nasipuri, D.K. Basu2, M.Kundu, L.Malik 2 G. G. Rajput, S. M. Mali 3 Pooja Agrawal, M. Hanmandlu, Brejesh Lall 4 U. Pal, N. Sharma, T. Wakabayashi and F. Kimura Title of Paper Data Set Used Methods Results (%) Study of Different Features on Character Fourier Descriptor based Isolated Marathi Numeral Recognition Coarse Classification of Hindi Characters Off-Line Character Recognition of Script 1500 basic characters samples of Marathi numerals basic characters MLP 98.61% NN KNN SVM Coarse classificatio n 97.05% 97.04% 97.85% 97.25% MQDF 94.24% 5 Umapada Pal, Sukalpa Chanda Tetsushi Accuracy Improvement of samples of MQDF SVM 94.24% 94.15% 90

3 Wakabayashi, Fumitaka Kimura Character Recognition Combining SVM and MQDF 6 Satish Kumar Performance Comparison of Features on Handprinted Dataset 7 Sandhya Arora, Debotosh Bhattacharjee, Mita Nasipuri, Dipak Kumar Basu, Mahantapas Kundu 8 Sandhya Arora, Debotosh Bhatcharjee, Mita Nasipuri, Latesh Malik 9 Alex Graves, Marcus Liwicki, Santiago Fernandez, Roman Bertolami, Horst Bunke, and Jurgen Schmidhuber 10 Bikash Shaw, Swapan Kumar Parui, Malayappan Shridhar 11 Marcus Liwicki, Horst Bunke, James A. Pittman, Stefan Knerr 12 Bikash Shaw, Swapan Kumar Combining Multiple Feature Extraction Techniques for Character Recognition. A Two Stage Classification Approach for Characters A Novel Connectionist System for Unconstrained Handwriting Recognition A Segmentation Based Approach to Offline Word Recognition Combining diverse systems for text line recognition Offline basic characters characters basic characters samples of characters. IAM-DB consists of lines, containing 86,272 instances of 11,050 distinct words and images of words of 100 word classes. IAM-OnDB-t2 Benchmark. 39,700 samples of MLP SVM 91.9% 94.1% MLP 92.80% NN 89.12% HMM Word accuracy RNN Word accuracy 64.5% 74.1% 81.8% RNN Character accuracy HMM 81.63% BLSTM HMM 81.2% 79.2% HMM 80.2% Sonali G Oval and R V Dagade 91

4 RECOGNITION OF HANDWRITTEN DEVANAGARI WORDS USING NEURAL NETWORK Parui, Malayappan Shridhar 13 Marcus Liwicki, Alex Graves, Horst Bunke Jurgen, Schmidhuber Word Recognition: A holistic approach based on directional chain code feature and HMM A Novel Approach to On- Line Handwriting Recognition Based on Bidirectional Long Short-Term Memory Networks town names IAM-OnDB-t2 Benchmark RNN 74.0% From the literature survey we came to know that neural network gives better results than other techniques. Also there are different techniques for feature extraction. Lots of work has been done on numerals, characters and very few were done on words. Recognition rate of numerals is very good, but for words it is less. [2]DEVANAGARI SCRIPT script came into existence at around AD This script emerged out of Siddham script an immediate descendant of Gupta script ultimately deriving from the Brahmi Script. It follows left to right fashion for writing [8]. This script is cursive in nature. has 13 independent vowels or svara, 33 independent consonants or vyajana and 12 dependent vowel signs shown in [Figure.2]. Figure: 2 Isolated Character Set. 92

5 Most of the consonants can be joined to one or two other consonants so that the inherent vowel is suppressed. The resulting conjunct form is called a ligature or a compound character. Commonly used compound characters appearing in our lexicon of words are shown in [Figure.3.] another distinctive feature of is the presence of a horizontal line on the top of all characters. This line is known as header line or shirorekha [see Figure.3.]. The words can typically be divided into three strips: top, core, and bottom, as shown in [Figure.3] the header line separates the top and core strips and a virtual base line separates the core and lower strips [4]. Figure: 3 Three strips of a word in script. [3] SYSTEM ARCHITECTURE The proposed system can be implemented an architecture where pre-processing, segmentation and classification are as shown in [Figure 4]. Preprocessing first perform gray scale on scanned input image. It generates grayscale image as a result. The next step is to perform binarization on grayscale image. Thresholding algorithm is used Binarize the gray scale image and generate binarized image as an output. On binarized output perform thinning. The result of thinning gives as an input to shirorekha detection and removal operation. For detecting and removing shirorekha of word from the image we have used scan line algorithm. The removed shirorekha image gives as an input to segmentation. Segmentation divides word into characters. Extract the features from segmented word and generate the template. The generated template gives as an input to recognition operation. In the recognition operation it compare generated template to the training dataset and produce the closest match as a result. Figure: 4 System Architecture Diagram [4] MATHEMATICAL MODELLING System S can be defined as: Sonali G Oval and R V Dagade 93

6 RECOGNITION OF HANDWRITTEN DEVANAGARI WORDS USING NEURAL NETWORK S = {Input, Output} The mathematical relation between input I and output O can be stated as: Where, m= W. Pi >= pt (default T (Threshold)) W- Number of characters present in the word. [4.1] ALGORITHM The handwriting recognition process includes the following steps: 1. Scan the word image and convert that image into grayscale. 2. Binarize the gray scale image by applying Thresholding algorithm to obtain binary image with word representing binary 1 and background Perform thinning (Stentiford algorithm) on binarized image. 4. Detect the shirorekha from the image and Remove that shirorekha. 5. Perform segmentation on that image. 6. Create the training dataset for recognition. 7. Recognize the input image using RNN classifier. [4.2] PREPROCESSING ALGORITHM Grayscale A grayscale or grayscale digital image is an image in which the value of each pixel is a single sample, that is, it carries only intensity information. Images of this sort, also known as black-and-white, are composed exclusively of shades of gray, varying from black at the weakest intensity to white at the strongest. Algorithm: Gray scale Conversion Input: Scanned Image Output: Gray scaled image 1. Get the red, green, and blue values of a pixel. 2. Use fancy math to turn those numbers into a single gray value. Gray = (Red + Green + Blue) / 3 3. Replace the original red, green, and blue values with the new gray value. Binarization The next step is to binarize the grayscale image. In Binarization, color or gray-scale image is converted into binary image with the help of Thresholding. Algorithm: Thresholding algorithm Input: gray scale image Output: binarized image 1. Initialize fgcount = 0,bgCount = 0 2. Scan the image vertically and horizontally i.e. 3. for(int y = 0; y < h; y + +) 94

7 4. for(int x = 0; x < w; x + +) 5. Get the value of pixel i.e. col = p2[y][x] 6. Get the value of blue value of pixel i.e b = col& 0xff 7. Get the value of green value of pixel i.e. g = (col >> 8)& 0xff 8. Get the value of red value of pixel i.e r = (col >> 16)& 0xff 9. Calculate the average of that pixel i.e. gs = (r + g + b)\3 10. if (gs < 128) then 11. Increment the background value count. i.e. bgcount Change the value of that pixel i.e. p2[y][x] = end if 14. else 15. Increment the foreground value count. i.e. fgcount Change the value of that pixel. i.e. p2[y][x] = end for 18. end for Thinning The next step is thin the binarized image. For performing thinning we have used Stentiford algorithm. Algorithm: Stentiford Thinning Algorithm Input: Binarized image Output: Thinned image 1. Find a pixel location (i, j) where the pixels in the image match those in template T1. With this template all pixels along the top of the image are removed moving from left to right and from top to bottom. 2. If the central pixel is not an endpoint, and has connectivity number = 1, then mark this pixel for deletion. Endpoint pixel: A pixel is considered an endpoint if it is connected to just one other pixel. That is, if a black pixel has only one black neighbor out of the eight possible neighbors. 3. Repeat steps 1 and 2 for all pixel locations matching T1. 4. Repeat steps 1-3 for the rest of the templates: T2, T3, and T4. T2 will match pixels on the left side of the object, moving from bottom to top and from left to right. T3 will select pixels along the bottom of the image and move from right to left and from bottom to top. T4 locates pixels on the right side of the object, moving from top to bottom and right to left. 5. Set to white the pixels marked for deletion. Shirorekha Detection & Removal Algorithm: Algorithm for Shirorekha Detection & Removal Input: Thinned image Output: Shirorekha removed image Sonali G Oval and R V Dagade 95

8 RECOGNITION OF HANDWRITTEN DEVANAGARI WORDS USING NEURAL NETWORK 1. Initialize ysum=0, ytotal=0, yavg 2. Scan the image both vertically and horizontally i.e. 3. for(x=0; x<=w; x++) 4. for(y=0; y<=h; y++) 5. Check each value stored in matrix p2. i.e. 6. If(p2[y][x]!=0) then 7. ysum+=y, ytotal++ 8. break 9. calculate yavg=ysum/ytotal 10. Remove the first pixel belonging to average i.e. 11. If(y<=yavg) 12. Change the value of that pixel i.e. p2[y][x]=0 13. Elseif (y-yavg < 2) then 14. Change the value of that pixel i.e. p2[y][x]=0 15. Repaint the image Segmentation Algorithm: Algorithm for Segmentation Input: Shirorekha removed image Output: Segmented word image 1. Initialize found, foundfirst. 2. Scan image vertically i.e. 3. for(y=0;y<h;y++) 4. Select the first white pixel value, which appear vertically. 5. Set found = false and Scan image horizontally 6. Set p2[y][x]==1 single white pixel is found 7. found =true 8. Store the pixel value which is found in array. 9. Again scan the image horizontally. 10. Repeat step 1 to 5 till p2[y][x]==0 11. Cluster white pixels found in pervious step. 12. Check for the white space. 13. If white space is occurred draw a rectangle outside the character. 14. Extract the features of character and store in the template. 15. Repeat step 1-14 till not found last white pixel in image [4.3]CLASSIFICATION ALGORITHM A recurrent neural network (RNNs) is a connectionist model containing a selfconnected hidden layer [1]. RNN's provide a very elegant way of dealing with (time) sequential data that embodies correlations between data points that are close in the sequence. [Figure 5] shows a basic RNN architecture with a delay line and unfolded in time for two 96

9 time steps. In this structure, the input vectors are fed one at a time into the RNN [1]. One of the key benefits of RNNs is their ability to make use of previous context [2]. Figure: 5 Structure of RNN Algorithm: Algorithm for Recognition Input: Segmented image Output: recognize word image 1. Initialize the variable count=0. 2. Select the rectangles which are present in the input segmented image. 3. If rectangle is found then 4. Compare the generated template p[y][x] to template from training set t[y][x]. 5. If pixel p[y][x]==t[y][x] i.e. pixel value is match 6. then count++; 7. Else 8. count--; 9. Store count value in result array r[ ]. 10. get.next.t[y][x] 11. Repeat step 1 to 10 to match all templates of training data set. Till get.last.t[y][x] 12. Compare the values result array to select maximum count. i.e. 13. If (r[i] >= r[i-1]) 14. Then select template having count value r[i], i++; 15. Else 16. Repeat step 13till r[i]==r[i-1]. 17. Select these templates, which count is high and show recognized output. Sonali G Oval and R V Dagade 97

10 RECOGNITION OF HANDWRITTEN DEVANAGARI WORDS USING NEURAL NETWORK [5] RESULT AND DATASETS Figure: 6 Results [Figure 6] Show the output of the system. Figure: 7 Graph of result From this graph we come to know that without modifiers i.e. kana, Marta we get better result than with modifiers i.e. kana, Marta s. Datasets The database is used a legal amount words. A database contained 26,720 legal amount words written in Hindi and Marathi languages. The database was constructed by taking data from ninety writers. [6] CONCLUSION As Script Recognition has a huge scope in many areas, the researchers should use the most efficient techniques to get the desired results. For recognizing the word is difficult task. In this system we have used grayscale method, Thresholding algorithm & Stentiford thinning algorithm as a pre- processing techniques. Scan line an algorithm is used for detecting the shirorekha & then remove that detected shirorekha. For Segmentation of words into characters we have used the scan line algorithm. After segmentation of words into characters we have extracted the features of those characters. The training dataset was generated using the features of the characters which we extracted. Finally we build the classifier to recognize the input word. The future scope of this project is to use the complex words from the database for recognition. Apply 98

11 more preprocessing techniques, feature extraction techniques to improve the result. Also recognize the multiple lines. REFERENCES [1] Graves, M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke, and J. Schmidhuber: A novel connectionist system for unconstrained handwriting recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 31, , May [2] Graves, S. Fernandez, M. Liwicki, H. Bunke, and J. Schmidhuber: Unconstrained Online Handwriting Recognition with Recurrent Neural Networks, Advances in Neural Information Processing Systems 20,1-7, [3] Shaw, S. K. Parui, and M. Shridhar: A segmentation based approach to offline word recognition, in Proc. IEEE Int. Conf. Inf. Technol., pp , [4] Shaw, S. K. Parui, and M. Shridhar: Offline word recognition: A holistic approach based on directional chain code feature and HMM, in Proc. Int. Conf. Inf. Technol., pp , [5] G. G. Rajput and S. M. Mali: Fourier descriptor based isolated Marathi numeral recognition, Int. J. Comput. Appl., vol. 3, no. 4, pp. 9-13, [6] Marcus Liwicki, Alex Graves, Horst Bunke, Jurgen Schmidhuber: A Novel Approach to On- Line Handwriting Recognition Based on Bidirectional Long Short-Term Memory Networks, , ICDAR [7] M. Hanmandlu, P. Agrawal, and B. Lall: Segmentation of Hindi text: A structural approach, Int. J. Comput. Process. Lang., vol. 22, no. 1, pp. 1-20, [8] Olivier Morillot, Laurence Likforman-Sulem, Emmanu'ele Grosicki: Comparative study of HMM and BLSTM segmentation-free approaches for the recognition of text-lines, IEEE [9] P. Agrawal, M. Hanmandlu, and B. Lall: Coarse classification of Hindi characters, Int. J. Advanced Sci. Technol., vol. 10, pp , [10] S. Arora, D. Bhatcharjee, M. Nasipuri, and L. Malik: A two stage classification approach for characters, in Proc. Int. Conf. Comput. Intell. Multimedia Appl., pp , [11] S. Kaur: Recognition of script using features based on Zernike moments, zoning and neural network classifier, M.Tech Thesis, Dept. Comput. Sci. Eng., Punjabi University, Patiala, India, [12] S. Kumar: Performance comparison of features on hand printed dataset, Int. J. Recent Trends, vol. 1, no. 2, pp , [13] U. Pal, N. Sharma, T. Wakabayashi, and F. Kimura: Offline character recognition of script, in Proc. 9th Conf. Document Anal. Recognit., pp , [14] U. Pal, S. Chanda, T. Wakabayashi, and F. Kimura: Accuracy improvement of character recognition combining SVM and MQDF, in Proc. 11th Int. Conf. Frontiers Handwrit. Recognit., pp , Sonali G Oval and R V Dagade 99