Neural Network Based Threshold Determination for Malaysia License Plate Character Recognition

Transcription

1 Neural Network Based Threshold Determination for Malaysia License Plate Character Recognition M.Fukumi 1, Y.Takeuchi 1, H.Fukumoto 2, Y.Mitsukura 2, and M.Khalid 3 1 University of Tokushima, 2-1, Minami-Josanjima, Tokushima, Japan fukumi@is.tokushima-u.ac.jp 2 Okayama University, 3-1-1, Tsushima-Naka, Okayama, Japan 3 University Teknologi Malaysia, City Campus, Kuala Lumpur, Malaysia Abstract In this paper, a method to recognize characters of vehicle license plate in Malaysia by using a neural network based threshold method is presented. Vehicle license plate recognition is one of important techniques that can be used for the identification of vehicles all over the world. There are many applications such as entrance admission, security, parking control, airport or harbor cargo control, road traffic control, speed control, toll gate automation and so on. For separation of characters and background, a threshold of digitalization is important and is determined using a three-layered neural network in this paper. Furthermore, in the extracted character portions, we segment characters and recognize them by obtaining their features. Computer simulations show that character segmentation and recognition can be effectively carried out using the present method. 1. Introduction Owing to the growth in the number of vehicles all over the world, even in Malaysia, it is needed to improve existing systems for identification of vehicles [1]. A fully automated system that can identify vehicles is in demand in order to reduce the dependency on labor. One of typical methods to identify vehicles is to identify vehicle's license plate characters, which uniquely distinguish each vehicle. There are a number of useful applications for vehicle license plate recognition system, for example electronic toll collection, security, parking control, speed detection, law enforcement and so on. There have been a number of commercialized systems that can be used for identification of license plates, which are mainly from USA, Europe, and Japan. However, they cannot be readily applied to Malaysian vehicles, because their license plate characters differ in styles and formats (See Fig.1). The license plates of vehicles in Malaysia are divided into several different colors and formats. The license plate for normal private vehicles has a black background and white characters as shown in Fig.1. In Fig.1, white square regions, which conceal a character, are put for privacy protection. Malaysia license plates usually have a single row or double rows structures. Furthermore, the characters have many kinds of fonts. Such facts make plate detection and character recognition difficult. The license plate for taxis has a white background and black characters. Diplomatic license plates have a red background and yellow or white characters. Another types of license plates, which are occasionally introduced, are special plates such as Proton, Satria, Sukom, Tiara and several others. The characters used in the license plate is 33 besides alphabet I, O, and Z, because I is similar to digit 1, O is similar to digit 0, and Z is similar to digit 2. In this paper, we present a vehicle license plate character recognition system and its simulation results. In character recognition, a threshold value for separating characters and background is in particular important. We determine a threshold value by using a three-layered neural network (NN) [2][3]. Furthermore, in extracted character portions, characters are segmented by character information, and then recognized by obtaining their directional features and by using NN. The remainder of the paper is organized as follows. The next section describes the neural network for determining a threshold value of binarization, and explains the technique of image processing for performing character recognition. This is followed by discussions on the experimental results and conclusion. In addition, inter-frame difference can perform the car extraction from a video easily. Since plate location can be mostly decided on the basis of the vehicle type after car location by the inter-frame difference, preliminary experiments using images with roughly detected plate regions are conducted.

2 Teacher data for NN training is set up manually. In other word, we determined the optimal threshold for each license plate by changing its value. The number of teacher data is 20 in this paper. This process can be carried out even by using genetic algorithms [8]. Acquisition of license plate (a) License plate Binarization by neural network Segmentation of character regions Character feature extraction (b) Roughly segmented plates Fig.1. Typical formats of Malaysian Vehicle license plates. White squares are for privacy protection. 2. The system flow Suppose that license plate location in vehicles is detected and its rough region is obtained in the system. This process can be carried out using the inter-frame difference for a moving image and plate position information in vehicles [3]. In roughly segmented license plates, binarized images are obtained from the license plate portion by determining a threshold value using the three-layered neural network as shown in Fig.2. Next, labeling and segmentation of characters are carried out in order to extract character domains. Thin line formation is then performed to the obtained characters and each character is divided into nine domains of 3x3. Each segmented domain yields the four amounts of the features, "vertical direction ( )", "horizontal direction ( - )", "direction of 45-degree slant ( / )", and "direction of 135-degree slant ( \ )". Each process is described below Neural network A neural network (NN) is an artificial network model, which emulates the cerebral nerve network in the brain. A typical NN is a feed forward neural network trained by the popular back-propagation method [4]. A sigmoid function is used as an output function of hidden neuron units in this paper. As shown in Fig.3, NN in this paper has 20 hidden units and one output unit. The histogram in grayscale values and the average value of grayscale image are used as input. The number of pixels between 0 and 9 in gray scale value is an input fed to the first input unit of NN. Input to the second unit is the number of pixels corresponding to grayscale values between 10-19, and the 26th input is the number of pixels corresponding to grayscale values between The 27th input is the average value of the whole grayscale image. The output unit has a sigmoid function and yields a threshold value for binarization. [0-9] [10-19] [ ] Average value Input (27) (1) Character recognition Fig. 2. The system flow layer Threshold value Middle layer (20) Output layer Fig.3. A neural network estimating a threshold value for binarization Binarization A license plate image is binarized for extracting its character domains. The character in a license plate and brightness usually varies with various situations. It is therefore difficult to carry out binarization with a given threshold value. In this paper, the method of determining a threshold value for each image by using NN is proposed. NN that carries out it in this paper is a three-layered NN without feedback connections. Input layer units are given the number of pixels corresponding to each grayscale value range and the average value of a whole image as shown in Fig.3. This NN can give a good threshold based on distribution of gray scale values. In this paper, only a single threshold value for each license plate is yielded to segment characters from background. Although the method of extracting a character is shown by reference [1], changing a threshold value, judgment of the threshold value used is a comparatively difficult task.

3 2.3. Character domain extraction Labeling is performed in order to extract a character domain from an obtained binarized image. Labeling is the operation which assigns the same label to all the pixels belonging to the same connection ingredient, and assigns a different label to a different connection ingredient. This operation is illustrated in Fig.4 [5]-[7]. It is clear that a different domain (a small region) should be deleted. Furthermore, small regions connected to the outside square are removed. The size of characters is almost the same for all and the number of characters is 7 in the maximum case. Therefore the connected characters can be separated using their size information. On the one hand, characters on a license plate form a single row or double rows. Then incorrectly segmented character can be restored using this information in some cases. direction of 135-degree slant ( \ ) are calculated. STEP 3: The feature of 36(9x4)-dimensional vector is obtained. These operations are carried out to all the character domains that exist in each license plate. Fig. 6 shows the character E after thin line formation. The feature of each character is calculated beforehand. The feature obtained from a new license plate is evaluated compared with the features calculated beforehand, and the character with the shortest distance is regarded as the class of each character in license plate. 3. Computer simulations In this section, the effectiveness of the present method is demonstrated for Malaysia license plate character recognition Threshold value determination In this paper, NN tries to determine a threshold value automatically from input images shown in Fig.7. Fig.4. Labeling process for segmenting characters Thin line formation Thin line formation is the operation of narrowing a line width from a given figure and extracting the central line of width 1. This does not change the connectivity of an original-drawing form. Although there are various methods in thin line formation algorithm, the method by Hilditch is adopted in this paper. This processing method is a model to carry out the thin line formation one by one, and uses 8-connectivity as a form of the diagram. The result is shown in Fig.5. Fig.6. Thin-lined image for E In this experiment, the grayscale images of 228 sheets of the license plate domain were used. These are roughly extracted from vehicle image sequence. A threshold value which performs the binarization manually and is the best for the license plate image is used as a teacher data. Twenty images obtained by the above-mentioned procedure are prepared, and they are used as the teacher images. Fig.5. Thin line formation Directional line-element feature Directional line-element features are obtained from the character transformed into the thin line form. The following operations are carried out in order to obtain the directional line-element features. STEP 1: A character image is divided into the nine partial domains of 3x3. STEP 2: The quantity of directional elements, vertical direction ( ), horizontal direction ( - ), direction of 45-degree slant ( / ), and Fig.7. Input image examples Neural network architecture The number of units in NN is shown in Table 1. As described in Section 2, the number of input layer units is 27 and the number of hidden units is set as 20. The number of output layer units to yield a threshold value was 1. Learning step width was 0.3, and the number of

4 learning cycles was 20,000 times, or if an error value was less than 1.0x10-6, then learning stopped. Table 1. The number of units in NN. Input layer 27 Hidden layer 20 Output layer Experimental result In this experiment, the rate of images in which the character portion and the background were separated accurately was 93.4% [212/227 sheets]. Resulting images are shown in Figs.8 and 9. In these figures, pixels with higher value than the threshold show white. The images whose separation was not completed well were images with low brightness on the whole, with complicated plate images, etc as shown in Fig.9. Such an image was not included in teacher data. In order to improve binarization accuracy, variance of gray scale values is maybe important. More samples are necessary to improve the binarization accuracy Character recognition experiment In this section, character recognition for the images binarized using the system was carried out. The images used for the experiment were binarized ones obtained by NN, and 212 sheets were used. In particular, we carried out it by using two recognition systems, which are the minimum distance classifier (Method A) and a three-layered NN (Method B). In the method A, thin line formation was performed for the images which succeeded in labeling and in quest of directional line-element features, and then character recognition was performed for the images which succeeded in the thin line formation. In the method B, segmented characters after the labeling are recognized only by NN. This NN is different from the threshold determination NN as shown in Fig.3. The labeling result was only 93.9% [199/212] accuracy for the images transformed into binarized images. 199 sheets include 1,242 characters. We show the results obtained using this system in Table2. Fig.10 shows the thin-lined character for 3. The directional line-element features obtained for the character '3' are shown in Table3. Moreover, the labeling was performed regarding as one character domain when the background and the character were joined together in some case. However this situation is improved by using size information of characters Thin line formation is 98% accuracy. The character recognition using the directional line-element features is 83.57% accuracy. As a cause of having yielded incorrect recognition, it is considered that generated mustache portions in the thin line formation had a bad effect for recognition accuracy in the method A. In the method B, NN was trained by using numerals segmented from license plates. The size of input images is 30x20 pixels. Learning samples are shown in Fig.11. There are many fonts. In NN learning, alphabet characters I, O, and Z are not included. The number of characters is therefore 33 and one sample for each character is used for NN training. Because the character R is very few in database. The number of input, hidden, and output units in this character recognition NN is 600, 50, and33, respectively. The recognition accuracy is 89.21%, as shown in Table2. This accuracy is not very good, because the number of training samples is not many. The increase of training samples would improve recognition accuracy. Fig.8. An example of accurately binarized image Fig.9. An example of failure in binarization. Table 2. Simulation results for character recognition. Method A Method B Correct characters 1038/ /1242 Accuracy 83.57% 89.21% (7) (8) (9) (4) (5) (6) (1) (2) (3) Fig.10. Image division for 3

5 Table 3. Directional line-element features for 3 ( ) (-) (/) (\) Region Region Region Region Region Region Region Region Region Fig.11. Image examples for digit 0. There are many fonts. method using neural networks", Proc. of KES'2002, Crema, Italy, pp (2002). [3] S. Yoshimori, Y. Mitsukura, M. Fukumi, & N. Akamatsu, "License Plate Detection System in Rainy Days", Proc. of CIRA'2003, Kobe, Japan, pp (2003). [4] J. Hearts, A. Krogh, Richard, and G. Palmer, Introduction to The Theory of Neural Computation, Addison-Wesley Publishing Company, The Advanced Book Program, 350 Bridge Park0way, Redwood City, (1991). [5] T.Agui, and T.Nagao, Processing and Recognition of Image, Shoumeidou, (1992). in Japanese. [6] H.Tamura, A guide to computer image processing, Shoukensyuppan, pp.3-162, (1999). in Japanese. [7] M.Inoue, N.Yagi, M.Hayashi, H.Nakasu, K.Mitami, and M.Okui, Practice image processing studied by the C language,ohmsha, pp.4-99, (1999). in Japanese. [8] Y.Mitsukura, M. Fukumi, and N. Akamatsu, "A Detection Method of Face Regions in Color Images by Using Evolutionary Computation", Proc. of International Joint Conference on Neural Networks'2001, pp , Washington, D.C. (2001). [9] M.Fukumi and Y.Mitsukura, "A Simple Feature Generation Method Based on Fisher Linear Discriminant Analysis", Proc. of IASTED International Conference on Signal and Image Processing'2005, pp , Hawaii (2005). [10] M. Fukumi and Y.Mitsukura, "Feature Generation by Simple FLD", Proc. of 9th International Conference on Knowledge-Based Intelligent Information & Engineering Systems,Part I, pp , Sept. 2005, Melbourne, Australia, (2005) 4. Conclusion In this paper, in order to extract character portions and recognize the characters in license plates, the technique of determining a threshold value using a neural network was proposed. By the present method, it enables us to determine a threshold value corresponding to change of character domain and brightness. Moreover, in order to obtain a character domain, labeling and thin line formation were performed and then directional line-element features were obtained. When character domain could be obtained, we can perform character recognition in high accuracy by a neural network. For better segmentation of license plate characters, we have to evaluate a method by two thresholds (upper and lower bounds) determined by neural networks or genetic algorithms [3]. Furthermore, we will evaluate a statistical feature generation method, Fisher linear discriminant analysis (FLDA). The FLDA is basically better than the principal component analysis in feature generation. We believe that the simple-flda [9] can improve recognition accuracy even in license plate character recognition. 5. References [1] Y.K.Siah, T.Y.Haur, M.Khalid, and T.Ahmad Vehicle License Plate Recognition by Fuzzy ARTMAP Neural Network, Proc.of WEC 99 pp19-22, (1999). [2] Y. Fukuta, Y. Mitsukura, M. Fukumi & N. Akamatsu: "High-speed face search by threshold value determination