Vehicle Registration Plate Recognition System Based on Edge Transition by Row and Column Profile on Still Image

Transcription

1

2 Vehicle Registration Plate Recognition System Based on Edge Transition by Row and Column Profile on Still Image Ankur Kr Aggarwal, Aman Kr Aggarwal Department of Computer Science Faculty of Management and Information Technology, Jamia Hamdard, New Delhi, India Abstract Vehicle Registration Plate Recognition System (VRPRS) has been intensively studied as it is an important part of the Intelligent Security and Traffic Management System. Registration plate recognition has been extensively applied in the Intellectual traffic system and public transit security system. In this paper, system is developed based on digital images and can be easily applied to commercial areas based on a smart and simple algorithm for vehicle s registration plate recognition system. The proposed algorithm consists of four major parts: Extraction of plate region, Segmentation of characters, cleaning of segmented information and Recognize characters on plate. Extraction of vehicle registration plate is performed by creating edge image based on row-column profile over the greyscale image. In Segmentation part, column profile is used based on character and background greyscale intensity variation. In cleaning phase, preprocessing of segmented character is done based on threshold value. And, finally row-column peaks and skeleton of character bitmap is used for recognition of plate characters. The performance of proposed algorithm has been tested on real images. Based on the experimental result of the proposed algorithm, we have shown the best performance in car registration plate recognition system for characters which are difficult to distinguish as they are very much alike, such as: B and 8, S and 5, V and Y, D and 0, G and C. 1. Introduction The Automatic Number Plate Recognition (ANPR) was invented in 1976 at the Police Scientific Development Branch in the UK. Vehicle Registration Plate Recognition System (VRPRS) is similar to ANPR. However, it gained much interest during the last decade along with the improvement of digital camera and increase in mathematical computational capacity of computers. As, different countries have different types of vehicle registration plate system, due to which Automatic Vehicle Identification (AVI) system based on number plate recognition system is different for each country. Vehicle Registration Plate Recognition System (VRPRS) has many applications in different security systems like lost vehicle identification, electronic toll collection, red light violation enforcement, custom and border checkpoints, etc. License plate recognition (LPR) is a form of automatic vehicle identification. It is an image processing technology used to identify vehicles by only their license plate. Since every vehicle carries a unique license plate, no external cards, tags or transmitters need to be recognizable, only license plate is enough. Real time LPR plays a major role in automatic monitoring of traffic rules and maintaining law enforcement on public roads [1]. The VRPRS system is performed in four major steps, first step is to extract vehicle registration plate which is performed by creating edge image based on row-column profile over the greyscale image. In second step, segmentation of characters is performed by column profile based on character and background greyscale intensity variation. In third step the cleaning phase, pre-processing (removing pixels such as the black bars above and below the characters, dirt spots, or nuts and bolts.) of segmented character is performed based on cross-over intensity values of rows and columns. And, finally rowcolumn peaks and skeleton of character bitmap is used for recognition of plate characters. 2. Related Work So many researches of car identification have been approached by car registration plate extracting and recognition, some of the related work is as follows: H.J. Kim, D.W. Kim, S.K. Kim, J.V. Lee, J.K. Lee [2] proposed a method of extracting plate region based on 1

3 color image segmentation by distributed genetic. Park et. al. [3] devised a method to extract Korean license plate depending on the color of the plate. Hontani et.al. [4] proposed a method for extracting characters without prior knowledge of their position and size in the image. Yoshimura and Etoh [5] used Gabor jets projection to form a feature vector for recognizing low resolution gray-scale character. Tavsanoglu and Saatci [6] proposed an approach to form orientation map as recognition feature using a Gabor filter for recognizing characters. S.K. Kim, D.W. Kim and H.J. Kim [7] used a genetic algorithm based segmentation to extract the plate region. E.R. Lee, P.K. Kim and H.J. Kim [8] used neural network for color extraction and a template matching to recognize characters. Kim [9] and Choi [10] proposed the method based on vertical edge using Hough transform (HT) for extracting the license plate. Venema and M.A.Westenberg [11] proposed fuzzy logic and neural networks for car LPR. This method used fuzzy logic for segmentation and discrete-time cellular neural networks (DTCNN S) for feature extraction. Fahmy [12] proposed bidirectional associative memories (BAM) neural network for number plate reading. It s appropriate for small numbers of patterns. Nijhuis, Ter Brugge, Helmholf J.P.W. Pluim, L. Spaanenburg, R.S. Johnson and Bird [13] proposed knowledge-guieded boundary following and template matching for automatic vehicle identification. Lotufo, Morgan and Johnson [14] proposed automatic number-plate recognition using optical character recognition techniques. The paper is organized as follows: Section III provides an overview of overall system. Extracting the plate region is explained in Section IV. Section V gives the segmentation of individual plate characters. Section VI deals with cleaning of segmented information. Section VII explains the recognition system for characters on registration plate. Section VIII discusses experimental results and the paper concludes with Section IX. 2.1 Difficulties while Extraction 3. In most cases, the detecting is performed without prior knowledge of the registration plate s location in the image. 4. Probability of facing some common drawbacks which could influence the efficiency of the extraction, such as, blurry image, uneven or low illumination, vehicle motion, low resolution of the image, distorted characters, dirty plate, shadows or reflection etc. In addition to the reasons above, the registration plate extraction becomes more difficult in real time systems, where the fast processing as well as accuracy is required. 3. Structure of VRPRS system The proposed system is designed for real-time registration plate extraction. Input to the system is an image which contains the registration plate, acquired from about 4-5 meters away by a digital camera of the front or rear of the vehicle; and its output is the extracted registration plate region. Figure 1 shows the structure of VPRS system. In this paper, the proposed algorithm is based on extraction of plate region, segmentation of plate characters and recognition of characters. Extraction of plate is a difficult task. Essentially, the difficulty can be due to the following reasons: 1. Registration plates normally, occupy a small portion of the whole image. 2. The difference of registration plates in formats, styles and colours from country to others. Figure 1. VRPRS overall system structure 2

4 4. Extraction of Plate Region In the extraction phase, we input the greyscale image (Figure 2 shows conversion of rgb to greyscale image) to the system and try to get the accurate location of the registration plate in output. The input image to the computer is just a matrix of values between 1 and 255 (representing grey scale). The features from image which have to be extracted (in this paper it is rectangle region as vehicle registration plate) was provided as input to algorithm. end end end end Figure 2. Car image conversion from rgb scale to greyscale for further feature extraction process The algorithm for the finding the accurate location of the registration plate is based on the following idea: a vehicle registration plate consists of alphabetic and numeric characters. Row pixels in the registration plate region are varied in intensity levels with characters as dark and background as light regions, respectively. So, the registration plate is categorized by rows of transitions from dark to light region. Greyscale image is converted to edge image for registration plate region extraction. The algorithm works over two consecutive pixels in row scan approach and check for high difference in grey level intensity. If difference is high we replaced the first pixel by an intensity of black pixel of the two consecutive pixels, otherwise by a white intensity pixel, the result of this process, as shown in Figure 3 (it is called as edge image for corresponding Figure 2). Algorithm: [x y] = size(image_gray); image = image_gray; for i = 1:x for j = 1:y if(j<y) diff = image(i,j)-image(i,j+1); if( diff>20 ) image(i,j)=1; else image(i,j)=255; Figure 3. Edge image generated based on high intensity difference In edge image we observed that there is high number of black pixels which defines the edges of vehicle registration plate. To extract the accurate registration plate region we decide on the region with high number of black pixels as edges of registration plate by using row profile over edge image. Row profile is the count of black pixels in each row of edge image and present this in graph. Graph shows peaks which represent position with high number of edges. The peak values of graph can be accustomed to determine all probabilities of horizontal position for the registration plate. Rows with high peak values are selected to crop edge image, as shown in Figure 4. 3

5 Figure 4. Row profile of Edge image (horizontal axis represents rows and vertical axis represents the number of black pixels in that row) and cropped edge image based on Row profile Cropped image is used by column profile for counting number of black pixel for the columns. Registration plate position in vertical direction is selected by taking likely peak values from column profile and truncated edge image is generated which have accurate location of registration plate, as shown in Figure 5. character data are of darker intensity as compare to background region. So, column profile of extracted image is used to segment character regions based on significant high intensity than their surroundings, as shown in Figure 7. Figure 5. Column profile of Edge image (horizontal axis represents rows and vertical axis represents the number of black pixels in that row) and Truncated edge image based on Column profile Figure 7. Column profile of cropped greyscale image (horizontal axis represents columns and vertical axis represents the number of pixels above threshold value) In column profile graph registration plate characters are cropped at the columns that are significantly higher in intensity than their surroundings. Threshold value is selected for this process based on that small image data set is obtained as character data, as shown in Figure Character Segmentation Figure 6 shows the extracted registration plate, location details from row and column profile graphs are used to truncate registration plate region from original greyscale image. Figure 6. Extracted registration plate image based on row and column pixels location from row-column profile. Segmentation of characters is directly performed on greyscale image based on the variation of intensity in characters and background region. Column with Figure 8. Character bitmaps as small image data set found as a result of the segmentation phase 6. Character Cleaning Before reorganization of characters we have to clean the character bitmaps by performing pre-processing. In cleaning phase we remove all dark pixels from the character bitmap that do not belong to the registration plate characters. In pre-processing we remove pixels such as the black bars above and below the characters, dirt spots, or nuts and bolts. Character bitmap image is converted into binary image. From binary character bitmap remove all rows and columns which have only black pixels, as they are black rectangle surrounding the registration plate. Next remove all small groups of pixels which are not connected to any other group, as these are no characters, either but dirt spots or bolts. Noise in 4

6 character bitmap is replaced by background pixels. Figure 9 shows the cleaned bitmap of the Figure 8. Figure 9. Character bitmaps as small image data set found as a result of the segmentation phase To extract junction and end-point feature, thinning operation is performed on character bitmap and find skeletonization of image. Junction feature is also known as branch point. It s a point where two points meet becomes Matlab command for junction/branch point extraction BWO = bwmorph(bwi, branchpoints ); 7. Recognize Characters Greyscale character bitmaps obtained in cleaning phase is input for recognizing character. Different methods are used for feature extraction for better generalization and low computation. Three character features: holes, junctions and end-points are considered, based on the number and position of these features we can classify characters. Decision table for character feature classification is given in Table 1. Table 1. Decision Table for Character Feature Classification ALPHABET HOLES END- JUNCTION POINT C L M N S U V W Z G J T Y F E K X H I O D P Q R A B End-point feature is a mark of termination or completion of line or curve of character. Every character has a number of end-points which plays a significant role to recognize a character becomes Matlab command used for end-point feature extraction BWO = bwmorph(bwi, endpoints ); But, these features are not adequate to recognize similar characters like 6 and 9, as they both have 1 hole, 1 junction and 1 end-point features. In this case, we can simply discriminate them just by looking at the position of the feature points. Divide the image into two equal parts upper half, and bottom half similarly left half and right half. If junction is at the right half and end-point at the bottom half, the character is recognized as 9 as shown in Figure 10. Hole feature is extracted by filling the holes of character bitmap image and taking logical difference with original image, shrink it and count the number of white pixels. Matlab command for hole feature extraction form BWI image BWO = imfill(bwi, holes ); Figure 10. Character bitmaps as small image data set found as a result of the segmentation phase With this classification method, the problem of recognizing characters that are very much alike is solved. 5

7 8. Experimental Result Experiments have been performed to test the proposal system and to measure the accuracy of the system. The system is designed in MATLAB R2009b for recognition of vehicle registration plates. The images for the input to the system are colored images with the size of 1200x1600. For improving the complexity and generality of the test database, the test images were taken under various illumination conditions (cloudy, sunny, and daytime). The result of the test cases is given in table 2. Table 2. Decision table for Character Feature Classification Units of VRPRS Number of Percentage of System Accuracy Accuracy Extraction of 107/ % Plate Region Segmentation 105/ % Recognition of 109/112 97% Characters It is shown that accuracy for the extraction of plate region is 95.5%, 94.6% for the segmentation of the characters and 97% is the percentage of accuracy of the recognition unit. 9. Conclusion In this paper, it proposes and designs the algorithm of the image processing and segmentation, based on the analysis of the variety of character features it is possible for algorithm to recognize similar characters based on features. Through experiments and improvements, the effect of the improvement is obvious and the character recognition rate increased substantially. Finally it shows accuracy of 95.5% for extraction of plate region, 94.6% accuracy for segmentation of characters for plate region and 97% accuracy for recognition of segmentation characters. This system can be extended for multiple car registration plate recognition system. 10. References [1] G. Bailey, D. Irecki, B.K. Lim and L. Yang Test bed for number plate recognition applications, Proceedings of First IEEE International Workshop on Electronic Design, Test and Applications ( DELTA 02 ), IEEE Computer Society, [2] H.J.Kim, D.W.Kim, S.K. Kim, J.V. Lee, J.K. Lee, 1997, Automatic Recognition of Car License Plates Using Color Image Processing, Engineering Design & Automation, 3(2), pp ,1997 [3] Park, S. H., Kim, K. I., Jung, K., and Kim, H. J., (1999), Locating car license plates using neural network, IEE Electronics Letters, vol.35, no. 17, pp [4] Hontani, H., and Koga, T., (2001), Character extraction method without prior knowledge on size and information, Proceedings of the IEEE International Vehicle Electronics Conference (IVEC 01), pp [5] Yoshimura H, Etoh M, Kondo K, Yokoya N. Gray-scale character recognition by Gabor jets projection, Proceedings 15th International Conference on Pattern Recognition, ICPR IEEE Comput. Soc. Part vol.2, 2000, pp vol.2, Los Alamitos, CA,USA. [6] Tavsanoglu V, Saatci E. Feature extraction for character recognition using Gabor-type filters implemented by cellular neural networks, Proceedings of the th IEEE International Workshop on Cellular Neural Networks and their Applications, IEEE. 2000, pp Piscataway,NJ,USA. [7] S.K. Kim, D.W. Kim, and H.J. Kim, 1996, A Recognition of Vehicle License Plate Using a Genetic Algorithm Based Segmentation, Proceedings of 3rd IEEE International Conference on Image Processing, V01.2., pp , 1996 [8] E.R. Lee, P.K. Kim, and H.J. Kim, 1994, Automatic Recognition of a Car License Plate Using Color Image Processing, Proceedings of the International Conference on Image Processing [9] H.S. Kim, et al., 1991, Recognition of a Car Number Plate by a Neural Network, Proceedings of the Korea Information Science Society Fall Conference, Vol. 18, pp , 1991 [10] H.J. Choi, 1987, A Study on the Extraction and Recognition of a Car Number Plate by Image Processing, Journal of the Korea Institute of Telematics and Electronics, Vo1.24, pp ,1987. [11] J.A.G. Nijhuis, M.H. Ter Brugge, K.A. Helmholt, J.P.W. Pluim, L. Spaanenburg, R.S. Venema, M.A. Westenberg, 1995, Car License Plate Recognition with Neural Networks and Fuzzy Logic, IEEE International Conference on Neural Networks,1995 [12] M.M.M. Fahmy, 1994, Automatic Number-plate Recognition : Neural Network Approach, Proceedings of VNIS 94 Vehicle Navigation and 6

8 Information System Conference, 3 1 Aug-2 Sept, 1994 [13] A.S. Johnson, B.M. Bird, 1990, NumberplateMatching for Automatic Vehicle Identification, IEE Colloquium on Electronic Image and Image Processing in Security and Forensic, Aprl, [14] R.A. Lotufo, A.D. Morgan, and AS. Johnson, 1990, Automatic Number-Plate Recognition, Proceedings of the IEE Colloquium on Image analysis for Transport Applications, V01.035, pp.6/1-6/6, February 16,

9 8