Robust Automatic Traffic Signs Recognition Using Fast Polygonal Approximation of Digital Curves and Neural Network

Transcription

1 Robust Automatic Traffic Signs Recogniti Using Fast Polygal Approximati of Digital Curves and Neural Network Abderrahim SALHI, Brahim MINAOUI 2, Mohamed FAKIR 3 Informati Processing and Decisi Laboratory, Sultan Moulay Slimane University, Abstract Traffic Sign Detecti and Recogniti (TSDR) has many features help the driver in improving the safety and comfort, today it is widely used in the automotive manufacturing sector, a robust detecti and system a good soluti for driver assistance systems, it can warn the driver and ctrol or prohibit certain actis which significantly increase driving safety and comfort. This paper presents a study to design, implement and test a method of detecti and of road signs based computer visi. The approach adopted in this work csists of two main modules: a detecti module which is based color segmentati and edge detecti to identify areas of the scene may ctain road signs and a module based the multilayer perceptrs whose role is to match the patterns detected with road signs correspding visual informati. The development of these two modules is performed using the C/C++ language and the OpenCV library. The tests are performed a set of real images of traffic to show the performance of the system being developed. Keywords Traffic sign; ; detecti; pattern matching; image processing; Polygal Approximati of digital curves I. INTRODUCTION A driver assistance system is designed to help the driver to better ctrol the vehicle in difficult circumstances (tired, stress, carelessness, poor visi...) to help increase safety and the safety of other drivers, pedestrians etc. involved in the traffic the roads. Computer visi is a promising approach for addressing this problem. Automatic ctrol of the braking system, automatic speed ctrol, generati of alerts and notificatis, correspding to the various events encountered the road etc., are some examples amg many installatis of driver assistance that could be developed, based this approach. The aim of this work is to design, implement and test a system of Traffic signs Recogniti. Traffic signs Recogniti system developed in this work like [-3] composed of two main s. The first is the detecti which is designed to detect and extract zes may ctain traffic signs in the image, based the particular color and geometry of these panels; and using image processing techniques such as color segmentati, threshold technique, Gaussian filter, canny edge detecti, it allows us to greatly reduce the amount of informati to be processed. This allows faster processing and facilitating the realizati of a system operating in real time. Fig.. Examples of the four types of panels csidered in this work The secd is the ; it helps identify road signs by comparing informati provided by the detecti with models of road signs that have been learned beforehand, based multilayer perceptrs. The algorithms are designed to detect and recognize red triangular and circular panels, and blue circular and rectangular panels. Making this choice of traffic signs to recognize, it covers a vast majority of comm traffic signs and also the most important signals that advertise a danger the road, a ban (speed limit) or require a driver to certain behavior (blue signals) figure (FIG ). Shows some examples of signs used in this work. II. SYSTEM DESCRIPTION The first secti in our system (FIG 2) is the detecti module, in this part we read the RGB image from real images sequence and cvert color to HSV space, to minimize the effects of changing brightness, followed by a color segmentati using threshold [], two masks are generated, the first e, is designed to find zes which may be a red circles or a red triangles shapes, using shape [8], the secd mask, is to find zes which may be a bleu circles or a bleu rectangles shapes by the same way. P a g e

2 Fig. 2. Block diagram of the system The secd secti is the Recogniti module, in this part we extract the s of zes (blobs) provided by the detecti module, to be matched with templates in database [] using Multilayer Perceptr (MLP) algorithm, and take the appropriate decisi. The image is read in RGB (Red Green Blue) color mode (FIG 3: a), and then cverted to HSV (Hue Saturati Value) color space, then we use the thresholding technique to generate two red mask and blue (FIG 3: a & FIG 3: b), these masks will be used later to find the ctours, and before that, we apply the technique of smoothing by using a Gaussian filter, and the technique of Canny edge detecti to improve the image and easily get the desired regi. The obtained binary image for each mask is processed to retrieve ctours by findctours functi for binary image and the retrieved ctours are returned and stocked in chain format III. TRAFIC SIGNS DETECTION MODULE Chain code: Fig. 4. Coding of Cnected Compents (Courtesy []) represent ctours in the OpenCV library, we use the Freeman method or the chain code. For any pixel all its neighbors with numbers from to 7 can be enumerated as Fig. 4. The -neighbor denotes the pixel the right side, etc. As a sequence of 8-cnected points, the border can be stored as the coordinates of the initial point, followed by codes (from to 7) that specify the locati of the next point relative to the current e. (Fig. 4 illustrates example of Freeman coding of cnected compents.) Fig. 3. a) BGR color image; b) thresholded Blue mask, c) thresholded Red mask. Fig.. Simplifying a piecewise linear curve with the Ramer-Douglas Peucker algorithm []. 2 P a g e

3 The extracted ctours are used to find the edges that can match the shapes of traffic signs, like triangular shapes, circular shapes and rectangular shapes. In the ctours returned from the red mask, we will look for the triangular and circular shapes, and in those returned from blue mask, we will search for rectangular and circular shapes. find these shapes will be based the algorithm of Ramer -Douglas Peucker []. The idea is to simplify a polyline (n nodes) and replace it with a single line (two-node) if the distance of the farthest from the line formed by the ends of the polyline node is below a threshold, as shown in (FIG ) The algorithm works recursively by the method of "divide and rule", to initialize the algorithm we select the first and last node (for a polyline), or any node (such as a polyg). These are the terminals. At each step, through all the nodes between the terminals and the farthest segment formed by the terminal node is selected. If there is no node between terminals algorithm ends. If this distance is less than a certain threshold are removed all the nodes between terminals. If it exceeds the polyline is cannot be directly simplified. Called recursively the algorithm two subparts of the polyline: the first terminal to the remote node and the remote terminal to the final node. The approxpolydp functi provided by the OpenCV library [] is used to implement this algorithm, and takes as input the ctours found by findctours functi, and the threshold, it return a table of points forming the new polyg approached to the real ctour. Our algorithm tests the size of the returned table of points, and if it ctains tree elements it means that the founded shape is a triangle, if it ctains four elements, it means that the shape founded correspding to a rectangle. For the circular shapes we empirically take the size of the table from beyd six elements. close that founded shapes to those of traffic signs, we impose certain criteria, such as: triangles should be equilateral with an error near; their surfaces must be included in a welldefined interval, and the something for rectangles. For circular shapes, we use the ellipse detecti technique [] provided by OpenCV library [] and then the sizes of its two axes are taken as criteria. The shapes that satisfy the criteria imposed will be cropped and extracted from the color natural image; these blobs will be resized to 32x32 to be used in the phase. This allows us to significantly reduce the number of shapes processed and computing time cost of the algorithm later; Figure (FIG 6) shows the flowchart of the detecti module. threshold using binary threshold for blue color & change image to binary Image Blue Ellipse Ellipse Rectangle Triangle Ellipse N N N N Criteria Criteria Criteria Criteria ctours in the binary image for Blue ellipses Start Read image in color Mode Cvert color to HSV space Blue Rectangl e threshold using binary threshold for Red color & change image to binary Image ctours in the binary image Y Y Y Y for Blue rectangle Red Triangle for Red triangles Red Ellipse for Red rectangle Fig. 6. flowchart of the detecti module 3 P a g e

4 (IJACSA) Internatial Journal of Advanced Computer Science and Applicatis, 2 Blue circular shapes 2 Blue rectangular shapes Red shapes are framed by a red rectangle & Blue shapes are framed by a blue rectangle in reel image. Cropped red and blue shapes found 4 Red triangular shapes 3 2 Resized shapes to 32x32. Fig. 7. Result of traffic signs Detecti. The traffic signs images database [] used in this paper ctains 3 color images with natural background, and with 3x8 size. Our program is tested all these images, the figure (FIG 7) shows e result of traffic signs Detecti. The program may extract also some blobs which are not a traffic signs, but all these blobs will be presented as input to Traffic Signs Recogniti to decide which the right traffic sign is, and which the bad is. Figure (FIG 8 & ) shows respectively the number of blobs extracted from the first images and the time spent in processing each image by Traffic Signs Recogniti Red circular shapes (d) Fig. 8. Number of blobs extracted from the first images 4 P a g e

5 (IJACSA) Internatial Journal of Advanced Computer Science and Applicatis, 4 Extracti time of the blue circular shapes (µs) Extracti time of the bleu rectangular shapes (µs) IV. TRAFFIC SIGNS RECOGNIATION MODULE A. Preprocessing Stage. Before passing the extracted blobs to the Traffic Signs Recogniti which is based neural network, a preprocessing stage was required to reduce the amount of informati to process, and reduce the computing time cost of the system thereafter. Through this stage, we proceed by extracting the of each blob. Blobs are presented in the 32x32 size, which is 24, and for the three layers R, G and B, the size of neural network input vector will be 2, and this will delay the system. For each channel, a projecti of the is de both vertical and horiztal axes. So for each point the horiztal axis: i= 2 32 () And for each point the vertical axis: j 2 32 (2) Extracti time of the red circular shapes (µs) Is the intensity of the pixel whose coordinates are (i, j) of the layer C. Is the red layer R, green layer G or blue layer B. And values are between and..6 Extracti time of the red triangular shapes (µs) The new is composed of 2 elements, the 32 elements of, and 32 others of for the three layers RGB. B. tarffic signs CORE.4.2 (d) Fig.. Time spent in processing each image by Traffic Signs Detecti and Recogniti Fig.. MLP Structure P a g e

6 The Traffic Signs Recogniti is implemented by using feed-forward artificial neural networks or, more particularly, multi-layer perceptrs (MLP), the most used type of Artificial neural networks [2]. These are a mathematical model, inspired by the brain that is often used in machine learning. It was initially proposed in the 4s and there was some interest initially, but it waned so due to the inefficient training algorithms used and the lack of computing power. More recently however they have started to be used again, especially since the introducti of autoencoders, cvolutial nets, dropout regularizati and other techniques that improve their performance significantly. The MLP includes at least 3 layers. The first e is the input layer; the last e is the output layer, and e or more hidden layers. Each layer of MLP ctains e or more neurs directially linked with the neurs from the previous and the next layer. The figure (FIG ) represents an example of a 3-layer perceptr with three inputs, two outputs, and the hidden layer including four neurs. All the neurs in MLP are similar. Each of them has several input links (it takes the output values from several neurs in the previous layer as input) and several output links (it passes the respse to several neurs in the next layer). The activati functi that used in this paper is binary sigmoid functi, which is defined as: With, it is shown like illustrate (FIG 2); the sigmoidal functi basically has some very useful mathematical properties, moticity and ctinuity. Moticity means that the functi f(x) either always increases or always decreases as x increases. Moreover, ctinuity means there are no breaks in the functi, it is smooth. These parameters are intrinsic properties eventually assist networks power to approximate and generalize functis by learning from data. The Traffic Signs module used in this paper is separated to four MLP functis; each e is used for e type of blobs, (blue circular blobs, blue rectangular blobs, red circular blobs and red triangular blobs). The results of Traffic Signs Recogniti are presented in table I, II and III below. TABLE I. () RESULTS OF RECOGNIZED RED TRIANGULAR TRAFFIC SIGNS Number of tested images Number of Recognized Signs Recogniti rate (%) TABLE II. RESULTS OF RECOGNIZED RED CIRCULAR TRAFFIC SIGNS Number of tested images Fig.. Computing process in a neur The values retrieved from the previous layer are summed up with certain weights, individual for each neur, plus the bias term. The sum is transformed using the activati functi that may be also different for different neurs (FIG ) In other words, given the outputs of the layer, the outputs of the layer are computed as: ( ) (3) (4) Number of recognized signs Recogniti rate (%) TABLE III RESULTS OF RECOGNIZED BLUE CIRCULAR TRAFFIC SIGNS Number of images Number of recognized sign Recogniti rate (%) P a g e

7 The system was ended with a Traffic Sign Recogniti based Neural Networks, which were trained, and validated with a database ctaining more than 3 real images, with complex natural background, to find the best network architecture. Fig. 2. sigmoid functi representati with V. PERSPECTIVES Tests show that the Detecti currently implemented is very fast and good enough (detecti rate %), even if it s depending size of images captured. by cs, the Recogniti is much less efficient (7%). improve performance of the proposed system, an improvement of the Recogniti is required, by using or combining other techniques. We can also use the techniques of multithreading, which allows parallel executi of several programs at the same time, this will significantly improve the executi time of the system which will be more robust and efficient for a real time implementati. VI. CONCLUSION Traffic Sign Recogniti is discussed in this study, by using Neural Networks technique. In the first time the Traffic signs has processed the input images using image processing techniques, such as, threshold technique, Gaussian filter, canny edge detecti, Ctours, algorithm of Ramer -Douglas - Peucker and Fit Ellipse. Next the Traffic Signs Recogniti based e the Neural Networks, were performed to recognize the traffic sign patterns. The main reas to select this method is to reduce the computatial cost inorder to facilitate the real time implementati. The strategy is to reduce number of MLP inputs by pre-processing blobs before giving them to Traffic Sign Recogniti REFERENCES [] A. Lorsakul and J. Suthakorn, Traffic Sign Recogniti for Intelligent Vehicle/Driver Assistance System Using Neural Network OpenCV, The 4th Internatial Cference Ubiquitous Robots and Ambient Intelligence (URAI 27) [2] R. Belaroussi J-P. Tarel, Détecti des panneaux de signalisatiroutière par accumulati bivariée,traitement du Signal 27, 3 (2) pp DOI :.366/ts [3] F.A. Aly and A.E. Alaa. Detecti, categorizati and of road signs for automous navigati.in Proceeding of Advanced Ccepts for Intelligent Visi System, Brussels, Belgium, Aug 24. [4] F.A. Aly and A.E. Alaa. Detecti, categorizati and of road signs for automous navigati.in Proceeding of Advanced Ccepts for IntelligentVisi System, Brussels, Belgium, Aug 24. [] H. X. Liu, and B. Ran, Visi-Based Stop Sign Detecti and Recogniti System for Intelligent Vehicle, Transportati Research Board (TRB) Annual Meeting 2, Washingt, D.C., USA, January, 2. [6] H. Fleyeh, and M. Dougherty, Road And Traffic Sign Detecti And Recogniti, Proceedings of the 6th Mini - EURO Cference and th Meeting of EWGT, pp [7] [4] Arturo de la Escalera, Luis E. Moreno, Miguel Angel Salichs, and José Maria Armingol. Road traffic sign detecti and classificati. IEEE Transactis Industrial Electrics, 44(6) :848, Dec. [8] S. Belgie, Shalpe Matching and Object Recogniti Using Shape Ctexts,IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL 24, NO 24, APRIL 22. [] DATASET THE GERMAN TRAFFIC SIGN DETECTION BENCHMARK, 23 [] Intel Corporati, Open Source Computer Visi Library, Reference Manual, Copyright -2, Available: [] Urs Ramer, "An iterative procedure for the polygal approximati of plane curves", Computer Graphics and Image Processing, (3), (2) [2] Laurence Fausett, Fundamentals of Neural Networks Architectures, Algorithms, and Applicatis, Prentice Hall Upper Saddle River, New Jersey 4. 7 P a g e