An Efficient Framework for Road Sign Detection and Recognition

Transcription

1 Sensors & Transducers, Vol. 165, Issue, February 014, pp Sensors & Transducers 014 by IFSA Publshng, S. L. An Effcent Framework for Road Sgn Detecton and Recognton Duanlng L, Qngxue Xu Bejng Unversty of Posts and Telecommuncatons, No. 10 Xtucheng Road, Hadan Dstrct, Bejng, , Chna Tel.: , fax: E-mal: Receved: 11 November 013 /Accepted: 8 January 014 /Publshed: 8 February 014 Abstract: Road sgn detecton and recognton s a sgnfcant and challengng ssue not only for assstng drvers but also navgatng moble robots. In ths paper, we propose a novel and fast approach for the automatc detecton and recognton of road sgns. Frst, we use Hue Saturaton Intensty (HSI) color space to segment the road sgns color. And then we locate the road sgns based on the geometry symmetry, as almost all the shapes of road sgn shapes are symmetrcal such crcle, rectangle, trangle and octagon. The proposed shape feature s further appled to classfy the shape ntally. Fnally, the road sgns are exactly recognzed by support vector machne (SVM) classfers. We test our proposed method on real road mages and the expermental results show that t can detect and recognze road sgns rapdly and accurately. Copyrght 014 IFSA Publshng, S. L. Keywords: Road sgn recognton, Shape symmetry, Shape detecton. 1. Introducton Road sgn detecton and recognton s a fundamental and sgnfcant topc n robot navgaton and computer vson. As road sgn detecton and recognton system provdes meanngful navgaton nformaton, t plays an mportant role not only n notfyng drvers the current state of the road but also n the navgaton of moble robots and other unmanned vehcles. The system warns drvers about danger of the traffc stuaton, and reduces traffc accdents effectvely. On the other hand, the system can automatcally navgate robots and vehcles by dentfcaton of road sgns, whch s urgently needed n warehouse storage ndustry and many scentfc research felds. The detecton and recognton system faces multple dffcultes and challenges because of the complex envronment of roads where there are a lot of dfferent objects on the roads such as pedestrans, vehcles, buldngs and trees. The low qualty of mages ncludng lght varaton and moton blur also brngs many challenges to the system. Another key dffculty of ths system s how to detect and recognze road sgns rapdly and n real-tme. Due to the sgnfcance and dffcultes of the road sgn detecton and recognton, a great amount of research has been done to solve ths problem. Usually, there are two major phases the road sgn recognton algorthms: 1) Detecton - locate road sgns from real mages, ) Recognton - dentfy the types of the road sgns. A road sgn has three most mportant characterstcs: color, shape, and pattern, whch are desgned to be dfferent from each other. The nformaton of road sgn s acqured from the basc meanng through the combnaton of these three 11 Artcle number P_1886

2 Sensors & Transducers, Vol. 165, Issue, February 014, pp propertes. Thus, most of the solutons rely heavly on these features of road sgns [1]. In the detecton phase, color segmentaton s the most common method. RGB space s used for the color segmentaton n [, 3]. As the RGB space s susceptble to the change of the lght, HIS space s preferred. The factors of hue and saturaton are nvarant to brghtness and shadow. [4, 5] use HIS space to extract color feature. Many other methods focus on the shape feature of road sgns, whch s another sgnfcant and vsble feature of road sgn. Shape s an mportant attrbute of road sgns both n detecton and recognton sectons. Color nformaton s no need n shape detecton. As the contour and the pattern of road sgns are both dfferent, a number of approaches based on shape have been proposed. [6] detects crcular sgns by two-tme Hough Transform algorthm. The algorthm classfes the edge ponts and uses the geometry feature of crcle to reduce the transform dmensons. In [7], the radal symmetry feature s adopted for locatng road sgns. Indvdual pxels vote for a common centre of symmetry wth ther results accumulatng n a common votng map. A shape symmetry method operatng on the gradent of grayscale mages s proposed n [8]. In [9, 10], template matchng approach s proposed to search for road sgns. All road sgn shapes are stored n the database. Each potental sgn s normalzed n sze and compared wth every template of the same shape. Ths method can be easly modfed to nclude new classes of sgns. [11] detects the trangular and rectangular road sgns by passng lnes ntersecton. In the second phase, neural networks are the most wdely used, due to ther powerful classfcaton capablty [1, 13]. When the algorthms are appled n road sgn classfcaton, neural networks can be traned to recognze road sgns wthn a regon of nterest. In [14], the road sgns are dentfed by template matchng approach. A lot of research based on SVM [15-17] wth dfferent extracted features has been appled n many applcatons. In addton, Fuzzy logc [18] and Zernke Moments [19] are also employed n the recognton of road sgns. Although many road sgn detecton and recognton algorthms have been proposed, there are stll many problems. Most of detecton methods lke Hough Transform, radal symmetry and template matchng are complcated to operate and also not fast enough, especally for large mages. In addton, the recognton approaches are usually dependng on statstcal theory. Lots of samples are needed and the calculaton s large. Thus, an effcent and rapd road sgn detecton and recognton algorthm s very mportant. In ths paper, we propose an effcent and rapd road sgn detecton and recognton algorthm. In the detecton step, we frstly apply the HSI color segmentaton to obtan the Regon of Interest (ROI), and then propose a novel method based on the shape symmetry to detect the road sgn accurately. In the recognton step, we propose a shape symmetry feature to classfy the shapes of detected road sgns and then apply SVM classfer to recognze road sgn accurately. The proposed system s much more stable and effcent comparng to the tradtonal methods. Especally, our detecton algorthm based on the shape symmetry s fast and effcent. The algorthm sets central axes n the segmented ROI and analyzes the symmetry by calculatng the area rato from dfferent sdes of the axs. In ths way, we locate the road sgns accurately from ntal ROIs segmented by the color segmentaton. To classfy the road sgns rapdly, the nterestng regons are dvded nto four parts by the vertcal axs and the horzontal axs, whch are through the centre of ther boundng boxes. The shape categores are classfed by analyzng whether they are symmetrc and the values of area ratos. To prove the effectveness of our framework, we test the proposed method on the database from []. The experments show that our system s very effcent to be used n real applcaton.. Methodology In ths paper, we present a fast and ntellgent detecton and recognton method whch s broadly llustrated n Fg. 1. The detals of each phase are presented as bellow: Fg. 1. Flow dagram of the proposed methodology. The approach conssts of two phases: detecton and recognton. And each phase has two steps. In the detecton phase: 1) Step 1: color segmentaton: extract the regons of road sgns from real background by usng HSI color thresholdng. 113

3 Sensors & Transducers, Vol. 165, Issue, February 014, pp ) Step : symmetry detecton of ROI: keep road sgns as canddate blobs and removes non-road sgns objects by geometry symmetry of the shapes. In the recognton phase: 3) Step 3: shape classfcaton: further use geometry symmetry and area to classfy the dfferent shapes. 4) Step 4: SVM recognton: dentfy exact road sgn types based on SVM..1. Color Segmentaton Color s often the man cue explored to fnd the areas where road sgns appear. The man defect of the color-based approaches s that outdoor llumnaton that mght affect the color acqured by the mage sensor. Compared to RGB space, HSI space s more smlar to human senstvty of colors. The HSI system encodes color nformaton by separatng out an overall ntensty value from two values ncludng hue and saturaton to make t more mmune to lghtng changes. Therefore, thresholdng n HSI color space s an effcent way due to ts strong robustness aganst lghtng varaton. The RBG space s converted to HSI space by the followng formulas (1)-(4): slow to be detected or recognzed by classfer n these nose regons. Thus, t s very necessary and mportant to remove the regons wthout road sgns n order to acheve better performance and ncrease the speed of the process of detecton and recognton. In the step, we propose an effectve and fast method to detecton the road sgns from the segmented regons based on that most of the road sgn shapes are symmetrcal. We know that most of the road sgn shapes are trangle, rectangle, crcle or octagon and the same characterstc of these shapes s axal symmetry. To detect the symmetrcal n the bnary regons, the canddate blobs are dvded nto left and rght parts by the vertcal central axs of ther boundng boxes as Fg.. The boundng box s the smallest rectangle whch contans the object. Accordng to the settng of µ, road sgns are detected. µ s the dfference between rato of the pxels n the left part of ROI to the pxels n the left part of boundng box and rato of the pxels n the rght part of ROI to the pxels n the rght part of boundng box. The pxels n the regon refer to ts area. 1 ( R G) ( R B) cos, ( R G) ( R B)( G B) Hue 360 B G B G (1) () Fg.. The road sgn shapes are dvded nto left and rght parts by the vertcal central axs. 3mn( RGB,, ) Saturaton 1 R G B, (3) l r l r, 0<µ<1, (5) l r b ROI ROI ROI ROI box box 1 Intensty ( R G B)/3, (4) where R, G, B represent the red, green and blue brghtness of a pxel wth ranges from The hue and saturaton components take values rangng from 0 to 360 and from 0 to 55, respectvely. After mage converson nto sutable color model for processng, segmentaton process s carred out to dentfy the possble canddate blobs. The thresholds for the crteron are fxed by a seres of tests. After the color segmentaton, Mnmum-Maxmum method s appled for flterng the blobs from the nosy envronment and then the mage s analyzed usng pre-processng methods lke medan operaton and morphologcal operaton. The result of ths color segmentaton step s a bnary mage wth some canddate blobs... Symmetry Detecton of ROI After color segmentaton n Step 1, most segmented regons are stll not the road sgn regons whch we want. Road sgns wll be very dffcult and where l ROI represents the pxels n the left part of ROI, represents the pxels n the rght part of ROI. r ROI s equal to pxels n the left part of boundng box and l box s equal to pxels n the rght part of boundng box, actually they are both 1/ pxels n the boundng box(b). The theoretcal value of µ s 0, consderng the breakage and deflecton of road sgns, the value of µ can be settng based on a lot of experments. The result of symmetry detecton s shown n Fg. 3. Ths shape detecton approach has strong robustness as t detects shapes based on edges, and wll effcently reduce the search for road sgns from the whole mage to a small number of pxels. The advantage of savng processng tme s descrbed as follows: The detecton algorthm operaton s no longer complex as other methods. Ths approach does not search for all the mage and pxels, t just computes the area of canddate blobs after color segmentaton wth low calculaton. 114

4 Sensors & Transducers, Vol. 165, Issue, February 014, pp Frst, the nterestng regons are dvded nto four parts by the vertcal axs and the horzontal axs whch are through the centre of ther boundng boxes as Fg. 4. (a) Fg. 4. The road sgn shapes are dvded nto four parts by the vertcal central axs and the horzontal axs. Then we use to classfy the shapes accordng to Table 1. r / b, = 1,, 3, 4, (6) (b) (c) where r, b represent the area of the nterestng regon and the boundng box n part 1, respectvely. Table 1. Road sgn shape classfcaton. Crcle (d) (e) Trangle Rectangle Octagon (f) Fg. 3. Detecton of ROI. (a) - real road mage from []; (b) - segmentaton result by blue thresholdng; (c) - detecton result based on geometry symmetry; (d) - segmentaton result by red thresholdng; (e) - detecton result based on geometry symmetry; (f) - road sgn detecton results..3. Shape Classfcaton Road sgn recognton s a hard and complex task. In ths step, the shape of road sgns s classfed by usng the part area rato after the detecton phase. Through the smply comparng of and confrmng the range of the values, crcle, trangle, rectangle and octagon can be dstngushed precsely. The four ratos of crcle, rectangle and octagon are n common n theory. Accordng to ths pont, the trangle road sgns are dentfed. Furthermore whether the trangle s regular or nverted can be known at the same tme by makng sure that 1, are bgger or not. Then dfferent values of rato are used to recognse crcle, rectangle and octagon. In ths step, we dvded the regon nto four parts, whch results n exact classfcaton. It s more accurate than the approach usng the whole area rato. And few pxels are examned n ths method. In general, shape classfcaton s effcent and fast. 115

5 Sensors & Transducers, Vol. 165, Issue, February 014, pp SVM Recognton The recognton step s used to classfy the exact sgn types based on Super vector machnes (SVMs). SVMs are pattern classfers wth hgh generalzaton ablty compared wth conventonal ones. The am s to fnd an optmal hyperplane n the hgher dmenson feature space that can separate the data n the best way. SVMs are formulated to solve bnary classfcaton [0]. In the case of two separable classes, the tranng data are labeled { x,y}, where d y { 1, 1}, x { R }, = 1,,, n. The vectors x are HOG features, whch wll be ntroduced later. The values y are 1 for one class and 1 for the others, d s the dmenson of the vector, and n s the number of tranng vectors. If a hyperplane {w, b} separates the two classes, the ponts that le on t satsfy: T x w b 0, (7) The recognton state nput s the features of ROI. In the classfcaton system desgn, features extracton of traffc sgns s the foundaton of well realzaton of road sgns recognton. In ths paper, the HOG features are extracted from the mage, whch represent the occurrence of gradent orentatons n the mage. The HOG descrptor provdes excellent performance compared to other exstng feature descrptors, due to ts fne-scale gradents, fne orentaton bnnng and hgh-qualty local contrast normalzaton n overlappng descrptor blocks [1]. The HOG features are computed on a dense grd of cells usng local contrast normalzaton. A nnebn hstogram of unsgned pxel orentatons weghted by magntude s created for each cell. These hstograms are normalzed over each overlappng block. The components of the feature vector are the values from the hstogram of each normalzed cell. Although ths ntensve normalzaton produces large feature vectors, t provdes hgh accuracy. The dmensons D of the HOG feature vector s computed usng R R wdth hegh D 1 1 B H, Mwdth Mhegh (10) where R s the regon, M s the cell sze, B s the number of cells per block, and H s the number of hstograms per cell. Road sgn examples and related HOG features are shown as Fg. 5. The road sgn detecton and recognton results are dsplayed n Fg. 6. Dfferent experments ncludng sunny, cloudy and rany condtons are done. And the results are apparent and effcent. where w s the normal to the hyperplane, b /w s the perpendcular dstance from the hyperplane to the orgn. In order to obtan the optmal hyperplane, the followng quadratc constrant optmzaton problem s needed to solve: mn (w) 1 w (8) A nature way to assgn extra cost for errors s to change the objectve functon to be mnmzed from w / to k w / C, where C s a parameter to be chosen by the user. A large C corresponds to assgnment of hgher penalty to errors. However, n many cases, the data cannot be separated by a lnear functon. A soluton s to map the nput data nto a dfferent space Φ(x). Ths paper uses RBF kernel functon K (x, x j ) as follows: Fg. 5. Road sgns and related hstograms of orented gradents (HOG features). K x x x x (9) (, ) exp( ), 0 Fg. 6. Some results of road sgn detecton and recognton on the mages from []. 116

6 Sensors & Transducers, Vol. 165, Issue, February 014, pp Expermental Results To prove the effectveness of our framework, we test the proposed method on the database from []. Each mage s segmented usng the hue component of a HSI mage. Hstograms of hue are bult for red, blue, and yellow sgn colors. Geometry symmetry s appled to dscard non road sgn objects and speed up the process. And shape symmetry and area are further used to classfy the dfferent sgn shapes ntally. SVMs wth RBF kernel s then used to recognze each sgn type based on a tranng set of between 50~100 mages per class on an unspecfc number of classes. A model s obtaned after SVM tranng and t s used to recognze canddate regons drectly. Recognzng road sgns wthout tranng the database every tme saves a lot of processng tme. To assess the performance of the system, a test data set s collected and road sgn mages are obtaned from the Internet. Images are all scaled to pxels. The mplementaton s runnng on a 1.5 GHz Intel Core T550 central processng unt under Vsual Studo 008. There are two mportant crtera that defned for evaluatng the system as below: Processng tme: the tme of recognzng road sgns from an mage. Ths crteron s calculated based on the average processng tme of detecton and recognton. Accuracy rate: the accuracy of recognzng the exact road sgn types n the mages. Ths measure s calculated as follows: rato of the number of sgns that have been correctly recognzed to the total number of road sgns. Accordng to the experments, the average processng tme s 1.3 second, and the accuracy rate of dfferent condtons can be seen n Table. Stuaton Table. The accuracy rate of experments. Image samples Correct recognton Accuracy rate Sunny % Cloudy % Rany % Important ponts of ths new method are expresson: smple search procedure, the approprate speed n both detecton and recognton phases, ndependent of the scale, flexblty and ablty to change database easly wthout any problem, supportng a large number of road sgns. Some more experments are dong to study penalty C and kernel parameter γ from SVMs mpact the forecastng accuracy. When one parameter s fxed, we can see how the other one nfluence the accuracy rate n Fg. 7 and Fg. 8, respectvely. The optmal C s 34 and γ s 0.5. accuracy rate accuracy rate 95,00% 90,00% 85,00% 80,00% 0 kernel parameter 4 γ 6 Fg. 7. Accuracy rate wth dfferent γ (C=34). 100,00% 95,00% 90,00% 85,00% 80,00% parameter C Fg. 8. Accuracy rate wth dfferent C (γ=0.5). 4. Conclusons In ths paper, a new method s provded for road sgn detecton and recognton. The method depends manly on color segmentaton, shape detecton and classfcatons and SVM recognton exactly. In the system, the especally novel pont s the detecton and classfcaton based on the geometry symmetry and shape area, as all the road sgns shape lke crcle, rectangle, trangle, octagonal are symmetrcal. Currently, collectng numerous road sgn mages n real envronment and mprovng the system s ongong to enhance the system performance. For future work, more samples need to be tested n order to get a better evaluaton of the system and more methods wll be proposed to keep the road sgn detecton and recognton system more effcent. Furthermore, the approach based on symmetry, the property of some shapes, wll also be consdered to apply to other felds. Acknowledgment The authors gratefully acknowledge the assstance of the revewers and the support of Natonal Scence Foundaton Grant No & ; Program for New Century Excellent Talents n Unversty Supported under Grant No. NCET ; Specalzed Research Fund for the Doctoral Program of Hgher Educaton under Grant No ; the Trbology Scence Fund of State Key Laboratory of Trbology and the Fundamental Research Funds for the Central Unverstes under Grant No. 01LD03, though any opnons, fndngs, and conclusons or recommendatons are those of the authors and do not necessarly reflect the vews of the Foundatons 117

7 Sensors & Transducers, Vol. 165, Issue, February 014, pp References [1]. C. Y. Fang, C. S. Fuh, P. S. Yen, S. Cherng, et al., An automatc road sgn recognton system based on a computatonal model of human recognton processng, Journal of Computer Vson and Image Understandng, 96,, 004, pp []. Xng Yang, Enhancement for road sgn mages and ts performance evaluaton, Internatonal Journal for Lght and Electron Optcs, 14, 14, 013, pp [3]. W. Rtter, F. Sten, R. Janssen, Traffc sgn recognton usng colour nformaton, Journal of Mathematcal and Computer Modellng,, 4, 1995, pp [4]. A. de la Escalera, J. Ma Armngol, M. Mata, Traffc sgn recognton and analyss for ntellgent vehcles, Journal of Image and Vson Computng, 1, 3, 003, pp [5]. Jn-Y Wu, Chen-Chung Tseng, Chun-Hao Chang, et al., Road Sgn Recognton System Based on Gentle Boost wth Sharng Features, System Scence and Engneerng (ICSSE), Macao, June 8 10, 011, pp [6]. Huang Sheng, Gao Chenqang, Meng Shang, Crcular Road Sgn Detecton and Recognton based on Hough Transform, n Proceedngs of the 5 th Internatonal Congress on Image and Sgnal Processng (CISP), Chongqng, Schuan, Chna, Oct , 01, pp [7]. Luke Fletcher, Gareth Loy, Nck Barnes, et al., Correlatng drver gaze wth the road scene for drver assstance systems, Journal of Robotcs and Autonomous Systems, 5, 1, 005, pp [8]. G. Loy, N. Barnes, Fast shape-based road sgn detecton for a drver assstance system, n Proceedngs of the Internatonal Conference (IROS), Senda, Japan, Sep./Oct. 004, pp [9]. Andrzej Ruta, Yongmn L, Xaohu Lu, Real-tme traffc sgn recognton from vdeo by class-specfc dscrmnatve features, Journal of Pattern Recognton, 43, 1, 010, pp [10]. J. Torresen, J. W. Bakke, L. Sekanna, Effcent recognton of speed lmt sgns, n Proceedngs of the 7 th Internatonal IEEE Conference on Intellgent Transportaton Systems, Oct. 3-6, 004, pp [11]. Fard A. L., Chalechale A., Razav Ebrahm S. A., Recognton of the trangular and rectangular road sgns usng passng lnes ntersecton, n Proceedngs of the 16 th CSI Internatonal Symposum on Artfcal Intellgence and Sgnal Processng (AISP), Shraz, Fars, May -3, 01, pp [1]. J. Stallkamp, M. Schlpsng, J. Salmen, et al., Man vs. computer: Benchmarkng machne learnng algorthms for traffc sgn, Journal of Neural Networks, 3, 01, pp [13]. A. Brogg, P. Cerr, P. Medc, P. P. Porta, et al., Real tme road sgns recognton, n Proceedngs of the IEEE Intellgent Vehcles Symposum, Istanbul, June 13-15, 007, pp [14]. Malk Raba, Khurshd Javad, Ahmad Sana Nazr, Road Sgn Detecton and Recognton usng Colour Segmentaton, Shape Analyss and Template Matchng, n Proceedngs of the Internatonal Conference on Machne Learnng and Cybernetcs, Hong Kong, Chna, Aug. 19-, 007, pp [15]. Pazhoumand-Dar H., Yaghob M., DTBSVMs: a New Approach for Road Sgn Recognton, Computatonal Intellgence, n Proceedngs of the Communcaton Systems and Networks (CICSyN), Lverpool, July 8-30, 010, pp [16]. S. Maldonado Bascón, J. Acevedo Rodríguez, S. Lafuente Arroyo, et al., An optmzaton on pctogram dentfcaton for the road-sgn recognton task usng SVMs, Journal of Computer Vson and Image Understandng, 114, 3, 010, pp [17]. P. Gl-Jmenez, H. Gomez-Moreno, P. Segmann, et al., Traffc sgn shape classfcaton based on Support Vector Machnes and the FFT of the sgnature of blobs, n Proceedngs of the IEEE Intellgent Vehcles Symposum Istanbul, Turkey, June 13-15, 007, pp [18]. Chen-Chuan Ln, Mng-Sh Wang, Tang-Chun Yang, Fuzzy adaptve pre-processng models for road sgn recognton, n Proceedngs of the Second World Congress on Nature and Bologcally Inspred Computng (NaBIC), Dec , 010, pp [19]. Hasan Fleyeh, Dougherty M., Aenugula D., Invarant Road Sgn Recognton wth Fuzzy ARTMAP and Zernke Moments, n Proceedngs of the Intellgent Vehcles Symposum, Istanbul, June 13-15, 007, pp [0]. Bernhard E., Isabelle M., Vladmr N., A Tranng Algorthm for Optmal Margn Classfers, n Proceedngs of the Ffth Annual Workshop on Computatonal Learnng Theory, New York, NY, USA, 199, pp [1]. Dalal, N., Trggs B., Hstograms of orented gradents for human detecton, n Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton, San Dego, CA, USA, June 5, 005, pp []. Gao X. W., Podladchkova L., Shaposhnkov D., Hong K. & Shevtsova N., Recognton of traffc sgns based on ther colour and shape features extracted usng human vson models, Journal of Vsual Communcaton and Image Representaton, 17, 4, 006, pp Copyrght, Internatonal Frequency Sensor Assocaton (IFSA) Publshng, S. L. All rghts reserved. ( 118