Detection and Recognition of Alert Traffic Signs

Transcription

1 Detection and Recognition of Alet Taffic Signs Chia-Hsiung Chen, Macus Chen, and Tianshi Gao 1 Stanfod Univesity Stanfod, CA 9305 {echchen, macuscc, tianshig}@stanfod.edu Abstact Taffic signs povide dives impotant infomation fo safety and efficient navigation. Automatic detection and ecognition of taffic signs inevitably become moe popula. In this pape, an efficient algoithm/platfom is pesented to achieve automatic alet taffic signs detection and ecognition. Histogam of Gadient (HOG) is adapted to extact featues and an ove-complete set of 1680 featues is designed. A cascade classifie fo each sign is tained and built with Suppot Vecto Machine (SVM) as the single stage classifie. To encode the colo infomation, featues fom diffeent layes in RGB space ae combined into a single vecto as the featue descipto. Futhemoe, colo segmentation is pefomed to educe the seach egions and a specially designed integal image is used to extact featues in a look-up manne. Expeimental esults show that ou system can achieve invaiance to illumination, scale and pose. The smallest detectable size is 1!1. The aveage detection ate is aound 98% and the false positive ate is aound 1.6%. The pocessing time fo a typical 60!80 image is aound 7-9 seconds. 1. Intoduction Alet taffic signs ae usually bight in colo in hope to attact dives attention. Howeve, dives may fail to notice, especially when they ae facing glaing light o when the signs ae too small, etc. As such, accuate detection of these signs unde diffeent illumination, pose, and scaling becomes a main issue. Symmety of taffic signs can be exploited fo detection. Howeve, it is subjected to the poblem of illumination and pose vaiance as edges may be blued in images and symmety may be skewed due to elative poses of the camea. SIFT [1] can captue geneal featues with decent efficiency. Howeve, the ecognition pocedue using SIFT equies counting the numbe of matched points. If the taget sign is too small, SIFT can only extact a few key points. Even when the matching is pefect, the numbe of matched points is still small. Due to this disadvantage, it is had and ineffective to use such a counting-based method fo detection and ecognition tasks. Edge histogam infomation is used in taffic sign detection in [2]. Howeve, this appoach used in this pape is specially designed fo cetain shaped signs, and cannot easily be extended to moe geneal signs easily. In [3,], the authos pesent an algoithm fo human being detection, using Viola s face detection taining algoithm and HOG to extact featues. The esults seem to pomise a possible solution to ou poblem. Howeve, the featues ae designed mainly fo the human body and ae not eadily to be used in the taffic sign detection, especially since they do not take the colo infomation into consideation. Based on the analysis above, in ou appoach we adapted HOG to extact featues and designed a 1680 ove complete set of featues. Fom the set of featues that best descibe and distinguish the signs ae selected. The colo infomation is also encoded indiectly into the featues, and a cascade classifie fo each sign is tained and built using SVM as the single stage classifie. Futhemoe, to speed up the pocessing time, colo segmentation is pefomed to educe the seach egions and a specially designed integal image is used to extact featues in a look-up manne. The emaining of this pape is oganized as follows: Section 2 descibes an oveview of ou appoach, while Section 3 and intoduces the featue design, epesentation and leaning pocedue. In Section 5 ou expeimental setup and esults ae shown, and Section 6 concludes this pape. 2. Oveview of ou System As shown in Figue 1, the input image is fist scaled down to ou pe-defined size, passed though colo segmentation and egion eduction, and then candidate egions ae fed into ou detection and classification system. In the colo segmentation and egion eduction step, we naow down the seach egion by filteing out negative egions that do not contain the desied colos and that fail in the gadient magnitude test. The egion with desied colos (ed and white in ou case) and that passed gadient magnitude test, foms a block, and is subsequently sent to the cascade classifie. In each cascade classifie, desied 1 The names of the authos ae odeed alphabetically.

2 Input Image Pepocessing & Colo Segmentation Block Scan Featue Extaction Cascade Classifie Result Figue 1: Oveview of the algoithm. Figue 2: Sub-blocks of the image. featues ae extacted fo classifying. Blocks that fail the tests at any stage in the cascade classifies ae declaed as negative and no futhe tests ae equied. On the othe hand, those blocks that pass the tests though all stages in the cascade classifie ae declaed as positive. 3. Featue Design and Repesentation Based on the obsevation that alets taffic signs have vey bight colo, shape, and patten attibutes, we designed ou featues to catch this infomation Colo Encoding The colo of the alet taffic signs is a valuable clue in sign detection. Howeve, since taffic signs ae exposed unde diffeent lighting conditions, it is had to achieve illumination invaiant detection using the absolute colo values diectly. Theefoe, we d like to encode the colo infomation indiectly, and take advantage of the elative appeaance instead of the absolute values on diffeent colo layes. Take a stop sign fo example. On the R laye in RGB space, since both white and ed pixels colo have high intensity values, the magnitude of the gadient on this laye is small. Howeve, on the G and B layes, ed pixels have low value while the intensity value of white pixels ae still high; thus the magnitude of the gadient on those edges between white and ed pixels is high. As a esult, the gadient of the same pixel on diffeent layes may exhibit vey diffeent values, and these values ae coelated. Theefoe, we combine the gadient values on the thee layes into a vecto and nomalize it using the sum of the gadient magnitude ove the thee layes HOG Based Featue Like SIFT, we adapted the HOG to extact featues. Thee ae two benefits of using HOG-based methods. Fist, Figue 3: Templates and oientation bins. taffic signs have distinctive shapes, and the pattens on diffeent layes may have stong edges. Theefoe, the gadient can efficiently captue these featues. Second, using HOG can help to achieve scale invaiance. In addition, inspied by featue selection in face detection, we designed an ove complete set of the Haa-like featues, fom which we can select featues that can best descibe and distinguish diffeent signs. The details ae descibed in the following subsections Featue Definition Fist, we divide the image into 3-by-3 squae egions and define 1 sub-blocks shown as the shaded egions in Figue 2. Second, we design 15 templates and divide the oientation ange [0 2"] into 8 bins as shown in Figue 3. Each template is a squae block with a shaded cell. Given a colo image, the featue is evaluated on diffeent layes at the i-th sub-block, using the j-th template and the k-th oientation. Each featue is denoted by F( i, j, k ) # [ f (,, ), (,, ), (,, )] T i j k f g i j k fb i j k, whee the thee components coespond to the thee RGB layes. Denote the i-th sub-block by B i and the shaded cell egion in j-th template by C j. Then fo f ( i, j, k ), we fist calculate the gadient at

3 each pixel (x, y) on R laye as: G # [ $ 1 01]* I x T G # [ $ 101] * I y the stength of the gadient at (x, y) is G # G % G 2 2 and the oientation is: G (, ) 1 y $ & # tan ' ) ( * + Gx, Remembe that we divided the oientation ange [0 2"] into 8 bins, and denoted the value of the k-th bin to be / G, if &. bin - k 0 1 0, othewise The featue value fo f ( i, j, k ) is defined as f ( i, j, k) #. Bi - (, ) (, ) k. C j 2G % Gg % Gb 3 whee G ( x, y ) and G ( x, y) ae the magnitudes of the g gadient on the G and B layes, espectively. Similaly, we have f ( i, j, k) # g f ( i, j, k) # b Finally, we have b. Bi. Bi - (, ) (, ) gk. C j 2G % Gg % Gb 3 - (, ) (, ) bk. C j 2G % Gg % Gb 3 F( i, j, k) # [ f ( i, j, k), f ( i, j, k), f ( i, j, k)] T g b Since thee ae 1 sub-blocks, 15 templates, and 8 oientations, we have a total of 1!15!8 = 1680 featues Featue Evaluation Integal images ae used extensively in ou algoithms and thus cut down epeated computation. The pecomputation enables fast computation in pe-pocessing and also featues extaction steps. The following fomula shows the computation of integal image. Any eading in integal image epesents the sum fom (1,1) to this point. As such, summation of a egion C can then be done by k Figue : Flow chat of taining phase using only aithmetic computation of eadings on the fou cones of this egion.. C 05x' 5x,05 y' 5 y 2 3 IGk # - k x', y ' x, y # I x $ 1, y $ 1 % I x % w $ 1, y % h $13 k Gk c c Gk c c c c 2 1, , 13 $ I x $ y % h $ $ I x % w $ y $ Gk c c c Gk c c c. Leaning Methods The set of 1680 featues is ove-complete. Theefoe, we need to select featues fo each of the signs that can best descibe and distinguish them. Moeove, the pocess of selecting featues inteweaves with the pocess of building up the classifie. The oveview of the taining phase is shown in Figue. Ou goal is to build a cascade classifie fo each of the signs. The details ae as follows: Fist, fo each sign we ceated a taining set in which images of the cuently taining sign ae positive samples while all othe signs and andom selected images ae negative samples. Afte that, the 1680 featues fo each of the samples ae computed. Second, fo each of the featues, we use Suppot Vecto Machine with RPF kenel to classify all the samples. Since the numbe of the negative samples is about times moe than that of the positive samples, we adjusted the penalty paametes of the eo tem fo the two classes to deal with this unbalanced situation. Futhemoe, we used diffeent sets of penalty paametes with diffeent emphasis on the positive and negative samples to tade off the detection ate and false positive ate. Thid, afte getting the classification esults using each featue, we selected seveal top featues with the highest detection ate and a elatively smalle false positive ate. We enumeated diffeent selections and combinations of these featues and fo each combination we stacked the selected featues as a highe dimension featue and then use the new single featue to classify the taining set. The new

4 Figue 5: Example of sign/non-sign images in ou data set. featue with the highest detection ate and a elatively smalle false positive ate is selected as a fist stage single classifie. Fouth, having obtained the fist stage single classifie, we used anothe validation set to test the cuent classifie to make sue it has a vey high detection ate ( > 98.5% ) and ecoded the false positive samples. Then, we combined false positive samples fom both the taining set and the validation set as a new negative sample set. This new negative sample set and the positive samples fom the taining set ae combined into a new taining set. Finally, using the new taining set, we epeated the second to fouth steps to fom anothe new single stage classifie which is cascaded afte the cuent classifie. This pocedue is epeated until the cascade classifie achieves the pedefined false positive ate while having a detection ate highe than a pedefined lowest detection ate. 5. Expeimental Results The alet taffic sign detection algoithm was extensively examined in this eseach. Hee we pesent esults on seveal sequences of images. The input image is fist scaled down to 60x80, and then passed into ou alet taffic sign detection system. In these sequences, some signs appea alone, while multiple-sign scenaios ae also included. These sequences also contain diffeent lighting, scale, and pose conditions fo vaious signs, and ae suitable fo examining the obustness of ou algoithm Data Set and Taining In this pesentation, we ceated a taffic sign taining data set with 1921 sign/non-sign samples fom 800+ photos we took aound Stanfod campus. The data set consists of seveal image sequences, which include taffic signs with diffeent poses and lighting conditions, and with size anges fom 16 to 220 pixels. These signs ae hand labeled and scaled to 2!2 by bilinea intepolation. Non-sign Figue 6: The fist two stages of featues fo stop and do not ente sign, espectively. samples ae andomly copped fom these input images, with vaious sizes, and ae also scaled to 2!2 by bilinea intepolation. The final taining set includes 161 stop signs, 72 yield signs, 73 no left tun signs, 83 do not ente signs, and 1532 non-sign images. Fo each sign, the taining set is divided into two classes, whee the positive samples ae the cuent taget sign images while the negative samples ae all othe signs and andomly selected non-sign images. Some of the taining set images ae shown in Figue 5. In Section 3.2, we have demonstated how featues ae epesented, and in Section, we have demonstated how to lean and iteatively combine o stack featues fom the featue pool of 1680 featues. The geneal ule is to select featues with a high detection ate and a low false positive ate. These two guidelines should be consideed as a whole, not individually. In Figue 6, the selected featues fo the fist two stages in the cascade classifies fo the stop and do-not ente signs ae shown, espectively Cascade Sign Detectos The cascade classifies fo diffeent signs ae combined seially to fom the final alet taffic sign classifie. As shown in Figue 7, an input image is fist examined by the the stop sign classifie. If the image passes though all stages of stop sign cascade classifie, we label it as a possible stop sign egion. Othewise, we pass it into the do not ente sign classifie, and the same pocedue goes on fo the no left tun and yield classifies. If we find out that the cuent image does not belong to any of these signs, we declae it as a non-sign egion.

5 Input Image Gadient magnitude + ROI 5/12 Region 1 1/ Non Sign Region 2 1/3 Stop Do not ente 5.3. Speed Consideation No left tun Yield Figue 7: Cascade sign detectos Gadient Magnitude Gadient magnitudes help to quickly naow the seach egion. The alet signs have a high contast in RGB space; fo example, white is [255, 255, 255] as compaed to ed [255, 0, 0]. As such, an edge in RGB images will have no edge in the ed laye, but will have edges in the othe two layes. Using this fact, we design a hype-plane with [-1 ½ ½] as the diection vecto. The sum of the gadient magnitudes of thee layes effectively sepaate signs fom non-signs with 99.% positive detection and 20% of false positive. This geatly educes the seach egion and impoves computational efficiency. The hype-plane coefficients ae intuitive, since they punish the gadient in the ed laye and ewad the geen and blue layes. Anothe thing to note is that absolute gadient magnitudes ae used instead of nomalized magnitudes. This is moe effective to eliminate negative images with noise since nomalized ones could give a lage magnitude atio even though the oiginal magnitudes ae not lage. This hype-plane method is scale invaiant. Clealy, un-nomalized magnitudes will also give a faily lage amount of false positives, especially it also gives positive esults fo lage egions that contain desied signs. To futhe tackle this poblem, we intoduce anothe detection featue that exploits the aw colos of signs by using colo segmentation Colo Segmentation In addition to the integal image technique discussed in Section 3.2.2, we also incopoated a ed-white colo segmentation to futhe educe the amount of calculation. The colo segmentation is designed based on two obsevations. Fist, in pactical settings taffic signs aely show up on the bottom pat of an image, and theefoe, we can ignoe the bottom pat diectly. Second, signs on the top pat of an image tend to be a lot lage than signs in the Image Close (a) Input Image Region 1 Region 2 (b) Image Dilate th 1 th 2 Combine Output Image Figue 8: Colo segmentation (a) two egions fo diffeent consideation, and (b) segmentation algoithm. middle pat of the image. Based on these obsevations, we divide the top half of the image into two sub egions, and apply diffeent decision ules to eliminate the non-sign egion while peseving ou egion of inteest. The two egions we employed in ou system ae as shown in Figue 8(a). Note that we delibeately ovelap the two egions in ode to decease the possibility that the signs ae cut in half, and to ensue that we do not lose any egions with signs duing the colo segmentation pocess. We opeate ou ed-white colo segmentation on the YCbC colo space [5]. The thesholds ae chosen based on the statistical mean and standad deviation of the CbC plane. To bette diffeentiate signs in egion 1 with extemely bight sky and buildings, we pefom image closing on the colo thesholded image. Fo egion 2, howeve, we need to intensify the esponse fo small signs. Thus, we pefom dilation on the colo thesholded egion 2 image. The colo segmentation flow is shown in Figue 8(b). 5.. Expeiments on Real-Wold Test Set The poposed algoithm is implemented in Matlab, and is tested on a test set of 99 images of size 60!80. This test set contains seveal image sequences with single and

6 multiple signs unde diffeent lighting, scale, and pose conditions. The fou alet taffic signs detectos ae tested independently in ode to examine the efficiency and coectness of ou design. Figue 9 shows some examples of detection esults, and Table 1 gives the pefomance of the poposed algoithm. Fom Table 1, we can see that ou system has an aveage un time aound 7~9 seconds fo 60!80 images, and is able to achieve a vey high detection ate (DR), while keeping a low false positive ate (FP). TABLE 1. PERFORMANCE OF THE PROPOSED ALGORITHM. Avg Time Smallest Size DR FP Stop x % 0.20% Yield 7.3 1x % 1.20% No left tun x %.81% Do not ente x % 0.20% 6. Conclusion and Discussion In this pape, we have demonstated an efficient alet taffic sign detection and ecognition system with novel obust featue extaction and epesentation. Ou expeiment shows that the system can achieve a high detection ate of 92%~100% (aveage of 98%) and a false positive ate of 0.19%~5% (aveage of 1.6%) fo diffeent signs. The system can also achieve illumination, scale, and pose invaiance. With an ove-complete set of featues, ou system is able to fully extact distinguishable featues fo diffeent taffic signs, and theefoe can be easily extended to accommodate new signs. The pocessing time in Matlab fo a 60x80 image is aound 7-9 seconds. When used with tacking, the coelation between successive fames can be exploited, and eal time taffic sign detection can be achieved. Refeences [1] D. Lowe. Distinctive Image Featues fom Scale-invaiant Keypoints. Intenational Jounal of Compute Vision, 60(2):99-110, 200. [2] B. Alefs, G. Eschemann, H. Ramose, C. Beleznai. Road Sign Detection fom Edge Oientation Histogams. IEEE Intelligent Vehicles Symposium, [3] H.X. Jia, Y.J.Zhang. Fast Human Detection by Boosting of Oientated Gadient. Fouth Intenational Confeence on Image and Gaphics. [] Q. Zhu, S. Avidan, M. Yeh, and K. Cheng. Fast Human Detection Using a Cascade of Histogams of Oiented Gadients. IEEE Compute Society Confeence on Compute Vision and Patten Recognition [5] Y.B. Damavandi, K. Mohammadi. Speed Limit Taffic Sign Detection and Recognition. IEEE Confeence on Cybenetics and Intelligent Systems, 200.

7 Figue 9: Examples of detection esults. These esults ae selected to demonstate ou system achieves illumination, scale and pose invaiance.