Traffic Sign Classification Using Ring Partitioned Method

Transcription

1 Traffic Sign Classification Using Ring Partitioned Metod Aryuanto Soetedjo and Koici Yamada Laboratory for Management and Information Systems Science, Nagaoa University of Tecnology 603- Kamitomioamaci, Nagaoa Niigata JAPAN a_aryu@alice.nagaoaut.ac.jp, yamada@js.nagaoaut.ac.jp Abstract---Traffic sign recognition usually consists of two parts : detection and classification. In tis paper we describe te classification stage using ring partitioned metod. In tis metod, first te RGB image is converted into gray scale image using color tresolding and istogram specification tecnique. Tis gray scale image, called as specified gray scale image is invariant to te illumination canges. Ten te image is classified using ring partitioned metod. Te image is divided by several concentric areas lie rings. In every ring te istogram is used as an image descriptor. Te matcing process is done by computing te istogram distances for all rings of te images by introducing te weigts for every ring. Te metod doesn t need a lot of samples of sign images for training process, alternatively only te standard sign images are used as te reference images. Te experimental results sow te effectiveness of te metod in te matcing of occluded, rotated, and illumination problems of traffic sign images. I. INTRODUCTION Road traffic signs are important parts in te transportation system. An intelligent system for recognizing traffic signs would be benefical in several ways. It could assist te driver before passing tem; a warning system sends signal to te driver as response to te traffic sign, for example te speed limitation.te driver assist system will mae te driving more safer and easier. In te autonomous veicle, in wic no uman driver in te car, te traffic sign recognition become essensial part in order to guide te veicle wen running in te street. Te maintenance and inventory of tose signs are te oter benefits. By recognition of te status or condition about te traffic signs, te maintenance and inventory can be effectively done using intelligent vision system rater tan manual inspection by uman. Researcers divide te traffic sign recognition system into two stages : te detection stage and te classification stage [,3,,5,7]. In te detection stage, color or sape features are used to generate te candidate regions were traffic signs migt exist in a particular image. In te classification stage, te candidate signs obtained before are classified. Tere are no standard algoritm in te bot stages. Neural networs are used in te classification stage in [3,,7 ]. In [5], te tecnique of Marov model is used to classify te sign. Te matcing pursuit filter is used for classification in []. Te above-mentioned metods consider te rotated, scaled, illumination canges of te traffic sign images. In [3,,7] te rotated, scaled, and imperfect signs were used as training patterns of artificial neural networs. Tose signs were also used as training patterns of matcing pursuit filter in []. In [], since matcing pursuit filter uses wavelet to extract te features, te recognition is invariant to te illumination canges. In [7], te ligting, weater, and sadows influences are reduced by converting te original image to te new image using a pre-selected formula. After conversion, only pixels wose original red components dominate te oter two components (green and blue) can ave a nonzero red component in te new image. Te ue and saturation components were taing into account in order to avoid ligting condition [3]. In [], te template based correlation wic was immune to te ligting canges was used. Te partial occluded images were considered in [3,7]. In [3] te occlusion affects te ue values, producing te ig variance. In te classification stage, te occlusion information obtained in te first analysis is used to decide te matcing image given by neural networ. If te networ discovers tat tere is no learnt pattern similar enoug to te sign presented, te system verifies if tere is occlusion. In te case of occlusion, te maximum value is taen altoug it was low. On te oter and, if te networ response is ig but tere is an occlusion, it means tat te occlusion is small and affects only te sign border. In [7], te partial occluded signs were used in te training process by added Gausssian noise and sift te perfect signs to te left, rigt, up and down. In te previous wors, artificial neural networ or matcing filter are used to classify te traffic signs. Tose metods need a lot of samples of signs in te training process to acieve a good recognition of te disturbed signs. Concerning wit te occlusion problem, in [7] te result depends on te sample patterns used in te training process. Meanwile, in [3] te occlusion information obtained in te previous analysis is used to justify te classification of te neural networ. Here, we propose a new simple metod to cope wit te problems of occlusion, rotation, and illumination canges of traffic sign images. Te metod doesn t need a lot of samples of sign images for training process, alternatively only te standard sign images are used as te reference images. In order to cope wit te occlusion problem, we tae into account te degree of occlusion in te matcing process. It will increase te robustness of te matcing regarding to te occlusion. Te metod divides a rectangular image into several rings, called as ring partitioned metod. An additional metod is introduced in te pre-processing stage to convert te RGB image into a gray scale image wic is invariant to illumination canges, called as specified gray scale image. Te istogram distance matcing is used to matc te target

2 images wit te reference images. Since, te istogram is used, te rotation invariant properties is acquired. Comparing wit te oters, te proposed metod is simple, fast, and robust. Te outline of te paper is as follow. In section, we introduce te pre-processing stage to convert RGB image into a specified gray scale image. Te ring partitioned matcing metod is explained in section 3. Section contains te experimental results. Conclusion is covered in section 5. II. PRE-PROCESSING Te traffic sign images often come wit unperfect images. Te signs are partially occluded by objects or interfered by sadows, rotation, oblique (image tat taen from side view) and affected by illumination canges. Since te illumination canges influence te istogram of image, a pre-processing stage is needed to convert a RGB image into a gray scale image wic is invariant to te illumination canges and sadows. Here, we propose a simple metod based on color tresolding to convert a RGB image into a gray scale image. Fig. illustrates te proposed metod. in some columns of te image will be missed. If it occurs, te pixels along tose columns are replaced wit te blac color. To examine te columns of image wic is occluded, te projection tecnique [5] of red color is employed. Te sign mages come wit te different sizes. Te specified gray scale image obtained before is resized to te size 70x70 pixels. Te resizing image is used to speed up te matcing process. Beside tat, by resizing te image into a square image, an oblique image wic te circular red sign appears as an ellipse can be converted into a circle. III. RING PARTITIONED MATCHING A. Basic idea Histogram is a simple descriptor for an image. Since, te istogram only counts te number of pixels for eac gray scale (or color) value, te spatial information is lost. Some different images will ave te same istograms, suc as sown in fig.. Suppose tere are two blac and wite images A and B, if we divide te images into four areas as sown in fig. 3, te above problem can be reduced. Figure. Two different images wit te same istogram. Figure. Pre-processing. Te traffic signs used in te experiment are limited to te circular sign wit red color border (include te no entry sign). Using tis limitation, tere are tree inds of colors in te signs, i.e. red, blue, and wite. Te color tresolding ten applied to te RGB image to extract te red, blue, wite, and oter colors of te image. After color tresolding, te gray scale image for eac color can be obtained. Since an image is segmented by four color tresolding, tere are four gray scale images corresponding to eac color. Using istogram specification tecnique [], we can create an image wit a particular istogram. By selecting te different gray scale ranges in eac image, finally we can combine four images into an image were gray scales range for red, blue, wite, and oter colors(blac color) are definitely separated. Anoter problem in te real sign image is te occlusion. Te sign often occluded by oter object suc as trees, poles, leaves, etc. Most of te occlusions are caused by trees and poles. Te trees appear wit te brown or green color. In tis case, te color tresolding is able to classify te trees as blac object properly, liewise te dar or blac poles. However, te wite or gray poles can not be solved by te color tresolding. Since tey are similar to te wite color, tey will ave te same appearance wit te wite color of te sign. Here, we assume tat te wite pole wic occlude te sign image is straigt pole and cover te image sign from upper side until lower side of te circle. By tis assumption, te wite pole can be separated from wite color in te sign bacground using te information from te red circle of te image. Sortly, if a pole occludes te sign image, te red color Image A Image B Figure 3. Image partition. Let A and B are istograms of image A and B, respectively. A, A, A3, A, B 3, B are istograms of sub-images A, A, A 3, A 3, respectively. Suppose tat, A (0) = p ; A () = m p ; A (0) = q ; A () = m q; A3 (0) = r ; A3 () = m r ; A (0) = s ; A () = m s; B (0) = q ; B () = m q ; B (0) = p ; B () = m p; B3 (0) = r ; B3 () = m r ; B (0) = s ; B () = m s; () were, * (0) = number of blac pixels, * () = number of wite pixels, m = total pixels in a sub - image, p, q, r, s : positive integers. Ten, te istograms of image A and B are : ( 0) = p + q + r + s; ( ) = m ( p + q + r + s) () A A (0) = q + p + r + s; () = m ( q + p + r + s) B B Te distance between A and B ( dist _ AB ) can be computed using te Euclidean s distance. In general, te distance between A and B in Euclidean space n R is given by n dist i i i = In te case of (), tis distance is given by _ AB = A B = ( A B ). (3)

3 dist _ AB = ( (0) (0)) + ( () Ai Bi Ai i = i = In te case of (), tis distance is given by Bi ()). () dist _ AB = ( (0) (0)) + ( () ()). (5) A B A B If we compute te distance between A and B ( dist _ AB ) using sub-images suc as in (), ten from () and () we obtain dist _ AB = ( p q) + ( p q) (6) = ( p q). Meanwile, if we compute te istogram distance between A and B ( dist _ AB ) using (), ten from () and (5) we obtain dist _ AB = 0. (7) From (6) and (7) (wit assumption tat p q ), we obtain tat two different images wit te same istograms can be recognized as two different images if we mae partition of te image and compute te istogram distance based on te partition. Te above idea can be extended to solve te occlusion problem. In te fig., image A is an image witout occlusion, image B is te anoter image, and image C is image A wit occlusion. Human can intepret easily tat image C is more similar to A tan B. Here, we will sow tat te partition metod sows better performance tan non-partition metod to interpret te occluded image C. Image A Image B Image C (a) (b) (c) Figure. Image partition in te occluded image. Let images in fig. be partitioned into four areas in te same way as fig. 3. By inspecting te pictures, we ave A (0) = p ; A (0) = q ; A3 (0) = r ; A (0) = s; B (0) = p ; B (0) = q ; B3 (0) = r ; B ( 0) = s β ; C (0) = p ; C ( 0) = q α ; C3 ( 0) = r ; C ( 0) = s α ; (8) were β = α ; p, q, r, s, α, β : positive integers. Te distance between image A and C (dist_ac_p) and distance between image B and C (dist_bc_p) for partitioned images are obtained as dist _ AC _ P = α, (9) dist _ BC _ P = α 5. (0) If te distances are calculated for non-partitioned images, te results are given as dist _ AC _ NP = α, () dist _ BC _ NP = α. () From (9)-(), if we compute te distance of image witout partition, te distance between image A and C is te same as te distance between B and C, or image C (image A wit a partial occlusion) can be recognized as image A or as image B. However, using partition metod, te distance between image A and C is lower tan te distance between image B and C, tus image C is more similar to image A tan to image B, similarly to te uman interpretation. Let us observe te general case to mae te partitioning effects clear. We can discuss it wit two partitions witout any loss of generality. Let A and B are te reference images and C is a target image. Te target image C is an occluded image of A. Suppose tat (0) = p; (0) = q; A A (0) = p + β ; (0) = q + β ; (3) B B (0) = p γ ; (0) = q γ ; C C were 0 p m; 0 q n; p β m p; q β n q; 0 < γ p; 0 < γ q; m: number of pixels of te st partition; n: number of pixels of te nd partition. Te distances between A and C and between B and C for partitioned images are obtained as dist _ AC _ P = ( γ + γ ), () dist _ BC _ P = ( β + γ ) + ( β + γ ). (5) For non-partitioned images, te distances are obtained as dist _ AC _ NP = ( γ + γ ), (6) dist _ BC _ NP = ( β + β + γ + γ ). (7) In te partitioned case, C is more similar to A tan B, if te next condition is satisfied. ( β + γ ) + ( β + γ ) > γ + γ. (8) In te non-partitioned case, te condition is ( β + β + γ + γ ) > ( γ + ) γ. (9) or ( β + γ ) + ( β + γ ) > γ + γ + γ γ (0) ( β + γ )( β + γ ). If β and β are positive, te inequalities in (8) and (9) are always satisfied. It means tat by using partitioned or non-partitioned metod, we can classify image C as te original image A properly. In te case were β and β are negative or ave te different signs, tere is no guarantee tat (8) and (9) are satisfied. However, (8) is always satisfied wen (9) olds, if te next condition is satisfied. γ γ ( β + γ )( β + γ ) > 0. () Namely if te above inequality olds, te partitioned case sows te better performance tan te non-partitioned case.

4 Let us examine te case were () olds. From (), we get te following inequality (it is assumed tat β 0 and β 0, for convenience). β γ β γ β < if β > γ ; β > if β < γ. β + γ β + γ () In te case of β = γ, () always olds. For given occlusion, namely γ and γ, te combinations of β and β satisfying () are sown in fig. 5. As sown in fig. 5, te area were () is satisfied is larger tan te area were () is not satisfied, if te occlusion ( γ, ) is not so large compared wit (p,q) of te original γ image. Tus te partitioned case sows te better performance tan te non-partitioned case in many cases. -p -γ β n-q -γ -q m-p β : area were bot te partitioned and te nonpartitioned cases conclude C is similar to A, : area were () is satisfied (te partitioned case is better tan te non-partitioned case), : : area were () is not satisfied (te non-partitioned case is better tan te partitioned case). Figure 5. Solution areas of (). Te effect of te partitioning will increase by partitioning te image into more tan two areas. Here, we use te ring (circle) partition, were an image is partitioned into several areas lie rings. An image is partitioned into -rings wit te same space/widt as sown in fig. 6. Te ring is numbered from te innermost ring. Te outermost ring actually doesn t form a ring. Since an image is rectangular, te outermost ring is used to mae a complete partition of an image. Comparing wit square partition suc as in fig. 3, te ring partition as te following advantages : - Since an image is partitioned into ring areas, it is invariant to te rotation. - Since te traffic sign images are circular signs, te inner parts of te ring partitions ave more important information tan te outer parts. Tus using ring partition, we can adjust te weigting factors easier rater tan square partition. Figure 6. Ring partition. B. Fuzzy istogram Te istogram provides te information about te dynamic range of gray distribution in an image. Te gray value n in a digital image can be (n+) or (n-) witout any appreciable cange in te visual perception. Suc imprecisions are not considered in te normal/usual istogram. Te fuzzy istogram considers te uncertainty arising out of te imprecision of gray levels of te image. Te fuzzy istogram is expressed as [0] : f( ) = ( ' ) µ ( ' ) ' µ (3) were ( ) is usual normalized istogram. µ (') is te resemblance degree of gray level to te gray level. In te case of symmetrical triangular membersip function, µ (') is expressed : ( ') = ' µ max 0, () α α is a positive real number. A fuzzy istogram provides a smooted istogram. By smooting te istogram, suc imprecisions of image caused by te natural of image or along te digital image processing can be reduced. Since in te digital image we wor wit te discrete level, te specified istogram obtained by te istogram specification tecnique is not exactly similar to te particular istogram. Te fuzzy istogram maes te specified istogram similar to tis particular istogram. C. Matcing strategy Matcing between target image and reference images basically finds te reference image wic as te igest similarity wit te target image or te lowest distance to te target image. Te distance or similarity is computed from te te extracted features of image called image descriptor. Te istogram is used as an image descriptor. Te normalized istogram for every ring is calculated and saved as an image descriptor. After discarding te occluded part, te new normalized istogram is saved as an image descriptor. Tis istogram is used in te istogram matcing by computing te istogram distance. Te euclidean s distance is used to calculate te distance between target and reference image. Te distance between target and reference image is calculated as: TR d = α ( TH RH ) α ( TH RH ). (5) were

5 TH i = istogram of target image of ring i, RH i = istogram of reference image of ring i, α i = weigting factor of ring i. α, α, α 3,..., α are weigting factors (between 0 and ) and denote te occlusion degree in every ring, for no occlusion and 0 for occlusion at all. Te reference image wit te lowest distance is selected as matced image. IV. EXPERIMENTS Te algoritm was evaluated to recognize te occluded, illumination cange, sadowed, oblique, and rotated circular traffic signs. Te tested images are taen from real scene using digital camera Te images are taen in te daytime on te sunny and cloudy weater. Since te classification or recognition stage is investigated, te real sign image is selected in te rectangle size from te real scene image by manually using image software tools. Te tested images consist of 80 images, some of tose images are sown in fig. 7. Te algoritm is implemented using MATLAB. Te figs. 8- sow some of te specified gray scales images obtained from te target images. In every figure, te upper side sows te usual gray scale image and its istogram, te lower side sows te specified gray scale image obtained by te proposed metod and its istogram. In fig. 8 and fig. 9, two sign images wit different illumination are sown. Te istograms of te usual gray scale images sow te difference in te range of gray scale. In fig. 9, were te image is darer tan fig. 8, te gray scale for red and blue color is closer compared wit te image in fig. 8. Meanwile, te proposed specified gray scale images ave te same range of gray scales. In te fig. 0, tere are leaf sadows in te sign image. Te proposed metod can reconstruct te image properly. Fig. sows te occluded image. Te occluded object (tree) can be detected and sown as blac object. Figure 9. Low brigtness image. Figure 0. Sadowed image. Figure. Occluded image. In te matcing process, six approaces are evaluated : istogram matcing witout partition, fuzzy istogram matcing witout partition, istogram matcing wit square partition, fuzzy istogram matcing wit square partition, istogram matcing wit ring partition, and fuzzy istogram matcing wit ring partition. Table sows te experiment results. Figure 7. Some of te tested sign images. Figure 8. Hig brigtness image. Table Matcing results. Metod Matcing rate Execution time No partition Usual istogram 9. % s Fuzzy istogram 6.7 % s Square partition (3x3) Usual istogram 7.8 % 0.07 s Fuzzy istogram 8.7 % 0.37 s Ring partition (7 rings) Usual istogram 6.8 % 0. s Fuzzy istogram 93.9 % 0.0 s

6 Te result sows te effectiveness of ring partitioned metod in te recognition of real sign images. Comparing wit te usual istogram, te fuzzy istogram sows te better performance. V. CONCLUSION In tis pa per, we propose a m etod to classify te traffic sign image. First te RGB image is pre-processed using color tresolding and istogram specification tecnique to mae a specified gray scale image. Te specified gray scale image is invariant to te illumination canges. Ten te ring partitioned metod, wic divide an image into several ring areas is used to matc te image by computing te istogram for every ring. It follows from te experiment tat te ring partitioned metod sows te best performance in te matcing of te occluded, rotated, sadowed, and illumination canges images, comparing wit non-partitioned and square partitioned metod. Te color tresolding and specified istogram tecnique wor effectively in te images wit varying illumination and sadows. Te fuzzy istogram tecnique affords te excelent result in te matcing process regarding to te te imprecision, uncertainty of te real digital image. From te experiment, te execution time is 0. second. Te execution time can be faster if te algoritm is written in C languange instead of MATLAB. Te proposed tecnique provides a simple algoritm, terefore te real time implementation can be acieved. Te problems wic may arise using tis proposed classification tecnique are segmentation of traffic sign according to te certain color and detecting te occluded object for various traffic sign images and occluded objects. Te proposed classification metod is part of a traffic sign recognition system. Te future wors will cover te detection stage of traffic signs and classification for anoter type of traffic signs, suc rectangular signs and triangular signs. Recognition in disturbing Environments, Proceedings of te Fourteent International Symposium on Metodologies for Intelligent Systems (ISMIS'03). Maebasi City, Japan, 8-3 October 003. [8] Seanina Luas, Torresen Jim. Detection of Norwegian Speed Limit Signs, Proc. of te 6t European Simulation Multiconference, Delft, NL, SCS, 00. [9] C.V. Jawaar and A.K.Ray. Fuzzy statistics of digital image, IEEE Signal Processing Letters,vol. 3(8), pp. 5-7, 996. [0] R. Zang and M. Zang. A Clustering-based approac to efficient image retrieval, Proc. Of te t IEEE Int l Conf. On Tool wit Artificial Intelligence, Wasington DC,USA, November 00. [] R.C. Gonzalez. Digital Image Processing, Addison-Wesley, 987. REFERENCES [] S.-H. Hsu, C.-L. Huang. Road sign detection and recognition using matcing pursuit metod, Image and Vision Computing, vol. 9, pp. 9-9,00. [] A. de la Escalera, L. Moreno, M.A. Salics, J. M. Armingol. Road Traffic Sign Detection and Classification, IEEE Transaction on Industrial Electronics, vol. (6), pp , 997. [3] A. de la Escalera, J.M Armingol, M. Mata. Traffic sign recognition and analysis for intelligent veicles, Image and Vision Com puting, vol., pp. 7-58, 003. [] D.M. Gavrila. Traffic sign Recognition Revisited, Proc. of te st DAGM Symposium fur Mustereennung, pp , Springer Verlag, 999. [5] J.C. Hsien and S.Y. Cen. Road Sign Detection and Recognition Using marov Model, Te t Worsop on OOTA, 003. [6] J. Miura, T. Kanda, and Y. Sirai. An Active Vision System for Real Time Traffic Sign Recognition, Proceeding 000 IEEE Int. Conf. On Intelligent Transportation Systems, pp. 5-57, Dearborn, MI, Oct [7] H.M. Yang, C.L. Liu, and S.M. Huang. Traffic Sign