Learning-based License Plate Detection on Edge Features

Transcription

1 Learnng-based Lcense Plate Detecton on Edge Features Wng Teng Ho, Woo Hen Yap, Yong Haur Tay Computer Vson and Intellgent Systems (CVIS) Group Unverst Tunku Abdul Rahman, Malaysa Abstract Ths paper presents Adaboost learnng-based method for lcense plate detecton n unconstraned envronment (cluttered scenes, changng llumnaton, n-plane and out-plane rotaton of lcense plates). Our approach s motvated by the dea that learnng-based method can mplctly derve a robust object model through tranng usng large set of postve and negatve samples. In addton, edge rather than ntensty nformaton s used to tran lcense plate detector (LPD) snce edge nformaton usng canny edge detector has shown better representaton than ntensty for lcense plate problem. We present comparatve results of our approach aganst ntensty, selecton of dfferent number of stages as well as our LPD detecton speed. Our approach acheves true postve rate of ~70%, wth detecton speed ~80 ms for mage sze of 320 x 240. Key words: lcense plate detecton, machne learnng, Adaboost, canny edge detecton 1. Introducton As the number of vehcles s growng, lcense plate detecton s becomng more mportant. It can be appled to applcatons such as traffc control, securty system, automated vehcle verfcaton, car park payment system and etc. A lcense plate detector (LPD) locates the poston of the lcense plate from a gven mage. It s challengng to detect a lcense plate from a cluttered background, and mage wth a lot of noses such as llumnaton, rotaton, and etc. The rest of the paper s presented n the followng structure. Secton 2 presents background nformaton related to lcense plate detecton. The archtecture of our algorthm s ntroduced n secton 3, and secton 4 s ntroduces Haar-lke features wth AdaBoost learnng algorthm. Expermental results and dscusson are presented n secton 9. We conclude ths paper n secton Background Vola et al. [3] presented a framework for face detecton that acheves hgh detecton rate and yet wth extremely rapd mage processng. Motvated by [1], they ntroduced a new mage representaton known as the ntegral mage that allows the features used n the detecton to be calculated very rapdly. They use the Haar-lke features to classfy the patterns for an mage, and the Haar-lke features used n ther framework are remnscent of Haar wavelet used by Papageorgou and Poggo [1].Then, they used AdaBoost learnng algorthm to select a smple Haar-lke features from the over-complete features set. Each feature s known as a weak classfer, and the weak classfers wll be combned to become a strong classfer. Chen and Yulle [4] demonstrated an algorthm for detectng text n natural mages, also based on AdaBoost. They clamed that the set of features used for face detecton by Vola and Jones [3] mght not be sutable for detectng text. Ths s because there s less spatal smlarty for the text compare to face; a face can be regarded as spatal smlar object snce t conssts of facal features such as eyes, nose, and mouth that are approxmately the same spatal poston for any face. Some of the algorthms are desgned specfcally for a partcular object detecton problem such as the adaptve algorthm for text detecton from natural scenes by Gao and Yang [5]. They developed a prototype system that can recognze Chnese sgn nputs. The algorthm s desgned n herarchcal structure wth dfferent condtons regulated n each layer. The algorthm can be appled to other languages text by modfyng the layout constrants. Nevertheless, t s dffcult to desgn an algorthm that can be used to detect dfferent object wthout changng the archtecture of the system. Shapro et al. [6] have presented mage-based car lcense plate recognton (CLPR) system. The system conssts of few processes such as the lcense plate localzaton that s used to locate the lcense plate for an mage. They use heurstc scheme for estmatng the plate s vertcal boundares whch are requred durng the vertcal projecton. In the system, they have assume that the plates are orented horzontally and are characterze wth frequent ntensty alteratons between the characters foreground and the

2 plate s background. Wth heurstc scheme, LPD arguably s less tolerant than learnng-based approach when t comes to varyng condtons (llumnatons, scales, sgnfcant blur, occluded lcense plate, etc). 3. System Archtecture We developed a framework that conssts of two man components,.e. the tranng and the testng stage. We tran the LPD usng Adaboost learnng algorthm to select the best feature as the weak classfer and combne all the selected classfers n cascade as shown n Fgure 1. Our lcense plate detecton archtecture has few stages; the ntal step s to get the orgnal mage from the user and the system converts the mage to gray-scale format. After convertng the mage, the next stage s to pre-process the gray-scaled mage usng an edge flter. We mplement canny edge detecton n our mage preprocessng. After usng canny edge detecton to flter the noses from the mage, we use the traned classfer to determne whether the gven regon s lcense plates or not. Orgnal Image Convert to gray-scale mage canny edge detector n our lcense plate detecton. Canny edge detector s one of the most popular edge detecton methods, because t provdes optmal detecton wth no false detecton, better localzaton wth mnmum dfference wthn the actual edge poston and the detected edge. It s capable of havng a sngle response to remove multple responses to a sngle edge [8]. 4. Haar-lke Features Rather than operatng on pxels drectly, lcense plate detecton classfers may act on smple features. Two motvatons of usng smple features are (1) features often contan salent doman-knowledge nformaton than pxels essental for learnng and (2) system operates much faster wth features than pxels. In ther paper [3], Vola et al. proposed to use smple Haar-lke bass functons as features. In general, these Haar-lke bass functons are smple 2D wavelet constructs consstng of at least two nonoverlappng rectangular regons, depcted as whte or black. Feature can be computed from the subtracton of pxels summaton wthn the black regon and from the pxels summaton wthn the whte regon. In our work, we use three types of features as orgnally proposed by Vola et al. as shown n Fgure 2. Canny Edge-detecton Lcense Plate Detector Result Fg 1. LPD System Archtecture 3.1 Canny Edge Detecton Although the archtecture we mplement s a learnng based algorthm, we also pre-process the orgnal mage before the detecton. Also we use canny edge detector to further process the tranng samples, so we wll test our system usng edge detected mages. Edge detecton s one of the most mportant processes n mage analyss [7]. An edge represents the boundary of an object whch can be used to dentfy the shapes and area of the partcular object. When there s contrast dfference between the object and the background, then after applyng edge detecton, the object edges wll be more obvous. In our lcense plate detecton archtecture, applyng canny edge detector can mprove the detecton rate snce t wll remove some of the noses and make the lcense plate text edges more vsble. There are many edge detecton methods; we are mplementng the Fg 2. Smple Haar-lke bass functons used as features n lcense plate detecton scheme. 5. Lcense Plate Learnng wth Adaboost Adaboost stands for adaptve boostng, a machne learnng technque ntroduced by Y. Freund et al. [9]. It s used prmarly to boost the classfcaton performance of a smple algorthm (for e.g. a smple perceptron) by combnng collecton of weak classfers to form one stronger classfer. A weak classfer means any smple classfer that delvers performance slghtly better than chance and preferably wth low computaton tme requrement. Y. Freund et al [9] dscovered that a commttee of weak classfers when combned properly often outperforms strong classfers such as Support Vector Machnes (SVM) and Neural Networks. Boostng algorthm comes wth many varants such as Dscrete Adaboost, Real Adaboost, and Gentle Adaboost [10].

3 Haar-lke features are overcomplete n certan sense because for an assocated wndow, the number of rectangle features s far larger than the number of pxels, for nstance a 24x24 wndow has 45,936 possble Haar-lke rectangular features [3] compared to 576 (24x24) pxels. Even though these rectangle features can be calculated effcently usng ntegral mage, Vola et al. [3] postulated that only small number of these features can be combned to form one good classfer. The challenge now s to fnd these features. In our lcense plate detecton system, we used Gentle Adaboost to tran strong classfer. Gentle Adaboost s chosen because t outperforms other varants n an emprcal analyss carred out by A. Kuranov et al. [11]. Note that, boostng algorthms only dffer n the procedure on how to re-weght tranng examples after the tranng teraton. To boost the performance of a strong classfer, AdaBoost algorthm search over a pool of weak classfers to fnd one wth the lowest classfcaton error for the subsequent combnaton. Ths learnng method s also known as greedy feature selecton process. Whle tranng a classfer, t s called upon to classfy tranng examples so that these examples can be re-weghted to emphasze those whch were ncorrectly classfed for the next tranng teraton. After tranng, we have a weghted combnaton of weak classfers n the form of perceptrons and a smple bnary threshold value determned automatcally. Every weak classfer has an assocated weght where good classfers assgned larger weght whle poorer classfers assgned smaller weght. Fgure 3 shows the learnng algorthm [3]. Gven example mage ( x 1, y 1),, ( x n, y n ), where y = 0, 1 for negatve and postve examples respectvely. Intalse weghts 1 1 w 1, =, for y = 0, 1 2 m 2l respectvely, where m and l are the number of negatve and postve respectvely. For t = 1,, T : 1. For each feature, j, choose a classfer, h t by mnmzng a weghted squared error ε = w h ( x ) y j j 2. Update the classfer F( x) F( x) + ht ( x) 3. Update weghts by w w e Fnal strong classfer s T F( x) = sgn α h ( x) + b = 1 y h t ( x ) Fg 3. Gentle Adaboost procedure used to construct a strong classfer. T s the number of weak classfers. The fnal strong classfer s a weghted lnear combnaton of the T weak classfers h (x) wth based offset b. 6. Tranng a Cascade of Strong Classfers A cascade of classfers can be constructed n subsequent stages wth Adaboost algorthm to acheve hgh detecton rate whle radcally reducng computaton tme. Cascaded classfers can acheve fast detecton speed because n the ntal stage of cascadng classfers, majorty of non-lcense plate wndows are quckly rejected whle almost all postve wndows are detected. Ths mechansm s effectve n lowerng false postves because wthn the mage, majorty of the wndows are negatve. In subsequent stages, complex classfers are only called upon on to focus ther attenton on small fractons of canddate wndows. The overall detecton process can be depcted as a degenerated decson tree. See Fgure 4. Fg 4. Schematc dagram of a 4 stages cascaded classfers. Every processng node s a strong classfer. Sub-wndows wthn the mage are fltered by the processng nodes. Intally, large number of negatve wndows s rejected wth very small processng tme. Subsequent nodes elmnate addtonal negatve wndows wth addtonal processng tme. After several stages, only small amount of canddate wndows are consdered by complex classfers for fnal decson. The constructon of cascaded classfers s drven by a set of detecton and performance goals. In our experment, stage classfer was traned to acheve hgh ht rates of for a frontal lcense plate patterns and very low false postve rates of 0.5; a total of 16 stages of cascaded classfers were traned. Theoretcally, our cascaded classfers can obtan optmum performance at false postve rates about and ht rates of about For detaled dscusson on

4 how to determne performance rates gven cascaded classfers, readers can refer to [3]. In the prevous secton, Adaboost procedure attempts only to mnmse errors, but not desgned to optmse performance tradeoffs to obtan hghest ht rates at the lowest possble false postve rates. One smple scheme to trade off performance over error rates s to adjust threshold of the classfer created by Adaboost. Gven a stage classfer denoted below T F( x) = sgnα h ( x t ) + b wth b = 0 = 1 Any stage classfer can be post-optmsed for a gven ht rate. The free parameters are t : threshold, b: offset, whle must be chosen accordng to the Adaboost loss functon to preserve the propertes of Adaboost [12]. Parameters t and b are selected n a gradent-descent manner by slowly ncreasng t value whle ensurng performance does not degrade. However, true gradent cannot be mplemented snce F(x) s not contnuous. 7. Detecton Archtecture Our LPD runs detecton wndow across the mage at multple scales and locatons. Detecton wndow s scaled at 1.1, means that wndow sze ncreased at 10% rate between subsequent scans, startng wth mnmum sze of The features can be easly determned by scalng the base wndow features by current scale factor, ths operaton can be done at any scales wth the same cost. LPD also runs across dfferent locatons. Subsequent locatons are shfted n some number of pxels,. The shftng amount s dependent on the scale of the detector, for nstance f the current scale s s, then the wndow s shfted by round ( s ), where round s the round up to the nearest nteger operaton. Two ntegral mages are ndependently determned from the gven mage and they correspond to ntensty ntegral, and squared ntensty ntegral, 2 respectvely. Recall that, ntegral mages are used rather than pxel ntensty because features can be evaluated very rapdly, sometmes n an order of hundreds tmes faster. The detecton wndow may proceed only f ts sze s smaller than the maxmum wndow sze, maxwnsze, whch can be computed as (wdth-10) x (heght x10). For nstance, f the mage sze s 720x486, maxwnsze would be 710x476. Detecton wndow must always be smaller than the mage sze to avod out of boundary scannng. The output of the fnal detecton often contans multple lcense plates detected around each lcense plate snce LPD s nsenstve to small translatons and/or scale changes. These extraneous lcense plates nstances can be categorsed as one knd of false postve. In practse, multple detectons are combned n a smple manner to return the fnal detecton. To do ths, overlappng wndows must be recognsed accurately before combnng them nto a sngle detecton. In certan cases, ths combnaton scheme decreases number false postve rates when many wndows overlapped. 8. Experments We prepared 2000 artfcal lcense plates as our postve samples, and 4000 non-lcense plate as our negatve samples. One of the man reasons of usng artfcal lcense plates s because the real world lcense plate databases are smply not suffcent. We rescaled the lcense plates to resoluton sze of 40 x 10, and added some noses, brghtness effects, nplane and out-of-plane rotaton, and etc. In our work, we conducted two experments, n the frst experment orgnal artfcal lcense plate mages are used as tranng samples, and n the second experment, edge-detected mages are used as tranng samples. We used canny edge detector to pre-process our orgnal artfcal lcense plates nto edged mages, Fgure 8 shows the orgnal artfcal lcense plates and Fgure 9 shows the edged mages of our tranng samples. Durng the AdaBoost tranng, we set the targeted number of cascade classfers to 15 stages and both of the experments are usng the same settngs so that we can compare the results farly. Fg 8. Artfcal Lcense Plate wthout usng canny edge detecton. Fg 9. Artfcal lcense plate usng canny edge detecton. To compare the performance, both LPDs traned wth and wthout edge-detected tranng mages are tested on our test sets. Our test sets consst of 83 mages collected from real-world stuaton varyng

5 llumnaton, n-plane and out-of-plane rotatons, dfferent lcense plate confguratons (fonts, spatal arrangement, and etc). For each test mages, we manually labeled the ground-truth locaton of lcense plates so that we can quantfy the LPD performance by judgng how much ts output devate from the expected ground truth poston and szes, as shown n Fgure 10. whle non-edge traned LPD only managed to acheve TPR of 0.1. Another nterestng observaton s that LPD traned wth more stages sometmes deterorates than mprovng TPR, as evdent n Fgure 13, where 14 stages LPD consstently outperformed LPD of 17 stages when FAR s fxed around at 0.5 and above. One possble explanaton s longer cascade classfers behave more strctly than ts shorter counterpart due to the longer flterng ppelne. Ths behavour may cause correct lcense plate area rejected at these extraneous traned stages thus leadng to lower TPR. Some LPD detecton results on our test mages are shown n Fg. 14. Our LPD costs ~ 80 msec to process an mage sze of 320 x 240, wth AMD Athlon 64x2 Dual Core Processor 2.01GHz, 2GB RAM. Fg 10. Some mages from our test set. 9. Results and Dscusson Fg 11. Top-left: Intensty mage. Bottom-left: Lcense plate n ntensty mage. Top-rght: Edgedetected mage (Canny). Bottom-rght: Lcense plate n edge detected mage. Lcense plate area has large denstes of vertcally and horzontally connected edges (See Fgure 11). Whereas, non-lcense plate area are nose found durng edge detecton process. Unlke lcense plate area, usually non-lcense plate areas have random drectons. Ths fndng mples that edge nformaton contans salent nformaton mportant to dstngush the lcense plate from the non-lcense plate. Such fndng agrees wth the prevous method found n [6]. As llustrated n Fgure 12, true postve rate (TPR) ndcates fracton of test mages that acheves correct detecton wth regards to postons and scales, whereas false alarm rate (FAR) ndcates a fracton of area ncorrectly dentfed as lcense-plate. Evdently, LPD traned wth edge-nformaton greatly outperformed LPD traned wthout edge nformaton. For nstance, from the Fgure 12, wth FAR fxed at ~ 1.0, edge-traned LPD obtaned TPR near to 0.7 Fg 12. Performance comparson of LPDs. Edgetraned LPD conssts of 14 stages cascaded classfers. Gray traned LPD conssts of 8 stages of cascaded classfers. Fg 13. Performance comparson of LPD wth dfferent number of stages traned on canny edgedetected mages.

6 [2]. Papageorgou, C. P., & Poggo, T. (2000). A Tranable System for Object Detecton. Internatonal Journal of Computer Vson 38(0), [3]. Vola, P. & Jones, M. (2004). Robust Real-Tme Face Detecton. Internatonal Journal of Computer Vson 57(2), [4]. Chen, X.R., & Yulle, A.L. (2004). Detectng and Readng Text n Natural Scenes. Proceedngs of the 2004 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04). Fg 14 Detecton results for Canny edge-traned LPD. 10. Concluson and Future Works In ths paper, we have proposed Adaboost learnng based method to construct LPD on Haar-lke features. Rather than ntensty mage, we proposed to learn Haar-lke features on edge nformaton. Edge nformaton has demonstrated better dscrmnatve power compared to ntensty nformaton snce lcense plate area s hghly responsve to horzontal and vertcal edges. Our experments have shown that our approach acheves sgnfcant mprovements n accuracy over ntensty approach. The total tme costs only ~ 80 ms to process an mage of 320 x 240. However, our approach s n ts prelmnary development, further mprovement s necessary to reduce false alarm rate as well as ncreasng true postves. One possble approach s to vsualze features selected by Adaboost learnng to get ntutve dea whether Haar-lke features s adequate n lcense plate detecton problems. Another possble approach s to nvestgate whether alternatve features are more approprate for learnng lcense plate. 11. Acknowledgements Ths research s partly funded by Malaysan MOSTI ScenceFund SF SF0019. [5]. Gao, J., & Yang, J. (2001). An Adaptve Algorthm for Text Detecton from Natural Scenes. Proceedngs of the 2001 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR 04). [6]. Shapro, V., Gluhchev, G., & Dmov, D. (2006). Towards a Multnatonal Car Lcense Plate Recognton System. Machne Vson and Applcatons (2006), 17: [7]. Parker, J.R. (1997). Algorthms for Image Processng and Computer Vson. Wley Computer Publshng, John Wley & Sons, Inc, Professonal, Reference and Trade Group, Unted States of Amerca. [8]. Nxon, M., & Aguado, A. (2002). Feature Extracton & Image Processng. (Frst edton).brtsh Lbrary Catalogung n Publshng Data. [9]. Freund, Y., Schapre, R. E. (1995). A decsontheoretc generalzaton of on-lne learnng and applcaton to boostng. Computatonal Learnng Theory: Eurocolt 95. Sprnger-Verlag. pp [10]. Freund, Y., Schapre, R. E. (1996) Experments wth a new boostng algorthm. Machne Learnng: Proceedngs of the Thrteenth Internatonal Conference. San Francsco: Morgan Kaufman. pp [11]. Kuranov, A., Lenhart, R., et al. (2003) Emprcal analyss of detecton cascades of boosted classfers for rapd object detecton. In: Lecture Notes n Computer Scence. Hedelberg: Sprnger-Verlag [12]. Lenhart, R., Maydt, J. (2002) An extended set of Haar-lke features for rapd object detecton. IEEE Internatonal Conference on Image Processng. September, pp References [1]. Papageorgou, C. P., & Poggo, T. (1999). A Tranable Object Detecton System: Car Detecton n Statc Image, A.I. Memo No 1673, C.B.C.L Paper No 180. Massachusetts Insttute of Technology.