Road Boundary Detection in Complex Urban Environment based on Low- Resolution Vision

Transcription

1 Road Boudary Detectio i Complex Urba Eviromet based o Low- Resolutio Visio Qighua We, Zehog Yag, Yixu Sog, Peifa Jia State Key Laboratory o Itelliget Techology ad Systems, Tsighua Natioal Laboratory for Iformatio Sciece ad Techology, Departmet of Computer Sciece ad Techology, Tsighua Uiversity, Beijig, 00084, Chia weqh06@mails.tsighua.edu.c, yagzehog@sia.com, sogyixu@sohu.com, dcsjpf@mail.tsighua.edu.c Abstract I this paper, we proposed a real-time road boudary detectio method i complex urba road eviromet. The detectio difficulty lies i road wear, both existece of marked ad umarked boudary ad low-resolutio visio. The idea of the algorithm is to extract the road surface firstly usig improved regio growig method based o edge ehacemet. The road boudary is the estimated by fittig the edges of the extracted road surface. A Bezier splies algorithm with optimizatio of cotrol poits is proposed to estimate the road boudary. The algorithm is implemeted o the video collected i BEIJING urba streets ad achieves good performace. Keywords: drivig assistace system, road boudary detectio, lae detectio, regio-growig, Bezier splie. Itroductio It is assumed that up to 90% car accidets are caused by driver faults []. So drivig assistace system which could work as exteded eyes to help the driver to perceive the blid area i the road ad as early warig to remid the driver the potetial dager had become a hot topic sice 990s. I the drivig assistace systems, road detectio is a key ad idispesable issue because road boudary defies the drivable safe-drivig area i both umarked ad marked road ad ofte performs as the prelimiary stage for further obstacle detectio ad trackig. Road boudary ad lae markig detectio are collectively called road boudary detectio for short i the rest of this paper. May sesors such as radar ad laser had bee used to perceive the road scee. But visio-based camera is thought ca capture more iformatio tha other sesors. So visio-based, either moocular visio or stereo visio, road boudary detectio has bee received a lot of attetio. The road boudary detectio i structured road had bee well-researched [][3]. The curret work maily focus o more complex urba eviromet i which the ifluece factors iclude wear, crossroads, the occlusio of vehicle, shadows, illumiatio effects, sigs o roads ad so o. The complex road coditio makes it difficult to desig a geeral detectio algorithm which ca deal with all of these problems. So some literatures deal with specific road scee istead [4]. Road boudary detectio geerally ca be divided ito two steps. The first step is segmetatio which gets the positio of lae i the image. Thresholdig color of

2 the image after filterig ad image trasformatio such as IPM is commo used but oly fits i clear road []. Clusterig ad classificatio method is ofte used i umarked road [7][8][0]. Some costraits are ofte assumed to limit the road coditio. Utilizig the priori kowledge is supposed useful i this step. The secod step is represetatio which is to state the true positio of the road boudary i the image. The simplest way is straight lie model usig Hough Trasform. Other models iclude hyperbola model [5], parabola model [6] ad clothoid model. This step may correct the error brought from the first step. I this paper, we proposed a method for typical urba road streets. For the reaso of eergy ad price cost, the lowresolutio camera is adopted to record the road scee. The difficulty of detectio i the road sceario icludes () road wear. () existece of both marked ad umarked boudary. (3) low-resolutio visio. A example of the road video as follows. the proposed method. I the ed, the coclusio ad future work are preseted.. Proposed method By aalyzig the target video, we ca fid edge-based method is ufeasible Because of the wearig boudary. The ot ecessary existece of boudary markig also make the algorithm relied o the color high-cotrast betwee the markig ad its surroudig ivalid. The existece of both marked ad umarked boudary makes the algorithm proposed should be fit to both coditios without assumig the either oe i advace. I aother aspect, the low-resolutio visio makes the algorithms relied heavily o thresholdig image color ivalid. But the advatage brought from low-resolutio image is to make the color of road surface seem cosistet, as show i Fig.. The proposed algorithm is based o the color cosistecy of the road surface. We first compute the road surface color ad its variace by assumig the road color obeys to Gaussia distributio, ad the extract the road surface usig a improved regio growig method with edge ehacemet. I the ed the edges of the road surface are fitted as the road boudary usig Bezier splie algorithm with optimizatio of cotrol poits... Preprocessig Fig. : Example of the road video The rest of the paper is orgaized as follows. I Part, we first aalyze the problem ad itroduce the basic priciple of the proposed method, the preset the method i detail. I Part 3, the experimet results of the algorithm ruig o differet road eviromets are preseted as well as some aalysis of the effectives of For the task of road boudary detectio, we do t eed deal with the whole regio of image. Usig the height of the camera from the groud ad its yaw agle ad pitch agle, we ca estimate the vaishig poit of the road approximately. To reduce computatio cosumptio ad to remove uecessary oise, the we determiate the regio of iterest (ROI) from the bottom of the image to the vaishig poit of the road for further road boudary detectio. For the examied

3 35*40 video, our ROI is the 35*50 area as i Fig Estimatig road surface color I the asphalt road, the road surface is approximately gray, ad the differece of its red, gree, blue compoet are cosidered less tha 5 [8]. The lowresolutio visio also makes the hue cotrast of differet object i the image ot clear. So it is eough to oly cope with its gray image for the sake of reducig computig time cost. The gray image is first smoothed by Gaussia filter. Assumig the itesity of the pixels of the road surface is subject to Gaussia distributio. We use ˆ ad ˆ represet the mea ad variace of the Gaussia distributio i frame ( x ) Px ( ) e () The ˆ ad ˆ are take as the road surface color ad its variace. Sice the road surface is just part of the image. We do t eed to compute the whole image. The chage betwee adjacet frames is slight, so we take the estimated road surface area of the frame - as the part for compute the Gaussia distributio i frame. For the first frame, we take the triagle area i the middle of the image which is the coservative estimate of the road surface for the most frames as the iitial road surface as illustrated i Fig.. The ad ca be computed by the maximum likelihood estimate procedure. Which yield ˆ Ii (, j) m (, i j) () ˆ ( Ii (, j) ˆ ) m (, i j) Where is the mask area defied by Figure 3, ad m is the total pixel umber of the area. I is the itesity of the pixels..3. Extractig road surface area.3.. Ehacig road edge Ehacig image edge, icludig road edge, is a critical step i the proposed method. Two kid of edge are extracted: the lie edge ad o-lie edge. For the lie edge, the Probabilistic Hough Trasform is used. For the o-lie edge, the firstorder Sobel operator is employed. We oly ehace the edge of the o-road surface area. The two edges are added to the origial image. The aim of ehacig road edge is to make the wearig boudary more preset ad visible, so as to extract the road surface correctly i the Road surface extractig module. A example comparig the detectio result with ad without ehacig road edge step as follows. Fig. 3 Top: without the ehacig road edge. Bottom: with the ehacig road edge step Fig. Up: the mask of computig area for the first frame. Bottom: the road surface area of the frame - as the mask of computig area for frame.3.. Road surface extractio After ehacig the edge, the regio growig method is employed to extractig the road surface. Regio growig method groups the pixels which have the 3

4 same property to form a coective regio, startig from a seed poit. There are two iitial seed poits, a ad b, which adaptively selected i the middle top ad bottom of the image respectively. a ad b satisfy ˆ ˆ ab, ˆ ˆ, otherwise, the earest eighbor pixels aroud a ad b which satisfy the coditio to replace them. The two coective regios startig from these two seeds mix together as the road surface. There are two criteria work as the ed coditio of regio growig. They are Iseed Ieighbor (3) I ˆ ˆ eighbor Where I eighbor is the itesity of the adjacet pixel of the seed poit, is the differece limit of the adjacet pixel pair. ˆ, ˆ are the road surface color ad its variace determied i the.. step..4. Estimatig road boudary The previous steps have give us the cadidate road surface, which firstly deoised by morphological operatios i this step. The third degree Bezier splie usig Berstei polyomials [9] is the used to fit the left ad right edge of the road surface, which is supposed as the true road boudaries. The advatage usig Bezier splie lies o it ca fit arbitrary shape of the curve with eough cotrol poits. It is icomplete to assume the predefied model [5] of the road boudary i the ustructured road such as urba or campus streets. The third degree Bezier splie is defied by four cotrol poits ad is a cubic polyomial as P( t) ( t) P 3 t( t) P 3 t ( t) P 6t P P P t t t P 0 0 0P3 (,,,) (4) Where is to cotrol the fittig fieess. P0, P, P, P3 are the cotrol poits. P(0) P0, P() P3, the iterior poits P, P cotrol the shape of the splie. We fit the Bezier splies usig leastsquared error method. For the left boudary, give the poit series sampled from the left edge of the road surfacep0, P,, P, the umber of poits determied by the samplig precisio. Let the cotrol poit P0 set as the poit p 0, the cotrol poit P set as the poit p. Oly the two iterior poits P, P is variable poits, they are selected i the rest edge poits P,, P by miimizig the error betwee the origial boudaries with the Bezier splie fitted by P 0, P, P, P 3. Here the origial boudary deotes the left edge of the road surface. The error is measured by the area formed by the origial boudary ad the Bezier splie as show i Fig.4. To avoid time-cosumig itegral calculus, it approximately estimated by coutig the pixels betwee the origial ad fitted boudary. The poit series p,,, 0 p,0, i pj p i j with the least pixel cout by miimize (5) are take as the optimal cotrol poits. A(, i j) Cout( i ) (5) i Where is the whole area betwee the origial boudary ad the fitted splie R. p 0 R p i p p j Road surface Fig. 4: Bezier splie fittig algorithm 4

5 3. Experimet ad aalysis The algorithm is ru o three video clips which are collected from BEIJING Fig. 6: Detectio samples show robustess i wearig boudary, umarked road, marked road, i presece of vehicle ad curves Fig. 5: Result of road boudary detectio i a straight road. From up to bottom: ROI with detectio result, gray image after smoothig, o-lie image edge, lie image edge, gray image after ehacig image edge, the mask of road surface urba streets. The Clip # sceario is the straight road with wearig ad umarked boudary, the Clip # is the umarked curve road, ad the Clip #3 is the marked sceario with crowd vehicle. Fig. 6 shows the robustess of the algorithm i the supposed road sceario. Fig. 7 shows some false detectio. I two coditios, the false detectio occurs. They are () a majority of the road boudary is occluded by vehicles or pedestria, () a majority of the road surface is occupied by illumiatio area. For the first case, however, it is reasoable to take the outside of the vehicle as the boudary because the true boudary is ivisible ad the area occupied by the vehicle is ot drivable area. For the secod case, a improved regio growig algorithm which is ivariat to shadig ad highlight effects usig the Mixture of Pricipal Compoets algorithm [] ca be itroduced to tackle this problem, this is part of our future work. Fig. 7: Examples of false detectio Table : Average computatioal time Test Module Clip# Clip# Clip#3 Average time per frame (ms) This algorithm was implemeted i C++ usig the ope source OpeCV library o PC with Itel P4 CPU 3.0 GHz 5

6 ad GB memory. For the examied 35*50 video, the average computatioal time per frame is less tha 5 ms. Table lists the average computatioal times for the three test video clips. The time cost is proportioal to the road surface area. 4. Coclusio ad future work I this paper, a road boudary detectio solutio is proposed for the complex urba eviromet characterized by wearig edge ad both marked ad umarked boudary o low-resolutio video with moocular. The experimet o the real road video shows the robustess of the method. The future work iclude: () Advaced regio growig algorithm cosiderig the illumiate effect will be itroduce to remove illumiate effect. () Sice this method oly detects the road boudary, but the dashed lae markig, if ay, i the middle of the road surface is useful to road uderstadig, the lae markig detectio uder kow road boudary is also a future work. (3) The o-road area betwee the detected left ad road boudary ca be classified as obstacles (road marker, vehicles, pedestrias etc). The further classificatio ad trackig of this o-road area will be a useful work for road uderstadig. 5. Ackowledge This work is supported by The Natioal High Techology Research ad Developmet Program of Chia. 6. Refereces [] Mohamed Aly, Real time Detectio of Lae Markers i Urba Streets, /publicatios /aly08realtime.pdf [] K. Kluge, Extractig Road Curvature ad Orietatio From Image Edge Poits Without Perceptual Groupig Ito Features, Proceedigs of the Itelliget Vehicles Symposium, pp 09-4, 994. [3] C. Kreucher, S. Lakshmaa, A frequecy domai approach to lae detectio i roadway images, Proceedigs of the Iteratioal Coferece o Image Processig 999, Volume, pp. 3-35, 999. [4] Hsu-Yug Cheg, Bor-She Jeg. Lae Detectio With Movig Vehicles i the Traffic Scees, IEEE Trasactios o itelliget trasportatio systems, VOL. 7, NO. 4, pp.57-58, 006 [5] W. N. Zhu, Lae Detectio i Some Complex Coditios, Proceedigs of the 006 IEEE/RSJ Iteratioal Coferece o Itelliget Robots ad Systems, pp.7, 005. [6] K. Y. Chiu ad S. F. Li, Lae Detectio usig Color-Based Segmetatio, Itelliget Vehicles Symposium, 005, pp.706 7, 005 [7] J. D. Crisma ad C. E. Thorpe, SCARF: A color visio system that tracks roads ad itersectios, IEEE Tras. Robot. Automatio, vol. 9, pp.49 58, 993. [8] J. Crisma ad C. E. Thorpe, UN- SCARF: A color visio system for the detectio of ustructure roads, Proc. IEEE It. Cof. Robotics ad Automatio, pp ,99 [9] David Solomo, Curves ad Surfaces for Computer Graphics, Spriger, 006. [0] Ola Ramstrom ad Herik Christese, A method for followig of umarked Roads, Itelliget Vehicles 05, pp , 005. [] Slawo Wesolkowski ad Paul Fieguth, Color Image Segmetatio Usig a Regio Growig Method, ts/oral/wesolkowski_fieguth.doc 6

7 7