Video Traffic Monitoring for Flow Optimization and Pollution Reduction

Transcription

1 Vieo Traffic Monitoring for Flow Optimization an Pollution Reuction P. Payeur, Y. Liu Abstract - Vehicles are wiely recognize as one the main sources of pollution in urban areas. In the context of the Kyoto protocol, means nee to be ientifie to reuce pollution emission without compromising economic activities. Computer vision technologies can avantageously be put to contribution in eveloping such solutions by implementing intelligent systems for traffic monitoring, management an optimization. Improving the flow of traffic an reucing roa congestion with more efficient synchronization of traffic lights can result in important reuction of carbon monoxie emissions as the uration of travels is reuce an constant spee cruise is favore. Most existing vision-base traffic monitoring systems suffer from a lack of robustness with respect to weather an lighting variations that make them unsuitable for reliable traffic light control. The present paper introuces a refine algorithm for real-time etection of moving vehicles approaching an intersection from sequences of color images. Efficient an robust vehicle etection uner various weather an illumination conitions is achieve by means of an aaptive backgroun representation an an original ualmoe feature space segmentation using HSV color mapping. Experimental results using real outoor traffic sequences emonstrate the system's robustness uner various an ifficult environmental conitions. I. INTRODUCTION Over the last ecae, the number of vehicles circulating on our roas has been continuously increasing. The attractive cost an efficiency of transportation by trucking in comparison with massive an restricte train an naval expeition massively contribute to this increase in traffic flow on highways an urban areas. Along with the popularity of sport utility vehicles, this resulte in important increases in carbon oxies emissions an ha a very negative impact on environment all aroun the worl [1]. Unfortunately, the technology of electric vehicles is still far from being available for wie commercialisation as power storage an practical istribution of sufficient power sources still remain important issues to be aresse [2]. Difficult climates such as those foun in Northern parts of Canaa an Europe also preempt this technology to be wiely use. Combustible fuel is therefore expecte to remain the main source of energy use in transportation over the next ecae. It is then necessary to evelop approaches to optimize traffic management in orer to protect environment. The aoption of the Kyoto protocol, some important variations in petrol cost an the increasing congestion of roas has initiate new research efforts in traffic monitoring, management an optimization. In urban areas, important improvements on traffic flow can be achieve by an optimal programming of traffic lights accoring to the current traffic flow rather than time of the ay. For now, this is partially achieve by means of magnetic loop etectors. Unfortunately, these loops require highly invasive installation techniques an o not allow an optimal control of traffic lights as their etection surface is limite to a few meters within the intersection area. Gray level or color image sensors are able to collect richer information about vehicles approaching an intersection as they monitor a wier area that allows to anticipate the arrival of cars at the intersection. This creates opportunities to implement higher level of intelligence in traffic light controllers such as aaptive an preictive control. Applications of computer vision in traffic monitoring usually rely on motion etection, tracking an feature matching. However, achieving reliable etection of moving objects in natural outoor environments remains a critical issue as the worl uner observation continuously changes with lighting an weather conitions. Most existing vision base vehicle etection systems suffer from a lack of robustness in ealing with these various natural conitions [3, 4]. In the literature, a large boy of vision research has been targete at motion etection uner fair weather situations an clear lighting only [5], while specific approaches have been evelope to operate uner precise weather conitions such as snowfall [6], heavy rain [7], or uner low lighting [8]. Even though several improvements have been mae on the etection of moving vehicles uner various conitions, most systems o not emonstrate sufficient robustness uner the vast variety of conitions uner which they must operate. More sophisticate techniques propose to use ifferent etection moes accoring to lighting conitions (ay/ nigh in orer to classify vehicles by types [9]. These systems generally succee to etect vehicles both at ay an uring the night but neglect other aspects such as the effect of shaows. This results in misclassification of vehicles in noisy conitions such as snowfall or highly reflective wet surfaces. The authors are with the Vision, Imaging, Vieo an Auio research laboratory, School of Information Technology an Engineering, University of Ottawa, Ottawa, Canaa, K1N 6N5. [ppayeur, yliu055]@site.uottawa.ca

2 Typically, the extraction of moving-object regions from outoor images is performe on gray scale images by backgroun subtraction [10] or spatio-temporal ifferentiation [11]. Backgroun subtraction implies a ynamic backgroun upating proceure [12, 13, 14, 15, 16]. Techniques using color as a cue to extract information from images have also been introuce. Since color images provie more information about the objects in the scene than gray level images, these algorithms combine motion an color segmentation to etect moving objects [17, 18, 19]. Motion segmentation is obtaine by frame ifferentiation while color segmentation consists of a splitan-merge algorithm that leas to an over segmentation of the image that makes objects stan out. In this paper, an approach is introuce to etect moving objects in outoor scenes with high robustness to environmental conitions. The propose approach uses color image sequences an allows to simultaneously etect moving objects in various parts of the scene. By means of these multiple areas of etection, specific information about cars that are reaching or approaching an intersection as well as their location on some specific lanes (e.g. turning lanes) can be provie in real-time to the traffic light controller. The algorithm operates on images encoe in the huesaturation-value (HSV) color space to automatically aapt the feature segmentation algorithm accoring to the current environmental conitions [18, 20]. The following sections etail the propose etection system an analyze the operation at various aytimes. Experimental results on real traffic sequences are presente to emonstrate the robustness of the propose scheme uner weather an lighting conitions foun in practice. II. PROPOSED APPROACH The perception contrast that exists in outoor images of traffic scenes appears to be a critical factor that influences the etection of moving objects as it etermines the features on which the etection nees to be base. For example, uner highly contrasting sunny conitions, the complete surface of moving vehicles can be extracte. However, at night, the low contrast only allows vehicles' healights to appear as clear features in the images. Therefore it seems suitable to evelop a etection strategy that groups environmental conitions in two categories. However, such a ual processing moe requires a proper moe selector to automatically switch between ifferent sets of features that are searche for. The propose approach illustrate in figure 1 relies on ual etection schemes that work in a complementary manner to extract appropriate features from moving regions in color images. The moe selector uses the HSV color mapping as it is known to be less sensitive to lighting variations. In orer to provie multiple sensing an control areas along the roa an to spee up processing, search winows are efine aroun areas of interest in the images collecte from a static viewpoint. The number an position of search winows are initially selecte by the operator at setting time. In the example shown in figure 2, two main areas of interest (soli lines) are efine to allow an anticipation of cars arrival at the intersection. Supplementary (otte lines) areas can easily be ae for specific irection lanes etection. Using multiple areas of etection enables programming of traffic lights to minimize ile perios for cars waiting at the intersection an to improve traffic flow thus reucing emissions. motion etection ual-moe feature etection 1st search winow setting Aaptive backgroun upate high contrast Shaows removal Rectangle matching Status of vehicle etection in 1st winow Sequence of RGB images Subimage selection Backgroun ifferentiation Ghosts removal Morphological filtering RGB - HSV mapping HSV moe selector 2n search winow setting Healights etection Circle pair matching low contrast Status of vehicle etection in 2n winow Fig. 1. Structure of the propose robust motion etector. Fig. 2. Multiple interest winows efinition.

3 III. MOTION DETECTION Sequentially processing each search winow, motion etection is achieve by computing a binary mask representing pixel-wise ifferences between a ynamically upate backgroun representation an the current color frame. Morphological filtering is applie to refine the efinition of moving areas before the backgroun moel is upate accoring to the latest binary mask to keep the track of environmental conitions. Ghost effects occurring when moving cars are present in the image at initialization or when stoppe cars start to move are also eliminate from the backgroun representation. A. Backgroun ifferentiation A binary mask putting in evience all moving objects is compute by comparing the current frame with an evolving backgroun image that represents static objects an allows to moel lighting an weather evolution. As color images appear to be less sensitive to illumination variations an inter-reflections than gray level ones, the principle of image binarization base on brightness level is extene to color images segmentation. The three color components of the current image, [ I R, I G, I B ], are compare with the previous backgroun image components [ BR, BG, BB ] to compute a temporal ifference image, Dt (), as follows: Dxyt (,, ) = I R ( xyt,, ) B R ( x, y, t 1) I G ( xyt,, ) B G ( xyt,, 1) I B ( xyt,, ) B B ( xyt,, 1) This ifference image is threshole at a given level, t D, in orer to extract only the pixels where a significant change in all color components has occurre. This results in a binary mask image, Mt (), where pixels that correspon to moving objects have a value equal to one, while all other pixels are set to zero: Mxyt (,, ) = 1 if D( xyt,, ) > t D (2) Mxyt (,, ) = 0 otherwise The threshol value, t D, is ynamically estimate from the histogram of the ifference image, Dt (), following an aaptation of the approach propose in [14]. Depening on environmental conitions, the sprea of the histogram of Dt () can vary significantly as shown in figure 3. Therefore, an aaptive threshol estimation technique has been esigne to automatically fin a proper value that retains only those pixels that exhibit a significant ifference between the current frame an the latest version of the backgroun image. t D = s if max( D( x, y, ) 10 max( D( x, y, ) t D ( 1) max( D( x, y, ) = if 0 < < (1) (3) a) b) t D(x,y, D(x,y, t Fig. 3. Histogram of D(x,y,: a) small number of changing pixels, b) uner wiely sprea illumination changes. where maxdxyt ( (,, )) is the maximum ifference value between the current frame an the backgroun image, an is the istance between the peak an the origin of the graph. When the ratio maxdxyt ( (,, )) is large, the histogram is steep an the peak tens to be closer to the origin as it correspons to situations where changes in illumination are limite to a small number of pixels (e.g. night operation). The parameter s has been experimentally set to 7 in orer to optimize performances uner all environmental conitions. B. Aaptive backgroun representation Vehicles motion generates faster changes in images than illumination an weather conitions. Representing the evolution of environmental factors over the time to allow reliable segmentation between fixe an moving components requires an aaptive backgroun mapping to be efine an frequently upate. To achieve this goal, a temporary backgroun image, TB() t, is compute from the binary mask image, Mt () obtaine previously. TB() t combines the previous backgroun pixels, Bt ( 1), for areas corresponing to moving objects with the current image pixels, It (), for static areas as follows: TB( x, y, = Mxyt (,, ) Ixyt (,, ) + (4) Mxyt (,, ) Bxyt (,, 1) where the binary mask, Mt (),is use as a selection factor. The upate backgroun image, Bt (), associate with the current frame is then compute as a weighte average of the temporary backgroun, TB() t, an the previous backgroun, Bt ( 1), as follows: Bxyt (,, ) = ( 1 α) Bxyt (,, 1) + α TB( x, y, (5) The weighting parameter, α, etermines the backgroun transition rate an is aaptively estimate to increase robustness accoring to the magnitue of illumination variations. As the steepness of the histogram of the ifference image, Dt (), mainly epens on illumination

4 a) initial backgroun b) after 3 sec. c) after 5 sec. ) after 7 sec. Fig. 4. Evolution of the aaptive backgroun representation. changes, α can be efine proportionally to the ratio of the istance between the peak an the origin of the histogram of Dt () an the maximum ifference value between the current frame an the previous backgroun image. A maximum weight of 0.5 has been set experimentally to prevent abrupt transitions in the backgroun moel. α = min , 0.5 (6) max( D( x, y, ) In practice, this results in an increase backgroun transition rate when illumination changes are important as the backgroun representation must then map faster variations in the static components of the scene. Figure 4 shows an example of a backgroun image upate with the propose approach over a perio of time of 7 sec. Here α has been set to a value of as illumination is quite constant uner clear weather conitions. C. Ghosts removal Given the fact that the backgroun image is progressively upate with some of the current frame information, a static object initially represente in the backgroun that starts to move (e.g. a stoppe car) requires a short perio of time before it faes out from the backgroun image. Therefore it influences movement etection over this perio of time as shown in figure 5b. These regions of the binary mask image that result from the temporary memory of the backgroun image are calle ghosts as they correspon to virtual objects that temporarily appear in the backgroun moel. These ghosts must be eliminate to preserve coherence in the binary mask that must map only real moving objects. A correction is applie by ifferentiating the current an the previous frames over all moving pixels (ientifie in the binary mask). Ghosts pixels have a small ifference between two successive frames since their motion is virtual an results from erroneous backgroun pixel values. Therefore a proper correction consists in eliminating these apparently moving pixels that have a small interframe ifference from the binary mask. Figure 5c illustrates such a correcte binary mask. IV. DUAL-MODE FEATURE MAPPING Refinement of the etecte moving objects is performe through a ual moe processing scheme that maps vehicles as rectangular patches that surroun the complete structure of the vehicle uner clear an highly contrasting aytime conitions. Uner night or lighter contrasting conitions such as rain, snow or fog, vehicles are mappe as groups of circular regions corresponing to their healights. An automatic switching approach has been evelope that relies on RGB color images converte to the Hue- Saturation-Value (HSV) color mapping. A. Moe selection The HSV mapping of color images is a nonlinear relationship of the RGB color scheme. Uner HSV mapping, it has been observe that color histograms of images collecte uner ifferent environmental conitions clearly iffer as shown in figure 6. The istribution of clusters of points in the HS histogram tens to move towars ifferent areas of the polar representation with changes in weather an illumination. On the other han, HS histograms of ifferent color vehicles collecte uner similar conitions remain rather similar one to each other. This behavior is exploite to automatically aapt the motion etection process for ifferent environmental conitions. Comparing the number of pixels in the image, NU, that are mappe to the upper half of the HS polar plot, with the number of pixels, NL, that are mappe to the lower half of the HS polar plot, the most appropriate etection moe is selecte. NU an NL are compute by an examination of the Hue angular value for each pixel in the search winow to etermine its location in the HS polar plot. The ratio of highlighte pixels in the image also reveals to be critical in the selection of the etection moe. Following the HSV color mapping scheme, highlighte pixels are characterize by low Saturation an high Value values. Inee, highlighte areas appear as colorless an bright regions in the image. A ratio of highlighte pixels, H, with respect to the total number of pixels containe in the search winow, W, is then efine an compute as: a) b) c) Fig. 5. Effect of ghost pixels on binary mask: a) initialization image, b) mask with ghost pixels, c) mask after ghost removal.

5 a) sunny ay b) rainy ay c) clear night ) clouy ay e) snow ay f) snow night Fig. 6. Hue-Saturation histograms of images uner ifferent environmental an contrast conitions. H ratio = ---- (7) W Combining the istribution of pixels on the HS histogram an the ratio of highlighte pixels, the selection between the full feature (FF) an the healights-base (HB) etection moes is efine as follows in pseuocoe: If NU < NL an ratio < τ Then moe=ff; Else if NU < NL an ratio >= τ Then moe=hb; Else Then moe=hb; A proper ratio threshol level, τ, has been experimentally estimate to be 2% of the pixels being highlighte. This value tens to slightly emphasize the application of the healights-base etection moe as it provies a clearer representation of vehicles, reuces the impact of occlusion problems between vehicles an is less sensitive to shaow effects. B. Shaows removal Shaows in images generate unwante changes in the aaptive backgroun an ten to be etecte as moving components [18, 21]. In the propose approach, this effect is eliminate by iscriminating shaows from moving objects taking avantage of the HSV color space mapping. Given the fact that the luminance of the cast shaow is lower than that of the backgroun while the chrominance of the shaow remains similar to that of the backgroun, a shaow mask, SM() t, is compute to extract pixels corresponing to shaow areas from the set of pixels initially tagge as moving ones in the binary mask, Mt (). SM( x, y, = 1 if SM( x, y, = 0 otherwise V I x y t (,, ) a < < b V B( x, y, H I( x, y, H B( x, y, S I( x, y, S B( x, y, (( ) < ε H ) (( ) < ε S ) (8) where 0<(a,b)<1 as shaow areas have lower luminance than the backgroun, an ( ε H, ε S ) are kept very small uner the assumption that the chrominance of shaow an nonshaow areas is similar. C. Healights etection Healights etection use in the low contrast operation moe is base on the fact that pixels of interest measure uner those conitions have low Saturation an high Value characteristics. Therefore, a healight mask, HM() t, of etecte healights pixels is compute by emphasizing these conitions: HM( x, y, = 1 if ( S I( x, y, < c) ( V I( x, y, > ) (9) HM( x, y, = 0 otherwise where c is set to 0.1 an equals 0.99 in our experimental work. V. EXPERIMENTAL RESULTS In the context of traffic monitoring at a roa intersection, a single camera is to be use per irection of traffic to provie information about as many observation winows are require. This may inclue ifferent lanes associate with turning signals, anticipative etection of vehicles approaching the intersection an eventually peestrians etection to immobilize traffic in specific irections. The approach has been evaluate on sequences of real traffic scenes uner various weather an illumination conitions for the etection of vehicles in multiple winows. The output is a count of vehicles occupying the area inclue in each observation winow that can be irectly interprete by a high level traffic light controller. Figure 7 shows a subset of traffic images uner ifferent environmental conitions on which the corresponing binary masks have been superimpose. The segmentation results using aaptive backgroun ifference an HSV color mapping were stable over a long perio of time an compare avantageously with manual segmentation of images for moving objects. The approach emonstrate excellent robustness to lighting an weather changes, succeeing to etect vehicles uner intense rain or snow conitions both uring the ay an at night. Shaow etection an ghost removal moules also showe very goo performances. The approach has been encoe using C++ language an Intel s CV library. A frame rate of 5 frames per secon can currently be achieve without coe optimization on a Pentium II 350 MHz processor. Interfacing the information provie by the propose etection mechanism with an intelligent preictive traffic light controller is currently uner investigation. It is expecte that smoother traffic flow will be achieve with reuce waiting times on re lights. The resulting improvements shoul lea to reuction of pollutants emission an waste of energy as well as to an increase efficiency in transportation.

6 a) sunny ay b) clouy ay c) rainy night ) snowy night Fig. 7. Images an corresponing binary masks uner various weather an lighting conitions. VI. CONCLUSION This paper introuce an approach for the monitoring of moving vehicles base on color image processing uner various uncontrolle weather an lighting conitions. The goal is to monitor traffic in various areas surrouning a roa intersection from a single visual sensor to improve traffic light programming by aaptive an preictive control. Optimize traffic flow can significantly reuce pollution emission an waste of energy in transportation. Taking avantage of the HSV color mapping scheme, the system automatically selects the most salient features uner given environmental conitions. The propose algorithm emonstrate robustness in etecting vehicles arriving at an intersection uner ifficult environmental conitions an in efficiently hanling shaow effects uner strong illumination. VII. REFERENCES [1] U.S. Dept. of Energy an Environmental Protection Agency, ( [2] Miller, J.M.; Emai, A.; Rajarathnam A.V.; Ehsani, M.: Current Status an Future Trens in More Electric Car Power Systems, in Proc. of the IEEE Vehicular Technology Conf., vol. 2, pp , Jul [3] Rowe, E.: The Los Angeles Automate Surveillance an Control (ATSAC) System, in IEEE Transactions on Vehicular Technology, vol. 40, no 1, pp , [4] Matsubara, M.; Tokutome, H.; Nishiyama, K.; Aoki, M.; Tanaka, K.: Development of a New Multi-Purpose Image Processing Vehicle Detector an its Implementation in the Tokyo Metropolitan Traffic Control System, in Proc. of the Vehicle Navigation an Information Systems Conference, pp , [5] Soto, A.; Cipriano, A.: Image Processing Applie to Real- Time Measurement of Traffic Flow, in Proc. of the IEEE Southeastern Symposium on System Theory, pp , [6] Hase, H.; Miyake, K.; Yonea, M.: Real-Time Snowfall Noise Elimination, in Proc. of the IEEE Int. Conf. on Image Processing, vol. 2, pp , [7] Kyo, S.; Koga, T.; Sakurai, K.; Okazaki, S.: A Robust Vehicle Detection an Tracking System for Wet Weather Conitions Using the IMAP-VISION Image Processing Boar, in Proc. of the IEEE/IEEJ/JSAI Int. Conf. on Intelligent Transportation Systems, pp , [8] Taktak, R.; Dufaut, M.; Husson, R.: Vehicle Detection at Night Using Image Processing an Pattern Recognition, in Proc. of the IEEE Int. Conf. on Image Processing, vol. 2, pp , Nov [9] Cucchiara, R.; Piccari, M.; Mello, P.: Image Analysis an Rule-Base Reasoning for a Traffic Monitoring System, in IEEE Transactions on Intelligent Transportation Systems, vol. 1, no. 2, pp , June [10]Ohta, N.: A Statistical Approach to Backgroun Subtraction for Surveillance Systems, in Proc. of the IEEE Int. Conf. on Computer Vision, vol. 2, pp , [11]Zhu, Z.; Yang, B.; Xu, G.; Shi, D.: A Real-Time Vision System for Automatic Traffic Monitoring Base on 2D Spatio- Temporal Images, in Proc. of the IEEE Workshop on Applications of Computer Vision, pp , Dec [12]Tsuchikawa, M.; Sato, A.; Koike, H.; Tomono, A.: A Moving-Object Extraction Metho Robust Against Illumination Level Changes for a Peestrian Counting System, In Proc. of the Int. Symposium on Computer Vision, pp , [13]Harville, M.; Goron, G.: Woofill, J.: Aaptive Vieo Backgroun Moeling Using Color an Depth, in Proc. of the IEEE Int. Conf. on Image Processing, vol. 3, pp , [14]Gupte, S.; Masou, O.; Martin, R.F.K.; Papanikolopoulos, N.P.: Detection an Classification of Vehicles, in IEEE Transactions on Intelligent Transportation Systems, vol. 3, no. 1, pp , March [15]Karmann, K.P.; von Brant, A.: Moving Object Recognition Using an Aaptive Backgroun Memory, in V. Cappellini (e.), Time-Varying Image Processing an Moving Object Recognition, 2, Elsevier, Amsteram, The Netherlans, [16]Haritaoglu, I.; Harwoo, D.; Davis, L.S.: W4: Real-Time Surveillance of People an Their Activities, in IEEE Transactions on Pattern Analysis an Machine Intelligence, vol. 22, no 8, pp , Aug [17]Zhang, C.; Siyal, M.Y.: A New Segmentation Technique for Classification of Moving Vehicles, in Proc. of the IEEE Vehicular Technology Conference, vol. 1, pp , Spring [18]Cucchiara, R.; Grana, C.; Piccari, M.; Pratti, A.; Sirotti, S.: Improving Shaow Suppression in Moving Object Detection with HSV Color Information, in Proc. of the IEEE Int. Conf. on Intelligent Transportation Systems, pp , [19]Dubuisson, M.-P.; Jain, A.K.: Object Contour Extraction Using Color an Motion, in Proc. of the IEEE Conference on Computer Vision an Pattern Recognition, pp , [20]Forsyth, D.A.; Ponce, J.: Computer Vision, A Moern Approach, Prentice Hall, [21]Mikic, I.; Cosman, P.C.; Kogut, G.T.; Trivei, M.M.: Moving Shaow an Object Detection in Traffic Scenes, in Proc. of the IEEE Int. Conf. on Pattern Recognition, vol. 1, pp , 2000.