An Efficient Vehicle Queue Detection System Based on Image Processing

Transcription

1 An Efficient Vehicle Queue Detection System Based on Image Processing Michele Zanin ITC-irst Povo, Trento, Italy Abstract This paper describes a method for the real-time measurement of vehicle queue parameters in a video-based traffic monitoring experimental system. The method proposed here is based on vehicle presence detection and movement analysis in video sequences acquired by a stationary camera. Queues are detected and characterized by a severity index. Intensive experiments show the robustness of the method under varying illumination and weather conditions. The system is presently undergoing an on-field testing phase in a double ways road near Trento, Italy, where queues frequently occur. 1. Introduction The growth in traffic congestion has been recognized as a serious problem in many urban and suburban areas, with significant effect on the accident risk, economy, pollution and discomfort for millions of road users. The automatic measurement of traffic parameters, and among those the queue presence and duration, can be a great help to traffic engineers providing a quantitative evaluation of the traffic situation on the roadway net. Previous works about queue detection using image processing are mainly focused on measuring queue length at intersections where queues are caused by a traffic lights system [7, 12, 9]. Fathy and Siyal approach [7, 12] is based on computing motion and presence of vehicles on profiles placed in strategic zones of the image. Profiles have a shape determined by camera and geometry parameters and their output gives information about queue length. In [12] output of profiles is interpreted by a neural net in order to obtain a better queue length estimation. Aim of these system is to give an instantaneous precise estimation of queue length. On extra-urban roads, measuring queue length from a single station is not very useful; when queues occurs, ve- (Italy) This work was partially supported by Provincia Autonoma di Trento hicles fill the entire field of view in few seconds. A global description of the complete queue event, including temporary nterruptions of queue status, could be more interesting for traffic engineers. The approach presented in this paper is based on the definition of a vehicle queue as a traffic event characterized by the following situations: presence of vehicles on the monitored roadway; no motion on the roadway. Measuring the quantities presence and motion and combining them, are the basis of the developed system, that can establish the forming of queues in each monitored lane, and characterize them with a severity index. The algorithm has been implemented in a real-time system and tested under several illumination condition in an experimental traffic monitoring station on a two ways road near Trento (Italy). The paper is organized as follows: the measure of presence is discussed in Section 2, while Section 3 presents the movement measure technique. Section 4 provides a method for stabilize the measures through the time. Thresholds tuning problem is addressed in Section 5. A severity index for queues is proposed in Section 6. Section 7 describes the onfield experimental station along with performances of the queue detection algorithm. Section 8 concludes the paper. 2. Vehicle presence detection Detection of each single vehicle in the traffic scene is a very complex task due to problems such as occlusions and shadows. In order to measure a global presence of vehicle on the monitored road, this task can be avoided and the presence parameter can be measured at pixel level. There are two main approaches to estimate the presence of new objects in an image sequence taken from a static camera: background based methods and gradient based methods. Background methods perform the difference of the current frame with an updated background image, while the second approach is based on the detection of vehicle edges.

2 After an extensive experimental performance measurement we decided to adopt the second strategy, which guarantees a very fast computation and good results. In fact, considering that the road surface presents an almost flat grey level and that vehicles generate strong discontinuities in the intensity, edges can be a good indicator for the presence of vehicles. Under the assumption that edges generated from vehicles are stronger in a direction perpendicular to the road axis, we adopted a directional gradient computation algorithm, where the direction depends on the camera view. In our experimental environment the road axis is vertical with respect to the image, so we used a five levels directional Robinson convolution kernel [6] for the computation of horizontal discontinuities. This straightforward method permits to gain sufficient precision in a very low computation time. The gradient map restricted to the road portion in absence of vehicles, shadows and road markers presents mainly low values, while its histogram computed for scenes with vehicles, presents a strong bimodality: the biggest peak is in correspondence of low gradient values, while the other is near the maximum value and is due to the vehicle presence. It is not difficult to get a reasonable threshold separating vehicle contribution to road surface contribution: a simple but effective choice for the threshold is the middle of the range of the possible gradient values. Let G be the gradient map of a frame and t e the aforementioned threshold value. Let us consider the set E of pixels such that: E = {(x, y) R, G(x, y) t e } (1) where R is the portion of the image relative to the monitored lane. Let N(E) be the number of pixels in E and A(R) the area of the region R measured as the number of its pixels. Then their ratio: p edge =: N(E) (2) A(R) is related to the probability of vehicles to be present in the scene. We claim that the condition presence of vehicles is satisfied if p edge is greater than a certain threshold value P e. Establishing a value for P e is a quite difficult task, because inappropriate values for this threshold can lead to high error rates. The discussion about P e setting is postponed in Section Vehicle motion detection Many computer vision applications, like video compression or segmentation, require the detection and description of objects movement, making the literature about this subject very wide. Beauchemin and Barron [1] wrote an interesting survey about this topic. They proposed a classification of existing motion measuring techniques. Some of them are based on the solution of differential equations of pixels intensities. Others are based on Fourier transformed versions of the original images (frequency-based methods). Another class of techniques is based on finding correlation among sequence images (correlation based methods). For queue detection we considered two approaches: the first one is based on the phase of Gabor transformed images [3, 11] and the second one relies on differencing frame by frame. Quantitative measures [3, 4, 8] proved that the Gabor based methods are precise, reliable and can provide an estimate of the velocity of the moving points. Performances of Gabor based method for the queue detection application are described in [13] and [14]. The frame by frame difference algorithm computes the movement map by comparing corresponding pixel values in consecutive frames. This second approach is very fast, but don t yield up information about velocity. Nevertheless, experiments proved that it is sufficiently accurate to detect queues. In the following we describe the motion detection algorithm based on frame by frame difference. Let F t be the current frame and F t 1 the previous one. We define the difference map D as D = {(x, y) R, F t 1 (x, y) F t (x, y) t m } (3) where t m is a threshold. If N(D) is the number of elements in D, then p mov =: N(D) (4) A(R) is related to the probability that something is moving in the region of interest. We claim that the condition motion on the roadway is satisfied if p mov is greater than a certain threshold P m. Experiments show that setting a value for the threshold t m is not critical with respect to the error rate. A factor 0.15 of the maximum admissible value of D is a good compromise between obtaining a sufficient sensitivity to movement, meanwhile discarding low differences due to noise. Setting of P m, analogously to P e, is derived from the analysis of the error rate. In Section 5 is discussed a technique for the threshold tuning. 4. Time integration The output at the time τ of the queue detection algorithm is queue if p edge P e (presence) and p mov < P m (no motion). Otherwise, it is no queue. Since values of p edge and p mov exhibit high variations in correspondence of camera vibrations, acquisition noise or other phenomena, the queue / no queue response is unstable through the time. For example, the system could output queue for 0.1 seconds in the middle of a no queue period. To overcome this difficulty the values of p edge and p mov have to be smoothed in the

3 time. The smoothing is performed by a moving average of the values, where width and center of the moving window are determined as explained in the following. Let W be the number of past measures of p edge and p mov to be averaged and T int the integration time, i.e. T int = W/N s where N s indicates the number of frames per second. In order to estimate a proper value for T int, we considered a 15 minutes video sequence acquired at 25 frames per second by a camera overlooking a traffic light controlled intersection. An operator labelled by hand each frame as queue, noqueue, or doubtful. Let ɛ W be the number of missclassification at frame level produced by the queue detection algorithm with respect to the manual labelling, using the averaged values p edge and p mov. We calculated ɛ W on the traffic light sequence for each value of W in [1, 170]. For each W we computed and used optimal threshold values for P e and P m. The computation of ɛ W depends not only on the W value, but also on the shift of the window with respect to the current frame τ. We considered two different positioning of the window: previous interval: I (P ) = [τ W + 1, τ]; centered interval: I (C) = [ τ W 2, τ + W 2 The first choice permits to get p edge and p mov immediately after capturing the τ-th frame. In the second case there is a delay of W/2 frames after the capturing time. Diagram in Figure 1 shows ɛ W as a function of W for both the interval positions. In the case of previous interval, there is a wide zero error plateaux for W between 17 and 48. The middle point of this segment corresponds to an integration time of about 1.25 secs. In the case of centered interval, errors are present for every value of W. For this reason and the aforementioned delay problem, we opt for an integration time of 1.25 secs with a previous positioned window, assuming that it is suitable for each situation of queues. 5. Tuning of thresholds When the queue detection system is installed on field, it is necessary a training phase to determine proper threshold values. We developed an algorithm that, starting from a manually labelled sequence, computes automatically optimal values for the edge and movement thresholds P e and P m for p edge and p mov, respectively. The labelling of the traffic light sequence is a reference classification T arget(τ), where τ is the frame counter. The algorithm is divided into two phases: 1. Let S be the set of tuples ( p edge, p mov, τ), each one corresponding to a frame of the training subsequence. Let C be the subset of S containing only the tuples concerning ]. ε N previous inteval (fr. by fr. diff.) centered interval (fr. by fr. diff.) N int Figure 1. Missclassifications of queue frames as a function of the moving window width. Continuous line for previous interval; dotted line for the centered interval. frames labelled as queue: C = {( p edge, p mov, τ) S T arget(τ) = queue} (5) The first phase threshold values are: P e = min{ p edge ( p edge, p mov, τ) C} (6) P m = max{ p mov ( p edge, p mov, τ) C} (7) 2. There are two cases. In the first case the missclassification error rate on the considered training subsequence using P e and P m in the queue detector is not zero; in the second case the error rate is zero. In the first case we search for thresholds that minimize error rate in the following ranges: P e [P e; (1 + δ edge ) P e] (8) P m [(1 δ mov ) P m; P m] (9) with δ edge = 0.15 and δ mov = 0.25, in our experiments. In the second case, although the error is zero, in order to obtain robust thresholds, we perturb them without altering error rate on the training set. Let L, M be defined as: L = {( p edge, p mov, τ) S T arget(τ) = noqueue} (10) M = {( p edge, p mov, τ) L p edge P e} (11) Then, the P m threshold is: P m = (P m + min{ p mov ( p edge, p mov, τ) M})/2. (12)

4 Analogously, with: G = {( p edge, p mov, τ) L p mov P m } (13) the P e threshold results to be: P e = (P e + max{ p edge ( p edge, p mov, τ) G})/2. (14) The evaluation of the proposed threshold finder algorithm is based on a variant of the k-fold cross validation paradigm, where training is performed each time on a single subset and training error rate is computed using the remaining k 1 subsets. Analysing the training error rate as a function of the training time, we observe that for the traffic light sequence the threshold finding algorithm needs only just two minutes of training sequence to get an averaged error rate of about 1% on the not-training part of the sequence. Therefore, it is sufficient to train the system with a labelled sequence containing only one formation and one dissipation of a queue event. 6. Queue severity index Depending on their cause, queues may be either moving or completely stopped. Stopped queues occur when there is completely interruption of road for a significant amount of time (e.g. due to accident), periodical queues are typical at traffic light locations, while in traffic congestion the internal queue dynamic may involve a series of starts and stops. In order to classify queues, we have developed a state machine with four states: no queue, queue, maybe queue, maybe no queue (see the transition diagram in Figure 2). If the system N no queue Q N Q (t c t s <T NQ ) ^ t s :=t c Q^ (t c t s >T NQ ) maybe queue N^ (t c t e >T QN ) T tot :=t e t s P:=T Q /T tot save data t r :=t s T Q :=0 Q t r :=t s queue N maybe no queue Q t e :=t c T Q :=T Q +t e t r N^(t c t e <T QN ) Figure 2. Queue severity machine. Q and N mean that the system measures an instantaneous queue and no queue event, respectively. t c is the current clock time. t s, t e and t r are auxiliary time variables. The small arrows indicate actions during the state transitions. is in no queue, then queue events shorter than a latency time T NQ are ignored; if a queue persists more than T NQ, then the system changes to queue. It remains in queue, until a no queue interval occurs longer than a latency time T QN. The other two states are introduced to implement the latency time effects. We define the duration of a queue event as the interval between entering into queue status and the final leaving. For every queue event are computed the starting time, its duration, and the severity index P. P is defined as P = T Q /T tot (15) where T tot is the total duration of the queue event (excluding the last permanence in maybe no queue status) and T Q is the total time passed in queue and in maybe queue status. The severity index P is a percentage related to the stationarity of the queue. More this number is close to 1 and more the queue is blocked. Latency times T NQ and T QN are parameters that can be changed in run time, in dependence on the application. Low values permit high precision measures, but cause severity index to be meaningless. In the experiments we used T NQ = 6 secs and T QN = 30 secs. We remark that events classified as queue having a low severity index (for example less than 0.6) can be automatically rejected. 7. The on-field experimental system The queue detector described in this paper belong to a traffic control system that includes a vision-based module for vehicle parameters extraction. The experimental station is installed in a suburban area near Trento (Italy) overlooking a high traffic two way road where queue occurrence is due mainly to commuter trips. The camera feeding the queue detector monitors a long section of the road. By means of a graphical user interface, at the installation time the operator selects the regions of interest (lanes). We can take advantage of the vehicle parameter module of the system in order to extend the capability of the queue algorithm to detect slow moving steams. In fact, in day light conditions, the parameter module makes available for each vehicle its velocity, and in particular the average velocity of last ten vehicles passed in each lane. In this case we can detect traffic events defined by presence of vehicles, movement and average speed less than, for example, 25 Km/h. The experimental system is working continuously onfield for about 15 months, providing a huge amount of data for evaluation and performance considerations, for both lanes, and in different illumination and weather conditions. 7.1 Evaluations The evaluation phase was performed on a 6 months period. In order to evaluate the correctness of the algorithm,

5 acquired images and the correspondent values of p edge and p mov have been saved every 15 seconds in case of queue response, and every 20 minutes otherwise. This choice is prone to evaluate false alarms (response is queue, really there is no queue), rather than missing alarms (response is noqueue, really there is a queue). This depends on the road management policy, that prefers to miss a dispatch of a queue alarm to the motorists, than to alert them unnecessarily. In the 6 months period the system measured a total queue time of about 48 hours, 44 in town direction and 4 in the opposite lane. 75% of total queue time is spent from 7 to 9 am in working days. This result accords with the motorist feeling (see Figure 3). During this period there was a false time Sun Mon Tue Wed Thu Fri Sat 6,0% 4,0% 2,0% 0,0% 12,0% 10,0% 8,0% week day Figure 3. Percentage contribution of different days of the week to the total queue time. The hottest period is from 7:00 to 9:00 in working days, with a shorter queue time on Friday. The Sunday afternoon queue is also to notice. alarm total time of about 4 hours. False detection events have usually a duration of about ten minutes, and occur in sunny days with low traffic levels. They are mainly caused by shadow effects. In order to measure missing alarms and analyze their causes we have considered a shorter experimentation (4 days) with a more frequent image saving. During this period the system detected a queue with an erroneous duration measure. The system missed the beginning and the ending of the event estimating a shorter queue time. This event happened in the middle of the night. The analysis of results gives reasons for most occurrences of false alarms and miss alarms: the former are mainly due to shadow presence, while the latter occurs in night conditions. In the following we analyze the errors and suggest some improvements. queue time percentage Shadows - In outdoor applications shadows are a common problem tackled by many researchers [10]. Shadows affect the queue detection algorithm mainly overestimating the presence of objects on the roadway. In our application we can consider different kind of shadows: 1. active shadows (vehicles); 2. rocking shadows (trees); 3. passive shadow (stationary structure). Rare (two) false alarms are due to the first kind of shadows. In fact, stopped vehicles in a lane cast strong shadows on the other, empty, lane. In this case the algorithm detects a queue in the second lane too, because there is presence of edges and there is no movement. Rocking shadows, mainly due to vegetation, affect the movement measures. Tree shadows can generate a number of movement points sufficient to inhibit the alarm of a queue. During the experiments no missing alarm due to rocking shadows was noticed. Finally, some false alarms are due to shadows produced by fixed structures. Their intensity and shape vary with weather conditions, day time or season, making their modeling hard. False alarms are due to the presence of strong edges of shadows in sunny days and no movement. In order to improve the algorithm for this last case, we assume that structural shadows vary slowly with the natural change of the sun position. The improvement we propose is based on varying the P e threshold as a function of the number of edges of a background image, using the Boninsegna-Bozzoli algorithm [2] for background extraction and updating. The threshold varying rule is: P e = { Pe if p edge P e f s P e if p edge > P e (16) where p edge is computed on the background image, and f s is a factor greater than 1 (a first estimate leads to f s = 1.2). While the background image is updated frequently (five times per second), Equation 16 should be applied on a longer period (once a minute, for example). A frequent application could cause the premature queue alarm stopping in case of very long and stopped queues. In fact, in this case, the background image absorbs vehicles, having as a consequence the P e growing, queue alarm stopping, and therefore a missing alarm. Night - Night conditions could be a source of missing alarms. Intensity level in night images in absence of vehicles is very low. When some vehicle is passing, the salient visual features are headlights and their beams [5], making unusable many algorithms for vehicle detection, otherwise appropriate in day illumination. As the proposed algorithm takes into account only features at pixel level (edge and moved points), it is sufficient to modify the edge threshold P e during night. In fact, in the development phase, we noticed some missing alarm in dark conditions, because the

6 average measure of vehicle presence is lower than in day illumination. Our improvement is to lower automatically the edge threshold as in the following, when the background image intensity is low: f n P e if p edge f P P e P e = P e if p edge (f P P e, P e ] (17) f s P e if p edge > P e where p edge is computed on the background image and f n, f s, f P are factors to be determined (a first estimate leads to f n = 0.5, f s = 1.2, f P = 0.1). Further experimentations at the station proved that the modified threshold updating rule is sufficient to solve the missing alarm problem in night conditions. 7.2 Execution time The computer installed at the experimental session is a Pentium III 700 MHz PC with 256MB RAM. The acquisition device is a Hauppauge WinTV PCI card and the operative system is Linux. The system acquires grey-level images that are used for transmission to a remote control center. The queue detection algorithm works on undersampled pictures at a frame rate of 25 frame per second. In order to operate in real time, the available time for each frame is therefore 40 ms. In the following are reported the average computing times measured on a one hour execution (90, 000 frames). Execution times are measured using TAU (Tuning and Analysis Utilities) performance analysis environment ( For each frame, the execution time is spent as follows: 4.2 ms to compute edge map on undersampled image; 0.4 ms to compute frame by frame difference map; 1.2 ms to apply thresholds and points counting; less than 0.1 ms to compute queue condition and to update the state machine; 2.1 ms to update the full size background image (10.6 ms every 5 frames); 0.4 ms for system overhead. The average computing time per frame is about 8.4 ms. Further 2.3 ms are used for JPEG compression of full size images for the transmission to the control center. Remaining 29.3 ms per frame can be exploited to improve the system or to add new features. 8. Conclusions In this contribution we have presented a real system for traffic queue detection. It is able to detect queues in each of the monitored lanes, providing their duration along with a severity index. By means of an extensive testing phase, the system shows to be robust with respect to weather conditions and illumination variations. Future works include an analysis of the proposed improvements in order to minimize false and missing alarms. Acknowledgments. We wish to thank Stefano Messelodi and Carla Maria Modena for the fruitful discussions and comments about the content and the structure of this brief report. References [1] S. Beauchemin and J. Barron. The Computation of Optical Flow. ACM Computing Surveys, 27(3): , September [2] M. Boninsegna and A. Bozzoli. A Tunable Algorithm to Update a Reference Image. Signal Processing: Image Communication, 16(4): , November [3] A. Cozzi, B. Crespi, F. Valentinotti, and F. Wörgötter. Performance of phase-based algorithms for disparity estimation. Machine Vision and Applications, 9: , [4] B. Crespi and G. Tecchiolli. Adaptive Gabor filters for phase-based disparity estimation. Int. J. of Pattern Recognition and Artificial Intelligence, 13(5): , [5] R. Cucchiara and M. Piccardi. Vehicle detection under day and night illumination. Proc. of ISCS-IIA99, Special Session on Vehicle Traffic and Surveillance, Genoa, Italy, [6] E. Davies. Machine vision: theory, algorithms, praticalities. Academic Press Inc., [7] M. Fathi and M. Siyal. A real-time image processing approach to measure traffic queue parameters. IEE Proceedings. Vision, Image and Signal Processing, 142(5): , October [8] D. Fleet and A. Jepson. Stability of phase information. IEEE Trans. on Pattern Analysis and Machine Intelligence, 15(12): , [9] M. Higashikubo, T. Hinenoya, and K. Takeuchi. Traffic Queue Length Measurement Using an Image Processing Sensor. In 3rd Annual World Congress on ITS, Orland, [10] A. Prati, I. Mikic, R. Cucchiara, and M. M. Trivedi. Comparative evaluation of moving shadow detection algorithms. In 3rd Workshop on Empirical Evaluation in Computer Vision, December [11] T. Sanger. Stereo disparity computation using Gabor filters. Biol. Cybern., 59: , [12] M. Siyal and M. Fathy. A neural-vision based approach to measure traffic queue parameters in real-time. Pattern Recognition Letters, 20: , [13] M. Zanin. Realizzazione di un rilevatore di code di veicoli basato su tecniche di visione artificiale. ATTI della Scuola IAPR-IC: La Visione delle Macchine 2000, Modena, Italy, October [14] M. Zanin. Tecniche di visione artificiale applicate al controllo del traffico: rilevamento di code e conteggio classificato di veicoli. Master s thesis, Università degli Studi di Bologna, Corso di Laurea in Ingegneria Informatica, AA