Estimating Velocity Fiels on a Freeway from Low Resolution Vieo Young Cho Department of Statistics University of California, Berkeley Berkeley, CA 94720-3860 Email: young@stat.berkeley.eu John Rice Department of Statistics University of California, Berkeley Berkeley, CA 94720-3860 Email: rice@stat.berkeley.eu Abstract We present an algorithm to estimate velocity fiels from low resolution vieo recorings. The algorithm oes not attempt to ientify an track iniviual vehicles, nor oes it attempt to estimate erivatives of the fiel of pixel intensities. Rather, we compress a frame by obtaining an intensity profile in each lane along the irection of traffic flow. The spee estimate is then compute by searching for a best matching profile in a frame at a later time. Because the algorithm oes not nee high quality images, it is irectly applicable to a compresse format igital vieo stream, such as mpeg, from conventional traffic vieo cameras. We illustrate the proceure using a 15 minute long VHS recoring to obtain spee estimates on a one mile stretch of highway I-80 in Berkeley, California. I. INTRODUCTION Traffic cameras offer the potential to complement or substitute for loop etectors. Because they can provie finer spatial an temporal resolution, they have many avantages over loop etectors. In principle, vieo from cameras can also be use to etect lane-changing, accients, an queuing patterns an to extract macroscopic traffic parameters, such as flow, spee, an ensity. Also, cameras are becoming less expensive to purchase an maintain. However, in orer to use images from cameras to stuy traffic, a large amount of vieo must be processe an an efficient an practical system to extract traffic parameters is thus essential. After being igitize, an hour vieo can be up to a several gigabytes. The obective of this paper is to present a simple algorithm to estimate a velocity fiel, localize in space an time, from vieo ata covering a wie area with limite spatial resolution. The localization is fine enough to reveal the temporal an spatial formation an issipation of shockwaves. To emonstrate practicality of the algorithm, we present results from a 15 minute long vieo filme by a Berkeley Highway Laboratory camera. Figure 1(a) shows the layout of the Berkeley Highway Laboratory. Figure 1(b) shows a single frame covering about one mile of freeway. Kastrinaki et al. [1] provie an extensive survey of state of the art traffic applications of vieo processing, incluing roa traffic monitoring. Our methoology falls into the general category of optical flow, techniques of which are reviewe in Beauchemin an Barron [2]. Applications of optical flow to traffic monitoring have been base on etecting an tracking iniviual vehicles to estimate spee, ensity, an flow. The following are representative examples. Autoscope [3] etects an tracks vehicles within a etection zone (roughly a rectangle the size of a vehicle) an integrates their spatial an temporal signatures to measure their spees. The ACTIONS system [4], etects an tracks moving obects by estimating optical flow vectors which are then clustere to create caniate moving obects. The MORIO system [5] infers polyheral moels for obects moving relative to a stationary camera. The TITAN system [6] uses mathematical morphology to extract iniviual vehicle features, aggregates them into iniviual vehicles, an tracks them. It is capable of monitoring stretches of the motorway of up to about 1000 feet, epening on the height of the camera. The images of iniviual cars nee to be separate. In Fathy an Siyal [7], a morphological ege etector an backgroun ifferencing are use to ientify an track vehicles an calculate traffic parameters. Coifman et al. [8] evelope a feature-tracking algorithm to extract iniviual vehicle traectories from vieo ata by etecting preefine features from images, grouping them, an tracking the groups of features to prouce traectories. Dailey et al. [9] evelope a metho to estimate mean traffic spee, using an ege-etecting algorithm to fin centrois an estimating mean spee from centroi movement in successive images. However, algorithms that rely on ientifying an tracking iniviual vehicles are not feasible for use with images of poor quality an over a wie area. If the spatial resolution is poor, vehicles in the frame o not show istinctive lines or features throughout the whole span of view an thus can be neither clearly ientifie nor tracke. If a vehicle only occupies a small number of pixels, its features may be har to ientify an furthermore can change as the precise position of the vehicle within the pixel gri changes. Vibration ue to win causes further ifficulties, as o shaows an occlusions in congeste traffic. Grant et al. [10] report on an extensive test of Autoscope on freeways in Atlanta, Georgia, showing that counts egrae in accuracy as the istance of the count location to the camera increase out to a maximum of about 400 ft. Although they i not irectly measure the quality of spee estimation as a function of istance, they conecture that it is similar to that of the volume counts. In contrast, the metho set forth in this paper oes not
epen on explicitly ientifying an tracking iniviual vehicles. We emonstrate that it is robust to occlusion an shaows, which can be seen in Figure 1(b), an to the camera motion that is evient in the vieos. We compare the estimates obtaine from the vieo to those from high frequency loop etectors. II. DATA The ata use in this stuy were generate from a 15- minute vieo of about a one mile stretch of highway I-80 in Berkeley, California. A vieo test be, the Berkeley Highway Laboratory, consists of 12 cameras on the roof of Pacific Park Plaza, a 30-story builing besie the highway. The analog cameras have S-VHS vieo recorers attache. Figure 1(a) illustrates the setup an the coverage of each camera. Among six cameras looking north, the fiel of view of camera N6 is furthest own I-80E. It covers the longest stretch (about one mile long), from the Ashby Avenue on-ramp to University Avenue off-ramp. But it prouces relatively poor quality images ue to the poor angle an resolution. Figure 1(b) is a still frame from camera N6. The spatial resolution is such that a pixel in the near fiel of view is about 5 feet, whereas those furthest from the camera are about 15 feet. The images also suffer from occlusions by vehicles an their shaows an from camera motion. Despite the poor quality of the images, the ata from camera N6 is potentially informative, for example for stuying the effect of on-ramp flow on highway performance. Also, it covers the longest stretch, about one mile, an in principle, information extracte from this camera can be combine with that from the higher resolution cameras, which have smaller fiels of view. Algorithms evelope to analyze images from camera N6 shoul be applicable to other conventional traffic cameras. For the analysis, a 15 minute long tape was igitize at the rate of 10 frames per secon an the results save in ppm file format. Each frame, like Figure 1(b), has 800x640 pixels an each pixel has intensities for re, green, an blue channels. III. METHODOLOGY Because of the camera resolution, it woul be at best extremely ifficult to apply a feature etecting an tracking algorithm. Hence, it is necessary to evelop a new way to extract information from the images. We evelope an algorithm for this purpose. The algorithm procees as follows: First, an intensity profile of each lane in the irection of traffic flow is extracte from each frame. Arranging the profiles in time orer gives intensity flow in the time-space omain, in which vehicles appear as stripes, or moving peaks. Secon, the spee estimate at (t, x) in the time-space omain is compute via searching a pair of best matching patterns at time t + τ an t τ in terms of the L 1 norm (sum of absolute values of ifferences) an estimate as the slope of the line connecting the two centers of the pair. For iscussion an references to such correlation-base matching methos, see Beauchemin an Barron [2]. The algorithm has a number of avantages. First, it oes not involve vehicle tracking, which can be computationally very expensive, an hence makes it more efficient to process a large amount of ata. Seconly, in estimating local spee it oes not compute a graient, or a weighte average of spees of moving features aroun the location of interest. It uses the L 1 norm in fining the best matching pattern an hence is robust to noise. Later in this section, we show that uner certain conitions the algorithm is equivalent to fining a weighte meian of the spees of moving peaks. A. Intensity Profiles To generate the intensity profile for a lane, a mask (M) is create on a particular frame, a so-calle reference frame. The mask passes intensities of pixels in a region of interest. A maske image is presente at the top of Figure 2. Once maske, a frame contains intensities of pixels in the region. The pixels have three integers between 0 an 255 for re, green, an blue channels. Because these three intensities are strongly correlate at the pixel level, we take the average of re, green, an blue intensities. Then, we scan across the lane (orthogonal to the irection of traffic) an calculate the maximum average intensity along the irection of traffic, which we refer to as a maximum intensity profile. The maximum intensity profile corresponing to the top image of Figure 2 is presente in the bottom of Figure 2. The mask shoul be create for each frame, because the camera may be constantly shaking ue to strong win. To create masks automatically, we fin mappings (Ψ t ) from the reference frame to every frame uring the time perio an use the new transforme masks (Ψ t (M)). First, we choose four fixe obects (reference obects) on the reference frame an place square winows centere at them. Then, in each frame we search for the best matching patterns corresponing to the square areas centere at the reference obects. Once the patterns are foun, we use the coorinates of the center pixels of the square winows to compute the proection matrix. For more information, refer to chapter 5 of Hartley an Zisserman [11]. We repeat this process for each frame an stack the maximum intensity profiles in time orer to obtain an intensity flow on the time-space omain. The intensity faes as the vehicle moves away from camera, an the roa surface intensity varies epening on the location. To correct for this, at each location of the highway we etermine the maximum an minimum intensity uring the 15 minutes an form the ratio of the ifference between the intensity an minimum to the ifference between maximum an minimum. After this backgroun correction, the intensity is stanarize between 0 an 1. To change units from pixels to feet, we compute the proection matrix from image to the real worl, using the real imension of I-80 (For a etaile computation, also refer to chapter 5 of Hartley an Zisserman [11]). As a final step, we interpolate the intensities on a finer gri, which are equally space by about 5 feet. Two examples are presente in Figure 3.
(a) Berkeley Highway Laboratory Layout (b) A frame from Camera N6 Fig. 1. Camera Setup The resulting intensity flow is similar to a traectory plot in the time-space omain in that the stripes, or moving peaks, contain information about the traffic flow on the highway. But it iffers in that one curve oes not necessarily correspon to one vehicle. Rather one vehicle can be shown as two lines, or two vehicles traveling closely together can appear as one moving peak. Small or ark vehicles may be barely perceptible. Fine iscrimination is not neee for subsequent spee estimation. Fig. 2. B. Spee Estimation A filtere image an the corresponing intensity profile The iea behin the estimate is that a locally constant intensity pattern represents travel at a locally constant spee. Hence, after a short perio of time, the same pattern will show up again at the travel istance above the previous location in the time-space omain. To estimate the local spee at (t, x) in the time space omain, we choose two rectangular winows of size w t by w x an center one at (t + τ, x) an the other at (t τ, x). We then move the winow centere at t + τ up (increase x) an the other one at t τ own (ecrease x), computing the L 1 norms for the pair. The norm will be minimize at the travel istance. By the nature of the image, the shifts are integer-value. To eal with this iscretization, we fit the L 1 norms near the minimizer to a quaratic function to fin the interpolate istance which minimizes the norm. The spee is estimate as the ratio of the istance to the shifting parameter, τ, which is equivalent to the slope between the two centers of the pair of the best matching patterns. In the examples presente here, we use the L 1 norm criteria an set the winow size to 30 secons by 90 feet, τ as 3 secon, an the searching area as 0 to 80 MPH. The step by step outline of the algorithm is as follows. Let I(t, x) enote the intensity at location (t, x) in the timespace omain. 1) Fix the local winow size, w t w x, an the shift parameter τ, 2) For = 0,..., m, compute D() = t 0+w t x 0 +w x I(t+τ, x+) I(t τ, x ). x=x 0 w x t=t 0 w t 3) Fit D() near the minimum to a quaratic function of. 4) Fin 0, which minimizes D. 5) The spee is estimate as 0 /τ. Note that the algorithm oes not involve graient computation or slopes for iniviual lines on the intensity flow. After running the algorithm, we run a 2-imensional meian filter to remove noise, of size 40 secon by 370 feet. Also note that it simultaneously shifts two square winows (centere shifting), instea of fixing one winow centere at (t, x) an shifting the other winow at either t + τ (forwar shifting) or t τ (backwar shifting) in time. As can be seen
from a Taylor series expansion, erivatives are estimate more accurately by central ifferences than by un-centere ones. The erivative of a quaratic function is estimate exactly by central ifferencing, but not by one-sie ifferencing. Because the algorithm involves computing the sums of absolute ifference over 2 square winows, it can be quite slow. To spee up, we use a subgri within the square winow instea of using all the intensities. Base on the empirical comparisons, we fin even a coarse gri of resolution one secon by about 10 feet (using 5% of ata points) is sufficient to prouce an estimate equally goo as using all the ata. Also, we nee not evaluate the estimate at every location. Instea, we estimate the spee on a sparse gri an then interpolate. To gain some insight into the nature of the estimate prouce by the algorithm we now consier an iealize continuous version. Suppose features, A (t, x), have isoint supports I, are parabolic on I (3r an higher orer erivatives are zeros), an travel at the spee of v. The features correspon to the contributions of iniviual vehicles to the intensity profile, which we write as f(t, x) = A (t, x) = A (x v t). Now consier the minimizer of 1 2τ t 0 +w t x 0 +w x f(t + τ, x + ) f(t τ, x ) xt. t=t 0 w t x=x 0 w x For small an τ 1 ˆv f(t + τ, x + ) 2τ t x f(t τ, x ) xt 1 A (t + τ, x + ) 2τ t x A (t τ, x ) xt 1 2( v τ)a 2τ (x v t) xt t x τ v t S S τ v x A (x v t) xt where S = t 0 +w t x0 +w x t 0 w t x 0 w x A (x v t) xt an S = S. The minimizer of the final expression above is a weighte meian of the iniviual velocities v in which the weights are S /S. That is, it is the meian of a iscrete probability istribution which has masses S /S on the values v. Vehicles with large erivatives of their iniviual intensity profiles thus contribute most heavily to the estimate. The meian, however, is insensitive to extreme velocities. By contrast, if we were to use the sum of square eviations rather than the sum of absolute eviations, the argument above shows that the estimate woul be a weighte mean, an less robust to extreme v. This argument formalizes the notion that the shifting an matching algorithm estimates a weighte meian velocity over a region of space an time. IV. RESULTS Two intensity flows uring 15 minutes from 3:00pm on 17th of December 2001 to 3:15pm on the same ay are presente in Figure 3. We picke two lanes; the right-most (5th) lane of I-80E an the 3r lane of I-80W. We chose the two lanes for the following reasons. The 5th lane merges with the Ashby onramp at the near fiel of the frame (at aroun 500 feet) an the inflow creates congestion. The 3r lane of I-80W experience the worst stop-an-go traffic an ha more trucks than any other lanes uring the 15 minutes. During the 15 minutes, the east boun traffic experience moerate congestion, shown in the intensity flow as changes in slopes of lines. Examining the figure carefully, one can see some lines isappear an appear, cause by lane-changing an occlusions from the shaows of vehicles traveling in the next lane. The west boun lanes experience very heavy traffic. Also recall that in the 3r lane there were the most trucks. In the intensity flow, trucks appear as broa stripes. In the intensity flow, we observe flat patterns lining up, which shows shockwaves propagating against the traffic. The lane also experience the most frequent occlusion from vehicles an their shaows, ue to the stop an go traffic in the next lane. In this lane, there are marks on the roa to signify the off-ramp an they create horizontal stripes aroun 30, 60, 460 pixels even after backgroun correction. However, the spee estimate is robust to these artifacts. From the spee estimate of I-80E, we observe congestion ue to the inflow from the Ashby on-ramp an corresponing shockwaves. We suspect that a traffic signal on Ashby Avenue cause perioic fluctuations in inflow an hence the pulsating series of shockwaves. Also note a pronounce shockwave originating at aroun 360 secon an 0.5 mile from the University exit an travelling against the traffic at about 10 MPH. The I-80W spee estimate shows even stronger oscillations shockwave evolution, an some variation in their velocities of propagation. The figure shows that the shockwaves typically travel at about 10 MPH. Because we o not observe where they originate an issipate, we cannot verify how long the shockwaves travele before issipating, base solely on camera # 6. For now, we conecture that the shockwaves were create further ownstream on I-80W, about 1.3 miles south of the Ashby off-ramp, at the notorious split of I-80W into I-580S, I-880, an I-80W. Further investigation using tapes from cameras # 1-5 woul reveal more information. To check our estimates, we compare them to loop etector ata. Loop etectors are locate at stations 3, 4, an 5 in the orer of istance from the University exit; refer to Figure 1(a). Unfortunately, the stretch ha been pave recently an we coul not locate precisely where the loops were. So, we approximate the loop locations by those of the cabinets an pull-boxes of the loop counter stations, which are locate at the sie of the wall of I-80E. In Figure 5, the ots are the point
estimates(vehicle by vehicle) from the loop ata. The spee estimates corresponing to the cabinet(pull-box) locations are shown as the soli lines. TABLE I MEANS AND STANDARD DEVIATIONS (MPH) East boun West boun Station 3-3.7 (2.1) -2.0 (3.8) Station 4 0.3 (1.6) 0.0 (1.8) Station 5 4.3 (1.8) 1.6 (3.1) The figures show that the estimates are very close to the loop ata uring the 15 minutes an pick up most of the oscillations. There are some systematic ifferences, which may be attributable to the imprecision of loop etector locations. Note that the spee ranges an traffic conitions for the west an east boun lanes are very ifferent, yet the estimates are very consistent in both cases. The means an stanar eviations of the errors between the estimates an the loop ata are reporte in Table I. V. CONCLUSIONS AND DISCUSSION The results above emonstrate the potential of our algorithm for processing a vieo recoring from a traffic camera, proviing a useful tool to stuy numerous traffic issues, such as the effect of an on-ramp, the evolution an issipation of queuing an congestion, an for monitoring highway performance. Despite its poor quality image, camera N6 provies very useful information in these regars. For some purposes, simple functionals of the estimate velocity fiel may be sufficient. For example, travel times can be estimate by tracing through the fiel, or the average velocity over space at a given time can be compute. Although the results we have shown are quite reasonable, we will stuy several issues in more etail in the future. One is the choice of the region on which to base shifting an matching. In principle, the rectangle coul be as small as one pixel in time an several pixels in space, or vice-versa. The computing time is faster for smaller rectangles, but the results are noisier (a efect which can be ameliorate, however, by smoothing the estimates). Smaller rectangles yiel a finer resolution in space an time, but again at the cost of noise. Larger rectangles localize less an are computationally more expensive, but prouce less noisy estimates. In principle, the regions nee not be rectangular an weight functions, such as Gaussian kernels, can be use instea of uniform weighting. Initial experimentation inicates that the final results are quite insensitive to these choices, but further stuy is necessary to optimize the algorithm for spee an accuracy. In aition to further improving spee estimation, we are eveloping algorithms to extract the other macroscopic parameters, flow an ensity, from the intensity profiles. This is more ifficult than velocity estimation. Counting is more feasible in the near fiel of view, an the results can be propagate through the estimate velocity fiel to obtain estimates of ensity an flow in the far fiel of view. We will also investigate the still more challenging problem of etecting lane changing. Finally we mention that we have use our metho on MPEG an AVI compresse vieo, with little egraation of the results. This may be useful if ata are to be transmitte prior to analysis. ACKNOWLEDGEMENTS This research was supporte by Partners in Avance Highways an Transportation an by a grant from the National Science Founation. We wish to thank Dan Lyy for assistance with the ata, Ryan Lovett an Phil Spector for assistance with computation an Peter Bickel, Z Kim, Jaimyoung Kwon, an Erik van Zwet for helpful iscussions. REFERENCES [1] V. Kastrinaki, M. Zervakis, an K. Kalaitzakis, A survey of vieo processing techniques for traffic applications, Image an Vision Computing 21, pp. 359 381, 2003. [2] S. Beauchemin an J. Barron, The computation of optical flow, ACM Computing Surveys, vol. 27, no. 3, pp. 433 467, Nov. 1995. [3] P. Michalopoulos, Vehicle etection vieo through image processing: the autoscope system, IEEE Trans. Veh. Technol., vol. 40, pp. 21 29. [4] W. Enkelmann, Interpretation of traffic scenes by evaluation of optical flow fiels from image sequences, IFAC Control, Computers, Communications in Transportation, 1989. [5] L. Dreschler an H.-H. Nagel, Volumetric moel an 3-traectory of a moving car erive from monocular tv-frame sequences of a street scene, Computer Vision, Graphics, an Image Processing, vol. 20, pp. 199 228, 1982. [6] J. Blosserville, C. Krafft, F. Lenoir, V. Motvka, an S. T. Beucher, New traffic measurements by image processing, IFAC Control, Computers, Communications in Transportation, pp. 35 42, 1989. [7] M. Fathy an M. Siyal, An image etection technique base on morphological ege etection an backgroun ifferencing for real-time traffic analysis, Pattern Recognition Letters, vol. 16, pp. 1321 1330, 1995. [8] B. Coifman, D. Beymer, P. Mclauchlan, an J. Malik, A real-time computer vision system for vehicle tracking an traffic surveillance, Transportation Research, Part C, vol. 6C(4), pp. 271 288, Aug. 1998. [9] D. Dailey, F. Cathey, an S. Pumrin, An algorithm to estimate mean traffic spee using uncalibrate cameras, IEEE Trans. Intell. Transport. Syst., vol. 1, no. 2, pp. 98 107, June 2000. [10] C. Grant, B. Gillis, an R. Guensler, Collection of vehicle activity ata by vieo etection for use in transportation planning, ITS Journal 5, pp. 342 361, 2000. [11] R. Hartley an A. Zisserman, Multiple View Geometry in Computer Vision. Cambrige, UK: Cambrige University Press, 2000.
(a) I-80 East Boun (b) I-80 West Boun Fig. 3. Intensity flows (a) I-80 East Boun (b) I-80 West Boun Fig. 4. Estimate velocity fiels
(a) I-80 East Boun Fig. 5. Comparsion between the estimate an the loop ata (b) I-80 West Boun