NEW APPROACHES FOR REAL TIME TRAFFIC DATA ACQUISITION WITH AIRBORNE SYSTEMS

NEW APPROACHES FOR REAL TIME TRAFFIC DATA ACQUISITION WITH AIRBORNE SYSTEMS I. Ernst a *, M. Hetscher a, K. Thiessenhusen a, M. Ruhé a, A. Börner b, S. Zuev a a DLR, Institute of Transportation Research, 12489 Berlin, Gerany, (Ines.Ernst, Matthias.Hetscher, Kai-Uwe.Thiessenhusen, Martin.Ruhe, Sergey.Zuev)@dlr.de b DLR, Optical Inforation Systes, 12489 Berlin, Gerany, Anko.Boerner@dlr.de KEY WORDS: Traffic Monitoring, Airborne Systes, online data processing, car detection, velocity estiation ABSTRACT: The topic of the paper is the adaptation of airborne reote sensing techniques and ethodology in transportation. All traffic relevant applications require real-tie derivation of traffic flow describing paraeters. This paper illustrates approaches in hard and software for fulfilling these deands. Two systes for traffic data collection for different operations will be explained and detailed inforation for online georeferencing, real-tie pattern recognition, speed easureent and car classification will be given. The results will be shown and discussed. As a short outlook necessary enhanceents and possible sensor extensions will be presented. 1. INTRODUCTION Intelligent transport systes require a new kind of data acquisition ethods to fulfil the deands of today s traffic anageent. Prediction of traffic, dynaic routing, off board navigation, and standardisation of traffic flow paraeters are the challenges we are faced with. Airborne systes are well suited to fit these deands. The advantages of airborne data are spatial availability, the broad variety of extractable paraeters and the speed of collection. Beside of these advantages, however, there is a tie gap between collecting and processing the data. The highly dynaical syste of transport is extreely tie dependent. The broad variety of fields of interest requires different types of operational systes. We focus on two different priary scenarios of airborne traffic data collection. One we call traffic paraeter easureent for supporting traffic flow odelling, siulation, and prediction allowing any applications in transport. For this a fast data collection over a large area is needed as well as the availability at all weather conditions and the traffic flow description including data aggregation. The ost suitable configuration consists of an airplane, a theral infrared caera and a non tracking downlink syste. A network of receiving stations covered the whole Berlin city area and surrounding regions. The second approach is traffic onitoring as a tool to provide integrated solutions addressing issues of traffic onitoring, fleet anageent, and eergency services support, e.g. for the organization of large scale events. Its ain feature is the coverage of a certain area of interest for a longer tie. For this, a flexible airborne platfor, a high resolution caera in the visible spectral range and a precise navigation syste are needed. Additionally to the traffic flow, single car characteristics have to be described too. These requireents lead to a syste using a helicopter or an airship with a caera and a oving receiving station at the ground. The ost deanding challenge for both approaches is the software, i.e. the developent of suitable fast and robust iage processing procedures. 2. SYSTEM DESCRIPTION 2.1 Caeras Several scientific and coercial caera systes in the visible range of the electroagnetic spectru were applied and tested (e.g. Kührt 2001). For the traffic paraeter easureent syste we selected caeras working in the infrared range (IR). They have the advantage to be applicable even under difficult illuination conditions. In ost cases the spectral texture in the infrared allows an easier iage data interpretation as with visible caeras. The ain disadvantage of infrared caeras is the sall nuber of pixels (320 x 240) - so either the swath or the ground resolution has to be reduced. Besides, IR caeras are still very expensive. Due to its low nuber of pixels a high frae rate can be expected (up to 25 Hz) despite the liited downlink data rates. For the traffic onitoring systes caeras in the visual range have been chosen. They allow a high resolution which is necessary for car identification. A caera with a ediu nuber of pixels (1k x 1k, 2k x 1k) guarantees a sufficient ground resolution and is able to cover reliably the area to be observed. The frae rate of the caeras has to be deterined according to the application. For traffic onitoring a low frae rate (about 0.2 Hz) was found to be sufficient, while for traffic flow easureents frae rates in the order of 5 Hz have to be realized depending of the velocities of the car and the airborne platfor. The ground resolution is a coproise between certain paraeters, e.g. detector technology, expected oveent of the platfor and provided data transfer rate. 69

CMRT05: Object Extraction for 3D City Models, Road Databases, and Traffic Monitoring - Concepts, Algoriths, and Evaluation 2.2 2.3 2.4 Data transission The data transission rate is one of the ost liiting factors for real tie airborne systes. It defines the axiu possible iage acquisition rate. Focusing on freely accessible radio frequencies coercially downlink systes were chosen. They deliver data rates up to 5 Mbps. For the traffic paraeter easureent a syste of three ground stations with a distance of about 25 k between the is used. Directional radio links lead the data fro each station to the server with a data rate of 2 Mbit/s, where the best of the three data packages is selected and provided to the iage processing. The downlink for the traffic onitoring syste is based on a digital transission. A GPS-supported transportable antenna with echanical bea steering allows two channel counications. Its data rate aounts up to 5 Mbit/s, The syste has a range of about 10-15 k. Iage and attitude data received on the ground station are transitted to the traffic coputer for iage processing. Inertial easureent unit For both approaches a real-tie onboard georeferencing is required. This iplies that all six paraeters of the so-called exterior orientation (three translations, three rotations) of the caera for each snapshot have to be deterined. Depending on the desired accuracy of data products, these paraeters have to be easured with an accuracy in the order of one ground pixel distance and one instantaneous field of view. A typical technical solution for direct and precise easureents of the exterior orientation paraeters of the caera during iaging is to use an integrated GPS/Inertial syste, which cobines efficiently inertial sensors technology and GPS technology. DLR owns such a syste (POS-AV 410 of Applanix Corp.), which fulfils the required specifications (Lithopoulos 1999, Scholten et al 2001). The syste consists of an Inertial Measureent Unit (IMU) LN200 and a control unit with integrated D-GPS receiver. The IMU realizes easureents of accelerations and angular velocities and records oveents of the caera/imu. The tie synchronized IMU and GPS data are processed in a control unit within an inertial navigation algorith. The syste provides real tie output of position and orientation with a rate up to 200 Hz. In cobination with a differential GPS correction, an absolute accuracy for position of 0.5 to 2 eters and for attitude of 0.015 to 0.05 degree can be obtained. Syste integration Depending on the platfor, the caeras were ounted directly on shock ounts (helicopter) or on a stabilizing platfor (airplane). Two ain deands had to be fulfilled: firstly, the target area had to be observed reliably and secondly, the reaining vibrations ust not influence the iage quality even for long exposure ties (blurring). The IMU sensor head was ounted close to the caera in all configurations. Iage and attitude data have to be recorded synchronously. Therefore, Applanix trigger pulses were onitored and used for coanding the iage acquisition process. Caera and IMU require coputing units for controlling and data pre-processing. 3. DATA PROCESSING 3.1 Direct georeferencing The real tie orientation data stored in a control coputer describes the actual position of the caera. This position is given by longitude, latitude and ellipsoid height with respect to the geodetic datu WGS84 and the rotation angles of IMU easureent axes given by roll, pitch, and heading with respect to the local tangential coordinate syste. The isalignent between the IMU and the caera axes (bore sight angles) has to be estiated offline once per syste installation (using the aero triangulation ethod). Paraeters describing pixel positions on the focal plane (socalled interior orientation) are necessary for georeferencing the iage data. They were deterined during a calibration process in optical laboratories at DLR. During the easureent flights, real tie orientation data, isalignent angles and interior orientation define a transforation fro iage space to object space and vice versa. Assuing a ediu terrain height, the position of the vehicles can be estiated. Consequently, for each pixel of interest a (x,y)-tuple can be deterined and each real world object corresponds to an e- quivalent pixel in the iage. Figure 1 illustrates the projection geoetry with directions of used coordinate fraes. Figure 1. Illustration of projection geoetry with directions of used coordinate fraes Thereafter, paraeters of interior and exterior orientation of the caera are known. Assuing a known interior caera geoetry, for an observed point P the following equation describes the relation between iage space and object space P = C + ( P C) = C + α dp (1) xp g b c dp = Cg Cb ( Φ, Θ, Ψ) Cc Ci yp f P, (2) where xp, yp are coordinates of P in iage frae (i) f eans focal length of the caera, C is the caera projection centre, i 70

Φ, Θ, Ψ (Roll, Pitch, Heading) are Euler angles for the transforation fro navigation frae (b) to geographic frae (g). l C k describes an appropriate rotation for transforation fro frae (k) to frae (l). Thus it is possible to project an iage (i) to the digital ap frae () in a world coordinate syste and vice versa. 3.2 Iage processing After deriving the relations between object space and iage space, relevant traffic objects have to be detected in the iage data. Theatic iage processing is the ost deanding part of the project. Different algoriths were developed and tested (e.g. Hetzhei et al 2003). The preferred approach (Ernst et al 2003) bases on a digital road ap (Navteq). Using the a priori knowledge about roads (e.g. nodes, directions) and anually acquired paraeters (e.g. bus or restricted lanes, parking lots), iages can be asked considering a argin depending on the accuracy of different data (etc. GPS/IMU data quality, aps). The roads are now the only iage sections to be investigated. Histogra based estiators can additionally liit the car search region. In this anner search area and calculation tie per iage can be reduced significantly. To get accurate knowledge about the apped roads all street segents of a sufficient area around the recorded region have to be tested regarding their intersection with the iage. This inforation enables the aggregation of vehicle data fro iage sequences later on. The vehicle detection is done on the asked iage fro the previous step. The sizes of the expected vehicle in the iage space are dynaically adapted to the current navigation data values (height over ground, attitude of the aircraft). The vehicle recognition works on single iages. Approaches based on difference iages or estiated background iages do not work reliably for test flights with airplanes due to their fast speed. The vehicles have a variety of appearances in the captured iages depending on sensor type, object properties and environental conditions (e.g. illuination, teperature). Most of the traffic objects can only be recognized as coarse rectangular shapes to be ore or less in contrast to the background. Therefore the algorith searches for characteristic contours (of suitable sizes) in edge iages retrieved by applying edge detection operators, e.g. Sobel. If a higher pixel resolution is available (VIS caera), further properties of vehicles such as the existence of special cross edges can be included in the search process. Pixel values theselves fro the original iages give additional inforation for consolidation or rejection of vehicle hypotheses or indications of the probable driving direction. Evaluating the nuber of vehicles per scene gives a easure for traffic density that can be provided to a central processing coputer. High frae rates allow the deterination of velocities. The frae based inforation is now processed for successive iages in cobination to deterine vehicle velocities. Virtual car positions are obtained fro real car position data fro one iage and navigation data fro the following iage. Velocity vectors can be extracted by coparison of these virtual car positions and real position data fro the second iage. The repeated recognition of a car in the following iage ephasizes the correctness of the car hypothesis. Assuing a tie difference of 1/5 s between two iages and a pixel resolution of 0.5, velocities can be detected with a discrete step width of 9k/h. On the other hand, a sall car oving with 80 k/h changes its position fro iage to iage in the range of its length, which akes an object atching difficult. The vehicle recognition algorith delivers rough size estiation so that accepted car hypotheses can be divided into a nuber of length classes. Using three classes has proven to be very practical; a larger nuber reduces the exactness of the classification. Thus essential shapes of cars, vans and long vehicles can be detected. The traffic data extraction within the airborne traffic onitoring projects is done per iage and road segent first. Densities and/or velocities are calculated for each vehicle class fro the obtained vehicle nubers and positions. The extracted data for single iages are cobined for copletely observed road segents using size and position of the scanned streets. The calculated average velocities and densities per road segent of the digital ap and per tiestap can be used now as input data for siulation and prognosis tools. 4. RESULTS AND DISCUSSION Real tie airborne traffic data acquisition systes and their validation have been deonstrated. During one year we perfored several flights for testing and iproving the technical equipent as well as iage processing procedures for both systes. This results in a final technical configuration with satisfying perforance and a shortening of processing tie. 4.1 Software syste ipleentation Different software packages were developed and tested for the iage processing and the real tie extraction of the relevant traffic inforation. Figure 2 gives an exaple of a typical scene during processing. Figure 2. Iage capture of data processing For better understanding and orientation in the iage the street naes were blended in the iages and in parallel a sall ap region with the iage projection is displayed. Both inforation cae fro the digital street ap offered by the copany Navteq. 71

CMRT05: Object Extraction for 3D City Models, Road Databases, and Traffic Monitoring - Concepts, Algoriths, and Evaluation For the syste which collects traffic paraeters the average processing tie is 0.16 s per iage what corresponds to a frae rate of 6 Hz. For the traffic onitoring syste with a lower frae rate of 0.5 Hz the ean processing tie is 1.0 s. The size of the fraes is also different: the IR-Caera frae chosen for the traffic paraeter easureent gives just a sall picture. The Caera for onitoring applications has a wider field of view. Both values eet the criterion of real tie processing. During a typical flight tie of 90 in the tie gap between transission to the ground and input of extracted traffic data into the data bases never exceeded 30 s. Before starting coparison of derived results to other traffic data sensors we first focussed on the algorith validation on a single iage basis to prove the correctness of the approach. An additional software tool for the coparison of anually counted cars to the autoatically detected ones in every single iage has been developed. We analysed iages in nuerous sequences with various traffic situations under different weather conditions. Figure 3 shows an iage of the coparison tool. Blue squares indicate autoatically detected, red squares show the anually detected cars, and green circles ark the atched ones. 4.3 Discussion and iproveent Since we wanted to achieve the real tie feasibility first, also the recognition rate is not too poor. However, there are still several opportunities to tune the algorith. As already entioned excluding parking cars increases the recognition rate significantly. The discriination between parking cars and vehicles only standing in right lane will be one challenge for the next future. Another reason for not finding all cars correctly is that the distances between cars standing at a light signal are soe sall that they ight be below the pixel resolution of the caera. New versions of the algorith will account for that. The quality of detection strongly depends on the quality of the digital ap. The algorith accounts for street inforation like nuber of lanes or directions of polygons to create the best possible extraction of the streets in the iages. Any inconsistency in the ap inforation leads to systeatic errors in calculation of the traffic paraeters. Especially the increasing nubers of lanes around crossroads leads to a decrease of detected vehicles. Ipleenting this crossroad inforation will iprove the detection, too. Finally, an exact deterination of the detection error and of its dependence on road type and other influence would allow a better conclusion on the real traffic densities. 4.4 Validation and coparison to other traffic sensors The airborne systes easure the nuber of cars (i.e. the traffic density) and their speeds. Fro these values, the traffic flow (vehicles per tie unit) can be derived. This would allow a coparison with data fro other sources, e.g. stationary sensors. We copared the data with stationary induction loop detector data fro the Traffic Manageent Centre Berlin (VMZ) as well as with easureents fro a video detection syste (Autoscope SOLO). Both data atch very well (see Fig. 4). 120 100 SOLO (stationary) airborne, locally airborne, long segent Figure 3. Coparison of auto and anually detected cars 4.2 Results of test capaigns For the helicopter syste we found as a preliinary result for arterial city roads that 61 % of all cars were detected correctly by the autoatic detection syste. Only 20% were falsely counted cars, i.e. cars that not do exist. Siilar results have been obtained for the airplane syste. Moving cars can easier be identified than parking cars. If we exclude parking cars, the autoatic detection rate increases to about 75% for the traffic paraeter easureents. For this case exclusion of parking car is tolerable because they do not contribute to the traffic flow. Only 8 % falsely counted cars are then found for arterial roads. In every iage sequence the nuber of autoatically found cars is saller than anually seen. The derived average velocities per car class are in good agreeent with long ter experiences of urban traffic observation. They never exceed the speed liitations. The evaluation of averaged velocity estiation fro observed iages is ipossible so is has to be done on a single car basis, to be discussed beneath. traffic density (vehicles/k) 80 60 40 20 0 0 10 20 30 40 50 60 tie (inutes after 6 p..) Figure 4. Coparison of traffic densities gained fro a stationary sensor (SOLO video syste; red curve) with airborne easureents fro a 90 long road segent (green) and averaged over a 440 long road section (blue). Each stationary detector, however, easures only at one location (as function of tie); airplane based ethods easure only during a few oents of tie (as function of position). Thus, direct coparisons only are possible at the location of the detector and for the tie of the flight (see Fig. 4). For checking the accuracy of the velocity easureents it is necessary to copare the airborne results with data fro other sources. For this purpose, we used a test car which was identified in the airborne iages, due to its special reflection behavi- 72

our. The speeds easured on board the car and fro the airborne systes have been copared (see table 1). Flight 06.05. 2003 03.11. 2003 16.12. 2003_A 16.12. 2003_B Velocity on board (k/h) Velocity fro airplane (k/h) Difference (k/h) 27.1 25.6 1.5 5.5 Relative Error (%) 24.4 20. 4.4 18.2 25.3 19.5 5.8 22.9 23.0 19.8 3.2 13.9 Table 1. Velocity deterination autoatically fro airplane iagery The average relative error of velocity deterination for one test car within the iage sequence of ~1 sec length is about 15 %. This error is inverse proportional to the absolute value of velocity. It also depends on the geoetric resolution and the radioetric conditions of iages. Despite the sall nuber of easures the calculation algoriths deonstrate its validity well. Because densities and speeds are quantities varying not only with tie but also with position, the airborne systes are able to provide inforation which cannot be obtained by stationary sources. 5. SUMMARY AND OUTLOOK 7. REFERENCES 1. H. Runge et al., Iage is everything. Traffic Technology International. No 2, 2003 2. M. Ruhé et al., Air- and space borne reote sensing systes for traffic data collection - european contributions. The IEEE 6th International Conference On Intelligent Transportation Systes Shanghai, China, October 12-15, 2003 Proceedings on CD-ROM 3. R. Kühne et al., Fro vision to reality. Traffic Technology International, No. 8, 2002 4. E. Kührt et al.,. An autoatic early warning syste for forest fires. Annals of Burns and Fire Disasters. Vol. XIV, No. 3, Septeber 2001, pp 151-154 5. E. Lithopoulos, 1999, The Applanix approach to GPS/INS integration, Photograetric Week 99, FRITSCH/SPILLER (Eds.), Wichann Verlag, Heidelberg, pp. 53-57 6. F. Scholten et al.,. High Resolution Stereo Caera Airborne (HRSC-A): 4 Years of Experience. Direct Sensor Orientation of a Multi-Line Pushbroo Scanner. ISPRS Proceedings Sensors and Mapping fro Space 2001. University Hannover. 7. H. Hetzhei. Analysis of hidden stochastic properties in iages or curves by fuzzy easure and functions or Choquet integrals. Proc. World Multiconf. Systeics, Cybernetics and Inforatics, SCI 99 and ISAS 99, Orlando, Florida, Vol. 3, 1999, pp. 501-508 8. I. Ernst et al., LUMOS Luftgestütztes Verkehrsonitoring-Syste. Held on Verkehrswissenschaftlichen Tage in Dresden, Septeber 2003, proceedings on CD-ROM Both technical configurations are suitable to collect traffic relevant airborne data and therefore fit the requireents for real tie data processing. Fro the technical point of view a cobination of VIS and IR caeras and the fusion of their iage data proise a high potential. High geoetric and radioetric resolution is required. High frae rates are needed for velocity estiations. In order to overcoe the bottle neck resulting fro data downlink, onboard processing could be considered. The iproveent of the basic ap inforation becoes of great iportance. Ipleentation of ap and a priori knowledge in a Geoinforation Syste will led to better results. For disaster anageent, the digital street ap has to be ore flexible. Up to now it is possible to reduce the network autoatically depending on the current traffic situation (e.g. accidents or road works). There are a lot of scientific papers dealing with the idea to create a digital roadap by using reote sensing. These approaches should be extended to create a network of optional streets in the cause of catastrophes. Therefore it is also necessary to siulate the traffic flow depending on social data like e.g. nuber of households, nuber of persons, or nuber of cars in one area. For a higher disposal of such an airborne syste for traffic data collection RADAR or SAR-Sensors should be used. 6. ACKNOWLEDGEMENTS The authors would like to express their thanks to the partners of the projects LUMOS and Eye in the Sky. Further thanks to Rotorflug GbH Berlin for the Helicopter and to Air Tepelhof GbH for the Airplane. Navteq and VMZ Berlin supported the research in these projects with special offers for soe datasets. 73