Pedestrian Recognition in Urban Traffic using a vehicle based Multilayer Laserscanner

Transcription

1 Pedestrian Recognition in Urban Traffic using a ehicle d Multilayer aserscanner Kay Ch. Fuerstenberg, Klaus C. J. Dietmayer Uniersity of Ulm, Department of Measurement, Control and Microtechnology Albert-Einstein-Allee Ulm, Germany kay.fuerstenberg@e-technik.uni-ulm.de Volker Willhoeft IBEO Automobile Sensor GmbH System Deelopment Department, Fahrenkroen Hamburg, Germany, wi@ibeo.de. Abstract Mounted on a ehicle aserscanners are able to obsere the ehicles enironment in order to detect, track and classify the surrounding objects and thus proiding data for actie safety systems. The latest deelopment of IBEO combines seeral innoations. The receier diodes are arranged in an array, which enables simultaneous measurements in 4 horizontal planes, e.g. to compensate pitching of the ehicle. In addition a multi target capability is integrated. This technique enables the detection of two distances with a single measurement, thus enhancing the robustness against rain. This paper introduces improed high speed object detection and high performance object tracking algorithms for real-time data processing. Additionally a classification of the road users is possible. A system architecture for detection and modelling of dynamic traffic scenes is introduced in order to proide a general idea of the different tasks necessary to reach the aim of a complete enironmental model using a sensor for a wide range of applications. suitable sensorial systems must be integrated into future cars. These safety systems will include object recognition in order to detect dangerous situations. It is ery important to assess the potential risk of an object for the passengers of the ehicle in case of an accident. Besides, it is of interest to get information about the potential hazard for the objects e. g. if a pedestrian is inoled, to initiate safety actions to protect the pedestrian. Therefore, special sensors and algorithms are needed. IBEO aserscanners and their integrated algorithms outlined in the following are able to delier data for such actie safety systems [1],[2],[3],[4]. To meet the manifold requirements a system architecture was designed in order to exhibit the dependencies between four main models used for detection and modelling of dynamic traffic scenes around a driing passenger car [5]. The compensation of the ehicles motion will be explained, which combines the relatie elocity of the objects determined by the aserscanner and the ehicle motion using the ESP-data. An absolute elocity of the objects is achieed, which is essential to obtain a reliable tracking and classification. INTRODUCTION Studies estimated that the annual rate of deaths and serious injuries was reduced by 12., since the introduction of passie safety systems. Howeer new passie systems will not offer such a significant progress for saing lies. There are still 1.2. accidents eery year in the European traffic with about 1.6. injured persons and 42. deaths. In order to obtain further improement, actie safety systems such as collision aoidance or Pre Crash as well as Fig. 1 Multilayer aserscanner D M

2 SYSTEM ARCHITECTURE The system architecture, shown in Fig. 2, outlines the dependencies between the different models for the detection and modelling of dynamic traffic scenes. It consists of four main parts The model of the sensing deices, a model of the street the car is presently driing on, a dynamic model of the ego ehicle and a cluster containing dynamic models of all objects to be identified around the ehicle. Street model Street knowledge Street type classification Street dynamic ane detection Object model Object knowledge Object detection Object tracking Object classification Object management Fig. 2 System architecture A. Sensor model Sensor model Sensor knowledge Basic Segmentation Adanced Segmentation Vehicle model Vehicle knowledge Vehicle parameters out of the single distance measurements. In addition, the outgoing laser pulse is ertical-line-shaped. The reflected light is picked up by an array of receier diodes, allowing to measure four scan planes. These scan planes hae a total opening angle of approx D M Outgoing pulse, shortly before hitting the target Echo pulses Outgoing pulse (partly reflected) Target Fig. 3 Measurement principle of the D M Fig. 3 shows an illustration in which the lower part of the laser beam hits a target, causing only the lower two scan planes to detect the obstacle. Using information about the sensor geometry een the obstacle height can be roughly estimated. The multi-target capability is another ery important feature of the aserscanner. This technique allows the sensor to detect two distances with a single measurement. Typically, this occurs if the beam hits a target with low reflectiity and high transmission or small sized targets such as glass or raindrops. A part of the outgoing beam is deflected back to the sensor, triggering the first measurement. ater, the remaining beam hits a second target, and the echo of this target triggers the second measurement. This allows the implementation of functions such as rain detection and remoal. 1) IBEO Multilayer aserscanner The adardigital Multiayer aserscanner is the latest member of IBEO AS's automotie aserscanner family. From the experiences with the preious D Automotie model, the D M is now using 4 scan planes to scan the enironment of the sensor [6]. This allows a compensation of the ehicles pitch angle, as the distance information is aailable for different heights. In the following the technical data are listed as an oeriew ariable scan frequency, 5 to 4 Hz the angle resolution aries with the scan frequency, at 1 Hz the angle resolution is.25, thus proiding 18 measurements per channel and scan ariable scan area, up to 27 range up to 5 m 4 channels, with two distance measurements per channel 2) Scanning In order to obtain the distance to a target, the D M uses a time-of-flight measurement principle. By rotating a prism in the scanner head, a two-dimensional range profile is formed Fig. 4 Scan data of a 4-plane aserscanner Fig. 4 shows the scan data of a 4-plane aserscanner. In front of the aserscanner, a ehicle is parked, while the other (on the right) is moing towards us. ooking at this ehicle, it can be seen ery well how the lower two scan

3 planes hit the front of the ehicle (white and green). Both upper scan planes hit the windshield approx. 1 m behind its front. B. Vehicle model The aserscanner is moed with the ehicle. Therefore the elocity calculation of the objects, d on the aserscanner data, is relatie to the co-ordinate system of the aserscanner. In order to achiee a reliable tracking and classification of the objects, the absolute elocity of the objects in the global co-ordinate system is necessary. To determine the objects absolute elocity the relatie elocity of the object can be combined with the motion of the ehicle. A ehicle model is the to calculate the ehicles motion of the own ehicle, using ESP data. The design of a suitable ehicle model is d on a linear two wheel model, the well known bicycle model, which can perform this task. Four basic assumptions are made in order to achiee the bicycle model [7] No accelerating or braking forces No changing in wheel load aerodynamic forces are neglected Small drift angles (cross acceleration < 4 m/s 2 ) a h? ß Fig. 5 bicycle model h a l h F Sh d l F S F C The equilibrium of lateral forces and the equilibrium of moments are determined as follows ψ& l ψ& lh m ( ψ& + β& ) = CR δ β + CRh β + (1) ψ& l ψ& lh Jgz ψ & = CR l δ β CRh lh β + (2) C J gz R, C Rh α α yaw momentof inertia of the z axis drift angle front/rear free- floating position angle Front wheelsteeringangle, h β δ cornering power ψ& m elocity yaw rate massof the ehicle To calculate the steering angle δ as a function of the steering wheel angle δ, the assumption of a PT 1 behaiour of the steering system is made. δ V T δ & = 1 T δ + steering wheel angle gear ratio delay time V T δ Equations (1), (2) and (3) are resoled and combined in equation (4). C + Rh C β & m ψ CRh lh C && = J gz δ & δ & δ & R R l C Rh lh CR l 1+ 2 m 2 2 CRh lh CR l J gz CR m CR l J gz 1 T V T (3) β ψ& δ δ δ & 1 (4) The ESP system proides the steering wheel angle δ, the steering wheel rate δ & and the yaw rate ψ&. Applying a Kalman-Filtering algorithm to equation (4) the free-floating position angle β can be estimated with a reliable accuracy. C. Object model 1) Object Detection and Tracking Usually the measurement points for one reolution of the rotating head (scan) are diided into clusters, which are assumed to belong to the same object - the so called segments. These segments are represented by seeral parameters, such as left, right and closest point to the sensor, as well as the geometrical centre of graity of all measurement points of the segment. This leads to a massie data reduction. Comparing the segment parameters of the current scan with predicted parameters of known objects from the former scan(s), quite a few of these objects will be recognised. In our case a Kalman Filter predicts the object state including the calculation of the longitudinal and lateral elocity of the object as well. Unknown segments become objects, initiating with default dynamic parameters. 2) Object Classification Object classification is performed by distinguishing between typical object-outlines (static data), like cars, trucks/busses, poles/trees, crash barriers, motorcycles/bicycles and pedestrians. Incorporating

4 additional knowledge from the past and also the dynamic behaiour of the object gien by the tracking algorithm, it is possible to achiee a classification of the objects [8]. Fig. 6 shows a pedestrian, who appears between parking cars and crosses the street. The relatie elocity is displayed at eery object. Using the ehicle elocity, as outlined before, the absolute elocity of the pedestrian is calculated. The moing direction is not displayed, howeer it is integrated in the data set as well. A simple algorithm can classify road users with the typical angular-outline, as those exhibited by cars, using just the geometric data [9]. It is more difficult to classify the pedestrian by looking just at this snapshot. Fig. 7 A passing bicycle in front of a car. Fig. 6 A picture and object data (right) of a pedestrian and parking cars at both sides of a road. To classify a pedestrian its typical moement can be used to differentiate between e.g. a lamppost and a pedestrian. The range of elocity, the typical appearance of the moing legs and the object information from the past are additional parameters for the classification of a pedestrian. If there is not enough information about the object, a hypothesis can be generated, because of the objects current appearance. The temporary assignment is alid, as long as there is no iolation of a limiting parameter. Howeer the classification is checked eery scan to erify the assignment of the specified class. An enironmental model, as described aboe, supports the selection of a suitable class. In future the understanding of the traffic situation could help further to find a classification as well. 3) Object occlusion and reconstruction The standard detection and tracking algorithm had to choose a compromise between the ability of separating the objects, and aoiding the disintegration of an object. Gaps in the object outline, caused by occlusions, normally result in disintegration of the object, as shown in Fig. 8. Using the improed object detection and tracking algorithm including detection of object coering and object reconstruction, the necessary distance to separate two objects from each other is decreased [1]. Segments, belonging to the same object, separated by gaps in the object outline, are fusioned using the improed algorithms, oerruling the simple distance criterion, as shown in Fig , bicycle Scanner Fig. 8 A passing bicycle is occluding parts of the car, which causes the unintended object disintegration into fie new objects. Using this improed algorithm, it is also possible to detect and classify pedestrians which are nearby other classified object. 2, car 17, bicycle Scanner Fig. 9 A segment fusion algorithm combines the fie segments which belong to the cars outline. D. Street model The street model should proide all important information of the road ahead on which the car is driing on. The main parameters are the curature, the dimensions, the number of lanes and the street type [11]. It is also possible to determine the region of interest (ROI), e.g. the road and the related footpath.

5 CONCUSION The aserscanner D M, IBEO s latest deelopment, is able to obsere the ehicles enironment. A reliable detection and tracking of the surrounding objects is performed. These and a classification of the objects, proide data for future actie safety systems. ACKNOWEDGMENTS Parts of this work was financed by the European Commision in the 5 th framework program within the PROTECTOR project with the following partners CRF, DC, MAN, IBEO, SIEMENS, TÜV, DIS, IKA, CSST, TAMAN, RAMOT. REFERENCES [1] Weisser, H.; Schulenberg, P.; Bergholz, R.; ages, U. (1998) Autonomous Driing on Vehicle Test Tracks Oeriew, Motiation and Concept. IEEE International Conference on Intelligent Vehicles, Stuttgart, IV 21, IEEE Intelligent Vehicles Symposium, IV 21 Tokyo, Paper 2-1. [9] Fuerstenberg, K. Ch.; Hipp, J.; iebram, A. (2) A aserscanner for detailed traffic data collection and traffic control. Proceedings of ITS 2, 7th World Congress on Intelligent Transport Systems, ITS 2 Turin, Paper [1] Fuerstenberg, K. Ch.; Willhoeft, V. Pedestrian Recognition in urban traffic using aserscanners. Proceedings of ITS 21, 8th World Congress on Intelligent Transport Systems, ITS 21 Sidney, Paper 551. [11] Sparbert, J.; Dietmayer, K. C. J; Streller, D. ane Detection and Street Type Classification using aser range Images. Proceedings of ITSC 21, IEEE 4th International Conference on Intelligent Transport Systems, ITSC 21 Oakland, page [2] Willhoeft, V.; Ewald, A. (2) aserscanners for Obstacle Detection in Automotie Applications. Proceedings of IV 21, IEEE Intelligent Vehicles Symposium 2, IV 2 Dearborn, Paper IVS-8. [3] ages, U. (2) New Sensor for Stop & Go - Innoatie Approach to Pedestrian Recognition. Proceedings of ITS 2, 7th World Congress on Intelligent Transport Systems, ITS 2 Turin, Paper [4] Fuerstenberg, K. Ch.; Willhoeft, V.; Dietmayer, K. (21) New Sensor for 36 Vehicle Sureillance - Innoatie Approach to Stop & Go, ane Assistance and Pedestrian Recognition. Proceedings of IV 21, IEEE Intelligent Vehicles Symposium, IV 21 Tokio, Paper 5-1. [5] Dietmayer, K. C. J.; Sparbert J.; Streller, D. A System Architecture for the Detection and Modelling of Dynamic Traffic Scenes. Proceedings of ITS 21, 8th World Congress on Intelligent Transport Systems, ITS 21 Sidney, Paper 78. [6] Willhoeft, V.; Fuerstenberg, K. Ch. Quasi-3d Scanning with aserscanners. Proceedings of ITS 21, 8th World Congress on Intelligent Transport Systems, ITS 21 Sidney, Paper 55. [7] Zomotor, A. Fahrwerktechnik Fahrerhalten. 2. Auflage, Vogel Verlag, Würzburg, [8] Dietmayer, K. C. J.; Sparbert J.; Streller, D. Model d Object Classification and Object Tracking in Traffic scenes from Range Images. Proceedings of