QUASI-3D SCANNING WITH LASERSCANNERS

QUASI-3D SCANNING WITH LASERSCANNERS V. Willhoeft, K. Ch. Fuerstenberg, IBEO Automobile Sensor GmbH, vwi@ibeo.de INTRODUCTION: FROM 2D TO 3D Laserscanners are laser-based range-finding devices. They create a range image of the environment. Unlike their smaller brothers, the fixed-beam lasers, they do not measure one direction (1d), but create a range profile of the surrounding. Their performance, such as the scan angle, angular resolution, range and accuracy all vary greatly from scanner to scanner. The Laserscanners available today which are suitable for automotive use are typically 2d scanners. They have only one scan plane in which they detect obstacles. Although this is sufficient for a range of applications, a 3d view with multiple scan planes will enhance most applications and add extra safety to the results of the sensors. The information from extra scan planes becomes especially valuable if the vehicle on which the Laserscanner is mounted is pitching. This pitch movement can be caused by bumpy roads, but varying loads also change the overall pitch angle. If only some additional information is measured without actually creating complete new profiles, the term 2.5d is used throughout this document. An example of a simple 2.5d extension is shown below in Fig.1. Top view Side view Top view Side view Fig. 1: Configuration of 2d (left) and 2.5d (right) Laserscanner FUNCTION PRINCIPLE OF A LASERSCANNER LD A AF: A 2D SCANNER The Laserscanner described here is from IBEOs Ladar Digital A AF series, which is a Laserscanner adapted for automotive use. It is a 2d Laserscanner with a single scan plane. The LD A AF uses a time-of-flight measurement principle to measure the distance to a target. An infrared laser light pulse is emitted by a laser diode and sent towards the target. The reflected beam is then picked up by a photo diode. The time between the emission of the pulse and the reception of the echo signal is measured by a high-speed timer with a resolution of 1/256 m (this equals approx. 10 ps) and an accuracy of ±5 cm (1σ). By rotating its head with a deflection mirror, the measurements form a scan plane with up to 270 opening angle. The angular resolution Fig. 2: LD A AF (3P-Version with mirrors) Page 1 of 7

is 0.25, so that the resulting scan consists of 1080 individual scanpoints. The range of the Laserscanner depends highly on the reflectivity of the target surface, but is typically 100 m on natural, non-reflective targets while being eye-safe with laserclass 1. The LD A AF Laserscanners contain a TMS320C32 DSP computer. This computer runs sensor-internal signal processing algorithms such as the object tracking described later. 2.5D EXTENSION In order to create some additional scan planes, a simple mirror construction may be added to the LD A AF. This extension is called "2.5d" because it does create more than a 2d view, but is not quite what is commonly refered to as "3d". Two products have evolved from the LD A AF, both using two mirrors behind the scanner head to deflect a part of the main scan plane to a "region of interest". The principle of this method is shown on the right side of Fig. 1. The first implementation, the LD A AF (FrontMirror), is Fig. 3: LD A AF (FrontMirror version) shown in Fig. 3. Although its mirrors seem to be transparent, they act as mirrors for the infrared light. This scanner is configured for use in the middle of the vehicle front, so it uses only up to 180 of its 270 scan plane. The rest of the scan area forms two 45 sections at both ends of the scan area. From these areas, approx. 25 of each section are deflected to the front. Since the mirrors are slightly tilted (Fig. 1, right side), these two sections form two additional scan planes, one above and one below the main scan plane. 3D LASERSCANNER: THE MOTIV In the course of the MOTIV project (see [hipp00]), IBEO has developed the prototype of a 3d Laserscanner. It had a fixed lower part for the mouting of the scanner, and a rotating head, containing the laser sender, the receiver APD- Array and the measurement hardware. This hardware was built using the components of the LD A AF, one for each channel. The measured data was sent from the rotating to the fixed part using IrDA infrared transceivers. Like the LD A AFs, the MOTIV has a TMS320C32 DSP for internal signal processing. The Laser is shaped like a vertical standing line. The reflected echo is picked up by the receive lens and focused on the 25-point APD array. To obtain 4 independent scan planes, 24 of these points are used, grouped as 4 x 6 receive diodes. The four channels are picked up by four individual measurement units from the LD Laserscanner, mounted in the scanner head. A special transceiver Fig. 4: The MOTIV Laserscanner board then reads the four different profiles, assembles one combined scan and transmits the scan to the transceiver board in the fixed base of the scanner. The scan is then read by the DSP, which runs the object tracking algorithm. The MOTIV sends object data like the LD A AF, but has Page 2 of 7

some extra information available due to its four scan planes. It has a 175 field of vision with an angular resolution of 0.35, resulting in 500 measurements per scan plane. Like the LD scanners, the MOTIV has a scan frequency of 10 Hz. An image of the resulting 4-profile scan is show in the next section. SCAN DATA The scan data of the laserscanners is shown from a birdview perspective. Fig. 5 (left) shows a scan from a 2d Laserscanner. The scanner covers 270, starting at 180 (straight downward), and scanning clockwise to -90 (to the right). Each red dot is a measured distance. One can easily see the typical clusters of raw data which are the outline of objects seen by the scanner. Fig. 5: Scan data of the LD A AF (2d, left side) and the FrontMirror Version (2.5d, right side) This is the most common configuration and can be used for a range of applications around automated vehicle operation and driver assistance, such as collision avoidance or turn-off and lanechange assists. It can also be used to monitor areas and survey the objects moving there. However, the range of the scanner is effectively limited if the vehicle is pitching. In this case, the scan plane often either points too high upward or down into the ground, limiting the range to approx. 10-20 m around the scanner. Those pitch movements can be caused by rough terrain, speedbumps, load changes etc. To compensate for those pitch movements, the FrontMirror sensor has two additional scan planes. The scan of this sensor (Fig. 5, right side) is shown in the same perspective, but here, all three scan planes are overlaid into one image. The main scan line is shown in blue, while the two extra scan lines are shown in red (upper) and green (lower). In this figure, the vehicle onto which the Laserscanner is mounted is pitching violently, causing both the lower and the main scan line to hit the ground 10 m in front of the scanner. Although the upper scan line does not cover a very wide opening angle, it still sees road markings along the road and an obstacle blocking the road 50 m in front of the vehicle. This information can be vital to continue driving safely. The LD A 3P with its bigger mirrors is the extension of the FrontMirror sensor. It has two much wider extra scan planes, each covering approx. 40. The 3P can be used in the same way as the FrontMirror sensor, to compensate pitch movements. In another application, both mirrors are pointing downward, so that both extra scan planes are monitoring the ground in front of the Laserscanner. This configuration is shown in Fig. 6 (upper right). On even ground, the measurements of the extra scan lines form an X in front of the sensor, as shown in the same image. On uneven ground, the straight lines are bent toward the sensor or away from it, depending on the Page 3 of 7

geometry of the ground. Fig. x (left) shows a scan from a LD A 3P which is standing just right of a steep ramp. The main scan line sees the ramp to the left of the scanner, but cannot decide if this is a ramp or a wall. The first extra scan line (red) forms a sharp turn in front of the scanner. This means a steeply ascending slope on the left side. The second extra scan line (green) shows an almost even line along the slope but slightly elevated, confirming that it is a ramp and not a wall. From the range of the extra scan lines, together with the known range values from the calibration on even ground, the elevation of the single scanpoints can be calculated. Top view Side view Fig. 6: Scan data of a LD A 3P (left), the system configuration (upper right), and the height plot of the extra scan lines (lower right) Fig. 6 (lower right) shows a height plot of the scanpoints of the extra scan lines. The sensor is mounted where the small vehicle is drawn (not up to scale!); each gray scale is 0.1 m in height and 0.5m length, respectively. The steepness of the slope can now be calculated easily from the height gradient of this scan data. This configuration allows the navigation of the vehicle on rough terrain, monitoring the steepness of ascending or descending slopes in front of the vehicle. Also, small obstacles in the vehicles path, such as curbstones or debries, can easily be detected. However, due to the high pitch angle and the thereby limited range of the extra scan lines, the vehicle must be comparatively slow in order to allow the sensor to detect all obstacles. MOTIV SCAN DATA The MOTIV has a configuration like a standard 2d Laserscanner, but it covers a vertical opening angle of approx. 4 with its 4 scan planes. A sample scene is shown in Fig. 7. In Fig. 7: Scan data from the MOTIV sensor. The colors are (from lower to upper scan plane): white, green, blue, red Page 4 of 7

this image, the different planes are displayed in different colors: The lowest one is white, the next is green, blue and the upper one is shown in red. The sensor is at the lower middle of the image. A vehicle is coming from above, passing us (middle of the image). The lower two scan lines are on the vehicle front, while the upper two scan lines hit the windscreen above. All four scan lines measure along the side of the vehicle. The slope on the left (upper left of the image) is the entryway to IBEO, Hamburg. It is slightly ascending from the street. The lowest scan line hits the ground first, then the next one 2-3 m further, and the upper two scan lines measure into the bushes beyond. OBJECT TRACKING All of IBEOs automotive laserscanners contain a DSP for sensor-internal signal processing. This DSP runs a complete object detection and tracking algorithm. This algorithm splits the scan data into objects and tracks those objects through subsequent scans. The result is, instead of the raw scan data, a set of object data with information like position, size, outline and velocity. The object data is sent to the host system on a standard CAN bus. This frees the host computer from the task of isolating the relevant information from the huge amount of scan data that the laserscanners produce. An overview of the standard (2d-) tracking algorithm is shown in Fig. 8. After receiving the scan data, it is split into segments. Segments are clusters of raw data that are believed to belong to one object. Then, the characteristics for each segment are calculated, such as the position, size, and number of scanpoints. At this stage of the algorithm, all those characteristics are purely static values. In parallel, the prediction of the object movement is calculated using the output of the Kalman filter. All objects are extrapolated one step into the future to predict their position in the current scan. Then, the segments of the current scan are matched with the predicted objects, and the best matches are assigned. More than one segment may be assigned to one object, because parts of the object may be blocked from view by some smaller object in the foreground. Finally, the object properties (position, size, velocity, uncertainties) are updated, using the precisely measured position of the assigned segment. This is done by updating the state vector of the object and running this vector through a Kalman filter. Unassigned segments are stored as new objects with default properties. After the object detection and tracking is complete for the scan, the objects are sent to the host computer on a CAN bus. The information for each object consists at least of a set of points on the object outline including the leftmost, rightmost and closest points, a velocity, and uncertainties for all values. All information is given in both x- and y-direction. 2.5D EXTENSION OF THE OBJECT TRACKING Receive scan Segment scan Segment characteristics Object/segment assignment Object filtering (Kalman filter) Object prediction Fig. 8: Overview of the object tracking algorithm Using a 2.5d Laserscanner, additional information becomes available that can be used in the object tracking. To use this information, extensions have to be made to the tracking itself. A typical disadvantage of the 2d tracking is that it is impossible to decide between ground (or a shallow Page 5 of 7

slope) and an obstacle such as a wall. This means that a measurement into the ground - e.g. due to vehicle pitching - or at a wall looks the same for the Laserscanner, and both are tracked and reported as obstacles blocking the path. Using the data from the lower scan plane with its known configuration, a slope or ground can be plausibly detected and removed from the object list (see Fig. 5, right side). In this case, because the main scan line hits the ground or a ramp, objects that have been tracked in the main scan plane are lost from view, so the data from the upper scan plane becomes valuable and is integrated into the object tracking. In Fig. 5, although both the lower and main scan planes hit the ground, the upper scan plane sees an object 50 m in front of the vehicle. If the vehicle is pitching upwards, the same process applies to the lower scan plane which is then used for the object tracking. In the configuration of the LD A 3P, where both extra scan planes are tilted downwards (see Fig. 6), information is generated about the near field only. In order to be effective, the two scan planes must hit the ground quite close to the scanner, typically 2-5 m in front of the sensor. In this distance, the scanpoints are close enough to allow a precise calculation of what is seen on the ground. Both the steepness Fig. 9: Curbstone, seen by the right extra scan plane of ascending or decending slopes and obstacles such as curbstones can be detected with the data from the extra scan planes. Fig. 9 shows the data of the two scan planes on the ground. The left scan plane (red) shows even ground and no obstacles, but the beginning curbstone can easily be detected from the z-shaped right scan plane. Also, the curbstone heigth (10 cm) can be calculated by comparing the measured distances with the expected values for even ground. Using this configuration makes it possible to manouver a vehicle in rough terrain while watching the ground for traps such as slopes or obstacles. 3D-EXTENSION OF THE OBJECT TRACKING Using a 3d Laserscanner such as the MOTIV requires a new design of the object detection and tracking algorithm. Although basic strategies such as the object assignment and Kalman filter can be used, the information from multiple scan palnes offers a whole set of new information. Therefore, the segmentation process which clusters the raw data into separate segments must not only consider neighbors in its own scan plane, but in the other planes as well, effectively making it work in 3d space. The resulting segments do not only have size information in width and length, but also in height. Because the pitch angle of the vehicle is not known to the Laserscanner, this is a relative height information. However, the height will enhance a classification of the objects, if desired. Due to the fact that at least one scan plane is pointing downwards, ground is almost always seen by one scan plane. This means that an effective ground removal algorithm has to be used with a multi-plane Laserscanner. On the MOTIV, this was realised using a heuristic of the caracteristic ground shape (appearance in the scan data) and position, taking into account the neighborhood points. CONCLUSION AND OUTLOOK Classic single-plane Laserscanners are currently used in a range of applications around either automatic vehicles or driver assistance functions. Their big field of vision and excellent measurement accuracy make them well suited for precise measurement tasks. However, adding some extra scan planes by deflecting parts of the single scan plane can greatly increase the flexibility of the scanner, either adding extra safety to the object tracking or allowing the operation Page 6 of 7

under extreme conditions. However, these modifications are only the first step on the way to true multi-line Laserscanners. The MOTIV prototype has proven the capabilities of such a Laserscanner. Currently, the next generation of multi-line Laserscanners is under development in several projects supported by the European Commission, some of which will be presented at this conference. [hipp00] [reich97] [am00] [ae00] [vwi00] REFERENCES Hipp, E.; Reichart, G.: "MOTIV - Fahrerassistenzsysteme". In: Mobilitätsforschung für das 21. Jahrhundert: TÜV Verlag, Köln 2000, S. 85 107, Final presentation on 04./05.05.2000. Reichart, G.: "MOTIV - A cooperative Research Programme for the Mobility in Urban Areas". Proceedings of ITS 97: 4th World Congress on Intelligent Transport Systems, 21. - 24.10.1997, ICC Berlin. C. Ameling, A. Kirchner: "The emergency braking module for an electronic copilot design and first results", 9 th IFAC symposium 2000, Braunschweig A. Ewald, V. Willhoeft: "Object Detection with Laserscanners for Automotive Applications", 9 th IFAC symposium 2000, Braunschweig V. Willhoeft: "Laserscanners for Automotive Applications in the AF Project", ITS2000, Torino Page 7 of 7