Pattern Recognition for Autonomous. Pattern Recognition for Autonomous. Driving. Freie Universität t Berlin. Raul Rojas

Transcription

1 Pattern Recognition for Autonomous Pattern Recognition for Autonomous Driving Raul Rojas Freie Universität t Berlin

2 FU Berlin Berlin 3d model from Berlin Partner

3 Freie Universitaet Berlin

4 Outline of the talk Outline of the talk The Urban Grand Challenge Drive-by by-wire: the hardware The sensors Pattern recognition Control Final reflection: a short history of AI

5 Urban Grand Challenge 2007 Urban Grand Challenge 2007 We have registered as participants in the DARPA Urban Grand Challenge 2007 The race will cover 60 miles in a city environment Maximum speed is 35 mph Vehicles must respect traffic rules Prize for the winner: 2 Million Dollars

6

7 New safety systems New safety systems

8 I Hardware I Hardware New cars are robots A computer screen will become standard Brake, gas, and steer by wire will be standard (through the CAN bus) Three buses: automotive electronics, entertainment, and telecommunications Programmable cars Cars will learn your driving style and adapt

9 Essential Components for an Autonomous Car GPS/IMU (1m accuracy DGPS) (Antennas on top) - 2 rotating lasers degree view - 1 revolution per minute Rotating LIDAR Omni-Video cameras - IBM Cell Blade server - Batteries LIDAR Drive by wire Communication for external control (WiFi and two radio bands)

10 Drive by wire Drive by wire

11 Computing Power Computing Power Four IBM dual core boards: a) for Navigation b) for computer vision c) for laser scanners d) for logging data and laptops as interface One Kilowatt needed from an additional generator

12 Safety

13 II Sensors II Sensors GPS and IMU Navigation Laser scanners one dimensional 3D Laser scanner Video Cameras

14 GPS Positioning IMU: : Inertial Measurement Unit generates a true representation of vehicle motion in all three axes Military grade IMUs can cost millions of dollars

15 GPS and IMU integration: Kalman Filter Applanix

16 Frontal Laser Scanner Frontal Laser Scanner Ibeo

17 Fraunhofer 3D Laser Scanner Half a revolution per second

18 Point cloud from scan Point cloud from scan

19 Omnicamera

20 Video with the omnicam

21 III Pattern recognition aspects III Pattern recognition aspects Colors Street Segmentation Shadows Removing from images 3D structure Recognizing moving or static objects In the future: Recognizing traffic signs

22 Computer vision: segmenting Computer vision: segmenting streets

23 Removing shadows Removing shadows log(b/g) log(r/g) Log-opponent chromaticities for 6 surfaces under 9 spectra

24 Rotate colors using principal Rotate colors using principal components log(b/g) log(r/g) aa Color opponents: B/G, R/G Finlayson, Hordley Drew, 2002 In the new color space, the measurement is independent of the spectrum

25 With a real camera With a real camera Can be improved 1 processing the edges aa

26 Clustering moving objects Clustering moving objects Video Ulm Ibeo

27 IV Control IV Control Simulator for vehicle dynamics High-level navigation planner Low-level reactive control

28 Dynamics Simulator Dynamics Simulator

29 The car driving The car driving

30 V Final reflection V Final reflection When there will be autonomous cars? But: will there be autonomous cars? A short history of AI The road to the future

31 Driving is a cognitive task Driving is a cognitive task When we drive, our cognitive apparatus is processing tons of information

32 A short history of AI A short history of AI 1950 Chess playing 1960 Computer vision: recognize objects brute force wins: compute all possible moves (for a certain depth) brute force works only partially 1997 world champion defeated 2007: computers cannot recognize objects

33 Solution: map the whole city Solution: map the whole city Berkeley mapped with laser scanners and video cameras

34 Map everything Map everything All streets and lanes All intersections All traffic lights All lane lines All traffic signs And let cars communicate And tell the autonomous car where they are

35 Intermediate step: Driver Intermediate step: Driver Assistance Systems

36 Schedule for our project Schedule for our project April 2007: Vehicle operational in Berlin June 2007: Demo of the autonomous car in San Antonio October 2007: qualification event in California November 2007: Grand Challenge race in California

37 Southwest Research Institute in Southwest Research Institute in San Antonio

38 Conclusions Modern cars are robots Multiple sensors will be integrated in future cars: advanced GPS, video cameras and LIDAR Hybrid cars have enough power for advanced electronics Developing an autonomous car is becoming more of a software problem