Image processing techniques for driver assistance. Razvan Itu June 2014, Technical University Cluj-Napoca

Size: px

Start display at page:

Download "Image processing techniques for driver assistance. Razvan Itu June 2014, Technical University Cluj-Napoca"

Audra McKenzie
5 years ago
Views:

1 Image processing techniques for driver assistance Razvan Itu June 2014, Technical University Cluj-Napoca

2 Introduction Computer vision & image processing from wiki: any form of signal processing for which the input is an image computer vision - subfield of artificial intelligence in computer science sometimes referred as the emulation of human vision by a machine

Popular libraries and software OpenCV - most popular library, the internet is full of examples & tutorials, good documentation FastCV - library for mobile only, from Qualcomm the makers of Snapdragon

3 Popular libraries and software OpenCV - most popular library, the internet is full of examples & tutorials, good documentation FastCV - library for mobile only, from Qualcomm the makers of Snapdragon processors found on most of today's smartphones matlab - also used, especially in the research field, provides one of the best camera calibration modules, very good to quickly prototype & test algorithms

Adaptive cruise control Lane change assistance Collision avoidance system (pre-crash) Traffic sign recognition Vehicular communication systems ADAS ADAS - Advanced

4 Adaptive cruise control Lane change assistance Collision avoidance system (pre-crash) Traffic sign recognition Vehicular communication systems ADAS ADAS - Advanced Driver Assistance Systems Systems to help the driver in the driving process, examples: Automatic braking, parking, etc. Image source: Continental Automotive Systems

cameras Image source: Texas Instruments sensors used to capture proximity/ distance to objects:

5 ADAS - Computer vision research 2 main categories for ADAS using image processing: single camera = monocular stereo cameras = binocular (2 or more cameras) some systems use more sensors in addition to the cameras Image source: Texas Instruments sensors used to capture proximity/ distance to objects: radar(radio Detection And Ranging) lidar(light Detection And Ranging) sonar(sound Navigation And Ranging) Image source: Bosch

Monocular not so popular, single camera = cheap, maybe improved speed/performance (you process half the number of frames/images compared to a stereo setup) reduced

6 Monocular not so popular, single camera = cheap, maybe improved speed/performance (you process half the number of frames/images compared to a stereo setup) reduced accuracy, one camera means no depth perception/knowledge Image source: SAE International mostly used to: detect lanes, detect traffic signs, maybe traffic light(s)?

Stereo vision most usages in automotive industry, expensive systems use special cameras that are calibrated & setup for the specific hardware that is being used high accuracy, the 2 cameras mimic the

7 Stereo vision most usages in automotive industry, expensive systems use special cameras that are calibrated & setup for the specific hardware that is being used high accuracy, the 2 cameras mimic the human vision, the two eyes add a "depth" perception of the surroundings good usages in detection & tracking of the obstacles Image source: Subaru EyeSight System good accuracy because of the two camera setup that adds depth mono vs stereo: in monocular approaches there are assumptions of a flat road, a constant pitch angle or absence of a roll angle

Stereo vision - continued More on depth: you can get depth information from a pair of images of same scene Create a disparity map Disparity refers to difference in

8 Stereo vision - continued More on depth: you can get depth information from a pair of images of same scene Create a disparity map Disparity refers to difference in image location of an object seen by the left & right eyes => in computer vision, disparity = the difference in coordinates of similar features within a pair of stereo images

9 Stereo vision - continued Disparity map: Mars rover uses it to navigate through obstacles! Image source: NASA

10 Basic algorithms & approaches 1. Camera calibration using special software and by taking repeated photographs of a known pattern OpenCV provides a calibration software, most used is from the matlab through calibration process you get to know some of the camera parameters: intrinsic & extrinsic Image source: dreamincode.net forums Image source:

search the RED color to detect the STOP sign shape based: detect the round shape of

contain both good & bad examples (ex: use Viola Jones detector) Viola Jones

11 Basic algorithms & approaches! 2. Traffic sign recognition Image source: Mercedes-Benz color based: for example search the RED color to detect the STOP sign shape based: detect the round shape of the SPEED LIMIT sign machine learning based: use a set of training images that contain both good & bad examples (ex: use Viola Jones detector) Viola Jones detector works by sliding detection window across image At each position, decide if desired object inside window Uses Haar features Image source: Siemens

12 Basic algorithms & approaches!! 3. Lane detection apply filter to image, ex Canny to find edges apply a Hough Transform to detect lines filter these lines input image Images source:

13 !! Basic algorithms & approaches 4. Obstacle detection & tracking Monocular appearance based methods: use colour and shape to detect image regions that belong to obstacles Monocular motion based methods: use motion of image features and optical flow optical flow = pattern of apparent motion of image objects between two consecutive frames caused by the movement of object or camera (definition adapted from wiki) you have a set of points in an image, find the same points in another image Stereo methods use the disparity map to get depth of image features. => detection of the ground plane (if it is unknown) and the detection of obstacles = any feature not on the ground plane Image source: OpenCV

research & proposed solutions, some examples: Feature

features =(sum of pixels in black region) - (sum of

subtraction Many other methods, most based on stereo

14 !! Basic algorithms & approaches 4. Obstacle detection & tracking - continued A lot of research & proposed solutions, some examples: Feature based detection (ex: using Haar features): Haar features =(sum of pixels in black region) - (sum of pixels in white region) Detection by background subtraction Many other methods, most based on stereo imaging Image source: behance.net Image source: atlas.web.ua.pt Image source:

15 Other tips only process what's needed/required from an image, ex: processing road lanes, remove the sky/horizon from the image, you only want to process the road pixels (this is called Region Of Interest = ROI) use pre-defined, pre-computed values, for example you calibrate the camera once, and store the obtained results, re-use them when needed

16 Drive Assist App monocular system based on Android devices fast, cheap & robust object detection (why? => use a mobile phone, portable, everyone has a phone, only requirement is to use a windshield mount for the phone not new, or breakthrough app, but performs better than competition real time processing & feedback (displaying the results)

17 Drive Assist App Some testing results & effective range Images source:

18 Drive Assist App Performance test

19 Drive Assist - Comparison

20 Drive Assist - Demo

21 Q&A Thank you! Razvan Itu twitter.com/razvan

Machine learning based automatic extrinsic calibration of an onboard monocular camera for driving assistance applications on smart mobile devices

Technical University of Cluj-Napoca Image Processing and Pattern Recognition Research Center www.cv.utcluj.ro Machine learning based automatic extrinsic calibration of an onboard monocular camera for driving