Autonomous Vehicles:

Transcription

1 Autonomous Vehicles: Research, Design and Implementation of Autonomous Vehicles Research Group - GPVA Tutorial page: html Dr. Fernando S. Osório - Applied Computing Post Dr. Christian R. Kelber PIPCA Post-Grad. Program PIPCA - Electrical Engineering / Computer Eng. Dr. Cláudio R. Jung - Applied Computing M.Sc.. Program PIPCA M.Sc. Farlei Heinen - Computer Engineering B.Sc. (Director) Grupo de Pesquisas em Veículos Autônomos Autonomous Vehicles Research Group - Unisinos 1 Veículos Autônomos Inteligentes Introduction Robotic: Automatons, Mobile Robots and Autonomous Robots Perception, Action, Locomotion e Communication and Intelligence Intelligent Vehicles Technologies for Vehicle Automation pyramid Intelligent of Autonomous Vehicles : Computational Architectures Simulation of Autonomous Vehicles Computer Vision Practical Applications Agenda 2 1

2 CONTROL: : Computational Architectures => From where do I start? Modeling and Simulation Models: - Sensorial Models - Actuator Models - Kinematics Models - Environment Models - A.I. Models (Path Planning, Agents,...) Simulation: - Validate models - Test robustness - Improve design : Computational Architectures 17 CONTROL: : Computational Architectures => From where do I start? Modeling and Simulation Scientific American January 2007 : Computational Architectures 17 2

3 CONTROL: : Computational Architectures Sensorial Models Kinematics Models C2 C1 C0 M1 C3 C4 C5 M2 Sensorial Model: Sonar Infrared Radar, Compass, Odometer Kinematics Model: Differential Aeckerman C7 C6 φ Y Aeckerman θ X θ = V / L * Sin (Φ) X = V * Cos (Φ) * Cos (θ) Y = V * Cos (Φ) * Cos (θ) : Computational Architectures 18 CONTROL: : Computational Architectures Sensorial Models Kinematics Models Robotic : * Reactive * Deliberative * Hierarchical * Hybrid Environment Maps * Building Maps * Path Planning * SMPA - Sense Model Plan Act Problems: * Complex tasks * Avoid Obstacles: Static / Mobile - Unexpected obstacles * Robot actual position estimation - Where am I? : Computational Architectures 17 3

4 CONTROL: : Computational Architectures Complexity... P.Bessièrre * Action Planning * Ability to Perceive the Environment * Ability to Decide * Ability to Act * High Level Tasks Planning * Reaction: Sensorial-Motor * Estimate Actual and Future States * Adaptation and Learning * Robustness * Unexpected Situations => From where do I start??? : Computational Architectures 20 CONTROL: : Computational Architectures Complexity... : Computational Architectures 21 4

5 CONTROL: : Computational Architectures Complexity... Simplify! How? : Computational Architectures 22 CONTROL: REACTIVE Architecture Complexity... Simplify! How? MIT - OCW Reactive: Sensorial-Motor Integration Able to Act Able to Perceive the Environment Able to React Reactive Architecture 23 5

6 CONTROL: REACTIVE Architecture Reactive: Sensorial-Motor Integration Reactive S2 S1 S0 M1 S7 S3 S4 S5 M2 S6 IF S1 < Threshold and S2 < Threshold and S3 < Threshold and S4 < Threshold THEN Action (Go_Forward) IF S1 < Threshold and S2 < Threshold and S3 > Threshold and S4 > Threshold THEN Action (Turn_Left) IF S2 > Threshold and S3 > Threshold and S2 > S3 and S1 > S4 THEN Action (Turn_Right) Sensorial-Motor: Perceive => Act Reactive 24 CONTROL: REACTIVE Architecture Reactive: Sensorial-Motor Integration Reactive Robotic Lawn Mowers - Toro imow - Husqvarna Auto Mower - Automower Electrolux Sensorial-Motor: Avoid Obstacles, Wall Following, Wander Electrolux Trilobite Robotic Vacuum Cleaner ZA1 Reactive 25 6

7 CONTROL: REACTIVE Architecture Reactive: Sensorial-Motor Integration Reactive Sensorial-Motor: - Avoid Obstacles - Wall Following - Wander Simple behaviors Robustness? Complex tasks? Reactive 26 CONTROL: DELIBERATIVE Architectures Deliberative: Planning + Action Deliberative SIMROB (2D) - Map - Configuration Space - Visibility Graph - Optimized Path (Dijkstra) Robotic Arm: Pre-defined paths Deliberative Architecture 27 7

8 CONTROL: DELIBERATIVE Architectures Deliberative: Planning + Action Deliberative Tarefas Complexas... Robustez? Imprevistos? Ambiente pouco conhecido? Deliberative Geometric Map based Navigation: Planning: Graph+Dijkstra, A* Grid based Navigation: Planning: A* 28 CONTROL: HIERARCHICAL and HYBRID Architectures Combining: Deliberative + Reactive Hierarchical and Hybrid Hierarchical and Hybrid Hierarchical : - Layers - Priorities - Information Exchange Figures From: Brooks, R. A. MIT A.I. Memo 864 Sept Brooks - Subsumption Architecture 29 8

9 CONTROL: HIERARCHICAL and HYBRID Architectures Building the Environment Map: SMPA - SENSE / MODEL / PLAN / ACT Hierarchical and Hybrid MODEL S E N S E PLAN: AStar Dijkstra ACT! Hierarchical and Hybrid Sebastian Thrun / CMU 30 CONTROL: Simple HYBRID Architectures Hybrid Farlei Heinen PLAN: Dijkstra ACT & ReACT Hybrid 31 9

10 CONTROL: Simple HYBRID Architectures Hybrid Farlei Heinen References: SEVA2D / SEVA3D Autonomous Vehicle Parking TASK PLANNING & CONTROL: Finite State Automata (FSA) Artificial Neural Net (ANN) ACTION: Sense, Act React (change state) SEVA-A (Automaton) Farlei Heinen SEVA-N (Neural) Farlei Heinen Fernando Osório Luciane Fortes Milton Heinen Publications: SBRN 2002 WCCI 2006 Hybrid 32 CONTROL: Simple HYBRID Architectures Hybrid SimRob3D Simulation SEVA3D 3D World Kinematics: Estimation of Position and Orientation Robot Model Perception: Sensor Simulation Motor Actions Sensorial Information Commands Sensors : SEVA3D-A (FSA) SEVA3D-N (Neural) Visualization Hybrid 32 10

11 CONTROL: : Computational Architectures Sensorial Models Kinematics Models Robotic : * Reactive * Deliberative * Hierarchical * Hybrid Environment Maps * Building Maps * Path Planning * SMPA - Sense Model Plan Act Problems: * Complex tasks * Avoid Obstacles: Static / Mobile - Unexpected obstacles * Robot actual position estimation - Where am I? 33 PROBLEMS: System Task Execution * Avoid Obstacles - Known Obstacles - Unknown Obstacles (static / no movement) - Unknown Obstacles (dynamic / moving objects) * Positioning - How to determine the exact actual position of the robot? - How to maintain the control of exact position after displacement? - Error and Imprecision: Move forward / Rotate 34 11

12 PROBLEMS: * Avoid Obstacles System Task Execution - Known Obstacles - Unknown Obstacles (static / no movement) - Unknown Obstacles (dynamic / moving objects) 35 PROBLEMS: * Positioning System Task Execution - How to determine the exact actual position of the robot? - How to maintain the control of exact position after displacement? - Error and Imprecision: Move forward / Rotate 36 12

13 COHBRA / HyCAR Robot [SimRob3D] Sensors Actuators Layers Behaviors Environment Representation Maps Positioning Estimator (Monte Carlo) Sequencer Polygonal Modules Path Planning Grid Topological and Semantic Shared Memory COHBRA e Híbrido de Robôs Autônomos HyCAR - Hybrid for Autonomous Robots 37 COHBRA / HyCAR [SimRob3D] Simulation using SimRob3D 38 13

14 COHBRA / HyCAR [SimRob3D] Simulation using a static environment Position estimation based on Monte Carlo Localization Method 39 COHBRA / HyCAR [SimRob3D] Simulation using a static environment Position estimation based on Monte Carlo Localization Method 40 14

15 COHBRA / HyCAR [SimRob3D] Simulation using a static environment Environment was changed related to the original map Internal robot representation is different from actual world configuration 41 COHBRA / HyCAR [SimRob3D] Simulation using a dynamic environment (mobile obstacles) 42 15

16 COHBRA / HyCAR [SimRob3D] Position estimation based on Monte Carlo Method: Robot was moved, starting in a new and unknown position 43 COHBRA / HyCAR [SimRob3D] Virtual Environment: 3D Realistic Environment SimRob3D Simulation Tool 44 16

17 COHBRA / HyCAR [SimRob3D] Virtual Environment: 3D Realistic Environment SimRob3D Simulation Tool 45 COHBRA / HyCAR [SimRob3D] SEVA 3D SimRob3D Simulation Tool 46 17

18 Intelligent Autonomous Robots and Vehicles << Intelligence >> * Task and Actions Planning * Ability to Perceive the Environment * Ability to Decide * Ability to Act * High Level Tasks Planning * Reaction: Sensorial-Motor Integration * Estimate Actual and Future States: Environment + Behavior = Interaction * Adaptation and Learning * Robustness: Unexpected Situations Next steps... GPVA 47 Intelligent Autonomous Robots and Vehicles << Intelligence >> * Task and Actions Planning * Ability to Perceive the Environment * Ability to Decide * Ability to Act * High Level Tasks Planning * Reaction: Sensorial-Motor Integration * Estimate Actual and Future States: Environment + Behavior = Interaction * Adaptation and Learning * Robustness: Unexpected Situations Next steps... DARPA Challenge - Desert (2004, 2005) DARPA Challenge - Urban (2007) GPVA 47 18

19 Intelligent Autonomous Robots and Vehicles << Intelligence >> * Task and Actions Planning * Ability to Perceive the Environment * Ability to Decide * Ability to Act * High Level Tasks Planning * Reaction: Sensorial-Motor Integration * Estimate Actual and Future States: Environment + Behavior = Interaction * Adaptation and Learning * Robustness: Unexpected Situations Next steps... Computational Vision Computational Vision 48 Intelligent Autonomous Robots and Vehicles << Intelligence >> Computational Vision Path following: - Follow Me, Lane Follow Avoid danger situations: going out of the track - Lane Detection Obstacle detection: pedestrians, cars, etc Traffic signs detection and recognition Visual Navigation (Based on Images) Computational Vision 49 19

20 Intelligent Autonomous Robots and Vehicles << Intelligence >> Computational Vision Lane Follow Lane Departure Detection Follow Me Computational Vision 50 Image Database: Path defined by a sequence of image Visual Navigation Reference ImagemAtual NCC (match) PróximaImagem Next Imagemcapturada Robot acquired image pelorobô Navigation based on Monochromatic Images [Jones et al. 1997] Algorithm: NCC Normalized Cross-Correlation Correlation Visual Navigation 51 20

21 Visual Navigation [Matsumoto et al. 1996] Matlab Code IR: Reference Image ICR: Image Captured by the robot [Righes 2004, 2005] Visual Navigation 52 Visual Navigation [Righes 04] Visual Navigation 53 21

22 Visual Navigation Mobile Robot Localization and Mapping with Uncertainty using Scale-Invariant Visual Landmarks Stephen Se, David Lowe, Jim Little (UBC, CA) Algorithm: SIFT Reference Int. Journal of Robotics Research Vol. 21, No. 8, August 2002, pp , Visual Navigation 54 Visual Navigation Omnidirectional Cameras Visual Navigation 55 22

23 Aerial Visual Navigation Vision System for Unmanned Aerial Vehicles Correlation: Satellite image and Helicopter Results... Not good at all! Visual Navigation 55 Aerial Visual Navigation Vision System for Unmanned Aerial Vehicles Referential Correlation in the Crossing Point Using helicopter only images Very Good Match! Visual Navigation 23

24 Vehicle Visual System Vision system used to identify traffic signs [F.Alves 2003] [H]SI RGB Color HSV Color Color based sign segmentation Artificial Neural Network Recognition Visual Systems Multiple Vehicles: Fire fighting squad Planning, Navigation, + Strategy, Cooperation Visual Systems 24

25 Google: Veículos Autônomos 25