AASS, Örebro University

Transcription

1 Henrik Andreasson, Abdelbaki Bouguerra, Marcello Mobile Cirillo, Robotics Dimitar Dimitrov, and Olfaction Dimiter Driankov, Lab, Martin Magnusson, Federico Pecora, and Achim J. Lilienthal AASS, Örebro University 1

2 Content 1. Transport 2

3 Content 1. Transport 2. The SAUNA Project o Safe Autonomous Navigation 3

4 Content 1. Transport 2. The SAUNA Project 3. Sensors 4

5 Content 1. Transport 2. The SAUNA Project 3. Sensors 4. Rich 3D Perception o what and why? 5

6 Content 1. Transport 2. The SAUNA Project 3. Sensors 4. Rich 3D Perception 5. Rich 3D in SAUNA 6

7 1 Professional Service Robots for Transport Applications at AASS 10

8 3. 1. Major Projects at the AASS MR&O Lab ( Application Area Mobile Robot Olfaction Robot Vision Safe Navigation & Planning Research Area Rich 3D Perception Other Industrial Vehicles SAVIE SaNa SAUNA ALL-4-eHAM ELWAYS Autonomous Unloading of Containers Grasping and Manipulation RobCab RobLog HANDLE Robosanis Artificial Olfaction Systems Gasbot DIADEM 11

9 3. 1. Major Projects at the AASS MR&O Lab ( Application Area Mobile Robot Olfaction Robot Vision Safe Navigation & Planning Research Area Rich 3D Perception Other Industrial Vehicles SAVIE SaNa SAUNA ALL-4-eHAM Autonomous Unloading of Containers Grasping and Manipulation RobCab RobLog Artificial Olfaction Systems 12

10 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) speed x 2 13

11 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) o environment with a dynamic "background" 14

12 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) o environment with a dynamic "background" o requires 3D sensing 1 meter drop to the railway tracks 15

13 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) Wheel Loaders (VolvoCE, VolvoTech, NCC) 17

14 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) Wheel Loaders (VolvoCE, VolvoTech, NCC) Mining Vehicles (Atlas Copco, Fotonic) picture from the actual mine! 18

15 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) Wheel Loaders (VolvoCE, VolvoTech, NCC) Mining Vehicles (Atlas Copco, Fotonic) Hospital Transport Vehicles (RobCab) 20

16 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) Wheel Loaders (VolvoCE, VolvoTech, NCC) Mining Vehicles (Atlas Copco, Fotonic) Hospital Transport Vehicles (RobCab) Garbage Bin Collection and Cleaning (RoboTech) 24

17 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) Wheel Loaders (VolvoCE, VolvoTech, NCC) Mining Vehicles (Atlas Copco, Fotonic) Hospital Transport Vehicles (RobCab) Garbage Bin Collection and Cleaning (RoboTech) Requirements o perception o manipulation (for picking up and unloading material / goods) o planning and execution o safe navigation 27

18 1. Professional Service Robots for Transport Applications Forklift Trucks (Danaher Motion, Linde MH, Stora Enso) Wheel Loaders (VolvoCE, VolvoTech, NCC) Mining Vehicles (Atlas Copco, Fotonic) Hospital Transport Vehicles (RobCab) Garbage Bin Collection and Cleaning (RoboTech) Requirements o perception o manipulation (for picking up and unloading material / goods) o planning and execution o safe navigation SAUNA! 28

19 2 SAUNA Leading Edge Research in Safe Autonomous Navigation 30

20 2. SAUNA Mobile Service Vehicles for Autonomous Transportation o transportation of goods by one or more vehicles o time and resource constraints o non-holonomic vehicles 31

21 2. SAUNA Mobile Service Vehicles for Autonomous Transportation o transportation of goods by one or more vehicles o time and resource constraints o non-holonomic vehicles Challenges o fleets of mixed autonomous and human-operated vehicles o high speeds (up to km/h) o rich 3D perception for enhanced safety and performance o automated mission planning capabilities at several levels of abstraction o flexible operation, accommodate run-time changes o collision avoidance throughout mission planning, trajectory computation and execution 32

22 2. SAUNA Scientific Objectives o Rich 3D Perception (depth + additional information)» mapping, localization» object identification and localization» real-time performance» long-term operation 33

23 2. SAUNA Scientific Objectives o Rich 3D Perception (depth + additional modalities)» mapping, localization» object identification and localization» real-time performance» long-term operation o Safe Motion» obstacle detection 34

24 2. SAUNA Scientific Objectives o Rich 3D Perception (depth + additional modalities)» mapping, localization» object identification and localization» real-time performance» long-term operation o Safe Motion ( very reliable collision avoidance)» tracking of vehicles and humans predicted collisions our vehicle a v other vehicle 35

25 2. SAUNA Scientific Objectives o Rich 3D Perception (depth + additional modalities)» mapping, localization» object identification and localization» real-time performance» long-term operation o Safe Motion ( very reliable collision avoidance)» tracking of vehicles and humans» collision avoidance (trajectory modification) our vehicle» real-time response at medium and high speeds (up to 30km/h) a v other vehicle 36

26 2. SAUNA Scientific Objectives o Rich 3D Perception (depth + additional modalities)» mapping, localization» object identification and localization» real-time performance» long-term operation o Safe Motion ( very reliable collision avoidance)» tracking of vehicles and humans» collision avoidance (trajectory modification)» real-time response at medium and high speeds (up to 30km/h) o Flexible Operation (no fixed action plans)» automated mission planning process (both mission and motion planning)» multiple types of requirements/constraints» incomplete prior knowledge 37

27 2. SAUNA Realization o in simulation o in real-world prototype robots 40

28 2. SAUNA Assumptions on the Environment o minimal infrastructure 42

29 2. SAUNA Assumptions on the Environment o minimal infrastructure High-Speed Navigation without Additional Infrastructure (MALTA) speed x 1 43

30 2. SAUNA Assumptions on the Environment o minimal infrastructure o environment is 3D, but navigation is in 2D» 3D workspace map 44

31 2. SAUNA Assumptions on the Environment o minimal infrastructure o environment is 3D, but navigation is in 2D» 3D workspace map» planning in 2D locally flat environment 45

32 2. SAUNA Assumptions on the Environment o minimal infrastructure o environment is 3D, but navigation is in 2D o fully dynamic objects» other vehicles» humans» navigation areas can be blocked 46

33 2. SAUNA Assumptions on the Environment o minimal infrastructure o environment is 3D, but navigation is in 2D o fully dynamic objects o slow dynamic changes» e.g., piles of objects/gravel 47

34 2. SAUNA Assumptions on the Environment o minimal infrastructure o environment is 3D, but navigation is in 2D o fully dynamic objects o slow dynamic changes o incomplete prior knowledge about the whereabouts of objects to transport 48

35 3 Sensors 56

36 3. Sensors for Autonomous Vehicles Sensors for Autonomous Vehicles o laser scanner (2D LRF) 57

37 3. Sensors for Autonomous Vehicles Sensors for Autonomous Vehicles o laser scanner (2D LRF) o laser scanner (3D LRF) 58

38 3. Sensors for Autonomous Vehicles Sensors for Autonomous Vehicles o laser scanner (2D LRF) o laser scanner (3D LRF) o laser scanner (3D a-lrf) 59

39 3. Sensors for Autonomous Vehicles Sensors for Autonomous Vehicles o laser scanner (2D LRF) o laser scanner (3D LRF) o laser scanner (3D a-lrf) o TOF camera Fotonic B70 Data Fotonic B70 ( 5k ) 60

40 3. Sensors for Autonomous Vehicles Sensors for Autonomous Vehicles o laser scanner (2D LRF) o laser scanner (3D LRF) o laser scanner (3D a-lrf) o TOF camera o range sensor + perspective camera ( Rich 3D) courtesy B. Huhle 61

41 3. Sensors for Autonomous Vehicles Sensors for Autonomous Vehicles o laser scanner (2D LRF) o laser scanner (3D LRF) o laser scanner (3D a-lrf) o TOF camera o range sensor + perspective camera ( Rich 3D)» Kinect sensor ( Rich 3D) Kinect Data Kinect ( 0.2k$) 63

42 3. Sensors for Autonomous Vehicles Sensors for Autonomous Vehicles o laser scanner (2D LRF) o laser scanner (3D LRF) o laser scanner (3D a-lrf) o TOF camera o range sensor + perspective camera ( Rich 3D)» "industrial Kinect" 64

43 4 Rich 3D Perception 66

44 4. What is Rich 3D? Rich 3D Perception o rich 3D = spatial data augmented with additional information» additional dimensions: e.g. colour coloured point cloud (example of rich 3D data) [Andreasson, 2005] 67

45 4. What is Rich 3D? Rich 3D Perception o rich 3D = spatial data augmented with additional information» additional dimensions: e.g. colour coloured point cloud (example of rich 3D data) [Andreasson, 2005] 68

46 4. What is Rich 3D? Rich 3D Perception o rich 3D = spatial data augmented with additional information» additional dimensions: e.g. colour, temperature "thermal" point cloud (example of rich 3D data) [Andreasson, 2006] 70

47 4. What is Rich 3D? Rich 3D Perception o rich 3D = spatial data augmented with additional information» additional dimensions: e.g. colour, temperature, remission, semantic label, confidence estimate, x i = (x i, y i, z i, r (1) i,..., r (n) i) 71

48 4. Why Rich 3D Perception? Why do we need "rich" 3D? 73

49 4. Why Rich 3D Perception? Why do we need "rich" 3D? o helps to establish the identity of obstacles» contains more information» can be important to maintain people tracks» can help to identify drivable areas 74

50 4. Why Rich 3D Perception? Why do we need "rich" 3D? o helps to establish the identity of obstacles o improved localization in areas without geometrical features 75

51 4. Why Rich 3D Perception? Why do we need "rich" 3D? o helps to establish the identity of obstacles o improved localization in areas without geometrical features o object separation in cluttered scenes may work with 3D perception... 76

52 4. Why Rich 3D Perception? Why do we need "rich" 3D? o helps to establish the identity of obstacles o improved localization in areas without geometrical features o object separation in cluttered scenes... here rich 3D is necessary 77

53 4. Why Rich 3D Perception? Why do we need "rich" 3D? o helps to establish the identity of obstacles o improved localization in areas without geometrical features o object separation in cluttered scenes o additional dimensions can carry the meaning of a map» classification labels, uncertainty 78

54 4. Why Rich 3D Perception? Why do we need "rich" 3D? o helps to establish the identity of obstacles o improved localization in areas without geometrical features o object separation in cluttered scenes o additional dimensions can carry the meaning of a map o may even allow for more compact representations (because it provides ways to instantaneously identify the most important information in the "rich 3D space") 79

55 5 Rich 3D Perception in SAUNA 81

56 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o (1) develop algorithms to create and maintain a single compact and truthful rich 3D workspace model 89

57 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o (1) create/maintain compact and truthful rich 3D map o (2) localize within rich 3D workspace model 90

58 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o (1) create/maintain compact and truthful rich 3D map o (2) localize within rich 3D workspace model o (3) detect obstacles wrt rich 3D workspace model 91

59 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o (1) create/maintain compact and truthful rich 3D map o (2) localize within rich 3D workspace model o (3) detect obstacles wrt rich 3D workspace model 92

60 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o (1) create/maintain compact and truthful rich 3D map o (2) localize within rich 3D workspace model o (3) detect obstacles wrt rich 3D workspace model 93

61 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o (1) create/maintain compact and truthful rich 3D map o (2) localize within rich 3D workspace model o (3) detect obstacles wrt rich 3D workspace model 94

62 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o (1) create/maintain compact and truthful rich 3D map o (2) localize within rich 3D workspace model o (3) detect obstacles wrt rich 3D workspace model o (4) identify drivable areas in rich 3D workspace model 95

63 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o (1) create/maintain compact and truthful rich 3D map o (2) localize within rich 3D workspace model o (3) detect obstacles wrt rich 3D workspace model o (4) identify drivable areas in rich 3D workspace model 96

64 5. Rich 3D Perception => Produce Really Useful Maps Rich 3D Perception in SAUNA Expected Outcome o Create and maintain a compact and useful rich 3D workspace model and use it! 100

65 Henrik Andreasson, Abdelbaki Bouguerra, Marcello Mobile Cirillo, Robotics Dimitar Dimitrov, and Olfaction Dimiter Driankov, Lab, Martin Magnusson, Federico Pecora, and Achim J. Lilienthal AASS, Örebro University 171