UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT RECONASS

Transcription

1 UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT RECONASS Norman Kerle. ESA Department ITC-OOA-Group (with materials by Markus Gerke, Jorge Fernandez and Anand Vetrivel)

2 ITC/ University Twente International Institute for Geoinformation science and Earth Observation independent for 60 years Faculty of University of Twente 3 years ago Various degree courses and certificates in Disaster management Earth sciences Geoinformatics Governance Land administration Natural resources Urban planning Water resources Student numbers Netherlands Indonesia 2

3 ITC/ University Twente Houses the United Nations University- ITC Centre for Spatial Analysis and Disaster Risk Management Training, education and curriculum development Knowledge development and research collaboration Advisory services In collaboration with many partners 3

4 ITC Geographer with study background in Hamburg (Germany), Ohio State (US) and Cambridge (UK) Since early 1990s work in the disaster field, with focus on remote sensing PhD in volcano remote sensing (lahars) Advanced image analysis, and focus on object-oriented analysis (OOA), increasingly with oblique data from UAV 4

5 RATIONALE & MOTIVATION Following disasters, rapid and comprehensive damage information is critical Assessment is challenging 5

6 RATIONALE VERTICAL IMAGE-BASED ASSESSMENT HAS LIMITATIONS Port-au-Prince, Haiti 6

7 RATIONALE PROMISE OF OBLIQUE DATA Oblique data are promising, but traditionally have limitations Early experiments with Pictometry data that may overcome those Not a UAV, but a MAV, 5 views Use of data ideally in an automated manner, using object-based approaches 7

8 PICTOMETRY BASICS OF THE SYSTEM 5 digital cameras (1 nadir, 4 oblique) GSD 15cm (nadir images, flying height ca. 1000m) overlap for stereo coverage 8

9 OBJECTIVES Which Pictometry features allow damage mapping? (elevation, geometrics features, textures, etc.) Identification of planar (=intact) and spectrally homogenous facades and roof sections Supervised classification of image segments Effect of training on the classification 9

10 DAMAGE MAPPING: METHODOLOGY Orthorectification of oblique images and disparity images (depth maps) Feature extraction from ortho images and DSM Assumption: intact buildings have planar surfaces Find evidence for planarity through Disparity maps (geometric homogeneity) Image information (radiometric homogeneity) 3D point cloud segmentation to find planar surfaces 10

11 EARTHQUAKE DAMAGE ASSESSMENT CLASSIFICATION OF IMAGE SEGMENTS 5 classes were defined: a. Intact roof b. Broken roof/ rubble c. Intact facade d. Bare ground e. Vegetation Direct damage indicators Training data by two interpreters Classification using a total of 22 features in machine learning approach Final step: combination per building D1 D2 D3 D4 D5 11

12 EARTHQUAKE DAMAGE ASSESSMENT CLASSIFICATION PER BUILDING Test on a 6 block area of Port au Prince, Haiti Depth images from stereo pairs (for different views) 3D point cloud 12

13 PARALLAX DISPARITY FOR IMAGE PAIR MERGE OF MULTIPLE VIEWS Planar areas have limited disparity Rubble and broken features appear noisy 13

14 DAMAGE MAPPING RESULTS: WESTERN VIEW EXAMPLE Ortho image Detected damage Intact roof Broken roof/ rubble Intact facade Bare ground Vegetation 14

15 Results using training data by 2 analysts For details see PE&RS paper, Gerke & Kerle,

16 ACCURACY ASSESSMENT & CONCLUSIONS Overall accuracy of approx. 70% (63% per building) Errors at image top where heights were missing Problem: damage indicators do not add up linearly 16

17 ONGOING RESEARCH: USING UAV DATA & ECOGNITION FOR THE CLASSIFICATION INSTEAD OF TRAINING DATA Limitations: Expensive Pictometry data Single view, limited spatial resolution, limited control Adding damage indicators Avoid use of training data Solution: Get your own UAV and ecognition Aibot X6 (Aibotix ) 17

18 OBJECTIVES Obtain data with a pole-based camera and a UAV Identify damage indicators in multi-perspective images with OOA Create a rough 3D building model with images and damage features draped Decision tree for semantic reasoning 18

19 DATA ACQUISITION 19

20 DATA COLLECTION IN ITALY, GERMANY, NETHERLANDS 20

21 METHODOLOGY Point cloud UAV images Segmentation Collapsed (D5) Roof Noncollapsed (D1-D4) Facade Building level Collapsed Standing Rubble piles Building s surroundings No rubble piles Facade s geometry Colors indicate orientation: red is horizontal, blue vertical. Here change in color signals sloping roof D5 D4 Inclined Noninclined 21

22 METHODOLOGY Point cloud Collapsed (D5) Segmentation Roof UAV images Noncollapsed (D1-D4) Facade Building level OOA 3D construct visualization Expert analysis Extracted features: Random scenarios Collapsed Stan ding Building s surroundings Facade assessment Per-view score D1-D2-D3-D4 Rubble piles No rubble piles Per-building decision tree D5 D4 Facade s geometry Inclined Noninclined Uncertainty evaluation D4 D3 D2 D1 D1 View/ facade level 22

23 DAMAGE DETECTION WITH OOA 23

24 PER-BUILDING ASSESSMENT Challenges: How to process damage evidence? How to deal with occlusion? 24

25 EXPERT-BASED ASSESSMENT ID Facade 1 Facade 2 Façade 3 Façade 4 Roof 1 EMS-98 score D0 D3 Occluded D4 D1 25

26 EXPERT-BASED ASSESSMENT Problem: simple building primitive Preferable: based on actual DSM 26

27 Multi-view images from UAV of complex structures Rich 3D model data from, e.g., from Italy Challenge is how to use such complex information 3D point clouds Segmentation for 3D damage assessment ITC, University of Twente 27

28 Multi-view images from UAV Demolished factory building. 300 images taken, combined into a 3D point cloud. ITC, University of Twente 28

29 RECONASS 2 FP7 projects on damage mapping RECONASS: Reconstruction and Recovery Planning: Rapid and Continuously Updated Construction Damage, and Related Needs Assessment (start: 1 Dec 2013; 3 ½ years) INACHUS: Technological and Methodological Solutions for Integrated Wide Area Situation Awareness and Survivor Localisation to Support Search and Rescue Teams (start: late 2014; 4 years) 29

30 OBJECTIVES OF RECONASS Provide rapid assessment on structural integrity of high-value buildings (ministries, hospitals), based on wireless sensor networks, finite element modelling, and external surveillance with UAV technology 3.5 years, 10 partners (German THW as disaster response organization, ITC for geoinformatics, and 8 engineering organizations) ITC covers one work package (Synergistic Damage Assessment with Air- and Space-borne Remote Sensing), with full-time AIO researcher 4 objectives 30

31 OBJECTIVES Detailed and element-specific damage assessment on per building level using multi-view oblique aerial images Synergistic use of UAV and wireless sensor network for complete, reliable and accurate damage assessment. Integration of satellite- and airborne data-based information for synoptic damage assessment Extension of sensor network with chemical and biological sensors for critical hazard monitoring 31

32 TASKS - Consider specific damage - Joint use of spectral/ color image information (texture, minute features), and 3D point cloud spalling Crack 32

33 TASKS - Relate to information from accelerometers, strain sensors, position tags and thermometers - Relate to data or derived deformation/instability information? - Semantic relationship between sensor reading and damage as observed outside? - Use UAV to patch gaps/ failed nodes? 33

34 TASKS Damage map of Port-au-Prince, Haiti Source: SERTIT - First damage data available from satellites - Limitations: vertical view, lack of familiarity of analyst with disaster area - Use UAV information to validate and calibrate the damage maps 34

35 TASKS - If more than 1 building is equipped with sensors, extra insights can be generated - piggy bag additional sensors, such as chemical or biological - Combine with environmental/atmospheric data to assess such threats 35

36 FROM RECONASS TO INACHUS 4 year project with > 20 partners Provide search & rescue forces with current and increasinglydetailed intelligence on the situation Nested integration initial satellite assessment Hotspot identification Scenario modelling using synthetic data UAV deployment (area-based monitoring) Dasymetric population modelling Sensors and robots 36

37 FINAL NOTES Legislation remains unclear (nogo in various countries) Reliability is a major issue 37

38 FINAL NOTES 38

39 SUMMARY Good progress in image-based damage mapping (2D to 3D, single- to multi-view, cognitive and semantic analysis) UAVs promising in term of image quality and sensor control Progress in photogrammetry and OOA has been critical FP7 projects give us space for further developments Limitations Recognizing and understanding damage (physical vs functional) Legal issues with UAV deployment Maturity of UAVs 39

40 Questions? For ITC s OOA work see Same for full references Papers also on For RECONASS see Or n.kerle@utwente.nl 40

41 RECONASS team Norman Kerle Markus Gerke her/ _markus_gerke/ Anand Vetrivel ITC, University of Twente 41