Presentation Outline Preserving the Evidence: Multi-Platform 3D Reality Captures for Tornado Damage J. ArnWomble, Ph.D., P.E. Assistant Professor of Civil Engineering West Texas A&M University Canyon/Amarillo, T awomble@wtamu.edu Richard L. Wood, Ph.D. Assistant Professor of Civil Engineering University of Nebraska Lincoln Lincoln, Nebraska rwood@unl.edu Introduction (Learning from tornado damage) Equipment (Lidar and low-level aerial) Data processing Case Study (Pampa, T Tornado -Nov. 2015) NSF RAPID Response Grant Learning from Tornado Damage Need tornado wind speeds for engineering designs Few wind speed measurements (logistics and dangers) Damageto structures is a proxy for wind speed Damage to engineered buildings is especially helpful Revised/expanded DODs for EF Scale Need to validate (correct) wind speed estimates in EF Scale Preserving damage scenes is helpful for ongoing/future studies and validation of EF Scale Objectives 1) Identify objective digital platforms for survey and assessment of structures 2) Prepareresultant data for quantitative analyses 3) Assess collected datafor key features of interest 4 Objectives Digital Platforms 1) Identify objective digital platforms for survey and assessment of structures 2) Prepare resultant data for quantitative analyses 3) Assess collected datafor key features of interest Ground Based Laser Scanner Unmanned Aerial System (abroad deployment) Motivation and Advantages Detailed and geometrically accurate Time efficient Cost effective Archived digital documentation Ease and safety of exposure: Debris Trip/Fall Precarious structure 5 6 1
http://en.wikipedia.org/wiki/file:lidar-scanned-sick-lms-animation.gif 2/20/2017 Digital Platforms: LiDAR Equipment Overview LiDAR Survey Light Detection and Ranging or laser scanning Determine surface geometries Traditionally a pulse of light sent and time of flight calculated Uses an exterior camera to capture RGB color indices Creates a point cloud Vertices in 3D space Measures distance, compute area, and volume Mesh or surface creation LiDAR Mechanism Digital Platforms: LiDAR Equipment Constraints Line of sight technology creates occlusions Empty areas of the point cloud Various causes include: Architectural features Adjacent buildings Tree branches and other landscaping features Utility lines Moving objects: people, vehicles, equipment, etc. Occlusion in LiDAR Point Cloud due to Trees 7 8 Digital Platforms: LiDAR Alleviating Constraints Minimizing occlusions Use of multiple scans Time consuming Accessibility issues Typical solutions: Mobile LiDAR via vehicle or robotic devices Airborne LiDAR Example Airborne LiDAR Image obtained from: http://www.terraremote.com/ mining/active-operations/ Multiple Scanner Placements 9 (example) Digital Platforms: LiDAR Example Previously Deployed Platform Ability to quickly map surface geometry Phase based scanner Range up to 130 m Speed up to 976,000 points per second Accuracy of approx. 0.15mm at 25 m 10 Digital Platforms: Low Level Aerial Equipment Overview Low level aerial imaging Traditional fixed-wing aircraft Unmanned Aerial Systems UAS Platforms can be flown: Remote controlled Semi-autonomous Fully-autonomous Aerial onboard sensors include: Cameras LiDAR Multi-spectral Image obtained from: http://www.terraremote.com/mining/active-operations/ Image obtained from: https://i.ytimg.com/vi/f3uyztrodzc/maxresdefault.jpg Image obtained from: https://sustainablesecurity.files.wordpress.com/2013/10/mq 1-predator-drone.jpg 11 Digital Platforms: Low Level Aerial Equipment Constraints Line-of-sight technology in a moving reference frame Flight path determined by: Environmental factors (e.g. wind, precipitation) Regulations Geometry of the target structure or system On-board sensors Situational obstacles Simulated Flight Path 12 2
Digital Platforms: Low Level Aerial Alleviating Constraints Flight plan optimization: Desired resolution of onboard sensors Duration (e.g. battery) Altitude (e.g. detail level) Obstacle avoidance Direction (e.g. N-S, E-W) Example Autonomous Path Image obtained from: http://bestdroneforthejob.com/wp-content/uploads/2015/09/agriculture-drone-buyers- Guide-flight-plan-for-quadcopter.jpeg Digital Platforms: Low Level Aerial UAS before Takeoff in Bungamati, Nepal (abroad deployment) Previously Deployed Equipment DJI Phantom 2 with modified accessories Range: 1000 m (3000 ft) Duration: approx. 23 min. Payload: 1300 g (2.8 lbs) GPS auto-pilot system First person camera view GoProHero 3+ Black Ed. 12 MP photo capture 1440 k video capture 13 Compact HD Camera 14 Objectives Resultant Data: LiDAR 1) Identify objective digital platforms for survey and assessment of structures 2) Prepareresultant data for quantitative analyses 3) Assess collected datafor key features of interest 15 LiDAR Scanner Positions Development Each scan position outputs point cloud in local coordinates Desired global coordinates for the entire system Point cloud registration done via targets Retro-reflective spheres Checkerboard patterns Surveyed coordinates 16 Resultant Data: Aerial Imaging Resultant Data: Aerial Imaging Image Collection Image Collection Complexly Distorted Image Classification (Keep?) Image Classification (Keep?) es No es No Distorted Discarded Distorted Discarded Undistorted Distortion Parameters Undistorted Distortion Parameters Undistorted Formation Formation Filtering Filtering Scale SfM Scale SfM 17 18 3
Resultant Data: Aerial Imaging Resultant Data: Aerial Imaging Image Collection Image Classification (Keep?) es No Distorted Discarded Distortion Parameters Undistorted Formation Filtering Development Applied computer vision techniques to reconstruct 3D geometry Structure-from-Motion General reconstruction process: Identified image pairs via feature detection Estimated image location and orientation Reconstruct scene geometry Image Collection Image Classification (Keep?) es No Distorted Discarded Distortion Parameters Undistorted Formation Filtering Set of Photo-Derived Scale SfM Scale SfM 19 20 Resultant Data: Aerial Imaging Objectives Image Collection Image Classification (Keep?) es No Distorted Discarded Distortion Parameters Undistorted Formation Filtering Development Arbitrarily scaled Physical scale factor needed: Real world measurement Ground control targets 1) Identify objective digital platforms for survey and assessment of structures 2) Prepare resultant data for quantitative analyses 3) Assess collected datafor key features of interest Scale SfM Example Ground Control Target 21 22 Data Assessment: Geometry Convert (x, y, z) vertices to dimensioned drawings Computer Aided Drawings, Building Information Modeling Measure and geometric computations identify key dimensions 23 Assess surface geometry to identify features Global response Displacement tracking Structural drift Local response Cracking Spalling Corrosion Basic steps Data filtering and outlier removal Point clustering and geometric computations Statistical variation to detect changes 24 4
Global Response: Residual Drift Global Response: Residual Drift Noisy profile Factor for Standard Deviations & Neighboring Points Hampel Filter Window Size FIR Average a Smooth Profile a Note: No data available in the direction for the 4 th floors due to occlusion 25 26 Global Response: Residual Drift 0.4 5.5 2.5 4.3 1.1 0.7 0.4 n/d * 2.8 1.5 0.3 2.0 1.9 n/d * 3.9 4.3 First story (all dimensions are in cm) Third story (all dimensions are in cm) 0.2 6.5 2.3 2.2 4.6 0.4 n/d * n/d 3.7 * 0.5 0.2 1.0 2.9 n/d * n/d * 4.1 Second story (all dimensions are in cm) Roof level (all dimensions are in cm) Local response Cracking, spalling, and corrosion is evident on surface due to local geometric variations Investigate the local geometric variations over the neighborhood Surface normal vectors Curvature Complications Ordered point cloud for computational efficiency Computations for each point cloud vertex Point cloud varies from 5k to 500M points Requires high computational power n/d * : No data available due to occlusion. 27 28 Example structure: El Centro, CA 2-story Masonry Infill RC Frame N Calculate Surface Normals Central Single Bay Computed Normals Front View Lidar survey Pretest Detailed Section N Posttest 29 30 5
Data Assessment: Validation Data Assessment: Validation Scalable Damage Detection and Validation Qualitative assessment of detected damage Superimposing the detected damage to the point cloud Scalable Damage Detection and Validation Qualitative assessment of detected damage Superimposing the detected damage to the point cloud Input Point Cloud Input Point Cloud Detected Surface Damage 31 Detected Surface Damage 32 Data Assessment: Validation Case Study 2015 Pampa Tornado 33 34 Tornado Outbreak Nov. 16-17, 2015 (T OK KS) Pampa, T N NOAA/NWS Damage Survey Viewer NOAA/NWS Damage Survey Viewer 6
Helicopter and News Media (First Look) Halliburton Oil Field Facility Low resolution, but helps to identify pristine damage areas Why is this site so interesting? Engineered buildings in close proximity - different resistances (Damage levels serve as proxy for wind speed) Revised/expanded DODs for EF Scale Metal buildings with overhead crane structure Facility was struck by a tornado in 1982 (under construction) No direct access (safety and security) What can we do to preserve these important data? N CPI N Aerial Oklahoma Image (November 20, 2015) Aerial Oklahoma Image (November 20, 2015) Aerial Oklahoma Image (November 20, 2015) Aerial Oklahoma Image (November 20, 2015) 7
Before Tornado (Google Earth) After (Aerial OK composite; Google Earth overlay) Before (Google Earth) After (Aerial Oklahoma) Before (Google Earth) After (Aerial Oklahoma) 8
Before (Google Earth) After (Aerial Oklahoma) Aerial Oklahoma (FoDAR 3D model) (Aerial SfM) Debris Field LiDAR (UNL and TTU) 9
Low-Level Aerial A puzzling damage pattern (as viewed from ground) The detailed video reveals why (overhead crane) Photogrammetry (PhotoModeler) Current exploration for effectiveness (time and expense) compared to FoDAR and LiDAR Preliminary results: Low cost Relatively rapid data collection Fine details possible Collection and processing area art forms Center-Pivot Irrigation Systems Potential new rural DI for EF Scale (ASCE Standard) 10
Center-Pivot Irrigation Systems Lidar Conclusions First-available imagery is critical (any resolution) to distinguish pristine damage 2D data generally first available for initial analysis 3D can come later No single platform emerges as clearly the best always overall Choice of platform depends on situation (location, timing, personnel, budget) High-Resolution Commercial Satellite Readily obtained Timing is tricky (revisit times and clouds) Expensive and relatively low resolution Near-IR imaging can be beneficial (Satellite is excellent for tornado paths over larger areas) Aerial imaging tends to win over satellite for details of individual structures Possibility for 3D imaging based on overlapping images Especially good for following a tornado track Costs can vary with location Contract basis is excellent for urban areas; Custom/FoDAR system for other areas Conclusions cont. Low-Level Aerial Highly useful Government limitations and regulations Lidar Excellent and straightforward results Relatively slow strategic choices only Digital photogrammetry Data collection can be rapid Relatively inexpensive rapid data acquisition, moderate acquisition speed Data collection and post-processing are acquired Acknowledgements Halliburton Corporation National Science Foundation (RAPID Response Grant #1623553) Killgore Research Center (WTAMU) NHERI/DesignSafe-ci Thank you! J. Arn Womble Assistant Professor of Civil Engineering West Texas A&M University awomble@wtamu.edu Richard L. Wood Assistant Professor Department of Civil Engineering University of Nebraska-Lincoln rwood@unl.edu 11