Runway Centerline Deviation Estimation from Point Clouds using LiDAR imagery

Runway Centerline Deviation Estimation from Point Clouds using LiDAR imagery Seth Young 1, Charles Toth 2, Zoltan Koppanyi 2 1 Department of Civil, Environmental and Geodetic Engineering The Ohio State University Phone: 614-292-7681 2 Satellite Positioning and Inertial Navigation (SPIN) Lab Center for Aviation Studies E-mail: koppanyi.1@osu.edu ICRAT 2016 Drexel University, Philadelphia, PA June 20-24, 2016 1

Introduction Motivation Remote sensing concept Laser scanning technologies Sensor selection Initial test results Tests with taxiing aircraft 1-LiDAR sensor configuration LiDAR point cloud processing Methods Method comparison Tests with touch&go aircraft 4-LiDAR sensor configuration Results and statistical analysis Tests with landing aircraft 5-LiDAR sensor configuration Results and statistical analysis Conclusion/Future work Outline 2

Potential Airport planning: stochastic models. Wingtip clearance, airfield separation Veer-off models Source: ACRP Report 51, Risk Assessment Method to Support Modification of Airfield Separation Standards Tracking take-offs and landings 4

Light Detection And Ranging Airborne LiDAR/ALS INS Courtesy of Ayman Habib Google s driverless car Mobile LiDAR/MLS 5

Concept LiDAR Sensor Trajectory 6

Sensor Comparative Tests Laser technology is rapidly advancing Application specific sensors are introduced (UAS, autonomous driving) Performance evaluation needed to optimize sensor selection to provide reference for low-end sensors Profilers/scanners considered: SICK LMS30206 UTM-30LX-EW Velodyne VLP-16: 16 channels 100+ m range 300K points/second Dual-return capability 360 x 30 FOV Small size: 100 mm x 65 mm IbeoWeight: Alasca 600 grams XT Low power requirements Easy to install No external rotating parts Relatively inexpensive sensor Velodyne HDL-32E Velodyne HDL-64E Bridger s HRS-3D-1W 8

Single Scanner Test Site location: OSU Don Scott Airport The sensors attached to a data acquisition platform (GPSvan) Test aircraft: Cessna 172 Scenarios: aircraft passing diagonally and perpendicularly w.r.t sensors at various velocities (3, 6, 9,12 knots) 9

Test 3: Airplane Georeferencing 10

Data Preprocessing 1. Point cloud filtering: remove no aircraft body points, such as runways, buildings, light fixtures, etc. 2. Filtering by time: select those time intervals when aircraft was in the field of view of the sensor 11

Methods: Challenges Sparse point cloud Difficult to model aircraft body Sample raw data 12

Methods under Investigation 1. Center of Gravity (COG) Point cloud center of gravity estimated and tracked 2. ICP-based 2-DoF 3D ICP (only heading and velocity estimated) 3. Volume minimization (VM) new approach, specifically developed for the project Bin-based random optimization (entropy minimization) 13

Method 1: COG Calculate the center of gravity (COG) of each frame (one point cloud) 14

Method 2: Iterative Closest Point Problem: find the rotation matrix and translation vector that minimize the sum of the distance of the closest points between two points clouds min αα,ββ,γγ,δxx,δy,δz ii=1 NN pp [MM(αα, ββ, γγ, Δxx, Δy, Δz) ss ii rr ii ] 2 MM = where MM is the 4-by-4 transformation matrix, ss ii = [xx ss,ii, yy ss,ii, zz ss,ii, 1] is the iith point from the samples, rr ii = [xx ss,ii, yy ss,ii, zz ss,ii, 1] is the closest iith coordinates on the reference body curve rr 1,1 rr 1,2 rr 1,3 Δxx rr 2,1 rr 2,2 rr 2,3 Δyy rr 3,1 rr 3,2 rr 3,3 Δzz 0 0 0 1 Solve this minimization problem between the consecutive frames Source: Point Cloud Library, http://pointclouds.org/documentation/tutorials/int eractive_icp.php 15

Motion Models Uniform motion Trajectory is a straight line xx = vv xx tt yy = vv yy tt Curvilinear motion Trajectory is a 2 nd order polynomial xx = vv xx tt + aa xx 2 tt2 yy = vv yy tt + aa xx 2 tt2 Free motion Trajectory is a higher older polynomial xx tt = aa 0 + aa 1 tt + aa 2 tt 2 + + aa nn tt nn, yy tt = bb 0 + bb 1 tt + bb 2 tt 2 + + bb nn tt nn 16

Method 3: VM Reconstruction Problem: reconstruct aircraft body using the equation of motion at various velocities 17

Method 3: Volume metric Decimating space into cubes (voxels); optimized to average point cloud density; 0.5x0.5x0.5 m was used The goodness of the reconstruction is measured by the number of cubes occupied by the reconstruction 18

Visual Comparison 19

Statistical Comparison COG may provide good solution but the full aircraft body has to be captured; difficult to achieve ICP is able to provide acceptable results only in short ranges and when the motion direction w.r.t sensors is close to ~45 ; in other cases, it is likely to fail VM (volume/entropy minimization) gives the best and most robust solution 20

Tests 4 and 5 22

System Components Sensor station With one vertical and one horizontal sensor GPS Antenna Precise timing and station coordinates Installation (from top to down) Sensor station, GPS antenna, logging laptops, power 23

Sensor Configuration GPS Antenna Horizontal sensor Vertical sensor GPS Receiver Sensor interface box Battery Logging laptop 24

Network Configuration 25

Sample Sensor Data Horizontal sensor axis Vertical sensor axis Landing aircraft 26

Typical Raw Data 27

Point Clouds Point clouds acquired by all sensors (rear view) Point clouds acquired by all sensors (top view) 28

Aircraft Variations Info Sample 1 Sample 2 Aircraft type Cessna Learjet Operation Landing Taxiing Number of sensors 4 2 Observation time [s] 7 6.1 Observed length [m] ~100 ~27 Number of backscattered points 5,970 13,133 29

Trajectory Estimation 30

Derived Data 31

Tests 6 33

Sensor Network Configuration 34

Obtaining Reference: Surveying Goals: Precise centerline reference Precise global coordinates Aiding sensor calibration Equipment: GPS static solution for control points Total Station for mass points 35

Sensor Configuration Station 1 (2 horizontal, 1 vertical) Station 2 (2 horizontal) GPS Antenna Logging laptops, batteries Sensor Cabling UTP & Power GPS Receiver Interface Box 36

Sensor Calibration Sensor position is known from GPS, but the sensor orientation is unknown The sensor orientation is estimated during the calibration Three rotation/attitude angles: roll, pitch, yaw Very accurate orientation determination is required for precise positioning, as data from various sensors should be integrated FOV: unknown FOV: unknown Sensor X,Y,Z position is known from GPS 37

Using Planar Target for Sensor and Intra-sensor Calibration Calibration board Captured by the LiDAR sensors Sensor station 4 prisms installed at corners Board corners in global system are measured by Total Station 38

Sensor Calibration Surface normal-based orientation estimation. Reference normal is calculated from TS measurements. Minimize the angle difference between the boards and the reference norms. Station This is an overdetermined homogenous linear equation system that can be solved by SVD. 39

Data Acquisition Landing airplane Sensor station 40

Method 4: Cube Trajectories Initial reconstruction from VM Point association by decimating the space into cubes The points with different timestamps inside one cube defines the cube s trajectory Calculating the cubes velocities based on the points Filtering the velocity time sequence by a Gaussian filter Trajectory can be derived by integrating the velocity function 41

Estimated Velocities 42

Results (Video) 43

Estimated Altitudes (34 planes) 44

Estimated Altitudes (34 planes) 45

Conclusion LiDAR technology allows tracking taxiing and landing aircraft Single LiDAR sensor configuration may not provide acceptable results, depending on the sensor quality (affordability) A multi-lidar prototype system developed The interactive target-based sensor network calibration process provided good accuracy During a real four-hour long test, the landing and/or taxiong of 34 aircrafts were recorded and processed From the LiDAR point clouds, 2D and 3D trajectories were estimated ~10 cm level accuracy was obtained; depending on sensorobject distance, aircraft and sensor types 47

Future Work Assess the reliability and accuracy of the system using GPS reference on aircraft (DGPS, cm-level accuracy) The optimal sensor network formation and configuration have still open questions Tests with larger aircraft Very little or no effort has been devoted to investigate the use of complementary sensors, such as cameras and radar/uwb Point cloud processing has advanced and more sophisticated feature extraction techniques could be exploited for plane type detection There have been rapid sensor developments, so the affordability of a sensor configuration for an entire runway is becoming a reality Ford signed an agreement with Velodyne targeting a $500 unit price 48

Thank you for the attention! Q&A 49