Artificial Intelligence for Real Airplanes

Artificial Intelligence for Real Airplanes FAA-EUROCONTROL Technical Interchange Meeting / Agency Research Team Machine Learning and Artificial Intelligence Chip Meserole Sherry Yang Airspace Operational Efficiency Boeing Research & Technology Presented at: TIM / ART Workshop on Machine Learning and Artificial Intelligence EUROCONTROL, Paris, France 24-25 September 2018 Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 1

Outline Introduction Motivation Challenges Background History Aviation applications Modalities Robustness Deep sense learning Recent examples Summary Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 2

Machine Learning (ML) is defined by widely accepted definitions as: A computer program is said to learn from experience with respect to some class of tasks and performance measure if its performance at those task improves with experience. Tom Mitchell or: Field of study that gives computers the ability to learn without being explicitly programmed. Arthur Samuel, ca. 1959 Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 3

Supervised & Unsupervised Learning Both Provide System Performance Enhancements Fewer Errors Better Efficiency Increasing Accuracy Yielding Smarter Devices Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 4

Challenges for Machine Learning in Aviation Non-stationary stochastics and covariant shift Anomaly detection Learning in on-line settings Validation of on-line learning Others include: Real-time operation Competing systems Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 5

Essential Elements of Machine Learning Initial Customers Users Community System Knowledge Public Acceptance Algorithms/ Products Applications Training Verification/ Validation Developers Real-Time Databases Experience Research More Efficiency Accuracy Safety Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 6

Complexity A Sample of Machine Learning Applications Profile 50 Years of History Human-like performance Very High Difficult Degrees of Improvement Very High Surgical Assistant Medium Very High Autonomous Cars Advanced Robotics Low Low Mathematics Problems Low Societal Planning Stock Trends High 1980s 1990s 2000s 2010s 2020s Time Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 7

Applications to Aviation Across 5A Model Airspace Route planning Communications Autonomy (control of assured comms) Trajectory planning Maintenance Autonomous Operations Air Traffic Control Deconfliction Situational awareness Autonomy (command) Airline Fuel consumption Luggage / package awareness Autonomy (operations) Airplane Airport Departure / arrivals planning Passenger awareness Autonomy (security) Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 8

Why Not Oversee Processor Training FAA & EUROCONTROL has been at the forefront of training pilots to safely fly airplanes As pilots are augmented by processors, why not apply the same safety training to the processors? Otherwise, the distribution of data and how it is handled could cause unexpected outcomes Unexpected outcomes are not acceptable for aviation safety Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 9

Potential Applications for AI/ML on Airplanes Avionics performance efficiencies Trajectory and airspace efficiency Safe, autonomous operations Clean operations Electrical power optimization Fuel burn efficiencies Ride quality optimization Aircraft traffic situational awareness Adverse weather rerouting Communications management and quality of service Passenger experience improvements / safety Runway and tarmac safe operations Cargo / luggage management Structural monitoring Maintenance predictions/ safety checks Performance Enhancer Safety Enhancer Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 10

Consequence of Occurrence Importantly, ML Cannot Ignore Training at the Corners but Methods Do Not Extrapolate High Low Low Frequency of Occurrence High Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 11

Automation Aided by AI / ML Present and Future Past & Present Routine tasks Predictions Patterns, Clustering, Ranking, Scoring Potential Future Complex decisions Sequential Decisions Handling true anomalies (unknown unknowns) Accurate risk estimation based on common-sense knowledge Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 12

High-Stakes Applications Require Robust AI In aviation, best practice for AI / ML robustness must accommodate: Incorrect models Unmodeled phenomena Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 13

Why is Unmodeled Phenomena a hard fundamental problem? 1) It is impossible to model everything Qualification Problem: It is impossible to enumerate all of the preconditions for an action Ramification Problem: It is impossible to enumerate all of the implicit consequences of an action 2) It is important NOT to model everything Fundamental theorem of machine learning model complexity error rate sample size Corollary: If sample size is small, the model should be simple We must deliberately oversimplify our models! Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 14

Deep Sense Learning (DSL) Objective Need for robust scene understanding in cluttered environments and data analytics with few labeled data Our objective is to dramatically reduce false alarms for autonomous recognition and learn with 1000x less labeled data Benefits Autonomous navigation Novelty detection Target recognition Performance Huge improvement with DSL For 50 labeled samples per class, DSL reduced errors from 70% for deep learning to only 18% Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 15

Deep Sense Learning (DSL) [cont d] Different from deep learning, Deep Sense Learning can extract meaningful component groups from data without labels Attr.1 Attr.2 Attr.3 Attr.4 Attr.k Raw data Dog bark Bird song Siren Our unsupervised component extraction can improve scene understanding and explainability of machine learning systems Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 16

Explainable Machine Learning Why / How? Bicycle misclassified as Traffic Cop Deep Neural Network (Black Box) Input image of Bicycle Deep Sense Learning technology explains the inner workings of complex machine learning models Deep Sense Learning 3.0 12a 31a 23b 4c 2.0 2.0 2.1 2.5 3b 22a 2a Influence from traffic cone causes misclassification 3.2 White Box 2.9 DSL explains machine learning mistakes and how to correct them Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 17

Challenge High-performance ML models are black boxes Innovation Explainable Traces (X-trace) provide finegrained view of ML decisions Approach Decompose input into hierarchy of decisionrelevant attributes Results Explainable Machine Learning [cont d] Bicycle misclassified as Traffic Cop because deep network focuses on traffic cone and bright yellow color DSL provides detailed insights into complex ML models Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 18

Summary AI / ML is a nascent discipline for applications to avionics and autonomous operations Training, especially at the corners, is essential to enhance safety for aviation Robustness is essential, or ML will produce unexpected outcomes The science is evolving; the applications are exploding; and the promises are extraordinary Copyright 2018 Boeing. All rights reserved. 18-029v4 9/24/2018 20