Large-Scale Flight Phase identification from ADS-B Data Using Machine Learning Methods

Transcription

1 Large-Scale Flight Phase identification from ADS-B Data Using Methods Junzi Sun PhD student, ATM Control and Simulation, Aerospace Engineering

2 Large-Scale Flight Phase identification from ADS-B Data Using Methods Overview Introduction on the research, ADS-B & Air Traffic Data Flights extraction using machine learning Flight phase determination Results & Conclusions 2

3 Introduction 3

4 Background / Introduction Research group on Air Traffic Management, Aerospace Engineering, TU Delft Aircraft performance modelling with open ADS-B data Goal of this paper/research - developing applicable machine learning methods to process and mining ADS-B data 4

5 What is ADS-B? Automatic dependent surveillance broadcast Determine aircraft position via satellite navigation system (e.g.: GPS) Broadcast information through Mode-S transponder at 1090 MHz ADS-B data containing: a. aircraft ICAO address b. aircraft position (latitude, longitude), and altitude c. aircraft velocity, heading d. aircraft callsign Constantly monitored by global receiver network (e.g.: FlightRadar24) 5

6 Open ADS-B Data Advantages: a. b. c. Large quantity of data Open, unencrypted and available for everyone No limitation of usage and distribution Each day from our receiver: ~ 3 thousand ICAOs ~ 12 million ADS-B messages ~ 5 million positions decoded ~ 5.5 million velocities decoded ~ 2.5 GB raw + compressed data Challenges: a. b. c. d. e. Arbitrary aircraft models and owners Atmospheric conditions uncertainty Wind uncertainty Aircraft weight uncertainty Thrust settings An small Global A320 Dataset (3 days) ~ 60 million entries of data ~ 4 thousand ICAOs ~ 20GB disk storage 6

7 ADS-B receiver and data service 1. Hardware setup 2. Python ADS-B library a. b. Decode position / velocity / ID c. Aggregate position and velocity data 3. ADS-B decoding guide a. 7

8 Single ADS-B receiver 8

9 FlightRadar24 Our receiver is part of FR24 network, which is made by contributors like us Continues tracking with ADS-B with large coverage Example: 24 hours 63 million entries 30,000 airplanes North America, Europe, and South Asia dominates the air traffic 9

10 Technical challenges How to extract information from scattered ADS-B data? How to deal with large variance in the data set? How to deal with unknown number of aircraft contains in a dataset? How to distinct different s carried out by the same aircraft? How to segment trajectory data into separate phases? How to maintain efficiency and scalability? 10

11 Data processing chain 1. ADS-B 2. Data pre-processing 3. Extraction of continuous trajectory, using unsupervised machine learning clustering a. 4. Smoothing, filtering, and interpolating missing data (optional) Flight phase segmentation Raw ADSB data 11

12 Flight Extraction 12

13 Step 1., data formating Mongo DB data External ADS-B Data in different format (e.g. FR24) data 13

14 Preprocessing - encoding, feature scaling 1. Encoding: Convert non numerical label to numerical data (ICAO, ICAO Index Models) 2. Scaling: Project numerical data of different features to the same kts range [0, 100] data 14

15 Choosing a clustering method Selection criterias: 1. Scalability: with very large dataset 2. N-Clusters: non-fixed number of clusters 3. Geometry: distance between nearest 4. Other: Possibility of outlier removal and batch clustering scikit-learn.org data 15

16 Choosing a clustering method - K-Means Idea is from H. Steinhaus (1957) Centroid-based algorithm Predefined number of clusters Minimizing the Euclidian distance of all data point to its cluster centroid data 16

17 Choosing a clustering method - K-Means data 17

18 Choosing a clustering method - BIRCH T. Zhang et al. (1996) BIRCH - balanced iterative reducing and clustering using hierarchies Step 1, scan all data to build an initial CF (Characteristic Feature) Tree, where the leaf contains several sub-clusters. Threshold: Each sub-cluster radius limit by distance T Branching Factor: Each non-leaf node has at most B entries, each leaf node has at most L CF entries Step 2, condensed into desired length by constructing a smaller CF Tree Step 3, global clustering data 18

19 BIRCH (Build CF Tree) Branching Factor (B, L) = 3 data 19

20 BIRCH (Build CF Tree) Branching Factor (B, L) = 3 data 20

21 BIRCH (Build CF Tree) Branching Factor (B, L) = 3 data 21

22 Choosing a clustering method - DBSCAN M. Ester et al. (1996) DBSCAN - Density-based spatial clustering of applications with noise Key parameters: Core, reachable, and outliers Eps: Maximum distance between two data point in the same cluster MinPts: number of data samples in the neighbourhood of a core point To form a cluster: All in a cluster are density connected A new point is part of the cluster if it is reachable from any of a cluster data 22

23 Choosing a clustering method - DBSCAN data 23

24 Results of BIRCH data 24

25 Results of BIRCH data 25

26 Results of BIRCH data 26

27 Results of DBSCAN data 27

28 Results of DBSCAN data 28

29 Results of DBSCAN data 29

30 Flight Phase Segmentation 30

31 Flight Phase Segmentation Goal: To separate data into different phases Challenges: 1. Data size 2. Uncertainty behavior a. b. c. duration altitude velocity 3. Accuracy of the algorithms data 31

32 Design Member functions design: Membership functions design Altitude {high, low, and ground} RoC: {zero, positive, negative} Ground speed: {high, medium, low} Flight phases: {ground, climbing, descending, cruise} data 32

33 Segmentation Flow data 33

34 Segmentation result (Example) data 34

35 Conclusion 35

36 Conclusion Raw ADSB data 36

37 Conclusion 1. The chosen methods have shown promising results. a. b. Clustering (DBSCAN, BIRCH) i. Large size of dataset ii. Unknown number of aircraft and s Flight phase segmentation ( ) i. Different profiles ii. Advantage over noisy trajectory data 2. Limitations: a. b. Lack of stream data clustering Offline use only 3. Open Python libraries / code: a. b

38 [github] 38