Data Processing for Classification. Dean Keiswetter, Ph.D., M.B.A. Chief Scientist, Leidos Holdings Inc.

Transcription

1 Data Processing for Classification Dean Keiswetter, Ph.D., M.B.A. Chief Scientist, Leidos Holdings Inc.

2 Data Processing Objective Determine which anomalies, if any, result from buried munitions The result of the analysis is a decision regarding the nature of the source of the measured signatures The analysis should be: Transparent Quantitative Documented 2

3 UX-Analyze software (ESTCP MR-0910) Data analysis algorithms embedded into Geosoft s Oasis montaj UX-Analyze = Commercial mapping, processing, & visualization software for earth sciences Commercial mapping, processing, & visualization software + Physics Induced EMI Response t 2 H 2 2 Commercial mapping, Principal Axis Polarizabilities processing, & visualization software for earth sciences Analysis Algorithms UX-Analyze = H H 0 t + TEMTADS 5x5 Physics Induced EMI Response t 2 H 2 2 H H 0 t Metal Mapper Principal Axis Polarizabilities TEMTADS 2x2 Classification Sensors

4 Software Overview UX-Analyze is fully integrated into Oasis montaj as a menu driven set of functions for geophysical target characterization and classification. These functions permit users to effectively classify buried sources as Targets-of-Interest, or not. Released to the US Government and commercial contractors (free of charge to recipients) 4

5 Topics Review Polarizations the basis of the classification decision Processing Fundamentals 1. Construct a Library (Expected Munitions and Clutter) 2. QC Measured Data (Blind and Background) 3. Invert and Look for Expected Munitions 4. Look for Unexpected Munitions 5. Prioritize Results and Final Products 5

6 Plan view of TEM Polarizations the Basis of the Decision Dipole Response Model Location & Orientation Transients from Rx cube Polarizability P 1 P 2 P Polarizations Time (ms) 6

7 Polarizabilities Principal axis polarizabilities completely describe EM response of target intrinsic to the target invariant to burial depth or target orientation Polarizability (m 3 /A) Time (ms) 10 Polarizability (m 3 /A) Time (ms) 7

8 Topics Polarizations the basis of the classification decision Processing Flow Fundamentals 1. Construct a Library (Expected Munitions and Clutter) 2. QC Measured Data (Blind and Background) 3. Invert and Look for Expected Munitions 4. Look for Unexpected Munitions 5. Prioritize Results and final products Construct a Library QC Measured Data Invert and Look for Expected Munitions Look for Unexpected Munitions Prioritize 8

9 BTW - the processing flow applies to all of the Advanced EMI Sensors Advanced EMI sensors are physically different, but the extracted polarizabilities are not... MetalMapper MPV HandHeld BUD TEMTADS 2x2 sphere sphere time (ms) Sphere mm small iso small iso 60 mm Small ISO time (ms) time (ms) time (ms) 9

10 #1: Construct a Library Document what we are Looking for Simply stated: We want to specify which munitions are present at our current site and store their respective polarizations in a library. New items can be added to the list if encountered. Why it is important: Classification performance suffers if we look for munition items that are not actually present Construct a Library QC Measured Data Invert and Look for Expected Munitions Look for Unexpected Munitions Prioritize 10

11 Library Master Database, maps and scatter plot are linked Build a library for the munitions expected at your site. It should include all anticipated munitions and unique clutter items (if any). 11

12 Munitions Signature Examples 2.36inch 105mm 155mm 81mm 37mm 20mm 12

13 Munitions Variability Variability among munitions needs to be considered. Site specific varieties, whether anticipated or discovered during the program, must be added to the library if unique 13

14 Seeds can be Used to Verify Library Manage Library Tool : Checks adequacy of library Identifies clusters Library Overlay Library Current Close Target Matches Small ISO Color coded by number of matches 0, 2, 5, 10, 20 Small ISO Ellis TP

15 #2: QC Measured Data Simply stated: We need to examine the integrity of the measured data files and the individual sensor readings Why it is important: Garbage in garbage out Construct a Library QC Measured Data Invert and Look for Expected Munitions Look for Unexpected Munitions Prioritize 15

16 QC Sensor Data for Blind Sources GPS Data Spatial registration GPS EMI Sensor Data Sensor Inertial Measurement Unit Orientation data Evaluate Sensor Data bounds take Action on Outliers 16

17 QC Data Backgrounds Sensor data acquired over areas believed to be free of metallic objects are subtracted to remove sensor drift, biases, and ground response 17

18 R1 R2 R3 Background Example R4 R5 R6 R7 Tx 3 backgrounds are different from the rest in varying degrees 1150 (orange) is most variable followed by 1111 (green) and 1265 (red) Ty Z_6Y Tz 18

19 Low amplitude signatures are preferentially affected by questionable backgrounds Result of subtracting the three different backgrounds shown earlier on a seeded 37mm 37mm 1150 as background No Match 1111 as background 37mm; 0.35 metric 1265 as background 37mm; 0.86 metric 19

20 #3: Invert and Look for Expected Munitions Simply stated: In this step, we want to invert the measured sensor data to obtain polarizabilities and compare them against our library of anticipated munitions. Why it is important: Provides a quantitative comparison of intrinsic source features Construct a Library QC Measured Data Invert and Look for Expected Munitions Look for Unexpected Munitions Prioritize 20

21 Primary Turn the Crank Inversion Process Process 10 1 Location & Orientation + Polarizability P 1 P 2 Parameters P Time (ms) 21

22 Intrinsic Polarizations Intrinsic Polarizations Qualitative Visual-based Library Match mm mm Blue Unknown Grey Library from APG Time (ms) Time (ms) mm mm Time (ms) 22

23 Quantitative Library Match 0.93 Match Type (ID) Parameters Multiple sources view All Three Curves 0.94 Options Only Two Curves 0.99 Only One Curve 23

24 Size / Decay UX-Analyze: QC environment Database Data maps Library Match Cluster Fit Results Interactive review with multiple linked views scatterplot, database, images, and polarizations A mouse click in the scatterplot or database changes all User can zoom to see scatterplot details 24

25 BTW -- It ain t that easy to unravel issues Lots and Lots of Individual Steps Analysts need training and practice Third party QC review is critical 25

26 #4: Look for Unexpected Munitions Simply stated: We need to look for repeat source signatures (multiple sources that are similar) that are not explained by the library Why it is important: We have to expect the unexpected. Historical records may be incomplete. Site usage may vary by location Construct a Library QC Measured Data Invert and Look for Expected Munitions Look for Unexpected Munitions Prioritize 26

27 Unanticipated UXO May Need to be Added Manage Library Tool : Checks adequacy of library Identifies clusters Library Current Target Library Overlay Close Matches Color coded by number of matches 0, 2, 5, 10, 20 27

28 Unanticipated TOI: Fort Sill, OK?? unique polarizations and best library match 28

29 Unanticipated TOI: Fort Sill, OK?? 3.5inch rocket 29

30 Fort Sill, OK Ground Truth 40mm Frag Ball no training data, unexpected 30

31 #5: Prioritization Simply stated: An anomaly is flagged for digging if the inverted source: Matches a signature in the site-specific library of munitions Is part of a previously unidentified cluster Large and deeply buried Why it is important: A numerical process produces a transparent, quantitative, and consistent classification decision Construct a Library QC Measured Data Invert and Look for Expected Munitions Look for Unexpected Munitions Prioritize 31

32 Prioritization Decision & Validation Digs This Section: Not Site Dependent Bounds set by controlled tests & rules to deal with distance-to-flag scenarios Parameters Library Match Metrics: Can be simplified to a single threshold -- one that is chosen to account for all site specific UXO. Possible approach for recommending Validation Digs: #1: Interrogate x% of anomalies below selected threshold #2: Analyst recommended checks 32

33 Topics Polarizations the basis of the classification decision Processing Fundamentals 1. Construct a Library (Expected Munitions and Clutter) 2. QC Measured Data (Blind and Background) 3. Invert and Look for Expected Munitions 4. Look for Unexpected Munitions 5. Prioritize Results and Final Processing Products 33

34 Results and Final Processing Products The decision Transparent Quantitative Documented and the documentation. Prioritized Dig List Digital files data maps, polarizations, library comparisons, final decision 34

35 Dynamic Data Issues Processing Perspective Additional Tasks: Pick Targets Extract spatial data 35

36 Dynamic Data Standard flow after anomalies are selected 75mm Dynamic Stationary Construct a Library QC Measured Data Invert and Look for Expected Munitions Look for Unexpected Munitions Prioritize Dynamic Stationary Greater variability in secondary polarizations, but they are still sufficient to identify TOI s ISO s 36

37 Technology Transfer - Workshops Year Location ½ Day 1 Day 2 Days Brief/Workshop # Attendees 3/2010 Huntsville, AL x workshop 32 4/2010 Denver, CO x brief 2 12/2010 Washington, DC x brief 40 1/2011 Washington, DC x workshop 21 1/2011 Denver, CO x workshop 21 3/2011 Huntsville, AL x workshop 30 12/2011 Huntsville, AL x workshop 20 2/2012 Denver, CO x workshop 24 4/2012 Washington, DC x workshop 20 6/2012 Denver, CO x workshop 24 2/2013 Washington, DC x workshop 31 2/2013 Denver, CO x workshop 31 5/2013 Huntsville, AL x workshop 10 TOTAL 306 Representatives from over 40 firms 37

38 Technology Transfer, continued 38

39 Results and Final Processing Products The decision and the documentation. Transparent Quantitative Documented Prioritized Dig List Digital files data maps, polarizations, library comparisons, metric matches 39