Covert and Anomalous Network Discovery and Detection (CANDiD)

Transcription

1 Covert and Anomalous Network Discovery and Detection (CANDiD) Rajmonda S. Caceres, Edo Airoldi, Garrett Bernstein, Edward Kao, Benjamin A. Miller, Raj Rao Nadakuditi, Matthew C. Schmidt, Kenneth Senne, Steven T. Smith, Leah Weiner GraphEx 2014 This work is sponsored by the Assistant Secretary of Defense for Research & Engineering under Air Force Contract #FA C Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the United States Government.

2 Program Motivation Network Detection Applications ISR Cyber Social Network Detection Techniques Signal Processing for Graphs Space-Time Threat Propagation Challenge: What are the performance bounds for graph detection techniques when used on application datasets? Presentation Name CANDiD - GraphEx Initials MM/DD/YY R.Author Caceres

3 Network Detection Performance State of Art Real-World Full Covert Networks Fidelity Empirical These Do Not Exist Closed-Form Analysis (complex model) Analytical Monte Carlo Simulations Empirical performance Real-world examples Single datasets Monte Carlo simulations Single Experiment Empirical Closed-Form Analysis (simple model) Analytical These Exist Empirical s 1000s Number of Cases Analytical performance Currently limited to simple models

4 Outline Motivation and Background Detection on Graphs Random Matrix Theoretic Bounds Inference and Estimation on Graphs Optimum Bayesian Network Detection Model Selection Future Work

5 Spectral Graph Detection Bounds Using Random Matrix Theory to Define Bounds Graph Model Graph Matrix as Low Rank Matrix plus Random Matrix Closed-Form Expression of Graph Spectrum Detection Bound Definitions Uniform Edge Probability (Erdos-Renyi) Clique ~Wigner A = k(1 p)uu T + random P(detect) = 1 if k 2 /n > p/(1 p) Arbitrary Expected Degrees (Chung-Lu) Degrees ~Wigner A = dd T /V + random N/A Community Structure (Stochastic Block) A = ½(c in +c out )11 T + ½(c in c out )uu T + random E-R Community ~Wigner Prior work with RMT has produced graph spectra and detection bounds for only simple models of background graphs

6 Expanding Random Matrix Theory to More Complex Graph Models 1 Complex Graph Model Spectrum with Blockmodel and Arbitrary Degrees Model: Community structure with arbitrary expected degree 1 Random matrix theory used to derive closed-form expression of graph model spectrum 2 Ongoing 1 P(detect) = 1 if N fg p fg > [½(c in +c out )] Background Model: Community structure with arbitrary expected degree Foreground Model: Clique PD N fg = 24 N fg = N fg p fg Academic collaborators have expanded theory to define spectrum and bounds for more expressive graph models 1 Brian Karrer, Mark E. J. Newman: Stochastic Block Models and community structure in networks, Phys. Rev. E (2011). 2 Xiao Zhang, Raj Rao Nadakuditi, Mark E. J. Newman: Spectra of random graphs with community structure and arbitrary degrees. Submitted to Phys. Rev E. (2014)

7 Outline Motivation and Background Detection on Graphs Random Matrix Theoretic Bounds Inference and Estimation on Graphs Optimum Bayesian Network Detection Model Selection Future Work

8 Hybrid Mixed-Membership Block Model Poisson Interaction Rate: Chung-Lu Model λ ij = λ iλ j λ π T i Bπ j I ij k k Mixed Membership Block Model Erdös-Rényi Model Observed Interaction Data: Log-likelihood function: l M ( i ) / X j2i ij =1 CL ij m ij P ois( ij T ) m ij is the number of interactions between i and j [m ij log T i B j CL ij T T i B j] Intuition: Setting the first derivative to zero shows that the log-likelihood is maximized when the observed interaction equals the expected interaction Hybrid Mixed-Membership Block model captures important properties of real-world network data m ij λij

9 Estimation Bounds for the Hybrid Mixed-Membership Block Model Fisher Information Derivation Cramér-Rao Bounds The inverse of the Fisher information (i.e. the Cramér-Rao bound) gives the minimal variance of any unbiased estimate of a membership vector Cramér-Rao bounds on membership estimation lead to theoretical bounds for the detection problem Line Leads - 10 of 12 CANDiD MCS 12/13/ GraphEx 2014

10 Outline Motivation and Background Detection on Graphs Random Matrix Theoretic Bounds Inference and Estimation on Graphs Optimum Bayesian Network Detection Model Selection Future Work

11 Optimal Bayesian Network Detection Stochastic Realization Threat Propagation Optimal Detector Neyman-Pearson optimal detector defined using likelihood-ratio test v - Threat at vertex v z u - Observation of threat at Threat exists at a node if it propagated thru the node to the observation of the threat The probability of threat at a vertex is averaged over all random walks u Threat at a vertex is a weighted sum of it s neighbors threat Efficient to solve for unknown threat given observed threat This is equal to probability of threat given observations The harmonic threat propagation values provide an optimal detection test for a given CFAR Harmonic threat propagation yields N independent tests Stochastic realization used to show the optimality of Harmonic Threat Propagation detection method for given diffusion models CANDiD Line Leads - GraphEx - 12 of R. MCS Caceres 12/13/2013 Smith et al., Bayesian Discovery of Threat Networks, IEEE Transaction on Signal Processing, 2013

12 Outline Motivation and Background Detection on Graphs Random Matrix Theoretic Bounds Optimum Bayesian Network Detection Inference and Estimation on Graphs Model Selection Future Work

13 Model Selection for Graphs Research Question Given an observed graph and a collection of generative graph models, what model is the observed graph the closest to? Technical Approach Construct graph ensembles using given generative models Embed each graph into feature space Build classifier Apply classifier to the observed graph M 1 M 2 M 3 f 2 Graph Feature Space Graph Instances f 1 G obs Discriminate generative models in topological feature space Presentation Name - 14 Author CANDiD Initials - GraphEx MM/DD/YY 2014

14 Model Selection for Graphs Realistic Models - Community structure - Power law - Sparsity - Large n Topological Measures - Density, triad count, etc Random Forest - Robust to noise - Computationally feasible GRAPH MODELS GRAPH ENSEMBLE GENERATION GRAPH INSTANCES FEATURE EMBEDDING FEATURE DATA GRAPH CLASSIFICATION MODEL GRAPH CLASSIFIER Theoretical Detectability bound Presentation Name - 15 Author CANDiD Initials - GraphEx MM/DD/YY 2014

15 Outline Motivation and Background Detection on Graphs Future Work

16 Operational Network Detection Operational CANDiD Community Structure Commercial HADR DoD/IC Cyber Topological Realism CANDiD Cliques Single Source Perfect Knowledge Multi-Source Data Realism Data Uncertainty

17 Multisource and Uncertainty Preliminary Results o v ψ 3 Multi-INT: Detection Threat Network o MOVINT track P2 D5 3 P1 D2 P3 P5 D6 4 1 D3 P4 D7 ψ 1 o v Positive + entries Adjacency Matrix (Symmetric) D1 D4 2 GEOINT TXT ψ 2 SIGINT ψ 4 Bayesian model Multi-INT Threat at v determined by neighboring threat P(o) P(2 o) P(3 o) P(4 o) P(5 o) P(v o) x b x i = ψ v v P(o) P(2 o) P(3 o) P(4 o) P(5 o) P(v o) Threat Probabilities Multi-INT: Aggregation Detect communities based on various observed types of interactions Boosting approach enables community detection via aggregation Performance meets the theoretical bound for the underlying graph Probability of Detection Uncertainty Cluster Detection in Web of Science fusion significantly reduces false alarms Probability of False Alarm Uncertainty models Missing edges False edges Missing vertices Metadata confusion Hypothesis: Data fusion recovers performance e P(e) 1/2 (1,2) 1/2 Recent efforts demonstrate potential of multi-observation fusion Operational CANDiD will lead to relevant performance bounds Smith et al., IEEE Trans. SP, 2014; Collins and Smith, Proc. FUSION, 2014 Caceres et al., Proc. SDM Workshop Mining Large Networks and Graphs, 2014 Miller and Arcolano, Proc. ICASSP, 2014

18 Papers Xiao Zhang, Raj Rao Nadakuditi, Mark E. J. Newman: Spectra of random graphs with community structure and arbitrary degrees. Phys. Rev E. (2014). Rajmonda S. Caceres, Kevin C. Carter, Jeremy Kun, A Boosting Approach to Learning Graph Representations, SDM Workshop: Mining Large Networks and Graphs, (2014). Benjamin A. Miller, Nicholas Arcolano, Spectral subgraph detection with corrupt observations, Proc. ICASSP, 2014 Steven T. Smith, Edward K. Kao, Kenneth D. Senne, Garret Bernstein, Scott Philips, Bayesian Discovery of Threat Networks, IEEE Transaction on Signal Processing, (2013). Benjamin A. Miller, Nadya T. Bliss, Patrick J. Wolfe, and Michele S. Beard, Detection Theory for Graphs, Lincoln Laboratory Journal, (2013). THANK YOU

19 Candid - GraphEx 2014 RSC 08/21/2014 BACKUP

20 CANDiD Overview Current Evaluation Method Uncertainty CANDiD Focus Areas Closed-Form Bounds Run detection method on observed data Validate any detections Could another detection method have done better? Random Matrix Theory Optimal Network Detection Membership Estimation Bounds Model Selection Run detection method on many simulated graphs Evaluate performance across simulations Does detection performance transfer to observed data? Topological Classification Establishing closed-form bounds and a framework for model selection address uncertainties of current evaluation methods Line Leads - 21 of 12 CANDiD MCS 12/13/ GraphEx 2014

21 Space-Time Threat Propagation* Space Space-Time Graph G T = (V T, E T ) Non-Threat Inferred Threat Bayesian Prior: Observed Threat Time Optimum Neyman-Pearson network detection Maximizes Probability of Detection 80 STTP BFS Algorithm: For each node, propagate threat belief to neighboring nodes Harmonic analysis on graph Comparable to Google s Page Rank algorithm Space-Time graph analytics infers threat at network sites based on Bayesian propagation model PD (%) 40 Random 0 0 NFA 200 λ P(site s i red) = α s N(s i ) v N (s i ) P(v red) + (1 λ) max v N (s i ) P(v red) Candid - GraphEx 2014 RSC 08/21/2014 *Smith et al. Lincoln Laboratory Journal (2012)