Localized Anomaly Detection via Hierarchical Integrated Activity Discovery

Transcription

1 Localized Anomaly Detection via Hierarchical Integrated Activity Discovery Master s Thesis Defense Thiyagarajan Chockalingam Advisors: Dr. Chuck Anderson, Dr. Sanjay Rajopadhye 12/06/2013

2 Outline Parameter Estimation PLSA PLSM IPLSM Anomaly Detection Application to Brain Computing Interfaces (BCI) 2

3 Parameter Estimation What do we need to know to guess the outcome of a biased coin? 3

4 Parameter Estimation The probability of either heads or tails 4

5 Maximum Likelihood Estimation-MLE Given a set of observations The probability of observing Find the parameter that will produce this observation is the number of heads 5

6 Graphical Model x x 1 x 2 x 3 x N N 6

7 Maximum Aposteriori Estimation-MAP Incorporates a mechanism for including a belief or previous results in the parameter estimation x N 7

8 Generalize We need 5 parameters for a cube and N-1 parameters for an N sided die 8

9 Two dice problem x = {1,2,3,4,5,6} z = {Dice1,Dice2} z x N 9

10 Two dice Incomplete data problem x = {1,2,3,4,5,6} z = {Dice1,Dice2} n(x) = count vector z x N 10

11 Expectation Maximization-EM algorithm Use EM algorithm to complete the data and find the parameters 11

12 Expectation Maximization-EM algorithm E Step M-step 12

13 Probabilistic Latent Semantic Analysis (PLSA) 13

14 Generative Model Pick a document d with probability - P(d) Pick a topic z given d with probability - P(z d) Pick a word w given z with probability - P(w z) 14

15 PLSA Illustration w w d z d 15

16 PLSA Learning Input Term-document matrix Number of topics Output 16

17 PLSA Learning E-step M-step 17

18 PLSA Inference Input Term-document matrix Topics Output 18

19 PLSA on images Has been successfully adapted to capture frequently co-occuring pixels in images 19

20 Topics Captured by PLSA Pros Captures polysemy Captures spatial alignment of topics in images successfully Cons Needs other models like Hidden Markov Model to capture temporal patterns Over fitting 20

21 Probabilistic Latent Sequential Motifs (PLSM) 21

22 PLSM Learning ts z tr w ta tr 22

23 PLSM Model 23

24 PLSM Learning Input Term-time-document matrix Number of motifs Length of each motif Output Motifs Motif start time 24

25 PLSM Inference Input Term-time-document matrix Motifs Output Motif start time 25

26 PLSM Advantages o o Can identify temporal patterns Can identify the start of temporal patterns and multiple temporal patterns can start at the same time Disadvantages o o Scaling No restriction of patterns at a given time 26

27 Motifs 27

28 Temporal Order of Topics 28

29 PLSA on top of PLSM ts z z ll tr ts tr 29

30 PLSA + PLSM Dimensionality Reduction using PLSA Capture motifs on topics discovered from PLSA with PLSM Input to PLSM is the output of PLSA 30

31 Cons Lower level (PLSA) is independent of PLSM The information captured by PLSA may not be temporally relevant 31

32 My Contributions Integrating PLSA and PLSM Anomaly Detection Application to BCI 32

33 My Contributions Integrating PLSA and PLSM Anomaly Detection Application to BCI 33

34 Integrated PLSM (IPLSM) Integrated PLSM formulates information feedback as Dirichlet prior 34

35 IPLSM Intuition for PLSM as corrector for PLSA 35

36 Reconstructed video 36

37 IPLSM How much prior is required? The prior strength should be less than the input strength 37

38 Traffic camera 38

39 Hierarchical Model PLSM on top of PLSMs detect patterns spanning multiple cameras 39

40 Metro Station -1 Iteration 0 Iteration 4 40

41 Metro Station -2 Iteration 0 Iteration 4 41

42 My Contributions Integrating PLSA and PLSM Anomaly Detection Application to BCI 42

43 Anomaly Detection Anomalies are unusual patterns in data Vanilla Algorithm 43

44 Issue 44

45 Vanilla Algorithm Problem : Recognizes frames with high activity as abnormal Solution : Vanilla algorithm normalized 45

46 Issue Anomaly detected only when frame as a whole is abnormal 46

47 Blocked algorithm Solution : Divided the images into blocks or sub images and apply the normalized algorithm on each block 47

48 Localization of anomaly Apply Kadane 2D algorithm on the blockerror matrix to obtain the spatial anomaly To apply Kadane 2D algorithm is mean subtracted 48

49 Anomaly on metro 49

50 Result - Anomaly detection on traffic cameras 50

51 UQM Roundabout 51

52 UQM Junction 52

53 Traffic Junction VASA vehicle stopping after stop line. ZC people crossing the road away from zebra crossing PA car stopping in the pedestrian area Traffic Junction 53

54 My Contributions Integrating PLSA and PLSM Anomaly Detection Application to Brain Computing Interfaces 54

55 Application to BCI Dataset Consists of EEG signal of 8 channels of 256Hz Sampling Frequency 9 subjects were used in the study 4 Mental tasks each 10 seconds long were performed by all subjects 6/5 trials were performed, each trial includes all 4 mental tasks Mental Tasks silently sing a song (S) visualize a rotating cube (R) imagine right hand clenching (F) counting backwards from 100 in steps of three (C) 55

56 BCI Channels F3, F4, C3, C4, P3, P4, O1, O2 56

57 Experimental Setup Features FFT of window length 256 with 90% overlap 89 windows for a 10 second signal 129 frequency components in each channel is concatenated with 8 channels to form 1032 words dictionary 89 windows represent each task Model PLSA was used for dimensionality reduction and detecting cooccurring frequencies PLSM for temporal pattern extraction SVM for classification 5-fold cross validation for model selection 57

58 Results PLSA 58

59 Accuracy Subject Impaired 5 trials 6 trials Topics Accuracy Topics Accuracy 11 Yes Yes No No No No No No No No No

60 Topic distribution 60

61 Counting activity Delta wave activity seen during sleep and high concentration tasks. 61

62 Singing and Rotating 62

63 Eye blink Power decreases from frontal to parietal region 63

64 Results PLSM on top of PLSA 64

65 Accuracy for 5 trails Subject Impaired Motifs Motif Length Accuracy 11 Yes Yes No No No No No

66 66

67 Prior Belief 67

68 Expectation Maximization-EM algorithm Given a set of incomplete data The Likelihood of X The ML estimate is given by 68

69 Reconstruction Error 69