Anomaly Detection in Cyber Physical Systems

Transcription

1 Anomaly Detection in Cyber Physical Systems Maggie Cheng Illinois Institute of Technology December 11, 2018 IEEE Big Data Conference Seattle, WA

2 Outline Introduction Outlier Detection Sequential Change Point Detection Anomaly Identification Introduction 2

3 Types of Anomalies Outliers location outliers scatter outliers Change Points on Time Series continuous beginning of a new trend change of state of the underlying process Introduction 3

4 Change point may not be an outlier Outlier vs. Change Point Pictures/changepoint-eps-converted-to.pdf Outlier may not be a change point Pictures/outlier-eps-converted-to.pdf Introduction 4

5 Outline Introduction Outlier Detection Sequential Change Point Detection Anomaly Identification Outlier Detection 5

6 Detection Techniques Statistical outlier detection (Barnett and Lewis 1994) Projection-based outlier detection Distance-based outlier detection (Knorr and Ng 1998) - based on notion of proximity - high-dimensional data: notion of proximity ( ) - sparsity makes every data point an outlier Outlier Detection Algorithms for High-Dimensional Data Naïve brute force: exhaustive search (Slow!) Principle components (for dimension reduction) Outlier Detection 6

7 Outline Introduction Outlier Detection Sequential Change Point Detection Anomaly Identification Sequential Change Point Detection 7

8 Sequential Change Point Detection Sequential Change Point Detection To detect a change point in a time series {X 1, X 2,..., X n }, it is assumed that the pre-change density is f 0, and if a change occurs at time ν, then the post-change density becomes f 1 beginning from moment ν + 1. The hypotheses are then formulated as: H 0 : {X 1, X 2,..., X n } f 0 H 1 : {X 1, X 2,..., X ν } f 0, and {X ν+1, X ν+2,..., X n } f 1 The Change Point Detection Problem is to decide (1) which hypothesis is true? (2) if H 1 is true, ν =? The time instance ν, at which the state of the process changes is referred to as the change point or time of change. Sequential Change Point Detection 8

9 Sequential Change Point Detection Algorithms Cumulative Sum Algorithm (CUSUM) Shiryaev-Roberts Procedure Sliding Window Algorithm Performance Metrics False positive and false negative rates - If a change occurred but the detection procedure failed to detect it: false negative (misdetection) - If the detection time N < ν: false positive (false alarm) Detection delay - If there is a true change and the time of change is ν, the detection time is N, then detection delay τ = N ν Sequential Change Point Detection 9

10 CUSUM (Page, 1954) Optimality CUSUM is optimal in the sense of minimizing worst case detection delay. Assumptions Observations X 1, X 2,... X n are independent, iid pre-change and iid post-change Probability density functions: f 0 before change; f 1 after change Assume f 0 and f 1 are known The only thing unknown is ν, the time of change Parameter Detection threshold h Sequential Change Point Detection 10

11 CUSUM Parametric CUSUM: Based on Maximum Likelihood Principle Detection statistics: where W 0 = 0 Z n = log L n W n = max(w n 1 + Z n, 0) for n 1 L n is the likelihood ratio: L n = f1(xn Xn 1 1 ), or L f 0(X n X n 1 1 ) n = f1(xn) f 0(X n) for i.i.d. The procedure declares a change as soon as the detection statistics W n exceeds a preset threshold h: N = min{n 1 : W n h} Sequential Change Point Detection 11

12 Shiryaev-Roberts Procedure The Problem Setting Limited to a change of mean or to normal observations Original Setting: Quickest Detection of a Disorder in a Stationary Regime - Detecting a change in the mean of a Brownian motion - The change is possibly taking place at a far horizon A randomized version for a general discrete time setting - Brownian motion techniques are not applicable in discrete time Applications: target detection and tracking, rapid detection of intrusions in communication networks, environmental monitoring - Early detection of changes that may occur in a distinct future Sequential Change Point Detection 12

13 Shiryaev-Roberts Procedure The Algorithm 1. Shiryaev-Roberts statistic R n = n k=1 2. From independence assumption: R n = n p(x 1,...,X n ν=k) p(x 1,...,X n ν= ) n f 1(X i) f 0(X i) k=1 i=k 3. R n can be computed recursively: R n = (1 + R n 1 ) f1(xn) f 0(X n), for n 1; R 0 = 0 4. Stopping time: R AB = min{n 1 : R n A B } Parameter: A B is chosen such that E N AB = B B is a preset value before surveillance begins. Sequential Change Point Detection 13

14 Shiryaev-Roberts Procedure Detection Delay Shiryaev-Roberts procedure is the best in terms of minimizing the expected detection delay (asymptotically). Theorem Shiryaev-Roberts procedure minimizes E k (N k) + k=1 over all stopping times N that satisfy E (N) B. CUSUM and S-R Procedure Based on ratio of likelihoods - S-R procedure is a CUSUM-type of algorithm Difficult to apply when f 1 and f 0 are unknown Sequential Change Point Detection 14

15 Preset Parameters Sliding Window Algorithm window size m (m N, the total number of data points) the level of significance α (e.g., α = 0.05) Sliding Window Algorithm (Cheng & Wu, 2016) 1. Set window offset d = Compute the sum S 1 = d+m i=d+1 X i, and S 2 = d+2m i=d+m+1 X i. 3. If S 2 S 1 zσ 2m, declare a change point ˆν = d + m 4. Else set d = d + 1, go to line 2. Remarks: z is the critical value that provides an area of α in the upper tail of the standard normal distribution. σ 2 is the variance, updated as the window moves Sequential Change Point Detection 15

16 Sliding Window Algorithm Algorithm Properties Be able to detect a change in state without knowing the actual preand post- change densities Relate detection threshold to a tolerable false alarm rate controlled trade-off Relate detection threshold to the dynamic characteristics of the data and not use a preset value Be able to detect abrupt changes as well as slow and subtle changes Avoid mistaking an isolated outlier as a change for a new state Sequential Change Point Detection 16

17 Applications Using Sequential Change Point Detection DoS Attack Detection Attack Detection in Wireless Networks Power Grid Anomaly Detection Sequential Change Point Detection 17

18 DoS Attack Detection A Common DoS Attack: SYN Flood Attack Attacker sends control packets to compromised nodes A large number of flooding sources send an excessive number of SYN requests to the victim The victim server returns SYN/ACK packet to the client waiting for ACK until timeout Flooding sources never return an ACK Exhaust the victim server s backlog queue all connection requests dropped Challenges: preset threshold ( ) Traffic patterns vary from site to site, from time to time Per-flow state information not known Normal traffic models hard to define Sequential Change Point Detection 18

19 DoS Attack Detection How to detect w/o prior knowledge of flow and traffic info? Detection mechanism must be insensitive to site and traffic patterns. There is no normal traffic model or flow rate, but there is normal behavior Baseline: protocol behavior (TCP connection management) - Normal: FINs match with SYN requests from clients - Packet drop/retransmission cause small discrepancy - Under SYN flood attack: Large difference between the number of SYNs and FINs received Sequential Change Point Detection 19

20 SYN Flood Attack Attackers create a large number of open connections Change in network measurement: SY N F IN shows abrupt increase Sequential Change Point Detection 20

21 DoS Attack Detection Detection Procedure Monitor the number of SYNs and FINs - at egress router (near the flooding source) - at ingress router (near the victim server) Generate time series on (SYNs FINs) Perform sequential change point detection on time series - Non-parametric version Sequential Change Point Detection 21

22 DoS Attack Detection Non-Parametric CUSUM for Change Point Detection Tunable parameters: a, N Observations: S: number of SYNs; F : number of FINs 1. D n = S n F n 2. R n = α(r n 1 ) + (1 α)f n 3. X n = D n /R n 4. choose constant a > E(X n ) 5. Test statistic: y n = (y n 1 + (X n a)) +, y 0 = 0 6. Detection: first n such that y n > N Remarks Algorithm very sensitive to N and a. Difficulty: determining N and c before monitoring begins. Sequential Change Point Detection 22

23 Wormhole Attack in Wireless Ad Hoc Networks Routing: A category of routing protocols use shortest path routing. Nodes exchange local information and relay to others Nodes collectively decide a route towards a destination Select the best route based on hop count (shortest path routing) Wormhole Attack Adversary controls two end points and a tunnel between them Attract traffic to go through the controlled wormhole tunnel by making false route advertisement a shorter path towards a destination Consequences of a Wormhole Attack Route changes not necessarily for the better Further damage: other attacks possible after a wormhole tunnel is established Drop packets, launch a denial of service attack, or gain unauthorized access to other information, etc. Sequential Change Point Detection 23

24 In-Band vs. Out-Band Wormhole Attack In-Band Wormhole Attack Wormhole tunnel consists of other wireless nodes controlled by the adversary Re-routed packets go through these wireless nodes Out-Band Wormhole Attack Wormhole tunnel is an external link A wired link A wireless link (e.g., a long-range directional link) A B A B This talk: address in-band wormhole attack Sequential Change Point Detection 24

25 Wormhole Attack Detection Performance Degradation in an In-Band Wormhole Attack End-to-end Delay increases Throughput decreases Packet Deliver Ratio drops (If the wormhole endpoints drops packets arbitrarily) and more... Detection Logic If a route change (to the shorter one) is coupled with an abrupt increase in end-to-end delay, it is very likely the packets are detoured to go through an in-band wormhole tunnel. In a non-attack scenario, changing to a shorter route is often coupled with decreased delay Proposed Method Model end-to-end delay of a flow as a time series Detect changes in the delay time series at the moment of attack Use a Change Point Detection method to detect the change Sequential Change Point Detection 25

26 Stationary Network Setup An in-band wormhole tunnel is established between node 1 and node 2 at 50 seconds. wormhole tunnel: advertised as one hop 1-2 Two flows, without other traffic in the background Flow 18 28: Before 50 seconds: use path After 50 seconds: use path Flow 17 38: Before 50 seconds: use path After 50 seconds: use path Sequential Change Point Detection 26

27 Stationary Network Result I End to end Delay (seconds) Simulation Time (seconds) Sequential Change Point Detection 27

28 Stationary Network Result II Three flows that changed routes: 9 2, 18 28, There are other flows in the background that stayed on the original routes End to end Delay (s) End to end Delay (s) Simulation Time (s) Simulation Time (s) Figure: Packet size 256B, interval=0.01s, 0.025s Figure: Packet size 256B, interval=0.02s, 0.01s Sequential Change Point Detection 28

29 Mobile Network Setup Mobility Model All nodes are wandering within a small range around their original positions using the random walk 2d mobility model. Case Study A benign case, node 1 are node 2 move towards each other, no wormhole attack. Route change is caused by node mobility, and the actual path length is indeed decreased. An attack case, node 1 and node 2 start a wormhole tunnel. Route change is caused by wormhole, and the actual length is increased while it is believed to be decreased. Sequential Change Point Detection 29

30 Mobile Network Result End to end Delay (seconds) End to end Delay (seconds) Simulation Time (seconds) Simulation Time (seconds) Figure: Benign case: nodes 1 and 2 are moving towards each other. Figure: Attack case: nodes 1 and 2 start a wormhole tunnel. Sequential Change Point Detection 30

31 MAC-Layer Attack Detection in Wireless Networks IEEE MAC CSMA/CA RTS-CTS-DATA-ACK Sequential Change Point Detection 31

32 MAC Layer Misbehavior in IEEE Networks Sender Selfish Behavior Manipulation on carrier sense time Manipulation on back-off value during contention Consequences Channel-capturing effect: other nodes have less chance to transmit Receiver Selfish Behavior RTS dropping attack Consequences Clear channel for itself Sender waste resource retransmit RTS Sequential Change Point Detection 32

33 MAC Layer Misbehavior Detection Other Flows Experience Performance Degradation End-to-end delay increases Throughput decreases Packet interval increases Attack Case vs Non-Attack Case If the trend of change in end-to-end delay (throughput, packet interval, etc) is not the same for everyone, it is likely someone is acting selfishly In a non-attack scenario, increased traffic degrades performance, and has the same effect on everyone Detection Method Monitor packets received Compute per-flow end-to-end delay (or throughput, packet interval) as a time series Use the sliding window change point detection method to detect the change on time series Sequential Change Point Detection 33

34 Simulation Setup Case 1: Shorter DIFS attack normal sender: DIFS=SIFS+2 slot-time; attacker: switch to DIFS=SIFS starting at 50s SIFS=10 µs, slot-time=20 µs Case 2: Shorter DIFS and smaller back-off window γ normal sender: following binary exponential back-off, γ [32, 1024] attacker: use fixed γ = 2 Case 3: RTS dropping attack normal receiver: respond CTS for every RTS request attacker: RTS to CTS ratio 20:1 Sequential Change Point Detection 34

35 MAC Layer Misbehavior Detection Results Case 1: Five victim flows 19 1, 14 1, 12 1, 10 1, 6 1 Case 2: Same as case 1 Case 3: 20 2, 11 2, 8 2, 7 2, 5 2 Node 2 is the attacker in all cases Sequential Change Point Detection 35

36 Result I: Case 1 Delay (s) Delay Throughput (kbps) Throughput Simulation Time (seconds) Simulation Time (seconds) (d) Delay (e) Throughput Delay Mean (s) Delay Mean Throughput Mean (kbps) Throughput Mean Simulation Time (seconds) Simulation Time (seconds) (f) µ D (g) µ T Sequential Change Point Detection 36

37 Result II: Case 2 Delay (s) Delay Throughput (kbps) Throughput Simulation Time (seconds) Simulation Time (seconds) (h) Delay (i) Throughput Delay Mean (s) Delay Mean Throughput Mean (kbps) Throughput Mean Simulation Time (seconds) Simulation Time (seconds) (j) µ D (k) µ T Sequential Change Point Detection 37

38 Result III: Case 3 Tx Data Rate (kbps) Tx Data Rate Transmitted Packet Interval (s) Tx Packet Interval Simulation Time (seconds) Simulation Time (seconds) (l) Throughput (m) Packet Interval Tx Data Rate Mean (kbps) Tx Data Rate Mean Tx Packet Interval (s) Cumulative Average Simulation Time (seconds) Simulation Time (seconds) (n) µ T (o) µ I Sequential Change Point Detection 38

39 Jamming Attack Detection in Wireless Networks Attacks All nodes exposed to open medium Jamming signals: using higher transmission power, do not have to follow MAC protocol Legitimate nodes suffer - TDMA: collision, increased packet error rate and drop rate - CSMA: collision, channel capturing Detection Procedure Detect changes from network measurements (delay, throughput, error rate, packet delivery ratio, signal strength, IFS, etc) Distinguish - Jamming weak signals from legitimate nodes - Jamming network congestion among legitimate nodes Sequential Change Point Detection 39

40 Jamming Attack Detection in Wireless Networks Detection Methods Use summary information in a time interval, compare against a preset detection threshold - Not suitable for highly dynamic networks Use change point detection on time series - Test statistic: delay, throughput, received packets IFS e.g.: Throughput when jamming signal duration varies (p) s (q) 0.8s (r) 1.5s Sequential Change Point Detection 40

41 Anomaly Detection in Power Grids Types of Anomalies Line outage - wild animals - weather - over-grown trees - coupled with aging infrastructure + lack of maintenance Generator outage Transformer fault Human errors Cyber attacks Sequential Change Point Detection 41

42 Error Detection in Power Grids Methods Detection Based on State Estimation - Least Square, Weighted Least Square Works for measurement errors (bad data detection) Line outage: topology change often causes conforming errors - Errors may go undetected Slow - Not suitable for real-time detection Sequential Change Point Detection 42

43 Error Detection in Power Grids Other Methods: Skip State Estimation Wu & Cheng 2016 real-time anomaly detection + identification in two steps Sequential Change Point Detection 43

44 Change Point Detection in High-Dimensional Data Multi-Stream Change Point Detection Detection Procedure Sequential Change Point Detection 44

45 Outline Introduction Outlier Detection Sequential Change Point Detection Anomaly Identification Anomaly Identification 45

46 Challenges Missing Data or Gappy Data Detection with Missing Data Anomaly Identification 46

47 Challenges High-Dimensional, Heterogeneous Data with Lead and Lag Anomaly Identification 47