Dynamic Pattern Detection with Temporal Consistency and Connectivity Constraints

Transcription

1 Dynamc Pattern Detecton wth Temporal Consstency and Connectvty Constrants Skyler Speakman School of Computer Scence, Carnege Mellon Unversty Abstract Ths work explores scalable and accurate dynamc pattern detecton methods n graph-based data sets. The proposed Dynamc Subset Scan method s appled to the task of detectng, trackng, and source-tracng contamnant plumes spreadng through a water dstrbuton system equpped wth nosy, bnary sensors. Whle statc patterns affect the same subset of data over a perod of tme, dynamc patterns may affect dfferent subsets of the data at each tme step. These dynamc patterns requre a new approach to defne and optmze penalzed lkelhood rato statstcs n the subset scan framework, as well as new computatonal technques that scale to large, real-world networks. To address the frst concern, ths work develops a new subset scan methods that allows the detected subset of nodes to change over tme, whle ncorporatng temporal consstency constrants to reward patterns that do not dramatcally change between adacent tme steps. Second, the Addtve GraphScan algorthm allows ths novel scan statstc to process small graphs 500 nodes n 4.1 seconds on average whle mantanng an approxmaton rato over 98% compared to an exact optmzaton method, and to scale to large graphs wth over 12,000 nodes n 30 mnutes on average. Evaluaton results across multple detecton, trackng, and source-tracng tasks demonstrate substantal performance gans acheved by the Dynamc Subset Scan approach. 1 Introducton Detectng patterns n massve data sets has multple real-world applcatons n felds such as publc health, law enforcement, and securty. The subset scan approach to pattern detecton treats the problem as a search over subsets of data, wth the goal of fndng anomalous subsets. Ths approach poses two man challenges: approprately evaluatng the anomalousness of a gven subset, and the computatonal ssue of searchng through the exponentally many subsets of the data. Prevous approaches [8, 12, 10, 11] have addressed the frst concern by scorng each subset usng lkelhood rato statstcs such as the expectaton-based Posson EBP [12, 10] or expectaton-based bnomal EBB [8] scan statstcs. Ths current work allows for a more sophstcated scorng functon by penalzng the lkelhood rato statstc, consderng addtonal pror nformaton from each data element. The penalzed lkelhood rato typcally does not satsfy useful propertes such as lnear-tme subset scannng [11], makng effcent optmzaton over subsets a challengng task. However, the EBB lkelhood rato statstc can be wrtten as an addtve functon, enablng effcent optmzaton of a penalzed verson of that statstc over subgraphs. The frst maor contrbuton of ths work s the development of temporal consstency constrants whch allow for addtonal penaltes or rewards to act on the scorng functon, rewardng spatal subsets that are temporally consstent wth each other, and effcent optmzaton of the resultng, penalzed scan statstc to detect dynamc clusters subect to these constrants. The 1

2 second maor contrbuton s the Addtve GraphScan algorthm, whch effcently dentfes anomalous hgh scorng, connected subgraphs, thus ncorporatng both hard connectvty constrants and soft temporal consstency constrants. Each of these contrbutons wll be dscussed n detal below. Many complex data sets contanng emergng events or patterns are commonly represented n a known and fxed graph structure. Examples of ths nclude water ppelnes, transportaton routes, power grds, and supply chans n general. Whle other recent work [9, 5] has focused on learnng graph structure, here we assume a gven graph structure and wsh to detect whch nodes are currently affected, by observng data produced at the nodes of the graph on each tme step. The motvatng example comes from the feld of publc health: wth focus on detectng, trackng, and source-tracng contamnant plumes n a water dstrbuton system. Creatng sensor networks for detectng delberate or accdental contamnaton of these systems has been a popular research doman followng the terror attacks of September 11, The Battle of the Water Sensor Networks" BWSN [1] provded real-world data to teams tasked wth placng perfect sensors to quckly detect contamnants and lmt the amount of contamnated water consumed by the populaton. The placement problem s an nterestng one explored further n [3, 7]. Ths current work focuses on the complementary problem of fusng data collected from nosy sensors assumng a gven placement. Sensor fuson attempts to combne data from multple dstrbuted sensors n order to ncrease the detecton power of the entre network [14]. The smulaton proceeds by modelng smple, bnary sensors at each ppe uncton graph node n the system wth a fxed false postve rate e.g., FPR = 0.1 and true postve rate e.g., TPR = 0.9. An addtonal assumpton s that each sensor operates ndependently of the others n the network. The smulatons use the network structure and plumes provded n the BWSN data to generate sensor readngs over the course of 12 one-hour ntervals. The task s then: Gven 1 the graph structure ppe network, 2 false postve and true postve rates of the sensors, and 3 ndependent observatons from the sensors over tme, the methods must provde: A whether or not a contamnant s present n the system for each onehour tme step, B hour-to-hour trackng of whch ppe nodes have been affected by the plume on the current and recent past tme steps, and C source-tracng to determne whch nodes n the system spawned the contamnant. Correspondng evaluaton metrcs nclude: A average tme n hours to plume detecton as a functon of false postve rate number of false alarms per month, B spatal-temporal overlap coeffcent between the true and detected subsets of nodes over tme, and C spatal overlap coeffcent between the true and dentfed subsets of source nodes. Ths s not the frst work to apply spatal or subset scan statstcs to contamnaton early warnng systems. Koch and McKenna [6] used Kulldorff s spatal scan [8] to detect statstcally sgnfcant crcular clusters of anomalous actvty. They used propertes of the ppe network to create a dstance metrc based on travel tme between sensng nodes n order to defne ther crcles. However, they were not able to enforce connectvty constrants and take advantage of the topology of the network. Through our Addtve GraphScan algorthm, we are able to search over connected subsets of the ppe network to fnd anomalous connected subgraphs. Berry et al. [2] have also consdered the detecton power of a network of mperfect sensors, showng that t s worth deployng a sensor network even when ndvdual sensors have low detecton probablty. However, ther experments dd not allow for sensors wth false postves, makng the detecton and source tracng problems much easer to solve as compared to the more dffcult scenaro consdered here. Spatal scan statstcs attempt to dentfy regons of nterest or hot spots. Ths s acheved by 2

3 maxmzng a scorng functon FS, typcally defned as the lkelhood rato FS = PrData H 1S, PrData H 0 over spatal regons S. In ths expresson H 1 S assumes ncreased actvty n regon S, and H 0 assumes regular behavor. Ths work montors bnary sensors s each producng c Bernoull FPR trggers under H 0 or c Bernoull TPR trggers for H 1 S contanng node s. Ths makes the expectaton-based bnomal scan statstc [8] a logcal choce. Spatal-temporal scan statstcs ncorporate the tme dmenson. It s standard to aggregate ths temporal nformaton over a tme wndow w so that c = t=1...w c t. Once the temporal nformaton has been aggregated for each wndow w = 1... W, maxmzng the spatal-temporal scan statstc for that wndow proceeds dentcally to the regular spatal scan statstc; t then maxmzes over all wndow szes from 1 to W. However, an nherent assumpton n ths aggregaton of temporal nformaton s that the affected spatal-temporal subset does not change over tme. Therefore, ths approach wll be referred to as the Statc scan method throughout ths text. The fundamental goal of ths work s to relax ths strong assumpton on the spatal-temporal structure n order to ncrease the power to detect dynamc patterns that change the affected regon over tme. One smple approach s to optmze each of the w tme steps ndependently. Ths allows for each tme step t to dentfy an entrely dfferent spatal regon, but does not allow the sharng of nformaton between tme steps, possbly reducng detecton power. Ths approach s referred to as the Independent scan method throughout ths text. As a compromse between Statc and Independent methods, the Dynamc Subset Scan s proposed whch enforces temporal consstency constrants to allow temporally adacent tme steps to share nformaton forward and backward n tme. As demonstrated below, ths flexblty ncreases power to detect and track dynamc patterns whle scalng to the sze of real-world networks. The rest of the paper s lad out as follows. Secton II ntroduces temporal consstency constrants, appled to the expectaton-based bnomal scorng functon, and demonstrates how the penalzed scan statstc can be effcently optmzed over not necessarly connected dynamc subsets. Secton III explans the Addtve GraphScan algorthm, whch effcently dentfes hghscorng, connected subsets of data wth an underlyng graph structure. Secton IV provdes emprcal results of the smulatons of the water dstrbuton network from the BWSN, comparng the Dynamc Subset Scan approach to several other approaches, and demonstratng mprovements n detecton, trackng, and source-tracng performance. Fnally, Secton V concludes the paper. 2 Temporal Consstency Constrants Ths secton has four sequental obectves. The frst s to demonstrate how the expectatonbased bnomal EBB scorng functon may ncorporate addtonal constrants whle remanng straghtforward to optmze over all possble subsets S.e., show that EBB can be wrtten as an addtve functon over the data elements s S. Second, s to show that these constrants may be nterpreted as the pror log-odds for a gven node s to be n the detected subset. The secton contnues by provdng the formal defnton of temporal consstency constrants based on a probablstc generatve model that ncorporates both forward and backward temporal consstency. The secton concludes wth a descrpton of the teratve optmzaton process that lnes up the spatal-temporal regon accordng to the provded temporal consstency constrants. 3

4 2.1 Addtve Scorng Functon and Addtonal Terms Condtoned on the false and true postve rates FPR, TPR of the sensors, the EBB statstc can be wrtten as an addtve set functon over the data elements s S. Ths s an mportant feature for two reasons. Frst, addtve functons are easy to optmze over all possble subsets. Wthout connectvty constrants, the optmzaton process s smply ncludng all records makng a postve contrbuton and excludng the rest. Determnng the most postve connected subset s more complcated, and s covered n the Addtve GraphScan secton below. Second, addtve functons allow for addtonal penalty terms to be ncluded at the element level whle the total penalzed scorng functon remans addtve and thus amenable to effcent optmzaton. Theorem 1. The expectaton-based bnomal statstc may be wrtten as FS = s S λ, where λ depends only on the bnary sensor response c for sensor s.e., whether that sensor trggers or not as well as the false and true postve rates of the sensors n general. Proof. The log-lkelhood rato form of the EBB scan statstc can be wrtten as follows: Then λ = c log TPR FPR FS = log PrData H 1S PrData H 0 = log s S Prc BernoullTPR s S Prc BernoullFPR TPR = log c 1 TPR 1 c s S FPR c 1 FPR 1 c [ ] TPR 1 TPR = c log + 1 c FPR log 1 FPR s S + 1 c log 1 TPR. 1 FPR Next, assume a bonus or penalty for each s S. These can easly be ncorporated nto the score functon. Defne: F pen S = FS + = λ +. s S s S F pen S s a penalzed form of the EBB scan statstc that s stll addtve over the data elements s. Note that the terms are assumed to be a functon of only the gven data element s ; they cannot depend on the entre subset S. Ths as a lmtaton of the current work and wll be nvestgated n extensons to more sophstcated penaltes n future work. 2.2 Pror Log-odds Interpretaton These soft constrants,, have a convenent nterpretaton as the pror log-odds that each data element s wll be ncluded n the detected subset. Let p be the pror probablty that data element s wll be contaned n the detected subset. Then can be defned as log p 1 p. The pror log probablty of selectng a subset S s then: 4

5 log PrS = log s S = s S = s S p 1 p s / S log p log1 p + N =1 N =1 log1 + exp. log1 p However, note that the term N =1 log1 + exp s constant and does not affect the probablty of selectng any partcular subset. Thus, ths term can be gnored when optmzng over all subsets of the data, and be subtracted once the hghest-scorng subset of the data has been dentfed. When > 0, record s s more lkely to be ncluded n the detected subset, and the opposte s true when < 0. When = 0 for all, whch s the default settng for spatal-temporal scan statstcs, then every subset S s consdered equally lkely a pror. 2.3 Dervaton of t Ths secton derves the formulas for t that correspond to the followng generatve model for temporal consstency. Let p t be the pror probablty that data element s wll be contaned n the detected subset S t on tme step t. Let x t be 1 f data element s s ncluded n S t, and 0 otherwse. Let n t be the number of neghbors of s that are ncluded n S t, and let k be the degree of node s. Then the generatve model of event propagaton, whch ncorporates temporal consstency constrants, s defned as: log p t 1 p t = β 0 + β 1 x t 1 + β 2 n t 1 k. 1 As a concrete example of the nterpretaton of ths model, assume β 0 = 1.5, β 1 = 5, and β 2 = 0. Then, f a node s ncluded n the prevous detected subset, S t 1, t has a 97% pror probablty of beng ncluded n the current detected subset, S t. If t was not ncluded n the prevous subset, then t only has an 18% probablty of beng ncluded n the current subset. When β 2 > 0, the proporton of neghbors ncluded n S t 1 wll further nfluence the pror probablty of s beng ncluded n the current subset. Consder t as the total mpact of ncludng x t on the overall penalzed log-lkelhood rato score FS, as compared to the score FS \ x t when xt s excluded. The log-lnear model of p t above provdes: t = logp t log1 pt + S t+1 + S t+1 logp t+1 x t logpt+1 x t log1 p t+1 x t log1 pt+1 x t. 2 x t 1 In equaton 2, the ntal dfference results from the pror probablty of x t, condtoned on and ts number of ncluded neghbors n t 1 from the prevous tme step. Ths dfference can 5

6 be calculated drectly from the model: logp t log1 pt = β 0 + β 1 x t 1 + β 2 n t 1 k. 3 The two sums n 2 account for the fact that ncludng x t changes the pror probabltes of x t+1 and ts neghbors n t+1 for the next tme step. These sums can be rewrtten as: logp t+1 x t logpt+1 x t + log1 p t+1 x t log1 pt+1 x t S t+1 S t+1 = β 0 + β 1 x t + β 2 S t+1 n t k x t f β 0 + β 1 x t + β n t 2 x t k 4 β 0 + β 1 x t + β 2 S t+1 n t k x t + f β 0 + β 1 x t + β n t 2 x k t, where the functon f x = log1 + expx. Next, note that the contrbutons to equaton 4 are equal to 0 for all nodes except for node and ts neghbors. For =, the correspondng terms n 4 smplfy to: β 1 x t+1 + f β 0 + β 2 n t k f β 0 + β 1 + β 2 n t k For each neghbor of, the correspondng terms n 4 smplfy to: x t+1 β 2 + f β 0 + β k 1 x t + β 2 n t k. 5 f β 0 + β 1 x t + β n t k Addng the contrbutons of equatons 3, 5, and 6 provdes: t = β 0 + β 1 x t 1 + x t+1 + f + f β 0 + β 2 n t k f β 0 + β 1 x t + β 2 + β 2 nt 1 + k β 0 + β 1 + β 2 n t n t k k 1 k S t+1 f β 0 + β 1 x t + β n t + 1 2, k 7 where the sums are taken over all neghbors of. In the specal case of β 2 = 0, equaton 7 smplfes to: t = β 0 + β 1 x t 1 + x t+1 + f β 0 f β 0 + β 1. 8 Note that, when β 2 = 0, t can be computed exactly. When β 2 = 0, t can must be approxmated assumng that β 0 + β 2 0 and β 0 + β 1 0. Notng that f x 0 when x 0, and 6

7 f x x when x 0, provdes: t β 0 + β 1 x t 1 + x t+1 n t β 0 + β 1 + β 2 k S t =β 1 x t 1 + x t β 2 + β 2 S t nt 1 k + 1 k S t+1 β 0 + β 1 + β 2 n t + 1 k nt k k S t+1 S t β 0 + β 1 + β 2 n t k k k + 9 However, equaton 9 assumes knowledge of whch other elements are contaned n S t, and ths nformaton would not be known n advance. Thus the fnal sum over S t s approxmated wth half the correspondng sum over all neghbors of : t β 1 x t 1 + x t β 2 nt k 1 k S t k k The ntutve role of t s that t must smultaneously make S t appear lkely to have been generated from S t 1 and able to generate S t+1, thus conveyng temporal consstency nformaton both forwards and backwards n tme. 2.4 Iteratve Convergence The prevous secton provded defntons and nterpretatons for F pen S = s t S λt + t for the EBB scan statstc wth temporal consstency constrants. However, recall that the values of t for a gven tme step t depend on the detected subsets at t 1 and t + 1, whch creates a computatonal paradox. To solve ths, the Dynamc Subset Scan uses an teratve method that converges to a local optmum. To better approach the global optmum, multple restarts and smulated annealng whch gradually ncreases the strength of the t from 0 to ther full values are wrapped around steps 3-13 n the algorthm below. 3 Addtve GraphScan The prevous secton outlned how the expectaton-based bnomal EBB scorng functon may be penalzed wth temporal consstency constrants F pen S = s Sλ + whle remanng an addtve set functon over the data elements s S. Optmzng addtve functons wthout connectvty constrants s very straghtforward and conssts of ncludng all records wth postve contrbutons λ + > 0 and excludng the rest. Enforcng hard connectvty constrants on addtve functons.e. determnng the most postve connected subset s an nterestng and dffcult problem. For example, not all nodes makng postve contrbutons wll be ncluded n a hgh-scorng connected subset because they are lkely dsconnected n the underlyng graph structure. Also, a hgh-scorng connected subset may nclude a node wth a negatve contrbuton n order to connect two postve nodes. The GraphScan algorthm [13] exactly dentfes the hghest scorng connected subset for any scorng functon that satsfes the Lnear-Tme Subset Scannng LTSS property [11]. It s trvally 7

8 Algorthm 1 Iteratve convergence to local optmum for Dynamc Subset Scan wthout multple restarts or smulated annealng 1: for wndow duraton w from 1 to max wndow W do 2: Intalze each of the w spatal subsets ndependently.e., separately compute the hghest scorng subsets S t for each tme step t, assumng t = 0 for all s. 3: repeat 4: Randomly select a tme step t that s not flagged as Checked. Copy current spatal subset S t. 5: Compute t for each node s gven subsets S t 1 and S t+1, usng equaton 8 or 10. 6: Compute new optmal subset S t for tme step t usng t. Wthout connectvty constrants, smply nclude all postve contrbutons λ t + t ; wth connectvty constrants, call Addtve GraphScan. 7: f new subset S t does not mprove penalzed log-lkelhood rato of spatal-temporal subset S then 8: Revert to S t and mark tme step t as Checked. 9: end f 10: f new subset S t does mprove penalzed log-lkelhood rato of spatal-temporal subset S then 11: Replace S t wth S t and remove Checked flags from tme steps t 1, t + 1, and t. 12: end f 13: untl no further changes mprove penalzed log-lkelhood rato of spatal-temporal subset S,.e., all tme steps have been flagged as Checked. 14: end for 15: Return the hghest scorng spatal-temporal subset Sw. shown that addtve functons satsfy LTSS, and therefore GraphScan could be used to determne the hghest scorng most postve, n the case of an addtve scorng functon connected subset. However, GraphScan s desgned to optmze over more complex scorng functons; most mportantly, ts computaton tme s exponental n the graph sze and therefore t does not scale well n ths settng. Therefore, Addtve GraphScan s proposed as an effcent heurstc alternatve to GraphScan whch can be used to dentfy hgh-scorng most postve connected subsets n a gven graph structure wth real-valued weghts at each node. 3.1 Addtve GraphScan Algorthm Addtve GraphScan makes use of the followng notaton. wn s the real-valued weght of node n. A path p s any connected subgraph of nodes. wp s the sum of weghts for every node n the path. gp s the gan that would result from mergng path p nto a sngle node. It s the dfference between the weght of the resultng merged node and the hghest weghted node n the path. Identfyng and mergng paths wth postve gans s an ntegral part of Addtve GraphScan. gn, p s the gan that would result from mergng two paths together. The frst path, p, s a prevously dentfed path of nterest wth postve gan. The second path s the shortest path between node n and any pont along path p. gn, p s the dfference between the weght of the resultng merged paths and maxwn, wp. pwn s the pathweght of a node used when calculatng sngle source, shortest paths traversng through the node. Note the dfference between the weght of a node wn whch may be postve or negatve and s used n the gan calculatons above and the pathweght of a node pwn whch s non-negatve and used n shortest path calculatons. Pathweghts of postve 8

9 nodes are set to 0, reflectng no penalty or reward for traversng postve nodes whle dentfyng shortest paths. Pathweghts for negatve nodes wth no postve neghbors are wn. Pathweghts for negatve nodes wth postve neghbors have wn pwn = mn 0, wn +. degreen pos neghbors,n These postve weghts may be thought as unformly dffusng over ther negatve neghbors and then usng ths altered weght as the pathweght for negatve nodes wth postve neghbors. In the case where a large postve node overwhelms ts negatve neghbor, the negatve neghbor s pathweght s set to 0. Fnally, sn a, n b, n c determnes a fourth node, n s n the graph as a Stener pont for n a, n b, and n c. A Stener pont n ths settng s a node that forms the shortest nterconnect between the three provded nodes usng the pathweghts of the graph. sn a, n b, n c returns the shortest nterconnectng path formed between the three nodes gong through n s. Some basc pre-processng may be appled to the graph before runnng Addtve GraphScan. For example, any postve node wth a postve neghbor may be merged together nto a larger, sngle postve node addng ther weghts and repeated untl no further merges exst. Also, any negatve nodes wth degree of 1 or less may be recursvely removed because these are guaranteed to not be ncluded n a hgh scorng connected subset. Lastly, any negatve node wth at least two postve neghbors may be merged nto a sngle node f the resultng merged node has a hgher weght than any ndvdual postve neghbor. Addtve GraphScan can then be appled to the pre-processed graph. The Addtve GraphScan algorthm scales as OkN 2 = ON 2.5, domnated by steps 3 and 5. Algorthm 2 Addtve GraphScan 1: whle postve gan path merges exst do 2: Identfy top-k postve nodes where k = N. 3: Compute path weghts pwn for all nodes and create sngle-source shortest paths from each top-k node. 4: Compute gp for each shortest path p between top-k pars. Determne hghest gan path p and record endponts as n a and n b. 5: Compute gn, p for each remanng top-k node, n. Determne hghest gan node for p and record as n c. If no postve gan exsts between p and any n, then merge p and restart. 6: Form new path p as the unon of p and the path connectng p to n c. 7: Compute sn a, n b, n c. Compare wsn a, n b, n c and wp. Merge the one wth hgher weght. 8: end whle 9: The hghest weght merged node s returned as the most postve connected subset found by Addtve GraphScan. Note that ths node may need to be unpacked to determne the contents n the orgnal graph form. 9

10 Fgure 1: An example graph to demonstrate the Addtve GraphScan algorthm. The large bolded numbers are node dentfers and the small numbers wthn each node are the nodes correspondng weghts. The most postve subgraph conssts of nodes {0, 1, 6, 3, 4, 8, 9} and s correctly dentfed by Addtve GraphScan. 3.2 Addtve GraphScan Example Ths secton concludes by applyng Addtve GraphScan to a sample pre-processed graph found n Fgure 1. The most postve connected subgraph conssts of nodes {0, 1, 6, 3, 4, 8, 9} where node 6 s the Stener pont used to connect nodes 0, 4, and 9. Addtve GraphScan correctly dentfes ths subgraph even though node 6 s not on the shortest paths connectng nodes 0 and 4 or nodes 4 and 9. A key nsght nto the strong performance of Addtve GraphScan s delayng path merges whle searchng for a potental Stener pont. Begn at step 2: Nodes 0, 4, and 9 are dentfed as the top-k nodes. 3: Dkstra s algorthm s called on nodes 0, 4, and 9 provdng sngle-source shortest path nformaton from each of them. 4: The shortest path from node 0 to node 4, p = {0, 1, 2, 3, 4}, has hghest gan of gp = = +1. Because a postve gan path was found between nodes n a = 0 and n b = 4, Addtve GraphScan contnues searchng for a thrd node, n c. 5,6: Node n c = 9 s found wth p = {0, 1, 2, 3, 4, 7, 8, 9} and wp = 8. 7: Calculate a Stener pont for nodes 0, 4, and 9 and note that node 6 forms the shortest nterconnect between these three ponts. Ths nterconnect s formed by the nodes {0, 1, 6, 3, 4, 8, 9} and ws0, 4, 9 = = 9. Because ws0, 4, 9 > wp the Stener nterconnect s0, 4, 9 s condensed nto a sngle node wth weght 9. After ths merge, no more postve gan path merges exst and the loop exts. 9: The hghest scorng connected subset s then {0, 1, 6, 3, 4, 8, 9}. Notce that greedly mergng ether p or p would have resulted n a sub-optmal merge. 4 Results 4.1 Comparson of Addtve GraphScan vs. GraphScan Ths secton compares the fast heurstc, Addtve GraphScan, to the slower, but exact, GraphScan algorthm. Frst, a runtme analyss s presented comparng the two optmzaton algorthms. The much larger network 2 provded n the Battle of the Water Sensor Networks [1] s used to create connected subgraphs of varous szes from 50 to 500 nodes from the network. The graphs are processed wth three dfferent scans: Dynamc Subset Scan wth GraphScan, Dynamc Subset 10

11 Fgure 2: Runtme comparsons for the Dynamc Subset Scan wth GraphScan and Addtve GraphScan as the optmzaton algorthm. Independent Scan wth Addtve GraphScan s also shown. Scan wth Addtve GraphScan, and Independent wth Addtve GraphScan. The average runtme for each method s reported and are shown n Fgure 2. GraphScan begns to struggle wth graph szes of 250 nodes whle Addtve GraphScan quckly scans graphs of 500 nodes n approxmately 4.1 seconds. Independent wth Addtve GraphScan processed the entre 12,000+ node Network 2 n 221 seconds whle Dynamc wth Addtve GraphScan requred 1830 seconds approxmately a half hour. Ths dfference represents the addtonal calls to Addtve GraphScan requred by Dynamc Subset Scan to algn the ndvdual spatal subsets accordng to the temporal consstency constrants. The comparson of Addtve GraphScan and GraphScan s concluded by analyzng the scores of the spatal-temporal subsets dentfed by the scannng methods usng both Addtve Graph- Scan and GraphScan. The approxmaton rato results compare the hghest-scorng subsets found by Addtve GraphScan and GraphScan as a percentage averaged over 2000 smulatons. Table 1 provdes detaled nformaton for the approxmaton ratos. The ratos over 100% n the Dynamc cases reflect the nose n the teratve convergence process outlned above n Secton 2.4. To be clear, Addtve GraphScan s not dentfyng a hgher scorng subgraph than GraphScan for an ndvdual tme slce. However, the local optmum after the teratve convergence of Addtve GraphScan-based optmzatons at each step may have a hgher score than the local optmum reached wth GraphScan-based optmzatons at each tme step. The Statc and Independent methods do not use ths teratve convergence process to dentfy the hghest scorng spataltemporal regon and may reflect a more drect comparson between the performance of Addtve GraphScan and GraphScan. The rato does not fall below 98.4% ndcatng that Addtve Graphscan s provdng a huge speed ncrease wth mnmal loss of accuracy compared to scan statstcs usng GraphScan. 4.2 Detectng, Trackng, and Source-Tracng Plumes Ths secton evaluates the detecton, trackng, and source-tracng abltes of the Dynamc Subset Scan. The 129-node Network 1 from the Battle of the Water Sensor Networks [1] served as the test bed for these evaluatons. Two smulatons were performed. One wth sensors at FPR = 0.1 and TPR = 0.9 and a second wth weaker sensors at FPR = 0.2 and TPR = 0.8. All results below 11

12 Table 1: Approxmaton ratos comparng scores for Addtve GraphScan and GraphScan for multple methods and FPR and TPR. method Background0.1 Inects0.9 Background0.2 Inects0.8 Statc % % 99.15% 99.83% Independent % 99.99% 98.48% 99.65% Dynamc % % 99.70% % Table 2: Summary of learned parameter values method FPR TPR β 0 β 1 β 2 Dynamc Dynamc Dynamc Alt Dynamc Alt are averaged over 200 contamnant plumes smulated for 12 hours each. A separate 100-plume tranng set was used for cross-valdaton for the scan statstcs that requred learnng parameters. Comparsons are made for 4 dfferent spatal-temporal scan statstcs: Statc scan does not allow the detected spatal regon to change over tme. Independent scan allows the detected spatal regon to change over tme but does not share temporal nformaton between tme steps. Dynamc scan allows the detected spatal regon to change over tme and uses temporal consstency constrants to algn the ndvdual tme steps. Dynamc Alt. scan s smlar to Dynamc scan but does not use any nformaton from neghbors when enforcng temporal consstency constrants,.e., forces β 2 = 0 when learnng the model parameters. For the temporal component of the scans, max wndow sze W = 12 s used. Ths allows Statc, Dynamc, and Dynamc Alt. to detect a spatal-temporal subset between 1 and 12 hours n duraton. However, for the Independent scan, the hghest scorng spatal-temporal regon wll always be maxmum duraton. Ths consequence of the Independent scan s dscussed further below. The β 0... β 2 parameters for the Dynamc scan, and the β 0 and β 1 parameters for Dynamc Alt. were set usng a grd search on the separate 100-plume tranng set. The parameter values that maxmzed spatal-temporal overlap n the tranng data are shown n Table 2. Note the large changes n β 1 when movng from the easer to harder scenaros, whle β 0 and β 2 reman relatvely constant. Fgure 3 reports the average tme requred by each method to detect a contamnant plume for varous false postve rates. These results were calculated by processng 2160 background hours approxmately 3 months of data wth no contamnants. These were compared wth scores produced by the scan statstcs durng the 200 smulated plumes. The 0 false postve alarms 12

13 Fgure 3: Detecton results for FPR = 0.1 and TPR = 0.9 sensors on the left and FPR = 0.2 and TPR = 0.8 sensors on the rght. The detecton results are reported through Actvty Montorng Operatng Characterstc AMOC curves. These show the average tme requred for each method to detect a contamnant n the system, assumng a fxed number of allowable false postves per month. nterpretaton s that t took 4.3 hours on average for the scores produced by Dynamc Scan to exceed the largest score found by Dynamc Scan n the 2160 background hours. As the threshold for detecton s lowered, the number of false postve alarms ncreases but the tme to detect decreases, as shown by the Actvty Montorng Operatng Characterstc AMOC curves [4] n Fgure 3. The Statc, Dynamc, and Dynamc Alt. methods acheve smlar power for event detecton n the easer scenaro FPR = 0.1, TPR = 0.9. However, Dynamc acheves the overall best performance 6.62 hours to detect at 0 false postves when detectng a weaker sgnal FPR = 0.2, TPR = 0.8. Note the nfluence that a node s neghbors have on dstngushng performance between Dynamc and Dynamc Alt. n the 0.2/0.8 scenaro. The easer scenaro dd not requre addtonal nformaton from neghbors n order to obtan smlar performance, but ths nformaton s mportant when workng wth weaker sensors. The Independent method s poor performance s due to the relatvely hgh subset scores found by Independent when no contamnant s present. Its unconstraned flexblty allows t to overft to nose n the background, makng detecton of a true contamnaton event more dffcult. Fgure 5 reports the methods trackng ablty over the duraton of a spreadng contamnant plume 12 hours. A scan statstc s trackng ablty s measured through spatal-temporal overlap. Spatal-temporal overlap s a combnaton of precson and recall appled to spatal-temporal subsets. A measure of 1.0 corresponds to perfect agreement between the affected and detected spatal-temporal regons, whle 0.0 means the affected and detected regons are dsont. For two spatal-temporal subsets, Affected and Detected, the overlap s defned as: Affected Detected Affected Detected. See Fgure 4 for detals. The relatve performance of the Statc and Dynamc methods n the easer scenaro demonstrates Statc s lack of trackng ablty as the plume grows over tme. Statc s trackng performance quckly levels off whle Dynamc contnues to acheve hgher spatal-temporal overlap over the course of the contamnaton event. Ths ncrease n performance s due to the constraned flexblty allowed to the Dynamc Subset Scan. The dfference n trackng performance between Statc and Dynamc methods s not as large n the harder scenaro, but note the mportance of ncorporatng nformaton from a node s neghbors. The poor performance of the Independent 13

14 Fgure 4: Ths fgure demonstrates the calculaton of spatal-temporal overlap for plume trackng. A plume spreads through a smple 5-node lne graph over the course of four tme steps. Affected nodes turn from whte to red as the contamnant spreads. The Statc scan method s constraned to keep the exact same detected spatal regon throughout the event duraton. Hence, t may fal to capture the plume at later tme steps. Dynamc Scan allows the detected spatal regon to change at each tme step, trackng the plume as t spreads. Due to connectvty constrants, both methods must return a connected subgraph as the detected spatal regon at each tme step. Spatal-temporal overlap s penalzed for both false postves and false negatves. A measure of 1.0 corresponds to perfect agreement between the affected and detected spatal-temporal regons, whle 0.0 means that the affected and detected regons are dsont. Fgure 5: Trackng results for FPR = 0.1 and TPR = 0.9 sensors on the left and FPR = 0.2 and TPR = 0.8 sensors on the rght. Trackng ablty s measured by reportng the spatal-temporal overlap of the detected and affected subsets over the course of 12 hours. 14

15 Fgure 6: Source-tracng results for FPR = 0.1 and TPR = 0.9 sensors on the left and FPR = 0.2 and TPR = 0.8 sensors on the rght. Source-tracng ablty s measured by reportng the spatal overlap between the earlest detected spatal regon and the orgnal affected nodes. scan, partcularly n the early stages of the contamnaton event, s due to ts tendency to report spatal-temporal regons of maxmum duraton whch are not a good match to the true affected regon. Fgure 6 reports the methods ablty to dentfy where the contamnant orgnated over the duraton of the plume 12 hours. Ths s measured through purely spatal overlap between the earlest tme step n the detected regon and the source nodes of the plume. Note that t s possble for a quckly spreadng plume to affect multple nodes wthn the frst hour. In such cases, all of these nodes are treated as source nodes. The source-tracng results clearly demonstrate the advantage of sharng nformaton between tme steps durng the optmzaton process. Statc s ablty to dentfy the source nodes actually decreases over the course of the contamnaton event as more nformaton s gathered. Explorng ths result further, note that Statc has very hgh spatal recall but very low precson for dentfyng the source nodes on hour 12. Ths suggests that Statc tends to return very large subsets at the later stages of the plume. The large regons returned by the Statc method harm ts ablty to accurately dentfy the source of the contamnant. The key to the Dynamc Subset Scan s success for source-tracng s the backwards flow of temporal consstency nformaton allowed n our model. Dynamc s able to change the detected subset for prevous tme steps based on new, more current data. Ths gves t superor sourcetracng abltes n both the easer 0.1/0.9 and harder 0.2/0.8 scenaros. The mportance of ncludng neghbor nformaton s evdent n the harder scenaro: whle the Dynamc method acheves smlar performance to Statc durng the early stages of the contamnaton event and much better performance n the later stages, the performance of Dynamc Alt. whch does not use neghbor nformaton does not surpass Statc untl the nnth hour. Fnally, note the substantal ncrease n performance of the Independent method at hour 12, though ts overall performance s stll low. Ths s an artfact of Independent preferrng to return 12-hour regons. 5 Conclusons Ths work ntroduced the Dynamc Subset Scan for detectng, trackng, and source-tracng dynamc patterns that change the affected subset over tme. Ths novel extenson of the well-known 15

16 spatal and subset scan statstcs s composed of two man contrbutons. Frst s the ncorporaton of temporal consstency constrants that may be enforced on temporally adacent, spatal subsets. These constrants are a frutful compromse between tradtonal spatal-temporal scan statstcs that do not allow the detected regon to change over tme Statc and the other extreme where temporal nformaton s gnored Independent. The key nsght to enforcng temporal consstency constrants s recognzng that the expectaton-based bnomal scorng functon may be wrtten as an addtve functon over the data records. Ths allows for addtonal terms constrants to be ncluded n the penalzed log lkelhood rato whle remanng effcent to optmze. Crtcally, these temporal consstency constrants were derved to allow temporal nformaton to be shared both forward and backward n tme. The second novel contrbuton s the Addtve GraphScan algorthm, whch allows the Dynamc Subset Scan to enforce both soft temporal consstency constrants and hard connectvty constrants whle scalng to large, real world networks. Addtve GraphScan s a fast, heurstc alternatve to GraphScan. However, the results demonstrate an approxmaton rato of over 99%, suggestng a very small sacrfce for dramatc gans n speed and scalablty. The Dynamc Subset Scan was evaluated on data provded through the Battle of the Water Sensor Networks [1]. Dynamc scan succeeded n detectng contamnaton events sooner and trackng these events more accurately compared to other competng methods. The gans were due to Dynamc Scan s constraned flexblty: competng methods ether faled to capture the dynamcs of the spreadng plume Statc or were susceptble to over-fttng from lack of constrants Independent. In scenaros wth a weaker sgnal to be detected, ncorporatng nformaton from a node s neghbors n the Dynamc Scan proved worthwhle, leadng to substantal gans n performance on the detecton, trackng, and source-tracng tasks. In summary, relaxng constrants on spatal-temporal regon shape must be done carefully. Strct temporal constrants work well when the affected subset of the data does not change over tme. However, removng them completely n order to track dynamc patterns performs poorly as shown by the Independent method results. Dynamc Subset Scan wth temporal consstency and connectvty constrants provdes a scalable soluton for future work n dynamc pattern detecton n graph-based or sensor network data. Acknowledgment A specal thanks to Professor Danel Nell and the Event and Pattern Detecton Laboratory for ther too many? deas and multple edts of ths work. Also, thanks to Yatng Zhang who helped me thnk through Addtve GraphScan at the mplementaton level. Thanks to R. Rav for provdng nsghts on the use of Stener ponts. Ths work was partally supported by NSF grants IIS , IIS , and IIS References [1] Av Ostfeld et al. The battle of water sensor networks: A desgn challenge for engneers and algorthms. Journal of Water Resources Plannng and Management, 1346: , [2] J. Berry, R. D. Carr, W. Hart, V. J. Leung, C. A. Phllps, and J. P. Watson. Desgnng contamnaton warnng systems for muncpal water networks usng mperfect sensors. Journal of Water Resources Plannng and Management, 1354: ,

17 [3] J. Berry, L. Flescher, W. Hart, and C. Phllps. Sensor placement n muncpal water networks. J. Water, 131: , [4] T. Fawcett and F. Provost. Actvty montorng: notcng nterestng changes n behavor. In Proc. 5th Intl. Conf. on Knowledge Dscovery and Data Mnng, pages 53 62, [5] M. Gomez-Rodrguez, J. Leskovec, and A. Krause. Inferrng networks of dffuson and nfluence. In Proc. 16th ACM SIGKDD Conf. on Knowledge Dscovery and Data Mnng, pages , [6] M. W. Koch and S. A. Mckenna. Dstrbuted sensor fuson n water qualty event detecton. Journal of Water Resources Plannng and Management, 137:10 19, [7] A. Krause, J. Leskovec, C. Guestrn, J. VanBresen, and C. Faloutsos. Effcent sensor placement optmzaton for securng large water dstrbuton networks. Journal of Water Resources Plannng and Management, 1346: , November [8] M. Kulldorff. A spatal scan statstc. Communcatons n Statstcs: Theory and Methods, 266: , [9] S. Myers and J. Leskovec. On the convexty of latent socal network nference. In Advances n Neural Informaton Processng Systems 23, pages [10] D. B. Nell. Expectaton-based scan statstcs for montorng spatal tme seres data. Internatonal Journal of Forecastng, 25: , [11] D. B. Nell. Fast subset scan for spatal pattern detecton. Journal of the Royal Statstcal Socety Seres B: Statstcal Methodology, 742: , [12] D. B. Nell, A. W. Moore, M. R. Sabhnan, and K. Danel. Detecton of emergng space-tme clusters. In Proc. 11th ACM SIGKDD Conf. on Knowledge Dscovery and Data Mnng, [13] S. Speakman and D. B. Nell. Fast graph scan for scalable detecton of arbtrary connected clusters. In Proc. Internatonal Socety for Dsease Survellance Annual Conference, [14] R. Vswanathan and P. K. Varshney. Dstrbuted detecton wth multple sensors: Part I Fundamentals. Proceedngs of the IEEE, pages 54 63,