Incremental Mining of Partial Periodic Patterns in Time-series Databases

Size: px

Start display at page:

Download "Incremental Mining of Partial Periodic Patterns in Time-series Databases"

Julie Page
5 years ago
Views:

1 CERIAS Teh Report Inremental Mining of Partial Periodi Patterns in Time-series Dataases Mohamed G. Elfeky Center for Eduation and Researh in Information Assurane and Seurity Purdue University, West Lafayette, IN 47907

2 Inremental Mining of Partial Periodi Patterns in Time-Series Dataases Mohamed G. Elfeky Feruary 3, Introdution A Time-Series Dataase is a dataase that ontains data for eah point in time; e.g., weather data that ontains several measures (e.g., the temperature) at different times per day. Some other examples are the stok pries and the power onsumption. Mining timeseries dataases involves two general kinds of periodi patterns: full periodi patterns and partial periodi patterns. In full periodi patterns, every point in time ontriutes to the yli ehavior of the time series for eah period [1]. For example, desriing the weekly stok pries pattern onsidering all the days of the week. The other kind is partial periodi patterns, whih speify the ehavior of the time series at some ut not all the points in time [1]. For example, disovering that a speifi stok pries are high every Saturday and low every Tuesday ut do not have suh regularity on other days. An effiient algorithm for mining partial periodi patterns is introdued in [1]. One of the important prolems of the data mining prolem is how to maintain the disovered rules or patterns over the time in the sense that the data is updated regularly. Espeially in time-series dataases, new data is added ontinuously over the time. Inremental mining onerns this prolem. It is how to mine a previously mined dataase after the addition of some data without running the mining algorithm again on the new dataase. A new algorithm is proposed for inrementally mine partial periodi patterns in time-series dataases ased on the algorithm disussed in [1]. Another prolem that is arisen here is that what if there are two time-series dataases and now it is wanted to omine them into one, how to disover the partial periodi patterns of the omined dataase, given that the two dataases were previously mined, without running the mining algorithm again on the omined dataase. The proposed algorithm is extended to solve this prolem. The rest of this report is organized as follows. In Setion 2, the algorithm of mining partial periodi patterns disussed in [1] is riefly introdued. Setion 3 disusses in detail the proposed algorithm for inremental mining of partial periodi patterns. In setion 4, some statistial analysis of the proposed algorithm is shown. Finally, Setion 5 presents a proposed solution to the latter prolem disussed aove. 2 Mining Partial Periodi Patterns 2.1 Prolem Definition Assume that a sequene of n time-stamped datasets have een olleted in a dataase. For eah time instant i, let D i e a feature derived from the dataset. Thus, the time series 1

3 of features is represented as, S = D 1, D 2,..., D n. A pattern is: s = s 1... s p over the set of features L and the letter *, suh that p is the period of the pattern. L-length of a pattern s is the numer of s i that is not *. A pattern with L-length j is also alled a j-pattern. A supattern of a pattern s is a pattern s =s 1... s p suh that for eah position i: s i = s i or s i = *. For example, if the period is 5, the pattern: a * de is alled 4-pattern. Two of his supatterns are: a * * e and ** d *. Eah segment of the form D ip+1... D ip+p is alled a period segment. A period segment mathes a pattern s if it is a supattern of s. The frequeny ount of a pattern in a time series is the numer of period segments of this time series that mathes that pattern. The onfidene of a pattern is defined as the division of its frequeny ount y the maximum numer of period segments in the time series. A pattern is alled frequent if its onfidene not less than a minimum threshold. For example, in the series aaeaed, the pattern: a *, whose period is 3, has frequeny ount 2, and onfidene 2/3 where 3 is the maximum numer of period segments of length Max-Supattern Hit Set Method A max-pattern C max is the maximal pattern that an e generated from F 1 (the set of frequent 1-patterns). For example, if F 1 = {a ****, * ***, ** **, **** d}, then C max = a * d. A supattern of C max is hit in a period segment S i if it is the maximal supattern of C max in S i. For example, if C max = a * d and S i = add, then its hit supattern is: a ** d. Algorithm: 1. San the dataase to find F 1, the frequent patterns of length 1, and form the andidate max-pattern C max. 2. San the dataase one again, and for eah period segment, onsider its max supattern and update the max-supattern tree (either add the pattern setting its ount to 1 if it is not exist, or otherwise inrease its ount y 1). 3. Traverse the tree to determine the atual ount of eah pattern, and determine the frequent ones. ad * 10 a d * d * 2 a * d * 50 a * d * 40 a ** 32 d a d a ** d * 18 * * d * 8 a ** d * 5 a * ** 19 a *** 2 Figure 1. An example to a max-supattern tree 2

4 Figure 1 shows an example of a max-supattern tree used to store the set of maxsupatterns hit in the time series. It shows that the root node is the C max. A hild node is a supattern of the parent node with one non * -letter missing. The link is laeled y this letter. The 1-patterns are not stored in the tree sine they already exist in F 1 tale. Eah node has a ount field that registers its numer of hits. A dotted link is a non-existing (virtual) link that represents a andidate parent of the hild node. The frequeny ount of eah pattern represented y a node is the sum of its ount and those of all its reahale parents (physial or virtual link). For example, the frequeny ount of ** d * is 80, and the frequeny ount of a ** d * is Inremental Mining of Partial Periodi Patterns 3.1 Prolem Definition The prolem of inremental mining takes as an input a previously mined dataase after some additions of new data, and suffiient information aout the previously mined patterns olleted efore from running the mining algorithm over the old dataase. Here, it is suffiient to store the max-supattern tree and all the 1-patterns along with their respetive ounts. 3.2 Algorithm 1. San the new data for the 1-patterns and add them to the whole list of 1-patterns, and determine the new max-pattern C max. 2. If C max is not hanged, there will e no hange on the tree. 3. If C max is hanged, there are two ases: deleted letters and inserted letters (either or oth). a. For the deleted letters, delete from the new C max the inserted letters, and onsider the resulted pattern, that is surely a supattern of the old C max, as the root of the new tree. Insert into the new tree the hit of eah pattern in the old tree with the same ount. Now, the old C max is the new C max.. For the inserted letters, the old C max is surely a supattern of the new one. Hene, onsider the new C max as the root of the tree making the old root a hild. San the old dataase, and for eah period segment ontaining at least one of the inserted letters, insert its hit in the tree, and derement the node that ontains the hit without the inserted letters. 4. San the new data and update the max-supattern tree as step 2 of the original mining algorithm. 5. Traverse the tree to determine the atual ount of eah pattern, and determine the frequent ones. The following example, followed from Figure 1, illustrates the idea ehind this algorithm. Assume that the new C max is a * e, whih means one deleted letter d and one inserted letter e. Following step 3a of the algorithm, the root of the new tree is: a **, resulting in Figure 2. Notie that: (i) the frequeny ount of eah pattern is the same as in the old tree, and this why the hits of the patterns of the old tree are inserted in the new one, (ii) the virtual links eome physial, (iii) a new node appears * ** resulted from inserting the hit of the node * d *. 3

5 a ** 42 a * ** 2 a * ** a *** Figure 2. The max-supattern tree of Figure 1. after deleting the letter d. Now, following step 3 of the algorithm, the root is a * e, resulting in Figure 3. Notie that: (i) there are new nodes other than the root, like a ** e, and this is why the old dataase must e resanned to disover the hits given the new max pattern, (ii) the frequeny ount of eah pattern is the same as in the old tree, and this why some nodes must e deremented. e a * e 4 a ** 38 a ** e 22 a e * ** 2 a * ** a *** a *** e 12 Figure 3. The max-supattern tree of Figure 2. after inserting the letter e. 4 Analysis Clearly, inremental mining will speed up the proess of mining a previously mined dataase after the addition of some new data, ut it relies on how muh often the algorithm is applied; i.e., with eah new segment of data or after a signifiant numer of additions. The following statistis may e inomplete ut give an idea aout the speed up of the proposed algorithm over applying the original mining algorithm on the new dataase again. Also, the proposed algorithm implies some overhead of the original mining algorithm in the sense that the reated tree must e stored on the physial storage, whih requires additional time and spae. 4

6 It an e inferred easily from the algorithm that it has a worst ase of one san over the old dataase if there are inserted letters in C max (Step 3). This san an e eliminated if a list of the segments that ontain any 1-pattern is stored along with this 1-pattern. This list is alled inverted list. Now, step 3 is modified suh that the period segments that must e reinserted in the tree are now loated in the inverted list without a need to resan the old dataase searhing for those period segments. 5 Merging Mined Dataases 5.1 Prolem Definition The prolem of merging two previously mined dataases and disover the partial periodi patterns from the new one an e onsidered a generalization to the prolem of inremental mining. Now the input is two trees and two 1-pattern lists, one for eah dataase. The worst ase in the proposed algorithm is to san eah dataase one, if the last modifiation is not onsidered, instead of sanning the new dataase twie if the original mining algorithm is onsidered. 5.2 Algorithm 1. Determine the C max that will e the root of the new tree. This an e done y either way of the following: a. Merge the two 1-pattern lists and determine the new C max as Step 1 of the original algorithm.. Interset the two C max s of the two trees, and for eah position that will have a *, selet either the letter of the first C max or the seond one ased on how frequent is this letter in the merged list. 2. Apply Steps 2 and 3 of the proposed algorithm for eah tree onsidering the new C max determined in the previous step. 3. Now, the two trees have the same root and then an e omined into one y seleting the one that has smaller numer of nodes and insert it into the other. 4. Traverse the tree to determine the atual ount of eah pattern, and determine the frequent ones. Referenes [1] Jiawei Han, Guozhu Dong, and Yiwen Yin. Effiient Mining of Partial Periodi Patterns in Time Series Dataases. In Proeedings of 1999 Int. Conf. on Data Engineering (ICDE 99), Sydney, Australia, Marh [2] Shiy Thomas, Sreenath Bodagala, Khaled Alsati, and Sanjay Ranka. An Effiient Algorithm for the Inremental Updation of Assoiation Rules in Large Dataases. In Proeedings of the 3 rd Int. Conf. on Knowledge Disovery and Data Mining, New Port Beah, California, August

7 Implementation Both the original mining algorithm and the proposed inremental one are implemented using Java programming language. To use the original mining algorithm: use java Mine arg1 arg2 arg3, suh that arg1 is the numer of the data file, arg2 is the period, and arg3 is the onfidene threshold. For examples, java Mine means to mine the first data file with period 7 (7 days per week), and the onfidene threshold is 0.6. For eah data file there are two versions, one with the original data, and the other after the addition of new data. Now, to use the inremental algorithm: use java InMine arg1, suh that arg1 is the numer of the data file. For example, java InMine 1 means to apply the inremental mining algorithm over the new data file. To ensure that the results of the inremental mining algorithm are orret, it is needed to apply the original mining algorithm over the new data file and this an e done using: java Mine 1i 7 60, suh that 1i is the numer of the new data file. 6

Path Sharing and Predicate Evaluation for High-Performance XML Filtering*

Path Sharing and Predicate Evaluation for High-Performance XML Filtering* Path Sharing and Prediate Evaluation for High-Performane XML Filtering Yanlei Diao, Mihael J. Franklin, Hao Zhang, Peter Fisher EECS, University of California, Berkeley {diaoyl, franklin, nhz, fisherp}@s.erkeley.edu