Efficient Mining of Partial Periodic Patterns in Time Series Database

Transcription

1 Wright tate University CORE cholar Kno.e.sis Pblications The Ohio Center of Excellence in Knowledge- Enabled Compting (Kno.e.sis) Efficient Mining of Partial Periodic Patterns in Time eries Database Jiawei an Gozh Dong Wright tate University - Main Camps, gozh.dong@wright.ed Yiwen Yin Follow this and additional works at: Part of the Bioinformatics Commons, Commnication Technology and New Media Commons, Databases and Information ystems Commons, O and Networks Commons, and the cience and Technology tdies Commons Repository Citation an, J., Dong, G., & Yin, Y. (1999). Efficient Mining of Partial Periodic Patterns in Time eries Database. 15th International Conference on Data Engineering: Proceedings, This Conference Proceeding is broght to yo for free and open access by the The Ohio Center of Excellence in Knowledge-Enabled Compting (Kno.e.sis) at CORE cholar. It has been accepted for inclsion in Kno.e.sis Pblications by an athorized administrator of CORE cholar. For more information, please contact corescholar@

2 1 Efficient Mining of Partial Periodic Patterns in Time eries Database Jiawei an chool of Compting cience imon Fraser University Gozh Dong Department of Compter cience and Engineering Wright tate University Yiwen Yin chool of Compting cience imon Fraser University Abstract Partial periodicity search, i.e., search for partial periodic patterns in time-series databases, is an interesting data mining problem. Previos stdies on periodicity search mainly consider finding fll periodic patterns, where every point in time contribtes (precisely or approximately) to the periodicity. owever, partial periodicity is very common in practice since it is more likely that only some of the time episodes may exhibit periodic patterns. We present several algorithms for efficient mining of partial periodic patterns, by exploring some interesting properties related to partial periodicity, sch as the Apriori property and the max-sbpattern hit set property, and by shared mining of mltiple periods. The max-sbpattern hit set property is a vital new property which allows s to derive the conts of all freqent patterns from a relatively small sbset of patterns existing in the time series. We show that mining partial periodicity needs only two scans over the time series database, even for mining mltiple periods. The performance stdy shows or proposed methods are very efficient in mining long periodic patterns. Keywords. Periodicity search, partial periodicity, timeseries analysis, data mining algorithms. 1. Introdction Finding periodic patterns in time series databases is an important data mining task with many applications. Many methods have been developed for searching periodicity patterns in large data sets [8]. owever, most previos methods on periodicity search are on mining fll periodic patterns, Research was spported in part by research grants from the Natral ciences and Engineering Research Concil of Canada and the Networks of Centres of Excellence Program of Canada Part of this work was done while visiting imon Fraser University dring his sabbatical from University of Melborne, Astralia. where every point in time contribtes (precisely or approximately) to the cyclic behavior of the time series. For example, all the days in the year approximately contribte to the season cycle of the year. A sefl related type of periodic patterns, called partial periodic patterns, which specify the behavior of the time series at some bt not all points in time, have not received enogh attention. An example partial periodic pattern may state that Jim reads the Vancover n newspaper from :00 to :30 every weekday morning bt his activities at other times do not have mch reglarity. Ths, partial periodicity is a looser kind of periodicity than fll periodicity, and it exists biqitosly in the real world. The prpose of the crrent paper is to fill the gap by considering the efficient mining of partial periodic patterns. Most methods for finding fll periodic patterns are either inapplicable to or prohibitively expensive for the mining of partial periodic patterns, becase of the mixtre of periodic events and non-periodic events in the same period. For example, FFT (Fast Forier Transformation) cannot be applied to mining partial periodicity becase it treats the time-series as an inseparable flow of vales. ome periodicity detection methods can detect some partial periodic patterns, bt only if the period, and the length and timing of the segment in the partial patterns with specific behavior are explicitly specified. For the newspaper reading example, we need to explicitly specify details sch as find the reglar activities of Jim dring the half-hor after :00 for the period of hors. A naive adaptation of sch methods to or partial periodic pattern mining problem wold be prohibitively expensive, reqiring their application to a hge nmber of possible combinations of the three parameters of length, timing, and period. Besides fll periodicity search, there are many recent stdies on time series data mining: Most concentrate on symbolic patterns, althogh some consider nmerical crve patterns in time series. Agrawal and rikant [3] developed an Apriori-like techniqe [2] for mining seqential patterns. Mannila et al. [10] consider freqent episodes in seqences, where episodes are essentially acyclic graphs

3 2 d $ - of events whose edges specify the temporal before-andafter relationalship bt withot timing-interval restrictions. Inter-transaction association rles proposed by L et al. [9] are implication rles whose two sides are totally-ordered episodes with timing-interval restrictions (on the events in the episodes and on the two sides). Bettini et al. [5] consider a generalization of inter-transaction association rles: these are essentially rles whose left-hand and right-hand sides are episodes with time-interval restrictions. owever, nlike ors, periodicity is not considered in these stdies. imilar to or problem, the mining of cyclic association rles by Özden, et al. [12] also considers the mining of some patterns of a range of possible periods. Observe that cyclic association rles are partial periodic patterns with perfect periodicity in the sense that each pattern reoccrs in every cycle, with confidence. The perfectness in periodicity leads to a key idea sed in designing efficient cyclic association rle mining algorithms: As soon as it is known that an association rle does not hold at a particlar instant of time, we can infer that cannot have periods which inclde this time instant. For example, if the maximm pe- does not riod of interest is and it is discovered that hold in the first time instants, then cannot have any periods. This idea leads to the sefl cycle-elimination strategy explored in that paper. ince real life patterns are sally imperfect, or goal is not to mine perfect periodicity and ths cycle-elimination based optimization will not be considered here. An Apriori-like algorithm has been proposed for mining imperfect partial periodic patterns with a given (single) period in a recent stdy by two of the crrent athors []. It is an interesting algorithm for mining imperfect partial periodicity. owever, with a detailed examination of the data characteristics of partial periodicity, we fond that Apriori prning in mining partial periodicity may not be as effective as in mining association rles. Or stdy has revealed the following new characteristics of partial periodic patterns in time series: The Apriori-like property among partial periodic patterns still holds for any fixed period, bt it does not hold for patterns between different periods. Frthermore, there is a strong correlation among freqencies of partial patterns. The main contribtions of this paper are as follows. We consider the efficient mining of partial periodic patterns, for a single period as well as for a set of periods. We propose several mining algorithms, by exploring some interesting properties related to partial periodicity sch as the Apri- It is important to point ot that [12] concentrates on the elimination of candidate itemsets for the association rle mining algorithm, althogh the cycle-elimination strategy does lead to a small redction on the nmber of patterns when we process the time series from left to right. Note that a modified strategy, where we stop considering certain patterns as soon as the length of the time series to be processed is not enogh to make the confidence higher than the threshold, can be sed. ori property and the max-sbpattern hit set property, and by shared mining of mltiple periods. The max-sbpattern hit set property is a vital new property which allows to derive the conts of all freqent patterns from a relatively small sbset of patterns mined from the time series. We show that mining partial periodicity needs only two scans over the time series database, even for mining mltiple periods. The performance stdy shows or proposed methods are very efficient. The proposed methods are also robst that can be applied in a variety of cases inclding mining mltiplelevel partial periodicity and mining partial periodicity with pertrbation and evoltion. The remaining of the paper is organized as follows. In ection 2, concepts related to partial periodicity are introdced. In ection 3, methods for mining partial periodicity in regard to both single and mltiple periods are stdied. In ection 4, the implementation of a novel data strctre, namely the max-sbpattern tree, for facilitating the conting of the hit maximal patterns, and the derivation of the set of freqent patterns from the hit maximal patterns, are presented. In ection 5, a comparison of the performance of the proposed algorithms is reported. We conclde or stdy in ection 6. 2 Problem Definition Assme that a seqence of timestamped datasets have been collected in a database. For each time instant, let be a set of featres derived from the dataset collected at the instant. Ths, the time series of featres is repesented as,! # %$$%$& ' Let ( be the nderlying set of featres. We will also se the don t care character ), which can match any single set of featres. We define a pattern * * *&- as a non-,++%+ empty seqence. over / ;:<4#)=. We will se > *8> to denote the length of *, and will say that > *?> is the period of the pattern *. Let the ( -length of * * ++%+ * - be the nmber of *@ which contains letters from (. A pattern with ( -length is also called an -pattern. Moreover, a sbpattern of a pattern * * ++%+ * - is a pattern *A *A ++%+ *@A sch that * and *@A have the same length, and *@ACB *@ for every position where *AED ). For example, the pattern F )G4 F I J8K is of length L and it is of ( -length (i.e., it is a 4-pattern); and F )M4 F I N)O) and )O) J8K are two of the P sbpatterns of F )Q4 F #J8K. The freqency cont and confidence of a pattern * in a time series R' are defined as $%$$ UT K%V WXK% Y Z WU\[/0*%9 >]4 N> ^_,`ba F cj If e&f is a singleton we will omit the brackets, e.g., we write ghi as h.

4 . 3 and the string s is tre in kj lj m +%++ kj lj mnj lj =>, UT KV WUK% Z /0*%9 Y Z WX\[/o*9 a where a is the maximm nmber of periods of length > *8> contained in the time series (i.e., a is the positive integer sch that ap> *?>q^rs`t/0atv 9%> *?> ). Each segment of the form kj lj m ++%+ kj lj mnj lj, where w^xy`xa, is called a period segment. We say a pattern * * ++%+ * - is tre in the period segment or the period segment matches *, if, for each position, either * is ) or all the letters in * occr in the {z} set of featres in the segment. Ths, if *@A is a sbpattern of *, then the set of seqences that can match * is a sbset of seqences that can match *@A. Example 2.1 For example, F ) is a pattern of period ; its freqency cont in the featre series F 4# I F K F KJ is 2; and its confidence is, where 3 is the maximm nmber of periods of length 3. The freqency cont of F 4# ) in F 4 J\ K F 4# I FF is also.. imilar to mining association rles [2], we say that a pattern is a freqent partial periodic pattern in a time series if its confidence is larger than or eqal to a threshold, ack. The mining of freqent partial periodic patterns Z in a time series is to discover, possibly with some restrictions, all the freqent patterns of the series for one period or a range of specified periods. More specifically, the inpt to mining incldes: A time series. A specified period; or a range of periods specified by two integers and Z U{?. An integer a indicating that the ratio of the lengths of and the patterns mst be at least a. This will ensre that the patterns mined wold be of vale to the application at hand. Remark: ometimes the derivation of the featre series from the original data series is qite involved, and the interaction of the periodic patterns with the derivation of featres may lead to improved performance. ence it is worthwhile to combine the mining of the featres from the datasets with the mining of the patterns, as is the case for the mining of cyclic association rles [12]. For or work on the mining of freqent partial periodic patterns thogh, this interaction is not sefl for achieving comptational advantage and ths we will assme that we are dealing with the featre time series in or stdy. 3 Methods for mining partial periodicity in time series In this section, we explore methods for mining partial periodicity in a time series, proceeding from mining partial periodicity for a single given period to mining partial periodicity for a specified range of periods (i.e., mltiple periods). 3.1 Mining partial periodicity for single period ingle-period apriori method A poplar key idea sed in the efficient mining of association rles is the Apriori property discovered in [2]: If one sbset of an itemset is not freqent, then the itemset itself cannot be freqent. This allows s to se freqent itemsets of size as filters for candidate itemsets of size \5. Interestingly, for each period, the property spporting the Apriori trick still holds: Property 3.1 [Apriori on periodicity] Each sbpattern of a freqent pattern of period is itself a freqent pattern of period. The proof is based on the fact that patterns are more restrictive than their sbpatterns. ppose *@A is a sbpattern of a freqent pattern *. Then *A is obtained from * by changing some set of letters to a sbset or ). ence * is more restrictive than *@A and ths the freqency cont of *@A is greater than or eqal to that of *. Ths *A is freqent as well. An algorithm for mining partial periodic patterns for a given fixed period based on this Apriori trick was presented in []. We inclde a simplied version here for the sake of completeness. Algorithm 3.1 [ingle-period Apriori] Find all partial periodic patterns for a given period satisfying a given confidence threshold min conf in time-series, based on the Apriori property Find, the set of freqent 1-patterns of period, by accmlating the freqency cont for each 1-pattern in each whole period segment and selecting among them whose freqency cont is no less than â } is the maximm nmber of Š a, where a 2. Find all freqent -patterns of period, for from 2 p to, based on the idea of Apriori, and terminate immediately when the candidate freqent -pattern set is empty. Nmber of scans over the time series. tep 1 of the algorithm needs to scan the time series once. tep 2 needs

5 > - $ $ > ^ 4 > > > J to scan p to 2Œ times in the worst case. Ths the total nmber of scans is no more than the period. pace needed. (1) At tep 1, sppose there exist a total of distinct featres at positions cr $%$$@ /oa2ž 9 cr in, where a is the nmber sch that ar ^ > >`s/0aû 9{. We need nits of space to hold the conts. In the worst case when every featre is distinct in the entire time series j \j, we need > R> nits of space P. After tep 1, we only need > > nits of space to keep, the set of freqent -patterns in. (2) At tep 2, the maximm nmber of > candidate sbpatterns that we may generate is > > > +%+%+ > > j \ &j 2C> >š2š. Considering that we still need > > space to keep the set of freqent 1- patterns, the total amont of space needed is j j 2p in the worse case in this comptation. owever, the average case shold be mch smaller than the worst case since if every featre is distinct in the time series, then there is no need to find periodic patterns. The existence of any periodicity in the time series will redce the memory needed ingle-period max-sbpattern hit set method Althogh the Apriori trick may redce the search space in partial periodicity mining in a similar way as association rle mining, it is important to note that the data characteristics in the two cases are very different. In mining association rles, the nmber of freqent -itemsets shrinks qickly as increases becase of the sparsity of freqent -itemsets in a large transaction database. owever, in mining partial periodicity, very often the nmber of freqent -patterns shrinks slowly (when, v ) as increases. The slow speed of decrease in the nmber of freqent -patterns is de to a strong correlation between freqencies of patterns and their sbpatterns. We now illstrate this point. Example 3.1 ppose we have two freqent 1-patterns, F ) and ), sch that Z / F )9 $œ and Z /0)9, $œ in a time-series. Then it mst be the case that $ž Z / F 9 ^v, as explained below. ince all period segments that match F match both F ) and ), Z $œ / F 9Ÿ^_ $œ holds. To derive the other ineqality, let F denote the predicate that a letter is not F, similarly. The confidence of F ) in is at most, becase Z / F )9 Q2 Z / F )9. imilarly, Z /o) %9q^. ince Z / F 9Ÿ v,2 Z / F )9c2 Z /o) 9, it follows that Z / F 9Q b. $ ž The slow redction of the set of candidate freqent - patterns as grows makes the Apriori prning of Algorithm 3.1 less attractive. Is there a better way? The nit of space is the space needed to hold the featre identifier and its associated cont, and its size is sally 2-8 bytes, depending on the implementation. This is eqal to the total space that the time series occpies. Obviosly, the derivation of freqent -patterns is still an effective way to dramatically redce the candidate set to be examined later becase there are sally only a small nmber of featres being freqent at a particlar position bt there cold be a large nmber of featres appearing in the position. This is especially tre when the average nmber of featres per position is larger than N ' oo'ü. Ths or discssion will be focsed on how to redce the search effort after the set of freqent -patterns,, is fond. Or key idea is based on the notions of max-patterns and hit patterns, defined next. A candidate (freqent) max-pattern, ªŸ, is the maximal pattern which can be generated from, the set of freqent -patterns. For example, if the freqent 1-pattern set is 4 F )O) ) ) ),) )) )«) ) ) )O)O)J8)8, the candidate max-pattern is F J8). Notice that a position in the candidate max-pattern may be allowed to have a disjnction of more than one non-) letter. For example, if the freqent 1-pattern set is 4 F ) )O)«) ) )O)) ) )«)) )O) )«) )G) )J8)8, the candidate max-pattern is F 4# J8). Let the ( -length of the candidate max-pattern, ªŸ, be > ªŸX>. A sbpattern of ªŸG is hit in a period segment of if it is the maximal sbpattern of ªŸG in. For example, for ª G F 4# J8), the hit sbpattern for a period segment F 4# # % 4#J #K is F 4 ) )), becase it is tre in and none of its sperpatterns F 4# ) ), F 4?)\J?), and F 4 J8), is in. The hit set,, of a time series is the set of all hit sbpatterns of ªŸ in. The seflness of hit max-patterns is: We can derive the complete set of partial periodic patterns, from the freqency conts of all the hit maximal sbpatterns of ª. This will be detailed below. We wold like to give an estimate of the bffer size needed in comptation based on the idea of hit patterns. One pper bond of the bffer size is estimated in terms of a, the total nmber of periods in. > Œ>, the size of the hit set in a time series, shold be no bigger than a, i.e., > Œ> ^ a. This is obvios since each period segment can generate at most one hit sbpattern, and a hit sbpattern may be hit in more than one period segment. The other pper bond of the bffer size is estimated in terms of the maximal nmber of patterns that can be generated from the set of freqent 1-patterns. ince each hit pattern of is a sbpattern of ªŸ, which is generated from, similar to the analysis performed in Algorithm 3.1, the size of the set of sbpatterns which can be generated from is > > > j j 2. ++%+ > > Therefore, > Œ>, the size of the hit set in a time series, shold be no bigger than j j 25. Combining both pper bonds, we have,

6 T Z T Z T T ž $ 5 ) ) ) J Property 3.2 [The bond of hit set] The size of the hit set is bonded by the formla, > Œ>±^²aCkn4#a j j 25 ³, where a is the total nmber of periods in, and set of freqent 1-patterns. is the Using this formla, we can calclate the bond of the maximal bffer size needed in the processing: Given the set of freqent 1-patterns,, the maximal (additional) bffer size needed for registering the conts of all the maximal sbpatterns of ª G is ackn4#a j j 25> >#2 ³. This property is very sefl in practice. For example, if we fond 500 freqent 1-patterns when calclating yearly periodic patterns for 100 years, the bffer size needed is at most 100; on the other hand, if we fond 8 freqent 1-patterns for calclating weekly periodic patterns for 100 years, the bffer size needed is at most ³Ó2 2s ³?µ. We can always select the smaller one in estimating the maximal bffer size needed in comptation. Before trning to or hit-set based algorithm, we examine the probability distribtions of maximal sbpatterns of ªŸ. eristic 3.1 [Poplarity of longer sbpatterns] The probability distribtion of the maximal sbpatterns of ª G is sally denser for longer sbpatterns (i.e., with the ( - length closer to > ªŸ±> ) than the shorter ones. This heristic can be observed in Example 3.1. From the example, we have $ž ^E %4 a F8 *WUo F [o[ik n/ F 9 F # ^, bt $œ %4 a F8 *%WUo F [o[ik n/ F 9 F )=Š^. In most cases, the existence of a short max-sbpattern indicates that the nonexistence of some non-) -letter, which redces the chance for the corresponding non-) letter patterns to reach high confidence. Ths we have the heristic. This heristics will imply that the nmber of nodes in the tree data strctre of the next section is sally small. It is also sefl for efficient bffer management: In order to redce the overall cost of access, the longer sbpatterns shold be arranged to be more easily accessible (sch as pt in main memory) than the shorter ones. We now present a main algorithm for mining partial periodic patterns for a given period, which is based on the discssions above. Algorithm 3.2 [Max-sbpattern hit-set] Find all the partial periodic patterns for a given period in a time-series, based on the max-sbpattern hit-set, for a given min conf threshold. 1. can once to find, the set of freqent 1-patterns of period, sing tep 1 of Algorithm 3.1. Form the candidate max-pattern, ª G, from. 2. can once. Dring the scan, for each period segment, if its hit set is nonempty, do the following: add the max-sbpattern into the hit set bffer (with the associated cont initialized to 1) if it is not already there; otherwise, increase the cont of the max-sbpattern by one. The hit set bffer is implemented in the form of a max-sbpattern tree, a novel data strctre, to be discssed in ection After the scan, derive the freqent patterns from the hit set. We will discss how to implement the finding of the conts of the hit patterns and how to se these conts to derive the freqent patterns in ection 4. It trns ot that both can be done efficiently. Nmber of scans over the time series. The first step of the algorithm needs to scan once. The second step needs to scan one more time. Ths the total nmber of timeseries scans is 2, independent of the period. pace needed. (1) The space needed for tep 1 is the same as Algorithm 3.1. After tep 1, we need > > nits of space to keep, the set of freqent -patterns in. (2) At the second step, sppose there are > > freqent -patterns in. According to Property 3.2, the total space needed for the hit set is at most ackn4#a j j 2 ³, where a is the total nmber of periods in. In comparison with Algorithm 3.1, Algorithm 3.2 redces the total nmber of scans of the time series from (the length of the period) to 2, and it also ses mch less bffer space in the comptation in most cases. This can also be seen from the following observation: ppose the hit sbpattern for a period segment is F J, which is not in the hit set yet. We need only one nit space to register the string and its cont 1. owever, for the Apriori techniqe, the candidate 2-patterns to be generated will be 4 F G)R) F ) F )Ž)J ) ),)RJ )) J\, 3-patterns to be generated will be 4 F F N)J F ) ) J\, and the 4-patterns will be 4 F J\, pls we have to pdate the cont associated with each of them. Ths, it is expected that the max-sbpattern hit set method may have better performance in most cases. We will compare the performance of the two algorithms in ection Mining partial periodicity with mltiple periods Mining partial periodicity for a given period covers a good set of applications since people often like to mine periodic patterns for natral periods, sch as annally, qarterly, monthly, weekly, daily, or horly. owever, certain patterns may appear at some nexpected periods, sch as every 11 years, or every 14 hors. It is interesting to provide facilities to mine periodicity for a range of periods.

7 ¼ ¼ $ / 6 To extend partial periodicity mining from one period to mltiple periods, one might wish to extend the idea of Apriori to compting partial periodicity among different periods, that is, to se the patterns of small periods as filters for candidate patterns of periods of the form ¹ for an integer ¹s º. This will work if all freqent patterns of period ¹ are freqent patterns of period. Unfortnately, this is not the case. For example, for the time series F J F K F J F K, Z /0) ) J?9», ppose the confidence threshold is. If we se from partial periodic patterns of period as filter for candidate partial periodic patterns of period, we will miss the partial periodic pattern )G) J. Given that we cannot extend the Apriori trick to mltiple periods, one obvios way to mine partial periodic patterns for a range of periods is to repeatedly apply the singleperiod algorithm for each period in the range. Algorithm 3.3 [Looping over single period comptation] Find all the partial periodic patterns for a set of periods in a given range of interest, $%$%$I X¼, in the time-series, with the given min conf threshold. 1. for each period ½ in the range of interest (i.e., %$$%$ U¼ ), apply Algorithm 3.2 ( max-sbpattern hitset ) on period ½. Nmber of scans over the time series. ince each period will take 2 scans of the time series, the total nmber of scans of the time series is ¹. pace needed. For compting partial periodicity for periods from to ¼, the space reqired is basically the sm of space for each ½. Notice that the space reqired for initial j \j tep 1 comptation is still > I> in the worst case since the space once sed in comptation for period ½, can be reinitialized and resed for compting other periods. Bt we need in total ½ > /š ½9%> nits of space to keep different sets of freqent 1-patterns, where /š ½ 9 is the set of freqent -patterns in derived for period ½. imilarly, it takes at most ½ ackn4#a½ j }¾ -% IÀ j 2! nits of space to compte all, where a ½ is the total nmber of periods ½ in. Algorithm 3.3 provides an iterative method for mining partial periodicity for mltiple periods. owever, when the nmber of periods is large, we still need a good nmber of scans to mine periodicity for mltiple periods. An improvement to the above method is to maximally explore the mining of periodicity for mltiple periods in the same scan, which leads to the shared mining of periodicity for mltiple periods, as illstrated below. J89 Algorithm 3.4 [hared mining of mltiple periods] hared mining of all the partial periodic patterns for a set of periods in a given range of interest, $%$%$I X¼, in timeseries, with the given min conf threshold. 1. can once, for all periods ½ in the range of interest, do the same as tep 1 in Algorithm 3.2. That is, for all periods ½ in the range of interest (i.e., %$$%$ ¼ ), find /Á ½ 9, the set of freqent 1-patterns of period ½, sing the same tep 1 as in Algorithm 3.1. For each set of freqent 1-patterns of period ½, form the candidate max-pattern, ªŸ\/Á ½ 9, from /Á ½ can once, for all periods ½ in the range of interest, do the same as tep 2 in Algorithm 3.2. A similar process which will not be explained in detail. Nmber of scans over the time series. The first step of the algorithm needs to scan once. The second step needs to scan one more time. Ths the total nmber of timeseries scans is 2, independent of the period. pace needed. The total space reqired in the worst case is same as in Algorithm 3.3. Algorithm 3.4 explores shared processing at mining partial periodicity for mltiple periods. The advantage of the method is that we only need two scans of time series for mining partial periodicity for mltiple periods. The overhead of the method is that althogh it redces the nmber of scans to 2, it will reqire more space in the processing of each scan than the mltiple scan method becase it needs to register the corresponding conts for each period ½ (for ẁâã`_¹ ). owever, since the shared featres will share the space as well (with conts incremented), and there shold be many shared featres in periodicity search (otherwise, why mining periodicity?), the space reqired will hardly approach the worst case. Therefore, it shold still be an efficient method in many cases for mining partial periodicity with mltiple periods. 4 Derivation of all partial patterns In this section, we examine the implementation considerations of or proposed algorithms. Algorithm 3.1 is an Apriori-like algorithm which can be implemented similarly as other Apriori-like algorithms for mining association rles (e.g. [2]). Algorithm 3.2 forms the basis for all the three remaining algorithms and reqires new tricks to achieve efficiency, and ths or discssion is focsed on its efficient implementation. Algorithm 3.2 consists of two steps: tep 1, scan the time series once and find freqent 1-pattern set ; and

8 . K tep 2, scan the time series one more time, collect the set of the max-sbpatterns hit in, and derive the set of freqent patterns. The implementation of tep 1 is straightforward and has been discssed in the presentation of Algorithm 3.1. owever, tep 2 is nontrivial and needs some good data strctre to facilitate the storage of the set of maxsbpatterns hit in and the derivation of the set of freqent patterns. A new data strctre, called max-sbpattern tree, is designed to facilitate the registration of the hit cont of each max-sbpattern and derivation of the set of freqent patterns, as illstrated in Figre 1. Its design is now otlined. The max-sbpattern tree takes the candidate max-pattern ªŸ as the root node, where each sbpattern of ªŸG with one non-) letter missing is a direct child node of the root. The tree expands recrsively, according to the following rles. A node, if containing more than 2 non-) letters, may have a set of children, each of which is a sbpattern of with one more non-) letter missing. Notice that a node containing only 2 non-) letters will not have any children since every freqent-1 pattern is already in. Importantly, we do not create a node if neither the node nor its descendant(s) containing more than 1 non-) letter is hit in ÅÄ. Each node has a cont field (which registers the nmber of hits of the crrent node), a parent link (which is nil for the root), and a set of child links; each child link points a child and is associated with a corresponding missing letter. A link can be nil when the corresponding child has not been hit. Notice that a non-) letter position of a max-sbpattern in a max-sbpattern tree may contain a set of letters, which matches the set of letters at the position in a period segment. For example, for ª = F 4 of the period segment F 4# 4 % #J X)cJ8), the max-sbpattern is F 4# )), and the segment will contribte one cont to this node. The pdate of the max-sbpattern tree is performed as follows. Algorithm 4.1 [Insertion in the max-sbpattern tree] Insert a max-sbpattern fond dring the scan of into the max-sbpattern tree Æ. 1. tarting from the root of the tree, find the corresponding node by checking the missing non-) letter in order. For example, for a max-pattern node ) )J8) in a tree with the root, ªŸG F 4# )«J?), there are two letters, F and, missing. The node can be fond by (1) following the F link (marked as Ç F in Figre 1) to )84# X)cJ8), and then (2) following the link to ) )OJ8), as shown in Figre 1. È we show sch a node h%é Ê sing a dotted box in Figre 1. G) 10 a{b1, b2}*d* a b1 b2 d *{b1,b2}*d* ab2*d* ab1*d* a{b1,b2}*** a a b1 a b1 b2 b1 d b2 8 b d 19 0 d 2 *b2*d* *b1*d* *{b1,b2}*** a**d* ab2*** ab1*** Figre 1. A max-sbpattern tree to store the set of max-sbpatterns hit in the time-series. 2. If the node is fond, increase its cont by 1. Otherwise, create a new node (with cont 1) and its missing ancestor nodes (only those on the path to, with cont 0), if any, and insert it (or them) into the corresponding place(s) of the tree. For example, if the very first max-sbpattern node fond in is ) )ŽJ8) for ªŸ F 4 R)ŠJ8), we will create the node ) ),J8) (with cont 1), after creating two ancestor nodes (with cont 0): = F 4#?) J8) (which is the root of the tree), and = )84 N)GJ8) (which is F link). The node ) ) J8) is s child, following the «s child, following the link. G Let the total nmber of non-) letters in ªŸG be. For a max-sbpattern containing UË (UËÅ ² ) non-) letters, we need to follow 2pUË links to find the node and create at most 2bUËp new nodes in the worst case. Therefore, the time complexity of node search and node creation will be less than. Also, since each insertion of maxsbpattern will create either only 0 node (when it hits) or less than nodes, the total nmber of the nodes in the tree is less than > Œ>, where > Œ> is the size of the hit set. In general, to insert a sbpattern we need to both locate the position and pdate the cont of the node if the node is fond, or otherwise insert one or several new nodes. Example 4.1 Let Figre 1 be the crrent max-sbpattern tree Æ. To insert a (max)sbpattern F )c)) into the tree, we search the tree starting with the root, ªŸ F 4# )J8). The first non-) letter missing is and the second non-) letter missing is J. Ths we first follow the branch to node F )J8), and then follow the J branch. ince the node F )G)) is located, its cont is incremented by 1. Before discssing the derivation of the set of freqent patterns, we need to introdce the concept of reachable ancestors. ince the traversal and creation of the children of a

9 F 8 Æ ) a ) node in the max-sbpattern tree follow the non-) letter position order, some of the ancestor nodes of a node may not be directly linked to a node. For example, in Figre 1, the node F ) )J8) is linked to only one parent F )OJ8) bt not the other F ) J?) (note: this missing link is marked by a dashed line in the Figre). In general, the set of reachable ancestors of a node in a max-sbpattern tree Æ is the set of all the nodes in Æ, which are proper sperpatterns of. It can be compted as follows: (1) derive a list of missing letters from based on ªŸG, which is roghly the position-wise difference, (2) the set of linked ancestors consists of those patterns whose missing letters form a proper prefix of, and (3) the set of not-linked ancestors are those patterns whose missing letters form a proper sblist (bt not prefix) of. Example 4.2 We compte the set of reachable ancestors for a node )Ì)3)J8) in a max-sbpattern tree with root ªŸ F 4 O)RJ8). The list of missing non-) letters is Í F. Ths, the set of linked ancestors is (1) IÎ 6 (missing nothing, which is the root); (2) F (i.e., missing F, which is the node )84 ;)J8K ); and (3) (i.e., missing F, then missing, which is the node ) )RJ8K ). The set of notlinked ancestors is: ) )«J8) (corresponding to the missing letter pattern F ), F ) J8) (corresponding to ), F )O)J8) (corresponding to ), and F ),J?) (corresponding to ). In other words, one can follow the links whose mark is not J in ordered way (to avoid visiting the same node more than once) and collect all the non- nodes reached in Æ. Essentially there is a tree traversal for each fixed pattern, except that we do not visit a node and its descendants if the node is not an ancestor pattern of or crrent pattern. The derivation of the freqent ¹ -patterns is performed as follows. Algorithm 4.2 [Derivation of freqent patterns from max-sbpattern tree] The derivation of the freqent ¹ - patterns for all ¹, given a max-sbpattern tree Æ, by an Apriori-like techniqe. 1. The set of freqent -patterns is derived in the first scan of Algorithm The max-sbpattern tree Æ is derived in the second scan of Algorithm 3.2. The set of freqent ¹ -patterns (¹ ) is derived as follows. for Ï to > > do 4 derive candidate patterns with ( -length from freqent patterns with ( -length /0@2C %9 by /0Ð 9 -way join. scan tree T to find freqency conts of these candidate patterns and eliminate the non-freqent ones. Notice that the freqency cont of a node is the sm of the cont of itself and those of all of its reachable ancestors. If the derived freqent -pattern set is empty, retrn. Let the total nmber of non-) letters in ª G be c. As shown in the analysis of Algorithm 4.1, the time complexity for searching a node is less than c. ince there are at most '8Ñ 2 c nodes to be generated from the max-pattern tree (inclding all the missing descendants), and there are at most > Œ> reachable ancestors in Æ, where > Œ> is the size of the hit set, the worst case time complexity for derivation of all the freqent patterns is O( ' Ñ > w> ), i.e., proportional to ' and the size of the hit set, bt exponential to (i.e., proportional to the size of the tree that can be generated by ª G ). ince an infreqent node will redce the nmber of candidates to be generated in the ftre ronds, the real processing cost is sally mch smaller than the cost in the worst case. We illstrate how to derive the freqent ¹ -patterns for ¹Š v from the max-sbpattern tree Æ. Example 4.3 Let Figre Ò 1 be the derived max-sbpattern tree Æ, and â } 8L. We can traverse the max- Z sbpattern tree to find all the freqent ¹ -patterns for ¹Ð ² as follows. tarting at level 2, we have the following freqent patterns: 4#) )ŸJ8) (68), ) )ŸJ8) (68), )84# Ó)Ÿ)) (4), F )G)J8) (119), F )Ÿ)) (92), F )Ÿ)) (84). We show the derivation of ) ) J8) (68) here: since the list of missing letters in this node is Í F, its set of reachable ancestors F ÔÎ,, and ths its freqent cont = is 4 6, 8 (itself) = 68. ince level-2 has no infreqent nodes, we search all the nodes at level-1 and have the following freqent patterns: 4 F );J8) (60), F ),J8) (50) ; ince there is one node infreqent, level-0 (root) has no freqent patterns. Notice althogh we only saved one node comptation in this case, it will save mch more when the tree is large and there are more missing nodes. From the above example, one can see that there are many freqent ¹ -patterns with small ¹ that can be generated from a max-sbpattern tree. In practical applications, people may only be interested in the set of maximal freqent patterns instead of all freqent patterns, where a set of maximal freqent patterns is a sbset of the freqent pattern set and every other pattern in the set is a sbpattern of an element in the set. For example, if the set of freqent ¹ pattern (for F )=, the set of maximal ¹w Õ ) is 4 F Ÿ) ) ) freqent patterns is 4 F F ) )8.

10 9 If a ser is interested in deriving the set of maximal freqent patterns, the MaxMiner algorithm developed by Bayardo [4] is a good candidate. The sccess of this algorithm stems from generating new candidates by joining freqent itemsets and looking head. owever, it still reqires to scan p to period times in the worst case. The mixtre of max-sbpattern hit set method and the MaxMiner can get rid of this problem and will be more efficient than pre MaxMiner. The details of the new method will be examined in ftre research. 5 Performance stdy In this section we report a performance stdy which compares the performance of the periodicity mining algorithms proposed in this paper. In particlar, we give a performance comparison between the single-period Apriori algorithm (Algorithm 3.1) (or simply called Apriori), and the max-sbpattern hit-set algorithm (Algorithm 3.2) (or simply hit-set) applied to a single period. This comparison indicates that there is a significant gain in efficiency by max-sbpattern hit-set over Apriori. ince there is more gain when applied to mltiple periods by sing max-sbpattern hit-set, it is clear that max-sbpattern hit-set is the winner. The performance stdy is condcted on a Pentim 166 machine with 64 megabytes main memory, rnning in Windows/NT. The program is written in Microsoft/VisalC Testing Databases Each test time series is a synthetic time-series databases generated sing a randomized periodicity data generation algorithm. From a set of featres, potentially freqent 1- patterns are composed. The size of the potentially freqent 1-patterns is determined based on a Poisson distribtion. These patterns are generated and pt into the time-series according to an exponential distribtion. LENGT the length of time series a period MAX-PAT-LENGT the maximal ( -length of freqent patterns > > the nmber of freqent 1-patterns Table 1. Parameters of synthetic time series The basic parameters sed to generate the synthetic databases are listed in Table 1. The parameters of LENGT (the length of time series) and (a period) are independently chosen. The parameters of MAX-PAT-LENGT (the maximal ( -length of freqent patterns) and > > (the nmber of freqent 1-patterns) are for a fixed, and they are controlled by the choice of some appropriate confidence threshold. We fond that other parameters, sch as the nmber of featres occrring at a fixed position and the nmber of featres in the time series, do not have mch impact on the performance reslt and ths they are not considered in the tests. 5.2 Performance comparison of the algorithms Figre 2 shows there is a significant efficiency gain by max-sbpattern hit-set over Apriori. In this figre, the maximal pattern length (the maximal ( -length of freqent partial periodic patterns) grows from to %. The other parameters are kept constant: L and > >. We rn two sets of tests, one with the length of the time series being % and the other being L. As we can see, the rnning time of max-sbpattern hit-set is almost constant for both cases, while Apriori is almost linear. When MAX-PAT-LENGT is, the gain by ž max-sbpattern hit-set over Apriori is abot doble. We expect this gain will increase for larger MAX-PAT-LENGT. Time (seconds) Apriori 500k itet500k Apriori 100k itet100k Max-Pat-Length Figre 2. Performance gain when MAX-PAT-LENGT increases: L, > >. It is important to note that, the gain shown in Figre 2 is done by keeping everything in memory, and by considering only one period. In general, this will be nlikely the case, and max-sbpattern hit-set will perform even better than Apriori for the following reasons: In general, the time series of featres may need to be stored on disk, de to factors sch as each may contain thosands of featres and the length of the time series can be longer. When the time series is stored on disk, there wold be a large amont of extra disk-io associated with Apriori, bt not with max-sbpattern hit-set since it only reqires two scans. Even when the time series is not stored on disk, Apriori will need to go over this hge seqence many more times than

11 10 max-sbpattern hit-set. Ths max-sbpattern hit-set will be far better than Apriori. When there are a range of periods to consider, max-sbpattern hit-set can find all freqent patterns in two scans bt Apriori will reqire many more scans, depending on the nmber of periods and the ( -length of the maximal freqent patterns. ence max-sbpattern hit-set will be again far better than Apriori. 6 Conclsions We have stdied efficient methods for mining partial periodicity in time series database. Partial periodicity, which associates periodic behavior with only a sbset of all the time points, is less restrictive than fll periodicity and ths covers a broad class of applications. By exploring several interesting properties related to partial periodicity, inclding the Apriori property, the maxsbpattern hit set property, and shared mining of mltiple periods, a set of partial periodicity mining algorithms are proposed, with their relative performance compared. Or stdy shows that the max-sbpattern hit set method, which needs only two scans of the time series database, even for mining mltiple periods, offers excellent performance. Or stdy has been confined to mining partial periodic patterns in one time series for categorical data with single level of abstraction. owever the method developed here can be extended for mining mltiple-level, mltipledimensional partial periodicity and for mining partial periodicity with pertrbation and evoltion. For mining nmerical data, sch as stock or power consmption flctation, one can examine the distribtion of nmerical vales in the time-series data and discretize them into single- or mltiple- level categorical data. For mining mltiple-level partial periodicity, one can explore levelshared mining by first mining the periodicity at a high level, and then progressively drilling-down with the discovered periodic patterns to see whether they are still periodic at a lower level. Pertrbation may happen from period to period which may make it difficlt to discover partial periodicity in many applications. For mining partial periodicity with pertrbation, one method is to slightly enlarge the time slot to be examined. Partial periodic patterns with minor pertrbation are likely to be caght in the generalized time slot. Another method is to inclde the featres happening in the time slots srronding the one being analyzed. We can frther employ regression techniqe to redce the noise of pertrbation. There are still many isses regarding partial periodicity mining which deserve frther stdy, sch as frther exploration of shared mining for mining periodicity with mltiple periods, mining periodic association rles based on partial periodicity, and qery- and constraint- based mining of partial periodicity [11]. We are stdying these problems and implementing or algorithms for mining partial periodicity in a data mining system and will report or progress in the ftre. References [1] R. Agrawal, G. Psaila, E. L. Wimmers, and M. Zait. Qerying shapes of histories. In Proc. 21st Int. Conf. Very Large Data Bases, pages , Zrich, witzerland, ept [2] R. Agrawal and R. rikant. Fast algorithms for mining association rles. In Proc Int. Conf. Very Large Data Bases, pages 48499, antiago, Chile, eptember [3] R. Agrawal and R. rikant. Mining seqential patterns. In Proc Int. Conf. Data Engineering, pages 314, Taipei, Taiwan, March [4] R. J. Bayardo. Efficiently mining long patterns from databases. In Proc ACM-IGMOD Int. Conf. Management of Data, pages 8593, eattle, Washington, Jne [5] C. Bettini, X. ean Wang, and. Jajodia. Mining temporal relationships with mltiple granlarities in time seqences. Data Engineering Blletin, 21:3238, [6] J. an and Y. F. Discovery of mltiple-level association rles from large databases. In Proc Int. Conf. Very Large Data Bases, pages , Zrich, witzerland, ept [] J. an, W. Gong, and Y. Yin. Mining segment-wise periodic patterns in time-related databases. In Proc Int l Conf. on Knowledge Discovery and Data Mining (KDD 98), New York City, NY, Agst [8]. J. Loether and D. G. McTavish. Descriptive and Inferential tatistics: An Introdction. Allyn and Bacon, [9]. L, J. an, and L. Feng. tock movement and n- dimensional inter-transaction association rles. In Proc IGMOD Workshop on Research Isses on Data Mining and Knowledge Discovery (DMKD 98), pages 12:1 12:, eattle, Washington, Jne [10]. Mannila, Toivonen, and A. I. Verkamo. Discovering freqent episodes in seqences. In Proc. 1st Int. Conf. Knowledge Discovery and Data Mining, pages , Montreal, Canada, Ag [11] R. Ng, L. V.. Lakshmanan, J. an, and A. Pang. Exploratory mining and prning optimizations of constrained associations rles. In Proc ACM-IGMOD Int. Conf. Management of Data, pages 1324, eattle, Washington, Jne [12] B. Özden,. Ramaswamy, and A. ilberschatz. Cyclic association rles. In Proc Int. Conf. Data Engineering (ICDE 98), pages , Orlando, FL, Feb