Combining The Global and Partial Information for Distance-Based Time Series Classification and Clustering

Transcription

1 Paper: Combnng The Global and Partal Informaton for Dstance-Based Tme Seres Hu Zhang, Tu Bao Ho, Mao-Song Ln, and We Huang School of Knowledge Scence, Japan Advanced Insttute of Scence and Technology, Nom, Ishkawa , Japan School of Computer Scence, Southwest Unversty of Scence and Technology, Manyang, Schuan , Chna E-mal: [Receved ; accepted ] Many tme seres representaton schemes for classfcaton and clusterng have been proposed. Most of the proposed representaton focuses on the promnent seres by consderng the global nformaton of the tme seres. The partal nformaton of tme seres that ndcates the local change of tme seres s often gnored. Recently, researches shown that the partal nformaton s also mportant for tme seres classfcaton. However, the combnaton of these two types of nformaton has not been well studed n the lteratures. Moreover, most of the proposed tme seres representaton requres predefned parameters. The classfcaton and clusterng results are consderably nfluenced by the parameter settngs, and, users often have dffculty n determnng the parameters. We attack above two problems by explotng the mult-scale property of wavelet decomposton. The man contrbutons of ths work are: (1) extractng features combnng the global nformaton and partal nformaton of tme seres (2) automatcally choosng approprate features, namely, features n an approprate wavelet decomposton scale accordng to the concentraton of wavelet coeffcents wthn ths scale. Experments performed on several benchmark tme seres datasets justfy the usefulness of the proposed approach. Keywords: Tme Seres, Haar Wavelets, Classfcaton, Clusterng, Feature Extracton 1. Introducton Tme seres data accounts for a huge amount of the data stored n fnancal, gene expresson, medcal and scence databases. Many algorthms have been proposed for mnng tme seres data [19, 28]. Tme seres classfcaton and tme seres clusterng are two mportant aspects of tme seres mnng. Tme seres classfcaton has been successfully used n varous applcatons such as medcal data analyss [12, 33], sgn language recognton [15], speech recognton [23], etc. Tme seres clusterng s a popularly used preprocessng technque n stock market analyss [9], gene expresson data analyss [13], and so on. The hgh dmensonalty, hgh correlaton between data, and hgh nose embedded n tme seres make tme seres classfcaton and clusterng challengng tasks. For effcency, most of the proposed methods classfy or cluster tme seres on the hgh level representaton of tme seres that takes the global nformaton of tme seres rather than classfyng or clusterng them drectly. The representaton ncludes Fourer Transforms [1, 27], Pecewse Lnear Representaton (PLR) [21], Pecewse Aggregate Approxmaton [17, 34], Regresson Tree Representaton [10], Haar Wavelets [3], and Symbolc Representaton [24]. Recently, Jn et al. proposed a tme seres representaton scheme based on the partal nformaton of the tme seres and showed that the partal nformaton s also mportant for tme seres classfcaton [14]. However, lttle work has done to combne these two types of nformaton n tme seres mnng lteratures. Most of the proposed representaton schemes n the lteratures requre predefned parameters and the correspondng classfcaton or clusterng algorthms are consderably nfluenced by these predefned parameters [20]. For example, when settng the number of straght lnes as the nput parameter for the PLR algorthm, the range of selecton s lmted from one to the length of raw tme seres data. If the nput parameter s one, the representaton s just a lnear regresson of the whole data set, and the classfcaton accuracy n ths case wll be lower than usng raw data for most data and algorthms. If we choose the length of raw tme seres data as the nput parameter, the represented data s actually the raw data. The selecton s not trval or easy for the users, and as a consequence, they usually have dffculty n determnng the parameters. Note that a doman-transform technque such as Fourer transform doesn t need nput parameters tself, but the later feature extracton process needs nput parameters n most cases. In ths paper we ntroduce a tme seres representaton combnng both the global nformaton and the partal nformaton of the tme seres data by Haar wavelet decomposton. Tme seres classfcaton and clusterng algorthms are performed wth the selected features of the representaton. We propose a novel non-parametrc feature extracton algorthm that extracts the approxmaton Journal of Advanced Computatonal Intellgence 1

2 Zhang, H. et al. (global nformaton) and change ampltudes (partal nformaton) of tme seres. The Eucldean dstances between the features of shfted tme seres and orgnal tme seres are smaller than those between the raw data. Hence, dstance-based classfcaton and clusterng algorthms are sutable for the features. The approprate features,.e., features wthn the approprate wavelet decomposton scale, are chosen wth respect to the concentraton of features. The approprate features are robust aganst nose embedded n the tme seres. Experments performed on several benchmark datasets demonstrate the effectveness of the proposed approach. The rest of ths paper s organzed as follows. Secton 2 brefly dscusses the related work. Secton 3 ntroduces our feature extracton algorthm and the correspondng dstance-based classfcaton and clusterng algorthms. Secton 4 contans an comparatve expermental evaluaton wth the proposed approach. Fnally, Secton 5 concludes the paper wth summarzng the man contrbutons of the work. 2. Related work A large number of wavelet-based feature extracton technques have been proposed. Chan and Fu ntroduced an algorthm for nearest neghbor queryng wth Haar wavelet coeffcents, and only frst few wavelet coeffcents were preserved for dmensonalty reducton [3, 4]. Popvanov and Mller used frst few Daubeches wavelet coeffcents nstead of Haar wavelet coeffcents for tme seres queryng [26]. Shahab et al. proposed an algorthm called TSA-tree whch queres ether the approxmaton part or detal part of Haar wavelet coeffcents n a specfc scale gven by the user [30]. Struzk and Sebes defned a new tme seres smlarty measurement on the correlaton of Haar wavelet coeffcents [31]. In the sgnal processng communty, Pter and Kamarth chose clustered wavelet coeffcents as features [25]. Tancel et al. only used approxmaton wavelet coeffcents as nput to an ART-2 type neural network [32]. Kalayc and Ozdamar suggested a method usng eght central detal coeffcents of scale m rangng from 1 to 5 as the features [16]. To date, all the proposed methods use global nformaton or partal nformaton, but no proposed algorthm combnes these two types of nformaton. Furthermore, no proposed work gves soluton for automatcally choosng approprate features for a gven dataset. We propose a method of usng entropy to choose approprate scale whch s smlar n sprt to the wavelet packet algorthm ntroduced by Cofman et al. [5], n whch the entropy s used to select the best bass for a wavelet packet. 3. Tme Seres Representaton and Feature Extracton Our basc dea s to extract features from tme seres and perform classfcaton and clusterng algorthms wth the extracted features. The mult-scale property of wavelets allows us to extract features wth the global nformaton and partal nformaton smultaneously. After obtanng the features, dstance-based classfcaton and clusterng algorthms can be appled n terms of the smlarty between the extracted features. Therefore, we decompose our task nto three sub-procedures: (1) representng the tme seres va wavelet coeffcents wthn varous scales whch contan both the global and partal nformaton of the tme seres data; (2) retrevng the features by selectng the approprate scale of the representaton; and (3) desgnng a smlarty measurng strategy, n whch most proposed smlarty models could be appled. The basc dea of Haar wavelet decomposton s ntroduced n Secton 3.1. We gve the tme seres representaton and correspondng feature extracton algorthms n Sectons 3.2 and 3.3. Secton 3.4 presents the method of nose reducton wth the features. We suggest the smlarty measure strategy and ts correspondng classfcaton and clusterng algorthms n sectons Haar Wavelet Decomposton Wavelet transform s a doman transform technque for herarchcally decomposng sequences. It allows a sequence to be descrbed n terms of an approxmaton of the orgnal sequence, plus a set of detals that range from coarse to fne. One property of wavelets s that the broad trend of the nput sequence s preserved n the approxmaton part, whereas localzed changes are kept n the detal parts. No nformaton s ganed or lost durng the decomposton process. The orgnal sgnal can be fully reconstructed from the approxmaton part and the detal parts. The Haar wavelet s the smplest and most popular wavelet proposed by Haar. The beneft of the Haar wavelet s that ts decomposton process has low computatonal complexty. Gven a tme seres wth length n, where n s an ntegral power of 2, the complexty of Haar decomposton s O(n). The Haar wavelet decomposton process needs a par of sequences assocated wth t. The sequences are called wavelet analyss flters, denoted as {h k,g k }. The Haar wavelet analyss flters h k and g k are gven by h k = [1/ 2,1/ 2] and g k = [1/ 2, 1/ 2] Haar wavelet decomposton s mplemented by a twostep process: down-samplng and a convoluton wth the down-sampled seres and wavelet analyss flters. Concrete mathematcal foundatons can be found n [2]. The length of the nput tme seres s restrcted to an nteger power of 2 n the process of wavelet decomposton. The seres wll be extended to an nteger power of 2 by paddng zeros to the end of tme seres f the length of the nput tme seres doesn t satsfy ths requrement. A tme seres X = {x 1,x 2,...,x n } can be decomposed nto an approxmaton part A 1 = {(x 1 + x 2 )/ 2,(x 3 + x 4 )/ 2,...,(x n 1 + x n )/ 2} and a detal part D 1 = {(x 1 x 2 )/ 2,(x 3 x 4 )/ 2,...,(x n 1 x n )/ 2}. The 2 Journal of Advanced Computatonal Intellgence

3 Fg. 1. An example of a tme seres and ts wavelet coeffcents to scale 2 A 1 are approxmaton coeffcents wthn scale 1 and D 1 are detal coeffcents wthn scale 1. The approxmaton coeffcents and detal coeffcents wthn a partcular scale j, A j and D j, both havng length n/2 j, can be decomposed from A j 1, the approxmaton coeffcents wthn scale j 1 recursvely. The th element of A j s calculated as a j = 1 (a 2 1 j 1 + a2 j 1 ), [1,2,...,n/2 j ].. (1) 2 The th element of D j s calculated as: d j = 1 2 (a 2 1 j 1 a2 j 1 ), [1,2,...,n/2 j ]... (2) The number of decomposng scales for X s J = log 2 (n). A J only has one element denotng the global average of X Tme Seres Representaton and Feature Extracton The concatenaton of decomposed wavelet coeffcents of a tme seres X = {x 1,x 2,...,x n } to a partcular scale k [1,2,...,J] shown n (3) s a representaton of X. An example of decomposng a tme seres to scale 2 s llustrated n Fg. 1. The X can be fully reconstructed from W k (X) wthout losng any nformaton. W k (X) = [ A k (X), D k (X),..., D 2 (X), D 1 (X)]. (3) Chan et al. proved that the Eucldean dstance s preserved through a Haar wavelet transform [3]. Assume we have two tme seres X = {x 1,x 2,...,x n } and Y = {y 1,y 2,...,y n }, the Eucldean dstance between X and Y s n Dsc( X, Y ) = (x y ) (4) =1 j The Eucldean dstance between W k (X) and W k (Y ), Dsc( W k (X), W k (Y )) s equal to Dsc( X, Y ). In ths case, f we just use W k as the features for a dstance based classfcaton or clusterng algorthm, the result should be the same wth that gotten from the orgnal tme seres. The th element of A j corresponds to the segment n the seres X startng from poston ( 1) 2 j + 1 to poston 2 j. The a j s proportonal to the average of ths segment and thus can be vewed as the approxmaton of the segment. Thus the approxmaton coeffcents wthn dfferent scales provde an understandng of the major trends n the data at a partcular level of granularty. From Eq. 2, we know that the detal coeffcents D j = {d 1 j j,d2 j,...,dn/2 } contan local changes of tme seres. Thus D j = { d 1 j, d2 j j } denote the ampltude of local changes. We defne the concatenaton of decomposed wavelet approxmaton coeffcents A k (X) and the absolute values of decomposed wavelet detal coeffcents D j (X), j = 1,2,...,k to a partcular scale k (k [1,2,...,J]) of a tme seres X as features. F k (X) = [ A k (X), D k (X),..., D 2 (X), D 1 (X) ] (5),... dn/2 j Ths defnton helps to overcome the well-known problem posed by the fact that wavelet coeffcents are senstve to shfts of seres. The Eucldean dstance Dsc( F k (X), F k (Y )) between the features of two tme seres X and Y s (a k ( X ) a k k ( Y )) 2 + ( d j ( X ) d j ( Y ) ) 2 (6) j=1 Because x y x y, we obtan Dsc( F k (X), F k (Y )) Dsc( W k (X), W k (Y )), and Dsc( W k (X), W k (Y )) = Dsc( X, Y ). If X and Y denote the orgnal tme seres and shfted tme seres respectvely, ths nequaton s stll tenable Approprate Scale Selecton Normally, for a tme seres X, the frst k, k < n coeffcents of W J (X) are taken as features after decomposng X to the scale J [3, 26]. The frst few wavelet coeffcents are correspond to the approxmaton wavelet coeffcents ncludng the global nformaton of X. However, settng the parameter k s not easy for the users and the later classfcaton or clusterng process are affected by poorly chosen parameter settngs. A non-parametrc feature extracton algorthm wthout any nput parameter s more convenent to the users. For our defnton of features, the nonparametrc feature extracton algorthm needs to fnd out whch features assocated wth a specfc scale are better than others for classfcaton and clusterng automatcally. If the energy of wavelet coeffcents wthn a scale s concentrated n a few coeffcents then just those mportant coeffcents can represent the whole, wth low error. Ths scale may gve valuable nformaton for classfcaton and clusterng. We need a functon to descrbe the en- Journal of Advanced Computatonal Intellgence 3

4 Zhang, H. et al. ergy concentraton of the wavelet coeffcents. The functon should be large when coeffcents are largely the same value, and small when all but a few coeffcents are neglgble. Shannon entropy, whch s a measure of mpurty wthn a set of nstances satsfes our requrement. It s defned below. H = p log 2 p (7) The approprate decomposng scale s defned as the scale wth the lowest entropy. The approprate features of a tme seres are defned as the wavelet coeffcents wthn an approprate decomposng scale. Approprate scale = argmn( p k log 2 p k ). (8) k here p k = F k (X) / n =1 F k (X) s the proporton between the absolute value of a coeffcent n a feature and the sum of the absolute values of a whole feature. p k s proportonal to the energy rato of each coeffcent to coeffcents wthn a feature. p k has propertes p k = 1 and p k Nose Reducton on The Approprate Features The dea of wavelet nose reducton s based on the assumpton that the ampltude of the spectra of the sgnal s as dfferent as possble from that of nose. If a sgnal has ts energy concentrated n a small number of wavelet coeffcents, these coeffcents wll be relatvely large compared to the nose, whch has ts energy spread over a large number of coeffcents. Ths allows thresholdng of the ampltude of the coeffcents to separate sgnals or remove nose. The thresholdng s based on a value τ that s used to compare all the detal coeffcents. Approprate scale, as defned n secton 3.3, should be robust to the nose because the energy of true sgnal gets concentrated n a few coeffcents and the nose remans spread out n that scale. Hence the energy of the few coeffcents s much larger than that of noses and these large coeffcents can domnate the classfcaton or clusterng process. To verfy the clam, we wll compare the classfcaton and clusterng results wth and wthout nose-reducton n the experments. Donoho and Johnstone [7] gave the threshold as τ = σ n 2log(N); here σn s the standard varaton of nose, and N s the length of the tme seres. Because we don t know the σ n of the tme seres n advance, we estmate t by the robust medan estmaton of nose method descrbed n [7]. The robust medan estmaton s the medan absolute devaton of the detal wavelet coeffcents at scale one, dvded by The wdely used hard thresholdng algorthm s a process of settng the value of detal coeffcents whose absolute values are lower than the threshold to zero [6]. The hard thresholdng algorthm for features defned n (8) s descrbed n Eq. 9. { d T hre( d j ) = j, d j > τ 0, d j τ (9) 3.5. The Smlarty Strategy and Its Correspondng Algorthms For two seres wth the same length, ther correspondng approprate scales may not be equal. We can t compare the smlarty of two sets of approprate features drectly because the meanng of each data entry s dfferent. For example, consder a tme seres X = {x 1,x 2,x 3,x 4 } wth approprate scale 1 and a tme seres Y = {y 1,y 2,y 3,y 4 } wth approprate scale 2. The features of seres X are F 1 (X) = {a 0 1 ( X ),a 1 1 ( X ), d1 1( X ), d1 1( X ) } and the features of seres Y are F 2 (Y ) = {a 1 2 ( X ), d2 1( X ), d2 2( X ), d2 3( X ) }. Comparng detal coeffcents wth approxmaton coeffcents wll nduce errors and be meanngless. To avod ths problem, we merge the dstance of features wthn dfferent approprate scales. The dstance of two features s replaced by the average of dstance computed on two features wth dfferent approprate scales. Suppose a tme seres X wth approprate scale m and another tme seres Y wth approprate scale n, the dstance between the approprate features of X and Y, Dsc( F m (X), F n (Y )), s defned as (Dsc( F m (X), F m (Y )) + Dsc( F n (X), F n (Y )))/2. We can smply use dstancebased classfcaton and clusterng algorthms for the proposed smlarty strategy NN Classfcaton Algorthm Usng The Proposed Smlarty Strategy The classfcaton algorthm s mplemented by means of a 1-nearest neghbor algorthm (1-NN), whch we call WCANN (Wavelet Classfcaton Algorthm based on 1- Nearest Neghbor). Table 1 shows an outlne of the classfcaton algorthm. The nput s S = { S 1, S 2,...} conssts of a set of labeled tme seres data (the traned tme seres datasets) and X (a new emerged testng tme seres). The output s x c, the label of X. X wll be labeled as a member of a class f and only f, the dstance between X and one nstance of the class s smaller than between other nstances. The classfcaton algorthm for nose reduced coeffcents, WCANR (Wavelet Classfcaton Algorthm wth Nose Reducton), s smlar to the WCANN algorthm. The only dfference s that the nose wthn approprate features s reduced before classfcaton Herarchcal Clusterng Usng The Proposed Smlarty Strategy Clusterng s one of the commonly used data mnng tasks for dscoverng groups and dentfyng nterestng dstrbutons and patterns n the underlyng data. Clusterng s useful n ts own rght as a descrpton tool, and also as a subtask n more complex algorthms. Herarchcal clusterng groups data objects nto a tree of clusters whch s among the best known of clusterng methods [8]. The herarchcal clusterng algorthms can be dvded nto agglomeratve and dvsve clusterng, n terms of whether the herarchcal decomposton s 4 Journal of Advanced Computatonal Intellgence

5 Table 1. The WCANN algorthm Input: S, X For each tranng example S, calculatng ts approprate scale m and correspondng approprate features F m (S ); Gven a testng nstance X, calculatng ts approprate scale n and approprate features F n (X); best-so-far = nf; for = 1 to length(s) do Calculate Dsc( F m (S ), F m (X)); Calculate Dsc( F n (S ), F n (X)); Dsc( F m (S ), F n (x)) = (Dsc( F m (S ), F m (X)) + Dsc( F n (S ), F n (X)))/2 f Dsc( F m (S ), F n (x)) < best-so-far then ponter-to-best-seres k = ; best-so-far = Dsc( F m (S ), F n (X)); end f end for return x c = the label of k; formed n a bottom-up or top-down fashon. As the computaton of decomposton s normally smple for the agglomeratve procedures, we only use agglomeratve clusterng algorthm wth the proposed smlarty strategy. Gven a set of N tme seres to be clustered, we generate a N N dstance matrx conssted of the par-dstance between each of two tme seres usng the defned smlarty strategy. The basc process of agglomeratve herarchcal clusterng s descrbed as below: 1 Assgn each tme seres to ts own cluster. For the gven N tme seres, we now have N clusters, each contanng just one tme seres. Let the dstances between the clusters equal the dstances between the tme seres they contan. 2 Fnd the closest (most smlar) par of clusters and merge them nto a sngle cluster, so that now we have one less cluster. 3 Compute dstances between the new cluster and each of the old clusters. 4 Repeat steps 2 and 3 untl all tems are clustered nto a sngle cluster of sze N. 4. Expermental Evaluaton To show the effectveness of our approach, we performed experments on fve benchmark tme seres datasets. We compared the classfcaton accuracy of our feature extracton algorthm wth other feature extracton algorthms. All the feature extracton algorthms were compared wth the one-nearest-neghbor algorthm (1-NN), evaluated by leave-one-out cross valdaton. We also compared the clusterng result of usng features to that of usng orgnal tme seres. The sngle lnkage s used for the agglomeratve herarchcal clusterng Data Descrpton We used fve datasets (CBF, CC, Trance, Gun and Realtycheck) from the UCR Tme Seres Data Mnng Archve [18]. As we need the label nformaton, we only took the classfed datasets for experments. There are sx classfed datasets n the archve. The Auslan data s a multvarate dataset wth whch we can t apply our approach drectly. All the other fve datasets are useded n our experments. As Realtycheck data only has one nstance wthn each cluster that s too smple for classfcaton, we only used t for clusterng. The man features of the used data sets are descrbed as below. Cylnder-Bell-Funnel (CBF): Contans three types of tme seres: cylnder (c), bell (b) and funnel (f). It s an artfcal data orgnal proposed by [29]. Instances are generated usng the followng functons: where c(t) = (6 + η) χ [a,b] (t) + ε(t) b(t) = (6 + η) χ [a,b] (t a)/(b a) + ε(t) f (t) = (6 + η) χ [a,b] (b t)/(b a) + ε(t) { 0, f t < a t > b χ [a,b] = 1, f a t b η and ε(t) are drawn from a standard normal dstrbuton N(0,1), a s an nteger drawn unformly from [16,32] and b a s an nteger drawn unformly from the range [32,96]. UCR Archve provdes the source code for generatng the samples. We generated 128 examples for each class wth length 128. Control Chart Tme Seres (CC): Ths data set has 100 nstances for each of the sx dfferent classes of control charts. The dataset can also be downloaded from UCI KDD Archve [11]. Trace dataset (Trace): The 4-class dataset contans 200 nstances, 50 for each class. The dmensonalty of the data s 275. Gun Pont dataset (Gun): The dataset has two classes, each contanng 100 nstances. The dmensonalty of the data s 150. Realty Check dataset (Realtycheck): Ths data set s desgned as a smple realty check for new dstance measures. The data set conssts of data from Space Shuttle telemetry, Exchange Rates and artfcal sequences. The data s a normalzed data so that the mnmum value s zero and the maxmum s one. Journal of Advanced Computatonal Intellgence 5

6 Zhang, H. et al. Table 2. The error rates (%) produced by varous feature extracton methods wth 1-NN classfcaton algorthm for all four data sets Methods CBF CC Gun Trace WCANN WCANR WC-NN DFT-NN SVD-NN NN Classfcaton results Keogh and Kasetty compared 12 dstance measurements for tme seres usng 1-NN classfcaton algorthm and shown that Eucldean s the best among the compared dstance measurements [19]. Therefore we used only Eucldean dstance for 1-NN algorthm n our experments. An unclassfed nstance s assgned to the same class as ts closest match n the tranng set. The accuracy of the classfed results s measured by error rates. We compared sx algorthms usng dfferent features extracted from the CBF, CC, Trace and Gun data sets descrbed above. WCANN and WCANR are our proposed algorthms descrbed n secton 3.5. NN s the 1-nearest neghbor algorthm that uses the raw data [19]. WC-NN s the algorthm that uses the 1-nearest neghbor algorthm wth decomposed wavelet coeffcents havng varous dmensons [4]. DFT-NN s the 1-NN algorthm usng frst few Fourer coeffcents as features [27]. SVD-NN uses Sngular Value Decomposton (SVD) to extract features wth 1-NN algorthm [22]. As WC-NN, DFT-NN and SVD-NN don t gve the soluton for choosng feature dmensonalty, we take the average error rates wth all possble feature dmensons produced by these algorthms. Table 2 gves the error rates produced by varous feature extracton algorthms wth 1-NN classfcaton algorthm. The WCANN algorthm acheves the same classfcaton accuracy as the WCANR algorthm on all the four data sets. Nose reducton upon the extracted approprate features doesn t affect the classfcaton result,.e., the approprate features are robust aganst nose n terms of classfcaton. WCANN and WCANR take the same classfcaton accuracy wth NN algorthm on CBF data and hgher accuracy on other three datasets. WCANN and WCANR algorthm outperform WC-NN, DFT-NN and SVD-NN on classfcaton accuracy for all the datasets used Clusterng Herarchcal clusterng s a good way to compare smlarty measures wth representaton, snce a dendrogram of sze N summarzes O(N 2 ) dstance calculatons. We can easly judge whch representaton measure s better by vsually observng whch representaton appears to create the most natural groupngs of the data. Fg. 2 shows Fg. 2. The dendrogram generated by agglomeratve clusterng algorthm for orgnal Realtycheck data set, sngle lnkage s used Fg. 3. The dendrogram generated by agglomeratve clusterng algorthm wth the features of Realtycheck data set, sngle lnkage s used the dendrogram produced by agglomeratve herarchcal clusterng algorthm wth the orgnal tme seres. The extracted features and nose-reduced features produce the same clusterng results and the results are shown n Fg. 3. The extracted features are not affected by the nose durng the clusterng process for the used datasets. The extracted features are robust aganst nose n terms of clusterng. The subgroup contans nstances {11, 12} and the subgroup contans nstances {13, 14} are smlar. They are successfully clustered nto one subgroup wth our defned features shown n Fg. 3 and are not n the same subgroup wth the orgnal tme seres shown n Fg Journal of Advanced Computatonal Intellgence

7 5. Conclusons We explot the mult-scale property of Haar wavelet transform to extract features combnng the global nformaton and partal nformaton together. We propose a method for automatcally choosng the approprate features by the concentraton of the feature coeffcents. We propose a dstance strategy for the approprate features wth whch dstance-based classfcaton and clusterng algorthms can be easly appled. We conduct experments on several wdely used tme seres datasets and compare the classfcaton results of our algorthms and those of other four algorthms. Our algorthms outperform other four algorthms on classfcaton accuracy. We also compare the clusterng results produced by the approprate features and the orgnal tme seres. The approprate features can create more natural groupngs of the data. The expermental results show that the approprate features always produce the same classfcaton and clusterng results wth nose-reduced approprate features. It ndcates that several mportant coeffcents wthn the approprate features domnate the classfcaton and clusterng processes. The extracted approprate features are robust aganst nose n terms of classfcaton and clusterng. Acknowledgements The authors thank Prof. Keogh for provdng the expermental data. The authors thank two anonymous revews for suggestons that markedly clarfed and mproved the paper. References: [1] R. Agrawal, C. Faloutsos, and A. Swam, Effcent Smlarty Search n Sequence Databases, In Proceedngs of the 4th Conference on Foundatons of Data Organzaton and Algorthms, pp , October [2] C. S. Burrus, R. A. Gopnath, and H. Guo, Introducton to Wavelets and Wavelet Transforms, A Prmer, Prentce Hall, Englewood Clffs, NJ, [3] F. K. Chan, A. W. Fu, and T. Y. Clement, Harr Wavelets for Effcent Smlarty Search of Tme-Seres: wth and wthout Tme Warpng, IEEE Trans. on Knowledge and Data Engneerng, 15(3): , [4] K. P. Chan and A. W. Fu, Effcent Tme Seres Matchng by Wavelets, In Proceedngs of the 15th Internaton Conference on Data Engneerng, pp , March [5] R. R. Cofman and M. V. Wckerhauser, Entropy-Based Algorthms for Best Bass Selecton, IEEE Trans. on Informaton Theory, 38(2): , [6] D. L. Donoho, De-nosng by soft-thresholdng, IEEE Trans. on Informaton Theory, 41(3): , [7] D. L. Donoho and I. M. Johnson, Ideal spatal adaptaton va wavelet shrnkage, Bometrka, 81: , [8] R. Duda, P. Hart, and D. Stork, Pattern Classfcaton, John Wley & Sons, New York, second edton, [9] M. Gavrlov, D. Anguelov, and P. Indyk, Mnng the Stock Market: Whch Measure s Best?, In Proceedngs of The 6th ACM SIGKDD Internatonal Conference on Knowledge Dscovery and Data Mnng, pp , August [10] P. Geurts, Pattern Extracton for Tme Seres Classfcaton, In Proceedngs of the Prncples of Data Mnng and Knowledge Dscovery,5th European Conference, pp , September [11] S. Hettch and S. D. Bay. The UCI KDD Archve [12] T. B. Ho, T. D. Nguyen, S. Kawasak, S. Q. Le, D. D. Nguyen, H. Yoko, and K. Takabayash, Mnng Hepatts Data wth Temporal Abstracton, In Proceedngs of the 9th ACM Internatonal Conference on Knowledge Dscovery and Data Mnng, pp , August [13] D. Jang, C. Tang, and A. Zhang, Cluster Analyss for Gene Expresson Data: A Survey, IEEE Trans. on Knowledge and Data Engneerng, 16(11): , [14] X. Jn, Y. Lu, and C. Sh, Smlarty Measure Based on Partal Informaton of Tme Seres, In Proceedngs of The 8th ACM SIGKDD Internatonal Conference on Knowledge Dscovery and Data Mnng, pp , August [15] M. W. Kadous, Learnng Comprehensble Descrptons of Multvarate Tme Seres, In Proceedngs of the 6th Internatonal Conference on Machne Learnng, pp , September [16] T. Kalayc and O. Ozdamar, Wavelet Preprocessng for Automated Neural Network Detecton of EEG Spkes, IEEE Eng. n Medcne and Bology, 14: , [17] E. Keogh, K. Chakrabart, M. Pazzan, and S. Mehrotra, Dmensonalty Reducton of Fast Smlarty Search n Large Tme Seres Databases, Journal of Knowledge and Informaton System, 3: , [18] E. Keogh and T. Folas. The UCR Tme Seres Data Mnng Archve. eamonn/tsdma/ndex.html, [19] E. Keogh and S. Kasetty, On the Need for Tme Seres Data Mnng Benchmarks: A Survey and Emprcal Demonstraton, Data Mnng and Knowledge Dscovery, 7(4): , [20] E. Keogh, S. Lonard, and C. A. Ratanamahatana, Towards Parameter-Free Data Mnng, In Proceedngs of the 10th ACM SIGKDD Internatonal Conference on Knowledge Dscovery and Data Mnng, pp , August [21] E. Keogh and M. Pazzan, An Enhanced Representaton of Tme Seres whch Allows Fast and Accurate Classfcaton, Clusterng and Relevance Feedback, In Proceedngs of the 4th Internatonal Conference of Knowledge Dscovery and Data Mnng, pp , August [22] F. Korn, H. Jagadsh, and C. Faloutsos, Effcently Supportng ad hoc Queres n Large Datasets of Tme Sequences, In Proceedngs of The ACM SIGMOD Internatonal Conference on Management of Data, pp , May [23] S. Lawrence, A. Back, A. Tso, and C. L. Gles, The Gamma MLP Usng Multple Temporal Resolutons for Improved Classfcaton, In Proceedngs of the 1997 IEEE Workshop on Neural Networks for Sgnal Processng VII, pp , Pscataway, NJ, 1997, IEEE Press. [24] J. Ln, E. Keogh, S. Lonard, and B. Chu, A Symbolc Representaton of Tme Seres, wth Implcatons for Streamng Algorthms, In Proceedngs of the 8th ACM SIGMOD Workshop on Research Issues n Data Mnng and Knowledge Dscovery, pp. 2 11, June [25] S. Ptter and S. V. Kamarth, Feature Extracton From Wavelet Coeffcents for Pattern Recognton Tasks, IEEE Trans. on Pattern Recognton and Machne Intellgence, 21(1):83 85, January [26] I. Popvanov and R. J. Mller, Smlarty Search over Tme-Seres Data Usng Wavelets, In Proceedngs of The 18th Internatonal Conference on Data Engneerng, pp , Feburary [27] D. Rafe and A. Mendelzon, Effcent Retreval of Smlar Tme Sequences Usng DFT, In Proceedngs of the 5th Internatonal Conference on Foundatons of Data Organzatons, pp , [28] J. F. Roddch and M. Splopoulou, A Survey of Temporal Knowledge Dscovery Paradgms and Methods, IEEE Trans. on Knowledge and Data Engneerng, 14(4): , [29] N. Sato, Local Feature Extracton and Its Applcaton Usng a Lbrary of Bases, PhD thess, Department of Mathematcs, Yale Unversty, [30] C. Shahab, X. Tan, and W. Zhao, TSA-Tree: A Wavelet-Based Approach to Improve the Effcency of Mult-Level Surprse and Trend Queres on Tme-Seres Data, In Proceedngs of the12th Internatonal Conference on Scentfc and Statstcal Database Management, pp , July [31] Z. R. Struzk and A. Sebes, The Harr Wavelet Transform n the Tme Seres Smlarty Paradgm, In Proceedngs of the 3rd European Conference on Prncples of Data Mnng and Knowledge Dscovery, pp , September [32] I. N. Tansel, C. Mekdec, and C. McLaughln, Detecton of Tool Falure n End Mllng wth Wavelet Transformatons and Neural Networks (WT-NN), Internatonal Journal of Machne Tolls and Manufacture, 35(8): , [33] Y. Yamada, E. Suzuk, H. Yoko, and K. Takabayash, Decson- Tree Inducton from Tme-Seres Data Based on Standard-Example Splt Test, In Proceedngs of the 20th Internatonal Conference on Machne Learnng (ICML03), pp , August [34] B. K. Y and C. Faloustos, Fast Tme Sequence Indexng for arbtrary L p norms, In Proceedngs of the 26th Internatonal Conference on Very Large Databases, pp , September Journal of Advanced Computatonal Intellgence 7