Clustering. Cluster Analysis 群聚分析. The K-Means Clustering Method. Clustering 的一般應用. Example. Pattern Recognition 圖樣識別. Cluster 群聚 : 一群 data objects

Transcription

1 The K-Means Clusterng Method Eample Clusterng Assgn each obects to most smlar center reassgn Update the cluster means reassgn K= Arbtrarly choose K obect as ntal cluster center Update the cluster means Cluster Analyss 群聚分析 Clusterng 的一般應用 Cluster 群聚 : 一群 data obects Pattern Recognton 圖樣識別 在同一群內相當相似 在不同群內非常不相似 Cluster analyss 把資料依相似性分群 Clusterng 是 unsupervsed classfcaton: 無預先設好的類別標籤 Spatal Data Analyss 空間資料分析 create thematc maps n GIS by clusterng feature spaces detect spatal clusters and eplan them n spatal data mnng Image Processng 影像處理 Economc Scence (especally market research) Typcal applcatons 作為了解資料分佈的工具 (stand-alone tool) 作為其他方法的 preprocessng step WWW Document classfcaton Cluster Weblog data to dscover groups of smlar access patterns

2 Eamples of Clusterng Applcatons 何謂 Good Clusterng? Marketng: Help marketers dscover dstnct groups n ther customer bases, and then use ths knowledge to develop targeted marketng programs Land use: Identfcaton of areas of smlar land use n an earth observaton database Insurance: Identfyng groups of motor nsurance polcy holders wth a hgh average clam cost Cty-plannng: Identfyng groups of houses accordng to ther house type, value, and geographcal locaton Earth-quake studes: Observed earth quake epcenters should be clustered along contnent faults 好的分群方法產生高品質的 clusters hgh ntra-class smlarty (cluster 內 : 高相似 ) low nter-class smlarty (cluster 間 : 低相似 ) 結果的品質決定於 clusterng 方法的 smlarty measure mplementaton clusterng 方法的品質也可以用 找出 ( 部分或全部 ) 隱藏的 pattern 能力 來度量 Clusterng 需求 擴充性 (scalablty ) 處理各種型態的屬性 (types of attrbutes) 找出任意形狀的 cluster 決定輸入參數時需盡量減少所需的 doman knowledge 處理 nose 及 outler 的能力 對輸入資料的順序要 nsenstve hgh dmensonalty 可以整合 user-specfed constrants Interpretablty usablty Data matr (two modes) n ob. * p var. 資料結構 Dssmlarty matr (one mode) n * n n d(,) d(, ) : d ( n,) f f nf d (,) : d ( n,) : p p np

3 Clusterng Qualty 的度量 Dssmlarty/Smlarty metrc: 以 dstance functon 表示,d(, ) Dstance functons 的定義依照變數型態而不同 nterval-scaled, boolean, categorcal, ordnal and rato varables 依照各個應用與資料的意義訂定變數的 weghts 有時很難定義 smlar enough or good enough 答案很主觀 Clusterng Analyss 的資料型態 Interval-scaled varables weght, heght, lattude, (roughly lnear) Bnary varables symmetrc: gender asymmetrc: fever (Y/N), test (P/N) Nomnal, ordnal, and rato varables map_color, weather; orderng; Ae Bt Varables of med types Interval-valued varables Smlarty and Dssmlarty Between Obects Standardze data 先標準化 ( f 變成 z f ) 算 mean absolute devaton Dstances: 度量兩 data obects 的 smlarty 或 dssmlarty where s = ( f n n m + f f f f nf f m = ( f f f nf 算 standardzed measurement (z-score) m f f z = f s 用 mean absolute devaton 比用 standard devaton 更 robust standard devaton 對差值平方 f m + + ). m ) m f : 平均 propertes d(,) d(,) = d(,) = d(,) d(,) d(,k) + d(k,) Manhattan dstance Eucldean dstance d(, ) = p p d(, ) = ( p p )

4 Smlarty and Dssmlarty (Cont.) Bnary Varables Mnkowsk dstance: q q q d (, ) = q ( ) p p = (,,, p ) and = (,,, p ) 是兩個 p- dmensonal 的 data obects, q 是正整數 A contngency table for bnary data 各種可能 Obect Obect sum a b a+ b c d c+ d sum a+ c b+ d p Manhattan dstance: q = Eucldean dstance: q = Smple matchng coeffcent (nvarant, f the bnary varable s symmetrc): d (, ) = b + c a + b + c + d Jaccard coeffcent (nonnvarant f the bnary varable s asymmetrc): d (, ) = b c a + + b + c Dssmlarty between Bnary Varables Nomnal Varables Eample gender 是 symmetrc attrbute 其他是 asymmetrc bnary attrbute 讓 Y 跟 P 為, N 為 Name Gender Fever Cough Test- Test- Test- Test- Jack M Y N P N N N Mary F Y N P N P N Jm M Y P N N N N + d ( ack, mary ) = = d ( ack, m ) = = d ( m, mary ) = =. + + bnary varable 的 generalzaton : 超過兩種狀態, 如 red, yellow, blue, green Method : Smple matchng m: # of matches, p: total # of varables Method : use a large number of bnary varables creatng a new bnary varable for each of the M nomnal states Show an eample d (, ) = p p m

5 Ordnal Varables Rato-Scaled Varables Can be dscrete or contnuous Order s mportant, e.g., rank Can be treated lke nterval-scaled 將 f 用他的 rank 替代, r f {,, M f } 將各個 ordnal varable 對應到 [, ] 取代 -th obect 的 f-th 變數 r f z = f M f Rato-scaled varable: a postve measurement on a nonlnear scale, appromately at eponental scale, such as Ae Bt or Ae -Bt Methods: treat them lke nterval-scaled varables not a good choce! (why? the scale can be dstorted) apply logarthmc transformaton, (log-log maybe) 用 nterval-scaled varables 的方法計算 dssmlarty y f = log( f ) Eample treat them as contnuous ordnal data as ordnal data, treat ther rank as nterval-scaled Varables of Med Types Maor Clusterng Approaches 資料庫可能同時包含這六種 varables symmetrc bnary, asymmetrc bnary, nomnal, ordnal, nterval and rato 可以用 weghted formula 來結合 Σ d (, ) = ( f ) ( f ) = p ( f ) δ f = 當 f 是 bnary or nomnal: d (f) = f f = f, otherwse d (f) = 當 f 是 nterval-based: d (f) = f - f /ma( f )-mn( f ) 當 f 是 ordnal or rato-scaled 算 ranks r f 把 z f 當 nterval-scaled p f Σ δ d z f = r f M f δ (f) = : () 缺 f 或 f () f = f =, f asymmetrc 其他 : δ (f) = Parttonng algorthms Construct varous parttons evaluatethem by some crteron Herarchyalgorthms Create a herarchcal decomposton of the set of data (or obects) usng some crteron Densty-based: based on connectvty and densty functons Grd-based: based on a multple-level granularty structure Model-based: A model s hypotheszed for each of the clusters and the dea s to fnd the best ft of that model to each other

6 Parttonng Algorthms: Basc Concept The K-Means Clusterng Method Parttonng method: Construct a partton of a database D of n obects nto a set of k clusters Gven k, the k-means algorthm s mplemented n four steps: Gven a k, fnd a partton of k clusters that optmzes the chosen parttonng crteron Global optmal: ehaustvely enumerate all parttons Heurstc methods: k-means and k-medods algorthms Partton obects nto k nonempty subsets Compute seed ponts as the centrods of the clusters of the current partton (the centrod s the center,.e., mean pont, of the cluster) k-means (MacQueen ): Each cluster s represented by the center of the cluster k-medods or PAM (Partton around medods) (Kaufman & Rousseeuw ): Each cluster s represented by one of the obects n the cluster Assgn each obect to the cluster wth the nearest seed pont Go back to Step, stop when no more new assgnment The K-Means Clusterng Method Comments on the K-Means Method Eample Strength: Relatvely effcent: O(tkn), where n s # obects, k s # clusters, and t s # teratons. Normally, k, t << n. K= Arbtrarly choose K obect as ntal cluster center Assgn each obects to most smlar center reassgn Update the cluster means Update the cluster means reassgn Comparng: PAM: O(k(n-k) ), CLARA: O(ks + k(n-k)) Comment: Often termnates at a local optmum. Weakness Applcable only when mean s defned, then what about categorcal data? Need to specfy k, the number of clusters, n advance Unable to handle nosy data and outlers Not sutableto dscover clusters wth non-conve shapes

7 K-Means 的問題 The k-means algorthm s senstve to outlers! Snce an obect wth an etremely large value may substantally dstort the dstrbuton of the data. K-Medods: Instead of takng the mean value of the obect n a cluster as a reference pont, medods can be used, whch s the most centrally located obect n a cluster. Herarchcal Clusterng Use dstance matr as clusterng crtera. Ths method does not requre the number of clusters k as an nput, but needs a termnaton condton Step Step Step Step Step PAM (Parttonng Around Medods, ) Step Step Step Step Step a b c d e a b d e c d e a b c d e agglomeratve (AGNES) dvsve (DIANA) AGNES (Agglomeratve Nestng) Implemented n Splus (e.g) Use Sngle-Lnkage method and the dssmlarty matr Merge nodes that have the least dssmlarty Go on n a non-descendng fashon Eventually all nodes belong to the same cluster Sngle-lnkage: cloest par Complete-lnkage: dstant Dendrogram Shows Herarchcal Clusterng dendrogram: 將 data obects decompose 為數層 nested parttonng (tree of clusters) clusters

8 DIANA (Dvsve Analyss) Implemented n statstcal analyss packages, e.g., Splus Inverse order of AGNES Eventually each node forms a cluster on ts own Densty-Based Clusterng Methods Clusterng based on densty (local cluster crteron), such as densty-connected ponts Maor features: Dscover clusters of arbtrary shape Handle nose One scan Need densty parameters as termnaton condton Several nterestng studes: DBSCAN: Ester, et al. (KDD ) OPTICS: Ankerst, et al (SIGMOD ). DENCLUE: Hnneburg & D. Kem (KDD ) CLIQUE: Agrawal, et al. (SIGMOD ) Model-Based Clusterng Methods Attempt to optmze the ft between the data and some mathematcal model Model-Based Clusterng Methods Statstcal approach Conceptual clusterng COBWEB(Fsher ) AI approach a prototype for each cluster (called eemplar) put new ob. to the most smlar eemplar Neural network approach Self-Organzaton feature Map (SOM) several unts competng for the current obect

9 Self-organzng feature maps (SOMs) What Is Outler Dscovery? Clusterng s also performed by havng several unts competng for the current obect The unt whose weght vector s closest to the current obect wns The wnner and ts neghbors learn by havng ther weghts adusted Useful for vsualzng hgh-dmensonal data n - or -D space Eample Tool 何謂 outlers? Mchael Jordon CEO 薪水 age = 那些跟其他資料相當不相似的資料 (consderably dssmlar!) Problem:fnd top k outlers among n obects Applcatons: Credt card/ Telecom fraud detecton Customer segmentaton Medcal analyss Approaches Statstcal-based Dstance-based Devaton-based Outler Dscovery: Statstcal Approaches Dstance-Based Approach 參數 : p ( 為一分數 ), D Dstance-based outler DB(p, D)-outler: dataset S 中的 obect O, S 中至少 p 的 obect 跟 O 的距離大於 D 沒有夠多的鄰居 Assume a model underlyng dstrbuton that generates data set (e.g. normal dstrbuton) Use dscordancy tests dependng on data dstrbuton dstrbuton parameter (e.g., mean, varance) number of epected outlers Drawbacks most tests are for sngle attrbute In many cases, data dstrbuton may not be known dstance-based outler mnng algorthms Inde-based algorthm Nested-loop algorthm Cell-based algorthm

10 Summary Constrant-Based Clusterng Cluster analyss groups obects based on ther smlarty cluster analyss has wde applcatons Measure of smlarty can be computed for varous types of data Clusterng algorthms can be categorzed nto parttonng methods herarchcal methods densty-based methods grd-based methods model-based methods Outler detecton and analyss useful for fraud detecton, etc. performed by statstcal, dstance-based or devaton-based approaches research ssues: constrant-based clusterng ATM allocaton problem