描述性資料採礦 Descriptive Data Mining

Transcription

1 描述性資料採礦 Descriptive Data Mining 李御璽 (Yue-Shi Lee) 銘傳大學資訊工程學系

2 Agenda Cluster Analysis ( 集群分析 ) 找出資料間的內部結構 Association Rules ( 關聯規則 ) 找出那些事件常常一起出現 Sequence Clustering ( 時序群集 ) Clustering + Sequence Mining 找出那些事件常常循序出現

3 Cluster Analysis

4 Purpose In cluster analysis, there is no pre-classified data and no distinction between dependent and independent data Instead, cluster analysis is a process to group similar objects together All members in one cluster are similar to each other and different from the members of other clusters, according to some similarity metric 同一集群內成員的相似性要愈高愈好, 而不同集群間成員的相異性則要愈高愈好

5 Cluster Analysis: Customer Segmentation Y (Age) Insurance ( 保險 ) Cluster Principal Guaranteed Investment ( 保本型投資 ) Customer (Object) Variables High Profit and Risky Investment ( 高獲利高風險投資 ) X (Income)

6

7

8

9

10

11 Categorization of Cluster Analysis Exclusive vs. Non-Exclusive (Overlapping) SQL Server EM Hierarchical Methods vs. Partitioning Methods Hierarchical Methods: SPSS, SAS Single Link Method Average Link Method Complete Link Method Partitioning Methods K-Means Methods: SQL Server 2012, SPSS Modeler, SAS EM K-Medoids Methods (PAM, ) Kohonen Self-Organizing Maps (SOM): SPSS Modeler, SAS EM Two Step: SPSS Modeler

12 K-Means Methods

13 K-Means Methods

14 K-Means Methods

15 K-Means Methods 7.0 Advantage Fast Disadvantages Sensitive to Outliers Not an Optimal Solution Can just handle numerical data

16 Categorization of Cluster Analysis Hard (Exclusive) Clustering Objects are assigned to a single cluster K-Means Soft (Non-Exclusive) Clustering Objects have probability distribution over clusters EM (Soft Version of K-Means)

17 EM Algorithm 隨機設置群中心 Initialization 決定每個 Object 屬於那一個群中心機率是多少 Estimation Step Maximization Step 重新計算群中心及其變異數 Convergence 直到群中心不再變動

18 K-Means (A Special Kind of EM) Initialization Randomly select cluster centers E Step Euclidean Distance to Each Cluster Center The probability is either 0 or 1 M Step Recalculate cluster centers Convergence Cluster centers do not change

19 EM Initialization Randomly select cluster centers and variances E Step Gaussian Distribution for Each Cluster The probability is between 0 and 1 M Step Recalculate cluster centers and variances Convergence Cluster centers do not change

20 Another Example

21 Microsoft Clustering Algorithm Parameters CLUSTER_COUNT 要建立之群集的數目 CLUSTERING_METHOD 1 可擴充 EM 2 不可擴充 EM 3 可擴充 K-Means 4 不可擴充 K-Means MINIMUM_SUPPORT 每個群集中的最小案例數目

22 Association Rules

23 Rakesh Agrawal 被譽為資料採礦之父的 Rakesh Agrawal 已離開 IBM, 成為 Microsoft 搜尋實驗室的技術人員 經歷 Microsoft Technical Fellow IBM Fellow-IBM 最高級研究員頭銜 IBM Intelligent Miner 的開發者 許多資料採礦演算法的設計者 Apriori (Association Rules) ArioriAll (Sequential Patterns)

24 What Is Association Mining? Association Rule Mining Finding associations among itemsets in transaction databases Applications Market basket analysis (marketing strategy: items to put on sale at reduced prices), cross-marketing, catalog design, shelf space layout design, etc. Examples Rule form: Body Head [Support, Confidence] buys(x, Computer ) buys(x, Software ) [2%, 60%] major(x, CS ) ^ takes(x, DB ) grade(x, A ) [1%, 75%]

25 Rule Measures: Support and Confidence Transaction ID Items Bought 1000 A,B,C 2000 A,C 3000 A,D 4000 B,E,F For rule A C Support = Support({A, C}) = 50% Confidence = Support({A, C})/Support({A}) = 66.6%

26 Market Basket Analysis Typically, association rules are considered interesting if they satisfy both a minimum support threshold and a minimum confidence threshold

27 Rule Explain: Support and Confidence

28 Minimum Support and Confidence Let minimum support 50%, and minimum confidence 50%, we have A C [50%, 66.6%] C A [50%, 100%] Transaction ID Items Bought 1000 A,B,C 2000 A,C 3000 A,D 4000 B,E,F

29 Analysis The number of itemsets of a given cardinality tends to grow exponentially

30 Mining Association Rules: Apriori Principle Transaction ID Items Bought 1000 A,B,C 2000 A,C 3000 A,D 4000 B,E,F For rule A C: support = support({a,c}) = 50% Min. support 50% Min. confidence 50% Frequent Itemset Support {A} 75% {B} 50% {C} 50% {A,C} 50% confidence = support({a,c})/support({a}) = 66.6% The Apriori principle: Any subset of a frequent itemset must be frequent

31 Mining Frequent Itemsets: The Key Step Find the frequent itemsets: the sets of items that have minimum support Use the frequent itemsets to generate association rules

32 Example

33 Apriori Algorithm

34 Apriori Algorithm

35 Apriori Algorithm

36 Interestingness Measurements Subjective Measures A rule (pattern) is interesting if it is unexpected (surprising to the user) it is actionable (the user can do something with it) Objective Measures Two popular measurements: Support Confidence

37 Criticism to Support and Confidence Example 1 Among 5000 students 3000 play basketball 3750 eat cereal 2000 both play basket ball and eat cereal play basketball eat cereal [40%, 66.7%] is misleading because the overall percentage of students eating cereal is 75% which is higher than 66.7% basketball not basketball sum (row) cereal not cereal sum (col.)

38 Criticism to Support and Confidence Example 2 X and Y: positively correlated, X and Z, negatively related Support and Confidence of X Z dominates We need a measure of correlated events corr A, B P( A B) P( A) P( B) Rule Support Confidence X=>Y 25% 50% X=>Z 37.50% 75% X Y Z

39 Criticism to Support and Confidence Correlation Taking both P(A) and P(B) in consideration P(A^B)=P(B)*P(A), if A and B are independent events A and B negatively correlated, if the value is less than 1 otherwise A and B positively correlated corr A, B P( A B) P( A) P( B) X Y Z Itemset Support Interest X,Y 25% 2 X,Z 37.50% 0.9 Y,Z 12.50% 0.57

40 Microsoft Association Rule Using PROBABILITY instead of Confidence Itemset IMPORTANCE is equal to Corr A,B Association Rule Importance is equal to P(B A) IMPORTANCEA B log( ) P(B not A) Algorithm Parameters MINIMUM_SUPPORT MAXIMUM_SUPPORT 刪除 Itemset 的 SUPPORT 高於此值者 MAXIMUM_ITEMSET_SIZE MINIMUM_PROBABILITY (MINIMUM_CONFEDENCE) MINIMUM_IMPORTANCE >0: Positively Correlated <0: Negatively Correlated =0: Independent

41 Itemsets 各產品的組合情況

42 Association Rules Touring-3000, Water Bottle Road Bottle Cage

43 Dependency Network 主要目的在於了解各產品間的關聯性

44 Model Prediction

45 Model Prediction

46 Sequence Clustering

47 Traditional Sequential Pattern Mining Microsoft Sequence Clustering algorithm is a hybrid of sequence techniques and clustering techniques

48 Traditional Sequential Pattern Mining Given Transaction Time, Customer Id, Items Bought Original Database Answer Set

49 Traditional Sequential Pattern Mining Customer-Sequence Version of the Database

50 Traditional Sequential Pattern Mining The sequential patterns in the answer set need not be consecutive in customer sequences Elements of a sequential pattern need not be simple items They can be itemsets Answer Set

51 Traditional Sequential Pattern Mining Disadvantages of Traditional Sequential Pattern Mining The candidate combinations are huge The mining time is very long It does not group customers into business segments

52 Microsoft Sequence Clustering Only consider simple item and the relationships between two items The candidate combinations are decreased The mining time is fast Group customers into clusters using the clustering algorithm and then calculate the probabilities between two items for each cluster

53 Microsoft Sequence Clustering EM Algorithm Group1: 1, 3 Group2: 2, 4

54 Microsoft Sequence Clustering 30 has 20% probability to 40 Group1: 1, 3 1/2 30 1/5 2/5 40 2/5 70 1/2 2/2 90 Group2: 2, 4 1/2 10 1/5 30 2/5 40 1/2 2/5 2/ has 50% probability to 40 1st Order Markov Chain

55 Microsoft Sequence Clustering Algorithm Parameters Cluster_Count It defines the number of clusters a model contains Setting this value to 0 will cause the algorithm to automatically choose the best number of clusters MINIMUM_SUPPORT It specifies the minimum number of cases in each cluster to avoid having clusters with too few cases

56 Clustering Results 共集出 15 群, 顏色越深代表越大群

57 State Transaction 主要目的在於了解各產品間的前後關係及機率

58 Model Prediction

59 Model Prediction

60