Decision Support Systems

Transcription

1 Decision Support Systems 2011/2012 Week 6. Lecture 11

2

3 HELLO DATA MINING!

4 THE PLAN: MINING FREQUENT PATTERNS (Classes 11-13) Homework 5 CLUSTER ANALYSIS (Classes 14-16) Homework 6 SUPERVISED LEARNING (Classes 17-24) Homeworks 7 and 8 WRAP- UP (Class 25)

5 Today Mining frequent paserns Basic concepts (Sec. 5.1) Frequent itemset mining: The Apriori algorithm (Sec. 5.2)

6 Mining Frequent PaFerns

7 Overview and Basic Concepts

8 What Is Frequent PaFern Analysis? Frequent pasern: A pasern (set of items, subsequence, substructure, etc.) that occurs frequently in a data set Finding inherent regulariyes in data: What products were ozen purchased together? What are the subsequent purchases azer buying a PC? What kinds of DNA are sensiyve to this new drug? Can we automaycally classify web documents? ApplicaYons: Basket data analysis, cross- markeyng, catalog design, sale campaign analysis, Web log analysis, DNA sequence analysis, etc.

9 Why Does FP Mining MaFer? Frequent paserns are intrinsic properyes of datasets FoundaYon for many data mining tasks: AssociaYon, correlayon, and causality analysis ClassificaYon Cluster analysis Data warehousing SemanYc data compression Broad applicayons

10 Basic Concepts: Frequent Itemset Itemset: Set of one or more items k- itemset: Set with k items X = {x1,, xk} Support of X (aka absolute support, support count) : Number of occurrences of itemset X RelaYve support: FracYon of transacyons that contains X An itemset X is frequent if its support is above a minsup threshold Tid Items bought 10 Beer, Nuts, Diaper 20 Beer, Coffee, Diaper 30 Beer, Diaper, Eggs 40 Nuts, Eggs, Milk 50 Nuts, Coffee, Diaper, Eggs, Milk Itemset X = {Beer, Diaper} Support: 3 RelaYve support: 60%

11 Basic Concepts: AssociaQon Rules AssociaYon rule X Y Support: Probability that transacyon contains X Y (occurrences of X Y) Confidence: CondiYonal probability of Y given X (occurrences of X Y over occurrences of X) AN associayon rule is strong if it verifies minimum support & minimum confidence Tid Items bought 10 Beer, Nuts, Diaper 20 Beer, Coffee, Diaper 30 Beer, Diaper, Eggs 40 Nuts, Eggs, Milk 50 Nuts, Coffee, Diaper, Eggs, Milk Ass. rule Diaper Beer Support: 60% Confidence: 75% Ass. rule Beer Diaper Support: 60% Confidence: 100%

12 AssociaQon Rule Mining General approach: Find all frequent itemsets Generate strong associayon rules from frequent itemsets How many itemsets are we talking about?

13 Closed Itemsets Every pasern contains a number of sub- paserns Number of subpaserns exponenyal in length of original pasern E.g., PaSern a 1,, a 100 contains as many as sub- paserns! An itemset X is closed if X is frequent; and There is no super- itemset Y X with the same support as X. Closed itemsets provide lossless compression of frequent pasern data.

14 Maximal Frequent Itemsets An itemset X is a maximal frequent itemset if X is frequent; and There is no frequent super- itemset Y X. Unlike closed itemsets, maximal frequent itemsets are insufficient to recover all frequent pasern informayon (such as counts, etc.)

15 Closed Itemsets and Maximal Frequent Itemsets Example: Consider the dataset D = { a 1,, a 100, a 1,, a 50 } and suppose that minsup = 1. What is the set of closed itemsets? a 1,, a 100 : 1 a 1,, a 50 : 2 What is the set of maximal frequent itemsets? a 1,, a 100 : 1 What is the set of all itemsets? Lots and lots!!

16 Scalable FP Mining Methods

17 The Apriori Property (for Frequent PaFerns) The Apriori property of frequent paserns: Any (nonempty) subset of a frequent itemset must also be frequent Example: If {beer, diaper, nuts} is frequent, then so is {beer, diaper} Every transacyon having {beer, diaper, nuts} also contains {beer, diaper}

18 Efficient Algorithms for FP Mining Apriori algorithm FP- growth Mining FP using verycal data format

19 Apriori: Candidate GeneraQon & Test Apriori pruning principle: Supersets of infrequent itemsets should not be generated/tested Apriori method: 1. Scan dataset once to generate all 1- itemsets (L 1 ) 2. Test candidates against dataset 3. Generate length (k + 1) candidate itemsets from length k frequent itemsets (set L k+1 from L k ) 4. Test the candidates against dataset 5. Repeat 2 and 3 unyl no frequent or candidate sets can be generated

20 Example We want to compute frequent itemsets with minsup = 2 STEP 1: Generate all 1- itemsets (C 1 ) Itemset Count {I1} 6 {I2} 7 {I3} 6 {I4} 2 {I5} 2 TID List of itemids T100 {I1, I2, I5} T200 {I2, I4} T300 {I2, I3} T400 {I1, I2, I4} T500 {I1, I3} T600 {I2, I3} T700 {I1, I3} T800 {I1, I2, I3, I5} T900 {I1, I2, I3}

21 Example STEP 2: Eliminate all itemsets not verifying minsup (they all do: C 1 = L 1 ) Itemset Count {I1} 6 {I2} 7 {I3} 6 {I4} 2 {I5} 2 TID List of itemids T100 {I1, I2, I5} T200 {I2, I4} T300 {I2, I3} T400 {I1, I2, I4} T500 {I1, I3} T600 {I2, I3} T700 {I1, I3} T800 {I1, I2, I3, I5} T900 {I1, I2, I3}

22 Example STEP 3: Generate all 2- itemsets by joining L 1 with L 1 (C 2 = L 1 L 1 ) Itemset Count {I1, I2} 4 {I1, I3} 4 {I1, I4} 1 {I1, I5} 2 {I2, I3} 4 {I2, I4} 2 {I2, I5} 2 {I3, I4} 0 {I3, I5} 1 {I4, I5} 0 TID List of itemids T100 {I1, I2, I5} T200 {I2, I4} T300 {I2, I3} T400 {I1, I2, I4} T500 {I1, I3} T600 {I2, I3} T700 {I1, I3} T800 {I1, I2, I3, I5} T900 {I1, I2, I3}

23 Example STEP 4: Eliminate all itemsets not verifying minsup Itemset Count {I1, I2} 4 {I1, I3} 4 {I1, I4} 1 {I1, I5} 2 {I2, I3} 4 {I2, I4} 2 {I2, I5} 2 {I3, I4} 0 {I3, I5} 1 {I4, I5} 0 TID List of itemids T100 {I1, I2, I5} T200 {I2, I4} T300 {I2, I3} T400 {I1, I2, I4} T500 {I1, I3} T600 {I2, I3} T700 {I1, I3} T800 {I1, I2, I3, I5} T900 {I1, I2, I3}

24 Example STEP 4: Eliminate all itemsets not verifying minsup Itemset Count {I1, I2} 4 {I1, I3} 4 {I1, I5} 2 {I2, I3} 4 {I2, I4} 2 {I2, I5} 2 TID List of itemids T100 {I1, I2, I5} T200 {I2, I4} T300 {I2, I3} T400 {I1, I2, I4} T500 {I1, I3} T600 {I2, I3} T700 {I1, I3} T800 {I1, I2, I3, I5} T900 {I1, I2, I3}

25 Example STEP 3: Generate all 3- itemsets by joining L 2 with L 2 (C 3 = L 2 L 2 ) 3- itemsets containing 2- itemsets that don t verify minsup are not generated (apriori property) Itemset Count {I1, I2, I3} 2 {I1, I2, I5} 2 TID List of itemids T100 {I1, I2, I5} T200 {I2, I4} T300 {I2, I3} T400 {I1, I2, I4} T500 {I1, I3} T600 {I2, I3} T700 {I1, I3} T800 {I1, I2, I3, I5} T900 {I1, I2, I3}

26 Example (Details) C 3 = L 2 L 2 = {(I1, I2, I3), (I1, I2, I5), (I1, I3, I5), (I2, I3, I4), (I2, I3, I5), (I2, I4, I5)} Analyze each 3- itemset for infrequent 2- subitemsets: All subitemsets of (I1, I2, I3) are frequent All subitemsets of (I1, I2, I5) are frequent (I3, I5) is not frequent, so (I1, I3, I5) can be removed (I3, I4) is not frequent, so (I2, I3, I4) can be removed (I3, I5) is not frequent, so (I2, I3, I5) can be removed Finally, (I4, I5) is not frequent, so (I2, I4, I5) can be removed.

27 The Apriori Algorithm (Pseudo- Code) C k : Set of candidate k- itemsets L k : frequent k- itemsets 1. set k = 1; 2. while L k 3. set C k+1 = Π(L k L k ) SelecYon of itemsets using apriori property Join 4. for all t D, i C k+1 5. if i t, count(i) = count(i) end 7. set L k+1 = {i C k+1 : count(i) > minsup} 8. end Prune 9. return k L k

28 GeneraQng AssociaQon Rules AssociaYon rules can be generated from frequent itemsets Idea: For each itemset i, and sub- itemsets s i, check if s i s verifies minimum support and minimum confidence Example: Considering itemset (I1, I2, I5) and minconf = 70% I1 I5 I2 (conf.: 100%) I2 I5 I1 (conf.: 100%) I5 I2 I1 (conf.: 100%) TID List of itemids T100 {I1, I2, I5} T200 {I2, I4} T300 {I2, I3} T400 {I1, I2, I4} T500 {I1, I3} T600 {I2, I3} T700 {I1, I3} T800 {I1, I2, I3, I5} T900 {I1, I2, I3}

29 Improvements to Apriori Method Major computayonal challenges: MulYple scans of transacyon database Huge number of candidates Tedious workload of support counyng for candidates Improving Apriori: general ideas Reduce passes of transacyon database scans Shrink number of candidates Facilitate support counyng of candidates

30 Scan Database Only Twice Two- stage method: Stage 1: ParYYon dataset An itemset potenyally frequent in dataset must be frequent in at least one of the paryyon sets Stage 2: Consolidate global frequent paserns minsup k = σ k minsup Dataset DS1 DS2 DS3 DS4 Global frequent pa<erns Local frequent pa<erns

31 Reduce the Number of Candidates A k- itemset whose corresponding hashing bucket count is below the threshold cannot be frequent When generayng L1 from C1, generate all 2- itemsets Store itemsets in Hash table, updayng bucket counts as dataset is scanned to construct L1 Itemsets in buckets with small counts cannot be frequent paserns Address Count Itemsets 0 2 {I1, I4}, {I3, I5} 1 2 {I1, I5}, {I1, I5} 2 4 {I2, I3}, {I2, I3}, {I2, I3}, {I2, I3} 3 2 {I2, I4}, {I2, I4} 4 2 {I2, I5}, {I2, I5} 5 4 {I1, I2}, {I1, I2}, {I1, I2}, {I1, I2} 6 4 {I1, I3}, {I1, I3}, {I1, I3}, {I1, I3}

32 Sampling for Frequent PaFerns Select a sample of original database Mine frequent paserns within sample using Apriori Less data requires smaller minsup Scan database once to verify frequent itemsets found in sample Scan database again to find missed frequent itemsets

33 Finally

34 Summary Basic concepts: Frequent itemsets AssociaYon rules Closed itemsets Maximal frequent itemsets Support and confidence Frequent itemset mining: The Apriori algorithm Improving performance of Apriori

35 Next Class FP- Growth