Product presentations can be more intelligently planned

Size: px

Start display at page:

Download "Product presentations can be more intelligently planned"

Garry Knight
5 years ago
Views:

1 Association Rules Lecture /DMBI/IKI8303T/MTI/UI Yudho Giri Sucahyo, Ph.D, CISA Faculty of Computer Science, Objectives Introduction What is Association Mining? Mining Association Rules Al ith ffor A Algorithms Association i ti R Rules l Mi Mining i Visualization ` ` ` ` `

2 Introduction You sell more if customers can see the product. Customers that purchase one type of product are likely to be interested in other particular products. Market-basket analysis studying the composition of shopping basket of products purchased during a single shopping event. Market-basket data the transactional list of purchases by customer. It is challenging, because Very large number of records (often millions of trans/day) Sparseness (each market-basket contains only a small portion of items carried) Heterogeneity (those with different tastes tend to purchase a specific subset of items). Introduction () Product presentations can be more intelligently planned for specific times a day, days of the week, or holidays. Can also involve sequential relationships. Market-basket analysis is an undirected (along with clustering) DM operation, seeking patterns that were previously unknown. Cross-selling The propensity for the purchaser of a specific item to purchase a different item Can be maximized by locating those products that tend to be purchased by the same consumer in places where both products can be seen.

3 What is Association Mining? Association rule mining (ARM): Finding frequent patterns, association, correlation, or causal structures among sets of items or objects in transaction databases, relational databases, and other information repositories. Frequent pattern: pattern that occurs frequently in a database. Motivation: finding regularities in data What products were often purchased together? Beer and diapers?! What are the subsequent purchases after buying a PC? What kinds of DNA are sensitive to this new drug? Can we automatically classify web documents? 5 Why is Frequent Pattern or Association Mining an Essential Task in DM? Foundation for many essential data mining tasks Association, correlation, causality Sequential patterns, temporal or cyclic association, partial periodicity, spatial and multimedia association Associative classification, cluster analysis, iceberg cube, fascicles (semantic data compression) Broad applications Basket data analysis, cross-marketing, catalog design, sale campaign analysis Web log (click stream) analysis, DNA sequence analysis, etc. 6

4 What is Association Mining? Examples: Rule form: A B [support, confidence]. buys(x, diapers ) buys(x, beers ) [0.5%, 60%] major(x, CS ) ^ takes(x, DB ) grade(x, A ) [%, 75%] A support of 0.5% for Assoc Rule means that 0.5% of all the transaction show that diapers and beers are purchased together. A confidence of 60% means that 60% of the customers who purchased diapers also bought beers. Rules that satisfy both minimum support and minimum confidence threshold are called strong. 7 What is Association Mining? A set of items is referred to as an itemset. An itemset that contains k items is a k-itemset. {beer, diaper} is a -itemset. If an itemset satisfies minimum support, then it is a frequent itemset. The set of frequent k-itemsets is commonly denoted by L k. ARM is a two-step process: Find all frequent itemsets Generate strong AR from the frequent itemsets. The second step is the easiest of the two. Overall performance of mining i AR is determined d by the first step. 8

5 Association Mining mining association rules (Agrawal et. al SIGMOD93) Better algorithms Problem extension Fast algorithm (Agrawal et. al VLDB9) Hash-based (Park et. al SIGMOD95) Partitioning (Navathe et. al VLDB95) Direct et Counting (Brin et. al SIGMOD97) Parallel mining (Agrawal et. al TKDE96) 9 Generalized A.R. (Srikant et. al; Han et. al. VLDB95) Quantitative A.R. (Srikant et. al SIGMOD96) Distributed mining Incremental mining (Cheung et. al PDIS96) (Cheung et. al ICDE96) N-dimensional A.R. (Lu et. al DMKD 98) Meta-ruleguided mining Many Kinds of Association Rules Boolean association rule: If a rule concerns associations between the presence or absence of items. Example: buys(x, diapers ) buys(x, beers ) [0.5%, 60%] (R) Quantitative association rule: Describes associations between quantitative items. Quantitative values for items are partitioned into intervals. Example: age(x, ) income(x, K-8K ) buys(x, LCD TV ) (R) Age and income have been discretized. Single-dimensional association rule R Multidimensional association rule R 0

6 Many Kinds of Association Rules Single-level association rule Example: age(x ) age(x, 39 buys(x, laptop computer ) Multilevel association rule Example: age(x, ) g( buys(x, computer ) y( Computer is a higher-level abstraction of laptop Various Extensions Mining maximal frequent patterns. If p is a maximal frequent pattern, then any superpattern of p is not frequent. Used to substantially reduce the number of frequent itemsts generated in mining. Mining Single-Dimensional Boolean Assocation Rules Given A database of customer transactions Each transaction is a list of items (purchased by a customer in a visit) Find all rules that correlate the presence of one set of items with that of another set of items Example: 98% of people who purchase tires and auto accessories also get automotive services done Any number of items in the consequent/antecedent of rule Possible to specify constraints on rules (e.g., find only rules involving Home Laundry Appliances).

Application Examples Market-basket Analysis * Fanta -- what the store should do to boost Fanta sales Bodrex * -- what other products should the store stocks up on if the store

X & Y Z with minimum confidence and support support, s, probability that a transaction contains {X, Y, Z} confidence, c, conditional probability that a transaction having {X,

7 Application Examples Market-basket Analysis * Fanta -- what the store should do to boost Fanta sales Bodrex * -- what other products should the store stocks up on if the store has a sale on Bodrex Attached mailing in direct marketing 3 Rule Measures: Support and Confidence Customer buys both Customer buys beer Customer buys diaper Find all the rules X & Y Z with minimum confidence and support support, s, probability that a transaction contains {X, Y, Z} confidence, c, conditional probability that a transaction having {X, Y} also contains Z. Transaction ID Bought 000 ABC A,B,C 000 A,C 000 A,D 5000 B,E,F Let minimum support 50%, and minimum confidence 50%, we have A C (50%, 66.6%) C A (50%, 00%)

8 Mining Association Rules -- Example Transaction ID Bought 000 A,B,C 000 A,C 000 A,D 5000 B,E,F Min. support 50% Min. confidence 50% For rule A C: support = support({a, C}) = 50% confidence = support({a, C})/support({A}) = 66.6% The Apriori principle: Any subset of a frequent itemset must be frequent. 5 Frequent et Support {A} 75% {B} 50% {C} 50% {A,C} 50% Mining Frequent ets: the Key Step Find the frequent itemsets: the sets of items that have minimum support A subset of a frequent itemset must also be a frequent itemset, ie i.e., if {AB} is a frequent itemset, both {A} and {B} should be a frequent itemset Iteratively find frequent itemsets with cardinality from to k (k-itemset) Use the frequent itemsets to generate association rules. 6

9 The Apriori Algorithm C k : Candidate itemset of size k L k : frequent itemset of size k L = {frequent items}; for (k = ; L k!= ; k++) do begin C k+ = candidates generated from L k ; for each transaction t in database do increment the count of all candidates in C k+ that are contained in t L k+ = candidates in C k+ with min_support end return k L k ; 7 The Apriori Algorithm Example Database D itemset sup. L itemset sup. TID C {} {} 00 3 {} 3 Scan D {} {3} 3 {3} {} {5} {5} 3 C itemset sup C itemset L itemset sup { } Scan D { } { 3} { 3} { 3} { 3} { 5} { 5} { 3} { 3} { 5} 3 { 5} 3 { 5} {3 5} {3 5} {3 5} 8 itemset S D itemset sup { 3 5} { 3 5} C 3 Scan D L 3

10 The Apriori Algorithm Example Tid Sample Database 9 et Tid Support,3,,5 80% () 3,,,5 80% (),,3,,5 00% (5) 5,,3 60% (3) 7,3 0% () 7,3 0% () 5 7,3 0% () 5 7,3 0%/ 5,,3 60% (3) 3 5, 0% () 3 5, 0% () 5,3 0% () 5,3 0% (),3,,5 80% () 3,,5 60% (3) 3,,5 60% (3) 3,,,5 80% () Frequent Patterns with MinSupport = 0% Rule Support Confidence 7 5 0% () 00% 7 0% () 00% 5 60% (3) 00% 3 60% (3) 75% 3 60% (3) 75% 3 80% () 00% 3 80% () 80% 80% () 00% 80% () 80% 5 7 0% () 00% 7 5 0% () 00% 3 5 0% () 00% 5 0% () 00% 3 60% (3) 00% 3 60% (3) 75% 3 60% (3) 75% Association Rules with MinSupport = 0% and MinConf = 70% The Apriori Algorithm Example 0 C Pattern Support Scan D for 3 Prune L count of each 5 infrequent Patt ern 5 3 candidate 6 patterns Generate C Candidates From L Generate C 3 Candidates From L C Pattern C 3 Pattern Scan D for count of each candidate Scan D for count of each candidate C Pattern Support C 3 Pattern Support Support Prune infrequent patterns Prune infrequent patterns L Pattern Support L 3 Pattern Support

11 Major Drawbacks of Apriori Apriori has to read the database many times to test the support of candidate patterns. In order to find a frequent pattern X with length 50, it has to traverse the database 50 times. On dense datasets with long patterns, as the length of the pattern increases, the performance of Apriori drops rapidly due to the explosion of the candidate patterns. TreeProjection Algorithm (Agarwal, Aggarwal & Prasad 000) null {753, 53, 75, 3, 3} Level 0 7 {5, 5} 5 {3, 3, } {3,, {,, 3 3, 3},} Level Level Level 3 Lexicographical Tree and Triangular Matrix for Counting Frequent Patterns with Length Two

12 Eclat Algorithm (Zaki et al. 997) {} 35 tid list intersection tid list intersection FP-Growth (Han, Pei & Yin 000) HeaderTable Item Head of node links Root :5 3: : 5: :3 5: 7: 5: 7: FP-Tree for the sample database

13 CT-PRO (Sucahyo & Gopalan 00) Tid (a) Frequent Tid (b) Mapped ItemTable I I C n t o d e x e u m n t P S T Level Level Level Level Level (c) Global CFP-Tree 5 Mining Very Large Database Partition Algorithm (Savasere, Omiecinski & Navathe 995) Tid FP : {{},{,},{,5},{,,5}, Scan P and P to find local {3}, {3,}, {3,,5}, {3,5}, frequent patterns {}, {,5}, {5} {,7}, {7} {,5,7}, {57} {5}, {5,7}, {7}} FP : {{},{,3},{,},{,3,} {3},{3,},{},. {8},{0} C: {{},{,3},{,},{,5}, {,3,},{,,5}, {3}, {3,}, {3,,5}, {3 {3,5}, {}, {,5}, {,7}, {,5,7}, {5}, {5,7}, {7}. {8},{0}} Scan database to count FP: {{},{,3},{,}, {,5}, {,3,},{,,5}, support for patterns in C {3}, {3,}, {3,,5}, {3,5}, {}, {,5}, {,7}, {,5,7}, {5}, {5,7}, {7}} 6

14 Mining Very Large Database Projection (Pei 00) Tid Tid <empty> 5 Projection Projection 5 Projection 3 Projection Projection Projection 5 Projection 7 Projection 3 <empty> Projection <empty> Projection (a) Parallel Projection (b) Partition Projection 7 Presentation of Association Rules (Table Form) 8

15 Visualization of Association Rule Using Rule Graph 9 Visualization of Association Rule Using Plane Graph 30

16 Conclusion Association rule mining probably the most significant contribution from the database community in KDD A large number of papers have been published Many interesting issues have been explored An interesting research direction Association analysis in other types of data: spatial data, multimedia data, time series data, etc. 3 References Jiawei Han and Micheline Kamber, Data Mining: Concepts and Techniques, Morgan Kaufmann, 00. David Olson and Yong Shi, Introduction to Business Data Mining, McGraw-Hill, 007. Agarwal, R. C., Aggarwal, C. C. & Prasad, V. V. V. 00, 'A Tree Projection Algorithm for Generation of Frequent Item Sets', Journal of Parallel and Distributed Computing (Special Issue on High-Performance Data Mining), vol. 6, no. 3, pp Han, J., Pei, J. & Yin, Y. 000, 'Mining Frequent Patterns without Candidate Generation', in Proceedings of the ACM SIGMOD International Conference on Management of Data, Dallas, Texas, USA, pp. -. Savasere, A., Omiecinski, E. & Navathe, S. 995, 'An Efficient Algorithm for Mining Association Rules in Large Databases', in Proceedings of the st International Conference on Very Large Data Bases (VLDB), Zurich, Switzerland, pp. 3-3

997, 'Parallel Algorithms for Fast Discovery of Association Rules', Data Mining and Knowledge Discovery: An International Journal,

17 References () Pei, J. 00, Pattern-growth Methods for Frequent Pattern Mining, PhD Thesis, Simon Fraser University, Canada. Zaki, M. J. 997, 'Parallel Algorithms for Fast Discovery of Association Rules', Data Mining and Knowledge Discovery: An International Journal, vol., no., pp Sucahyo, Y. G. & Gopalan, R. P. 00, 'CT-PRO: A Bottom-Up Non Recursive Frequent et Mining Algorithm Using Compressed FP-Tree Data Structure', in Proceedings of the IEEE ICDM Workshop on Frequent et Mining Implementations (FIMI), Brighton, UK. 33

Chapter 4: Mining Frequent Patterns, Associations and Correlations

Chapter 4: Mining Frequent Patterns, Associations and Correlations 4.1 Basic Concepts 4.2 Frequent Itemset Mining Methods 4.3 Which Patterns Are Interesting? Pattern Evaluation Methods 4.4 Summary Frequent