1. Interpret single-dimensional Boolean association rules from transactional databases

Transcription

1 1 STARTSTUDING.COM 1. Interpret single-dimensional Boolean association rules from transactional databases Association rule mining: Finding frequent patterns, associations, correlations, or causal structures among sets of items or objects in transaction databases, relational databases, and other information repositories. Single-Dimension Association rule: It is a rule that references only one dimension. Example: buys(x, computer ) buys(x, financial_software ). The single dimension is buys The following rule is a multi-dimensional AR: Age(X, ) income(x, 80K..100K ) buys(x, High Resolution TV) Basic Concepts: o A set of items: I={x1,, xk} Transactions: D={t1,t2,, tn}, tj I A k-itemset: {Ii1,Ii2,, Iik} ISupport of an itemset: Percentage of transactions that contain that itemset.large (Frequent) itemset: Itemset whose number of occurrences is above a threshold. o Example:

2 2 STARTSTUDING.COM I = { Beer, Bread, Jelly, Milk, PeanutButter} Support of {Bread,PeanutButter} = 3/5 = 60% Transaction-id Items bought A, B, C A, C A, D B, E, F

3 3 STARTSTUDING.COM Customer buys both Customer buys milk Customer buys read Association Rules o Implication: X Y where X,Y I and X Y = ; o Support of AR (s) X Y: Percentage of transactions that contain X Y Probability that a transaction contains XY. o Confidence of AR (a) X Y: Ratio of number of transactions that contain X Y to the number that contain X Conditional probability that a transaction having X also contains Y.

4 4 STARTSTUDING.COM Transaction-id Items bought A, B, C A, C A, D B, E, F Frequent pattern {A} {B} {C} {A, C} Support 75% 50% 50% 50%

5 5 STARTSTUDING.COM For rule A C: Support(AC) = P(AC) = support({a}{c}) = 50% confidence (A C) = P(C A) = support({a}{c})/support({a}) = 66.6%Another Example: Support Confidence Bread Peanutbutter = 3/5 %= 60% = (3/5)/(4/5)%=75% Peanutbutter Bread 60% = (3/5)/(3/5)%=100%

6 6 STARTSTUDING.COM Jelly Milk 0% 0% Jelly Peanutbutter =1/5 % = 20% = (1/5)/(1/5) % = 100% 2. Explain Apriori algorithm with example. Uses prior knowledge of frequent itemset properties. It is an iterative algorithm known as level-wise search. The search proceeds level-by-level as follows: First determine the set of frequent 1-itemset; L1 o Second determine the set of frequent 2-itemset using L1: L2 o Etc. The complexity of computing Li is O(n) where n is the number of transactions in the transaction database. Reduction of search space: o In the worst case what is the number of itemsets in a level Li? o Apriori uses Apriori Property : Apriori Property: It is an anti-monotone property: if a set cannot pass a test, all of its supersets will fail the same test as well. It is called anti-monotone because the property is monotonic in the context of failing a test. All nonempty subsets of a frequent itemset must also be frequent. An itemset I is not frequent if it does not satisfy the minimum support threshold: P(I) < min_sup If an item A is added to the itemset I, then the resulting itemset I A cannot occur more frequently than I: I A is not frequent

7 7 STARTSTUDING.COM Therefore, P(I A) < min_sup How Apriori algorithm uses Apriori property? In the computation of the itemsets in L k using L k-1 It is done in two steps: Join Prune Join Step: The set of candidate k-itemsets (element of L k), C k, is generated by joining L k-1 with itself: L k-1 L k-1 Given l 1 and l 2 of L k-1 L i=l i1,l i2,l i3,,l i(k-2),l i(k-1) L j=l j1,l j2,l j3,,l j(k-2),l j(k-1) Where L i and L j are sorted. L i and L j are joined if there are different (no duplicate generation). Assume the following: l i1=l j1, l i2=l j1,, l i(k-2)=l j(k-2) and l i(k-1) < l j(k-1) The resulting itemset is: l i1,l i2,l i3,,l i(k-1),l j(k-1) Example of Candidate-generation:

8 8 STARTSTUDING.COM L3={abc, abd, acd, ace, bcd} Self-joining: L3 L3 abcd from abc and abd acde from acd and ace Prune step C k is a superset of L k some itemset in C k may or may not be frequent. L k: Test each generated itemset against the database: Scan the database to determine the count of each generated itemset and include those that have a count no less than the minimum support count. This may require intensive computation. Use Apriori property to reduce the search space: Any (k-1)-itemset that is not frequent cannot be a subset of a frequent k-itemset. Remove from C k any k-itemset that has a (k-1)-subset not in L k-1 (itemsets that are not frequent) Efficiently implemented: maintain a hash table of all frequent itemset. Example of Candidate-generation and Pruning: L3={abc, abd, acd, ace, bcd} Self-joining: L3 L3 abcd from abc and abd

9 9 STARTSTUDING.COM acde from acd and ace Pruning: C4={abcd} acde is removed because ade is not in L Example Tid Database TDB Items A, C, D B, C, E A, B, C, E Itemset sup {A} 2 L 1 C 1 {B} 3 {C} 3 1 st scan {D} 1 {E} 3 Itemset {A} {B} {C} {E} sup B, E L 2 Itemset {A, C} {B, C} {B, E} {C, E} sup C 2 Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E} sup C2 2 nd scan Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E} C 3 Itemset 3 rd scan L 3 {B, C, E} Itemset {B, C, E} sup 2

10 10 STARTSTUDING.COM Pseudo-code Ck: Candidate itemset of size k Lk : frequent itemset of size k L1 = {frequent items}; for (k = 1; Lk!=Æ; k++) do - Ck+1 = candidates generated from Lk; - for each transaction t in database do increment the count of all candidates in Ck+1 in t; that are contained endfor; - Lk+1 = candidates in Ck+1 with min_support endfor; return k Lk;

11 11 STARTSTUDING.COM Challenges Multiple scans of transaction database Huge number of candidates Tedious workload of support counting for candidates Improving Apriori: o general ideas o Reduce passes of transaction database scans o Shrink number of candidates o Facilitate support counting of candidates 3. Discuss the techniques for improving efficiency of Apriori algorithm? Techniques for improving the efficiency of Apriori algorithm: Hash-based technique: A k-itemset whose corresponding hashing bucket count is below the threshold cannot be frequent Transaction reduction: A transaction that does not contain any frequent k-itemset can be removed in subsequent scans Partitioning: Any itemset that is potentially frequent in DB must be frequent in at least one of the partitions of DB Sampling: mining on a subset of given data, lower support threshold + a method to determine the completeness Dynamic itemset counting: add new candidate itemsets only when all of their subsets are estimated to be frequent Hash-based technique Hashing itemset counts Example: o Transaction DB: TID T100 List of Transactions I1,I2,I5

12 12 STARTSTUDING.COM T200 T300 T400 T500 T600 T700 T800 T900 I2,I4 I2,I3 I1,I2,I4 I1,I3 I2,I3 I1,I3 I1,I2,I3,I5 I1,I2,I3 Create a hash table for candidate 2-itemsets: Generate all 2-itemsets for each transaction in the transaction DB H(x,y) = ((order of x) * 10 + (order of y)) mod 7 A 2-itemset whose corresponding bucket count is below the support threshold cannot be frequent. Remember: support(xy) = percentage number of transactions that contain x and y. Therefore, if the minimum support is 3, then the itemsets in buckets 0, 1, 3, and 4 cannot be frequent and so they should not be included in C2. Bucket count Content {I1,I4} {I1,I5 } {I2,I3 } {I2,I4 } {I2,I5 } {I1,I2 } {I1,I3 } {I3,I5} {I1,I5 } {I2,I3 } {I2,I4 } {I2,I5 } {I1,I2 } {I1,I3 }

13 13 STARTSTUDING.COM {I2,I3 } {I2,I3 } {I1,I2 } {I1,I2 } {I1,I3 } {I1,I3 } Transaction reduction Reduce the number of transactions scanned in future iterations. A transaction that does not contain any frequent k-itemsets cannot contain any frequent (k+1)-itemsets: Do not include such transaction in subsequent scans. Other techniques include: Partitioning (partition the data to find candidate itemsets) Sampling (Mining on a subset of the given data) Dynamic itemset counting (Adding candidate itemsets at different points during a scan) 4. Discuss mining of frequent itemsets without candidate generation. Mining Frequent Itemsets without Candidate Generation Objectives: o The bottleneck of Apriori: candidate generation o Huge candidate sets: For 104 frequent 1-itemset, Apriori will generate 107 candidate 2-itemsets. To discover a frequent pattern of size 100, e.g., {a1, a2,, a100}, one needs to generate candidates. o Multiple scans of database: Needs (n +1) scans, n is the length of the longest pattern. Principal o Compress a large database into a compact, Frequent-Pattern tree (FP-tree) structure Highly condensed, but complete for frequent pattern mining o Avoid costly database scans Develop an efficient, FP-tree-based frequent pattern mining method A divide-and-conquer methodology: decompose mining tasks into smaller ones Avoid candidate generation: sub-database test only!

14 14 STARTSTUDING.COM Example: min_support = 0.5 TID Items bought (Ordered) frequent items 100 {f, a, c, d, g, i, m, p} {f, c, a, m, p} 200 {a, b, c, f, l, m, o} {f, c, a, b, m} 300 {b, f, h, j, o} {f, b} 400 {b, c, k, s, p} {c, b, p} 500 {a, f, c, e, l, p, m, n} {f, c, a, m, p} Header Table {} Item Supp. Count Node Link f:4 c:1 f 4 c:3 b:1 b:1 c 4 a 3 a:3 p:1 b 3 m 3 m:2 b:1 p:2 m:1 Node Structure: Item count node pointer child pointers parent pointer Algorithm:

15 15 STARTSTUDING.COM 1. Scan DB once, find frequent 1-itemset (single item pattern) 2. Order frequent items in frequency descending order, called L order: (in the example below: F(4), c(4), a(3), etc.) 3. Scan DB again and construct FP-tree a. Create the root of the tree and label it null or {} b. The items in each transaction are processed in the L order (sorted according to descending support count). c. Create a branch for each transaction d. Branches share common prefixes 1.1. Mining Frequent patterns using FP-Tree General idea (divide-and-conquer) o Recursively grow frequent pattern path using the FP-tree Method o For each item, construct its conditional pattern-base, and then its conditional FP-tree o Recursion: Repeat the process on each newly created conditional FP-tree o Until the resulting FP-tree is empty, or it contains only one path (single path will generate all the combinations of its sub-paths, each of which is a frequent pattern) 1.2. Major steps to mine FP-trees Main Steps: 1. Construct conditional pattern base for each node in the FP-tree 2. Construct conditional FP-tree from each conditional pattern-base 3. Recursively mine conditional FP-trees and grow frequent patterns obtained so far If the conditional FP-tree contains a single path, simply enumerate all the patterns Step 1: From FP-tree to Conditional Pattern Base Starting at the frequent header table in the FP-tree Traverse the FP-tree by following the link of each frequent item, starting by the item with the highest frequency. Accumulate all of transformed prefix paths of that item to form a conditional pattern base

16 16 STARTSTUDING.COM Example: Header Table {} Item Supp. Count Node Link f:4 c:1 f 4 c:3 b:1 b:1 c 4 a 3 a:3 p:1 b 3 m 3 m:2 b:1 p:2 m:1

17 17 STARTSTUDING.COM Conditional pattern bases Item Conditional pattern base c f:3 a fc:3 b fca:1, f:1, c:1 m p fca:2, fcab:1 fcam:2, cb:1 Properties of FP-tree for Conditional Pattern Base Construction: o Node-link property For any frequent item ai, all the possible frequent patterns that contain ai can be obtained by following ai's node-links, starting from ai's head in the FP-tree header. o Prefix path property To calculate the frequent patterns for a node ai in a path P, only the prefix subpath of ai in P need to be accumulated and its frequency count should carry the same count as node ai. Step 2: Construct Conditional FP-tree o Example: For each pattern-base Accumulate the count for each item in the base Construct the FP-tree for the frequent items of the pattern base m-conditional pattern base: fca:2, fcab:1 Header Table {} Item Supp. Count Node Link f:4 c:1 f 4 c:3 b:1 b:1 c 4

18 18 STARTSTUDING.COM Mining Frequent Patterns by Creating Conditional Pattern-Bases: Item Conditional pattern-base Conditional FP-tree p {(fcam:2), (cb:1)} {(c:3)} p m {(fca:2), (fcab:1)} {(f:3, c:3, a:3)} m b {(fca:1), (f:1), (c:1)} Empty a {(fc:3)} {(f:3, c:3)} a c {(f:3)} {(f:3)} c

19 19 STARTSTUDING.COM f Empty Empty Step 3: Recursively mine the conditional FP-tree Why is FP-Tree mining fast? o The performance study shows FP-growth is an order of magnitude faster than Apriori o Reasoning: No candidate generation, no candidate test Use compact data structure Eliminate repeated database scan Basic operation is counting and FP-tree building FP-Growth vs. Apriori: Scalability with the support Threshold [Jiawei Han and Micheline Kamber] 5. Explain mining Multi-dimensional Boolean association rules from transactional databases. Association rules that involve two or more dimensions or predictions can be refered as multidimensional association rules. Example: The following rule is a multi-dimensional Boolean association rule: Age(X, ) income(x, 80K..100K ) buys(x, High Resolution TV) Transactions Example TID Produce 1 MILK, BREAD, EGGS 2 BREAD, SUGAR 3 BREAD, CEREAL 4 MILK, BREAD, SUGAR 5 MILK, CEREAL 6 BREAD, CEREAL TID Products 1 A, B, E 2 B, D 3 B, C 4 A, B, D 5 A, C 6 B, C

20 20 STARTSTUDING.COM ITEMS: A = milk B= bread C= cereal D= sugar E= eggs

21 21 STARTSTUDING.COM Attributes converted to binary flags TID A B C D E Item: attribute=value pair or simply value usually attributes are converted to binary flags for each value, e.g. product= A is written as A Itemset I : a subset of possible items Example: I = {A,B,E} (order unimportant) Transaction: (TID, itemset) TID is transaction ID Support of an itemset sup(i ) = no. of transactions t that support (i.e. contain) I

22 22 STARTSTUDING.COM In example database: sup ({A,B,E}) = 2, sup ({B,C}) = 4 Frequent itemset I is one with at least the minimum support count sup(i ) >= minsup A: Example: Suppose {A,B} is frequent. Since each occurrence of A,B includes both A and B, then both A and B must also be frequent Similar argument for larger itemsets Almost all association rule algorithms are based on this subset property Association rule R : Itemset1 => Itemset2 Itemset1, 2 are disjoint and Itemset2 is non-empty meaning: if transaction includes Itemset1 then it also has Itemset2 Examples A,B => E,C A => B,C Given frequent set {A,B,E}, what are possible association rules? A => B, E A, B => E A, E => B B => A, E B, E => A E => A, B => A,B,E (empty rule), or true => A,B,E Suppose R : I => J is an association rule sup (R) = sup (I J) is the support count support of itemset I J (I or J) conf (R) = sup(j) / sup(r) is the confidence of R fraction of transactions with I J that have J Association rules with minimum support and count are sometimes called strong rules Given frequent set {A,B,E}, what association rules have minsup = 2 and minconf= 50%? A, B => E : conf=2/4 = 50% TID List of items 1 A, B, E 2 B, D 3 B, C 4 A, B, D 5 A, C 6 B, C 7 A, C 8 A, B, C, E 9 A, B, C

23 23 STARTSTUDING.COM Given frequent set {A,B,E}, what association rules have minsup = 2 and minconf= 50%? A, B => E : conf=2/4 = 50% A, E => B : conf=2/2 = 100% B, E => A : conf=2/2 = 100% E => A, B : conf=2/2 = 100% Don t qualify A =>B, E : conf=2/6 =33%< 50% B => A, E : conf=2/7 = 28% < 50% => A,B,E : conf: 2/9 = 22% < 50% TID List of items 1 A, B, E 2 B, D 3 B, C 4 A, B, D 5 A, C 6 B, C 7 A, C 8 A, B, C, E 9 A, B, C

24 24 STARTSTUDING.COM 6. Summarize the kinds of association rules. Kinds or Types of Association Rules Boolean AR: o It is a rule that checks whether an item is present or absent. o All the examples we have seen so far are Boolean AR. Quantitative AR: o It describes associations between quantitative items or attributes. o Generally, quantitative values are partitioned into intervals. o Example: Age(X, ) income(x, 80K..100K ) buys(x, High Resolution TV) Single-Dimension AR: o It is a rule that references only one dimension. o Example: buys(x, computer ) buys(x, financial_software ) The single dimension is buys o The following rule is a multi-dimensional AR:

25 25 STARTSTUDING.COM Age(X, ) income(x, 80K..100K ) buys(x, High Resolution TV) Multi-level AR o It is a set of rules that reference different levels of abstraction. o Example: Age(X, ) buys(x, desktop ) Age(X, ) buys(x, laptop ) Laptop desktop computer 7. Explain constraint based association mining. Interactive, exploratory mining giga-bytes of data? Could it be real? Making good use of constraints! What kinds of constraints can be used in mining? Knowledge type constraint: classification, association, etc. Data constraint: SQL-like queries Find product pairs sold together in Vancouver in Dec. 98. Dimension/level constraints: in relevance to region, price, brand, customer category. Rule constraints small sales (price < $10) triggers big sales (sum > $200). Interestingness constraints: strong rules (min_support 3%, min_confidence 60%). Rule Constraints in Association Mining Two kind of rule constraints: Rule form constraints: meta-rule guided mining. P(x, y) ^ Q(x, w) takes(x, database systems ). Rule (content) constraint: constraint-based query optimization (Ng, et al., SIGMOD 98).

26 26 STARTSTUDING.COM sum(lhs) < 100 ^ min(lhs) > 20 ^ count(lhs) > 3 ^ sum(rhs) > variable vs. 2-variable constraints (Lakshmanan, et al. SIGMOD 99): 1-var: A constraint confining only one side (L/R) of the rule, e.g., as shown above. 2-var: A constraint confining both sides (L and R). sum(lhs) < min(rhs) ^ max(rhs) < 5* sum(lhs) Constrain-Based Association Query Database: (1) trans (TID, Itemset ), (2) iteminfo (Item, Type, Price) A constrained asso. query (CAQ) is in the form of {(S1, S2 ) C }, where C is a set of constraints on S1, S2 including frequency constraint A classification of (single-variable) constraints: Class constraint: S A. e.g. S Item Domain constraint: S v, {,,,,, }. e.g. S.Price < 100 v S, is or. e.g. snacks S.Type V S, or S V, {,,,, } e.g. {snacks, sodas } S.Type Aggregation constraint: agg(s) v, where agg is in {min, max, sum, count, avg}, and {,,,,, }. e.g. count(s1.type) 1, avg(s2.price) 100 Constrained Association Query Optimization Problem Given a CAQ = { (S1, S2) C }, the algorithm should be : sound: It only finds frequent sets that satisfy the given constraints C complete: All frequent sets satisfy the given constraints C are found A naïve solution: Apply Apriori for finding all frequent sets, and then to test them for constraint satisfaction one by one. Our approach: Comprehensive analysis of the properties of constraints and try to push them as deeply as possible inside the frequent set computation. Categories of Constraints. 1. Anti-monotone and Monotone Constraints constraint Ca is anti-monotone iff. for any pattern S not satisfying Ca, none of the superpatterns of S can satisfy Ca A constraint Cm is monotone iff. for any pattern S satisfying Cm, every super-pattern of S also satisfies it 2. Succinct Constraint A subset of item Is is a succinct set, if it can be expressed as p(i) for some selection predicate p, where is a selection operator

27 27 STARTSTUDING.COM SP2I is a succinct power set, if there is a fixed number of succinct set I1,, Ik I, s.t. SP can be expressed in terms of the strict power sets of I1,, Ik using union and minus A constraint Cs is succinct provided SATCs(I) is a succinct power set 3. Convertible Constraint Suppose all items in patterns are listed in a total order R A constraint C is convertible anti-monotone iff a pattern S satisfying the constraint implies that each suffix of S w.r.t. R also satisfies C A constraint C is convertible monotone iff a pattern S satisfying the constraint implies that each pattern of which S is a suffix w.r.t. R also satisfies C Property of Constraints: Anti-Monotone Anti-monotonicity: If a set S violates the constraint, any superset of S violates the constraint. Examples: sum(s.price) v is anti-monotone sum(s.price) v is not anti-monotone sum(s.price) = v is partly anti-monotone Application: Push sum(s.price) 1000 deeply into iterative frequent set computation. Example of Convertible Constraints: Avg(S) V Let R be the value descending order over the set of items E.g. I={9, 8, 6, 4, 3, 1} Avg(S) v is convertible monotone w.r.t. R If S is a suffix of S1, avg(s1) avg(s) {8, 4, 3} is a suffix of {9, 8, 4, 3} avg({9, 8, 4, 3})=6 avg({8, 4, 3})=5 If S satisfies avg(s) v, so does S1 {8, 4, 3} satisfies constraint avg(s) 4, so does {9, 8, 4, 3} Property of Constraints: Succinctness Succinctness: For any set S1 and S2 satisfying C, S1 S2 satisfies C Given A1 is the sets of size 1 satisfying C, then any set S satisfying C are based on A1, i.e., it contains a subset belongs to A1, Example : sum(s.price ) v is not succinct min(s.price ) v is succinct Optimization:

28 28 STARTSTUDING.COM If C is succinct, then C is pre-counting prunable. The satisfaction of the constraint alone is not affected by the iterative support counting. 8. Describe in detail about multilevel association rules. Multiple-Level Association Rules Items often form hierarchy. Items at the lower level are expected to have lower support. Rules regarding itemsets at appropriate levels could be quite useful. Transaction database can be encoded based on dimensions and levels We can explore shared multi-level mining Food milk bread skim 2% white wheat Fraser Sunset Approach A top-down, progressive deepening approach: First find high-level strong rules: milk bread [20%, 60%]. Then find their lower-level weaker rules: 2% milk wheat bread [6%, 50%]. Variations at mining multiple-level association rules. Level-crossed association rules: 2% milk Wonder wheat bread

29 29 STARTSTUDING.COM Association rules with multiple, alternative hierarchies: 2% milk Wonder bread Two multiple-level mining associations strategies: Uniform Support Reduced support Uniform Support: the same minimum support for all levels One minimum support threshold. No need to examine itemsets containing any item whose ancestors do not have minimum support. Drawback: o Lower level items do not occur as frequently. If support threshold too high miss low level associations too low generate too many high level assoc. Level 1 min_sup = 5% Milk [support = 10%] Level 2 min_sup = 5% 2% Milk [support = 6%] Skim Milk [support = 4%] Reduced Support: reduced minimum support at lower levels There are 4 search strategies: o Level-by-level independent o Level-cross filtering by k-itemset o Level-cross filtering by single item o Controlled level-cross filtering by single item Level-by-Level independent: o Full-breadth search o No background knowledge is used.

30 30 STARTSTUDING.COM o Each node is examined regardless the frequency of its parent. Level-cross filtering by single item: o An item at the ith level is examined if and only if its parent node at the (i-1)th level is frequent. Level-cross filtering by k-itemset: o A k-itemset at the ith level is examined if and only if its corresponding parent k-itemset at the (i-1)th level is frequent. o This restriction is stronger than the one in level-cross filtering by single item o They are not usually many k-itemsets that, when combined, are also frequent: Many valuable patterns can be mined Controlled level-cross filtering by single item: o A variation of the level-cross filtering by single item: Relax the constraint in this approach o Allow the children of items that do not satisfy the minimum support threshold to be examined if these items satisfy the level passage threshold: level_passage_supp o level_passage_sup Value: It is typically set between the min_sup value of the given level and the min_sup of the next level. o Example: Level 1 min_sup = 12% level_passage_sup = 8% Milk [support = 10%] Level 2 min_sup = 4% 2% Milk [support = 6%] Skim Milk [support = 5%] Redundancy Filtering Some rules may be redundant due to ancestor relationships between items.

31 31 STARTSTUDING.COM Definition: A rule R1 is an ancestor of a rule, R2, if R1 can be obtained by replacing the items in R2 by their ancestors in a concept hierarchy. Example R1: milk wheat bread [support = 8%, confidence = 70%] R2: 2% milk wheat bread [support = 2%, confidence = 72%] Milk in R1 is an ancestor of 2% milk in R2. We say the first rule is an ancestor of the second rule. A rule is redundant if its support is close to the expected value, based on the rule s ancestor:r2 is redundant since its confidence is close to the confidence of R1 (kind of expected) and its support is around 2% = (8% * ¼) R2 does not add any additional information. Multi-Level Mining: Progressive Deepening A top-down, progressive deepening approach: o First mine high-level frequent items: milk (15%), bread (10%) o Then mine their lower-level weaker frequent itemsets: 2% milk (5%), wheat bread (4%) Different min_support threshold across multi-levels lead to different algorithms: o If adopting the same min_support across multi-levels then toss t if any of t s ancestors is infrequent. o If adopting reduced min_support at lower levels then examine only those descendents whose ancestor s support is frequent/non-negligible. 9. Discuss the mining of frequent item sets using vertical data format with example. The Apriori and the FP-growth methods mine frequent patterns from a set of transactions in TID-itemset format, where TIS is a transaction id and itemset is the set of items bought in transaction TID. This data format is known as horizontal data format. Alternatively data can also be presented in item-tid_set format, where item is an item name and TID_set is the set of transaction identifiers containing the item. This format is known as vertical data format. In the following table you can see the vertical data format of the example, shown in table 3: Itemset TID_set

32 32 STARTSTUDING.COM I1 {T100, T400, T500, T700, T800, T900} I2 {T100, T200, T300, T400, T600, T800, T900} I3 {T300, T500, T600, T700, T800, T900} I4 {T200, T400} I5 {T100, T800} The 2-itemsets in vertical data format: Itemset TID_set {I1, I2} {T100, T400, T800, T900} {I1, I3} {T300, T700, T800, T900} {I1, I4} {T400} {I1, I5} {T100, T800} {I2, I3} {T300, T600, T800, T900} {I2, I4} {T200, T400} {I2, I5} {T100, T800}

33 33 STARTSTUDING.COM {I3, I5} {T800} The 3itemsets in vertical data format: Itemset TID_set {I1, I2, I3} {T800, T900} {I1, I2, I5} {T100, T800} First we transform the horizontally formatted data to the vertical format by scanning the data set once. The support count of an itemset is simply the length of the TID_set of the itemset. Starting with k=1 the frequent k-itemsets can be used to construct the candidate (k+1)-itemsets. This process repeats with k incremented by 1 each time until no frequent itemsets or no candidate itemsets can be found. Advantages of this algorithm:

34 34 STARTSTUDING.COM - Better than Apriori in the generation of candidate (k+1)-itemset from frequent k-itemsets - There is no need to scan the database to find the support (k+1) itemsets (for k>=1). This is because the TID_set of each k-itemset carries the complete information required for counting such support. The disadvantage of this algorithm consist in the TID_set being to long, taking substantial memory space as well as computation time for intersecting the long sets. 10. Explain the mining of closed frequent itemsets. An itemset X is closed if X is frequent and there exists no super-pattern Y כ X, with the same support as X. An itemset X is a max-pattern if X is frequent and there exists no frequent super-pattern Y כ X. Closed pattern is a lossless compression of freq. patterns Exercise: Suppose a DB contains only two transactions o <a1,, a100>, <a1,, a50> o Let min_sup = 1 What is the set of closed itemset? o {a1,, a100}: 1 o {a1,, a50}: 2 What is the set of max-pattern? o {a1,, a100}: 1 What is the set of all patterns? o {a1}: 2,, {a1, a2}: 2,, {a1, a51}: 1,, {a1, a2,, a100}: 1 Algorithm for mining closed itemsets: CHARM

35 35 STARTSTUDING.COM

36 36 STARTSTUDING.COM Find all frequent itemsets: 2 m : NP-Complete. Assuming a bound on transaction length O (r n 2 L ). Generating confident rules: For each itemset of size k, 2 k potential rules. Complexity: O (f 2 L ). Galois connection: For all a in A and b in B: F (a) b G (b) a

37 37 STARTSTUDING.COM Example1: t (ACW) = t (A) t (C) t (W) = = 1345 i (245) = CDW ACW Í ACDW t (ACW) = 1345 Ê 135 = t (ACDW)

38 38 STARTSTUDING.COM A Closed Itemset X is an Itemset that is same as its closure. Example 2 : c it (AC) = i(t(ac) = i(1345) = ACW conclusion: AC is not closed. ACW is closed.