Minig Top-K High Utility Itemsets - Report

Size: px

Start display at page:

Download "Minig Top-K High Utility Itemsets - Report"

Shona Bradley
6 years ago
Views:

1 Minig Top-K High Utility Itemsets - Report Daniel Yu, yuda@student.ethz.ch Computer Science Bsc., ETH Zurich, Switzerland May 29, 2015 The report is written as a overview about the main aspects in mining top-k high utility itemsets from the paper Mining Top-K High Utility Itemsets written by Cheng Wei Wu et. al. from the National Cheng Kung University in 2012 [1]. 1 Introduction Utility mining, which refers to the discovery of itemsets with utilities higher than a user-specified minimum utility threshold, is an important task and has a wide range of applications, especially in e-commerce. But setting an appropriate minimum utility threshold is a difficult problem. If the minimum threshold is set to low, too many high utility itemsets will be generated and it takes a long time to compute, while setting the minimum threshold too high would result in too few results. Setting appropriate minimum utility threshold by trial and error is not very efficient. We want to discuss in this report how this can be done better. This report starts with a small example, to get a basic understanding about the general high utility itemsets mining. Then we ll look at a naiv approach and extend it with the increasing threshold mechanism by using the so-called transactional-weighted downward closure (TWDC), which is one of the most important basis of most optimisation mechanism. At the end, we will see a very short introduction to the UP-Tree, which is the state-of-the-art datastructure for mining high utility itemsets. 2 Problem Definition In top-k high utility itemset minig we want to calculate the top-k high utility itemset in D from the system (I, p, D, q): I is a finite set of distinct items I = {i 1, i 2,..., i m }. 1

2 p is a function p : (i j, D) N, which associate each item i j I with a positive number, called the external utility. D is transactional database, which consist of a set of transactions {T 1, T 2,..., T n }. Each transaction T c D is a subset of I and has an unique identifier c, called Tid. q is a function q : (i j, T c ) N, which associate each item i j in transaction i j T c with a positive number, called the internal utility. The profit of an itemset X in a transaction T c is denoted as s(x, T c ) is defined as: s(x, T c ) = ( p(ij, D) q(i j, T c ) ) i j X The utility of an itemset X in D is denoted as u(x) is defined as: u(x, D) = X T c T c D s(x, T c ) 3 Example TID Purchase P 1 (3, A), (5, B) P 2 (2, B) P 3 (2, A), (1, C) P 4 (2, A)(3, B)(1, C) Table 1: Example purchase database Item A B C D Profit 2$ 1$ 3$ 2$ Table 2: Example price table Suppose we are a shop, which sell fruits: Apple, Banana, Cherry and Date: I = {A, B, C, D}. We collected data of purchases of today stored in a so-called transactional database D = {P 1,..., P n }. Each purchase consist of several items and it s purchased quantity. E.g. the first customer purchased 3 Apples and 5 Bananas (see Table 1). There is a second database, wich stores the price of each item (see Table 2). With high utility itemset mining, we can answer the following question: Which product set has the highest profit out of these data? To answer 2

3 this question we define the price as external cost and the quantity as internal cost. With this mapping the utility of itemset {A} is in our example 14$: u({a}, D) = s({a}, P 1 ) + s({a}, P 3 ) + s({a}, P 4 ) = 3 2$ + 2 2$ + 2 2$ = 14$ and the itemset {B, C} has a utility of 6$: u({b, C}, D) = s({b, C}, P 4 ) = 3 1$ + 1 3$ = 6$ In classical high utility itemset mining, we choose the threshold as parameter and every itemset with a higher utility will be in the result set. Without any knowledge about the database D, it s quite hard to choose the threshold, because if you choose the threshold too low, let s say 2$, you will get too much itemsets. And if you choose it too high, let s say 20$, no high utility itemset will be found. We rather want an algorithm, which takes the number of result we want as parameter k. Setting k is more intuitive than setting the threshold because k represents the number of itemsets that the user want to find. We name this problem the Top-K High Utility Itemset Mining. In our example the top-3 high utility itemset would be: itemset {A} with a utility of 14$ itemset {A, C} with a utility of 14$ and itemset {A, B} with a utility of 18$ Please keep these example databases in mind, since it s used for all the example through the whole paper. 3

4 4 Basic Algorithm Generate all subset of I compute utility Choose top-k Figure 1: Basic Algorithm The Basic Algorithm computes the topk high utility itemset problem in three steps: 1. It first generates all the possible subsets of I. 2. Then it computes for all subsets their utilities. 3. Finally, it chooses the top-k high utility itemset out of all itemsets. 4.1 Analysis: Basic Algorithm Now we would like to compute the complexity of this naiv algorithm for finding the top-k high utility itemsets. 1. The number of all subset of I is by definition equals to the size of the powerset of I. Therefore: # subsets of I = P (I) = 2 n, where n = I. Generating all the subset has a complexity of O(2 n ), which is a exponential growth to the number of items. 2. For each subset, we have to calculate it s utility, which can be done with a complete tablescan. The size of the table has O(nm), where m is the number of transactions. It s bounded by n, since every transaction is a subset of I, which has a maximum size of n. 3. Choosing the top-k high utility itemsets is basically a simple scan of all subset, which can be done in O(2 n ). In total we get: O(2 n nm + 2 n ) = O(2 n nm) Calculating the utility of all itemsets seems to be very expensive. The problem is that the utility function is neither monoton nor anti-monotone. Calculating the utility of a itemset wouldn t give us any information about the utility of it s supersets or subsets. We also would like to have some mechanism to prune the search space, since the search space grows exponentially to the number of items as we have seen above. We will discuss this problem extensively in the next two chapters. 4

5 5 Transaction-weighted downward closure (TWDC) One of the major challenge is, that the utility function of an itemset is neither monotone nor anti-monotone. In other words, the utility of an itemset might be equal to, greater or lower than the utility of it s superset and subset. This makes hard to prune the search space, since the exact utility of an itemset won t give us information about it s supersets or subsets. In 2005, Liu et al. proposed in their paper [4] the Two-Phase algorithm, which uses the so-called Transaction-weighted downward closure. First we need the definition of Transactional weighted utility of an itemsets X: T W U(X) = X T i T i D s(t i, T i ) If we take the same example for chapter 3, the transactional weighted utility of the itemset {A} is 28$: T W U({A}) = s(t i, T i ) {A} T i T i D = s({a, B}, P 1 ) + s({a, C}, P 3 ) + s({a, B, C}, P 4 ) = 11$ + 7$ + 10$ = 28$ If we compare this to the actual utility, we see that the TWU function is an upper bound for the utility function, which will be proved below. This function has the nice property of downward closure, which means: If Y is a subset of X I, then the transactional weighted utility if Y is at most the transactional weighted utility of X. We want now proof the downward closure property: To prove: Y X T W U(Y ) T W U(X) proof: We assume Y X. We can show, that the transactional weighted utility of X is at least as the transactional weighted utility of Y. T W U(>) = Y T i T i D u(t i, T i ) X T j T j D u(t j, T j ) = T W U(X), since the collection of transaction containing X is a superset of the collection of transaction containing Y, because Y X. 5

6 6 Increasing Threshold We learnt a transactional weighted utility (TWU) is an upper bound for the utility function, which has the downward closure property. But how can we use it to prune the search space? In 2012, Wu et.al. proposed with their algorithm TKU Base [1] the following idea: The proposed algorithm uses an internal variable named border minimum utility threshold (denoted as min util). We only want to consider itemsets with a higher utility that the threshold. The algorithm initially set the threshold to 0 and gradually raise the threshold to prune the search space by using the TWDC. We can raise the threshold after a sufficient number of itemsets with higher TWU has been captured. For the algorithm, we need to calculate the lower and upper bound of an itemset. For the upper bound the TWU can be used. For the lower bound we use the definition of minimum item utility of an item a, denoted as miu(a): miu(a) = min T D u(a, T ) and minimum itemset utility of an itemset X = {a 1,..., a m }, denoted as MIU(X): MIU(X) = m miu(a i ) SC(X), i=1 where SC(X) is the support count of an itemset, which is the number of transaction containing X in D. This is cleary a lower bould for the utility function. If we take the data of chapter 3, the minimum itemset utility of itemset {A, B} is 12$: MIU({A, B}) = miu({a}) SC({A, B}) + miu({b}) SC({A, B}) = 4$ 2 + 2$ 2 = 12$ For the algorithm, we need to destinguish between three different cases (cf. Figure 2). For a itemset X: I. MIU(X) min util T W U(X) II. MIU(X) T W U(X) < min util III. min util MIU(X) T W U(X) 6

7 I. MIU min util TWU II. MIU TWU min util min util MIU TWU III. Figure 2: Three cases for min util, MIU and TWU These cases are complete, because all the other possible cases would violate the following fact: MIU(X) u(x) T W U(X) T W U(X) T W U(X) Let s analyze these three cases in detail: I. We call such a itemset a potential itemset, since the utility might be higher than the threshold min util. We have to keep these itemset, because they are a candidates for high utility itemset. II. Such an itemset X are definitely not part of the top-h high utility itemset and can be savely discarded (the proof is below in III.), since his exact utility is for sure below the threshold min util: u(x) T W U(X) < min util By applying the TWDC property of TWU, we can also prune all it s subsets X, which are less promising itemsets because of their lower TWU: u(x ) T W U(X ) T W U(X) < min util III. Such an itemset X is also candidate for high utility itemsets, so we have to keep it. Here the MIU(X) can be used to raise the border min. We need for this purpose a proof: To prove: Assume we are mining for the top-k high utility itemset. Let C = {X 1, X 2,..., X m } be a ordered set of itemsets, where m k and X i is the i-th itemset in C and MIU(X i ) MIU(X j ), i < j (ordered by MIU). For any itemset Y, if T W U(Y ) < min{miu(x i ) X i C, 1 i k}, Y is not a top-k high utility itemset. 7

8 proof: According to the definition of T W U and MIU we know, that: u(y ) T W U(Y ) < MIU(X i ) u(x i ), where X i C, 1 i k. If there already exist k itemsets whose utilites are higher that the utility of Y, by the definition of top-k high utility itemset, Y is not a top-k high utility itemset. What also follows from this proof is, that we can safely set the threshold min util to min{miu(x i ) X i C, 1 i k}, because there is no sense to consider itemsets, which are definitely not part of the top-k high utility itemset. How do we keep track of the itemset to efficiently update border min? We use a max-heap structure L to maintain the k highest MIUs of the candidate itemsets until now. Once k MIUs are found, min util is raised to the k th MIU in L. Each time a new candidate X is found and its MIU is higher than min util, X is added ti L and the lowest MIU in L is removed. After that, min util is raised to the k-th MIU in L. 7 Advanced Algorithm Generate all the subsets of I and discard all it s subsets Calculate MIU and TWU II. discard it I. III. save the candidate save the candidate and update the threshold Trash Calculate utility and choose top-k Figure 3: The TKU Base algorithm The new algorithm consists of three part: 1. generating all the itemsets 2. choose all the potential candidate for high utility itemstes with the increasing threshold method, which we have discussed in the last chapter extensively. We initialize the threshold with 0. For 8

9 each itemset, we check to which case it belongs (I., II. or III.) and act appropriate. To keep trach of MIUs to efficiently update border min, we use a max-heap L as discussed before. At the end we get a list of candidates stored in C. 3. Choosing the top-k high utility itemsets is basically a sinple scan of C. Algorithm 1: Advanced Algorithm // Initialization 1 L empty minheap; 2 C empty set; 3 min util 0 ; // Generate all the subsets of I 4 M subset(i); // Calculate MIU and TWU, case destinction 5 while M is not empty do 6 X take one itemset C; 7 if MIU(X) min util and min util T W U(X) then // Case I. 8 C X; 9 else if T W U(X) min util then // Case II. 10 C X; 11 L MIU(X); 12 update min util; 13 else // Case III. 14 M M subset(x); 15 end 16 end // Check the candidates in C 17 Calculate the exact utility for each itemset in C ; 18 Output the top-k high utility itemset in C ; Note: The subset(x) function generates all the subsets of X. 9

10 7.1 Analysis: Advanced Algorithm This new algorithm seems to have a quite overhead to calculate all the TWUs. Does it at least garantuee to perform better than the basic algorithm? The answer is sadly no. The TWDC with the increasing threshold doesn t give us any guarantee to perform better at all. In fact it could be slower. As a simple and short example, think of a database D with just one transaction D 1, which has all the items D 1 = I and assume p = q = 1. With such a database, the TWU for all itemsets would be equal, since the TWU is a overestimation: For any X I : T W U(X) = s(d i, D i ) X D i D i D = s(d 1, D 1 ) = s(i, D 1 ) = i j I p(i j, D) q(i j, D 1 ) = i j I 1 = I = n Which such a system, we wouldn t get any additional information about the utility of the itemsets. We couldn t prune the search space with TWDC, which means that we still have to check the utility of all possible subsets of I. However in practise, a online store like amazon which serves millions of products, it s very unlikely, that a person will purchase millions of products in one purchase. For the dataset which the authors used for performance testing, the transaction size was quite small. They doesn t have to consider this problem, since they only used real world datasets, where the transaction size is relative small to the number of Items. for example the Foodmart dataset has 1559 items and the average size of transactions was 4.4 or the Chainstore dataset has items and the average size of transactions was Up-Tree In this subsection, we briefly introduce the structure of the UP-Tree. We ll need this structure for the baseline approach for mining top-k high utility itemsets. In UP-Tree, each node N consists of thefollowing elements: name (the item name of N), count (the support count of N), nu (the node utility of N), parent (records the parent node if N) and link (is a node link which points to a node whose item name is the same as name). Due to time reasons, this datastuctrue can t be discussed in detail. For the details about the Up-Tree, readers can refer to the paper [2]. In 10

11 short, the UP-Tree can be constructed with only two tablescan of D. it s a datastructure, which can delete a itemset and all it s subset very efficiently. Also calculating the TWU and the support count, which we will use for calculating the upper and lower bound is just a traversation in the UP-Tree. For the algorithm,the UP-Tree is used for generating the next itemset to analyze. For case II, and III, the UP-Tree will be updated. For illustration, this is the UP-Tree for our example from chapter 3: Item TWU Link Root A 28 B C B,1,2 A,3,14 D 0 B,2,18 C,1,7 C,1,10 Figure 4: Example UP-Tree for min util = 0 9 Conclusion We have seen two algorithms to mine top-k high utility itemsets: the basic one and the advanced one. The advanced one has the increasing threshold mechanism to filter the candidiates by using the transactional weighted downward closure. We have also learned, that the increasing threshold method is not for all databases an improvement, since it relies heavly on the additional information by calculating the transactional weighted utility, which is not always the case. The author should have also test the TKU Base on different database than typical real world commerce data, since high utility mining doesn t refer only to commerce datasets. 11

12 References [1] C. W. Wu, B.-E. Shie, P. S. Yu and V. S. Tsend. Mining Top-K High Utility Itemsets. In Proc. of Int l Conf. in ACM SIGKDD. pp , [2] V.S. Tseng, C.-W. Wu, B.-E. Sie and P.S. Yu. UP-Growth: an efficient algorithm for high utility itemset mining. In Proc. of Int l Conf. in ACM SIGKDD. p , [3] C.F. Ahmed, S.K. Tanbeer, B.-S. Joeng and Y.-K. Lee. Efficient Tree Structures for High-utility Pattern Mining in Incremental Databases. In IEEE Transactions on Knowledge and Data Engineering, Vol. 21, Issue 12, pp , [4] Y. Liu, W.Liao, and A.Choudhary. A fast high-utility itemsets mining algorithm. In Proc. of the Utility-Based data Mining Workshop, [5] Y. Liu, J. Li, W.-K. Liao, A. Choudhary and Y.Shi. High Utility Itemsets Mining. In Int l Journal of Information Technology and Decision Making p

FHM: Faster High-Utility Itemset Mining using Estimated Utility Co-occurrence Pruning

FHM: Faster High-Utility Itemset Mining using Estimated Utility Co-occurrence Pruning Philippe Fournier-Viger 1, Cheng-Wei Wu 2, Souleymane Zida 1, Vincent S. Tseng 2 1 Dept. of Computer Science, University