Mining from Quantitative Data with Linguistic Minimum Supports and Confidences

Transcription

1 Miig from Quatitative Data with Liguistic Miimum Supports ad Cofideces Tzug-Pei Hog, Mig-Jer Chiag ad Shyue-Liag Wag Departmet of Electrical Egieerig Natioal Uiversity of Kaohsiug Kaohsiug, 8, Taiwa, R.O.C. Graduate School of Iformatio Egieerig I-Shou Uiversity Kaohsiug, 84, Taiwa, R.O.C. Abstract - Most covetioal data-miig algorithms idetify the relatioships amog trasactios usig biary values ad set the miimum supports ad miimum cofideces at umerical values. This paper thus attempts to propose a ew miig approach for extractig liguistic weighted associatio rules from quatitative trasactios, whe the parameters eeded i the miig process are give i liguistic terms. Items are also evaluated by maagers as liguistic terms to reflect their importace, which are the trasformed as fuzzy sets of weights. Fuzzy operatios are the used to fid weighted fuzzy large itemsets ad fuzzy associatio rules. A example is give to clearly illustrate the proposed approach. I. INTRODUCTION Amog the miig techiques, it is most commoly see i applicatios to iduce associatio rules from trasactio data. Most previous studies h oly show how biary valued trasactio data may be hadled. Trasactios with quatitative values are, however, commoly see i real-world applicatios. Srikat ad Agrawal proposed a method for miig associatio rules from trasactios with quatitative ad categorical attributes [3]. Their proposed method first determied the umber of partitios for each quatitative attribute, ad the mapped all possible values of each attribute ito a set of cosecutive itegers. Recetly, the fuzzy set theory has bee used more ad more frequetly i itelliget systems because of its simplicity ad similarity to huma reasoig [36][37]. The theory has bee applied i fields such as maufacturig, egieerig, diagosis, ad ecoomics, amog others [7][23][25][36]. Several fuzzy learig algorithms for iducig rules from give sets of data h bee desiged ad used to good effect with specific domais [5][7][3][6] [8]-[2][29][3][32]. Strategies based o decisio trees were proposed i [9][][2][27]-[29][33][34], ad based o versio spaces were proposed i [3]. Fuzzy miig approaches were proposed i [8][22][24][35]. Besides, most covetioal data-miig algorithms set the miimum supports ad miimum cofideces at umerical values. Liguistic miimum support ad miimum cofidece values are, however, more atural ad uderstadable for huma beigs. I this paper, we thus exted our previous fuzzy miig algorithm [22] for quatitative trasactios to the miig problems with liguistic miimum support ad miimum cofidece values. Also, items may h differet importace, which is evaluated by maagers or experts as liguistic terms. A ovel miig algorithm is the proposed to fid weighted liguistic associatio rules from quatitative trasactio data. II. REVIEW OF MINING ASSOCIATION RULES The goal of data miig is to discover importat associatios amog items such that the presece of some items i a trasactio will imply the presece of some other items. To achieve this purpose, Agrawal ad his co-workers proposed several miig algorithms based o the cocept of large itemsets to fid associatio rules i trasactio data []-[4]. They divided the miig process ito two phases. I the first phase, cadidate itemsets were geerated ad couted by scaig the trasactio data. If the umber of a itemset appearig i the trasactios was larger tha a pre-defied threshold value (called miimum support), the itemset was cosidered a large itemset. Itemsets cotaiig oly oe item were processed first. Large itemsets cotaiig oly sigle items were the combied to form cadidate itemsets cotaiig two items. This process was repeated util all large itemsets had bee foud. I the secod phase, associatio rules were iduced from the large itemsets foud i the first phase. All possible associatio combiatios for each large itemset were formed, ad those with calculated cofidece values larger tha a predefied threshold (called miimum cofidece) were output as associatio rules. Srikat ad Agrawal the proposed a miig method [3] to hadle quatitative trasactios by partitioig the possible values of each attribute. Hog et al. proposed a fuzzy miig algorithm to mie fuzzy rules from quatitative data [22]. They trasformed each quatitative item ito a fuzzy set ad used fuzzy operatios to fid fuzzy rules. Cai et al. proposed weighted miig to reflect /2/$. 22 IEEE

2 differet importace to differet items [6]. Each item was attached a umerical weight give by users. Weighted supports ad weighted cofideces were the defied to determie iterestig associatio rules. Yue et al. the exteded their cocepts to fuzzy item vectors [35]. The miimum supports ad miimum cofideces set i the above methods were umerical. I this paper, these parameters are expressed i liguistic terms, which are more atural ad uderstadable for huma beigs. III. THE PROPOSED ALGORITHM I the proposed algorithm, the fuzzy cocepts are used to represet item importace, item quatities, miimum supports ad miimum cofideces. Each attribute uses oly the liguistic term with the maximum cardiality i the miig process. The umber of items is thus the same as that of the origial attributes, makig the processig time reduced [22]. Details of the proposed miig algorithm are described below. The Algorithm: INPUT: A set of quatitative trasactio data, a set of m items with their importace evaluated by d maagers, four sets of membership fuctios respectively for item quatities, item importace, miimum support ad miimum cofidece, a pre-defied liguistic miimum support value α, ad a pre-defied liguistic miimum cofidece value β. OUTPUT: A set of weighted fuzzy associatio rules. STEP : Trasform each liguistic term of importace for item A, m, which is evaluated by the k-th maager ito a fuzzy set Wk of weights, k d, usig the give membership fuctios of item importace. STEP 2: Calculate the fuzzy rage weight W of each item A by fuzzy additio as: STEP 3: d W = W k d k =. Trasform the quatitative value V i of each item A i each trasactio datum D i ( to, = to m), ito a fuzzy set f i represeted as: f ( R i fi 2 f + + L + R R by usig the give membership fuctios for item quatities, where h is the umber of regios for A, R l is the l-th fuzzy regio of A, l h, ad f il is V i s fuzzy membership value i regio R l. STEP 4: Calculate the cout of each fuzzy regio R l i 2 ih h ) the trasactio data as: cout =. l f il STEP 5: Fid max cout = max( cout ), for = to h l= m, where m is the umber of items. Let max-r be the regio with max-cout for item A. max-r is the used to represet the fuzzy characteristic of item A i later miig processes for savig computatioal time. STEP 6: Calculate the fuzzy weighted support wsup of each item A as: max R W wsup =, where is the umber of trasactios. STEP 7: Trasform the give liguistic miimum support value α ito a fuzzy set (deoted misup) of miimum supports, usig the give membership fuctios for miimum supports. STEP 8: Calculate the fuzzy weighted set (wmisup) of the give miimum support value as: wmisup = misup (the gravity of I ), where I u I t t= =, u with u beig the total umber of membership fuctios for item importace ad I t beig the t-th membership fuctio. I thus represets the fuzzy rage weight of all possible liguistic terms of importace. STEP 9: Check whether the weighted support (wsup ) of each item A is larger tha or equal to the fuzzy weighted miimum support (wmisup) by fuzzy rakig. Ay fuzzy rakig approach ca be applied here as log as it ca geerate a crisp rak. If wsup i is equal to or greater tha wmisup, put A i the set of large -itemsets L. STEP : Set r =, where r is used to represet the umber of items kept i the curret large itemsets. STEP : Geerate the cadidate set C r+ from L r i a way similar to that i the apriori algorithm [4]. That is, the algorithm first ois L r ad L r assumig that r- items i the two itemsets are the same ad the other oe is differet. It the keeps i C r+ the itemsets, which h all their sub-itemsets of r items existig i L r. STEP 2: Do the followig substeps for each ewly formed (r+)-itemset s with items (s, s 2,, s r l /2/$. 22 IEEE

3 i C r+ : (a) Fid the weighted fuzzy set (Wf is ) of s i each trasactio data D i as: r+ Wfis = Mi (W f = s is ), where f is is the membership value of regio i D i ad Ws is the rage fuzzy weight for s. (b) Calculate the fuzzy weighted support (wsup s ) of itemset s as: wsup s Wfis =, where is the umber of trasactios. (c) Check whether the weighted support (wsup s ) of itemset s is greater tha or equal to the fuzzy weighted miimum support (wmisup) by fuzzy rakig. If wsup s is greater tha or equal to wmisup, put s i the set of large (r+)-itemsets L r+. STEP 3: IF L r+ is ull, the do the ext step; otherwise, set r = r + ad repeat Steps to 3. STEP 4: Trasform the give liguistic miimum cofidece value β ito a fuzzy set (micof) of miimum cofideces, usig the give membership fuctios for miimum cofideces. STEP 5: Calculate the fuzzy weighted set (wmicof) of the give miimum cofidece value as: wmicof = micof (the gravity of I ), where I is the same as that calculated i Step 9. STEP 6: Costruct the associatio rules from each large weighted q-itemset s with items (s, s 2,, s q ), q 2, usig the followig substeps: (a) Form all possible associatio rules as follows: sλ... Λs Λs + Λ... Λsq s, = to q. (b) Calculate the weighted cofidece value wcof R of each possible associatio rule R as: couts wcof R = Ws, cout where W s cout q s s q = s ( Mi fis ) = s = MiW. i ad (c) Check whether the weighted cofidece wcof R of associatio rule R is greater tha or equal to the fuzzy weighted miimum cofidece wmicof by fuzzy rakig. If wcof R is greater tha or equal to wmicof, keep rule R i the iterestig rule set. STEP 7: For each rule R with weighted support wsup R ad weighted cofidece wcof R i the iterestig rule set, fid the liguistic miimum support regio S i ad the liguistic miimum cofidece regio C with wmisup i- wsup R < wmisup i ad wmicof - wcof R < wmicof by fuzzy rakig, where: wmisup i = misup i (the gravity of I ), wmicof = micof (the gravity of I ), misup i is the give membership fuctio for S i ad micof is the give membership fuctio for C. Output rule R with liguistic support value S i ad liguistic cofidece value C. The rules output after step 7 ca the serve as liguistic kowledge cocerig the give trasactios. Membership value IV. AN EXAMPLE I this sectio, a example is give to illustrate the proposed data-miig algorithm. This is a simple example to show how the proposed algorithm ca be used to geerate weighted fuzzy associatio rules from a set of quatitative trasactios. The data set icludes six quatitative trasactios, as show i Table I. TABLE I THE DATA SET USED IN THIS EXAMPLE TID ITEMS (A, 4), (B, 4), (E, 9) 2 (B, 3), (C, 5), (F, 3) 3 (B, 2), (C, 3), (D, 2), (E, 8) 4 (A, 7), (C, 7), (E, 9) 5 (C, 2), (D, 2), (F, ) 6 (A, 4), (B, 3), (C, 5), (F, 2) Also assume that the fuzzy membership fuctios for item quatities are the same for all the items ad are show i Figure. Low 6 Figure : The membership fuctios for item quatities Item quatity The importace of the items is evaluated by three maagers as show i Table II /2/$. 22 IEEE

4 TABLE II THE ITEM IMPORTANCE EVALUATED BY THREE MANAGERS MANAGER ITEM A B C D E F MANAGER Very Assume the membership fuctios for item importace are give i Figure 2. Very Membership value Figure 2: The membership fuctios of item importace The liguistic terms for item importace give i Table II are trasformed ito fuzzy sets by the membership fuctios i Figure 2. The rage fuzzy weights of all the items are calculated, with results show i Table III. TABLE III MANAGER MANAGER 3 Very Very Weight THE AVERAGE FUZZY WEIGHTS OF ALL THE ITEMS ITEM A B C D E F The quatitative values of the items i each trasactio are represeted by fuzzy sets. The step is repeated for the other items, ad the results are show i Table IV, where the otatio item.term is called a fuzzy regio. The scalar cardiality of each fuzzy regio i the trasactios is calculated as the cout value. The fuzzy regio with the highest cout amog the three possible regios for each item is foud. Thus is chose for A, Low is chose for B, is chose for C, Low FUZZY WEIGHTED SUPPORT (.,.94,.278) (.233,.333,.4) (.28,.333,.458) (,.44,.) (.33,.2,.267) (.33,.33,.233) TABLE IV THE FUZZY SETS TRANSFORMED FROM THE DATA IN TABLE I TID FUZZY SETS.4.6 (, ( A. Low A. (.4.6 E. E..4.6, B. Low B. (.6.4, (.2.8, B. Low B. C. Low C..6.4 ( F. Low F. (.6.4, (.6.4, B. Low B. C. Low C. (.8.2, (.6.4 D. Low D. E. E. (.8.2, (.8.2, A. A. C. C. (.4.6 E. E. ( , (, C. Low C. D. Low D. ( F. Low F. ( , (, A. Low A. B. Low B. (.2.8, (.8.2 C. Low C. F. Low F. is chose for D, is chose for E ad Low is chose for F. The umber of item.regios is thus the same as that of the origial items, makig the processig time reduced. The fuzzy weighted support of each item is the calculated, with results for all the items show i Table V. TABLE V THE FUZZY WEIGHTED SUPPORTS OF ALL THE ITEMS ITEM A B C D E F AVERAGE FUZZY WEIGHT (.333,.583,.833) (.583,.833, ) (.47,.667,.97) (,.67,.47) (.5,.75, ) (.83,.333,.583) Assume the membership fuctios for miimum supports are give i Figure 3, which are the same as those i Figure 2. Also assume the give liguistic miimum support value is Low. It is the trasformed ito a fuzzy set of miimum supports, (,.25,.5), accordig to the give membership fuctios i Figure /2/$. 22 IEEE

5 Membership value Very Low Low Very Figure 3: The membership fuctios of miimum supports The gravity of I is calculated as.5. The fuzzy weighted set of miimum supports for Low is the (,.25,.5).5, which is (,.25,.25). The fuzzy weighted support of each item is the compared with the fuzzy weighted miimum support by fuzzy rakig. Ay fuzzy rakig approach ca be applied here as log as it ca geerate a crisp rak. Assume the gravity rakig approach is adopted i this example. A., B.low, C., E. ad F.Low are thus large weighted -itemsets. These -itemsets are put i L. (A., C.), (A., E.) ad (B.Low, C.) are the foud to be large weighted 2-itemsets. They are the put i L 2. I this example, L 3 is empty. The give liguistic miimum cofidece value is the trasformed ito a fuzzy set of miimum cofideces. Assume the membership fuctios for miimum cofidece values are the same as those i Figure 3. Also assume the give liguistic miimum cofidece value is. It is the trasformed ito a fuzzy set of miimum cofideces, (.25,.5,.75), accordig to the give membership fuctios. The fuzzy weighted set of miimum cofideces for is the (.25,.25,.375). The possible associatio rules from each large weighted itemset are costructed. The weighted cofidece values for the above possible associatio rules are the calculated. The weighted cofidece of each associatio rule is compared with the fuzzy weighted miimum cofidece by fuzzy rakig. I this example, six rules are output. Oe of them, for example, is show as follows: If a low umber of item B is bought The a middle umber of item C is bought, with a low support ad a very high cofidece. V. CONCLUSION Miimum support I this paper, we h proposed a ew weighted data-miig algorithm for fidig iterestig weighted associatio rules with liguistic supports ad cofideces from quatitative trasactios. Items are evaluated by maagers as liguistic terms, which are the trasformed ad raged as fuzzy sets of weights. Fuzzy operatios icludig fuzzy rakig are used to fid large weighted itemsets ad associatio rules. Compared to previous miig approaches, the proposed oe directly maages liguistic parameters, which are more atural ad uderstadable for huma beigs. Refereces [] R. Agrawal, T. Imieliksi ad A. Swami, Miig associatio rules betwee sets of items i large database, The 993 ACM SIGMOD Coferece, Washigto DC, USA, 993. [2] R. Agrawal, T. Imieliksi ad A. Swami, Database miig: a performace perspective, IEEE Trasactios o Kowledge ad Data Egieerig, Vol. 5, No. 6, 993, pp [3] R. Agrawal, R. Srikat ad Q. Vu, Miig associatio rules with item costraits, The Third Iteratioal Coferece o Kowledge Discovery i Databases ad Data Miig, Newport Beach, Califoria, August 997. [4] R. Agrawal ad R. Srikat, Fast algorithm for miig associatio rules, The Iteratioal Coferece o Very Large Databases, 994, pp [5] A. F. Blishu, Fuzzy learig models i expert systems, Fuzzy Sets ad Systems, Vol. 22, 987, pp [6] C. H. Cai, W. C. Fu, C. H. Cheg ad W. W. Kwog, Miig associatio rules with weighted items, The Iteratioal Database Egieerig ad Applicatios Symposium, 998, pp [7] L. M. de Campos ad S. Moral, Learig rules for a fuzzy iferece model, Fuzzy Sets ad Systems, Vol. 59, 993, pp [8] K. C. C. Cha ad W. H. Au, Miig fuzzy associatio rules, The 6th ACM Iteratioal Coferece o Iformatio ad Kowledge Maagemet, 997. [9] R. L. P. Chag ad T. Pavliddis, Fuzzy decisio tree algorithms, IEEE Trasactios o Systems, Ma ad Cyberetics, Vol. 7, 977, pp [] M. S. Che, J. Ha ad P. S. Yu, Data miig: a overview from a database perspective, IEEE Trasactios o Kowledge ad Data Egieerig, Vol. 8, No.6, 996, pp [] C. Clair, C. Liu ad N. Pissiou, Attribute weightig: a method of applyig domai kowledge i the decisio tree process, The Seveth Iteratioal Coferece o Iformatio ad Kowledge Maagemet, 998, pp [2] P. Clark ad T. Niblett, The CN2 iductio algorithm, Machie Learig, Vol. 3, 989, pp [3] M. Delgado ad A. Gozalez, A iductive learig procedure to idetify fuzzy systems, Fuzzy Sets ad Systems, Vol. 55, 993, pp [4] A. Famili, W. M. She, R. Weber ad E. Simoudis, "Data preprocessig ad itelliget data aalysis," Itelliget Data Aalysis, Vol., No., 997. [5] W. J. Frawley, G. Piatetsky-Shapiro ad C. J. Matheus, Kowledge discovery i databases: a overview, The AAAI Workshop o Kowledge Discovery i Databases, 99, pp [6] A.Gozalez, A learig methodology i ucertai ad imprecise eviromets, Iteratioal Joural of Itelliget Systems, Vol., 995, pp [7] I. Graham ad P. L. Joes, Expert Systems Kowledge, Ucertaity ad Decisio, Chapma ad Computig, Bosto, 988, pp [8] T. P. Hog ad J. B. Che, "Fidig relevat attributes ad membership fuctios," Fuzzy Sets ad Systems, Vol.3, No. 3, 999, pp [9] T. P. Hog ad J. B. Che, "Processig idividual fuzzy attributes for fuzzy rule iductio," accepted ad to appear i Fuzzy Sets ad Systems. [2] T. P. Hog ad C. Y. Lee, "Iductio of fuzzy rules ad membership fuctios from traiig examples," Fuzzy Sets ad Systems, Vol. 84, 996, pp [2] T. P. Hog ad S. S. Tseg, A geeralized versio space learig algorithm for oisy ad ucertai data, IEEE Trasactios o /2/$. 22 IEEE

6 Kowledge ad Data Egieerig, Vol. 9, No. 2, 997, pp [22] T. P. Hog, C. S. Kuo ad S. C. Chi, "Miig associatio rules from quatitative data", Itelliget Data Aalysis, Vol. 3, No. 5, 999, pp [23] A. Kadel, Fuzzy Expert Systems, CRC Press, Boca Rato, 992, pp [24] C. M. Kuok, A. W. C. Fu ad M. H. Wog, "Miig fuzzy associatio rules i databases," The ACM SIGMOD Record, Vol. 27, No., 998, pp [25] E. H. Mamdai, Applicatios of fuzzy algorithms for cotrol of simple dyamic plats, IEEE Proceedigs, 974, pp [26] H. Maila, Methods ad problems i data miig, The Iteratioal Coferece o Database Theory, 997. [26] J. R. Quila, Decisio tree as probabilistic classifier, The Fourth Iteratioal Machie Learig Workshop, Morga Kaufma, Sa Mateo, CA, 987, pp [27] J. R. Quila, C4.5: Programs for Machie Learig, Morga Kaufma, Sa Mateo, CA, 993. [28] J. Rives, FID3: fuzzy iductio decisio tree, The First Iteratioal symposium o Ucertaity, Modelig ad Aalysis, 99, pp [29] R. Srikat ad R. Agrawal, Miig quatitative associatio rules i large relatioal tables, The 996 ACM SIGMOD Iteratioal Coferece o Maagemet of Data, Moreal, Caada, Jue 996, pp. -2. [3] C. H. Wag, T. P. Hog ad S. S. Tseg, Iductive learig from fuzzy examples, The fifth IEEE Iteratioal Coferece o Fuzzy Systems, New Orleas, 996, pp [3] C. H. Wag, J. F. Liu, T. P. Hog ad S. S. Tseg, A fuzzy iductive learig strategy for modular rules, Fuzzy Sets ad Systems, Vol.3, No., 999, pp [32] R.Weber, Fuzzy-ID3: a class of methods for automatic kowledge acquisitio, The Secod Iteratioal Coferece o Fuzzy Logic ad Neural Networks, Iizuka, Japa, 992, pp [33] Y. Yua ad M. J. Shaw, Iductio of fuzzy decisio trees, Fuzzy Sets ad Systems, 69, 995, pp [34] S. Yue, E. Tsag, D. Yeug ad D. Shi, Miig fuzzy associatio rules with weighted items, The IEEE Iteratioal Coferece o Systems, Ma ad Cyberetics, 2, pp [35] L. A. Zadeh, Fuzzy logic, IEEE Computer, 988, pp [36] L. A. Zadeh, Fuzzy sets, Iformatio ad Cotrol, Vol. 8, No. 3, 965, pp /2/$. 22 IEEE