Interpreting Association Rules in Granular Data Model via Decision Logic

Transcription

1 Interpreting Association Rules in Granular Data Model via Decision Logic Tsau. Young.("T. Y.") Lin Department of Computer Science San Jose State University San Jose, CA Abstruct - Based on machine oriented modeling, a formal theory of association rules bas been developed; the theory allow us to mining association rule mining by solving a set of linear inequality. Unfommately, these rules are un-interpreted in the sense, we cannot generate a proper formula using the originally given symbols (attribute values) to described these discovered rules. In this paper, we develop a theory to generate such formula of given symbols to interpret them. I. INTRODUCTION In the Foundation on Data Mining and Discovery Workshop [6], we view that data mining as a procedure that transforms (extracts or discovers from) data into pattems. In [7], we took the most conservative view, namely, mathematical deductions are the only acceptable reasoning in the procedure and call it deductive data mining. In other words, we treat data as the "axiom" and patterns as "interesting theorems" to be derived kom axioms mathematically. In classical statistics, a similar view has been taken, and is called descriptive statistics. Taking such a view, does not imply that we are restricting ourselves in utilizing possible means and tools, but imply that data miners need to identify clearly what are the bidden assumptions. With this view, data mining (we will focus on association rule mining) bas the following ingredients: (1) Raw Data: A relational table is a representation of a universe, a set of entities. It is the input data for data mining. From data mining point of view, it is merely a table of symbols; these symbols play the roles of semantics primitive (1.1,) From systems point of view, symbols are bits-andbytes. (1.2) From buman point of view, symbols denote or represent some pieces of real world phenomena, concepts or objects. (1.3) None of the human view are implemented in the system. (2) Patterns: In association rule mining, high frequency subtuples are patterns. (3) Interpretations: If the relational tables are represented by bitmaps, the pattern may not be comprehensible to human. As symbols are meaningful to human, we need to represent the patterns in (algebraic or logic) expressions of symbols. As symbols and the operators in the expressions are meaningful to human, formally, we say the (algebraic or logic) expressions of symbols are interpretations or explanations of the pattems. 11. GRANULAR DATA MODEL A. Relation, Bilmapsfrom Database Theory In setting up a database, a relational table is often viewed as a representation of a set of real world entities in terms of a family of attributes A ={A', A2,..., A"}. Let us set it up more formally: Let Dom(-) be the active domain, that is, it consists of current distinct values of an attribute. A knowledge representation is a map E +K; v + (ki,kz,...) that maps each entity to a tuple of attribute values. The set of K c Doin(A') x Dom(AZ) x... x Dom(A") of these tuple is called a bag relation. In the traditional relational theory, the independent variables play no roles. So we are forced to consider bag relations, because distinct entities may have the same representation. So, we use the notion of graph (v, f(v)), the pair of independent variable and function. It will be referred to as an information table or simply table. Io this format, we can avoid the bag theory; see Table 2.1 For convenience, we may also treat an attribute as a map, A': --f Dom(A'), Then, we can consider an equivalence relation defined as follows: vl = v2 iff A'(v,)= A~(J The equivalence relation will be denoted by Q'; note that the inverse image of an attribute value defmes a granule (an equivalence class). We will see this will play the key role in this paper. B. Bitmap lndexes and Granular Data Model (GDM Bitmap index is a common notion in database theory; the following example is taken from a popular database text ([3], PP 702) /04/ Cnnvriuht 7nn4 TFFF 51

2 Example: Suppose we are given a set of six entities, called the universe of entities. We represent them in six tuples; see Table 2.1. This relation consists of two attributes, F and G, of type integer and string respectively. Table 2.1 v4 v6 baz foo bar baz A bitmap index for the fmt attribute, F, would have three bitvectors. The fust, for value 30, is , because the first, second, and sixth tuple have F = 30. The other two, for 40 and 50, respectively, are and , A bmap index for second attribute, G, would also have three bit-vectors: , , and ; see middle column of Figure 2.1. Fi e = = = Foo = Bar = Baz = Next, we will interpret the bit in terms of set theory. A bit vector is a representation of a subset of. We will refer this subset as an elementary granule [5], and the attribute value the name of the elementary granule; Previously, we have called it elementary neighborhood [8], [9]. The bit vector, , of F = 30 says that the fust, second and sixth entities are selected, that is, the bit vector represents the elementary granule, {el, e2, e6), and 30 is its name. Similarly, the bit vector, , of F = 40 represents the elementary granule (e3, e5) and 40 is its name. Note that the thud column of Figure 2.1 gives partition F and G, Using Figure 2.1, we replace attribute values of Table 2.1 by their respective bit vectors or elementary granules, we get Table 2.2 (Bit Table) and Table 2.3 (GDM). Both Bit Table and GDM are information tables of granules. In other words, their attribute values are granules (equivalence class); one represent by bit vectors, another by standard subset notations. For computation, we will use Bit Table, while conceptual discussions, we will use subset notation, because subset notations give better mental pictures for granules. Table 2.3 Granular Data Model (GDM) el, e4 e2, e5 el, e4 e2, e5 e3, e6 We will denote the GDM by (, (F, G)). By abuse of language, we may use GDM for both bit and set notations. As the granule in GDM has no symbols (Foo, Bar, Baz, 30.40, 50) attached; each granule, either in bit or set representation, bear no human interpretations, so it is an information table with un-interpreted attribute values; we will refer to it as uninterpreted information table SYNTACTIC NAKIRE OF DATA AND PATIFRNS A Syntactic Properties of Relational Tables From DDM point of view, an information table is a table of symbols stored in computer system. No human perceived semantics of these symbols are implemented (stored). The symbols, foo, bar, baz, 30, 40, 50 in Table 2.1, similarly, the symbols (110001), (OOlOlO),..., (001001) of Table 2.2 and {el, e2, e6}, {e3, e5},...,{e3, e6) of Table 2.3 are stored symbols, no human perceived semantics of these symbols are stored. So it is easy to see the followings Proposition. Three information tables are isomorphic: Table 2.k Table 2.2 ETable 2.3. in the sense one table may convert to another by replacing one set of symbols, foo,...,50 by their respective bitmaps, (1 loool),...,(001001) or respective granules {el, e2, e6},...,{e3, e6). From DDM view, three tables are exactly the same, they will produce the same set of pattems, namely, we have the following [5] Theorem. Isomorphic information tables (bag relations) have isomorphic pattems. 58

3 This theorem implies that we can do data mining on either original table or GDM. B. (Un-)Intetpretotions ofdata and Pattems The symbols, foo, bar, baz, 30,40, 50 in Table 2.1 are the typical data in DDM; in AI, they are called semantic primitive, which means (1) they are operated by the system as primitive (undefined notion), and (2) the adjective semantics refer to the interpretations by human, One such interpretation are not implemented in the system The symbols, foo, bar,...,50 of Table 2.1. and their formulas have apparent semantics to human. We call them interpreted symbols, or expressions. The association rules found in TABLE 2.1 are interpreted pattems. On the other hands, the expressions (110001) and (OOlOOl), and {el, e2, e6) and {e3, e6} do not carry the apparent semantics as foo and 50 do. We call the bitmaps, granules and their respective formulas un-intetpreted symbols and formula. The pattems found in GDMare un-interpretedpattems. It needs a special translation table, such as Figure 2.1, to support the interpretations. We would like to stress here that the capability of Figure 2.1. is very limited; it only do token interpretation. In this paper, we will extend the the translation beyond the Figure 2.1. I. AlTIUBUTES, PARTITIONS AND GENERALIZED PAlTERNS Most of the results are recalling from [5], but from different prospects. An attribute is also called a feature; we will use them interchangeably. A. Understanding the limits OfAtfributes in DDM The theoretical intent of relational schema is to define the semantics (of a table) that are perceived by human. However, the system has never been able to implement it, even a good approximation. Suppose there is an attribute, COLOR, in a relational schema. The intent of creating such an attribute is to represent what human think about the concept of COLOR. So the values are, of course, yellow, blue, orange, and etc. However, DDM is looking in reverse way, Could the set of such values in the systems define the concept of COLOR the way human haveperceived? The answer is an obvious No. The question is: What is an attribute in DDM? In [5], we concluded an attribute and its values are a named partition and namedgranules. We should stress that those names are given by human, so cannot be automated. This explains why we have been only able to fmd un-interpreted pattems. We will formalize the interpretations. B. A Theory of Un-interpreted atfributes Recall that A ={A, A*,..., A ) is the family of attribute under considerations, ne power set 2~ forms a lattice, in fact a Boolean algebra. Previously we note that each attribute induces an equivalence relation, the join and meet operations of this lattice comespond to the intersection and mion of equivalence relations (partitions) respectively. Please note the Gist from the intuition. Let n(v) be the lattice of all partitions on, where join is the intersection of equivalence relations and meet is the union; where the union is the smallest coarsening of its components. As we have observed earlier that an attribute induces a partition (equivalence relation) on. Using T. T. Lee s notation, he, the set of induced partitions of 2A is called a relation lattice and observed that 1. The join operator in he (induced from 2A) is different from that of n(). 2. So he is a subset, but not a sublattice, of no. Such an embedding is an unnatural one, but Lee focused his efforts on it. We have, instead, taken the natural embedding. The smallest lattice generated by hi3 is called the (Lin s) relation lattice and denoted by L(Q), where Q ={Q, Q2,..., Q ) is the set of the partitions induced by A ={A, A2,..., An). Note that every element (a partition) in ImI3 has an interpretation (derived from A ={A, A2,..., A } ). Unfortunately, not every element in L(Q) is interpreted. The difference between L(Q) and ImI3 is that former contains all the join of distinct attributes. Many such joins have not been interpreted by human, so they are un-interpreted attributes. We need a formal language that can generate logic formula based on the terms given in the original table (e,g, Table 2.1.); this problem is addressed in next section The pair (, L(Q)) is the GDM of (Lin s) relation lattice Definition The smallest lattice containing all the coarsening of L(Q) is called the complete relation lattice, and is denoted by L*(e). C. DerivedAtfributes - Feature Constructions Theorem 5.1. (, L*(Q)) consists of all possible derived attributes of (, Q), that is, it contains all possible feature constructions. 59

4 Next few theorems have reported in ICDMOZ foundation of data mining workshop [6], [7]. lo terms of current language they are un-interpreted patterns. Theorem 5.2. Any granule G in a partition P E L(Q*), whose cardinality exceeds the support is an un-interpreted generalized pattems (generalized undirected association rules). Theorem 5.3. Let Lo be the smallest element in L*(Q). All possible unions of the Lo-granules that meet the threshold is an un-interpreted generalized pattem (generalized association rules). Let us set up some notations. Let the variables x,. x2, x3, xa take either zero or one. Let us defme an operation of binary number x and a set S. We write S*x to be defmed by The two equations S*x=S, ifx=l S*x=0, ifx=0 Every element of L*(Q) is a coarsening, say C, of G. Hence every granule in C is a union of some G-granules (granules in G); they are denoted by C,, C2...,C,. Let 1. 1 be the cardinality of the set. Then the following expression represents the cardinality of all possible unions of G-granules thatare 2 threshold. Theorem 5.4. Then the following union Cl*Xl U... U c,* x,. is a granule that represents a un-interpreted (undirected) association rule, if its cardinality (**) z ICiI*Xj t s where s is the threshold Remark The cardinal number of L*(Q) is bounded by the Bell number [l] of (GI, where G is the smallest partition and IC/ is the cardinality of the partition. The total number of derived attributes is very large. However the complexity of (**) is not too high. Let s be the threshold and (GI=g, then the possible ''minimal solutions" is bounded by the combination gcs. We will report the calculation on real world data in future report soon. DECISION - A LOGIC OF A TA~LE In association rule mining, the only pattems that have been mined are sub-tuples. If we want to mine complex patterns, there are needs for a language to express them. We decide to borrow decision logic for this purpose [15]. A. The Syntax I) The Alphabet a) A ={A', A2,..., A"} - The set of attribute names; these names are meaningful to human, but they are primitive in the formal system;, they are not defmed by other notions in the formal system. b) = U Dom(A') - The set of attribute values of A' E A; each attribute value is very meaningful to human, hut formally, it is a primitive. d) 2={-, A, v, +, )-The set of connectives (negation, and, or, implication, equivalence); these connective represents usual logical operators. 2) Formulas R The smallest set satisfies the following: a) Expressions of the form, attribute value pair < Aj, v > called atomic formulas any A' E A and v E dom(a). b) If cp and q are formulas, so are "cp, (cp A q), (cp v q), (cp + q) and their labels are C(-rp)=C(cp), C(cp)~C(q), C(cp)vC( q), -C(cpW( q), where v and A are lattice operations. B. The Semantic 1) Interpretations We will denote v I= cp or v I= cp when is understood, if an object v E satisfies a formula cp in. So we will say v I= cp, iff v (= <A, v> iff p(v, A) = U v )= "cp iff non v I= cp v I= (9 A q) iff v I= cp and v /= q v I= (cp v q) iffv I=cp orv /= q We have many usual formulas, such as v (= (9 + q) iff v I=-9 v q We associate the formula cp, the following set lcpl =(v:veandvi= cp}. It is called the meaning of cp. A formula is said to be hue if IcpI = ; cp is logically equivalenf to q iff their meahgs are the same, i.e., Icp Iv = 1qIv Using GDM's terminology, we have )< A', v>i = the elementary granule of attribute value < A', >. 60

5 I9 +9Iv =-191v "I'II" I -@ Iv = - I9 Iv In [9], we approach the DL-language informally we "quote" here the general form. Theorem 1. There is a one-to-one correspondence between each column of a relational table and a partition of the universe. 2. Each attribute value is defined by one and only one elementary granule (equivalence class) of a column 3. A logical formulas of attribute values is defined by and only by a set theoretical relationship among elementary granules of all columns. C. The Deductive System ofdecision Logic 1) Inference rules Modus ponens is the only rule. 2) Axioms (1) The set of propositional tautologies (2) Specific axioms: (a) < A', v> A <A', U > = 0 for any A' E A and v, U EandvSu (b) vf < A', v> : for every v E dom(aj) and for every A' E A) = 1 (c) - < A', v> E v( < Ai, U> : for every U E dom(a1) and every A' E A, v # U ) We need few auxiliary notations and results: Let 0 and 1 denote falsity and truth. Formula of the form <A1, vl> A <A2, v2> A,....< A", v.> is called P-basic formula or P-formula, where vi E dom (A'), and P E A. The specific Axiom (a) follows from the assumption that each entity can have exact one value in each attribute. The Axiom (h) implies that each value of its domain must be taken once. This is saying that dom(a) is the active domain of attribute A. The Axiom (c) allows us to get rid of the negation in such a way that instead of saying that an object does not possesses a given property we can say that it has one of the remaining properties. It implies the closed word assumption. Let Zv (P), or simply X (P) denote the disjunction of all P-basic formulas satisfied in. The closed word assumption can he express in the following: Proposition I= Z (P) = 1. For any P T. v v Note that commercial DBMS usually have this assumption. For example, the output of not red colour consists of all nonred colours. A formula is a theorem, denoted by I- 9, if it is derivable from the axioms. The set of theorems of decision logic is identical with the set of theorems of propositional calculus with specific axioms (a)- (c). I. GENERALIZED PATTERNS AND INTERPRETATIONS In last section, we have several theorems on un-interpreted pattems. We need some formal way that can interpret them in terms of the symbols (semantic primitives) in the original table (e,g, Table 2.1.) Let us first recall a theorem and comments from [9]; The formal theorem was referred to single column representation; hut it also had commented that "One should note that this theorem is valid, even when we consider a collection of partitions." So the theorem is valid for general cases; so we "quote" the general form here. Theorem There is a one-to-one correspondence between each column of a relational table and a partition of the universe. 2. Each attribute value is defined by one and only one elementary granule (equivalence class) of a column 3. A logical formulas of attribute values is defmed by and only by a set theoretical relationship among elementary granules of all columns. In [9], we refer these granules as machine semantics of attribute values and logical formulas respectively. Let us interpret the un-interpreted (undirected) association rule developed in Theorem 5.4. Each C,, Cz...,C, is a granule of G=Q'nQ2n...nQ". So, in terms of elementary granules of (Q', Q',...,Q"}, we have Name(Ci )= Name(gi') AName(g{) A... AName(g:) So we have the interpretations of Theorem 5.4 as Theorem 6.2 Then the un-interpreted patterns C]*Xl U... U I&* x, has the following interpretation (a formula of DL) Name(CI )*XI v Name(C2)*xz... v Name(C,J* x, where Name(Cj)= Name(gjl) AName(g2) A... AName(go) is the Name (.) are name of 0. The last formula is a formula of decision logic [15] and Cj is referred to as the meaning of its name. J J 61

6 U. CONCLUSIONS Data, pattems and interpretations are three important ingredients in (undirected ) association rule mining. In this paper, we formalize the current state of database mining: Data are the bare data; it is a table of symbols. The pattems are the repeated data. The results are somewhat surprising: 1. Patterns are properties of the isomorphic class, not an individual relation - This implies that the notion of pattems have not matured yet, also explains why there are so many extracted association rules. In fact, many of them have no real world meanings. We have the following two striking results: 2. All possible attributes (features) can be enumerated 3. Generalized associations can be found by solving integral linear inequalities. Unfortunately, the computation cost is enormous. This may signifies the current notion of data and pattems (implied by the algorithms) are too primitive. 4. Decision logic can be viewed as a systematic buman interpretation of data. In the current state of arts in association rule mining, a pattem is simply a repeated data that may have no real world meaning. So some semantic oriented data mining may have to be explored [9], [IO], [Ill REFERENCES [I] Richard A. Brualdi, Introductory Combinatorics, Prentice Hall, [2] John E. Freund, Modem Elemen- Statistics, Prentice Hall, 1952 [3] Hector Garcia-Molina, JeErey D. Ullman. Jennifer Widom, Database Systems-The Complete Book, Prentice Hall, 2002 [4] T. T. Lee, Algebraic Theory of Relational Databases, The Bell System Technical Joumal ol 62, No 10, December, 1983, pp [5] T. Y. Lin, Attribute (Feature) completion-- The Theory of Attributes from Data Mining Prospect, in: the Proceedings of Intemational Conference on Data Mining, Maebashi, Japan, Dec 9-12,2002, pp [6] T. Y. Lin, Mathematical Foundation of Association Rules--Mining Associations by Solving Integral Linear Inequalities. In: the Proceedings of the Workshop on the Foundation of Data Mining and Discovery, which is a part of Intemational Conference on Data Mining, Maebashi, Japan, Dec 9-12,2002, pp [7] T. Y. Lin, Deductive Data Mining: Mathematical Foundation of Database Mining, in: the Proceedings of 9 Intemational Conference, RSFDGrC 2003, Chongqing, China, May 2003, Lecture Notes on Artificial Intelligence LNAI 2639, Springer-erlag, [8] Tsau Young Lin, Data Mining: Granular Computing Approach. In: Methodologies for Knowledge Discovery and Data Mining, Lecture Notes in Artificial Intelligence 1574, Third Pacific-Asia Conference (PAKDD1999), Beijing, April 26-28, 1999, [9] Tsau Young Lin, Data Mining and Machine Oriented Modeling: A Granular Computing Approach, Joumal of Applied Intelligence, Kluwer, ol. 13, No 2, September / October,2000, pp [IO] T. Y. Lin, Data Mining: Granular Computing Approach. In: Methodologies for Knowledge Discovery and Data Mining, Lecture Notes in Artificial Intelligence 1574, Third Pacific-Asia Conference, Beijing, April 26-28,1999, [I I] T. Y. Lin, Granular Computing on Binary Relations I: Data Mining and Neighborhood Systems. In: Rough Sets In Knowledge Discovery, A. Skowom and L. Polkowski (eds), Springer-erlag, 1998, [ 121 T. Y. Lin, The Power and Limit of Neural Networks, Proceedings of the 1996 Engineering Systems Design and Analysis Conference, Montpellier, France, July 1-4, 1996, ol. 7, [13] T. Y. Lin and N. Cercone, Rough Sets and Data Mining: Analysis of Imprecise Data, T. Y. Lm and N. Cercone (eds), Kluwer Academic Publishers, 1997, 2000(2nd print) [14] T. Y. Lin, Deductive Data Mining: Mathematical Foundation of Database Mining, in: the Proceedings of 9 International Conference, RSFDGrC 2003, Chongqing, China, May 2003, Lecture Notes on Artificial Intelligence LNAI 2639, Springer-erlag, [15]. Z. Pawlak, Rough sets. Theoretical Aspects of Reasoning about Data, Kluwer Academic Publishers, 1991 [I61 A. Barr and E.A. Feigenbaum, The handbook of Artificial Intelligence, Willam Kauhann