Discovering Attribute Relationships, Dependencies and Rules by Using Rough Sets

Transcription

1 Discovering Attribute Relationships, Dependencies and Rules by Using Rough Sets Wojciech Ziarko Computer Science Department University of Regina Regina, SK., Canada S4S OA2 Ning Shan Computer Science Department University of Regina Regina, SK., Canada S4S OA2 Abstract This paper reviews the methodology of the application of the theory of rough sets to the problem ofknowledge discovery in databases. The methodology is based on the idea of information generalization, to look at the data at various levels of abstraction, followed by the discovery, analysis and simplification of significant data relationships, dependencies, fundamental factors and rules. 1 Introduction Knowledge discovery in databases is a relatively new research area based on earlier results in database theory, machine learning, statistics, rough sets theory, and other areas[2-61. The main problem of knowledge discovery research is how to turn low level information represented by data into generalized knowledge about some characteristics, or relationships occurring in data. For example, the specific relationship between energy, current and temperature of a physical system may be unknown and its identification may require the usage of some statistical techniques. Interesting relationships occurring in the database may assume the form of functional, or partial functional dependencies and their discovery, analysis, and characterization may call for methods of rough sets theory. Really new and interesting knowledge is often not accessible due to lack of proper tools. This paper is concerned with the application of some of the research results of the theory of rough sets[l] to the knowledge discovery problem. It is basically a survey of relevant results along with their interpretations in the context of knowledge discovery problems. The following subproblems of the knowledge discovery problem are identified and addressed in the paper: 2 1. The analysis of information on various levels of representation granularity, from fine to coarse representation. 2. The discovery and simplification of relationships among attributes. 3. The discovery and analysis of functional or partial functional dependencies among attributes. 4. The identification of fundamental factors and interactions among attributes affecting the dependencies. 5 The identification of maximally generalized rules characterizing the discovered dependencies. Information Representation We are interested in discovery and analysis of relationships, dependencies etc., which characterize objects belonging to a certain domain such as medical patients, states of a chemical process, or a physical system etc. The objects are represented through information about them acquired from sensors or by other means. The information is expressed in the form of attributes and their values. Since each object e is assigned a unique value of an attribute,.e.g, HEIGHT(e2) = 148.0, attributes can be perceived as functions assigning unique values to objects. Formally, an information system[l-71 can be defined as a triple S =< OBJ, A, {VAL,}aE~ > where OBJ is the universe of objects, A is the set of attributes and for each attribute a E A, VAL, is the domain of the attribute a. An information system can be viewed as a table, as shown for example in Table 1. In the Table 1, S is an abbreviation for /95$4.00O1995IEEE Proceedings of the 28th Hawaii International Conference on System Sciences (HICSS '95) 293

2 Table 1: An example of an information system Table 2: An generalized information cretized attributes system with dis- SIZE, H for HEIGHT, E for ENERGY, C for CURRENT and T for TEMPERATURE. For the purpose of knowledge discovery it is often useful to replace the original attribute values of an information system with more general categories, for example, corresponding to value ranges. Formally, it means that a new information system is created with new attributes which are formed by combining old attributes with some discrete-valued functions. The advantage of such a reduction of information precision is that it exposes some repetitive data patterns and regularities which are not visible with high precision data representations. Some previously unknown data dependencies may show-up only after an appropriate conversion of the high precision data into less precise representation. By varying the degree of data generalization it is possible to scan the database at various levels of abstraction to find some interesting patterns or relationships. At each level, the techniques of rough sets can then be used to discover and analyze significant patterns or relationships. In the rest of the paper we will assume that the original data has been generalized to reach a certain specific level of abstraction and we will focus on rough sets-based discovery and analysis of such data. For the purpose of illustration we will use the information system given in Table 2, which has been generalized from Table 1 by replacing the original attribute values with some discrete ranges. The question of how to optimally discretize the attribute values is, in general, not yet settled and it is a subject of on-going research. In most applications, the discretization procedure is based on domain knowledge. 3 Analysis of Attribute Relationships The collection of discretized information vectors represents a certain relationship between attributes. Every relationship among attributes A corresponds to a classification of objects of an information system into disjoint classes where objects belonging to the same class have the same attribute values. Therefore, the classification corresponding to the set of attributes A can be represented by an equivalence relation R(A) E OBJ x OBJ. Two relationships, one between attributes & and the other between attributes P of an information system are said to be equivalent if they produce the same classification of OBJ, that is if R(P) = R(Q). Based on the above observations, one can ask the following questions about the relationship between attributes A of an information system: Are there any redundant attributes in the representation of the relationship? Would that be possible to describe the relationship in a simpler way, resulting in a simpler information representation and better understanding of the physical nature of the relationship? What are the most significant attributes in the analyzed relationship? The question (1) can be answered by computing attribute reducts known in the rough sets theory. The reduct is a subset of attributes RED c A such that (a) R(RED) = R(A), i.e., RED produces the same classification of objects as the whole attribute collection A, and (b) for any a E RED, R(RED - {a}) # R(A), that is, reduct is minimal subset with respect to the property (a). For example, in the information system given in Table 2 one can identify the following reducts: REDI = {SIZE, HEIGHT, ENERGY} (1) RED2 = {SIZE, HEIGHT, CURRENT} (2) 294

3 Each reduct represents an alternative and simplified way of expressing the relationship between attributes of an information system. Consequently, it is sufficient to measure the values of reduct attributes only to uniquely determine the relationship between ~11 attributes of an object. Since obtaining values of some attributes may be more costly than of the others, one could choose the reduct of lowest total cost for determining whether a specific relationship holds for an object. For instance, to determine which relationship holds between state variables of a system given in Table 1, it suffices to measure SIZE, HEIGHT and ENERGY instead of taking all measurements, whereby reducing time and cost of information gathering when applying the knowledge discovered from the information system to classify new states. In general the problem of computation of all reducts has been shown to be exponential. The complexity of finding the lowest cost reduct is NP-hard. However, good suboptimal results are usually obtained by using the linear time procedure described in [a]. To deal with the question (2) of this section we can use the concept of core attributes. The core attributes are the ones which cannot be eliminated from A without affecting our ability to classify an object into a particular relationship category. For example, the core attributes set of the generalized information system given in Table 2 are SIZE and HEIGHT which means that they are fundamental factors of the relationship between SIZE, HEIGHT, ENERGY, CURRENT, and TEMPERATURE. In other words, the relationship class of an object cannot be determined without knowing SIZE and HEIGHT attribute values. An interesting property of core attributes is that they equal to the intersection of all reducts of an information system[l]. The core is a context sensitive notion. In a greatly overspecilied system with many redundant attributes, core may be empty. This means that eventually every attribute of the information system could be substituted by another one while preserving the classification of objects. In this sense, no attribute is the most important one as the classification information contributed by each given attribute is already present in the system in terms of values of other attributes. 4 Analysis of Dependencies An important aspect of the knowledge discovery problem is the discovery, analysis, and characterization of dependencies among attributes. The problem of discovery of attribute dependencies has been studied independently by many researchers[2,4]. A good review of recent results in this area can be found in [12]. In this article, we specifically focus on the method for discovering functional and partial functional dependencies using some results of rough sets methodology. As in the case of relationships, we can have different dependencies on different levels of information generalization. In fact, a strong functional dependency on high level of information granularity implies functional dependency on lowest levels, but not vice versa. In our research, we are interested not only in functional dependencies but also in a spectra of partial functional dependencies derived from the concept of rough set. Essentially, at a given level of information abstraction we ask the question whether there is any dependency between groups of attributes P and Q, where P G A and Q C A. To formalize the notion of partial functional dependency, we need to use some elementary notions of rough sets theory as presented below[l]. Let R(P) be an equivalence relation among objects of an information system as introduced in the previous section. The pair (OBJ, R(P)) will be called an approximation space[l]. Also, let R*(P) be the collection of equivalence classes of R(P). That is, elements of R*(P) are groups of objects having the same values of attributes belonging to P. The lower approximation in the approximation space (OBJ, R(P)), or alternatively the interior INT(Y) of an arbitrary subset Y E OBJ, is defined as the union of those equivalence classes of R( P) which are completely contained by Y, i.e., INT(Y) = u{x E R*(P) : X c Y} The lower approximation characterize objects which can be classified into Y without any uncertainty, based on the available information. The lower approximation INT(Y) is a deterministic part of Y approximation. Since the set of the attributes Q corresponds to the partitioning R*(Q) of the universe OBJ, then the degree K(P, Q) of the deterministic, or functional dependency in the relationship between attribute collections P and Q can be defined as a total relative size of lower approximations of classes of the partition R*(Q) in the approximation space (OBJ, R*(P)). That is, if R (Q) = {Yr,Yz,... Ym} then where curd is a set cardinality. Ii(P, Q) assumes values in the range [O,l] with K(P, Q) = 1 for functional 295

4 dependency and K(P, Q) = 0 when absolutely none of the values of Q attributes can be uniquely determined by values of P attributes. K(P, Q) can be interpreted as a proportion of such objects in the information system for which it suffices to know the values of attributes in P to determine the values of attributes in Q. In practice, we are most often interested in analyzing dependencies where Q contains only a single attribute. For example, it is easy to verify based on Table 2 that for P = {SIZE, HEIGHT, ENERGY, CURRENT} and Q = {TEMPERATURE} the degree of dependency of K(P, Q) is 1. This means that this dependency is functional whereas the dependency between P = (ENERGY, CURRENT) and Q = {TEMPERATURE} is only partially functional with K(P, Q) = 0.5. Any dependency discovered from an information system can subsequently be simplified to eliminate any redundant attributes, and analyzed to find out the relative significance of the attributes involved in such dependency. The simplification of dependencies is based on the concept of relative reduct of rough sets theory[l]. The relative reduct of the attribute collection P with respect to the dependency Ii(P, Q) is defined as a subset RED(P, Q) E P such that (a) K(RED(P,Q), Q) = K(P, Q), i.e., relative reduct preserves the degree of inter-attribute dependency, and (b) for any a E RED(P,Q), I((RED(P,Q) - {a}, Q) # K(P, Q), that is the relative reduct is a minimal subset with respect to the property (a). For example, one possible relative reduct with respect to dependency between P = {SIZE, HEIGHT, ENERGY, CURRENT) and Q = (TEMPERATURE) is RED(P, Q) = (HEIGHT, ENERGY}. This means that the discovered dependency can be characterized by less attributes leading to possible savings in information representation, better understanding of the nature of the dependency and stronger patterns. In general, a number of alternative reducts can be computed for each analyzed dependency and one of the lowest total cost can be selected to represent the discovered dependency. The dependency represented in the reduced form is usually more regular as it reflects a stronger data pattern. This is illustrated in Table 3. The complexities involved in computation of relative reducts, or minimum cost reduct are of the same nature as in the case of reducts described in previous section. Some more HEIGHT ENERGY TEMPERATURE Table 3: Reduced representation between P and Q of the dependency recent research results and algorithms to deal with this problem can be found in [14]. To find fundamental factors contributing to the discovered dependency the rough sets the notion of relative core can be used. The relative core set of attributes with respect to the dependency K(P, Q) is a subset CORE E P such that for all a E CORE, K(P - {a}, Q) # K(P,Q). In other words, CORE is the set of the essential attributes which cannot be eliminated from P without affecting the dependency between P and Q. For example, the core of the dependency between P = (SIZE, HEIGHT, ENERGY, CURRENT) and Q = {TEMPERATURE} is {HEIGHT}. That is, HEIGHT is the fundamental dependency factor and it is included in every relative reduct for this dependency[l]. Clearly, the relative core, as well as the core described in section 3, is also context sensitive and can be empty. This can be interpreted, as in previous section, as a case of highly overspecified system with superfluous information inputs. 5 Rule Discovery Discovering rules from data is one of the most important goals of knowledge discovery in databases. Many systems and approaches for rule induction have been used for rule or decision tree discovery[b61. Rules can be perceived as data patterns which represent relationships between attribute values of an information system. In the rough sets approach, we distinguish two kinds of rules: (1) certain, or deterministic rules, and (2) possible, or non-deterministic rules. In the extended Variable Precision Rough Sets (VPRS) model the probabilistic rules can be computed as we11[8]. The decision matrix method for finding all maximally general rules, as briefly presented below for finding deterministic or possible rules, can be also applied with some minor modifications to produce

5 VPRS-based probabilistic rules. The maximally general rules minimize the number of rule conditions, For that reason, we will refer to them as minimal rules. In what follow, we present the basics of the method for discovering all minimal rules (either possible or certain) based on the idea of decision matrix[9] and discernibility matrix[lo] introduced by Skowron. The approach to generation of rules, as presented here, is based upon the construction of a number of Boolean functions[9,10] from decision matrices. Let V denote a value of the attribute d E A and let IV1 denote the set of objects with this particular value of the attribute d, referred to as a decision attribute. The rules can be computed either with respect to lower approximation INT(IVI) or upper approximation U PP( ] VI) in the approximation space (OBJ, R(A - {d})). When the lower approximation is used then deterministic rules are obtained and nondeterministic rules are obtained from upper approximation. In either case, the computational procedure is the same, the only difference is in the target class (concept). Consequently, to describe the method we can assume, without loss of generality, that IV1 is an exact set, or not rough, i. e., IV/1 = INT(IVI) = UPP(IVI) A rule with respect to a value V of the decision attribute d is defined here as a set of attribute-value pairs r = {(ail = KI), (ai = via),.... (ain = Kn>) such that and A, = ( ail, ~2,.... ~in ) C A (3) as cond(r) - dec(r). The set of all objects supp(r) in the universe OBJ whose attribute values match the rule conditions is called the rule support. From the knowledge discovery perspective, the most important problem is finding all minimal rules. The minimal rules minimize the number of rule conditions, subject to constraints (3) and (4), and represent the strongest data patterns. Before we define the concept of a decision matrix we will assume some notational conventions. That is, we will assume that all objects belonging to /VI and all objects belonging to the complement of IV1 are separately numbered with subscript i (a = 1,2,...y) and j (j = 1,2,...p) respectively. To distinguish positive from negative objects we will use superscripts V and - V, for instance, er versus e3 for the class V. A decision matrix M(S) = (Mij) of an information system S =< OBJ, A, {VAL,},,A > with respect to value V of an attribute Mij = {(a, u(ey)) : u(ey) # u(e7 )) d is defined as The set Mij contains all attribute-value pairs (&rib&e, value) whose values are not identical between ey and ey. 3 In other words, Mij represents the complete information distinguishing ey from ejnv. For example, the entry Ml1 of the decision matrix given in Figure 1 reflects the fact that case el differs from the case e2 in values of attributes SIZE, ENERGY, and CURRENT. This values are SIZE = 1, ENERGY = 2 and CURRENT = 1 for the positive case ez. The set of minimal decision rules IBi I for a given object ev (i = 1,2,...y) is obtained by forming the Boolean expression sup(r) = {e E OBJ : A,.(e) = VP} 5 IV1 where V, = ( Vi1, L&t,... Vi, ). In other words, the rule is a combination of values of some attributes such that the set of all objects matching this combination is contained in the set of objects with the value of decision attribute equal to V. Traditionally, the rule r is denoted as an implication r : (ail = V~~)A(CQ = t&), A...A(u~, = k$::,) + (d = V) (4 where A and v are respectively generalized conjunction and disjunction operators[9,10]. The Boolean expression, called a decision function BY is constructed out of row i of the decision matrix, that is (Mil,Mia,... Mip), by formally treating each attribute-value pair occurring in the component Mij as a Boolean variable and then forming a Boolean conjunction of disjunctions of components belonging to each set Maj I = 1,2,... p). The set of attribute-value pairs occurring on the left The decision rules IB, ] are obtained by turning hand side of the rule r is referred to as the rule condi- such an expression into disjunctive normal form and tion part, denoted cond(r), and the right hand side is using the absorption law of Boolean algebra to simplify the decision part, dec(r), so the rule can be expressed it. The conjuncts, or prime implicants of the simplified

6 also be used to select the most interesting rules[l3]. The discussion of these methods, however, is beyond the scope of this article. A comprehensive presentation of statistical rule validation methods is given by Piatetsky-Shapiro in [ll]. 6 Summary and Conclusions Figure 1: Decision matrixfor TEMPERATURE = 1 of Table 2. decision function correspond to the minimal decision rules[9,10]. Similarly, by treating the complement of the class V as a target concept, a set of decision rules can be computed for each object of the class N V using the same approach. For example, for the class TEMPERATURE = 1 of the system shown in Table 2, we can compute and simplify the following decision functions as derived from the corresponding decision matrix presented in Fig. 1. B: = ((S,~)V(E,~)~(C,~))A((H,O)V(E,~)~(C,~)) h((e, 2) v (C, 1)) = (E, 2) v CC, 1) B: = ((H, 2) v CC, 1)) A ((S,O) v (H, 2) v CC, 1)) A((S, 0) v (X, 2) v (C, 1)) = (H, 2) v cc, 1) B; = ((S, 1) V (ff, 2)) A ((H, 2)) A ((ff, 2)) = (H, 2) B: = ((S, I) v (H, 2) v (E, 2) v (C, 1)) A((H, 2) V (E, 2) V CC, 1)) A((H, 2) V (E, 2) V (C, 1)) = (H, 2) V (E, 2) V (C, 1) 8; = ((E,Z)v(C,l))A((S,O)V(~,O)V(E,2)V(C,1)) A((S, 0) V (E, 2) V (C, 1)) = (E, 2) V (c, 1) The above collection of decision functions is equivalent to the following set of minimal rules for TEMPERATURE = 1. (ENERGY = 2) + (TEMPERATURE = 1 ) (CURRENT = 1) --t (TEMPERATURE = 1 ) (HEIGHT = 2) - (TEMPERATURE = 1 ) All the minimal rules for the decision class 0 can be computed in a similar way. As indicated earlier, the decision matrix method only enables one to pinpoint some potentially interesting data relationships, called rules for traditional reasons, whose general validity would have to be yet confirmed using methods from statistical theory. Rule int.erestingness criteria can In the preceding sections we attempted to sketch a general methodology for comprehensive knowledge discovery using some of the methods of rough sets theory. The discussed methodology is aimed at discovering different kinds of knowledge using the same consistent framework. The key issue in this approach is information generalization as a primary means of identifying data patterns. The other discussed issues are relationship, dependency, and rule discovery. For each of the issues a concrete discovery and analysis method is presented. The presented methods of data analysis have been implemented in various systems. A comprehensive, open set of system tools to encompass all aspects of the presented methodology, called KDD-R[15], h as recently been implemented at University of Regina. This UNIX-based system contains facilities for data preprocessing (discretization), data reduction, dependency analysis, rule computation, prediction and validation of results. Future papers will report the results of computational experiments in knowledge discovery conducted with the help of KDD-R. 7 Acknowledgment The research reported in this paper was supported in part by an operating grant from the Natural Sciences and Engineering Research Council of Canada. Many thanks to Gregory Piatetsky-Shapiro for the suggestions and references on statistical verification of rules. Authors are grateful to referees for providing insightful and stimulating comments. References [l] Pawlak, Z. Rough Sets: Theoretical Aspects of Reasoning About Data, Kluwer Academic Publishers, [2] Piatetsky-Shapiro, G. (ed.) Proc. of AAAI-93 Workshop on Knowledge Discovery in Databases, Washington D.C.,

7 [3] Ziarko, W. (ed.) Rough Sets, Fuzzy Sets and Knowledge Discovery, Springer-Verlag, [4] Piatetsky-Shapiro, G. and Frawley, W. J. (eds.) Knowledge Discovery in Databases, AAAI/MIT Press Hu, X., Cercone, N., J. Han An Attribute- Oriented Rough Set Approach for Knowledge Discovery in Databases, In Ziarko, W. (ed.) Rough Sets, Fuzzy Sets and Knowledge Discovery, Springer-Verlag, 1994, pp [14] Kryszkiewicz, M., Rybinski, H. Finding Reducts in Composed Information Systems, In Ziarko, W. (ed.) Rough Sets, Fuzzy Sets and Knowledge Discovery, Springer-Verlag, 1994, pp [15] Ziarko, W. and Shan, N. KDD-R: A Comprehensive System for Knowledge Discovery in Databases Using Rough Sets, Proceedings of the International Workshop on Rough Sets and Soft Computing, RSSC 94, San Jose, [6] Piatetsky-Sh a p iro, G. Knowledge Discovery in Databases: Progress and Challenges, In Ziarko, W. (ed.) Rough Sets, Fuzzy Sets and Knowledge Discovery, Springer-Verlag, 1994, pp.l-10. [7] Pawlak, Z. and Orlowska, E. Expressive Power of Knowledge Representation, International Journal of Man-Machine Studies, 20, 1984, pp [8] Katzberg, J. and Ziarko, W. Variable Precision Rough Sets with Asymmetric Bounds, In Ziarko, W. (ed.) Rough Sets, Fuzzy Sets and Knowledge Discovery, Springer-Verlag, 1994, pp [9] Ziarko, W. and Shan, N. A Method for Computing All Maximally General Rules in Attribute-Value Systems, Computational Intelligence: An International Journal, in print. [lo] Skowron, A. and Rauszer, C. The Discernibility Matrices and Functions in Information Systems, In Slowinski, R. (ed.) Intelligent Decision Support: Handbook of Applications and Advances of Rough Sets Theory, Kluwer, [ll] Piatetsky-Shapiro, G. Discovery, Analysis, and Presentation of Strong Rules, In Piatetsky- Shapiro, G. Frawley, W. (eds). Knowledge Discovery in Databases, AAAI/MIT Press, 1991, pp [12] Schlimmer, J.V. Using Learned Dependencies to Automatically Construct Su@cient and Feasible Editing Views, Proceeding of AAAI Workshop on Knowledge Discovery in Databases, Washington D.C., 1993, pp [13] Major, J. and Mangano, J. SelectingAmong Rules Induced from a Hurricane Database, Proceeding of AAAI Workshop on Knowledge Discovery in Databases, Washington D.C., 1993, pp