Approximation Theories: Granular Computing vs Rough Sets

Transcription

1 Approximation Theories: Granular Computing vs Rough Sets Tsau Young ( T. Y. ) Lin Department of Computer Science, San Jose State University San Jose, CA tylin@cs.sjsu.edu Abstract. The goal of approximation in granular computing (GrC), in this paper, is to learn/approximate/express an unknown concept (a subset of the universe) in terms of a collection of available knowledge granules. So the natural operations are and and or. Approximation theory for five GrC models is introduced. Note that GrC approximation theory is different from that of rough set theory (RST), since RST uses or only. The notion of universal approximation theory (UAT) is introduced in GrC. This is important since the learning capability of fuzzy control and neural networks is based on UAT. Z. Pawlak had introduced point based and set based approximations. We use an example to illustrate the weakness of set based approximations in GrC. 1 Introduction Granular Computing (GrC) can be interpreted from three semantic views, namely, uncertainty theory, knowledge engineering (KE) and how-to-solve/compute-it. In this paper, we concentrate on the KE views: the primary goal of this paper is to develop and investigate the approximation theory that can approximate/learn/express an unknown concept (represented by an arbitrary subset of the universe) in terms of a set of basic units of available knowledge (represented by granules.) 2 Category Theory Based GrC Models It is important to note that the following definition is basically the same as the category model of relational databases that we proposed in 1990 [4]. This observation seems to say that the abstract structures of knowledge and data are the same. Let CAT be a given category. Definition 1. Category Theory Based GrC Model: 1. C = {Cj h,h,j,=1, 2,...} is a family of objects in the Category CAT, it is called the universe (of discourse). 2. There is a family of Cartesian products, C j 1 Cj 2... of objects, j =1, 2,... of various lengths. C.-C. Chan et al. (Eds.): RSCTC 2008, LNAI 5306, pp , c Springer-Verlag Berlin Heidelberg 2008

2 Approximation Theories: Granular Computing vs Rough Sets Each n-ary relation object R j, which is a sub-object of C j 1 Cj 2...Cj n, represents some constraint. 4. β = {R 1,R 2,...} is a family of n-ary relations (n could vary); so β is a family of constraints. The pair (C,β), called Category Theory Based GrC Model, is a formal definition of Eighth GrC Model. To specify the general category to the categories of functions, Turing machines and crisp/fuzzy sets, we have Sixth, Seventh and Fifth GrC models, respectively. In Fifth GrC model, by limiting n-ary relations to n = 2, we have Fourth GrC Models, which are information tables based on binary relations [8], [9]. By restricting the number of relations to one, we have Third GrC Model (Binary GrC Model). Again fromfifth GrC Model, we havesecond GrC Model (Global GrC Model) by requiring all n-ary relations to be symmetric. Note that a binary relation B defines a binary (right) neighborhood system as follows: p B(p) ={y mid (p.y) B}, By considering the collection of B(p) for all binary relations in the Fourth GrC Model, we have First GrC Model (Local GrC Model). 3 Approximations: RST vs GrC In this paper, we are taking the following view: A granule represents a (basic unit) of available knowledge. Based on this view, what should be the admissible operations? We believe and and or so we take intersection and union as basic operations. For technical considerations (for the infinitesimal granules, equivalently, topological spaces), we take finite intersections and unions of any subfamily as acceptable knowledge operations. We do believe a negation of a piece of available knowledge is not necessary a piece of available knowledge; so negation is not an acceptable operation. Note that the approximation in this sense is different from that of generalized RST, which is based on the sole operation or. (Is this a miss-interpretation of Pawlak s idea by the RST community? In classical RST, the intersections are not needed, since they are always empty.) Anyway, in practice generalized RST does not regard the and of two known concepts as a known concept. So strictly speaking, generalized rough set approximations are not concept approximations. GrC has eight models; each has its own approximation theory. We will provide a set of generic definitions here, then discuss the details for individual models. Definition 2. Three (point based) approximations (we will suppress point based in future discussions). Let β = C 1 be a granular structure (the collection of granules). Let G 1 be the collection of all possible finite operations of C 1 ; note that operations are either

3 522 T.Y. Lin finite intersections or point-wise finite intersections. Let G be a variable that varies through the collection G 1, then we define 1. Upper approximation: C[X] =β[x] ={p : G, such that, p G & G X }. 2. Lower approximation: I[X] =β[x] ={p : ag, such that, p G &G X}. 3. Closed set based upper approximation: [17] used closed closure operator. It applies closure operator repeatedly (for transfinite times) until the resultants stop growing. The space is called Frechet (V)-space or (V)-space. Cl[X] =X C[X] C[C[X]] C[C[C[X]]]...(transfinite). For such a closure, it is a closed set. The concept of approximations just defined is derived from topological spaces. For RST, they can also be defined as follows: Definition 3. Set based approximations 1. Upper approximation: C[X] =β[x] = {G : G, such that, G X }. 2. Lower approximation: I[X] =β[x] = {G : ag, such that, G X}. These definitions do not work as well for many GrC models. 4 Rough Set Theory (RST) Let us consider the easiest case first. Let U be a classical set, called the universe. Definition 4. Let β be a partition, namely, a family of subsets, called equivalence classes, that are mutually disjoint and their union is the whole universe U. Then the pair (U, β) is called RST GrC Model (0th GrC Model or Pawlak Model) The two definitions of approximations agree in RST, in this case G 1 partition plus. is the 5 Topological GrC Model Next, we consider the approximation theory of a special case in the First GrC Model(LocalGrCModel),namelythe classical topological space (U, τ), where τ is the topology.

4 Approximation Theories: Granular Computing vs Rough Sets 523 Definition 5. Topological GrC Model (0.5th GrC Model) is (U, τ). A topology τ is a family of subsets, called open sets, that satisfies the following (global version) axioms of topology: The union of any family of open sets is open and a finite intersectionsofopensetsisopen [1]. A subset N(p) in a topological space U is a neighborhood of p if N(p) contains an open set that contains p. Note that every point in this open set has regarded N(p) as its neighborhood. Such a point will be called the center of N(p) infirst GrC Model. The union of all such open sets is O(p) is the maximal open set in N(p). It is clear every point in O(p) regards N(p) as its neighborhood. So O(p) is the collection of center set. In First GrC, it is denoted by C(p). The topology can also be defined by neighborhood system. Definition 6. Topological GrC Model (0.5th GrC Model) is (U,T NS). Topological neighborhood system (TNS) is an assignment that associates each point p afamilyofsubsets, TNS(p), that satisfies the (local version) axioms of topology; see [1]. In this case topology is the family {TNS(p) pu}. 6 Second GrC Model Definition 7. Let β = C 1 = {F 1,F 2,...} be a family of subsets. Then the pair (U, β) is called Global GrC Model (2.nd GrC Model or Partial Covering Model). In this case G 1 is the family of all possible finite intersections of C 1. Theorem 1. The approximation space of Full Covering Model (point based) is a topological space. However, under rough set approximation, it is not a topological space. Let τ be the collection of all possible unions of G 1 (when C 1 is a full covering), then τ is a topology. Proposition 1. G 1 is a semi-group under intersection. The set based definitions may not be useful, for example, C[X] mayalwaysbe U if β is a topology. 7 First GrC Model Now, we will generalize this idea to First GrC Model. Let U and V be two classical sets. Let NS be a mapping, called neighborhood system(ns) NS : V 2 (P (U)), where P (X) is the family of all crisp/fuzzy subsets of X. 2 Y is the family of all crisp subsets of Y,whereY = P (U). In other words, NS associates each point p in V, a (classical) set NS(p) of crisp/fuzzy subsets of U. Such a crisp/fuzzy subset is called a neighborhood (granule) at p, andthesetns(p) is called a neighborhood system at p; notethatns(p) could be a collection of crisp sets or fuzzy sets.

5 524 T.Y. Lin Definition 8. First GrC Model: The 3-tuple (V,U,β) is called Local GrC Model, whereβ is a neighborhood system (NS). If V = U, the 3-tuple is reduced to a pair(u, β). In addition, if we require NS to satisfy the topological axioms, then it becomes a TNS. Let NS(p) be the neighborhood system at p. Let G(p) be the collection of all finite intersections of all neighborhoods in NS(p). Let G be a variable that varies through G(p). Definition 9. With such a G, the previous equations given above do define the appropriate notions of C[X], I[X], Cl[X] for First GrC Models. 7.1 Algebraic Structure of GrS Let N(p) represent an arbitrary neighborhood of NS(p). Let C N (p), called the center set ofn(p), consists of allthosepoints that haven(p) as its neighborhood. (Note that C N (p) is the maximal open set O(p) inn(p)). Now we will observe something deeper: The finite intersections of all neighborhoods in NS(p) is G(p). A hard question is: Do the intersections of neighborhoods at distinct points belong to G(p)? Proposition 2. The theorem of intersections 1. N(p) N(q) is in G(p) =G(q), iffc N (p) C N (q). 2. N(p) N(q) is not in any G(p) p, iffc N (p) C N (q) =. If we regard N(p) as a known basic knowledge, then we should define the knowledge operations: Let be the and of basic knowledge (a neighborhood). For technical reasons, is regarded as a piece of the given basic knowledge. Definition 10. New operations 1. N(p) N(q) =N(p) N(q), iffc N (p) C N (q). 2. N(p) N(q) =, iffc N (p) C N (q) =. The second property says that even though N(p) N(q) maynotbeequalto, it does not form a neighborhood, hence not a knowledge. 8 Third and Fourth GrC Model Let U and V be two classical sets. Each p V is assigned a subset B(p); intuitively, it is a basic knowledge (a set of friends or a neighborhood of positionsasinquantummechanics). p B(p) ={Y i, i =1,...} U Such a set B(p) is called a (right) binary neighborhood and the collection {B(p) p V } is called the binary neighborhood system (BNS).

6 Approximation Theories: Granular Computing vs Rough Sets 525 Definition 11. Third GrC Model: The 3-tuple (U, V, β), whereβ is a BNS, is called a Binary GrC Model. IfU = V, then the 3-tuple is reduced to a pair (U, β). Observe that BNS is equivalent to a binary relation(br): BR = {(p, Y ) Y B(p) andp V }. Conversely, a binary relation defines a (right) BNS as follows: p B(p) ={Y (p, Y ) BR}. So both modern examples give rise to BNS, which was called a binary granular structure in [8]. We would like to note that based on this (right) BNS, the (left) BNS can also be defined: D(p) ={Y p B(Y )} for all p V }. Note that BNS is a special case of NS, namely, it is the case when the collection NS(p) isasingletonb(p). So the Third GrC Model is a special case of First GrC Model. The algebraic notion, binary relations, in computer science, is often represented geometrically as graphs, networks, forest and etc. So Third GrC Model has captured most of the mathematical structure in computer science. Observe that BNS is a special cases of NS. So we have Definition 12. Let B be a BNS, then 1. B(p) B(q) =B(p) =B(q), iffc B (p) C B (q). 2. B(p) B(q) =, iffc B (p) C B (q) =. Note that B(p) B(q) may not be empty, but it is not a neighborhood of any point. Observe that in Binary GrC Model, two basic pieces of knowledge are either the same or the set theoretical intersection does not represent any basic knowledge. Next, instead of a single binary relation, we consider the case: β is a set of binary relations. It was called a [binary] knowledge base [8]. Such a collection naturally defines a NS. Definition 13. Fourth GrC Model: the pair (U, β), whereβ is a set of binary relations, is called a Multi-Binary GrC Model. This model is most useful in databases; hence it has been called Binary Granular Data Model(BGDM), in the case of equivalence relations, it is called Granular Data Model(GDM). Observe that Fourth GrC Model can be converted by a mapping say First Four, to First Model, and First GrC Model induces, say by Four First,to Fourth Model. So First and Fourth models are equivalent, however, the conversions are not natural, because, the two maps are not the inverse of each other.

7 526 T.Y. Lin 9 Fifth GrC Model Definition 14. Fifth GrC Model: 1. Let U = {Uj h,h,j,=1, 2,...} be a given family of classical sets, called the universe. Note that distinct indices do not imply the sets are distinct. 2. Let U j 1 U j 2... be a family of Cartesian products of various length. 3. A constraint is expressed by an n-ary relation, which is a subset R j U j 1 U j 2...Uj n. 4. The constraints are the collection β = {R 1,R 2,...} of n-ary relations for various n. The pair (U,β), called Relational GrC Model, is a formal definition of Fifth GrC Model. Note that this granular structure is the relational structure (without functions) in the First Order Logic, if n only varies through finite cardinal number. Higher Order Concept Approximations (HOCA). In Fifth GrC model, we consider the relations (subsets of product space) as basic knowledge, and any subset in a product space as a new concept. We will illustrate the idea in the following case: U j is either a copy of V or U. Moreover, in each product space, there is at most one copy of V, but no restrictions on the number of copies of U. If a Cartesian product has no V component, it is called U-product space. If there is one and only copy of V, it is called a product space with unique V. 1. u and u 1 is said to be directly related, if u and u 1 are in the same tuple (of arelationinβ), where u 1 couldbeanelementofu or V. 2. u and u 2 is said to be indirectly related, if there is a finite sequence u i,i = 1, 2,...,t such that (1) u i and u i+1 are directly related for every i, and(2) u = u 1 and u 2 = u t. 3. An element u U is said to be v-related (v V ), if u and v are directly or indirectly related. 4. v-neighborhood, U v, consists of all the u U that is v-related. Such a relational granular model (with unique V ) induces a map: B : V 2 U ; v U v, which is a binary neighborhood system(bns), where U v is a v-neighborhood in U, and hence induces a binary granular model (U, V, B). Next, we will consider the case U = V,then Definition 15. The high order approximations of Fifth GrC model is the approximations based on the v-neighborhood system. [Digression] In the case n=2, depending on the given relation is either V U or U V, the neighborhood systems so obtained is left neighborhood system or right neighborhood system. The algebraic notion, n-ary relations, in computer science, is often represented geometrically as hypergraphs, hyper-networks, simplicial complexes and etc.

8 Approximation Theories: Granular Computing vs Rough Sets Models in Other Categories Let us consider the category of differentiable, continuous or discrete real-valued functions (think of them as generalized fuzzy sets) on some underlying spaces (these spacescanbe differentiable manifolds, topologicalspaces,or classicalsets). In general any collection of functions can be a granular structure, but we will be more interested in those collection that have universal approximation property (for example, a Schauder base in a Banach space). In such case, the approximations are done under appropriate topology on functions spaces. For a category of Turing machines (algorithms), it is still unclear as how to define the concept approximations. 11 Future Directions 1) Higher Order Concept Approximations, we may consider v-direct-related neighborhood system. 2) Admissible operations in Granular Structure. For simplicity, we will consider the Global GrC Model (2nd GrC Model). In other words, β is a partial/full covering. In this paper, we have not introduced the admissible operations into GrS; GrS is represented by C 1, the admissible operations are carried in G 1. In this section, we will include the admissible operations into granular structure. Let A(GrS) be the algebraic structure generated by GrS using the admissible operations. The three approximations can be stated as follows: Let G be a variable that varies through A(GrS), then the same equations given in Section 3 will be used to define C[X], I[X] and Cl[X]. Based on this terminology, we say 1. RST-Based View. A(GrS) is a complete semi-group under union. 2. Topology Based View. A(GrS) is a topology (closed under finite intersection and general unions). This view is what we have adopted in this paper. Actually, what we hope is a bit more general. We would require only that A(GrS) to be a topology on a subset(= (A(GrS)), not necessarily the whole universe. 3. Complete Boolean Ring Based View. A(GrS) is an algebraic structure that is closed under intersections and unions (of any sub-family), but not the complement. 4. σ-ring based view. This is similar to the previous item, except that we restrict it to countable intersections and countable unions. 3) Set based Approximation Theory: Pawlak offered a set based approximation theory; in RST, both theories are the same. However, for other GrC model, they are different. In the case of Topological

9 528 T.Y. Lin GrC Model C[X] isalwaysequaltou; uninteresting. Now, we have noticed that some information will be lost in the set based approximation theory. Let us consider the non-reflexive and symmetric binary relational GrC Model (Second GrC Model). To be specific, we consider a finite universe U = {a, b, c}. LetΔ 1 be a binary neighborhood system defined by Δ 1 : U 2 U : a B 1 ; b B 2 ; c B 3, where {B 1,B 2,B 3 } are three distinct fixed subsets of U. Now, let us consider a new BNS, denoted by Δ 2, Δ 2 : U 2 U : b B 1 ; a B 2 ; c B 3. In fact, we could consider 6 (= 3!) cases. All these six BNS have the same covering. So these six BNS have the same set based approximation, and hence the same C[X], I[X], and Cl[X]. In other words, the set base approximation cannot reflect the SIX differences. Nieminen considered such approximation for tolerance relations [13], we have used point based notion [6]. It is implicitly in [3] as we have treated it as a generalization of topology. 4) Numerical Measure Based Approximation Theory: We will illustrate the idea from infinite RST: Let us consider a family of partitions on the real line. 1. The first partition P 1 consists of unit closed- open interval [n, n+1),where < n <. They form a partition of real line, 2. The second partition P 2 consists of 1/2 unit half closed-open intervals, [n.n+ (1/2)), The m-th partition P m consists of 1/(2 m ) unit half closed-open intervals, [n.n +(1/(2 m )). Let β be the family of the union of these families (m =1, 2,...). It is important to observe that β is a covering but not a partition, though every P m,m=1,... is a partition. Now we have the following universal approximation theorem: Let μ be the Lebesgue measure of real line. Definition 16. A subset X is a good concept, if for every given ɛ, we can find a finite set of granules such that μ(c[x] μ(i[x]) ɛ. Note that this is a variation of Pawlak s accuracy measure. Theorem 2. Every measurable set is a good concept. This is a universal approximation theorem. The learning capability of fuzzy control and neural network is based on such a theorem, see for exmaple [15].

10 References Approximation Theories: Granular Computing vs Rough Sets Kelley, J.: General Topology. American Book-Van Nostrand-Reinholdm (1955) 2. Lin, T.Y.: Neighborhood systems and relational database. In: Proceedings of CSC 1988, p. 725 (1988) 3. Lin, T.Y.: Neighbourhood systems and approximation in database and knowledge base systems. In: Proceedings of the Fourth International Symposium on Methodologies of Intelligent Systems (Poster Session), pp (1989b) 4. Lin, T.Y.: Relational Data Models and Category Theory (Abstract). In: ACM Conference on Computer Science 1990, p. 424 (1990) 5. Lin, T.Y.: Peterson Conjecture and Extended Petri nets. In: Fourth Annual Symposium of Parallel Processing, April, pp (1990) 6. Lin, T.Y.: Topological and Fuzzy Rough Sets. In: Slowinski, R. (ed.) Decision Support by Experience - Application of the Rough Sets Theory, pp Kluwer Academic Publishers, Dordrecht (1992) 7. Lin, T.Y.: Neighborhood Systems -A Qualitative Theory for Fuzzy and Rough Sets. In: Wang, P. (ed.) Advances in Machine Intelligence and Soft Computing, Duke University, North Carolina, vol. IV, pp (1997) ISBN: Lin, T.Y.: Granular Computing on binary relations I: data mining and neighborhood systems. In: Skoworn, A., Polkowski, L. (eds.) Rough Sets In Knowledge Discovery, pp Physica-Verlag (1998a) 9. Lin, T.Y.: Granular Computing on Binary Relations II: Rough set representations and belief functions. In: Skoworn, A., Polkowski, L. (eds.) Rough Sets In Knowledge Discovery, pp Physica-Verlag (1998b) 10. Lin, T.Y.: Granular Computing: Fuzzy logic and rough sets. In: Zadeh, L., Kacprzyk, J. (eds.) Computing with Words in Information/Intelligent Systems, pp Physica-Verlag (1999) 11. Lin, T.Y.: Chinese Wall Security Policy Models: Information Flows and Confining Trojan Horses. In: DBSec 2003, pp (2003) 12. Lin, T.Y., Sutojo, A., Hsu, J.-D.: Concept Analysis andweb Clustering using Combinatorial Topology. In: ICDM Workshops 2006, pp (2006) 13. Nieminen, J.: Rough Tolerance Equality and Tolerance Black Boxes. Fundamenta Informaticae 11, (1988) 14. Pawlak, Z.: Rough sets. Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht (1991) 15. Park, J.W., Sandberg, I.W.: Universal Approximation Using Radial-Basis-Function Networks. Neural Computation 3, (1991) 16. Polya, G.: How to Solve It, 2nd edn. Princeton University Press, Princeton (1957) 17. Sierpenski, W., Krieger, C.: General Topology (Mathematical Exposition No 7). University of Toronto Press (1952) 18. Spanier, E.H.: Algebraic Topology. McGraw Hill, New York (1966); (Paperback, Springer, December 6, 1994) 19. Zimmerman, H.: Fuzzy Set Theory and its Applications. Kluwer Academic Publisher, Dordrecht (1991) 20. Zadeh, L.A.: Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/ intelligent systems. Soft Computing 2, (1998) 21. Zadeh, L.A.: Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic. Fuzzy Sets and Systems 90, (1997)