Dynamically Managing Schema Changes in a HDE Pitfalls and Possibilities

Transcription

1 Dynamically Managing Schema Changes in a HDE Pitfalls and Possibilities Working paper # Sudha Ram Department of Management Information Systems Eller School of Management, The University of Arizona Tucson, AZ Phone : (520) ram@bpa.arizona.edu G. Shankaranarayanan Management Information Systems Department Boston University School of Management 595 Commonwealth Avenue, Boston MA Phone: (617) Fax: (617) gshankar@acs.bu.edu

2 Abstract The schema of a database changes and evolves extensively even in an operational database. Managing schema evolution is an important problem in databases and the large body of literature reflects the extensive work on this topic. Managing schema evolution in a heterogeneous data environment (HDE) has however not been addressed at all. Further more automating schema evolution has received little attention. Schema evolution must be managed dynamically and in a HDE this process requires automated support. In this paper we first highlight the importance of managing schema evolution in a HDE. Specifically, we describe the need for managing schema evolution dynamically in a HDE and the need for automated support for managing it. We then propose a formal framework based on a semantic data model and its graph representation (SM-graph) for managing dynamic schema evolution in a HDE. We describe a mapping scheme to map a semantic model onto a SM-graph. The framework consists of a set of rules that define the consistent state of the SM-graph, a comprehensive taxonomy of schema changes based on the graph with a rigorous examination of the implications of each change, and a set of graph-based operators to incorporate the changes. The completeness and correctness of these operators is also proved. A set of graph algorithms supplements the framework and assists in managing the process. Using this framework we present a architecture for propagating the schema changes to the databases in the HDE to address database evolution as well. Finally, we examine the implications for completely automating the process of managing dynamic schema evolution and investigate the extent of automation possible. 2

3 Dynamically Managing Schema Changes in a HDE Pitfalls and Possibilities The schema of a database changes and evolves extensively even in an operational database. Managing schema evolution is an important problem in databases and the large body of literature reflects the extensive work on this topic. Managing schema evolution in a heterogeneous data environment (HDE) has however not been addressed at all. Further more automating schema evolution has received little attention. Schema evolution must be managed dynamically and in a HDE this process requires automated support. In this paper we first highlight the importance of managing schema evolution in a HDE. Specifically, we describe the need for managing schema evolution dynamically in a HDE and the need for automated support for managing it. We then propose a formal framework based on a semantic data model and its graph representation (SM-graph) for managing dynamic schema evolution in a HDE. We describe a mapping scheme to map a semantic model onto a SM-graph. The framework consists of a set of rules that define the consistent state of the SM-graph, a comprehensive taxonomy of schema changes based on the graph with a rigorous examination of the implications of each change, and a set of graph-based operators to incorporate the changes. The completeness and correctness of these operators is also proved. A set of graph algorithms supplements the framework and assists in managing the process. Using this framework we present a architecture for propagating the schema changes to the databases in the HDE to address database evolution as well. Finally, we examine the implications for completely automating the process of managing dynamic schema evolution and investigate the extent of automation possible. 1. Introduction A database schema is the description of the database [Elmasri and Navathe, 1994] and is the overall design of the database [Silberschatz et al. 1997]. Defining the database schema is an evolutionary activity requiring more than a single iteration to capture the complex world of reference represented by the database. Even after defining it, the database schema may continue to change and evolve [Zanilo et al. 1997]. Schema changes are triggered by changes to the real world, changes to user perceptions and requirements, and by changes stemming from the application domain. Managing schema evolution is the process of incorporating changes into the database schema without loss of existing data and with minimal modifications to applications. Sjoberg has shown that schema changes are as common during the operational phase as during the design phase in organizational databases [Sjoberg 1993]. In his study of a single relational database for a health management system he observed 65 changes to relations each involving the 3

4 creation or removal of a relation and 470 changes to attributes each involving the addition or removal of an attribute during a 5-month design period. The corresponding numbers during a 13- month operational period were 299 and 2324 respectively. These numbers highlight the extent of schema changes in an operational database. Besides health management the impact of schema changes has been described in other application domains such as office information systems [Gibbs and Tsichristzis 1983, King and McLeod 1985], personal database systems [Lyngbaek and McLeod 1984], design engineering databases [Afsarmanesh et. al. 1985, Batory and Kim 1985], and artificial intelligence systems [Kerschberg 1984, Stefik and Bobrow 1986]. An accepted method for managing schema changes is database reorganization. Part or the entire database must be taken off-line for reorganizing and is hence inaccessible to users during the maintenance period. The frequency of changes makes it impractical to reorganize after each change. If performed periodically, the database is inconsistent with its world of reference during the intervals between reorganizations. Schema evolution must hence be dynamically managed. Dynamic schema evolution is defined as the ability of the database to incorporate schema changes in a timely manner, without loss of existing data, and without significantly impacting database operations. Organizations today typically use an integrated collection of several databases creating a heterogeneous set. In this paper the term heterogeneous database environment (HDE) describes a set of databases that store different types of data, are managed by different types of data management software systems (RDBMS, OODBMS, ORDBMS, and flat-files managed by 3GL), may be distributed across geographically dispersed locations, and data-associations (relationships) exist both within and across these multiple databases. Typically, a global or a federated schema is used to conceptually describe the world of reference of the HDE and 4

5 underlying databases. A change to any one database in this set could trigger changes to other databases in the HDE causing cascading changes because of the inter-relationships that exist. Hence the number of schema changes in a HDE is much more compared to a single database. Also, individual databases may be added or deleted to/from the HDE triggering changes to the federated/global schema. Consider a large heterogeneous administrative database system for a university consisting of financial, research projects, employee, and student databases. When a space database (information on building(s) owned / leased / rented by the university, including office and classroom space) is added it triggers changes to the global schema as income/expenditure associated with each building needs to be tied in with financial data, research labs in buildings have to be linked to projects, and faculty/departments need to be tied in with buildings where they are housed. Given the number of changes and the cascading changes resulting from each, manually tracking and managing each change is difficult. Incomplete (and incorrect) tracking of changes would result in inconsistencies that affect the quality of data in the databases. It is necessary to automate the process to efficiently manage dynamic schema evolution in a HDE. A large body of literature on schema evolution exists reflecting the extensive work in this area. However, no one has examined schema evolution in HDEs. While automation has been suggested and recommended for managing schema evolution, the literature does not address the extent to which the process can be automated nor does it specify the requirements for automating it. In this paper we propose a theoretical framework for dynamic schema evolution in a HDE. Using this we examine the implications for automating dynamic schema evolution. There are two aspects that differentiate this work from the rest of schema evolution literature. First, this research is one of the first to address dynamic schema evolution in a HDE. The framework is 5

6 based on a semantic data model and is general enough to be applied to all types of databases and is not restricted to relational or object-oriented databases. We propose a three-tier architecture that incorporates this framework for propagating the schema change to the databases. Second, using this framework we examine the implications for automating dynamic schema evolution. We identify the requirements for it and the problems associated with complete automation. The rest of the paper is organized as follows. Section 2 describes the relevant literature, differentiates our contribution from the existing literature, and defines the scope of this paper. The semantic model used in this research and its mapping to a graph is described in section 3. The graph-based framework for dynamic schema evolution is described in section 4 and section 5 examines the implications for automation using this framework. Conclusions are presented in section Relevant Literature and Scope of the Paper Schema evolution research can be classified from two viewpoints: the research activity and the data model used. Based on the first three well defined but inter-related activities can be observed: core schema evolution, version management, and application management. Based on the second viewpoint three data models can be identified: the object-oriented model, the relational model, and conceptual models including ERM, SDM, and NIAM. We address each of these in turn to differentiate and to define the scope of our research. Managing core schema evolution implies two activities: (1) identifying all possible changes to a database schema and incorporating these changes into the schema while preserving its consistent state [Zanilo et al 1997] and (2) propagating the changes to the corresponding data in the database. The first activity is also referred to as managing the semantics of schema changes [Banerjee et al. 1987, Peters and Özsu 1997]. The second is termed as database evolution [Zanilo et al 1997] and as propagation of schema changes [Banerjee et al. 1987, Peters and Özsu 1997]. 6

7 For managing core schema evolution some research target only the evolution of specific (sub) levels of abstraction in the object model such as object class evolution [Penny and Stein 1987, Bjornstedt and Britts 1988], object type evolution [Skarra and Zdonik 1987], method evolution [Morsi et al 1994], and attribute evolution [Clamen 1994] instead of examining all possible schema changes. Others are more comprehensive such as object class and type evolution [Zicari 1991, Breche et al. 1995], and evolution of the complete object schema [Banerjee et al. 1987, Ngyuen and Rieu 1989, Lerner and Habermann 1990, Ra and Rundenstiener 1997, and Peters and Özsu 1997]. In all of the above, a list of possible schema changes is defined based on the object-oriented data model and operators to incorporate each change are also identified. To preserve the consistent state of the object schema the prevalent method is to identify a set of invariants or axioms that define the consistent state of the schema and to create a set of rules that preserve them [Penny and Stien 1987, Banerjee et al. 1987, Ngyuen and Reiu 1989, Peters and Özsu 1997]. Other methods include the use of view mechanisms where the change is simulated and tested for consistency before being realized [Bertino 1992, Breche et al. 1995, Ra and Rundenstiener 1997] and leveraging polymorphism in object databases [Liu et al. 1997]. In this paper we focus on core schema evolution. We comprehensively identify all possible schema changes based on the semantic model representation of the schema and define the consistent state of the HDE based on this representation. Methods to propagate schema changes to the database include conversion on one side, and screening, filtering, or a hybrid on the other. Conversion coerces the existing data in the database to immediately conform to the changed / evolved schema. There is only one version of the database and schema available at any time. Data associated with an object that is no longer part of the new/evolved schema is not accessible from this schema and is lost. Screening and 7

8 filtering methods for propagating schema changes are employed when multiple versions of the schema and data are maintained. Screening creates a new version of the schema along with a set of associated functions. When an application accesses this version of the schema, the functions coerce the data to conform to this schema version (delayed conversion). Filtering is similar to screening except it does not coerce the data. Instead, the functions or filters associated with each version map the data to conform to that version. If screening or filtering is used for database evolution then schema versioning is necessary and an application can continue to access the version of the schema it is dependent upon. In this paper we use conversion to propagate schema changes and do not address versioning or application management. Considering core schema evolution using the object model we restrict our examination to only those research papers that address schema evolution and propagation of changes. In Orion an object-oriented database system supporting schema evolution a comprehensive taxonomy of schema changes is defined and the semantics of schema changes are managed by a set of invariants [Banerjee et al. 1987]. The taxonomy and invariants are defined based on the class lattice. Change propagation is by screening. Nguyen and Rieu present a set of schema changes similar to that in Orion [Nguyen and Rieu 1989]. While the semantics for schema changes is similar to Orion, the user is permitted to decide between screening and conversion for change propagation. Lerner and Habermann use invariants and rules to manage the semantics of schema changes in OTGen [Lerner and Habermann 1990]. The invariants are used to define default transformations for each schema change. The schema changes create a transformation table that defines how modifications should be performed to the affected data instances. Screening is used for change propagation. Zicari proposes a set of schema change operations to be incorporated in O 2, an object-oriented database system and programming environment [Zicari, 1991]. O 2 8

9 distinguishes between a class and a type and every class is associated with a type that describes the structure of its instances. The objective is to define a minimal set of primitive operations that can be used to update the schema in O 2 while preserving its consistency. This research raises the issue of completely automating schema evolution and emphasizes the importance of involving the designer but does not explicitly address both. Change propagation in O 2 is by using views [Breche et al. 1995].Users are provided with conversion functions that transform the database objects to conform to the new schema. Views are treated as virtual schemas to allow the user to test before materializing changes. View mechanisms to assist object schema evolution by creating personal schemas from a base schema are described in the TSE system [Ra and Rundensteiner 1997]. It includes a comprehensive set of schema changes and changes are managed by rules built into the view generation algorithms one set of algorithms for each schema change. Changes are propagated to object instances using filtering. The types of evolutionary schema changes in a relational data model are limited (changes to attributes and changes to relations). Research in schema evolution using relational model does not explicitly address schema changes and their propagation. Some discuss language support for schema evolution by extending / modifying relational algebra [e.g. McKenzie & Snodgrass 1990, Roddick 1992]. Others examine schema versioning by proposing temporal support for data and metadata [Dadam & Teuhola 1987, Clifford & Croker 1987, Ariav 1991, Roddick 1991, Snodgrass 1995, De Castro et. al. 1997]. Methods for automatically restructuring the relational schema have also been described [Schneiderman and Thomas 1982, Shu 1987]. Semantic models such as Entity Relationship model (ERM) have been used for managing core schema evolution. A methodology for implementing incremental and reversible schema change operations using the ERM has been proposed by [Markowitz and Makowsky 1987]. This 9

10 research defines an ER-consistent relational schema as a relational schema with all its key and inclusion dependencies that can be represented by an ER model. The possible changes in an ERconsistent schema are the addition and deletion of relational schemes, and the modification of key and inclusion dependencies. ER-consistent state of the database is defined by state mappings and ER-calculus is used to define rules that preserve this state. It addresses schema changes but not change propagation. Roddick et al. describe a taxonomy for schema evolution based on the ERM [Roddick et al. 1993]. Schema changes are classified into 3 types: changes to entity classes, changes to relationships, and changes to attributes. The focus is on managing schema evolution in relational databases and does not address change propagation. The Farandole 2 model for schema evolution in object databases employs a semantic model similar to ERM [Andany et al.1991]. The model is used to represent the schema as a whole or a part of it (semantic context). The model is then represented as a graph and schema changes defined using this graph. Liu et al. address schema changes by translating them into changes to an ERM [Liu et al.1993]. The schema changes described are the changes to attributes (add / delete attributes), and changes to objects that result in changes to attributes changes. Methods for managing core schema evolution have been comprehensively addressed using the object data model and to a lesser extent using the relational model. However, these examine schema evolution in single stand-alone databases and do not examine HDEs. Using the object (or relational) model restricts their application to object (or relational) databases only. Research has also shown that semantic models can be used to represent object schemas besides relational schemas. Our focus in this paper is on dynamically managing core schema evolution including change propagation in a HDE. We do not address versions or application management. The framework proposed is based on a semantic data model that offers two advantages. First, the 10

11 semantic model can be used to represent the schema of any database regardless of its type. Second, the semantic model supports explicit modeling of complex real-world relationships, many of which are not supported by the relational data model and are hidden within object methods in the object-oriented model. Information about the real world can be explicitly represented using semantic models and more types of schema changes can be explicitly identified and managed. The key contributions of this paper are: (1) it provides a framework for dynamic schema evolution in a HDE that has implications in data warehouses and corporate intranets. (2) It describes an architecture that incorporates the framework to comprehensively and dynamically manage schema and database evolution in a HDE. (3) It examines the requirements for automating dynamic schema evolution and the extent of automation possible. 3. The Semantic Model and its Graph Representation The semantic data model used in this paper is an extension of the Entity Relationship Model (ERM) [P. Chen, 1976]. The Unifying Semantic Model (USM) enhances existing semantic models with constructs to represent constraints on relationships between sub-classes and to represent constraints between attributes [Ram 1995]. It also formally defines complex object classes such as Composites and Groups / Aggregates. The elementary classes of the USM include Simple Strong, Simple Weak, Interaction Weak, Subclasses, Composites, Groups, Aggregates, Attributes, and their Domains. Each of these is related to one or more of the classes using relationships. Types of relationships include interaction, subclass, composite, group, and aggregate besides attribute and domain relationships. 3.1 Graph Representation A graph representation of a semantic model schema is formally defined on a graph G. Elementary classes in the semantic model schema are mapped as nodes and the relationships 11

12 between classes as links in the graph. The semantic model can be represented as a graph (SMgraph) G = (N, L) where N is a set of nodes representing the different classes and L the set of links representing the relationships between classes. The set N consists of 9 types of nodes N E, N W, N I, N C, N A, N D, N Ct, N At, and N M. The set L consists of 9 types of links L I, L W, L S, L C, L A, L D, L Ct, L At, and L M. Let V E be the set of nodes belonging to type N E in the graph. Similarly let sets V W, V I, V C, V A, V D, V Ct, V At, and V M be the sets of nodes of types N W, N I, N C, N A, N D, N Ct, N At, and N M respectively. Also, let sets E I, E W, E S, E C, E A, E D, E Ct, E At, and E M be the sets of links in graph G belonging to link types L I, L W, L S, L C, L A, L D, L Ct, L At, and L M respectively. Set N = V E V W V I V C V A V D V Ct V At V M is a disjoint union of 9 sets of nodes, each set being a collection of nodes belonging to a particular node type. Set L = E I E W E S E C E A E D E Ct E At E M is a disjoint union of 9 sets of links, each set being a collection of links belonging a particular link type. Sets N and L are listed in tables 1 and 2. Node Type N E N W N I N C N A N D N Ct N At N M Describes Simple Strong Entity Class Simple Weak Entity Class Interaction Weak Entity Class Complex Class Attributes Domains Class Constraint Attribute Constraint Composite class Member Table 1: Description of Nodes in the SM-graph Link Type Directionality Describes L I Bi-directional Interaction Relationship L W Unidirectional to simple weak class Weak Class Relationship L S Unidirectional to subclass Subclass Relationship L C Unidirectional to complex class Complex Class Relationship L A Bi-directional Attributes property Relationship L D Bi-directional Domain Association L Ct Bi-directional Class Constraint Relationship L At Bi-directional Attribute constraint Relationship L M Unidirectional to member Member Relationship Table 2: Description of Links in a SM-graph 12

13 Each object in our semantic model schema would correspond to a node or a link in its corresponding SM-graph. The nodes and links in the SM-graph are identified by the same name as in the schema. To represent inter-attribute (inter-class) constraints, the definition of each constraint is captured as a node of type N At (N Ct ) and is linked to the participating attribute (entity class) node(s) using attribute (class) constraint links of type L At (L Ct ). The interaction relationship link (type L I ) requires attention because: (1) it has associated cardinality constraints that describe the interaction and may have attributes and (2) an interaction relationship may relate more than two entity classes in the model (a link in the graph can relate only two nodes). Attributes of an Interaction relationship and Cardinalities: For representing attributes of an interaction relationship (note: not all interaction relationships have attributes) we use the modeling construct offered by USM, the interaction entity class. The attributes of the interaction relationship are represented as attributes of the interaction entity class. We treat the cardinality as metadata associated with the interaction relationship. For dynamically managing schema evolution we need to maintain the data associated with the different objects in the semantic model such as the name of the object, its description, and its function in the model. All of this data is stored in a repository that serves as the repository of model metadata. The cardinality (metadata of an interaction relationship) is stored here and is not represented in the SM-graph. Interaction relationships with degree >2: Each interaction relationship with participation degree greater than two is represented in the SM-graph as a set of interaction links. For instance an interaction relationship that relates three entity classes, STUDENTS, COURSES, and GRADE will be represented as three interaction links between nodes corresponding to the three classes ((Students, Courses), (Students, Grades), and (Courses, Grades)). The participating nodes form a fully connected sub-graph of the SM-graph. To distinguish between a ternary (or 13

14 quaternary) relationship link and a set of three (or four) binary relationship links between the same three (or four) entity class nodes, we specialize the interaction relationship link type L I into three types L IB, L IT, and L IQ. 3.2 The HDE for Global Climate Change and Hydrology Data Consider a subset of the schema for the HDE that stores the Global Climate Change and Hydrology data (GCH) used by atmospheric scientists and hydrologists. This HDE contains facts about atmospheric properties such as cloud characteristics, profiles of temperature, pressure, and humidity, vegetation, radiation, and run-offs. The data is measured using various instruments (probes) that are mounted on satellites, aircraft, and balloons. Observation stations distributed across the globe also measure and record some of the data captured here. The GCH includes data on precipitation and on water flow such as peak and mean daily discharge. Precipitation / gauging stations located on rivers and streams collect this information. In the semantic model schema for GCH (figure 1), a weak class is represented using a box with double borders, complex class using a box with dark borders, and simple strong class using a box with normal border. The subclass relationship is shown as a hexagon with the letter 's'. The SM-graph corresponding to the semantic model in figure 1 is in figure 2. The property links between attribute nodes and their corresponding entity class nodes are represented by dotted lines. Only a subset of attribute and domain nodes is shown for readability. Using this mapping, the semantic model schema of a database can be mapped onto its corresponding SM-graph. Schema changes can be interpreted and implemented as changes to the SM-graph. When changes are completely incorporated, the resultant SM-graph can be converted back into the evolved schema by reversing the mapping. 14

15 Zenith Name COUNTRIES SATELITE PROBES AIRCRAFT PROBES FIELD PROGRAMS COUNTIES Identify Determine Energy_bal ATMOSPHERIC PROPERTIES SEL Area GRP L C2 REGIONS Lat1 Long1 Lat2 Long2 L I1 Measure S PROBES SEL L S2 PROBE TYPES Probe_type Probe_name Frequency PRECIPTN. STATIONS GAUGING STATIONS S S_type Loc-in STATIONS SUBWATER SHEDS Belong-to motion CLOUD CHARACT. Weak Type CLOUDS SURFACE CONDITIONS DATA SEL L C1 PRECIPITATION Collect STREAM FLOW Flowtype Collect GRP WATERSHEDS GRP BASINS Toptemp Opt_den S S GRP LAND OCEAN DAILY MEAN DISCHARGE PEAK FLOW LARGE BASINS Figure 1: Semantic Model for the GCH - USM representation Real N D Countries Zenith L D L D Sname N E Satellite Probes Aircraft Probes N E L D Field Programs N E Name String N D Energy bal Atmospheric Properties N C N C Area L C L D Lat1 Lat2 Long2 Long1 L S L S Probes N E L S Name Frequency Probe_type L D Counties N E Regions Probe Types L I Motion L I Cloud Characteristics N W Type L W L C N E L I Data Clouds Surface Stream Precipitation Conditions Flow N E N E N E N E L I N C L I L C LC LC L C N C L I Precipitation Stations FType N E N E Gauging Stations L S L S Stations N E L I Subwater Sheds N E SType Toptemp Opt_den L S L S Daily Peak Land Ocean Mean N N E N E N E E L S L S L C Watersheds N C L C N C Basins L C N C Large Basins Figure 2: SM-Graph for GCH (Node and link types shown for clarity) 15

16 4. Framework for Dynamic Schema Evolution The framework consists of a set of rules that define the consistent state of the SM-graph (and schema), a comprehensive taxonomy of schema changes based on the graph, and a set of operators to implement them to the graph. It is supplemented by a set of graph-based algorithms that identify parts of the schema affected by a change and detects inconsistencies that arise when changes are incorporated. 4.1 Rules for Consistency Based on the graph representation we define the rules for modifying the semantic model in terms of the nodes and links of the SM-graph. Let G=(N, L) be a SM-graph corresponding to some semantic model schema defined using the USM. The definitions for sets V E, V W, V I, V S, V C, V A, V At, V Ct, V D, and V M and sets E I, E W, E S, E C, E A, E D, E Ct,, E At, and E M follow from section 3.1 For convenience, the set V S is used to represent the set of subclass nodes belonging to node type N S. A subclass node in the SM-graph could belong to sets V E, V W, or V C ). Table 3 lists each rule along with a brief description of the corresponding property of the semantic model that is enforced by this rule in the SM-graph G. Relevant graph definitions are included in table 3. Rule Statement Rule 1: For node X N in G = (N, L), d G (X) 0 X V D Rule 2: For nodes X, Y N in G = (N, L), for each X V E, V W, V I or V C there must exist at least one node Y V A that is connected to X by a link l, l E A except when X V S. Model Property and Relevant Definitions A domain is a pool of values from which attributes draw their values and is a symbolic object with a data type [Ram 1995]. Domains may be predefined and hence may be unconnected in the model, i.e a node of type N D may exist unconnected in G Definition: d G (n) = id G (n) + od G (n) n N where d G (n) is the degree of node n, id G (n) and od G (n) are the in and out degrees of node n Each entity class in a semantic model must have at least one attribute with the exception of subclasses (these may have no attributes of its own and inherited ones are not shown explicitly). Table 3: Rules for Defining the Consistent State of an SM-graph G 16

17 Rule Statement Rule 3: For node X N' in semantic model subgraph G' = (N', L'), d G (X) 1 X V E, V W, V I, or V C. Rule 4: For nodes X, Y, Z N in G = (N, L) every node X V W must be linked to node Y V E, V W, or V I using link l = (Y, X), l E W. If Y V W then the tree in G that contains X and Y must be rooted at some node Z V E or V I. Rule 5: For nodes X, Y N in G =(N. L) every node X V S must be linked to a node Y, Y V E, V W, V I or V C using link l = (Y, X), l E S. If Y V E then X V E and if Y V I then X V W. Also, if Y V W then X V W and the tree in G that contains Y and X must be rooted at some node Z V E or V I. Similarly, if Y V C then X V C and the tree in G that contains Y and X must be rooted at some node Z V E, V W, or V I. Rule 6: For nodes X, Y, Z N in G=(N.L) every node X V C must be linked to a node Y V E, V W, V I, or V C, using link l = (Y, X), l E C. If Y V C then the tree in G that contains X and Y must be rooted at some node Z V E, V W, or V I. Rule 7: For node X, X N in G = (N. L), if X V I then X must have at least two links l 1, l 2 E I incident on it. Rule 8: Every S G of G must be linked to (G-S G ) by at least one link l, l E I, E S, or E C. Model Property and Relevant Definitions A semantic model must be connected. An entity class that is unconnected has no meaning unless it is the first and/or the only entity class in the model. In defining connected attributes and associated property relationships, attribute constraints, class constraints, and constraint relationships are ignored. Let G' = (N', L') be a subgraph of G = (N, L). Set N' = N {V A V Ct V At } and set L' = L {E A E Ct E At } A simple weak class can exist only if it is associated with some existing strong entity class using a weak relationship A subclass can exist only if it has at least one superclass that is an existing strong / weak / complex entity class. A complex class can exist only if it has at least one base class that is an existing strong / weak / complex entity class. An interaction weak class explicitly captures the attributes associated with the interaction relationship and must always be associated with it. A collection consisting of a strong entity class together with all of its subclasses, complex, and weak classes is a component. A model with more than one component is consistent per according to rule 3 even if unconnected components exist. Definition: A component S G = (S N, S L ) of a SMgraph G = (N, L) is a subgraph of G where node X S N implies that X V E, V W, V C, or V S and link l S L implies that l E W, E S, or E C. There exists in S N at least one node X V E. All other nodes in S N are dependent on X, directly or indirectly. These dependency relationships are captured by the set of links in S L. Table 3: Rules for Defining the Consistent State of an SM-graph G (continued) 17

18 Rule Statement Rule 9: For nodes X, Y N in G = (N, L), if X V A and Y ADJ(X), it must be that Y V D, V At, V E, V W, V I, or V C only. ADJ(X) must include one and exactly one node from the set V D and must include one and exactly one node from one of the sets: V E, V W, V C, or V I. The set ADJ(X) may include zero or more nodes from the set V At. Each link l in the set of links incident on X must be from one of the sets E A, E D, or E At. It follows from rule 9 that id(x) 2, od(x) 2, and ADJ(X) > 0. Rule 10: For nodes X, Y N in G = (N, L), if X V D and Y ADJ(X), it must be that Y V A. Each link l in the set of links incident on X must be from the set E D. Rule 11: For nodes X, Y N in G = (N, L), if X V At and Y ADJ(X), it must be that Y V A. Each link l in the set of links incident on X must be from the set E At. Also, ADJ(X) > 0. Rule 12: For nodes X, Y N in G = (N, L), if X V Ct and Y ADJ(X), it must be that Y V S, V C, or V M. Each link l in the set of links incident on X must be from the set E Ct Also, ADJ(X) > 0. Rule 13: For nodes X, Y N in G = (N, L), if X V M and Y ADJ(X), it must be that Y V C or V Ct. ADJ(X) must have exactly one member from the set V C and ADJ(X) > 0. Each link l in the set of links incident on X must be from one of the sets E M or E Ct. Rule 13: For nodes X, Y N in G = (N, L), if X V M and Y ADJ(X), it must be that Y V C or V Ct. ADJ(X) must have exactly one member from the set V C and ADJ(X) > 0. Each link l in the set of links incident on X must be from one of the sets E M or E Ct. Model Property and Relevant Definitions Every attribute in a semantic model must describe exactly one entity class and must draw its value from one domain. An attribute may have one or more attribute constraints. Definition: Let ADJ(X) be the set of nodes adjacent to node X in G. Domains and attribute constraints must be associated with attributes only. Definition: Let ADJ(X) be the set of nodes adjacent to node X in G. Attribute constraints may be associated with one or more attributes. Definition: Let ADJ(X) be the set of nodes adjacent to node X in the G. Every class constraint (inter sub/complex class and on members of complex classes) must be linked to at least one entity class (or subclass / complex class / complex class member) A member of a complex class must be associated with its parent complex class and may be associated with member class constraints A member of a complex class must be associated with its parent complex class and may be associated with member class constraints Table 3: Rules for Defining the Consistent State of an SM-graph G (concluded) 4.2 The Taxonomy of Schema Changes A comprehensive taxonomy of schema changes is developed along with implications of each change based on the SM-graph. We further examine, for each change identified, how it must be captured in the semantic model. The implications of each change help define the atomic set of operations required to implement it. The complete taxonomy is listed in table 4. 18

19 Node Type Changes to Node Implications of Changes to Node Add Entity Class Add Interaction Relationship, Add Attributes. Simple Strong Entity Class Delete Entity Class Delete Interaction Relationship, Delete Attributes, Delete Attribute Constraints, Delete Domain associations, Delete Weak class, Delete Subclasses, Delete class constraints, Delete Complex Classes. N E Change Name of Class Add attributes, Add domain associations, Delete Attributes, delete domain associations. Add Entity Class Add Interaction Weak Relationship, Add Attributes, Add domain associations. Delete Entity Class Delete Interaction Weak Relationship, Delete Attributes, Delete Attribute Constraints, Delete domain associations, Delete Subclasses, Delete Complex Classes. N W Change Name of Class Add attributes, Add domain associations, Delete Attributes, Simple Weak Entity Class delete domain associations Add Entity Class Delete Interaction Relationship, Add Interaction Relationship, Add Attributes, Add domain associations Delete Entity Class Delete Interaction Relationship, Delete Attributes, Delete Attribute Constraints, Delete domain associations, Delete Entity Subclasses, Delete Complex Classes N I Change Name of Class Add attributes, Add domain associations, Delete Attributes, Interaction Weak Entity Class Subclasses of Weak and Strong Entity Class N E / N W / N C Complex Entity Class N C Interaction Relationship L I Add Subclass Delete Subclass Change Subclass constraints Add / Delete Change Definition of subclass Add Complex class Delete Complex class Change Definition of Complex class Add Interaction Relationship Delete Interaction Relationship Changes to Cardinalities delete domain associations. Add Subclass Relationship, Add Attributes, Add domain associations Delete Subclass Relationship, Delete Attributes, Delete domain associations, Delete Attribute constraints, Delete Subclasses, Delete Complex Classes, Delete Class constraint relationships, Delete Class constraint definitions Add Class constraint definitions, Add Class constraint relationships, Delete Class constraint relationships, Delete Class constraint definitions Delete Subclass Relationship, Add Subclass Relationship, Delete Class constraint relationship, Delete class constraint definitions Add Complex class Relationship, Add Attributes, Add domain associations, Add Member Classes, Add Member relationships. Delete Complex class relationship, Delete Attributes, Delete domain associations, Delete Members, Delete Member relationships, Delete Class constraint relationships, Delete Class constraint definitions, Delete dependent Complex Classes (Recursive). Delete Complex class relationship, Delete Members, Delete Member relationships, Delete Class constraint relationships, Delete Class constraint definitions. Add Cardinalities. Delete Cardinalities, Delete Entity (weak/strong/complex) if no other relations exist, Delete interaction class. Same as add / delete of Interaction relationship. Table 4: Taxonomy of Schema Changes based on the semantic model 19

20 Node Type Changes to Node Implications of Changes to Node Add Complex Add Cardinalities. Relationship Complex Relationship Delete Complex Relationship Delete Cardinalities, Delete Entity (weak/strong/complex) if no other relations exist, Delete Interaction class. L C Changes to Same as add / delete of Interaction relationship. Cardinalities ATTRIBUTE N A Add attribute Add attribute relationship, Add domain, Add domain associations. Delete attribute Delete attribute relationship, Delete Domain, Delete Domain associations, Delete attribute constraints, Delete Subclass, Delete Complex class. Changes to Definitions Of attributes 1 Delete Subclass(es), Delete Complex class(es). 1 Mandatory to Optional 2 Optional to Mandatory 3 Single Valued to Multi-valued 4 Multi-valued to Single valued 5 Derived to Nonderived 6 Non-derived to derived (Derived from existing attributes) 7 Manner of derivation 2 No direct changes. 3 Delete Subclass(es), Delete Complex class(es), Add Domain, Add Domain relationship, Delete Domain, Delete Domain relationship. 4 Add Domain, Add Domain relationship, Delete Domain, Delete Domain relationship. 5 Add Domain, Add Domain relationship, Delete Domain relationship. 6 Delete Domain, Delete Domain relationship, Add Domain relationship. 7 Delete Domain relationship, Add Domain relationship. Table 4: Taxonomy of Schema Changes based on the semantic model 4.3 Operators to Implement Schema Changes A set of schema change operators can be defined based on the taxonomy and rules. These operators change the structure of the SM-graph by adding/deleting nodes and links. Modification to a node or link is performed as a sequence of addition (adding a new node or link with the modified definitions) and deletion (deleting a node or link with the old definitions). The operators may be classified into three types. (1) Operators to add new nodes (NA) or links (LA) to the SM-graph. (2) Operators to delete existing nodes (ND) or links (LD) from the SM- graph. (3) Operators to modify existing nodes or links that can be defined in terms of the first two. The SM-graph is consistent if the structure of the graph does not violate any of the rules defined in section 4.1. To preserve its consistency it is necessary in some cases that a combination of basic operators be executed in an atomic fashion. For example adding a domain 20

21 node using the basic node addition operator NA6 would not create an inconsistency. Adding a new entity class node (operator NA1 alone) violates rule 1 if the node is not connected to the rest of the graph. Attribute nodes must be added (NA5) and linked to the entity class node (LA5) and to the corresponding domain nodes (LA6). All above operators must execute atomically to ensure consistency. The basic operators are combined to define consistency-preserving (CP) operators. Table 5 lists the basic operators that add or delete nodes/links from the SM-graph and table 6 lists the requirements to be met by each link addition operator before it can add a link of a specific type. The list of consistency preserving operators is listed in table 7. Operators NA-1 / ND-1 NA-2 / ND-2 NA-3 / ND-3 NA-4 / ND-4 NA-5 / ND-5 NA-6 / ND-6 NA-7 / ND-7 NA-8 / ND-8 NA-9 / ND-9 LA-1 / LD-1 LA-2 / LD-2 LA-3 / LD-3 LA-4 / LD-4 LA-5 / LD-5 LA-6 / LD-6 LA-7 / LD-7 LA-8 / LD-8 LA-9 / LD-9 Description of operation performed Adds / Deletes a Simple Strong Entity Class node N E Adds / Deletes a Simple Weak Entity Class node, N w Adds / Deletes an Interaction Weak Entity Class node, N I Adds / Deletes a Complex Class node, N C Adds / Deletes an Attribute node, N A Adds / Deletes a Domain node, N D Adds / Deletes a Class Constraint Node N Ct Adds / Deletes an Attribute Constraint Node, N At Adds / Deletes Composite Class Member, N M Adds / Deletes an Interaction Relationship link, L I Adds / Deletes a Weak Relationship link, L W Adds / Deletes a Subclass Relationship link, L S Adds / Deletes Complex Class Relationship link, L C Adds / Deletes Attribute Property link, L A Adds / Deletes Domain Association link, L D Adds / Deletes a Class Constraint Relationship link, L Ct Adds / Deletes an Attribute Constraint Relationship link, L At Adds / Deletes Composite Class Member Relationship link, L M Table 5: Node Addition and Node Deletion Operators Link Addition Adds link (X, Y) of Nodes X, Y N in graph G = (N, L) where Operator type: LA1 L I X, Y are of node types N E, N W, N I, or N C LA2 L W X is of node type N E or N W ; Y of node type N W LA3 L S X is of node type N E, N W, N I, or N C ; Y of node type N S LA4 L C X is of node type N E, N W, N I, or N C ; Y of node type N C LA5 L A X (Y) is of node type N E, N W, N I, or N C ; Y (X) of node type N A LA6 L D X (Y) is of node type N A ; Y (X) of node type N D LA7 L Ct X (Y) is of node type N E, N W, N I, N C, or N M ; Y (X) of node type N Ct LA8 L At X (Y) is of node type N A ; Y (X) is of node type N At LA9 L M X is of node type N C ; Y is of node type N M Table 6: Link Addition Operators and Node Types Associated with each 21

22 # Operator and Description Set of basic operators invoked in its execution 1 ADM Add Domain NA6 2 AAT Add Attribute NA5, LA5, {NA6} 0, LA6 3 ASE Add Strong Entity Class NA1, {AAT} 1, {[LA1 LA3 LA4]} 0 4 AWE Add Weak Entity Class NA2, {AAT} 1, LA2 5 AIE Add Interaction Weak Class NA3, {AAT} 1, LA1, DIR 6 ASUB Add Subclass [NA1 NA2 NA4], {AAT} 1, LA3 7 ACX Add Complex Class NA4, {AAT} 1, LA4 8 AACT Add Attribute Constraint NA8, {LA8} 1 9 ACCT Add Class Constraint NA7, {LA7} 1 10 ACM Add Composite Member NA9, LA9 11 AIR Add Interaction Relationship LA1 12 ASUBR- Add Subclass Relationship LA3 13 ACOMR Add Complex Relationship LA4 14 DACT Delete Attribute Constraint {LD8} 1, ND8 15 DCCT Delete Class Constraint {LD7} 1, ND7 16 DCM Delete Composite Member LD9, ND9 17 DAT Delete Attribute {DACT} 0, LD6, ND6, LD5, ND5 18 DIR Delete Interaction Relationship {LD1} 1 19 DSE Simple Strong Entity Class {{{DAT} 1, ND4} 0, LD4 } 0, {DIR} 0, {DCCT} 0, {{{DAT} 1, {ND1}} 0, LD3 } 0, {DIR} 0, {DCCT} 0, {{DAT} 1, ND2, LD2} 0, {DAT} 1, {DIR} 0, ND1 20 DWE Delete Weak Entity Class {{{DAT} 1, ND4} 0, LD4 } 0, {DIR} 0, {DCCT} 0, {{{DAT} 1, {ND2}} 0, LD3 } 0, {DIR} 0, {DCCT} 0, {DAT} 1, {DIR} 0, LD2, ND2 21 DIE Delete Interaction Weak Entity Class {{{DAT} 1, ND4} 0, LD4 } 0, {DIR} 0, {DCCT} 0, {{{DAT} 1, {ND2}} 0, LD3 } 0, {DIR} 0, {DCCT} 0, {DAT} 1, ND3, {DIR} 1 22 DSUB Delete (Strong/Weak) Subclass {{{DAT} 1, ND4} 0, LD4} 0, {DIR} 0, {DCCT} 0, {[LA3 {{{DAT} 1, [ND1 ND2 ND4]} 0, LD3 } 0, {DIR} 0 ]} 0, {DCCT} 0, LD3,{DAT} 1, [ND1 ND2 ND4], {DIR} 0, {DCCT} 0 23 DCX Operator Delete Complex Class {{{DAT} 1, {ND4}} 0, LD3 } 0, {DIR} 0, {DCCT} 0, {[LA4 {{{DAT} 1, ND4} 0, LD4} 0, {DIR} 0 ] } 0, {DCCT} 0 {LD4} 1, {DAT} 1, ND4, {DIR} 0, {DCCT} 0 Table 7: List of Consistency-Preserving (CP) Operators (adopting conventions listed below) Each operator on the left is defined by a comma-separated list of basic operators (right). The order of execution may vary depending upon the implementation. The list of one or more operators within square parenthesis ([]), separated by the vertical separator ( ), indicates the option of calling ONE and EXACTLY ONE of the specified operators from the list and ONE operator must be executed. The list of one or more operators within curved parenthesis {} indicate that the sequence of operators may be executed more than once. The superscript 0 indicates that the list may not be executed at all, and superscript 1 indicates that it is executed at least once. 4.4 Completeness and Correctness of Schema Change Operators A SM-graph G is consistent (or in a consistent state) if it satisfies the thirteen consistency preserving rules stated in section 3.1. An empty SM-graph is trivially consistent. Correctness (or 22

23 soundness) addresses whether the application of a set (or subset) of the consistency preserving operators listed in table 7 to a consistent SM-graph results in a consistent SM-graph. Correctness: To show that the schema evolution operators are correct, for each operator in the set we need to show that the application of this operator to a consistent SM-graph G results in a consistent SM-graph G. Theorem 1: Every consistency preserving operator (listed in table 7), when applied to a consistent SM-graph G results in a consistent SM-graph G, i.e., each operator preserves the consistency of the SM-graph. Proof: For each operator, we can show that it preserves the consistency of G. For brevity, the complete proofs are not discussed here and only an outline of the proof is presented. The consistency preserving rules specify the structure of each node/link type in G. For each node/link type in G the rules define how every node/link of that type must exist in G by specifying the types of nodes that must be adjacent to it and the types of links that must be incident on it. Therefore not all of the rules apply to each node/link type. A subset of rules is pertinent to each type of node/link in G. Graph G is initially assumed to be consistent and therefore each node/link in G satisfies all of the rules that are applicable to that node/link type. When a node/link in G is modified by some operator we show that the resultant graph G is consistent by showing that each affected node/link in G satisfies the subset of rules that is applicable to its node/link type. All other rules are trivially satisfied in G as they were satisfied in G and remain unaffected. Completeness: To prove completeness we need to show that the operators are sufficient to capture all schema changes that can occur in a database. The proof is completed in three steps. First, we state that the Unifying Semantic Model (USM), like other semantic models, is capable of representing the schema of any database. The purpose of the USM is to allow users and designers to capture the meaning of a database and as a formal mechanism for describing a real world [Ram 1995]. Second, we show that the SM-graph mapped from some semantic model 23