Stored Relvars 18 th April 2013 (30 th March 2001) David Livingstone. Stored Relvars

Transcription

1 Stored Relvars Introduction The purpose of a Stored Relvar (= Stored Relational Variable) is to provide a mechanism by which the value of a real (or base) relvar may be partitioned into fragments and/or merged with the values of other real relvars, with the results being physically stored instead of the relational value (= relvalue) of the original real relvar. This document describes stored relvars and their usage within the formal data storage model of the RAQUEL Database Management System (= DBMS). The Purpose of Stored Relvars A real relvar is a relvar whose value is physically stored. The simplest way to store its value is to store it in its entirety as a single, coherent whole that encompasses all the data in the relvalue. Sometimes this is sufficient. However sometimes it is preferable if the relvar s value is re-organised and the consequent results physically stored instead. There are three logical possibilities for re-organising the relvalue :- 1. The real relvar s value is fragmented into several smaller relvalues, and it is these relvalue fragments that are physically stored. 2. The real relvar s value is merged with the relvalues of other real relvars into a single, larger relvalue, and it is this single large relvalue that is physically stored. 3. A combination of the above two re-organisations. The purpose of stored relvars is to make possible the storage of a real relvar s value via a configuration of fragments and/or mergers of the real relvar s value. A stored relvar is one whose value is directly physically stored as a single, coherent whole. (It does not prevent the nature of the physical storage used being varied to optimise physical performance). Conversely each stored relvalue is the value of a stored relvar. By contrast, although a real relvar is also one whose value is physically stored, the storage may be direct or indirect. Direct storage means that its value is stored as a single, complete entity - i.e. the real relvar is also a stored relvar. Indirect storage means that its value can be split up into fragments and the fragment values physically stored; or its value can be merged with the values of other real relvars and the whole merged value physically stored; or some combination of the two.- i.e. the value of a real relvar is mapped to those of separate stored relvar(s). In the fragmentation approach, a relvar s value is fragmented horizontally or vertically. Horizontal fragmentation uses restrictions and/or semi-joins to split the original relvalue into a set of two or more sub-relvalues, such that they can be Page 1 of 10

2 unioned together to re-create the original relvalue. Vertical fragmentation uses projection to split the original relvalue into a set of two or more sub-relvalues, such that they can be naturally joined together to re-create the original relvalue. A fragment can itself be fragmented, either horizontally or vertically, so that the fragmentation can be continued recursively till fragments suitable for storage are obtained. Stored relvalues are the fragments chosen to be physically stored. In the merge approach, a relvar s value is merged together horizontally or vertically with the values of one or more other relvars. A horizontal merger is derived by unioning together compatible relvalues, such that the original relvalues can be re-created via restrictions on the merged relvalue. A vertical merger is derived by a natural join of appropriate relvalues, such that the original relvalues can be re-created via projections of the merged relvalue. Merged relvalues can themselves be merged together, either horizontally or vertically, so that the merging can be continued recursively till a merged result suitable for storage is obtained. Stored relvalues are the merged results chosen to be physically stored. The Rationale for Fragmenting and Merging Relvalues The fragmentation approach has been well developed for the design of DBs distributed over a network of computers, but can also be used to distribute data over multiple storage devices attached to one computer. In both cases, the distribution of fragments of a real relvar s value over multiple locations is in order to improve performance, typically to speed up access to stored data, also for other benefits such as improved resilience. Because fragmentation is purely a physical data storage issue, some DBMSs do not use relational algebra or its SQL equivalent to define it, but use methods developed as part of the DBMS s data storage management facilities. Traditionally mergers are not considered so attractive a strategy as fragmentation because in general it takes longer to access a small volume of data from a larger set than it does from a smaller set. This is exacerbated when relvalue sizes increase faster than the ability of storage devices to hold relvalues, where ability typically refers to storage capacity and access speed. Nevertheless vertical mergers are potentially beneficial. If a query involves joining relvars, physically storing the relvalue of the vertical merger that is the result of that join would remove the need for the (potentially time-consuming) physical join operation. This is the origin of the interest in materialised views as a data storage technique, and what underpins the halfway house of physical data clustering whereby data records holding tuples that would be merged in a join of two relvars are held in the same physical block. It is also why some denormalisation of relational DB designs occurs. However de-normalisation can lead to confusion and maintenance problems in the long run because the design no longer reflects the inherent nature and structure of the data in the database, despite the fact that queries and performance must be based ultimately on the inherent nature and structure of the data. Page 2 of 10

3 The combination of merging and fragmentation appears to be little used, but may be of interest (say) for use in supporting fragments of joins. Stored Relvars and RAQUEL s Formal Data Storage Model RAQUEL s formal data storage model deals with 3 aspects of physically storing the values of real relvars : 1. The flexibility to permit the values of real relvars to be stored in a variety of configurations, namely the ability to use the fragmentation and merging of real relvar values to create a suitable configuration of actual physically stored relvalues. Every physically stored relvalue is that of a stored relvar. 2. The ability for the value of a stored relvar to be stored in multiple locations. 3. Multiple kinds of storage facilities to handle the actual storage of and access to values of stored relvars. These 3 aspects are reflected in the 3 layers of the physical data storage architecture :- Storage Schema Stack Schema Location Schema Physical Schema The Storage Schema comprises the set of all the stored relvars that exists in the DB, analogously to the Logical Schema (in the layer above the Storage Schema) comprising the set of all the real relvars that exists in the DB. Each stack schema represents a particular location at which data is physically stored. A Stack Schema s Location Schema comprises all the stored relvars whose values are stored at that location, and its Physical Schema specifies the physical storage facilities used to store the value of a stored relvar at that location. The same relational model applies to both stored and real relvars, but stored relvars are an integral part of the RAQUEL formal data storage model, whereas real relvars are part of the logical model presented to an application by the RAQUEL DBMS. The storage model and the application model are disjoint. Nevertheless the front-end of the storage model uses the same relational model as the application model. The purpose of stored relvars is to be the containers of collections of attribute values that are to be physically stored. In principle, any kind of container, say an array of attribute values, could be used as long as it was effective and efficient in dealing with the storage and access of the physically stored data that forms a Page 3 of 10

4 relvalue, but RAQUEL uses relvars as containers. It uses them for the following reasons : It simplifies the construction of the DBMS as a whole because the same relational model is used for both the storage model and the application model. It is simpler for a DB Administrator to manage the physical storage of data, because s/he does not have to learn a new logical model to manage the storage system, nor does s/he have to learn how to map relations in the application model to a different kind of container of attribute values in the storage model. It provides the potential for a logical relational optimiser to operate not just at the level of virtual and real relvars but down to the level of stored relvars, without any additional overhead to the optimiser. It also simplifies what the physical storage optimiser has to achieve. Stored Relvars as Opposed to Real Relvars If stored relvars are to be used as the means of storing the relvalues of real relvars, there needs to be a means of mapping an individual real relvar to the one or more stored relvars that hold the former s value. This is done by binding a Storage Expression to a real relvar. A storage expression is a relational algebra expression that contains at least one relvar, such that every relvar in the expression Is a stored relvar. The binding is expressed in RAQUEL as follows : R ==Equate StorageExpression where R is the name of a real relvar which has the storage expression bound to it. The storage expression is said to define a storage configuration for R. The == prefix indicates that the assignment is a binding assignment and so part of the RAQUEL storage model. In an ==Equate assignment the storage expression is bound to the real relvar the expression equates to its value. Hence : The value of R is obtained by evaluating its storage expression. The value of R is changed by changing the value(s) of the stored relvar(s) in its storage expression. Every real relvar must be bound to some storage configuration in order that the DBMS can physically store the value of the real relvar. When a real relvar is first created, the DBMS in effect, as a default, executes the assignment : R ==Equate R This means that relvar R appears in both the application model and the storage model. It can have the same name in both models because it is the same relvar in both models. When a storage expression is assigned to a real relvar, it replaces the existing storage expression. So the default storage configuration of a real relvar can be replaced by a better one by assigning new storage expression to it at a later Page 4 of 10

5 date. Clearly care has to be taken if a real relvar already has a non-empty value when a new storage expression is assigned to it. This raises the question as to whether a storage expression could be a literal relvalue (as long as it is of the right type) : R ==Equate some-literal-relvalue This would effectively define R to be a constant whose value was that of the literal relvalue assigned. In fact the specification of ==Equate requires that StorageExpression include at least one stored relvar. However that doesn t exclude the possibility of a literal relvalue being included in StorageExpression. In general stored relvars form a different set of relvars to real relvars. As such it is necessary to be able to assign a relvalue to a stored relvar. This can be done in RAQUEL as follows : S <--Store relational-algebra-expression The RHS expression is evaluated and assigned to S as its value, overwriting any previously assigned value. S must be a stored relvar, although this does not exclude it from being a stored relvar which is also a real relvar. S is created if it does not already exist. In passing, note that RAQUEL also includes the two comparable assignments R <--Real relational-algebra-expression V <--View relational-algebra-expression where R is a real relvar and V is a virtual relvar. The <--Real assignment operates identically to <--Store, except that the LHS variable must be a real relvar. <--View operates like <--Real and <--Store except that the RHS expression is not evaluated but is assigned as is to V as its definition. Examples of the Usage of Stored Relvars Example One : Horizontal Fragmentation Let there be a real relvar, RU, such that it would be much more efficient to store and process its value as two disjoint sets of tuples, i.e. two sub-relvars. To ensure that the two relvars - let them be called R1 and R2 - are disjoint, there must be a condition such that when it is used as a parameter in a Restrict operation applied to RU, the tuples of RU for which the condition is True form the value of R1, and the tuples of RU for which the condition is False form the value of R2. In other words, if Condition expresses the condition, then : R1 RU Restrict[ Condition ] R2 RU Restrict[ ~ Condition ] It is then the case that : RU R1 Union R2 Page 5 of 10

6 Therefore it would be possible to store the value of the real relvar RU in two horizontal fragments corresponding to R1 and R2. To achieve this in RAQUEL requires first creating two stored relvars, R1 and R2, containing the relevant values, and then binding RU so that it takes its value from the union of R1 and R2. This would be expressed in RAQUEL as : R1 <--Store RU Restrict[ Condition ] R2 <--Store RU Restrict[ ~ Condition ] RU ==Equate R1 Union R2 The <--Store assignments create R1 and R2 as stored relvars, with the correct values derived from RU. This will happen whether RU has no tuples (because, say, it has just been created) - so an empty relvalue is assigned to each of R1 and R2 or whether R has a non-empty value. R1 and R2 are created with the correct type, which is derived from the RHS expression of the <--Store assignments, and so is the same type as that of RU. The ==Equate assignment replaces the previous storage expression of RU and hence the previous relvalue of RU with the new storage expression, which in fact will evaluate to the same value of RU. Note that it would be illogical to execute the ==Equate assignment before the <--Store assignments. Example Two : Vertical Fragmentation Let there be a real relvar, RJ, such that it would be much more efficient to store and process its value as two disjoint sets of attributes, i.e. two sub-relvars with no attributes in common. Actually they cannot be completely disjoint, because there would then be no way of re-combining the two fragments to re-create the value of RJ The solution is to make the attribute sets of the two sub-relvars disjoint except for a Candidate Key which is common to both sub-relvars. Then if : R3 RJ Project[ K, A ] R4 RJ Project[ K, B ] where K is a set of attributes that form a common Candidate Key, and A and B are two disjoint sets of attributes, then : RJ R3 Join[ K ] R4 The following statements in RAQUEL create the corresponding fragments as stored relvars : R3 <--Store RJ Project[ K1, K2, A1, A2, A3 ] R4 <--Store RJ Project[ K1, K2, B1, B2, B3, B4 ] RJ ==Equate R3 Join[ K1, K2 ] R4 This assumes that attributes K1 and K2 constitute the Candidate Key, attributes A1, A2, and A3 are those unique to R3, and attributes B1, B2, B3, and B4 are those unique to R4. Page 6 of 10

7 Example Three : A Merge of Horizontal Fragments Let there be two real relvars, RL and RR, such that it would be much more efficient to store and process their values as two merged relvars, each formed from two horizontal fragments of RL and RR. Initially let there be four fragments, RL1, RL2, RR1 and RR2, defined as follows : RL1 RL Restrict[ ConditionL ] RL2 RL Restrict[ ~ ConditionL ] RR1 RR Restrict[ ConditionR ] RR2 RR Restrict[ ~ ConditionR ] such that it is then the case that : RL RL1 Union RL2 RR RR1 Union RR2 Let relvar RL have attributes K1, K2, A1, A2 and A3. Let relvar RR have attributes K1, K2, B1, B2, B3 and B4. Let attributes K1 and K2 form a Candidate Key in both RL and RR. Let ConditionL and ConditionR be such that the use of ConditionL to split RL into fragments RL1 and RL2, and the use of ConditionR to split RR into fragments RR1 and RR2, has the following result. For each tuple in RL1 there is only one tuple in RR1 which matches it when RL1 is joined to RR1 on the common attributes K1 and K2; and vice versa. Likewise for each tuple in RL2 there is only one tuple in RR2 which matches it when RL2 is joined to RR2 on the common attributes K1 and K2; and vice versa. It is then the case that : RJ1 RL1 Join[ K1, K2 ] RR1 RJ2 RL2 Join[ K1, K2 ] RR2 such that : RJ1 Union RJ2 RL Join[ K1, K2 ] RR It would then be possible to create two stored relvars, RJ1 and RJ2 using the following RAQUEL statements : RJ1 <--Store ( RL Restrict[ ConditionL ] ) Join[ K1, K2 ] ( RR Restrict[ ConditionR ] ) RJ2 <--Store ( RL Restrict[ ~ ConditionL ] ) Join[ K1, K2 ] ( RR Restrict[ ~ ConditionR ] ) The values of the two original real relvars RL and RR could then be derived from the values of the two stored relvars RJ1 and RJ2 using the following RAQUEL statements : Page 7 of 10

8 RL ==Equate ( RJ1 Project[ K1, K2, A1, A2, A3 ] ) Union ( RJ2 Project[ K1, K2, A1, A2, A3 ] ) RR ==Equate ( RJ1 Project[ K1, K2, B1, B2, B3, B4 ] ) Union ( RJ2 Project[ K1, K2, B1, B2, B3, B4 ] ) Storing the relvalues of RL and RR via the stored relvars RJ1 and RJ2 could be more efficient if RL and RR were frequently used in queries which involved joins of the fragments RL1 and RR1 and/or joins of the fragments RL2 and RR2. In those queries the relational algebra expression of the query would include the sub- expression(s) whose value(s) has/have already been evaluated and stored under the designation RJ1 and/or RJ2, thereby reducing the data storage processing needed to accomplish the query. The Design of Storage Configurations The examples used sound design to ensure that the stored relvars represented a good physical storage configuration that mapped well to the real relvars. So what would happen if the design were poor? The answer is that the RAQUEL DBMS would still execute that design, because its data storage system is a formal model and the DBMS would simply execute the formalism. A poorly designed storage configuration may well execute slowly, or have undesirable consequences such as poor resilience. If the storage configuration is illogical, then executing the data storage formalism could result in the DBMS halting with an error or even returning a false result. In this respect, abusing the data storage formalism is no different to abusing any other mathematical formalism, such as integral calculus or differential calculus. The fact that the data storage system is a formal model does not make the use of it immune to poor design. There are design rules governing how fragmentation should be specified. For example, to get the most efficient and effective stored relvalues, horizontally created sub-relvalues should be disjoint (i.e. have no tuples in common); vertically created sub-relvalues should be disjoint (i.e. have no attributes in common) except for one common Candidate Key which is needed to join the sub-relvalues together so as to re-create the original relvalue. Design rules corresponding to those for fragmentation apply to the merging of relvalues. For example, when vertically merging two real relvars values with a join, it is important to do so in such a way that their values can be reliably and accurately re-created from the single merged relvalue. It is important to distinguish between the design of real relvars and the design of stored relvars : The design of real relvars concerns the succinct representation of the inherent nature and structure of all the data stored in the DB. Page 8 of 10

9 The design of stored relvars is dependent on the real relvars that they support, but is driven by data usage and the data storage units available. Data usage includes the frequency and relative importance of different DB transactions. In addition to optimising physical performance, recovery and security considerations need to be taken into account. The use of multiple networked computers and/or multiple storage units attached to one computer also affects the design. Hence the separate existence of the Logical Schema (for real relvars) and the Storage Schema (for stored relvars) is useful. For example, a fully normalised design may be applied to Logical Schema relvars with an un-normalised design in the Storage Schema in order to improve performance. Virtual, Real, and Stored Relvars The relationship of a virtual relvar to the real relvar(s) it is defined on is identical to the relationship of a real relvar to the stored relvar(s) that hold its relvalue. For instance, consider the earlier example where RJ was a real relvar such that : R3 RJ Project[ K, A ] R4 RJ Project[ K, B ] RJ R3 Join[ K ] R4 Two stored relvars R3 and R4 were created to hold the value of RJ : R3 <--Store RJ Project[ K1, K2, A1, A2, A3 ] R4 <--Store RJ Project[ K1, K2, B1, B2, B3, B4 ] RJ ==Equate R3 Join[ K1, K2 ] R4 Now suppose instead that it was decided to replace the real relvar RJ with a virtual relvar RJ that was defined on two real relvars R3 and R4 such that the same equivalences as before applied : R3 RJ Project[ K, A ] R4 RJ Project[ K, B ] RJ R3 Join[ K ] R4 This could be carried out with the following statements : R3 <--Real RJ Project[ K1, K2, A1, A2, A3 ] R4 <--Real RJ Project[ K1, K2, B1, B2, B3, B4 ] RJ <--View R3 Join[ K1, K2 ] R4 The two cases are logically equivalent. The only apparent difference is the use of a binding assignment ( ==Equate ) as opposed to a value assignment ( <--View ). However this difference is superficial. The binding assignment is needed because the expression assigned represents data storage; the value assignment is needed because the expression assigned represents a relational algebra definition within the relational model. Page 9 of 10

10 Both ==Equate and <--View assign the un-evaluated expression R3 Join[ K1, K2 ] R4 to the relvar named on their LHS. In both cases this expression must be evaluated to derive the value of RJ. In both cases when the value of RJ is changed, that change must cause corresponding changes in R3 and R4; the mechanism to do that in both cases is the view updating mechanism. In principle updating an underlying stored relvar(s) in order to update a real relvar is no different to updating an underlying real relvar(s) in order to update a virtual relvar. The correspondence applies generally. Date and McGoveran have proposed a generalised method of view updating for all logically-updateable virtual relvars. A comprehensive description was published in C. J. Date s 6 th edition of An Introduction to Database Systems (Addison Wesley, 1995) textbook. The method has been further developed and is the subject of C. J. Date s book View Updating and Relational Theory : Solving the View Update Problem (O Reilly,2013). Using this same method for the relvars underpinning both virtual and real relvars simplifies the construction of the DBMS and makes it easier for a DB Administrator who has to design all aspects of a DB 1. The execution of the view updating method takes account of the design of relvars, including their integrity constraints. With respect to design : Real relvars should canonically represent the inherent nature and structure of data in the DB. Stored relvars should reflect storage requirements. They are never seen by DB applications. Virtual relvars should meet user needs. They may be seen in conjunction with real relvars. Thus virtual relvars might be ad hoc in design, or they might be designed to canonically represent the inherent nature and structure of a sub-set of the DB s data that appears in a subschema. Given these different design approaches, the application of the view updating method in the two cases may tend to have different outcomes. 1 It also makes possible the use of unnamed views ( pseudovariables in the terminology of The Third Manifesto) in assignments such as ( RX Join[ K ] RY ) <--Insert LiteralRelvalue where K is a suitable Candidate Key common to both relvars RX and RY, and LiteralRelvalue has the same type as the LHS unnamed view. Consistent insertions to RX and RY are carried out by executing one statement, via the view update mechanism. Page 10 of 10