Illustrative Example of Physical Schema Usage

Transcription

1 Example of Physical Usage 14 th February 2014 (30 th March 2001) Illustrative Example of Physical Usage The example assumes that a small RAQUEL DB and its relational model schemas have already been created. Firstly those physical schemas are described that are automatically created when the DB is created. Secondly the physical schemas are altered to illustrate how they can be changed in a RAQUEL DB, to improve performance, resilience, etc. The Example DB and Its s The DB example is called ExampleDB. As part of the creation of the DB, all the system schemas in it are created. These include not only the logical system schemas, but also the physical system schemas and a single default stack schema. A Venn diagram of all these schemas immediately after the creation of ExampleDB looks like :- Logical Sink Source Virtual DB ExampleDB Storage Stack Stack1 Stack1Physical Stack1Location Notice that all the schemas are empty, except that the default stack schema Stack1 contains 2 system schemas. Assume that after DB creation, the logical relational model schemas together with the required subschemas are developed as described in the document Page 1 of 13

2 Example of Physical Usage 14 th February 2014 (30 th March 2001) Illustrative Example of Logical Database Creation. The following Venn diagram describes these logical relational model schemas and subschemas :- Subschema CompanyDB DB ExampleDB Subschema CompanyNorth Virtual UsersNorth DeptsNorth EmpsNorth DEPT EMP USER UsersSouth DeptsSouth EmpsSouth Logical Sink Subschema CompanySouth Source Default action provides the physical data storage required to support these logical schema. In order to explain that default action, the operation of the physical data storage model must first be considered. Physical Data Storage Model The RAQUEL DBMS takes 3 factors into account when physically storing the value of a real (or base) relvar : 1. The relational algebra Storage Expression assigned to the real relvar. The expression includes one or more relvar names, each relvar being a Stored Relvar, i.e. a relvar whose value is physically stored as a coherent whole in one location. When the expression is evaluated, it uses the values of the stored relvars to derive and return the value of the real relvar to which the expression has been assigned. 2. The storage location assigned to each stored relvar. Each location has a stack schema that handles physical data storage at that location. 3. The type of the physical storage mechanism or facility assigned to a stored relvar for use in a particular location. Page 2 of 13

3 Example of Physical Usage 14 th February 2014 (30 th March 2001) Each of the 3 factors is specified using an appropriate kind of assignment. The assignments are : 1. The assignment of a storage expression to a real relvar. The assignment takes the form : R ==Equate StorageExpression where R is the name of the real relvar concerned. ==Equate binds relvar R to the storage expression. 2. A stored relvar is assigned to a storage location, i.e. to the stack schema that handles that location. A relvar may be moved from one storage stack to another or copied from one to another (so that 2 copies exist in 2 storage stacks). The assignments take the form : StackName <--Move { { S } } StackName <--Copy { { S } } where S is the name of the stored relvar concerned. The <--Move and <--Copy assignments execute at the schema level, so the right-hand operand must also be a schema. A schema comprises a set of members and can be construed as a relvalue whose tuples constitute the members of the set. Therefore if the tuples contain the relevant valid attribute values here the names of relvars a relvalue can be used as the righthand operand. 3. The assignment of a physical storage mechanism or facility chosen from those available in the RAQUEL DBMS installation. The assignment takes the form : S ==Physical[ StorageMechanism ] where S is the name of the stored relvar concerned. == Physical binds the stored relvar to the specified physical storage mechanism. StorageMechanism may include not only the kind of physical storage mechanism required, but also associated parameters that define how the mechanism is to be used. The number and nature of the parameters depends on the kind of storage mechanism concerned. Two assignments fall into the Binding category because they bind the recipient operand to the corresponding aspect of its physical storage. The two value assignments assign a value that is a name (that of a stored relvar) to their recipient operand (which is a stack schema). Only real relvars affect physical storage. The creation and values of virtual relvars cannot affect it, because the values of virtual relvars are derived from the values of real relvars. The creation and values of source and sink relvars cannot affect physical storage, because the values of source and sink relvars are stored outside the DB and are not managed by the DBMS. Page 3 of 13

4 Example of Physical Usage 14 th February 2014 (30 th March 2001) The Example s Default Physical Storage s Both default and user-designed storage arrangements are created by use of the kinds of assignment described above. When a real relvar is created, two possibilities arise as to when to create the physical storage arrangements needed to store its value : 1. Create the physical storage arrangements immediately. They are available whenever thereafter the relvar receives any tuples. 2. Create the physical storage arrangements later, at any convenient point in time prior to tuples first being assigned to the relvar. The two timings are orthogonal to the two alternatives of default or user-designed storage arrangements. Thus a user can implement specially designed storage arrangements at any time prior to the receipt of tuples by the relvar. RAQUEL automatically defers the creation of default storage arrangements until the receipt of the first tuples by the relvar. So for ExampleDB, when tuples are first put into the real relvars DEPT and EMP, RAQUEL executes the default storage assignments for them. In this case they are as follows : 1. By default, a real relvar is made a stored relvar as well, i.e. the entire value of each real relvar is held as a single coherent volume of data in one storage unit. Therefore from within a subschema or the Logical the following ==Equate assignments are executed : DEPT ==Equate DEPT EMP ==Equate EMP The storage expressions consist solely of the real relvar s own name. 2. A default stack schema must always exist, created as part of the creation of the DB. By default its name is Stack1 (although if required a different default name could be assigned to a DBMS installation for use instead). Therefore after exiting the previous schema, enter Stack1 ready to create both the stored relvars, DEPT and EMP, in it : Stack1 <==Open 3. From the set of physical storage mechanisms available at each DBMS installation, one must always be specified as the default storage mechanism. By default the values of stored relvars are stored using the default mechanism. Therefore execute == Physical assignments to create the stored relvars in Stack1 : DEPT ==Physical[ DefaultStorageMechanism ] EMP ==Physical[ DefaultStorageMechanism ] Because it is a default, the same physical storage mechanism is used for both DEPT and EMP. Now exit Stack1 : Page 4 of 13

5 Example of Physical Usage 14 th February 2014 (30 th March 2001) Stack1 <==Close Note that no moving or copying of stored relvars is necessary in this case. A user could have executed these default statements earlier if required, and this is true in general for default storage. The following Venn diagram depicts how the default physical storage schemas and their contents relate to each other :- Logical Storage DEPT EMP Stack1 Stack1Location DEPTSt ; DefStorMech EMPSt ; DefStorMech Stack1Physical The name of the storage unit and the name of the default physical storage mechanism constitute the physical storage specification of a stored relvar. DefStorMech is assumed to be the name of the default physical storage mechanism. Because the name of a storage unit is always that of the stored relvar with the suffix St, there is no need to specify via an assignment the name of the storage unit to be used to hold the value of a stored relvar at a location 1. The above diagram includes the Logical because it is this schema and its contents that determine the physical storage schemas and their contents. It can be seen that while the schema architecture for physical data storage allows considerable potential to vary physical storage in a large number of ways, the defaults give the simplest possible storage arrangement. The Logical, Storage and Location s are seen to be identical, with each relvar in this common set mapping directly to its physical storage. 1 If it were required in future to avoid fixed names for storage units, another assignment (say ==Store) could be created as the means of assigning a name of choice. Even then, the current mandatory naming scheme would be useful as a default naming scheme. Page 5 of 13

6 Example of Physical Usage 14 th February 2014 (30 th March 2001) Improving the Physical Data Storage Arrangements At any time, the physical storage arrangements used to hold the value of a real relvar may be altered, regardless of whether they are default arrangements or were specially set up for the relvar. For illustration, let the default arrangements set up for DEPT and EMP be altered to something now considered preferable. It is assumed that both relvars are nonempty and therefore the default storage arrangements are being used to hold data. Assume it is the case that the 3 subschemas are found to be used by 3 separate groups of users. The group using the CompanyDB subschema is located mainly in the Head Office at Newcastle, the group using the CompanyNorth subschema mainly in Manchester and Leeds, and the group using the CompanySouth subschema mainly in Coventry and London. Since there are suitable computer servers in Newcastle, Manchester and Coventry, and the company s computer network links all company offices to all 3 servers, it is decided to store a copy of the data of each subschema on its corresponding local server, in order to store the data as close as possible to its users. To achieve this, an appropriate stack schema and its corresponding physical data storage 2 are to be created on each of these 3 servers. The stack schema on the Newcastle server is to be called CoHQ, that on the Manchester server CoNorth, and that on the Coventry server CoSouth. As a reminder, the attributes of relvars DEPT and EMP are, in terms of the original attribute assignment statements : DEPT <==Attribute[ DeptNo <== Text; Budget <== Number; Area <== { { North } { South } } ] EMP <==Attribute[ EmpNo <== Text; EName <== Text; DeptNo <== Text; Salary <== Number; Tax <== Number ] Given the relvar usage of the 3 groups of users, let it be decided to store the relvalues of DEPT and EMP in the following fragments : EMP-Fin. This is to hold the financial data about all employees i.e. attributes EmpNo, Salary and Tax. EMP-N. This is to hold the non financial data i.e. attributes EmpNo, EName, and DeptNo about all employees from the northern part of the company. 2 Note that data is physically stored separately from the stack schema that manages it, not within that stack schema. Page 6 of 13

7 Example of Physical Usage 14 th February 2014 (30 th March 2001) EMP-S. This is to hold the non financial data i.e. attributes EmpNo, EName, and DeptNo about all employees from the southern part of the company. DEPT-N. This is to hold department data i.e. attributes DeptNo, Budget, and Area about all departments in the northern part of the company. DEPT-S. This is to hold department data i.e. attributes DeptNo, Budget, and Area about all departments in the southern part of the company. Each fragment is implemented in RAQUEL as a stored relvar. One copy of all the stored relvars is to be held with the CoHQ stack schema. One copy of the stored relvars EMP-N and DEPT-N is to be held with the CoNorth stack schema. One copy of the stored relvars EMP-S and DEPT-S is to be held with the CoSouth stack schema. Thus there will be 2 copies of all the stored relvars except EMP-Fin, and the DBMS will have to keep the 2 copies consistent for those stored relvars. Implementing the Improved Physical Data Storage Arrangements The implementation proceeds in the following sequence : 1. Create any required Stack s that do not already exist. 2. Create the required additional stored relvars in the required Stack s by assigning the required physical storage mechanism or facility to it. 3. Assign the correct relvalue to each new stored relvar. 4. Assign a Storage Expression to each relevant real relvar so that it derives its value from the relvalues of its stored relvars via the Storage Expression. 5. Remove any now unused stored relvars. 1. Create New Stack s Let the default stack schema Stack1 and its corresponding data storage (which currently holds all the DB s data) be already on the Newcastle server. Then the first task is to create a new stack schema on each of the Manchester and Coventry servers, and rename the Newcastle stack schema CoHQ. Firstly the DB ExampleDB is opened to give access to its member schemas (which include the stack schemas) : ExampleDB <==Open 3 Then create the 2 new Stack s : 3 Security considerations are ignored here, so including an identifier parameter in the assignment is also ignored. Page 7 of 13

8 Example of Physical Usage 14 th February 2014 (30 th March 2001) where 4 : CoNorth ==Stack[ URLMan/DefaultMan/ExampleDB/DB/LocationN ] CoSouth ==Stack[ URLCov/DefaultCov/ExampleDB/DB/LocationS ] URLMan and URLCov denote the network links from the computer holding the DBMS installation to the Manchester and Coventry computers respectively on which the stacks will be created. DefaultMan and DefaultCov are the standard initial path names used from the root directory to the directory holding RAQUEL DBs on Manchester and Coventry computers respectively. ExampleDB is the name of the sub-directory holding the ExampleDB DB. DB is the name of the sub-directory holding the DB (as opposed to the sub-directory holding the Meta DB). LocationN and LocationS are the path and directory names of the locations that house the physical data of the stack schemas. Let the default stack be renamed CoHQ for consistency : CoHQ <--Rename Stack1 The Rename assignment has been generalised to handle schemas as well as relvars. 2. Create New Stored Relvars Each stored relvar is created in the location that will store the physical data that comprises the relvar s value. A stored relvar can be created by assigning to it the physical storage mechanism that it is to use. The assignment is executed inside the stack schema which manages the location in question. The statements to achieve this for all the new stored relvars in each of the 3 stack schemas are : CoNorth <==Open EMP-N ==Physical[ B-Tree, EmpNo ] DEPT-N ==Physical[ IndexSeq, DeptNo ] CoNorth <==Close CoSouth <==Open EMP-S ==Physical[ B-Tree, EmpNo ] DEPT-S ==Physical[ B-Tree, DeptNo ] CoSouth <==Close 4 The path names assume a Unix/Linux computer system and a traditional hierarchical directory structure. This is for purposes of illustration only, and should not be construed as preventing very different path names from being used if they are preferred. Page 8 of 13

9 Example of Physical Usage 14 th February 2014 (30 th March 2001) CoHQ <==Open EMP-Fin ==Physical[ Hash, EmpNo ] EMP-N ==Physical[ B-Tree, EmpNo ] EMP-S ==Physical[ B-Tree, EmpNo ] DEPT-N ==Physical[ IndexSeq, DeptNo ] DEPT-S ==Physical[ IndexSeq, DeptNo ] CoHQ <==Close It is not necessary to open a Location within a stack schema in order to execute a ==Physical assignment. The assignments have all been executed within the context of a stack schema. In fact it is never necessary to open a component schema of a stack schema. All the assignments required operate at the stack schema level. The significance of the component schemas within a stack schema is that they are used to organise the meta data needed to manage the physical storage of relvalue data. Therefore they are referenced when retrieving meta data via the Meta operator. Each ==Physical assignment creates a stored relvar by binding the relvar s name to the physical storage mechanism to be used to store its data. The parameters appended to the assignment specify which physical storage mechanism is to be used and the controlling factor(s) to be applied to its execution; the latter can vary since they depend on the nature of the mechanism. For purposes of illustration, it is assumed here that the only parameters needed are the name of the storage mechanism and the name(s) of the attribute(s) to be used as a physical key by the facility. The choice of physical storage mechanisms in the above statements a Hash File system, a B-Tree system, and an Index Sequential system and their usage for the stored relvars, is purely arbitrary and used to illustrate the possibilities that could be applicable. The name of the storage mechanism given as a parameter is the name used within the RAQUEL notation to refer to a particular storage stack plugged into the software architecture of the DBMS installation. (A storage stack is the software module that actually executes everything to do with the physical storage of data). The portfolio of storage stacks plugged into a particular installation can vary; so what can appear as a valid parameter(s) to a ==Physical assignment varies accordingly. The examples show that it is possible for different copies of a stored relvar to use different kinds of physical storage mechanism in different stack schemas. The copy of DEPT-S in stack CoSouth uses a B-Tree file while the copy in stack CoHQ uses an Index Sequential file. 3. Assign Relvalues to Stored Relvars Now that stored relvars have been created with physical storage, it is possible to assign them a relvalue. Page 9 of 13

10 Example of Physical Usage 14 th February 2014 (30 th March 2001) The assignment of a relvalue to a stored relvar can be carried out either within the Storage or within a stack schema. In both cases, for those stored relvars that have multiple copies in different stack schemas, the DBMS will automatically propagate the assigned relvalues to all the copies. In the current situation where all the stored relvars are to be dealt with, it is easier to deal with them from within the Storage, rather than visit each stack schema in turn. Therefore the following assignments are used : Storage <==Open EMP-Fin <--Store EMP Project[ EmpNo, Salary, Tax ] DEPT-N <--Store DEPT Restrict[ Area = North ] EMP-N <--Store EMP Project[ EmpNo, EName, DeptNo ] Join[[ DeptNo ] DEPT-N DEPT-S <--Store DEPT Restrict[ Area = South ] EMP-S <--Store EMP Project[ EmpNo, EName, DeptNo ] Join[[ DeptNo ] DEPT-S Storage <==Close In each <--Store assignment, the expression on the RHS is evaluated and physically stored as the value of the stored relvar on the LHS. (Hence <--Store is a value assignment, not a binding assignment). In this case, the relvalues of the stored relvars are derived from those of the preexisting relvars EMP and DEPT. The value of EMP-Fin is a projection of the relevant attributes of EMP. The value of DEPT-N is a restriction of DEPT such that the tuples in the result contain only the value North in the Area attribute. The value of EMP-N is derived by projecting out the relevant attributes of EMP and then limiting the tuples in the result to those corresponding to a northern department; the latter is achieved by means of a left-hand semi-join with DEPT-N, which contains only northern departments. The corresponding approach is used to derive the values of DEPT-S and EMP-S. Note : The above statements would still be valid even if EMP and DEPT were empty of tuples. The reltypes and keys of the stored relvars are automatically derived from the relvalues assigned to them. (Ad hoc constraints are enforced at the level of the logical relational model. They do not exist in the data storage model). 4. Assign Storage Expressions to Real Relvars Storage Expressions are now assigned to the real relvars EMP and DEPT. The assignment values overwrite the previous assignment values (which in this case were the default values). Page 10 of 13

11 Example of Physical Usage 14 th February 2014 (30 th March 2001) This is achieved as follows : Logical <==Open EMP ==Equate ( EMP-N Union EMP-S ) Join[ EmpNo ] EMP-Fin DEPT ==Equate DEPT-N Union DEPT-S Logical <==Close The value of the left-hand relvar is now obtained by executing the right-hand expression. In this case, the names EMP and DEPT previously referred to both stored relvars and real relvars. Now the names only refer to real relvars. Because previously each name referenced 2 relvars by default, when the stored relvars ceased to be used they are no longer referenced in the Storage Expressions that have just been assigned - the DBMS automatically removes the stored relvars to clean up after its default so to speak. 5. Remove Unwanted Stored Relvars There are no stored relvars that are no longer needed and which should therefore be removed. Thus the DB can now be exited : ExampleDB <==Close Note that the stored relvars named EMP and DEPT were only removed by default because they were created by default. If stored relvars named EMP and DEPT were to be created by deliberate user action, then they would have to be removed by deliberate user action; no default removal would occur as a result of the stored relvars having the same names as real relvars. The set of all relvar names appearing in the logical relational model those of real, virtual, source and sink relvars and the set of all relvar names appearing in the physical storage model those of stored relvars are disjoint. Therefore the same name can arise in both sets (although consideration of good design may affect the extent to which relvar names are duplicated). It would be undesirable for relvar names to be unavailable in the logical model due to their appearance in the physical model, since there is no requirement for the user of the logical model to have any awareness of its implementation, e.g. of the stored relvars that it uses. Furthermore circumstances may arise in which it is useful for a DB Administrator to employ the same relvar names in the physical model to help relate storage considerations to the logical model. Overview of Improved Physical Data Storage Arrangements The following Venn diagram illustrates the revised physical data storage : Page 11 of 13

12 Example of Physical Usage 14 th February 2014 (30 th March 2001) Stack CoHQ CoHQLocation EMP-FinSt ; Hash DEPT-NSt ; IndexSeq DEPT-SSt ; IndexSeq EMP-NSt ; B-Tree EMP-SSt ; B-Tree Storage EMP-Fin DEPT-N EMP-N DEPT-S EMP-S DEPT-NSt ; IndexSeq EMP-NSt ; IndexSeq DEPT-SSt ; B-Tree EMP-SSt ; B-Tree Stack CoNorth Stack CoSouth Note that the Storage and the Location of the CoHQ Stack are identical sets, due to the CoHQ Stack holding a copy of all the stored relvars in the ExampleDB. Comments on the Design Because the RAQUEL DBMS implements a formal model of physical storage, the physical data storage arrangements will be executed regardless of how well they are designed. Nevertheless good physical design is important if the resultant data storage is to be efficient and effective. The example discussed above includes two aspects of design that are important in practice but which the example glosses over : 1. The placing of data near to the users of that data, so that the minimum of overheads are incurred when users access that data. In the example, this was handled by storing the data of copies of fragments of the 2 real relvars EMP and DEPT with stack schemas in locations that were easily and efficiently accessed by the users. In reality one would need to investigate the access times and networking problems involved when spreading out the data across a network, and what overheads would be incurred by storing more than one Page 12 of 13

13 Example of Physical Usage 14 th February 2014 (30 th March 2001) copy of a fragment. The benefits of the resilience gained from storing multiple copies of the same data should also be considered, as should security considerations. 2. The design of fragments to minimise the overheads involved in accessing them. It is important to design fragments so that they are disjoint. This not only minimises data storage requirements but also means that access can be limited to the minimum number of fragments, thereby improving physical performance. This was achieved in the example as follows. For horizontal fragments, EMP-N and EMP-S, and DEPT-N and DEPT-S, the fragments were made completely disjoint by employing suitable restriction constraints. For vertical fragments, EMP-Fin with EMP-N and EMP-S, the fragments were made as disjoint as possible by projecting out complementary attributes. The only attribute overlap was the Key, which has to be common to all vertical fragments in order to allow them to be recombined. Appendix Use of the ==Stack Assignment In addition to the monadic use illustrated earlier, the assignment can also be used dyadically. For example : CoHQ ==Stack[ URLNcl/DefaultNcl/ExampleDB/DB/LocationNcl ] Stack1 This assignment creates a stack schema whose value is that of the right-hand operand (Stack1) and whose location is specified by the parameter, and binds it to the name given as the left-hand parameter which thereafter is used to reference that stack schema. Whereas the monadic use creates and assigns to the left-hand operand name a stack schema whose component schemas are empty, the dyadic use creates a copy of a stack schema value in a new location and with a new name; there is no reason why the component schemas of the copied stack schema need be empty. Use of the <--Remove Assignment The <--Remove assignment is generalised so that it applies to schemas as well as relvars. Thus stack schema Stack1 (say) could be removed with the statement : or Stack1 <--Remove Stack1 <--Remove[ PathName ] The effect of the optional parameter in the latter case is to archive the stack schema value denoted by Stack1 in the directory defined by PathName ; the archive could be used in future if necessary to restore the stack. Page 13 of 13