Official Statistics - Relational Database Management Systems. Official Statistics - Relational Database Management Systems

Size: px

Start display at page:

Download "Official Statistics - Relational Database Management Systems. Official Statistics - Relational Database Management Systems"

Alexander Wilkerson
5 years ago
Views:

1 Chapter 1 Database fundamentals 1.1. What is a database? A database is a stored collection of data designed for efficient information retrieval, efficient data maintenance and centralised control of an organisation's data 1.2. Database Structure A database contains one or more tables A table contains one or more records A record contains one or more fields A field is the smallest unit of data considered in a database If part of a field is used by itself it should be split into two fields, eg phone number - area code & number. All records in a table have the same structure, they are composed of the same fields however the field values differ from record to record. Examples: A college database would contain the following tables: Student table with one record for each student, each record would contain one field for each student detail recorded eg: Student_Number, Surname, Forename, Faculty, Department, Year, Enrolment_date. A stock/sales database could contain the following tables: Customer table containing fields such as Account_number, Name, Address, Phone, Balance, etc. Stock table, Order table, Invoice table, Credit Note table, Sales, Reps Table Section 1.3 Database Models Three different models: Hierarchical, Network, Relational. Most modern databases are Relational and that's what we'll concentrate on. The term relational stems from the fact that each record in the database contains information related to a single subject and only that subject. Also, data about two classes of information (such as customers and orders) can be manipulated as a single entity based on related data values. For example, it would be redundant to store customer name and address information with every order that the customer places. In a relational database system, the information about orders contains a field that stores data, such as a customer number, which can be used to connect each order with the appropriate customer information. In a relational database management system, sometimes called an RDBMS, the system manages all data in tables. Tables store information about a subject (such as customers or students) and have columns that contain the different kinds of information about the subject (for example, customers' or students' addresses) and rows that describe all the attributes of a single instance of the subject (for example, data on a specific customer or student). Even when you query the database (fetch information from one or more tables), the result is always something that looks like another table. 1 2

2 Section 1.4 Database Capabilities An RDBMS gives you complete control over how you define your data, work with it, and share it with others. The system also provides sophisticated features that make it easy to catalogue and manage large amounts of data in many tables. An RDBMS has three main types of capabilities: data definition, data manipulation, and data control. All this functionality is contained in MS Access. Data definition - You can define what data will be stored in your database, the type of data (for example, numbers or characters), and how the data is related. In some cases, you can also define how the data should be formatted and how it should be validated. Data manipulation - You can work with the data in many ways. You can select which data fields you want, filter the data, and sort it. You can join data with related information and summarise (total) the data. You can select a set of information and ask the RDBMS to update it, delete it, copy it to another table, or create a new table containing the data. Data control -You can define who is allowed to read, update, or insert data. In many cases, you can also define how data can be shared and updated by multiple users. Section 1.5 Some Relational Database Terminology Relation - Information about a single subject such as customers, orders, students, or colleges. A relation is usually stored as a table in a relational database management system. Attribute - A specific piece of information about a subject, such as the address for a customer or the amount of a contract. An attribute is normally stored as a data column, or field, in a table. Relationship - The way information in one relation is related to information in another relation. For example, customers have a one-to-many relationship with orders because one customer can place many orders, but any order belongs to only one customer. Secondary school students might have a many-to-many relationship with colleges because each student is interested in applying to multiple colleges, and each college receives applications from many students. Join - The process of linking tables or queries on tables via their related data values. For example, customers might be joined to orders by matching customer ID in a customers table and an orders table. You can also join information on related values from multiple tables or queries. For example, you can join student information with college application information to find out which students applied to which colleges. You can join employee information with contract information to find out which salesperson should receive a commission. 3 4

3 Section 1.6 Rules governing database structures An attribute's domain is the set of all possible values for that attribute. Every attribute value must belong to that attribute's domain. For example student_age must have a domain like [13, 110] even though most students in a college will be in the interval [17, 27] To be a relation a table must have the following three properties: a. No two rows can be the same, every row in a table must have a unique attribute or group of attribute values that distinguish it from all other rows in the table. This attribute or group of attributes will act as a primary key for the table. b. Each attribute must have a unique name, becaues attribute names identify attributes. c. The order of rows and attributes must not be significant. Section 1.7 Keys Primary keys A primary key is an attribute or a group of attributes whose value uniquely identifies each row in the table. A primary key has the following three properties: 1. It uniquely identifies each record in a table. No two records have the same primary key value. 2. Each record has a primary key value, ie it is never NULL. 3. A primary key is MINIMAL, ie no subset of its attributes is also a primary key. A key which consists of a single attribute is called a simple key. Often a relation's primary key is made up of more than one attribute. This is either a compound key or a composite key. Let's look at an example to illustrate the difference: Example of a Compound Key: A consulting company works on several database projects. The table below records data on consultant's time: Consultant # Project # Role Time 003 C232 Leader C771 Analyst A152 Interviewer C771 Leader 20.0 In this example no single attribute can act as a primary key. A consultant can work on many projects and projects may have many consultants assigned. The combination of Consultant # and Project # can uniquely identify a row. 5 6

4 Example of a Composite Key: Projects can have many activities Project # Activity # Start Week End Week C C C Again in this example no single attribute can act as a primary key. The combination of Project # and Activity # can uniquely identify a row. The difference between this and the previous example is that Activity # on its own is not meaningful it needs to be combined with Project # to make sense Foreign Keys A Foreign Key is an attribute in one table which is a primary key in another table. This is the way that records are related in a Relational Database. Below are two table schemes with primary keys underlined and foreign keys denoted by an asterix Project = Project # + Project Description + Project Type + * Client # Secondary (Alternate, Logical) Key A Secondary Key is an alternative way to find a record, it is often used when the value of the primary key is unknown. It does not need the properties of a primary key. eg Employee_Name instead of Employee_# 1.8 Information retrieval Information and Data are not the same. Information is Meaningful data. Summaries of data usually become information. Retrieval is concerned with getting information out of the database, the main kinds are: On-line Queries Reports On-line database services via a link eg accessing a DB on the WWW. Retrieval usually requires specification of 1. The tables to retrieve from 2. The fields to retreive from selected tables 3. Criteria to select a subset of the records. There are many retrieval features: SORT reports. SUMMARISE reports. CRITERIA to restrict selections. RESTRICT OUTPUT fields. (by date, name etc) (Totals, averages etc.) (beginning date - end date) (show names not addresses) Client = Client # + Name + Business Type 7 8

5 1.9 Relational Algebra Selection: Subset of records from a table Projection: Subset of fields from a table Union: Combine all records from two tables, tables must have same fields. Intersection: Collection of all records which are in both tables. Tables are normally stored on disks. But some tables may only exist in memory (RAM) for the duration of a process. These tables which are usually the result of some relational algebra operation are called virtual tables ( or views, queries or dynasets) Data Maintenance Data maintenance is keeping the data in the DB up to date and is considered the biggest chore in using a database. Most incorrect data in a database is due to entry errors. In building a database you must try to anticipate as many errors as possible and guard against them using referential integrity, domain look-up tables, validation rules, input masks etc. There are three main activities in Data Maintenance: Adding a new record Changing values in an existing records fields Deleting an existing record Referential Integrity Probably THE MOST IMPORTANT aspect of a database, however ignored by many database designers. (Database design unlike many other areas of IT is populated by amateurs and this leads to some really shoddy work.) Referential Integrity concerns cross-references between tables. Foreign keys are the only way to cross-reference tables in a relational database. A database has referential integrity IFF each foreign key value has a matching primary key value in the related table Integrity Constraints Integrity constraints are rules to ensure that a database retains its refereential integrity when changes are made. Integrity constraints ensure that either A change is not allowed if referential integrity would be lost or Appropriate follow-on changes are made to other tables so that integrity is preserved. The three types of integrity constraints and their effect when deleting a record whose primary key value occurs as a foreign key value in another table are: 1. Restrict Deletion is not allowed if the primary key value occurs as a foreign key value in another table 2. Cascade Immediately after a record is deleted, delete each record in the other database tables which has the deleted primary key as a foreign key. 3.Nulllify Immediately after a record is deleted, search for each record in the other database tables which has the deleted primary key as a foreign key and replace all the foreign keys with NULL values. Nullify is an unsatisfactory solution! 9 10

6 1.11 Order Physical Order A table is in physical/natural order if record number 1 is first, record 2 second etc. Records are always entered into a table in physical/natural order, new records being added to the end of the table and assigned the next highest record number. When a record is deleted subsequent records are re-numbered so that there's no gap in the numbering sequence Indices An index controls the sequence in which records are used. Indices have two man purposes: 1. For retrieving records in an order, eg alphabetical order 2. For fast access to a record, eg to quickly find a student with student number in a table with 100,000 present and past UCD students Indexes use very fast B-Tree methods not just sequential search methods. ASCII - American Standard Code for Information Interchange 1.12 RDBMS A Relational Database Management System is software to manage a database eg Oracle, Ingres, Informix, Sybase, SQL Server, Access A RDBMS usually includes SQL and a 4GL RDBMS usually feature: Easy to implement database design structures, indices etc. Triggers: When one event occurs a sequence of other events begins Helps to preserve Referential Integrity Transaction Processing Contention Prevention Automatically decides best methods for accessing data etc without users noticing Includes data entry validation features Most RDBMS do not have full automatic checking of referential integrity, programmers must ensure the preservation of Referential Integrity Transaction processing A transaction updates several tables in one chain of events. ROLLBACK - If a transaction goes wrong, undo it by playing it backwards to return to the original state. COMMIT - When a transaction has been successfully completed, permanently commit the updates to the database

7 Contention prevention Contention means problems due to several users trying to access the same data. This can be avoided using: FILE LOCK While one user accesses a table it is inaccessible to other users RECORD LOCK While one user accesses a record it is inaccessible to other users DIRTY READ Permit Reading data even if another user is also using it SQL Structured Query Language used in Relational DBs for information retrieval. 1. GUEST LECTURE FRIDAY 14 th Dr. K Mc Daid Example: SELECT SUM(price) FROM invoice WHERE cust = "731" GL 4 th Generation Language C ++, Visual Basic etc. allows Form, Report designs without writing 3gl programmes. A Form is a screen to make data entry and record viewing easier. Many Forms designed to look like office paper. A Report is typically columns of data, including headings, page numbers calculated fields and groupings. 2. Times for Computer Labs in ACCESS 13 14

8 Chapter 2 Setting up a Database When designing a database the first task one is faced with is to construct tables and relationships, there are some techniques which will help with this. 2.1 Normalisation This is a a transformation of complex data structures into simple tables or relations. It is a process of removing all data redundancy from a design. Data redundancy is defined as the storing of a fact more than once. A fact is an association of two data values, eg: ID#: 001, Name: John Smith Storing customer's names in a table of invoices as well as in a customer table would be redundant. Normalisation gets at the underlying logical dependencies between data elements, just because data appears together on a physical form does not mean they are part of the same grouping. 2.3 Example Below is a screen for a program to enter details for a new consultant onto the system of a consultancy firm. We will apply the normalisation procedure to this example. CONSULTANT DETAILS Cons No. Name Grade Salary Car Type 019 Murphy A1 S1 A Address: 1 South Avenue Abbingdon 0X2 3BH Code Description Qualifications SK01 C ++ BSc SK15 SSADM NCEA Cert SK10 CAD/CAM 3 Years 2.2 Normalisation Steps There are 6 steps involved in the process of normalising a database. Write the data of each source in an Un-Normalised form (UNF) notation. Convert the UNF into 1 st Normal form (1NF) Convert the 1NF into 2 nd Normal form (2NF) Convert the 2NF into 3 rd Normal form (3NF) Optimise the 3NF relations Apply the 3NF tests 15 16

9 2.3.1 Write data in UNF This stage requires 2 steps. A. Identify a primary key In an employee table a good key would be employee number or RSI number In general we should choose a primary key to be the shortest possible the least changeable the most meaningful to others the least number of attributes B. Write as a UNF Relation Notation: The primary key is underlined and braces enclose repeating data structures. Consultant = Car Type + { Skill Description + Qualification} Convert to 1 st Normal Form To transform from UNF to 1NF: 1. Remove any repeating structures from the UNF 2. Create a new relation for each such removed structure 3. Copy the primary key of the UNF relation into the new relation 4. Decide the primary key of the new relation UNF 1NF 2NF 3NF Car Type + { Skill Description + Qualification} Car Type Skill Description + Qualification Note: Composite keys are enclosed in parentheses in order to associate the qualified attributes with the qualifying attributes. Invisible data may also appear in the output data structure, eg the colour of a form, shaped holes on a ticket to indicate whether it is checked or not

10 2.3.3 Convert to 2 nd Normal Form To transform from 1NF to 2NF: Remove any part key dependencies. This is achieved by examining relations which have a compound or composite key. For each non-key attribute in the relation we ask "Does this attribute depend on the whole key or only part of it?" When part key dependencies are identified in a 1NF relation the attributes involved are moved to a new relation. UNF 1NF 2NF 3NF Car Type + { Skill Description + Qualification} Car Type Car Type Skill Description + Qualification Qualification Skill Description Convert to 3 rd Normal Form To transform from 2NF to 3NF, this is similar to the previous step. Here we examine all dependencies between attributes including key attributes. We ask "Does attribute A determine attribute B?" The dependent attribute is removed from the 2NF relation to form a new relation whose key is the determinant attribute. The determinant attribute also stays in the original 2NF relation as a foreign key shown with an asterix *. UNF 1NF 2NF 3NF Car Type Car Type Skill Description + Qualification Qualification Skill Description * Car Type Salary Scale Qualification Skill Description 19 20

11 2.3.5 Optimisation Optimisation comprises combining relations which have exactly the same key Apply the 3NF tests It is possible that data inter-dependencies have been introduced, so we must check the following: Given a value for the primary key of a relation, is there only one possible value for each of the associated attributes? Is each attribute directly dependent on the full key? Normalisation was developed by Dr. Codd of IBM in the 70's. "In perfectly normalised data, each attribute in a relation must depend on: the key, the whole key, and nothing but the key (so help me Codd)." Normalised data takes up the minimum of storage space however the use of foreign keys means a lot of cross referencing and so sometimes we de-normalise before implementing a database. Chapter 3 Logical Data Structures 3.1 Components of a LDS Entities 1. An entity is a collection of occurences of things about which the organisation stores information 2. The information on each entity occurrence is stored as logical groups of attributes 3. A Primary key exists or could be made which identifies each occurrence A Rectangular Box represents an entity Relationships A relationship is an association between two entities that models operational or business rules. 3.2 Degree of Relationship There are three possible degrees for a relationship: 1:1 - one to one 1:M - one to many M:N - Many to Many Question: what is the degree of the relationship between each of the following pairs of entities: Customer to Invoice Student to Subject Doctor to Patient Invoice to Product Employee to Department 21 22

12 3.3 Developing an LDS 1. Identify initial entities 2. Identify direct business/operational relationships between entities and determine degree of each relationship 3. Identify Operational Masters 4. Construct initial diagram 5. Identify additional characteristics of relationships 6. Resolve M:N and 1:1 relationships 7. Rationalise the LDS 8. Validate the LDS Identify initial entities Typical entities are Doers of operations (staff, departments etc) Records of operations (invoices, bookings etc) Domain look-up lists (countries, grades, course types etc) Identify Operational Masters Data can always be accessed in relations using the keys of the relation but sometimes it is necessary to access data using attributes other than keys. These non-key methods are called Operational Masters and are represented by an OVAL. An example would be that sometimes it is necessary to retrieve an employee by their name and not just their employee number. We need to investigate the types of access required and the types of groupings required for output Resolve M:N and 1:1 relationships M:N relationships cannot be implemented directly in a database and we need to insert linking entities. Example: athletes enter many events 1:1 relationships mean that the same entity has been included two ways Additional Relationship Features Entities are Master or Detail EX: Patient may be Detail of Ward and also of Doctor Participation is Mandatory or Optional EX: Political Parties and TD's Relationships can be Recursive EX: Employees supervise other Employees Relationships can be Multiple EX: Student is currently in a Department Student was previously in a Department Relationships can be exclusive EX: Customer pays by cash or credit not both 23 24

13 25

Relational Database Components

Relational Database Components Chapter 2 Class 01: Relational Database Components 1 Class 01: Relational Database Components 2 Conceptual Database Design Components Class 01: Relational Database Components