Managing the Database

Transcription

1 Slide 1 Managing the Database Objectives of the Lecture : To consider the roles of the Database Administrator. To consider the involvmentof the DBMS in the storage and handling of physical data. To appreciate different kinds of file organisation and access method. To appreciate the need for meta data.

2 Slide 2 Database Administrator(s) This can vary. 1 person part-time - full-time team. Depends on the nature, size and usage of the DB : Large DB shared by many users/applications - traditional. Relatively small DB for one person/team/application. Very large data warehouse DB for data mining. different DBMSs may be used for different purposes different levels of technical support required. Depends on how work allocated w.r.t. other computer staff. DBA may/may not be involved in : application development/support; computer system/network support. Members of a DBA team may each have different duties, ranging from strategic decision making to daily operational tasks - see later.

3 Slide 3 Why is DB Management Needed? A DB is a coherent, integrated collection of data. the DB needs to be managed because : whether the coherence applies to data for one application or many, the coherence must be created by design, and maintained as the DB evolves; data is now accepted as a valuable organisational asset, that must be cared for; DB management tasks physical data independence implies the performance tuning of the DB s physical storage. These 3 aspects each give rise to a set of database management tasks. Each set of tasks is now considered in more detail :-

4 Slide 4 DBA s Coherence Tasks Creation of the DB. Obtaining a suitable DBMS. Design of the Logical Schema. Design of Sub Schema(s). Design of Physical Schema. Provision of suitable hardware. Implementation of the DB design. Insertion/loading of valid data into the DB. Some or all, as required. Maintenance & Extension of the DB. A few/some/all of the creation tasks as appropriate. Liaising with : End Users. Application Developers and Systems Staff. To meet their aims & enforce realistic constraints. Slide 5 DBA s Caring Tasks Maintaining security of DB. Protection against unauthorised access. Ensure a requested operation on a requested object by a requesting user is acceptable. Need defence in depth; e.g. audit trails, data encryption. Protecting DB against loss or damage. Need backup copy of DB + Transaction Log. From latest valid copy of DB, roll forward through transaction log repeating transactions till current DB state restored. Maintaining Standards. Needed for procedures, software, documentation, etc to support other DB activities and ensure their effectiveness. Day-to-day maintenance operations. Managing DB restarts, keeping backup data, correcting DB errors, investigating problems, updating users authorisation, etc.

5 Slide 6 DBA s Performance Tuning Tasks (1) Purpose of Physical Data Independence is to allow a relations s data to be physically stored in many different ways without this affecting what a user/programmer writes in their (SQL) statements to use that relation. Thus if a relation s data is moved from one physical storage arrangement to another, the user/programmer is unaware of it. DBA can and should change a relation s physical storage if performance can be improved. Slide 7 DBA s Performance Tuning Tasks (2) Purpose - optimise trade-off between : User-level - update & retrieval, different users /applications needs; Hardware level - hard disc, RAM, CPU and network usage. Design the initial Physical Schema. Map base relations & views to physical files. Decide file locations w.r.t. discs and network nodes. Decide record formats, file organisation & access. Monitor usage and performance of DB. Amend Physical Schema when altered usage &/or requirements demand it; and when DB is extended/altered. To decide on the trade-offs, physical files and locations, we need to consider more about physical file storage.

6 Slide 8 Consideration of Physical File Storage In order to choose optimal physical file designs, the DBA must understand : How the DBMS handles statements input to it for execution. How a DBMS uses a computer s memory for the storage and handling of a DB. How files are organised and accessed in physical storage. What happens at the physical level to execute a DB statement. The performance characteristics of different physical file types. These topics are now reviewed. The intent is not to show how a DBA can optimise a DB s physical file design - this is a very large subject on its own - but to give sufficient background information to appreciate the nature of the problem. Slide 9 DBMS s Execution of a Statement On receipt of a (SQL) statement, the DBMS : 1. Determines what must be done to execute it; i.e. tokenises and parses it. 2. Follows the mappings between Sub, Logical and Physical Schemas to determine what data to physically read/write from/to disc; 3. Optimises the method of execution. 4. Executes the statement. Successful optimisation depends on the DBA s choice of physical file designs for data storage, as well as the DBMS s optimiser. The DBMS must output data in the form of relations, even when a relation s data is physically stored in a quite different way.

7 Slide 10 Example Retrieval Example SQL query :- SELECT * FROM Customer WHERE ACC_NO = ; Let Customer be a view. DBMS : Tokenise & parse. Determines what the statement means. Gets definition of view Customer. Translates query into a logical equivalent using base table(s). Gets location of file(s) holding the base table data, and their organisation(s) & access method(s). Determines optimum query method. Executes query. Use schema data. Slide 11 Executing the Example Retrieval Get the record with customer account number DBMS Here s the record you wanted That s on the 27th. page of the file called customer File Manager Here s the page of the file you wanted That s on page 14 of cylinder 127 Disk Manager Here s the page you wanted May be part of the DBMS or the Operating System. Part of the Operating System.

8 Slide 12 Simplified Computer Architecture CPU Random Access Memory (main) I/O Control Typically hard discs Backing Storage (secondary) This is an overview schematic of a computer system. The main memory is where data is held in order to be processed by the CPU. The secondary memory is where data is permanently stored. Data in secondary memory cannot be processed in situ but needs to be copied into main memory for processing; processing includes sending data to a peripheral (e.g. a printer) etc, as well as literally manipulating the data.. Other overall architectures are possible. For example, there can be tertiary memory for holding archived data. However these are ignored as they do not contradict what is given here, but elaborate on it.

9 Slide 13 Primary vs. Secondary Storage Primary : Fast For volatile data Expensive Limited in size Secondary : Slow For permanent data Cheap Removable / Expandable The relative costs of primary (= main) and secondary data storage remain permanently and broadly true, although the actual costs of both have consistently come down significantly over the years, a trend which is expected to continue into the indefinite future. Volatile data is that which changes rapidly and frequently, as opposed to that which is updated infrequently and kept for a long time. Various technologies have been used in the past to implement primary and secondary memory, and it is anticipated that newer technologies will be used in the future.

10 Slide 14 Magnetic/Hard Disks Rotating disc. Each surface comprises concentric tracks. Each track split into blocks. Read/write head for each surface. Head moves across to required track, waits till required block comes underneath, reads/writes data from/to block. Discs may be stacked onto one spindle; each surface accessed simultaneously. Cylinder corresponding tracks on each surface. parallel read/write of 1 cylinder. Operating systems read/write one page at a time. 1 page 1 / 2 / 4 / 8 /... block(s). Block and page nomenclature varies with operating system and hardware - check the terminology relevant to your computer system. Slide 15 Buffers & Cache Memory Computers read/write data from/to secondary memory via Buffers. They are a special part of RAM whose purpose is to handle disc I/O. Required for efficiency. 1 Buffer Buffer 2 4 Buffer use strategies : Double Buffering (alternate filling & emptying), Read Ahead, etc. Disc Cache is a special kind of buffer - holds frequently read data from disc, to minimise re-reading it from disc. DBMS must handle buffers. With the double buffering illustrated, data is copied into main memory from one buffer while simultaneously more data from disc is copied into the other buffer. This speeds up the overall disc-to-main memory copying process.

11 Slide 16 DB Use of Memory DBs typically : need to be kept for a long time; are large, and so need a lot of memory. store them on Secondary Storage. DBs users typically require significant processing of data. Picking out parts of relations, merging relations, doing calculations of stored data, sorting data, etc. read data into Primary Storage, and process the data there. DBMS handles all this for the user. In principle, a DB could also hold data archived on a tertiary memory system. We ignore such complications here..

12 Slide 17 File Structure and Content A file consists of a sequence of records. Records in a file may have : a fixed or variable structure, a fixed or variable length. A record consists of a sequence of fields. Each field holds a value of a certain data type. (A record often holds the values of one tuple). A disc block normally holds several records :- Records accumulate in a block till there is no further room in it. Data Item Data Item Data Item Data Item Data Item Data Item Data Item Data Item Data Item Data Item Data Item Data Item Data Item Data Item Data Item A large record could be spread over several blocks, or a large number of small records could be held in one block. Slide 18 A File Page 2 Page 3 etc. EOF record record record record record record record free space Page 1

13 Slide 19 Disc Access Time Access Read From OR Write To. Disc access is 10,000-1,000,000 times longer than RAM access. Actual speeds continually improve. Despite variations due to the technology used, the relative speeds of RAM and disc are always hugely different. always extremely important to minimise disc access times. Time taken for disc access depends on : whether access is Read or file organisation how the Write. records are laid out in the file. precisely what data is to file access method how the be accessed; e.g 1 specific required record(s) are found in record, a certain range of the file. records, all the file. This access time difference is despite the use of buffers.

14 Slide 20 Minimising Disc Access Time (1) Problem : File type (= organisation & method) best for one kind of user access (= read/write & data to be accessed) is worst for another. DB users have very varying access needs. Solution : DBA chooses suitable file types and clustering of files on pages. DBMS automatically optimises within these parameters. DBA may also have to set other parameters, e.g. buffer space, page size, that are then used by the DBMS. Possible and worthwhile parameters depend on the individual DBMS. Slide 21 Minimising Disc Access Time (2) Actual strategies used : Minimise data transfers to / from disc : Minimise search path (= no. of pages read). Once in main memory, maximise use of each page. Keep high hit areas (e.g. index pages) in data cache. Minimise disc handling : Minimise head movement: read whole cylinders. Minimise Latency: read whole tracks. Choose best compromise page size. Minimise CPU waiting: Buffer I/O, read ahead. Possible strategies available depend on the individual DBMS.

15 Slide 22 File Types File Organisation : Serial File - new records added to the end of a file; records not in sequence; may mark a deleted record with a tombstone. Sequential File - all records maintained in order of value(s) in one or more fields. (Could correspond to candidate key values). File Access Method : Sequential Access - go through file s pages in some sequence. Indexed Access - use an index to go straight to the required page. Many types of index : B-tree, Secondary, Bitmap, etc. Hashed Access - calculate page location with a hash algorithm. Pointer Chain - pointer in a retrieved record is used to access the next page. Different combinations of organisation & access method give very different performance characteristics. choose appropriately to serve required access of data. In principle, file organisation and file access method are two completely different and orthogonal aspects of file design. In practice, the way(s) in which the file will (mainly) be used are considered, and the two are chosen to complement each other in meeting the usage requirements.

16 Slide 23 Serial Organisation, Sequential Access Page 2 Page 3 etc. EOF Page 1 Slide 24 Sequential Organisation & Access EOF Page 2 Page 3 etc Page 1 When it comes to finding a particular record, the latter case allows the possibility of a more efficient sequence of page reading than the former case, as the former case has no specific sequencing of records on pages while the latter does.

17 Slide 25 Indexed (Access) Sequential File Page 3 Data pages Index key address etc Page 2 Page 1... Using an index to find the page on which a particular record exists is normally much more efficient than any kind of search through the whole file. It corresponds to using an index in a book to find a topic rather than skimming through the whole book until the topic is found.

18 Slide 26 DBMS s Need for Meta Data In addition to the DB s data, the DBMS needs considerable data about the DB data in order to function. For example, in order to operate, the DBMS needs to know : Names of DB relations, & whether base or view. Names & data types of attributes in relations. Mapping between relations and physical files Names and locations of physical files. Organisation & access methods of files. Buffering available. etc. meta data = data about data Meta data is stored in a Data Dictionary / SQL Catalog. This is another DB. It is stored & used in the same way as the main DB. The DBMS automatically updates it when relations, etc are created, retrieves from it to execute statements. Slide 27 DBA s Need for Meta Data The DBA needs similar data about the DB to carry out their functions : Coherence tasks : e.g. check on relations (attributes & data types, integrity constraints), schemas, files. Caring tasks : e.g. check on authorised users and their access privileges, state of backups and logs. Performance Tuning tasks : e.g. check on usage, file sizes, file types. Often used to look up data for mundane, everyday tasks, since a DB is often too big for the DBA to remember everything. Note that most of this meta data is also used by the DBMS.