Outline. Database Management and Tuning. Index Tuning Examples. Exercise 1 Query for Student by Name. Concurrency Tuning. Johann Gamper.

Size: px

Start display at page:

Download "Outline. Database Management and Tuning. Index Tuning Examples. Exercise 1 Query for Student by Name. Concurrency Tuning. Johann Gamper."

Johnathan Skinner
5 years ago
Views:

1 Outline Database Management and Tuning Johann Gamper 1 Free University of Bozen-Bolzano Faculty of Computer Science IDSE Unit Conclusion Acknowledgements: The slides are provided by Nikolaus Augsten and have been adapted from Database Tuning by Dennis Shasha and Philippe Bonnet. Johann Gamper (IDSE) Database Management and Tuning Unit 7 1 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 2 / 34 Exercise 1 Query for Student by Name The examples use the following tables: Employee(ssnum,name,dept,manager,salary) Student(ssnum,name,course,grade,stipend,evaluation) Student was created with non-clustering index on name. SELECT * FROM Student WHERE name= Bayer Problem: Query does not use index on name. Solution: Try updating the catalog statistics. Oracle, Postgres: ANALYZE SQL Server: sp createstats DB2: RUNSTATS Johann Gamper (IDSE) Database Management and Tuning Unit 7 3 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 4 / 34

2 Exercise 2 Query for Salary I Exercise 3 Query for Salary II Non-clustering index on salary. Catalog statistics are up-to-date. SELECT * WHERE salary/12 = 4000 Problem: Query is too slow. Solution: Index not used because of the arithmetic expression. Two Options: Rewrite query: SELECT * WHERE salary = Use function based index. Non-clustering index on salary. Catalog statistics are up-to-date. SELECT * WHERE salary = Problem: Query still does not use index. What could be the reason? Solution: The index is non-clustering. Many employees have a salary of 48000, thus the index may not help. It may still help for other, less frequent, salaries! Johann Gamper (IDSE) Database Management and Tuning Unit 7 5 / 34 Exercise 4 Clustering Index and Overflows Johann Gamper (IDSE) Database Management and Tuning Unit 7 6 / 34 Exercise 5 Non-clustering Index I Clustering index on Student.ssnum Page size: 2kB Record size in Student table: 1KB (evaluation is a long text) Problem: Overflow when new evaluations are added. Solution: Clustering index does not help much due to large record size. A non-clustering index avoids overflows. Employee table: 30 employee records per page each employee belongs to one of 50 departments (dept) the departments are of similar size SELECT ssnum WHERE dept = IT Problem: Does a non-clustering index on Employee.dept help? Solution: Only if the index covers the query. 30/50=60% of the pages will have a record with dept = IT table scan is faster than accessing 3/5 of the pages in random order Johann Gamper (IDSE) Database Management and Tuning Unit 7 7 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 8 / 34

3 Exercise 6 Non-clustering Index II Exercise 7 Statistical Analysis Employee table: 30 employee records per page each employee belongs to one of 5000 departments (dept) the departments are of similar size SELECT ssnum WHERE dept = IT Problem: Does a non-clustering index on Employee.dept help? Solution: only 30/5000=0.06% of the pages will have a record with dept= IT table scan is slower Auditors run a statistical analysis on a copy of Emplyee. Queries: count employees with a certain salary (frequent) find employees with maximum or minimum salary within a particular department (frequent) find an employee by its social security number (rare) Problem: Which indexes to create? Solution: non-clustering index on salary (covers the query) clustering composite index on (dept, salary) using a B + -tree (all employees with the maximum salary are on consecutive pages) non-clustering hash index on ssnum Johann Gamper (IDSE) Database Management and Tuning Unit 7 9 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 10 / 34 Exercise 8 Algebraic Expressions Exercise 8 Solution Student stipends are monthly, employee salaries are yearly. Which employee is paid as much as which student? There are two options to write the query: SELECT * SELECT * FROM Employee, Student FROM Employee, Student WHERE salary = 12*stipend WHERE salary/12 = stipend Index on a table with an algebraic expression not used. Problem: Which query is better? If index on only one table, it should be used. Index on both tables, clustering on larger table: use it. Index on both tables, non-clustering on larger table: small table has (much) less records than large table has pages: use index on large table otherwise: use index on small table Johann Gamper (IDSE) Database Management and Tuning Unit 7 11 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 12 / 34

4 Exercise 9 Purchasing Department Exercise 9 Solution Purchasing department maintains table Onorder(supplier,part,quantity,price). The table is heavily used during the opening hours, but not over night. Queries: Q1: add a record, all fields specified (very frequent) Q2: delete a record, supplier and part specified (very frequent) Q3: find total quantity of a given part on order (frequent) Q4: find the total value on order to a given supplier (rare) Problem: Which indexes should be used? Queries: Q1: add a record, all fields specified (very frequent) Q2: delete a record, supplier and part specified (very frequent) Q3: find total quantity of a given part on order (frequent) Q4: find the total value on order to a given supplier (rare) Solution: Clustering composite B + -tree index on (part,supplier). eliminate overflows over night attribute order important to support query Q3 hash index will not work for query Q3 (prefix match query) Discussion: Non-clustering index on supplier to answer query Q4? index must be maintained and will hurt the performance of much more frequent queries Q1 and Q2 index does not help much if there are only few different suppliers Johann Gamper (IDSE) Database Management and Tuning Unit 7 13 / 34 Exercise 10 Point Query Too Slow Johann Gamper (IDSE) Database Management and Tuning Unit 7 14 / 34 Exercise 11 Historical Immigrants Database Employee has a clustering B + -tree index on ssnum. Queries: retrieve employee by social security number (ssnum) update employee with a specific social security number Problem: Throughput is still not enough. Solution: Use hash index instead of B + -tree (faster for point queries). Digitalized database of US immigrants between 1800 and 1900: 17M records each record has approx. 200 fields e.g., last name, first name, city of origin, ship taken, etc. Queries retrieve immigrants: by last name and at least one other attribute second attribute is often first name (most frequent) or year Problem: Efficiently serve 2M descendants of the immigrants... Johann Gamper (IDSE) Database Management and Tuning Unit 7 15 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 16 / 34

5 Exercise 11 Solution Exercise 12 Flight Reservation System Clustering B + -tree index on (lastname,firstname): no overflow since database does not have updates use high fill factor to increase space utilization key compression should be used (long key, no update) index useful also for prefix queries on lastname Composite non-clustering index on (lastname,year): no maintenance cost (no updates) attributes probably selective enough Non-clustering indexes on all frequent attribute combinations? no maintenance, thus only limitation is space overhead useful only if selective enough An airline manages 1000 flights and uses the tables: Flight(flightID, seatid, passanger-name) Totals(flightID, number-of-passangers) Each reservation adds a record to Flight increments Totals.number-of-passangers Queries are separate transactions. Problem: Lock contention on Totals. Solution: Totals is a small table (1000 small records) and fits on few pages. Without index, update scans table and scanned records are locked. Clustering index on flightid avoids table scan and thus lock contention (row locking assumed). Johann Gamper (IDSE) Database Management and Tuning Unit 7 17 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 18 / 34 Outline What is a Transaction? Conclusion A transaction is a unit of program execution that accesses and possibly updates various data items. Example: transfer $50 from account A to account B 1. R(A) 2. A A W (A) 4. R(B) 5. B B W (B) Two main issues: 1. concurrent execution of multiple transactions 2. failures of various kind (e.g., hardware failure, system crash) 1 Slides of section are adapted from the slides Database System Concepts, 6 th Ed., Silberschatz, Korth, and Sudarshan Johann Gamper (IDSE) Database Management and Tuning Unit 7 19 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 20 / 34

6 ACID Properties Atomicity Database system must guarantee ACID for transactions: Atomicity: either all operations of the transaction are executed or none Consistency: execution of a transaction in isolation preserves the consistency of the database Isolation: although multiple transactions may execute concurrently, each transaction must be unaware of the other concurrent transactions. Durability: After a transaction completes successfully, changes to the database persist even in case of system failure. Example: transfer $50 from account A to account B 1. R(A) 2. A A W (A) 4. R(B) 5. B B W (B) What if failure (hardware or software) after step 3? money is lost database is inconsistent Atomicity: either all operations or none updates of partially executed transactions not reflected in database Johann Gamper (IDSE) Database Management and Tuning Unit 7 21 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 22 / 34 Consistency Isolation Motivating Example Example: transfer $50 from account A to account B 1. R(A) 2. A A W (A) 4. R(B) 5. B B W (B) Consistency in example: sum A + B must be unchanged Consistency in general: explicit integrity constraints (e.g., foreign key) implicit integrity constraints (e.g., sum of all account balances of a bank branch must be equal to branch balance) Transaction: must see consistent database during transaction inconsistent state allowed after completion database must be consistent again Example: transfer $50 from account A to account B 1. R(A) 2. A A W (A) 4. R(B) 5. B B W (B) Imagine second transaction T 2 : T 2 : R(A), R(B), print(a + B) T 2 is executed between steps 3 and 4 T 2 sees an inconsistent database and gives wrong result Johann Gamper (IDSE) Database Management and Tuning Unit 7 23 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 24 / 34

7 Isolation Durability Trivial isolation: run transactions serially Isolation for concurrent transactions: For every pair of transactions T i and T j, it appears to T i as if either T j finished execution before T i started or T j started execution after T i finished. Schedule: specifies the chronological order of a sequence of instructions from various transactions equivalent schedules result in identical databases if they start with identical databases Serializable schedule: equivalent to some serial schedule serializable schedule of T 1 and T 2 is either equivalent to T 1, T 2 or T 2, T 1 When a transaction is done it commits. Example: transaction commits too early transaction writes A, then commits A is written to the disk buffer then system crashes value of A is lost Durability: After a transaction has committed, the changes to the database persist even in case of system failure. Commit only after all changes are permanent: either written to log file or directly to database database must recover in case of a crash Johann Gamper (IDSE) Database Management and Tuning Unit 7 25 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 26 / 34 Locks Lock Compatibility A lock is a mechanism to control concurrency on a data item. Two types of locks on a data item A: exclusive xl(a): data item A can be both read and written shared sl(a): data item A can only be read. Lock request are made to concurrency control manager. Transaction is blocked until lock is granted. Unlock A ul(a): release the lock on a data item A Lock compatibility matrix: T 1 T 2 shared exclusive shared true false exclusive false false T 1 holds shared lock on A: shared lock is granted to T 2 exclusive lock is not granted to T 2 T 2 holds exclusive lock on A: shared lock is not granted to T 2 exclusive lock is not granted to T 2 Shared locks can be shared by any number of transactions. Johann Gamper (IDSE) Database Management and Tuning Unit 7 27 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 28 / 34

8 Locking Protocol Pitfalls of Locking Protocols Deadlock Example transaction T 2 with locking: 1. sl(a), R(A), ul(a) 2. sl(b), R(B), ul(b) 3. print(a + B) T 2 uses locking, but is not serializable A and/or B could be updated between steps 1 and 2 printed sum may be wrong Locking protocol: set of rules followed by all transactions while requesting/releasing locks locking protocol restricts the set of possible schedules Example: two concurrent money transfers T 1 : R(A), A A + 10, R(B), B B 10, W (A), W (B) T 2 : R(B), B B + 50, R(A), A A 50, W (A), W (B) possible concurrent scenario with locks: T 1.xL(A), T 1.R(A), T 2.xL(B), T 2.R(B), T 2.xL(A), T 1.xL(B),... T 1 and T 2 block each other no progress possible Deadlock: situation when transactions block each other Handling deadlocks: one of the transactions must be rolled back (i.e., undone) rolled back transaction releases locks Johann Gamper (IDSE) Database Management and Tuning Unit 7 29 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 30 / 34 Pitfalls of Locking Protocols Starvation Two-Phase Locking Starvation: transaction continues to wait for lock Examples: the same transaction is repeatedly rolled back due to deadlocks a transaction continues to wait for an exclusive lock on an item while a sequence of other transactions are granted shared locks Well-designed concurrency manager avoids starvation. Protocol that guarantees serializability. Phase 1: growing phase transaction may obtain locks transaction may not release locks Phase 2: shrinking phase transaction may release locks transaction may not obtain locks Johann Gamper (IDSE) Database Management and Tuning Unit 7 31 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 32 / 34

9 Two-Phase Locking Example Summary Conclusion Example: two concurrent money transfers T 1 : R(A), A A + 10, R(B), B B 10, W (A), W (B) T 2 : R(A), A A 50, R(B), B B + 50, W (A), W (B) Possible two-phase locking schedule: 1. T 1 : xl(a), xl(b), R(A), R(B), W (A A + 10), ul(a) 2. T 2 : xl(a), R(A), xl(b) (wait) 3. T 1 : W (B B 10), ul(b) 4. T 2 : R(B), W (A A 50), W (B B + 50), ul(a), ul(b) Index tuning examples Equivalent serial schedule: T 1, T 2 Johann Gamper (IDSE) Database Management and Tuning Unit 7 33 / 34 Johann Gamper (IDSE) Database Management and Tuning Unit 7 34 / 34

Outline. Database Tuning. Join Strategies Running Example. Outline. Index Tuning. Nikolaus Augsten. Unit 6 WS 2014/2015

Outline. Database Tuning. Join Strategies Running Example. Outline. Index Tuning. Nikolaus Augsten. Unit 6 WS 2014/2015 Outline Database Tuning Nikolaus Augsten University of Salzburg Department of Computer Science Database Group 1 Examples Unit 6 WS 2014/2015 Adapted from Database Tuning by Dennis Shasha and Philippe Bonnet.