Structured Query Language (SQL) Advanced Features

Transcription

1 Explicit Joins Structured Query Language (SQL) Advanced Features Joins are usually specified as part of the WHERE statement (implicit or innerjoins.) Further, LEFT, RIGHT, and FULL OUTER joins can also be specified. Joins can also be specified as part of the FROM clause explicitly (theta join) computed tables Instructor: Sharma Chakravarthy The University of Texas at Arlington Database Management Systems: Sharma Chakravarthy 1 SELECT select-list FROM table_1 {LEFT RIGHT FULL} [OUTER] JOIN table_2 ON join_condition_1 [{LEFT RIGHT FULL} [OUTER] JOIN table_3 ON join_condition_2] CROSS JOIN can also be used for Cartesian product INNER JOIN can also be used Database Management Systems: Sharma Chakravarthy 2 Example Union 26. Find the numbers of the players who are older than R. Parmenter SELECT FROM WHERE A.playerno PLAYERS as A JOIN PLAYERS as P on A.yob < P.yob P.name = 'Parmenter' AND P.initials = 'R' You can write Queries in the way that suits you! The optimizer will optimize it in the best possible way! Syntax SELECT_statement_1 UNION [ALL] SELECT_statement_2 [UNION [ALL] SELECT_statement_3] [ORDER BY order_by_list] Database Management Systems: Sharma Chakravarthy 3 Database Management Systems: Sharma Chakravarthy 4 1

2 Aggregate Functions AVG([ALL DISTINCT] expression) The average of the non-null values in the expression SUM([ALL DISTINCT] expression) The total of the non-null values in the expression MIN([ALL DISTINCT] expression) The lowest non-null values in the expression MAX([ALL DISTINCT] expression) The highest non-null values in the expression COUNT([ALL DISTINCT] expression) The number of non-null values in the expression COUNT(*) //can also use DISTINCT and attr names The number of rows selected by a query Aggregate Functions Aggregate functions (or column functions) perform calculation on the values in a set of selected rows (partition) Expression specified for AVG, SUM and COUNT functions must result in a numeric value. The expression for Min and MAX can result in a numeric, date, or string value By default, all values are included in the calculation regardless of whether they are duplicates. Use DISTINCT keyword to omit duplicates If you use an aggregate function in the SELECT clause, that clause cannot include non-aggregate columns from the base table (except group by attributes) Database Management Systems: Sharma Chakravarthy 5 Database Management Systems: Sharma Chakravarthy 6 Aggregate Functions Can a HAVING clause have aggregate functions? Yes! Can a SELECT clause have aggregate functions? Yes! Can a WHERE clause have aggregate functions? No! Can a WHERE clause have subselects in it? Yes! Can a HAVING clause have subselects in it? Yes! Aggregate Functions PLAYERS (playerno*, name, initials, yob, sex, year_joined, street, houseno, postcode, town, phoneno, leagueno); GAMES (teamno*, playerno*, won, lost); 27. List names of players who have won more games than they have lost (for all the teams they have played) select name from players p where (select sum(won) from games g1 where g1.playerno = p.playerno) > (select sum(lost) from games g2 where g2.playerno = p.playerno); Can a WHERE clause have subselects that have aggregates in them? Yes, need to make sure it is properly used! Database Management Systems: Sharma Chakravarthy 7 NAME Parmenter Baker Collins Moorman Brown Database Management Systems: Sharma Chakravarthy 8 2

3 Aggregate Functions 28. list names of players who have won more games than they have lost when they played for a team select name from players p where (select won from games g1 where g1.playerno = p.playerno) > (select lost from games g2 where g2.playerno = p.playerno); Gives error: single-row subquery returns more than one row! Tables Instances SQL> select * from penalties; PAYMENTNO PLAYERNO PDATE AMOUNT rows selected. Database Management Systems: Sharma Chakravarthy 9 Database Management Systems: Sharma Chakravarthy 10 Aggregate Functions PLAYERS (playerno*, name, initials, yob, sex, year_joined, street, houseno, postcode, town, phoneno, leagueno); PENALTIES (paymentno*, playerno, date, amount); 29. list names of players and their max penalty select playerno, (select max(amount) from penalties pe where pe.playerno=pl.playerno) as maxpmt from players pl order by maxpmt desc; PLAYERNO MAXPMT Database Management Systems: Sharma Chakravarthy Aggregate Functions A HAVING clause can refer to a column included in the select clause whereas a WHERE clause can refer to any columns in the base tables Can use AND and OR in the HAVING clause Can use comparisons using relational arithmetic operators Group by name HAVING sum(bonus) > sum(salary) Can use simple predicates in the HAVING clause, but aggregates cannot be used in the WHERE clause A SELECT can only include columns in the group by clause (+ aggregate functions)! Database Management Systems: Sharma Chakravarthy 12 3

4 Aggregate Functions (SQL Extensions) Several RDBMSs support OLAP (on-line analytical processing) used in data warehouses as opposed to OLTP (online transaction processing) used in traditional DBMSs ROLLUP Summary row is added for each group and the total CUBE Summary row is added for each dimension (or distinct combination of the group) and the total TOP n [PERCENT] output top n tuples or top n% tuples Overview What s data Analysis Overview of Tools for analysis in relational model Problems with GROUP BY and why SQL need to be fixed Need for ROLL UP and CUBE operators Top n [PERCENT] [WITH TIES] if the last row has ties (same values), all are presented Database Management Systems: Sharma Chakravarthy 13 11/8/2017 Sharma Chakravarthy 14 Requirements Data analysis - Summarize data value - Look for unusual pattern of data - Extract statistical information - Contrast one category with another Steps in data analysis - Extracting the aggregated data from the database into a file or table - Visualizing the result in graphical way, display trends, clusters and difference Questions? How do traditional relational database fit into this picture? How can relations (FACT table) represent multidimensional problem? How to use existing tools in SQL and enrich it further 11/8/2017 Sharma Chakravarthy 15 11/8/2017 Sharma Chakravarthy 16 4

5 A typical STAR Schema Time Altitude Fact Latitude Longitude Time Altitude Temp Pres Latitude... Longitude Multi-dimensional data Relational systems model N- dimensional data as a relation with N-attributes Weather Time Latitude Longitude Altitude Temp Pres 27/ 11/ 94: : 58: 33N 122: 45: 28W / 11/ 94: : 16: 18N 27: 05: 55W /8/2017 Sharma Chakravarthy 17 11/8/2017 Sharma Chakravarthy 18 Relational Tools Additional SQL functions SQL aggregate function: COUNT(), SUM(), MIN(), MAX(), and AVG() Example: The average of all measured temperatures is: SELECT AVG( Temp) FROM Weather; The following query counts the distinct number of reporting time SELECT COUNT( DISTINCT Time) FROM Weather 11/8/2017 Sharma Chakravarthy 19 Statistic: median, standard deviation, variance, etc. Physical: center of mass, angular momentum, etc. Financial analysis: volatility, alpha, beta, etc. Some system allow users to add new aggregation function 11/8/2017 Sharma Chakravarthy 20 5

6 Group BY Figure 1: The GROUP BY relational operator partitions a table into groups. Each group is then aggregated by a function. The aggregation function summarizes some column of groups returning a value for each group. Grouping Values Partitioned Table Sum() Aggregate Values 11/8/2017 Sharma Chakravarthy 21 Group-By Used to create a table of many aggregate values indexed by a set of attributes Weather Time Latitude Longitude Altitude Temp Pres 27/ 11/ 94: : 58: 33N 122: 45: 28W / 11/ 94: : 16: 18N 27: 05: 55W / 11/ 94: : 58: 33S 150: 10: 10W / 11/ 94: : 58: 33S 150: 10: 10W SELECT Time, Altitude, AVG( Temp) FROM Weather GROUP BY Time, Altitude; Weather Time Altitude Avg Temp 27/ 11/ 94: / 11/ 94: One may want to group by time, altitude, and both To do this, you have to write 3 or more queries and take union Sharma Chakravarthy 22 An Example SALES Model Year Color Sales Chevy 1990 red 5 Chevy 1990 white 87 Chevy 1990 blue 62 Chevy 1991 red 54 Chevy 1991 white 95 Chevy 1991 blue 49 Chevy 1992 red 31 Chevy 1992 white 54 Chevy 1992 blue 71 Ford 1990 red 64 Ford 1990 white 62 Ford 1990 blue 63 Ford 1991 red 52 Ford 1991 white 9 Ford 1991 blue 55 Ford 1992 red 27 Ford 1992 white 62 Ford 1992 blue 39 Roll-Up and Drill-Down Sales Roll Up by Model by Year by Color Sales by Model Sales by Sales Model Year Color by Year by Color Model by Year by Model Chevy 1994 black 50 white black 85 white Problem -not relational model (null value in the primary key not allowed) -not convenient, the number of columns grows as the power set of the number of aggregated attributes 11/8/2017 Sharma Chakravarthy 24 6

7 Roll-Up and Drill-Down A more convenient representation in relational model Sales Summary Model Year Color Units Chevy 1994 black 50 Chevy 1994 white 40 Chevy 1994 ALL 90 Chevy 1995 black 85 Chevy 1995 white 115 Chevy 1995 ALL 200 Chevy ALL ALL /8/2017 Sharma Chakravarthy 25 SQL Approach SQL statements to build the SalesSummary table from the raw sales data SELECT Model, ALL, ALL, SUM( Sales) FROM Sales WHERE Model = Chevy GROUP BY Model UNION SELECT Model, Year, ALL, SUM( Sales) FROM Sales WHERE Model = Chevy GROUP BY Model, Year UNION SELECT Model, Year, Color, SUM( Sales) FROM Sales WHERE Model = Chevy GROUP BY Model, Year, Color 11/8/2017 Sharma Chakravarthy 26 SQL Approach SQL Approach SELECT ALL, ALL, ALL, SUM( Sales) FROM Sales UNION SELECT Model, ALL, ALL, SUM( Sales) FROM Sales GROUP BY Model UNION SELECT ALL, Year, ALL, SUM( Sales) FROM Sales GROUP BY Year UNION SELECT ALL, ALL, Color, SUM( Sales) FROM Sales GROUP BY Color UNION SELECT Model, Year, ALL, SUM( Sales) FROM Sales GROUP BY Model, Year UNION SELECT Model, ALL, Color, SUM( Sales) FROM Sales GROUP BY Model, Color UNION SELECT ALL, Year, Color, SUM( Sales) FROM Sales GROUP BY Year, Color UNION SELECT Model, Year, Color, SUM( Sales) FROM Sales GROUP BY Model, Year, Color 11/8/2017 Sharma Chakravarthy 27 Notice: Roll up to one direction Still complicated Does not represent all aspects of roll up and drill down To extend from this example, for full roll- up and drill- down in 3D Needs 8 aggregate SQL statements Problem: complicated SQL query for 4D, 16 aggregates are needed 11/8/2017 Sharma Chakravarthy 28 7

8 N-dimensional Cube where Each Attribute is a Dimension N-dimensional Aggregate (sum(), max(),...) fits relational model exactly: a 1, a 2,..., a N, f() Super-aggregate over N-1 Dimensional sub-cubes ALL, a 2,..., a N, f() a 3, ALL, a 3,..., a N, f()... a 1, a 2,..., ALL, f() this is the N-1 Dimensional cross-tab. Super-aggregate over N-2 Dimensional sub-cubes ALL, ALL, a 3,..., a N, f()... a 1, a 2,..., ALL, ALL, f() SALES Model Year Color Sales Chevy 1990 red 5 Chevy 1990 white 87 Chevy 1990 blue 62 Chevy 1991 red 54 Chevy 1991 white 95 Chevy 1991 blue 49 Chevy 1992 red 31 Chevy 1992 white 54 Chevy 1992 blue 71 Ford 1990 red 64 Ford 1990 white 62 Ford 1990 blue 63 Ford 1991 red 52 Ford 1991 white 9 Ford 1991 blue 55 Ford 1992 red 27 Ford 1992 white 62 Ford 1992 blue 39 An Example CUBE DATA CUBE Model Year Color Sales ALL ALL ALL 942 chevy ALL ALL 510 ford ALL ALL 432 ALL 1990 ALL 343 ALL 1991 ALL 314 ALL 1992 ALL 285 ALL ALL red 165 ALL ALL white 273 ALL ALL blue 339 chevy 1990 ALL 154 chevy 1991 ALL 199 chevy 1992 ALL 157 ford 1990 ALL 189 ford 1991 ALL 116 ford 1992 ALL 128 chevy ALL red 91 chevy ALL white 236 chevy ALL blue 183 ford ALL red 144 ford ALL white 133 ford ALL blue 156 ALL 1990 red 69 ALL 1990 white 149 ALL 1990 blue 125 ALL 1991 red 107 ALL 1991 white 104 ALL 1991 blue 104 ALL 1992 red 59 ALL 1992 white 116 ALL 1992 blue 110 The Data CUBE Operator To simplify the computation mentioned above Use the ROLL UP and CUBE operator GROUP BY [ WITH ( CUBE ROLL UP) ] { ( <column name> <expression> ) } 11/8/2017 Sharma Chakravarthy 31 CUBE Operator If the application wants only a roll-up or drilldown report, the full cube is an overkill It is reasonable to offer other additional rollup functions in addition to CUBE. ROLLUP produces just the super-aggregates: (f1, f2,, ALL),. (f1, ALL,, ALL) (ALL, ALL,, ALL) 11/8/2017 Sharma Chakravarthy 32 8

9 Aggregate Functions (SQL Extensions) Several RDBMSs support OLAP (on-line analytical processing) used in data warehouses as opposed to OLTP (online transaction processing) used in traditional DBMSs ROLLUP Summary row is added for each group and the total CUBE Summary row is added for each dimension (or distinct combination of the group) and the total TOP n [PERCENT] output top n tuples or top n% tuples Top n [PERCENT] [WITH TIES] if the last row has ties (same values), all are presented Database Management Systems: Sharma Chakravarthy 33 ROLLUP INVOICES(invid, vid, invnum, invdate, description, total); Select vid, count(*) as invcount, sum(total) as invtotal From invoices Group by ROLLUP (vid); //multiple attributes are ok; // Oracle syntax Vid invcount invtotal Syntax varies from vendor to vendor GROUP BY vid with ROLLUP (for SQL server) You can also have order by Database Management Systems: Sharma Chakravarthy 34 Summary row ROLLUP VENDORS(venid, name, address, venstate, vencity); Select venstate, vencity, count(*) as qtyven, From vendors Group by ROLLUP (venstate, vencity); Order by venstate DESC, vencity DESC Venstate vencity qtyven NJ washington 1 NJ fairfield 1 NJ east brunswick 2 NJ NULL 4 IA washington 1 IA fairfield 1 IA NULL 2 NULL NULL 6 Summary row Summary rows ROLLUP select playerno, sum(amount) from penalties group by rollup (playerno); PLAYERNO SUM(AMOUNT) Summary row Database Management Systems: Sharma Chakravarthy 35 Database Management Systems: Sharma Chakravarthy 36 9

10 CUBE Select venstate, vencity, count(*) as qtyven, From vendors Where venstate in { IA, NJ ) Group by CUBE (venstate, vencity) Order by venstate DESC, vencity DESC Venstate vencity qtyven NJ washington 1 NJ fairfield 1 NJ east brunswick 2 NJ NULL 4 Summary row IA washington 1 IA fairfield 1 IA NULL 2 Summary row NULL washington 2 NULL fairfield 2 NULL east brunswick 2 NULL NULL 6 Summary rows Database Management Systems: Sharma Chakravarthy 37 CUBE PLAYERNO PDATE SUM(AMOUNT) select playerno, pdate, sum(amount) 480 from penalties group by cube (playerno, pdate); Summary row Summary row rows selected. Database Management Systems: Sharma Chakravarthy 38 CUBE Operator Comments on the CUBE Operator it aggregates over all <select list> as in standard group by then it unions in each super aggregate of the global cube substitute ALL for the aggregation column If the cardinality of the N attributes are C 1, C 2, C 3,., C n. The result will be (C i + 1) Previous Example: (1+1)* (2+1) = 6 rows Sales Example: (2+1)*(3+1)*(3+1) = 48 rows (18 in sales + 30 in data cube) The extra value is ALL Summary CUBE operator generalizes relational aggregates Needs ALL value to denote sub-cubes ALL values represent aggregation sets Needs generalization of user-defined aggregates Decorations and abstractions are interesting Computation has interesting optimizations Relationship to rest of SQL not fully worked out. 11/8/2017 Sharma Chakravarthy 39 10

11 Histograms Histograms (Contd.) In Standard SQL, GROUP BY operator does NOT allow a direct construction of Histograms (aggregation over computed categories) Ex: SELECT day, nation, MAX( Temp) FROM Weather GROUP BY Day( time) as day, Country( Latitude, Longitude) as nation; 11/8/2017 Sharma Chakravarthy 41 Some SQL system do support histogram (SQL92) SELECT day, nation, MAX( Temp) FROM (SELECT Day( Time) as day, Country( Latitude, Longitude) as nation, Temp FROM Weather ) AS foo GROUP BY day, nation; Not convenient 11/8/2017 Must construct a table-valued Sharma Chakravarthy expression and 42 then aggregate over resulting table Joins Vs. subselects Advantage of implicit/explicit joins The result of a join operation can include columns from both tables. Only outer table columns can be included when using a subquery A query with an implicit join typically performs faster than the same with a subquery Advantages of subqueries You can pass an aggregate value to the outer query A subquery tends to be more intuitive Long, complex queries can be easier to code using subqueries subqueries A subquery is a SELECT statement that is used within another SQL statement A subquery can return a single value, a result set that contains a single column, or a result set that contains one or more columns A subquery that returns a single value can be used anywhere an expression is allowed. A subquery that returns a single column can be used in place of a list of values A subquery that returns one or more columns can be used in place of a table in the FROM clause Some DBMSs allow multiple attributes to be compared Database Management Systems: Sharma Chakravarthy 43 Database Management Systems: Sharma Chakravarthy 44 11

12 Subqueries Subqueries can be used in 4 ways: In a WHERE clause as part of a condition (as long as it returns a single value) In a HAVING clause as part of a condition (as long as it returns a single value for a partition) In a FROM clause as a table specification //table expression In a SELECT clause as a column specification Database Management Systems: Sharma Chakravarthy 45 Subqueries Select distinct vendorname, (select MAX(invoicedate) form Invoices where Invoices.vendorID = Vendors.vendorID) as LatestInv from Vendors order by LatestInv DESC Alternatively, Select distinct vendorname, MAX(invoicedate) Form Vendors join Invoices on Invoices.vendorID = Vendors.vendorID) as LatestInv Group by VendorName order by LatestInv DESC Database Management Systems: Sharma Chakravarthy 46 Subqueries Alternatively, Select distinct vendorname, MAX(invoicedate) as latestinv Form Vendors, Invoices Where Invoices.vendorID = Vendors.vendorID Group by VendorName order by LatestInv DESC Subselect Subquery in a SELECT clause used for a column specification must return a single value Typically a correlated subquery Can be rewritten using a join instead of the subquery. Join is faster and more readable and hence subqueries are seldom used in a select statement Database Management Systems: Sharma Chakravarthy 47 Database Management Systems: Sharma Chakravarthy 48 12

13 Subselect in a FROM Clause Nested Queries SELECT Invoices.VendorID, MAX(InvoiceDate) FROM Invoices JOIN (SELECT TOP 5 VendorID, AVG(invoiceTotal) as AvgInvoice FROM Invoices GROUP BY VendorID ORDER BY AvgInvoice DESC) AS TopVendor ON Invoices.VendorID=TopVendor.VendorID Group By Invoices.VendorID Order BY LatestInv DESC Classification single-shot evaluation of sub/nested queries Example : Select employees whose salary is the same as the average salary Select From Where Name Employee Salary = (Select AVG (salary) From Employee) The subquery returns a single value. Hence the predicate can be modified as Salary = K where k is the result of the sub query Database Management Systems: Sharma Chakravarthy 49 Database Management Systems: Sharma Chakravarthy 50 Nested Queries (Contd.) Nested Queries (Contd.) Sub/nested query can return a set Example: select employees whose department is located in Denver Select Name From Employee Where Dept_number IN (Select Dept_Number From Dept Where location= Denver ) Modify the predicate appropriately at run time (as an IN clause which has an explicit set as part of the query Database Management Systems: Sharma Chakravarthy 51 The previous query can be re-written without nesting Select Name From Employee e, Dept d Where e.dept_number = d.dept_number and d.location= Denver The optimizer may not do (or most likely not able to do) this transformation Database Management Systems: Sharma Chakravarthy 52 13

14 Nested Queries (Contd.) Nested Queries (Contd.) Select names of Employees who earn more than their managers This query cannot be written like earlier queries with a subquery that can be evaluated once (or is independent) In principle, a correlation query must be re-evaluated for each candidate tuple from the referenced query block Correlated sub query a sub query that contains reference to a value obtained from a candidate tuple of a higher level query block Example Select Name From Employee e where Salary > (Select Salary From Employee where Employee.Num=e.Manager) The subquery has to be evaluated for each tuple of the outer query (nested loop query) Database Management Systems: Sharma Chakravarthy 53 Database Management Systems: Sharma Chakravarthy 54 Nested Queries (Contd.) Nested Queries (Contd.) Correlated sub query a sub query that contains reference to a value obtained from a candidate tuple of a higher level query block Example Select Name From Employee Where Dept_number IN (Select Dept_Number From Dept Where location= Denver ) The above query is not correlated The previous query (employee s salary greater than his/her manager) can be written without using subqueries (as a non-correlated query) Example Select From where Name Employee e, Employee m e.salary > m.salary AND e.num=m.manager Database Management Systems: Sharma Chakravarthy 55 Database Management Systems: Sharma Chakravarthy 56 14

15 Multi level correlated subqueries Select names of employee s who earn more than their manager s manager. L1 select name from Employee e where salary > L2 (select salary from Employee where Employee.ssn = L3 (select mgrssn From Employee where Employee.ssn = e.mgrssn)) Correlated subqueries Some correlated subqueries CANNOT be expressed in a simpler manner (as non-correlated subqueries) Example: Division queries Since L3 query references a L1 value, L3 need to be evaluated for each L1 tuple. Database Management Systems: Sharma Chakravarthy 57 Database Management Systems: Sharma Chakravarthy 58 Nested Queries Nested block is optimized independently, with the outer tuple considered as providing a selection condition. Outer block is optimized with the cost of `calling nested block computation taken into account. Implicit ordering of these blocks means that some good strategies are not considered. The non-nested version of the query is typically optimized better. SELECT S.sname FROM Sailors S WHERE EXISTS (SELECT * FROM Reserves R WHERE R.bid=103 AND R.sid=S.sid) Nested block to optimize: SELECT * FROM Reserves R WHERE R.bid=103 AND S.sid= outer value Equivalent non-nested query: SELECT S.sname FROM Sailors S, Reserves R WHERE S.sid=R.sid Database Management Systems: Sharma Chakravarthy 59 DIVISION SUPPLIER (s#, sname, status, city) PARTS (p#, pname, color, weight, city) SP (s#, p#, qty) Get supplier names for suppliers who supply ALL parts (SP[s#, p#] PARTS[p#]) join SUPPLIER on s# There is NO division operator or (logical) for all quantifier in SQL θ [any all] is not adequate to express division! How do we translate the above query into SQL? AND R.bid=103 Database Management Systems: Sharma Chakravarthy 60 15

16 DIVISION SUPPLIER (s#, sname, status, city) PARTS (p#, pname, color, weight, city) SP (s#, p#, qty) DIVISION SUPPLIER (s#, sname, status, city) PARTS (p#, pname, color, weight, city) SP (s#, p#, qty) It is based on the equivalence of the logical expression x ( y (P(x, y)) ( x ( y (P(x, y))) The following 2 statements are equivalent: 1. For all x (parts) there exists a y (supplier) such that P(x, y) is true (y supplies all x (parts)) 2. It is not the case (false) that there exists a x (part) for which no y (supplier) exists such that P(x, y) is true (y supplies part x) Database Management Systems: Sharma Chakravarthy 61 Get supplier names for suppliers who supply ALL parts parts ( supplier (supplies(supplier, parts)) ( part ( supplier (supplies(supplier, parts))) The following 2 statements are equivalent: 1. For all parts p there exists a supplier s such that supplies(s, p) is true 2. It is not the case (false) that there exists a part for which no supplier exists such that suplies(s, p) is true Database Management Systems: Sharma Chakravarthy 62 DIVISION SUPPLIER (s#, sname, status, city) PARTS (p#, pname, color, weight, city) SP (s#, p#, qty) SELECT sname FROM supplier s /*needed for output */ WHERE NOT EXISTS ( SELECT parts FROM parts p /*for all denominator*/ minus ( SELECT parts FROM SP sp /* there exists numerator*/ WHERE sp.s# = s.s# ) ) Database Management Systems: Sharma Chakravarthy 63 DIVISION SUPPLIER (s#, sname, status, city) PARTS (p#, pname, color, weight, city) SP (s#, p#, qty) SELECT DISTINCT sname //is distinct needed? FROM supplier s //needed for output WHERE NOT EXISTS (SELECT * FROM parts p //for all denominator WHERE NOT EXISTS (SELECT * FROM SP sp //there exists numerator WHERE sp.s# = s.s# AND sp.p# = p.p#) ) Database Management Systems: Sharma Chakravarthy 64 16

17 DIVISION SUPPLIER (s#, sname, status, city) PARTS (p#, pname, color, weight, city) SP (s#, p#, qty) Get suppliers who supply ALL red parts Other operators SELECT FROM WHERE DISTINCT sname supplier s NOT EXISTS (SELECT * FROM parts p //all variable comes here WHERE p.color = red AND NOT EXISTS (SELECT * FROM SP sp //there exists var WHERE sp.s# = s.s# AND sp.p# = p.p#) INTERSECT EXCEPT MINUS Creation of views Views are not stored Tables are stored Database ) Management Systems: Sharma Chakravarthy 65 Database Management Systems: Sharma Chakravarthy 66 Other Features Grant and revoke Create views Rollback and commit The catalog is maintained as a table Operations on meta data is done using the same language! Limitations of SQL SQL is not like other programming languages SQL is non-procedural and is not imperative You do not write algorithms, but express you intent of what you want to retrieve The System (or the DBMS query optimizer) actually generates the algorithm and code for each SQL query submitted. This is not easy or straight forward SQL is not Turing complete. In other words, it cannot be used in lieu of a programming language Cannot compute median or other customized computations Database Management Systems: Sharma Chakravarthy 67 Database Management Systems: Sharma Chakravarthy 68 17

18 Embedded SQL As the name implies, SQL is embedded into a programming language Provides features that are not supported in SQL directly (if-then-else, looping, ) Can write canned packages to accept parameters from users Combines the capability of SQL with that of a programming language Arbitrary computations can be performed on data retrieved and stored in tables Embedded SQL Main(){. exec sql begin declare section; declare variables exec sql end declare section; exec sql include sqlca; exec sql select cname, discnt into :cust_name, :cust_discnt from customers where cid= :cusr_id } A single row is extracted in the above! What if it is a set? Database Management Systems: Sharma Chakravarthy 69 Database Management Systems: Sharma Chakravarthy 70 Embedded SQL Cursors Declare a cursor with a name Open a cursor Fetch tuples one at a time into program variables Close the cursor Cursors can be used for read or update; need to be declared; Scrollable cursors are used to move back and forth with in a scroll window of tuples Compiled by the pre-processor Compiled and linked by the language processor Executed! Median and mode can be computed! Transitive closure can be computed! Database Management Systems: Sharma Chakravarthy 71 Dynamic SQL So far, we discussed static SQL. That is program variables held values but NOT SQL statements! In other words, the embedded SQL statements cannot change at run time! SQL statements cannot be generated or parameterized at run time based on user input! The above is needed for writing interfaces to accept inputs and generate appropriate SQL statements based on the input! Dynamic SQL solves this problem! You need to use this for phase 5 of the project Database Management Systems: Sharma Chakravarthy 72 18

19 Dynamic SQL Dynamic SQL (Contd.) Execute immediate Exec sql execute immediate :host_variable host_variable contains an sql statement as a string Example: char sqltext[] = delete from players where playerno=24 ; exec sql execute immediate :sqltext; Execute immediate The statement is compiled once The statement cannot change If you want to change it, needs recompilation of the code. How do you overcome the above? Provide a mechanism to compile dynamically Provide a mechanism to execute dynamically Database Management Systems: Sharma Chakravarthy 73 Database Management Systems: Sharma Chakravarthy 74 Prepare, execute, and using prepare Exec sql prepare delete_playerno from :host_variable Exec sql execute delete_playerno using :var1, :var2,, :vark; Example: char sqltext[] = delete from players where playerno= :pno ; exec sql prepare delete_playerno from :sqltext; Parses and generates compiled form which is parameterized exec sql execute delete_playerno using :pno; Executes the compiled form using the variable Prepare and execure In the previous example, there were NO values returned If the sql stmt has a select, there is a problem we cannot include arbitrary number of attributes to be returned In order to overcome the above problem, a descriptor area has been introduced Describe statement needs to be used Cursor is used to extract values iteratively Database Management Systems: Sharma Chakravarthy 75 Database Management Systems: Sharma Chakravarthy 76 19

20 PL/SQL, TransactSQL In addition to embedding sql in various programming languages, each vendor has a language in which programming language features have been included PL/SQL is Oracle s procedural extension of SQL TransactSQL is Sybase s procedural extension of sql. Seamless integration of sql and other language constructs Can compile and store PL/SQL code and can be invoked Procedures, Triggers and user-defined functions A stored procedure is one or more sql statements that have been compiled and stored with the database Stored procedures are compiled and optimized the first time they are executed. In contrast, SQL statements that are sent from the client to the server have to be compiled and optimized every time they are executed In addition to select statements, it can contain other sql statements. It can also contain control-flow language for conditional processing A trigger is a special type of procedure executed automatically when a table is modified. Triggers are typically used to check the validity of data in a row that is being updated. A user-defined-function (UDF) is a special type of procedure that can return a value or a table Database Management Systems: Sharma Chakravarthy 77 Database Management Systems: Sharma Chakravarthy 78 Procedures, Triggers and user-defined functions The three types of procedural programs stored procedure, triggers and user-define-functions are executable database objects. These are stored with in the database. Stored procedures are frequently written by SQL programmers for use by end users or application programmers. You can provide access to the database exclusively thru stored procedures. Both user-defined-function (UDF) and Triggers are used more often by sql programmers than application programmers or end users. UDF is a special type of procedure that can return a value or a table They differ in whether or not they can use parameters. Both stored procedures and user-defined-functions can use parameters, but triggers cannot. Triggers Create trigger trigger_name On {table_name view_name} {for after instead of} [insert] [,] [update] [,] [delete] As sql_statements With in a trigger, you can refer to two tables: inserted or deleted Syntax may be slightly different for each vendor; need to look it up and use it correctly! Database Management Systems: Sharma Chakravarthy 79 Database Management Systems: Sharma Chakravarthy 80 20

21 SQL Summary Although not Turing-complete, SQL is a powerful non-procedural language for relational data management SQL has been optimized extensively over 30 years SQL as a standard evolves every 4 to 5 years and a number of additional features are added to accommodate growing needs Triggers, abstract data types, object-oriented features have been added in addition to OLAP operators (cube, rollup etc.), XML, temporal features etc. etc. It will continue to evolve! Now NoSQL (not only SQL) is also becoming mainstream!! Database Management Systems: Sharma Chakravarthy 81 SQL Summary (Contd.) An English query can be translated into more than one SQL equivalent. Some may be easier to understand (by humans). Some are better optimized by the underlying system Mining and warehousing has brought a number of limitations of relational query optimizers They were developed assuming not more than 8 to 10 joins Today, SQL queries with 50+ joins are common when data mining is done on a database Tuners or wizards are being developed to simplify (or replace) the job of an administrator Database Management Systems: Sharma Chakravarthy 82 Thank You! 21