Chapter 5 Relational Algebra and SQL

Transcription

1 Chapter 5 Relational Algebra and SQL Now that we have some rough idea as how to create and set up a database based on a project specification, via an E/R chart, we will learn how to get information out of such a data base, which is what we do most of the time. A database query language is a special-purpose programming language that is designed and used to retrieve and update information stored in databases. The structured query language(sql) is the one that we use most of the time. A very important feature for SQL statements is that it only states what it does, but not how to do it, which is left for DBMS to figure out. 1

2 RA and SQL SQL is based on a mathematical body of knowledge, Relational algebra (RA), which serves as an intermediate language for the DBMS. Did we talk about this before? When a declarative SQL statement is parsed by a DBMS, it will be translated into an RA expression. Such an expression is then analyzed and optimized by a query optimizer to become an equivalent but more efficient algorithm, or, a query execution plan. Such a plan is then converted to a piece of executable code. The mathematical nature of the relational algebra makes such analysis and optimization, and proof of equivalence possible. Thus, RA is a key to understand the internal working of SQL statements, and is also helpful to design SQL statements. 2

3 What is relational algebra? A relational algebraic expression consists of a combination of some eight, or nine, simple operators. There are two groups of operators, some basic ones such as Restrict, Project, Union, Difference, Intersection, and Cartesian product; together with several derived ones: Join, and Division. Sometimes, renaming also plays a role... We use a combination of these eight or nine operators to come up with a data access program. 3

4 Eight operators illustrated... 4

5 ...and in words Union returns a relation containing all tuples that appear in either, or both, of the two specified relations. Intersect returns a relation containing all tuples that appear in both of the two specified relations. Product returns a relation containing all possible tuples that are a combination of two tuples, one from each of the two specified relations. Difference returns a relation containing all tuples that appear in the first but not the second of the two relations. We also mentioned, during the review, that the first three are communicative (?), but the last one is not. 5

6 Restrict returns a relation containing all tuples (rows) from a specified relation that satisfy a specific condition. Project returns a relation containing all (sub) tuples that remain in a specified relation after specified attributes (columns) have been removed. Join returns a relation containing all possible tuples that are a combination of two tuples, one from each of the two specific relations, such that 1) the two tuples contributing to any given combination have a common value for the common attributes of the two relations; 2) a common value appears only once, not twice, in the resulting tuple. Division takes two unary relations and one binary one as its inputs. As the output, it sends back a relation containing all the tuples from the first unary relation, each of which is matched with all the tuples in the second unary relation, as shown in the binary one. 6

7 Restriction(Select) We apply this operation to select a subset of tuples satisfying certain Boolean conditions. For example, if we want to get a list of computer science professors, we use a select operation to get it from the Professor table as follows: σ DeptId= CS (Professor) i.e., Select all tuples from the Professor relation that satisfy the condition that DeptId= CS. The general syntax is the following: σ selection condition (relation expression) 7

8 What could a condition be? A condition can be a simple one, such as attribute constant, e.g., DeptId= CS ; or attribute attribute, e.g., Teaching.ProfId = Professor.Id. It could also be a logic expression, i.e., an expression formed with logical operators, such as And, Or, and Not. 8

9 How do we get the nastiness? σ selection condition (relation expression) is just a string. But, when applied to a concrete database, it has a value as its meaning. Assume that it is applied to a relation instance r of type R, we define the values of such an expression σ selection condition (r) to be the collection of tuples in r that satisfies the selection condition. The important thing is that, when applied to a relation, this operator will result in another relation. Question: So what? This provides the basis for nested(nasty) queries, as a query can be put anywhere a table fits. 9

10 An example Given the following Person table, Id Name Address Hobby 1123 John 123 Main St. Stamps 1123 John 123 Main St. Coin 5556 Mary 7 Lake Dr. Hike 9876 Bart 5 Pine St. Stamps with the expression σ Hobby= Stamps (Person), we will get the following table back. Id Name Address Hobby 1123 John 123 Main St. Stamps 9876 Bart 5 Pine St. Stamps 10

11 Another example Given a more complicated condition σ StudId!= And (Semester= S2017 Or Grade= B ) (Transcript) for each and every tuple in the table, it will check if it satisfies the requirement, and throw that into the result bucket if it does. The condition part could be further extended, e.g., EmpSalary > (M ngrsalary 2) And (DeptId + CrsN umber) Like CrsCode where + is for string concatenation, and Like is for pattern matching. 11

12 What do we usually do? During the Registration period, we want to find all the courses taught by CS professors. How should we do it? We always start with the input and walk towards the ouput. It seems that two tables, Teacing and Professor are mentioned, and the input seems to be CS. Notice that those two tables share the professor id information, we can find the output from these two tables such that the tuples share the same professor id, and the professor is affiliated with Computer Science. Technically, we can have the following RA expression. σ Professor.DeptId= CS And ProfId=Professor.Id (Teaching Professor) 12

13 Given the following data of the Professor table, Id Name DeptId Jacob MG 2222 John CS 3333 David EE 4444 Mary CS and that of the Teaching table: ProfId CrsCode Semester MGT123 F CS305 S CS315 F EE101 F CS305 F How will it get the result? 13

14 The three yards A Cartesian product will be formed, which contains a (h)uge table of twenty rows and six columns. 2. Only those ten (?)rows, where DeptId keeps the CS value will be kept. 3. Finally, it keeps three (?) rows where CS professors and taught CS305 and CS All these three rows contain six attributes, and we will use projection to get out the course code for these courses, as we will see later. Question: What do you mean? Let s check out the details... 14

15 Mix them up by doing a Cartesian product Id# NAME DeptId ProfId CrsCode Semester 1111 Jacob MG 1111 MGT123 F Jacob MG 2222 CS305 S Jacob MG 2222 CS315 F Jacob MG 3333 EE101 F Jacob MG 4444 CS305 F John CS 1111 MGT123 F John CS 2222 CS305 S John CS 2222 CS315 F John CS 3333 EE101 F John CS 4444 CS305 F David EE 1111 MGT123 F David EE 2222 CS305 S David EE 2222 CS315 F David EE 3333 EE101 F David EE 4444 CS305 F Mary CS 1111 MGT123 F Mary CS 2222 CS305 S Mary CS 2222 CS315 F Mary CS 3333 EE101 F Mary CS 4444 CS305 F1995 Question: The first row is related, but not what we want; and the second is not related. 15

16 ... on the condition: Restriction... a. Only CS professors... Id# NAME DeptId ProfId CrsCode Semester 2222 John CS 1111 MGT123 F John CS 2222 CS305 S John CS 2222 CS315 F John CS 3333 EE101 F John CS 4444 CS305 F Mary CS 1111 MGT123 F Mary CS 2222 CS305 S Mary CS 2222 CS315 F Mary CS 3333 EE101 F Mary CS 4444 CS305 F1995 b.... and tuples have to be related, with matching Ids. Id# NAME DeptId ProfId CrsCode Semester 2222 John CS 2222 CS305 S John CS 2222 CS315 F Mary CS 4444 CS305 F1995 Question: Is this what we want? Answer: No. How could we focus on, e.g, CS305, and what is it? 16

17 Is there a better way? The above solution works, but it is bulky. We always want to have a smaller intermediate table to reduce the space, as well as time, to get something done. A better way might be the following: σ T.ProfId=Professor.Id ((σ Professor.DeptId= CS Professor) Teaching) Procedurally, we start with the Professor table to get all the professors who work in the CS department, then walk over to the Teaching table to get those tuples such that its ProfId match with those that we just found. Question: Why is this one better? Notice that we have to get the stuff from two tables. Thus, we are really Joining tuples from two related tables that agree on the ProfId attribute, shared among the two tables. 17

18 Focus through projection This is used to choose attributes. Let A denote an attribute of a relation, R, and let t be a tuple in r, and instance of R, then t.a denote part of t consisting of the column under A only. For example, if t is a tuple of the Professor table, then t.id refers to the Id value of this tuple. It is also clear what it means by t.[a 1,, A n ]. In general, we have the following notation: π attribute list (relation) When applied to a relation r with type R, where A 1,, A n are all attributes of R, then π A1,,A n (r), the projection of r on the list, returns the collection of tuples t.[a 1,, A n ], where t is a tuple of R. 18

19 An example Given the following Person table, Id Name Address Hobby 1123 John 123 Main St. Stamps 1123 John 123 Main St. Coin 5556 Mary 7 Lake Dr. Hike 9876 Bart 5 Pine St. Stamps with the expression π Name,Hobby (Person), we will get the following table back Name Hobby John Stamps John Coin Mary Hike Bart Stamps 19

20 Embedded (nested) expression The gist of DB programming is that RA operations can be combined. For example, given the following table Id Name Address Hobby 1123 John 123 Main St. Stamps 1123 John 123 Main St. Coin 5556 Mary 7 Lake Dr. Hike 9876 Bart 5 Pine St. Stamps and π Id,Name (σ Hobby= Stamps Or Hobby= Coins (Person)), we will get the following table back Id Name 1123 John 9876 Bart Notice the order of operation is inside-out, just like in an arithmetic expression. 20

21 Related to SQL Given π Id,Name (σ Hobby= Stamps Or Hobby= Coins (Person)), we immediately have the following SQL query: Select Distinct Id, Name From Person Where Hobby= Stamps or Hobby= Coins ; Question: What will the following get? Select Id, Name From Person Where Hobby= Stamps or Hobby= Coins ; Id Name John 1123 John 9876 Bart

22 déjà vu Question: What are all the courses taught by CS professors? The following RA expression finds out the course numbers of the courses taught by CS professors. π CrsCode (σ Professor.DeptId= CS And ProfId=Professor.Id (Teaching Professor)) Question: What are those courses, namely, names? Since course names are in Course tables, we get the stuff out of Teaching Professor Course, where the tuples agree on both Professor Id and Course Id. π CrsName (σ Course.CrsCode=Teaching.CrsCode (π CrsCode (σ Professor.DeptId= CS And ProfId=Professor.Id (Teaching Professor))) Course) 22

23 How would it work out? b. Courses that a CS person teaches (Cf. page16). Id# NAME DeptId ProfId CrsCode Semester 2222 John CS 2222 CS305 S John CS 2222 CS315 F Mary CS 4444 CS305 F1995 c. Get the course code: CrsCode CS305 CS315 d. Join with Course with matching CrsCode: CrsCode DeptId CrsName Descr CS305 CS Database On the road to high-paying job CS315 CS Trans. Proc. Recover from your worst crashes e. Get the names by making a projection on the course names. CrsName Database Trans. Proc. 23

24 Related to SQL Given the following RA query π CrsName (σ Course.CrsCode=Teaching.CrsCode And Professor.DeptId= CS And ProfId=Professor.Id (Teaching Professor Course)) We immediately (?) get the following mysql> Select Distinct CrsName -> From Course C, Teaching T, Professor P -> Where P.DeptId="CS" And T.CrsCode=C.CrsCode -> And T.ProfId=P.Id; CrsName Database Systems. Transaction Processi rows in set (0.00 sec) 24

25 Is there a better way? A better way might be the following: π CrsName (σ T.CrsCode=C.CrsCode π CrsCode (σ T.ProfId=Professor.Id (σ Professor.DeptId= CS Professor) Teaching) Course) Procedurally, we start with the Professor table to get all the Ids of those professors who work in the CS department, walk over to the Teaching table to get the Course Ids of those courses taught by CS professors. Finally, we walk over to the Course tables with those course Ids to get the names of those courses. Question: Why is this one better? Answer: All the intermediate tables will contain minimum information that we need to continue. Assignment: Use the current instance (Unit 4, Page23-24) to verify the answer. 25

26 Set operations Since relations (tables) are sets, the set operations are pretty straightforward. You must have played with them in either Finite Math., or Discrete Math.. Given two sets R and S, their union, intersection, and difference are represented as R S, R S, and R S, respectively. Notice that although the first two are symmetric, the difference is not, i.e., R S could be different from S R. Given two relations r and s, we immediately obtain r s,r s,r s as the collection of tuples that are in either r or s; in both r and s; and in r but not in s. Thus, the results are all sets, as well. 26

27 Union compatible To be meaningful, when we apply set operators, both relations must have the same attributes, i.e., union compatible. π CrsCode,Semester (σ Grade= C (Transcript)) π CrsCode,Semester (σ CrsCode= MAT123 (Transcript)) What are the courses, except MAT123, offered at any time, when at least one student got a C? π CrsCode,Semester (σ Grade= C (Transcript)) π CrsCode,Semester (σ CrsCode= MAT123 (Transcript)) When did we offer MAT123 in the past, and what are the other courses, offered at any time, when at least one student got a C? π CrsCode,Semester (σ Grade= C (Transcript)) π CrsCode,Semester (σ CrsCode= MAT123 (Transcript)) When we offered MAT123, where at least one student got a C? 27

28 What does it do? We can check them out with the following data: 28

29 Assignment: Evaluate the above queries. 29

30 Related to SQL If we want to find out, for any student, the courses and semesters when she took MAT123 or something else for which she got a C, we can do the following: The RA expression is just the following: π CrsCode,Semester (σ Grade= C (Transcript)) π CrsCode,Semester (σ CrsCode= MAT123 (Transcript)) And the MySQL query naturally follows: mysql> (Select CrsCode, Semester From Transcript Where Grade= C ) Union -> (Select CrsCode, Semester From Teaching Where CrsCode= MAT123 ); We will get the following: CrsCode Semester MAT123 S1996 CS305 F1995 MAT123 F On the other hand, neither intersection nor complement is supported by Version 5.7 of MySQL. 30

31 Cartesian product Given two relations r and s, r s, where r and s share no common attributes names consists of the set of all tuples (a, b), a r and b s. For example, Let r and s be the following, S# S1 S2 P# P1 P2 Then, the result of r s is the following, S# P# S1 P1 S1 P2 S2 P1 S2 P2 31

32 What happens... when r and s do share common attribute names? For example, T 1 (A, B) T 2 (B, C). If we do nothing, by the very definition, we will end up with a table T 3 (A, B, B, C), where the two B s have the same name, but potentially different values. This is not allowed by the relational data model. (Still remember data atomicity?) What we will do is thus to rename such attributes. This ninth operator, not a basic one, can take the following syntax: expression[a 1,, A n ], where A 1,, A n are the new names of the original relational expression, for the corresponding positions. Let s check out an example: 32

33 Mix up the profs and students... (π Id,Name (Student) π Id,DeptId (Professor)) [Student.Id, N ame, P rof essor.id, DeptId]) 33

34 Join Given two relation schemas, R and S, their join, R join condition S is a useful but complicated operation. Let A 1,, A n and B 1,, B n be two subsets of attributes of R and S, respectively, and 1,, n be the standard comparators such as =, <, etc., then a general join is R S, with the following restriction: (R.A 1 1 S.B 1 ) And And (R.A n n S.B n ). A join is thus not a basic operation, but one derived with product, restriction, and projection. 34

35 Natural join When all the operations used in a join are =, we call this special case a natural join. For example, considering two tables, Dept and Emp DEPT# DNAME BUDGET D1 Marketing 10M D2 Development 12M D3 Research 5M EMP# ENAME DEPT# SALARY E1 Lopez D1 40K E2 Cheng D1 42K E3 Finzl D2 30K E4 Salto D2 35K 35

36 Their natural join over DEPT#, a commonly shared attribute, is the following: DEPT# DNAME BUDGET EMP# ENAME SALARY D1 Marketing 10M E1 Lopez 40K D1 Marketing 10M E2 Cheng 42K D2 Development 12M E3 Finzl 30K D2 Development 12M E4 Salto 35K We notice the following two things about this table: 1. Those two tables are related through a commonly shared column, i.e., DEPT#. 2. When joining them, every row in the first table will be concatenated with another from the second row, as long as they are related, i.e., sharing the same DEPT# value. For example, since no row in the first table has a DEPT# value of D3, then no such row is contained in the joined table. 36

37 More specifically A Cartesian product of the two tables will be formed: D# DNAME BUDGET EMP# ENAME D# SALARY D1 Market 10M E1 Lopez D1 40K D1 Market 10M E2 Cheng D1 42K D1 Market 10M E3 Finzl D2 30K D1 Market 10M E4 Salto D2 35K D2 Develop 12M E1 Lopez D1 40K D2 Develop 12M E2 Cheng D1 42K D2 Develop 12M E3 Finzl D2 30K D2 Develop 12M E4 Salto D2 35K D3 Research 5M E1 Lopes D2 40K D3 Research 5M E2 Cheng D2 42K D3 Research 5M E3 Finzl D2 30K D3 Research 5M E4 Salto D2 35K 2. All rows that have different D# values will be deleted, since they are not related. Thus, D# DNAME BUDGET EMP# ENAME D# SALARY D1 Market 10M E1 Lopez D1 40K D1 Market 10M E2 Cheng D1 42K D2 Develop 12M E3 Finzl D2 30K D2 Develop 12M E4 Salto D2 35K 37

38 3. Finally, the duplicated D# column is deleted, since it would be redundant. Technically, we make a projection of the resulting table on all by one redundant attribute, D# in this case. D# DNAME BUDGET EMP# ENAME SALARY D1 Market 10M E1 Lopez 40K D1 Market 10M E2 Cheng 42K D2 Develop 12M E3 Finzl 30K D2 Develop 12M E4 Salto 35K After the normalization process, an RDB almost always consists of a bunch of simple and small tables. On the other hand, a general query needs information from several tables, when the join operation is applied to collect information from related tables. Let s look at a few examples... 38

39 An query example Question: Who taught a course in the fall semester of 1995? We want the names of the professors, not all of them, but the ones who taught in F1995. The general plan is always how to get the output, based on the input. One way to proceed is to start with the input, F1995, and find out where that input sits, Teaching (taught). This can be obtained via a restriction. σ Semester= F1995 (Teaching) With the current instance of the Teaching table, we get the following: ProfId CrsCode Semester CS305 F EE101 F

40 What do we really want? The above information does show the professors who taught in F1995, but only their numbers, not their names. Question: Where is the beef? Look at the other tables, we find out that their names can be found in the Professor table, related to the Teaching table via the professor s Ids. We can thus get a) the ids of those professors, then b) their names via a join. It is easy to get the Ids via a projection. π ProfId (σ Semester= F1995 (Teaching)) ProfId

41 Go over with join With the Id s in hand, we can connect the two tables via a join, as follows: Professor P.Id=T.ProfId (σ Semester= F1995 (Teaching)), Since join only keeps those rows sharing the same Id values, we get the following: Id Name DeptId Mary Doe CS David Jones EE 41

42 The final kick Since we only want the names, we have to do another projection on the Name attribute. π Name (Professor Id=ProfId (σ Semester= F1995 (Teaching))), This gets us the following: Name Mary Doe David Jones Question: What have we done? Answer: The restriction applied on Teaching finds out all the information about who taught what in F1995, including the professor Id. To get their names, we have to match up the selected tuples with the tuples in the Professor table with a natural join on their Id. Finally, since we only want the names, we make a projection on the Name. 42

43 Related to SQL Given the RA expression π Name (Professor Id=ProfId σ Semester= F1995 (Teaching)), we immediately have the following SQL query: mysql> select P.Name -> from Teaching T, Professor P -> Where T.Semester= F1995 And P.Id=T.ProfId; Notice that we use T, P as shortcuts for Teaching and Professor, and we also use a condition, P.Id=T.ProfId, to explicitly enforce the join. We will get the following result for this query: Name David Jones Mary Doe rows in set (0.63 sec) 43

44 Another query example Query: Who taught what in the fall semester of π CrsName,Name ((Professor P.Id=T.ProfId (σ Semester= F1995 (Teaching))) C.CrsCode=T.CrsCode Course) Question: What is going on? Answer: The restriction finds all the information from the Teaching table about who taught what in F1995, including the professor Id and course Id. To get the names of the professors and those of the courses, we have to match up the selected tuples with the tuples in the Professor table with a natural join. We similarly find out the names of those courses. Finally, since we only want to get the names, we make a projection on the respective names. 44

45 Related to SQL Given π CrsName,Name ((Professor P.Id=T.ProfId (σ Semester= F1995 (Teaching))) C.CrsCode=T.CrsCode Course) we immediately have the following SQL query: mysql> select P.Name, C.CrsName -> from Teaching T, Professor P, Course C -> Where T.Semester= F1995 And P.Id=T.ProfId -> And T.CrsCode=C.CrsCode; Notice again that we use additional conditions to explicitly enforce the two join operations. We will get the following result for this query: CrsName Name Database Systems. Mary Doe Electronic Circuits David Jones

46 Join in MySQL MySQL actually implements a Join in the form of A Join B (join condition) Thus, we can have the following alternative SQL expression, which gets us the same answer. mysql> Select distinct Name, CrsName -> From Professor join Teaching -> on (Professor.Id=Teaching.ProfId) -> join Course -> on (Teaching.CrsCode=Course.Crscode) -> Where Teaching.Semester= F1995 ; Name CrsName Mary Doe Database Systems. David Jones Electronic Circuits

47 Why natural join? Such equality based join is common, and natural, since it reflects a good design principle: the same stuff should be related, and nothing else. For example, different pieces of the same course should be related; while those information of different courses have nothing to do with each other. Such an attribute is often given the same name in different tables, collecting different information. The condition in a natural join actually equates all the related attributes in the relations being joined. Moreover, as we already discussed, since these attributes really mean the same thing, it keeps only one of them, while an equijoin keeps both. A natural join is defined as follows: π attribute (σ equation of the shared attributes (R S)) 47

48 Who taught whom? Those who took the same course as taught by the professor in the same semester will be so paired off. Below is the RA expression: π StudId,ProfId (Transcript C Teaching) We have the following with MySQL, where condition C is made explicit: mysql> Select T.StudId, H.ProfId -> From Transcript T, Teaching H -> Where T.CrsCode=H.CrsCode and T.Semester=H.Semester; StudId ProfId Question: Who are those people? 48

49 Let s find them out... Find out where their names sit, then join those tables. mysql> Select P.Name, S.Name -> From Transcript T, Teaching H, Professor P, Student S -> Where T.CrsCode=H.CrsCode and T.Semester=H.Semester -> and P.Id=H.ProfId and T.StudId=S.Id; Below is the answer: Name Name Ann White Jane Doe Adrian Jones Jane Doe David Jones Jesoph Public Ann White Jesoph Public Jacob Taylor Jesoph Public Mary Doe Bart Simpson Jacob Taylor Bart Simpson John Smyth Homer Simpson Mary Doe Joe Blow John Smyth Joe Blow

50 Who took at least two courses? There are several ways of doing this. We will start with the following one: π StudId (σ CrsCode CrsCode2 (Trancript T ranscript[studid, CrscCode2, Semester2, Grade2])) Question: Why don t we rename StudId? Answer: We need this value to connect all the courses taken by the same student, assuming each student has his/her unique Id. Question: Are you sure this stuf works? 50

51 Let s find it out... Goiven the following Transcript table, SId CrsC G Sem 1111 CS2370 C F CS3600 A F CS2370 B F2015 (T rancript T ranscript[sid, CrscC2, G2, Sem2])) will give us the following: SId CrsC G Sem CrsC2 G2 Sem CS2370 C F2015 CS2370 C F CS2370 C F2015 CS3600 A F CS3600 A F2015 CS2370 C F CS3600 A F2015 CS3600 A F CS2370 B F2015 CS2370 B F2015 (σ CrsC CrsC2 (Trancript Transcript[SId, CrscC2, G2, Sem2])) will give us SId CrsC G Sem CrsC2 G2 Sem CS2370 C F2015 CS36000 A F CS3600 A F2015 CS2370 C F2015 And the whole thing gives us SId

52 Given Related to SQL π StudId (σ CrsCode CrsCode2 (Trancript T ranscript[studid, CrscCode2, Semester2, Grade2])) we can have the following SQL query: mysql> Select distinct T1.StudId -> From Transcript T1, Transcript T2 -> Where T1.StudId=T2.StudId And T1.CrsCode <> T2.CrsCode; Notice that we use different names to get two separate copies of the same table. We will get the following result for this query: StudId Assignment: Find out who they are... 52

53 Something special Question: What is the natural join of R and S when R and S share the same attributes? Answer: By definition, once we construct the product of R and S, and apply the equality selection, only the identical pairs will stay. Finally, we will keep only one copy of those identical pairs. This leads to the collection of tuples that belong to both R and S. Hence, in this case, we have R S. Question: What is the natural join of R and S when they have no attribute in common. Answer: In this case, when we apply the equality selection, nothing will be kicked out. In the projection step, we also can project out no attributes. Then, in this case, the whole product stays. By the way, I just did 5.6 for you. 53

54 Division This might be the most complex operation. It is used in such scenarios that who has taught everything that is offered by the CS department or who has taken every course offered by a particular professor or who supplies every red part? This operator takes two unary relations, A, B; and one binary one, C, as its inputs. As the output, it sends back a relation containing all tuples from the first unary relation that appear in the binary relation, which is matched with all tuples in the second unary relation. In the teaching everything case, A and B refer to ProfId and CrsCode of all the CS courses, respectively; and C is π ProfId,CrsCode (Teaching). 54

55 Worth how many words? Let s check them out with the following data, and will get {1,2,3,4} divide by s per r = {2,3} : 55

56 A final example Query: Who have taken all the courses taught by Professor John Smyth? We know that we can use the division operator, by finding out the three tables, A, B and C. A = π StudId Transcript, i.e., those who have taken courses. To find out B, we have to find out the code of those courses taught by Prof. Smyth, i.e., π CrsCode (σ ProfId=(πProfId (σ Name= John Smyth (Professor)))(Teaching)). It is easy to get C, Who have taken what : π StudId, CrsCode (Transcript). We will see later on how to use MySQL to do division. 56

57 Now the SQL part SQL is the most widely used DB programming language, with MySQL being an incomplete implementation. We can submit individual SQL query statement directly to an DBMS through a terminal, as what we have been doing. But, in practice, we almost always embed them in a larger program that submits a collection of SQL statements to a DBMS at run time and process the returned results. We discuss the former case in this chapter, and talk about the embedding case in a later chapter, Unit9. 57

58 To kick off Query: Who are the professors working in the EE department? mysql> Select P.Name -> From Professor P -> Where P.DeptId= EE ; Name David Jones In the above, we use a tuple variable, P, which ranges over the tuples of the Professor relation. It is not necessary here, but quite useful when we have to deal with several tables with identical attribute names. 58

59 The evaluation process The basic algorithm for evaluating such an SQL statement is as follows: 1. The From part is evaluated to produce a Cartesian product of all the tables mentioned. 2. The Where part is evaluated to apply a restriction on the product where we keep only these rows that make the cut. 3. Finally, the Select part is evaluated to apply a projection to select those attributed from the leftover rows taken from the previous step. Thus, the previous query is nothing but π Name (σ DeptId= EE (Professor)). 59

60 A multi-table example Considering two tables, Dept and Emp DEPT# DNAME BUDGET D1 Marketing 10M D2 Development 12M D3 Research 5M EMP# ENAME DEPT# SALARY E1 Lopez D1 40K E2 Cheng D1 42K E3 Finzl D2 30K E4 Salto D2 35K and the query Select E.EName,E.Salary From Dept D, Emp E Where D.DEPT#=E.DEPT# and E.SALARY<40K; 60

61 The evaluation process 1. The Cartesian product of the two tables, Dept and Emp, as mentioned in the From part, is constructed as follows: D# DNAME BUDGET EMP# ENAME D# SALARY D1 Market 10M E1 Lopez D1 40K D1 Market 10M E2 Cheng D1 42K D1 Market 10M E3 Finzl D2 30K D1 Market 10M E4 Salto D2 35K D2 Develop 12M E1 Lopez D1 40K D2 Develop 12M E2 Cheng D1 42K D2 Develop 12M E3 Finzl D2 30K D2 Develop 12M E4 Salto D2 35K D3 Research 5M E1 Lopes D2 40K D3 Research 5M E2 Cheng D2 42K D3 Research 5M E3 Finzl D2 30K D3 Research 5M E4 Salto D2 35K 61

62 2. Then, the Where part is evaluated, so that only those tuples satisfying the condition D.DEPT#=E.DEPT# and E.SALARY<40K are kept. D# DNAME BUDGET EMP# ENAME D# SALARY D2 Develop 12M E3 Finzl D2 30K D2 Develop 12M E4 Salto D2 35K 3. Finally, the Select part is evaluated. Only those attributes as mentioned in the target list will be kept. ENAME Finzl Salto SALARY 30K 35K This is indeed the result of this query when applied to this instance. 62

63 Query with a join Query: Who taught in Fall 1995? mysql> Select P.Name -> From Professor P, Teaching T -> Where P.Id=T.ProfId And T.Semester= F1995 ; Name Mary Doe David Jones The evaluation of this query really follows the three step process as we discussed earlier. This one is nothing but π Name (Professor Id=ProfId (σ Semester= F1995 (Teaching))). 63

64 SQL and RA Given an SQL query Select TargetList From Rel1 V1,..., Reln Vn Where Condition its RA expression is essentially the following: π TargetList (σ Condition (Rel 1 Rel n )), where we do need to convert the Condition into its RA equivalent. RA will often give us clues as how to come up with SQL query, particularly for those tough ones, as we will see with numerous examples later on. 64

65 An example Query: Who taught what in Fall 1995? Its RA expression is something like the following: π CrsName,Name (Professor (σ Sem= F1995 Teaching) Courses), with appropriate join conditions. This can be immediately turned into an SQL query, as follows: mysql> Select C.CrsName, P.Name -> From Professor P, Teaching T, Course C -> Where T.Semester= F1995 And -> P.Id=T.Profid And T.CrsCode=C.CrsCode; CrsName Name Database Systems. Mary Doe Electronic Circuits David Jones

66 Self-join queries We once mentioned that, to compose a Cartesian product, we have to rename identically named attributes. In particular, we had the following RA expression to get all students who took at least two different courses: π StudId (σ CrsCode CrsCode2 (Trancript T ranscript[studid, CrscCode2, Semester2, Grade2])) Here we keep the table name, but rename the attributes. 66

67 Its SQL cousin To do it in SQL, we have the following, where we rename the table names. mysql> Select Distinct T1.StudId -> From Transcript T1, Transcript T2 -> Where T1.CrsCode<>T2.CrsCode -> And T1.StudId=T2.StudId; StudId Question: Why do we need Distinct? 67

68 We want distinct results If we don t use distinct we would get the following: StudId rows in set (0.00 sec) 68

69 RA SQL The mathematical RA is to get a set, while the practical SQL is to get a multiset. To return a true relations with no duplicates, the evaluator has to do another scan to take out all the duplicates, which can be arranged by putting up another operator mysql> Select Distinct T.ProfId, T.CrsCode -> From Teaching T; ProfId CrsCode MGT EE CS CS CS MAT MGT rows in set (0.00 sec) 69

70 Making comments We sometimes want to make comments to make queries more readable. mysql> #An example of select distinct mysql> Select distinct T.ProfId, T.CrsCode -> From Teaching T; ProfId CrsCode MGT EE CS CS CS MAT MGT rows in set (0.00 sec) 70

71 What is in Where? We have so far only seen simple conditions in the Where part. SQL provides some common operators for such a purpose, such as =, <, <>, (Same as! =, or even Not = ), etc.. In general, any Boolean expression will do. For example, the following query Select E.Id From Employee E, Employee M Where E.BossSSn=M.SSN And E.Salary>2*M.Salary And E.LastName= Mc E.FirstName returns all employees who make more than twice what his boss makes, and whose last name is Mc concatenated with his first name, such as Donald McDonald. 71

72 What about Select? The Select part can also come with a few special features. If you want to get everything from the From part, you put an * in the Select. For example, mysql> select * from Professor; is equivalent to the following: mysql> select Id, Name, DeptId, Age, Salary -> from Professor; Id Name DeptId Age Salary John Smyth CS Adrian Jones MG David Jones EE Qi Chen MA Mary Doe CS Jacob Taylor MG Ann White MA rows in set (0.00 sec) 72

73 Expression in Select SQL permits expressions in the target list, as well as new headings. The following finds out the average salary per year in age for all the professors. mysql> select Name, Age, Salary, Salary/Age As SalaryByAge -> from Professor; Name Age Salary SalaryByAge John Smyth Adrian Jones David Jones Qi Chen Mary Doe Jacob Taylor Ann White rows in set (0.00 sec) 73

74 What does Not mean? Any condition can be negated. For example, NOT (T1.CrsCode=T2.CrsCode) It could be even nested. For example NOT (E.BossSNN=M.SSN And E.Salary>2*M.Salary And NOT (E.LastName= Mc E.FirstName)) Question: What does the last piece mean? Answer: It means that if somebody s salary is more than twice that much of his boss, then his last name is Mc together with his first name, since (A B) A b A B. Labwork: Let s take care of Lawork

75 Set operations With SQL, we can use set operators as defined in RA, i.e., union, intersection and difference. Query: Who are those professors working either in CS or EE departments? mysql> Select P.Name From Professor P -> Where P.DeptId= CS -> Union -> Select P.Name From Professor P -> Where P.DeptId= EE ; Name John Smyth Mary Doe David Jones

76 Recall the following: An equivalent form A B {x : x A x B}, the previous query is quivalent to the following: mysql> Select Distinct P.Name -> From Professor P -> Where (P.DeptId= CS Or P.DeptId= EE ); Name John Smyth David Jones Mary Doe Question: Which one is to use? Answer: It is largely a personal preference in this case, but we don t have a choice for the other two. 76

77 This or that... Query: Who are those professors either affiliated with the Computer Science department or have ever taught a CS course? mysql> Select P.Name From Professor P, Teaching T -> Where P.Id=T.ProfId And T.CrsCode like CS% -> Union -> Select P.Name From Professor P -> Where P.DeptId= CS ; Name Mary Doe John Smyth Alternatively, we can also use or to do the same: mysql> Select Distinct P.Name -> From Professor P, Teaching T -> Where ((P.Id=T.ProfId) And (T.CrsCode like CS% )) -> Or (P.DeptId= CS ); 77

78 This and that... Query: Who are those students who took both CS315 and CS305? We might want to do the following: Select S.Name From Student S, Transcript T Where S.Id=T.StudId And T.CrsCode= CS305 ; Intersect Select S.Name From Student S, Transcript T Where S.Id=T.StudId And T.CrsCode= CS315 But, we can t do it with the current version of MySQL since it is not implemented there, yet. 78

79 What could we do? We can use logical operators. Does the following one work? mysql> Select S.Name From Student S, Transcript T -> Where S.Id=T.StudId And T.CrsCode= CS315 -> And T.CrsCode= CS305 ; Empty set (0.00 sec) Question: Is it really empty? If you check out the instance, Joe Blow took both. Question: Why is it incorrect? The CrsCode box of any tuple contains only one value. So, no CrsCode box of any tuple of Transcript may contain both CS305 and CS315. Have you forgot about data atomicity? 79

80 What should we do? Look for evidence in two tuples in the Transcript table for two different courses. mysql> Select S.Name -> From Student S, Transcript T1, Transcript T2 -> Where S.Id=T1.StudId And T1.CrsCode= CS315 -> And S.Id=T2.StudId And T2.CrsCode= CS305 ; Name Joe Blow Question: What is going on? We came up with two copies of transcript table, T1 and T2, where for the same student S, we look for her record of taking CS315 in T1 and CS305 in T2. Question: How do we make sure it is the same student? Join conditions via S.Id... 80

81 This but not that... Query: Who are those professors who are not affiliated with Computer Science department, but taught a CS course? Select P.Name From Professor P, Teaching T Where P.Id=T.ProfId And T.CrsCode like CS% ) Except Select P.Name From Professor P Where P.DeptId= CS ) This is not supported with the current version of MySQL, either. Question: Is there any alternative? Recall that A \ B = A B = A ( B) = {x x A} {x x B} = {x x A x B}. Hence, the following query should do the trick. mysql> Select P.Name -> From Professor P, Teaching T -> Where P.Id=T.ProfId And T.CrsCode like CS% -> And P.DeptId!= CS ; Empty set (0.00 sec) 81

82 Is it really? mysql> select * from Teaching; ProfId CrsCode Semester MGT123 F EE101 S CS305 F CS315 F MAT123 S EE101 F CS305 S MAT123 F MGT123 F mysql> select * from Professor; Id Name DeptId Age Salary John Smyth CS Adrian Jones MG David Jones EE Qi Chen MA Mary Doe CS Jacob Taylor MG Ann White MA Thus, every one who taught a CS course is from CS department. 82

83 Is it in? With SQL, we can also test whether something is a member of a finite set. Query: Who are those professors who work either in CS or EE department? Select P.Name From Professor P Where P.DeptId In { CS, EE } In MySQL, it looks like the following: mysql> Select P.Name From Professor P Where P.DeptId In ( CS, EE ); Name John Smyth David Jones Mary Doe

84 An application Query: Who taught in fall 1995? mysql> Select P.Name -> From Professor P, Teaching T -> Where P.Id=T.ProfId And T.Semester= F1995 ; Name Mary Doe David Jones This one is involved with a join, for which we have to work out first a Cartesian product, an expensive operation. Could we do something else? 84

85 The nested version... We can also do it in two steps: a) find out their ids from the Teaching table, then, b) using those ids, find out their names in Professor. mysql> Select P.Name From Professor P -> Where P.Id IN -> (Select T.ProfId From Teaching T -> Where T.Semester= F1995 ); Name Mary Doe David Jones Do you like this latter approach better? The fact that SQL statements can be nested makes it much more powerful, but potentially complex. Labwork: Let s take care of Lawork 3.2 next. 85

86 Nested (nasty) queries In the previous example, it might be more natural (?) to come up with the nested version. But, a much more important reason for nesting is that it increases SQL s expressive power in the sense that lots of things can t be done without this feature. Query: Who did not take any course? mysql> Select S.Name From Student S -> Where S.Id Not In -> # All students who takes some course -> (Select T.StudId -> From Transcript T); Name Mary Smith Question: Is it? 86

87 Let s test it out... mysql> Select id, Name From Student; id Name Jane Doe Jesoph Public Mary Smith Bart Simpson Homer Simpson Joe Blow mysql> Select Distinct StudId, CrsCode From Transcript; StudId CrsCode CS EE MAT MGT MGT CS CS MAT EE MAT MGT CS MGT MGT

88 About division Query: Who were taught by all the CS professors. Let s start with students who were not taught by at least one CS professor. mysql> Select Distinct S.Name -> From Student S, -> #All CS Professors -> (Select P.Id From Professor P -> Where P.DeptId= CS ) As CSP -> Where CSP.Id Not In -> #Is this CS professor NOT among those -> #who taught S? If this is the case, -> #we have found the evidence, so we -> #put S s Name into the output bucket. -> (Select T.ProfId -> From Teaching T, Transcript R -> Where T.CrsCode=R.CrsCode And -> T.Semester=R.Semester And -> S.Id=R.StudId); 88

89 What do we get? If you apply the above query to the database instance we have created in Lab 5, we get the following: Name Jane Doe Jesoph Public Mary Smith Bart Simpson Homer Simpson rows in set (0.00 sec) Each and every one of them is not taught by at least one CS faculty. Question: Should we trust Dr. Shen?? 89

90 Let s find it out... Question: Why is Jane in, but Joe out?. With our instance, Table CSP leads to the following two Ids for Computer Science professors: mysql> Select P.Id From Professor P -> Where P.DeptId= CS ; Id Question: Why should Jane be included in the output bucket? 90

91 Knowing that Jane s Id is , we find out the ProfIds of all the professors who have taught her are the following: mysql> Select T.ProfId -> From Teaching T, Transcript R -> Where T.CrsCode=R.CrsCode And -> T.Semester=R.Semester And -> R.StudId= ; ProfId Now, the query tries to check if any of the Id as contained in Table CSP, i.e., ProfId of the Computer Science professors, is not in the above ProfId table. If it is true, it would mean that at least one Computer Science professor did not teach her, thus Jane should belong to the bucket of this query. Neither is... None taught her.. That s why Jane is included in the output bucket. 91

92 How about Joe? Joe s Id is , the ProfIds of all the professors who have taught him are the following: mysql> Select T.ProfId -> From Teaching T, Transcript R -> Where T.CrsCode=R.CrsCode And -> T.Semester=R.Semester And -> R.StudId= ; ProfId Again, the query tries to check if at least one CSP professor did not teach Joe. The inside query fails for both CSP instances. Both taught him. That s why Joe Blow is not included in the output bucket. 92

93 Let s dig a bit deeper... Mathematically speaking, what we have got is the following: B = {s f ( CSP(f) ( Teaching(f, s)))}. Question: What is its complement? S \ B = {s f ( CSP(f) Teaching(f, s))} = {s f ( CSP(f) Teaching(f, s))} = {s f CSP(f) Teaching(f, s))} = {s f CSP(f) Teaching(f, s))}. Therefore, the complement of B collects all the students whom every CS faculty has taught. (Cf. Page 74) Question: How do we get the complement in SQL? Answer: Not in. 93

94 Let s play it out... mysql> Select Name From Student -> Where Id Not In ( -> #Below is the previous query in page 88 -> Select Distinct S.Id -> From Student S, -> #All CS Professors -> (Select P.Id From Professor P -> Where P.DeptId= CS ) As CSP -> Where CSP.Id Not In -> #Professors who has taught S -> (Select T.ProfId -> From Teaching T, Transcript R -> Where T.CrsCode=R.CrsCode And -> T.Semester=R.Semester And -> S.Id=R.StudId)); Name Joe Blow Question: Is this result correct? Answer: Let s find it out... 94

95 The whole 90 feet Question: Who took a CS course? mysql> select * from Transcript where CrsCode like CS% ; StudId CrsCode Semester Grade CS305 F1995 C CS305 F1995 A CS305 S1996 A CS315 F1997 A rows in set (0.00 sec) Question: Who taught a CS courses? mysql> select * from Teaching where CrsCode like CS% ; ProfId CrsCode Semester CS305 F CS305 S CS315 F rows in set (0.00 sec) 95

96 Question: Who are those CS professors? mysql> select Id from Professor where DeptId= CS ; Id rows in set (0.00 sec) Thus, only one student took all the CS courses taught by the two CS professors: whose id is Question: Who is this student? mysql> select Name from Student where Id= ; Name Joe Blow Thus, the result is correct. 96

97 Could we do it with RA? What we have got is the collection of students who have been taught by all the CS faculties. If we do it with Division, we would find out the following: where Result = A Divide B V ia C, A = π StudId Transcript, B = π StudId, ProfId (Transcript CrsCode,Semester Teaching) C = π ProfId (σ DeptId= CS (Professor)). The above MySQL code implements this devision, although MySQL does not provide a direct hit on division Question: Do we have to use four layers of nesting? Is there a simpler way to do it? 97

98 Quantified predicates Beginning with SQL 1999, we can also have a support of limited quantification, with the following basic format For All relation (condition) For Some relation (condition) The first returns true if, for all the tuples in relation, condition is true; while the second returns true if, at least one tuple in relation, condition is true. For example, the following tries to make sure that every professor is teaching at least one course. For All Professor (Id In (Select T.ProfId From Teaching)) 98

99 The exists operator It is often necessary to check if a nested subquery actually returns anything. Query: Who never took any computer science course? One way to do it is to find all CS courses a student has taken and return those students for which this list is empty. Thus, mysql> Select S.Name From Student S -> Where Not Exists ( -> #all CS courses taken by S.Id -> Select T.CrsCode -> From Transcript T -> Where T.CrsCode like CS% -> And T.StudId=S.Id); Name Jane Doe Jesoph Public Mary Smith Question: Can we use Not In in place of Not exists? Why not? 99

100 The All operator Query: Who has the highest GPA among all the students? mysql> Select S.Name, S.Id From Student S -> Where S.GPA >= All (Select S.GPA -> From Student S -> ); Name Id Bart Simpson row in set (0.00 sec) Here the operator All returns true whenever its left argument is at least as high as that of every student. 100

101 Is he the one? Let s check it out... mysql> select Name, GPA from Student; Name GPA Jane Doe 3.4 Jesoph Public 3.3 Mary Smith 3.5 Bart Simpson 3.6 Homer Simpson 3.2 Joe Blow rows in set (0.00 sec) So, the universal quantifier is supported in the MySQL version as currently installed in turing, ,as well! 101

102 An alternative solution MySQL also supports an aggregation operator, MAX, which can be used as follows. mysql> Select S.Name, S.Id -> From Student S -> where S.GPA >= (Select Max(S1.GPA) -> From Student S1); Name Id Bart Simpson row in set (0.03 sec) We will see how to embed this to a script later on. Labwork: It is about time to do Labwork

103 Aggregation As we have already seen, it is often necessary to calculate the average, the maximum, etc., of the values of certain attributes. It also helps to simplify some query (Cf. Labworks 3.3 Sampler). Thus, SQL provides a collection of such aggregated operators. When we want to find out the average age of all the seniors, we do the following: mysql> Select Avg(S.Age) -> From Student S -> Where S.Status= Senior ; Avg(S.Age)

104 Is it correct? mysql> Select S.Name,S.Status,S.Age -> From Student S; Name Status Age Jane Doe freshman 19 Jesoph Public sophomore 21 Mary Smith freshman 21 Bart Simpson senior 22 Homer Simpson senior 21 Joe Blow junior We have two seniors, with their average age being 43/2=21.5. Yes! 104

105 Another example mysql> #The youngest professor in Math mysql> #department mysql> Select P.Name,P.Age From Professor P Where P.DeptId= MA And P.Age=(Select Min(P1.Age) From Professor P1 Where P1.DeptId= MA ); Name Age Qi Chen Check it out... mysql> Select P.Name, P.Age -> From Professor P -> Where P.DeptId= MA ; Name Age Qi Chen 34 Ann White Question: Do we have to use P1? Answer: No. Check it out.... Why? 105

106 More aggregation examples The following finds out the number of professors in the Mathematics department. mysql> Select Count(P.Name) From Professor P -> Where P.DeptId= MA ; Count(P.Name) How about the number of different names in that department? mysql> Select Count(Distinct P.Name) -> From Professor P -> Where P.DeptId= MA ; Count(Distinct P.Name)

107 thou shall not do this You cannot mix an aggregated quantity and any ordinary attribute, like the following: Select count(*), S.Id From Student S Since count returns a single value for the set of tuples, while S.Id tries to send back a set of values for every tuple. The following does work with MySQL: a junior who has the highest GPA among all her peers., since it does not mix things up. mysql> Select S.Name, S.Id From Student S -> Where S.GPA >= (Select Max(S1.GPA) -> From Student S1) -> And S.Status= junior ; Empty set (0.00 sec) 107

108 Aggregation and grouping Although we know how to count professors for one department, what happens if we need this information for all the departments? This is where the Group by clause is used. This clause will group rows of a table that agree on values of a specified subset of attributes. 108