CS 5614: (Big) Data Management Systems. B. Aditya Prakash Lecture #2: The Rela0onal Model, and SQL/Rela0onal Algebra

Transcription

1 CS 5614: (Big) Data Management Systems B. Aditya Prakash Lecture #2: The Rela0onal Model, and SQL/Rela0onal Algebra

2 Data Model A Data Model is a nota0on for describing data or informa0on. Structure of data (e.g. arrays, structs) Conceptual model: In databases, structures are at a higher level. Opera@ons on data (Modifica0ons and Queries) Limited Opera0ons: Ease of programmers and efficiency of database. Constraints on data (what the data can be) Examples of data models The Rela0onal Model The Semistructured-Data Model XML and related standards Object-Rela0onal Model Prakash 2017 VT CS

3 The Model Student Course Grade Hermione Grainger Po0ons A Draco Malfoy Po0ons B Harry Pober Po0ons A Ron Weasley Po0ons C Structure: Table (like an array of structs) Opera@ons: Rela0onal alebgra (selec0on, projec0on, condi0ons, etc) Constraints: E.g., grades can be only {A, B, C, F} Prakash 2017 VT CS

4 The Semi-structured model <CoursesTaken> <Student>Hermione Grainger</Student> <Course>Potions</Course> <Grade>A</Grade> <Student>Draco Malfoy</Student> <Course>Potions</Course> <Grade>B</Grade>... </CoursesTaken> Structure: Trees or graphs, tags define role played by different pieces of data. Follow paths in the implied tree from one element to another. Constraints: E.g., can express limita0ons on data types Prakash 2017 VT CS

5 Comparing the two models Flexibility: XML can represent graphs Ease of use: SQL enables programmer to express wishes at high level. Prakash 2017 VT CS

6 The Model Simple: Built around a single concept for modeling data: the rela0on or table. A rela0onal database is a collec0on of rela0ons. Each rela0on is a table with rows and columns. Supports high-level programming language (SQL). Limited but very useful set of opera0ons Has an elegant mathema0cal design theory. Most current DBMS are rela0onal (Oracle, IBM DB2, MS SQL) Prakash 2017 VT CS

7 A rela0on is a two-dimensional table: Rela0on == table. Abribute == column name. Tuple == row (not the header row). Database == collec0on of rela0ons. A rela0on has two parts: Schema defines column heads of the table (abributes). Instance contains the data rows (tuples, rows, or records) of the table. Student Course Grade Hermione Grainger Po0ons A Draco Malfoy Po0ons B Harry Pober Po0ons A Ron Weasley Po0ons C Prakash 2017 VT CS

8 Schema CoursesTaken : Student Course Grade Hermione Grainger Po0ons A Draco Malfoy Po0ons B Harry Pober Po0ons A Ron Weasley Po0ons C The schema of a rela0on is the name of the rela0on followed by a parenthesized list of abributes. CoursesTaken(Student, Course, Grade) A design in a rela0onal model consists of a set of schemas. Such a set of schemas is called a rela0onal database schema. Prakash 2017 VT CS

9 Equivalent CoursesTaken : Student Course Grade Hermione Grainger Po0ons A Draco Malfoy Po0ons B Harry Pober Po0ons A Ron Weasley Po0ons C CoursesTaken(Student, Course, Grade) Rela0on is a set of tuples and not a list of tuples. Order in which we present the tuples does not maber. Very important! The abributes in a schema are also a set (not a list). Schema is the same irrespec0ve of order of abributes. CoursesTaken(Student, Grade, Course) We specify a standard order when we introduce a schema. How many equivalent representa0ons are there for a rela0on with m abributes and n tuples? m! n! Prakash 2017 VT CS

10 Degree and Cardinality CoursesTaken : Student Course Grade Hermione Grainger Po0ons A Draco Malfoy Po0ons B Harry Pober Po0ons A Ron Weasley Po0ons C Degree/Arity is the number of fields/abributes in schema (=3 in the table above) Cardinality is the number of tuples in rela0on (=4 in the table above) Prakash 2017 VT CS

11 Keys of Keys are one form of integrity constraints (IC) No pair of tuples should have iden0cal keys What is the key for CoursesTaken? Student if only one course in the rela0on Pair (Student, Course) if mul0ple courses What if student takes same course many 0mes? Student Course Grade Hermione Grainger Po0ons A Draco Malfoy Po0ons B Harry Pober Po0ons A Ron Weasley Po0ons C Prakash 2017 VT CS

12 Keys of Keys help associate tuples in different rela0ons SID CID Grade A B B... SID Student GPA 123 Hermione Grainger Draco Malfoy Harry Pober Ron Weasley 3.1 Prakash 2017 VT CS

13 Example Create a database for managing class enrollments in a single semester. You should keep track of all students (their names, Ids, and addresses) and professors (name, Id, department). Do not record the address of professors but keep track of their ages. Maintain records of courses also. Like what classroom is assigned to a course, what is the current enrollment, and which department offers it. At most one professor teaches each course. Each student evaluates the professor teaching the course. Note that all course offerings in the semester are unique, i.e. course names and numbers do not overlap. A course can have 0 pre-requisites, excluding itself. A student enrolled in a course must have enrolled in all its pre-requisites. Each student receives a grade in each course. The departments are also unique, and can have at most one chairperson (or dept. head). A chairperson is not allowed to head two or more departments. Prakash 2017 VT CS

14 Example Create a database for managing class enrollments in a single semester. You should keep track of all students (their names, Ids, and addresses) and professors (name, Id, department). Do not record the address of professors but keep track of their ages. Maintain records of courses also. Like what classroom is assigned to a course, what is the current enrollment, and which department offers it. At most one professor teaches each course. Each student evaluates the professor teaching the course. Note that all course offerings in the semester are unique, i.e. course names and numbers do not overlap. A course can have 0 pre-requisites, excluding itself. A student enrolled in a course must have enrolled in all its pre-requisites. Each student receives a grade in each course. The departments are also unique, and can have at most one chairperson (or dept. head). A chairperson is not allowed to head two or more departments. Prakash 2017 VT CS

15 Design for the Example Students (PID: string, Name: string, Address: string) Professors (PID: string, Name: string, Office: string, Age: integer, DepartmentName: string) Courses (Number: integer, DeptName: string, CourseName: string, Classroom: string, Enrollment: integer) Teach (ProfessorPID: string, Number: integer, DeptName: string) Take (StudentPID: string, Number: integer, DeptName: string, Grade: string, ProfessorEvalua0on: integer) Departments (Name: string, ChairmanPID: string) PreReq (Number: integer, DeptName: string, PreReqNumber: integer, PreReqDeptName: string) Prakash 2017 VT CS

16 Design Example: Keys? Students (PID: string, Name: string, Address: string) Professors (PID: string, Name: string, Office: string, Age: integer, DepartmentName: string) Courses (Number: integer, DeptName: string, CourseName: string, Classroom: string, Enrollment: integer) Teach (ProfessorPID: string, Number: integer, DeptName: string) Take (StudentPID: string, Number: integer, DeptName: string, Grade: string, ProfessorEvalua0on: integer) Departments (Name: string, ChairmanPID: string) PreReq (Number: integer, DeptName: string, PreReqNumber: integer, PreReqDeptName: string) Prakash 2017 VT CS

17 Design: Keys? Students (PID: string, Name: string, Address: string) Professors (PID: string, Name: string, Office: string, Age: integer, DepartmentName: string) Courses (Number: integer, DeptName: string, CourseName: string, Classroom: string, Enrollment: integer) Teach (ProfessorPID: string, Number: integer, DeptName: string) Take (StudentPID: string, Number: integer, DeptName: string, Grade: string, ProfessorEvalua0on: integer) Departments (Name: string, ChairmanPID: string) PreReq (Number: integer, DeptName: string, PreReqNumber: integer, PreReqDeptName: string) Prakash 2017 VT CS

18 Issues to Consider in the Design Can we merge Courses and Teach since each professor teaches at most one course? Do we need a separate rela0on to store evalua0ons? How can we handle pre-requisites that are or s, e.g., you can take CS 4604 if you have taken either CS 3114 or CS 2606? How do we generalize this schema to handle data over more than one semester? What modifica0ons does the schema need if more than one professor can teach a course? Prakash 2017 VT CS

19 SQL AND RELATIONAL ALGEBRA Prakash 2017 VT CS

20 Algebra Rela0onal algebra is a nota0on for specifying queries about the contents of rela0ons Nota0on of rela0onal algebra eases the task of reasoning about queries Opera0ons in rela0onal algebra have counterparts in SQL Prakash 2017 VT CS

21 What is SQL SQL = Structured Query Language (pronounced sequel ). Language for defining as well as querying data in an RDBMS. Primary mechanism for querying and modifying the data in an RDBMS. SQL is declara0ve: Say what you want to accomplish, without specifying how. One of the main reasons for the commercial success of RDMBSs. SQL has many standards and implementa0ons: ANSI SQL SQL-92/SQL2 (null opera0ons, outerjoins) SQL-99/SQL3 (recursion, triggers, objects) Vendor-specific varia0ons. Prakash 2017 VT CS

22 What is an Algebra? An algebra is a set of operators and operands Arithme0c: operands are variables and constants, operators are +,,,, /, etc. Set algebra: operands are sets and operators are, U, - An algebra allows us to construct expressions by combining operands and expression using operators has rules for reasoning about expressions a² + 2 a b + 2b, (a + b)² R (R S), R S Prakash 2017 VT CS

23 selec0on projec0on FUNDAMENTAL operators σ condition π att list cartesian product R X S set union R U S set difference R - S (R) (R) Prakash 2017 VT CS 5614 #23

24 The projec0on operator produces from a rela0on R a new rela0on containing only some of R s columns Delete (i.e. not show) abributes not in projec0on list Duplicates eliminated (sets vs mul:sets) To obtain a rela0on containing only the columns A1,A2,... An of R RA: π A1,A2,... An (R) SQL: SELECT DISTINCT A1,A2,... An FROM R; Prakash 2017 VT CS

25 Example S2 sid sname rating age 28 yuppy lubber guppy rusty (S2) π sname,rating π age ( S2) sname rating age yuppy 9 lubber 8 guppy 5 rusty Prakash 2017 VT CS

26 The selec0on operator applied to a rela0on R produces a new rela0on with a subset of R s tuples The tuples in the resul0ng rela0on sa0sfy some condi0on C that involves the abributes of R with duplicate removal RA: σ (R) SQL: SELECT *FROM R WHERE C; The WHERE clause of a SQL command corresponds to σ( ) c Prakash 2017 VT CS

27 Syntax of Syntax of condi0onal (C): similar to condi0onals in programming languages. Values compared are constants and abributes of the rela0ons men0oned in the FROM clause. We may apply usual arithme0c operators to numeric values before comparing them. RA Compare values using standard arithme0c operators. SQL Compare values using =, <>, <, >, <=, >=. Prakash 2017 VT CS

28 Example S2 sid sname rating age 28 yuppy lubber guppy rusty σ S rating >8 ( 2) sid sname rating age 28 yuppy rusty π σ ( ( S )) sname, rating rating>8 2 Combining Operators sname rating yuppy 9 rusty 10 Prakash 2017 VT CS

29 Set Union Standard defini0on: The union of two rela0ons R and S is the set of tuples that are in R, or S or in both. When is it valid? R and S must have iden0cal sets of abributes and the types of the abributes must be the same. The abributes of R and S must occur in the same order. Prakash 2017 VT CS

30 Set Union RA R U S SQL (SELECT * FROM R) UNION (SELECT * FROM S); Prakash 2017 VT CS

31 Set The intersec0on of R and S is the set of tuples in both R and S Same condi0ons hold on R and S as for the union operator RA R S SQL (SELECT * FROM R) INTERSECT (SELECT * FROM S); Prakash 2017 VT CS

32 Set Difference Set of tuples in R but NOT in S Same condi0ons on R and S as union RA R S SQL (SELECT * FROM R) EXCEPT (SELECT * FROM S); R (R S) = R S Prakash 2017 VT CS

33 Difference S1 sid sname rating age 22 dustin lubber rusty S2 sid sname rating age 28 yuppy lubber guppy rusty S1 S2 sid sname rating age 22 dustin Prakash 2017 VT CS

34 What about strings? find student ssns who live on main (st or str or street - ie., main st or main str ) Prakash 2017 VT CS

35 What about strings? find student ssns who live on main (st or str or street) select ssn from student where address like main% %: variable-length don t care _: single-character don t care Prakash 2017 VT CS

36 operators Are we done yet? Q: Give a query we can not answer yet! Prakash 2017 VT CS 5614 #36

37 operators A: any query across two or more tables, eg., find names of students in 4604 Q: what extra operator do we need?? STUDENT Ssn Name Address 123 smith main str 234 jones forbes ave SSN c-id grade A B Prakash 2017 VT CS 5614 #37

38 operators A: any query across two or more tables, eg., find names of students in 4604 Q: what extra operator do we need?? A: surprisingly, cartesian product is enough! STUDENT Ssn Name Address 123 smith main str 234 jones forbes ave SSN c-id grade A B Prakash 2017 VT CS 5614 #38

39 Cartesian Product The Cartesian product (or cross-product or product) of two rela0ons R and S is a the set of pairs that can be formed by pairing each tuple of R with each tuple of S. The result is a rela0on whose schema is the schema for R followed by the schema for S. RA: R X S SQL: SELECT * FROM R, S ; Prakash 2017 VT CS

40 Cartesian Product S1 sid sname rating age 22 dustin lubber rusty sid bid R1 day /10/ /12/96 S1 X R1 (sid) sname rating age (sid) bid day? 22 dustin /10/96 22 dustin /12/96 31 lubber /10/96 31 lubber /12/96 58 rusty /10/96 58 rusty /12/96 We rename abributes to avoid ambiguity or we prefix abribute with the name of the rela0on it belongs to. Prakash 2017 VT CS

41 selec0on projec0on REMINDER: FUNDAMENTAL operators σ condition π att list cartesian product R X S set union R U S set difference R - S (R) (R) Prakash 2017 VT CS 5614 #41

42 ops Surprisingly, they are enough, to help us answer almost any query we want! derived/convenience operators: set intersec0on ---(We have seen this) join (theta join, equi-join, natural join) rename operator division R S ρ ( R) R' Prakash 2017 VT CS 5614 #42

43 Theta-Join The theta-join of two rela0ons R and S is the set of tuples in the Cartesian product of R and S that sa0sfy some condi0on C. RA: R S SQL: SELECT * FROM R, S WHERE C; R S = σ (R x S) C Prakash 2017 VT CS

44 Theta-Join S1 sid sname rating age 22 dustin lubber rusty R1 sid bid day /10/ /12/96 S1 S1.sid<R1.sid R1 (sid) sname rating age (sid) bid day 22 dustin /12/96 31 lubber /12/96 R c S = σ c ( R S) Prakash 2017 VT CS

45 Natural Join The natural join of two rela0ons R and S is a set of pairs of tuples, one from R and one from S, that agree on whatever abributes are common to the schemas of R and S. The schema for the result contains the union of the abributes of R and S. (so duplicate cols. are dropped) Assume the schemas R(A,B, C) and S(B, C,D)./ RA: R S SQL: SELECT R.A, R.B, R.C, S.D FROM R, S WHERE R.B = S.B AND R.C = S.C; Prakash 2017 VT CS

46 Natural Join: Nit-picking What if R and S have not abributes in common? natural join à cartesian product Some (like Oracle) provide a special single NATURAL JOIN operator, but some (like IBM DB2) don t. So assume there is no special SQL natural join operator Prakash 2017 VT CS

47 Operators so far Remove parts of single Projec0on: Selec0on: Combining Projec0on and Selec0on: (A,B) (R) and SELECT A, B FROM R C(R) and SELECT * FROM R WHERE C (A,B) ( C (R)) SELECT A, B FROM R WHERE C Prakash 2017 VT CS

48 so far Set R and S must have the same abributes, same abribute types, and same order of abributes Union: R U S and (R) UNION (S) Intersec0on: R S and (R) INTERSECT (S) Difference: R S and (R) EXCEPT (S) Prakash 2017 VT CS

49 so far Combine the tuples of two Cartesian Product: R X S,. FROM R, S.. Theta Join: R S,. FROM R, S WHERE C./ Natural Join: R S Prakash 2017 VT CS

50 Ordering find student records, sorted in name order select * from student order by name asc asc is the default Prakash 2017 VT CS

51 Ordering find student records, sorted in name order; break 0es by reverse ssn select * from student order by name, ssn desc Prakash 2017 VT CS

52 Q: why? Rename op. ρ AFTER (BEFORE) A: shorthand; self-joins; for example, find the grand-parents of Tom, given PC (parent-id, child-id) Prakash 2017 VT CS 5614 #52

53 Rename op. PC (parent-id, child-id) PC PC PC p-id Mary Peter John c-id Tom Mary Tom PC p-id Mary Peter John c-id Tom Mary Tom Prakash 2017 VT CS 5614 #53

54 Rename op. first, WRONG abempt: PC PC (why? how many columns?) Second WRONG abempt: PC PC. c id= PC. p id PC Prakash 2017 VT CS 5614 #54

55 Rename op. we clearly need two different names for the same table - hence, the rename op. ρ PC ( PC) 1 PC1. c id= PC. p id PC Prakash 2017 VT CS 5614 #55

56 and Renaming RA: give R the name S; R has n abributes, which are ρ (R) S (A1,A2,... An) called A1, A2,..., An in S SQL: Use the AS keyword in the FROM clause: Students AS Students1 renames Students to Students1. SQL: Use the AS keyword in the SELECT clause to rename abributes. Prakash 2017 VT CS

57 and Renaming Name pairs of students who live at the same address: Students (Name, Address) RA: S1.Name,S2.Name ( S1.Address=S2.Address ( S1 (Students) S2 (Students))) SQL: SELECT S1.name, S2.name FROM Students AS S1, Students AS S2 WHERE S1.address = S2.address Prakash 2017 VT CS

58 and Renaming Name pairs of students who live at the same address: SQL: SELECT S1.name, S2.name FROM Students AS S1, Students AS S2 WHERE S1.address = S2.address Are these correct? No!!! the result includes tuples where a student is paired with himself/herself Solu0on: Add the condi0on S1.name <> S2.name. Prakash 2017 VT CS

59 Division Rarely used, but powerful. Example: find suspicious suppliers, ie., suppliers that supplied all the parts in A_BOMB Prakash 2017 VT CS 5614 #59

60 Division SHIPMENT s# p# s1 p1 s2 p1 s1 p2 s3 p1 s5 p3 ABOMB p# p1 p2 = BAD_S s# s1 Prakash 2017 VT CS 5614 #60

61 Division Observa0ons: ~reverse of cartesian product It can be derived from the 5 fundamental operators (!!) How? Prakash 2017 VT CS 5614 #61

62 Division Answer: r s = π ( R S )( r) π ( R S )[( π ( R S )( r) s) r] Observa0on: find good suppliers, and subtract! (double nega@on) Prakash 2017 VT CS 5614 #62

63 Division Answer: r SHIPMENT s# p# s1 p1 s2 p1 s1 p2 s3 p1 s5 p3 s ABOMB p# p1 p2 = BAD_S s# s1 r s = π ( R S )( r) π ( R S )[( π ( R S )( r) s) r] R: a=ributes of r S: a=ributes of s Observa0on: find good suppliers, and subtract! (double nega@on) Prakash 2017 VT CS 5614 #63

64 Division Answer: r SHIPMENT s# p# s1 p1 s2 p1 s1 p2 s3 p1 s5 p3 s ABOMB p# p1 p2 = BAD_S s# s1 r s = π ( R S )( r) π ( R S )[( π ( R S )( r) s) r] All suppliers All bad parts Prakash 2017 VT CS 5614 #64

65 Division Answer: SHIPMENT s# p# s1 p1 s2 p1 s1 p2 s3 p1 s5 p3 ABOMB p# p1 p2 = BAD_S s# s1 r s = π ( R S )( r) π ( R S )[( π ( R S )( r) s) r] all possible suspicious shipments Prakash 2017 VT CS 5614 #65

66 Division Answer: SHIPMENT s# p# s1 p1 s2 p1 s1 p2 s3 p1 s5 p3 ABOMB p# p1 p2 = BAD_S s# s1 r s = π ( R S )( r) π ( R S )[( π ( R S )( r) s) r] all possible suspicious shipments that didn t happen Prakash 2017 VT CS 5614 #66

67 Division Answer: SHIPMENT s# p# s1 p1 s2 p1 s1 p2 s3 p1 s5 p3 ABOMB p# p1 p2 = BAD_S s# s1 r s = π ( R S )( r) π ( R S )[( π ( R S )( r) s) r] all suppliers who missed at least one suspicious shipment, i.e.: good suppliers Prakash 2017 VT CS 5614 #67

68 Quick Quiz: Independence of Operators R \ S = R (R S) R./ C = C (R S) R./ S =?? Prakash 2017 VT CS

69 Quick Quiz: Independence of R./ S Operators Suppose R and S share the abributes A1,A2,..An Let L be the list of abributes in R \Union list of abributes in S (so no duplicate abributes) Let C be the condi0on R.A1 = S.A1 AND R.A2 = S.A2 AND.. R.An = S.An R./ S = L ( C (R S)) Prakash 2017 VT CS

70 Linear for Algebra Rela0onal algebra expressions can become very long. Use linear nota0on to store results of intermediate expressions. A rela0on name and a parenthesized list of abributes for that rela0on Use Answer as the conven0onal name for the final result The assignment symbol := Any expression in rela0onal algebra on the right Prakash 2017 VT CS

71 Example of Linear Name pairs of students who live at the same address. Normal expression: S1.Name,S2.Name ( S1.Address=S2.Address ( S1 (Students) S2 (Students))) Prakash 2017 VT CS

72 Example of Linear Normal expression: S1.Name,S2.Name ( S1.Address=S2.Address ( S1 (Students) S2 (Students))) Linear Nota0on: Pairs(P1, N1, A1, P2, N2, A2) := S1 (Students) S2 (Students) Matched(P1, N1, A1, P2, N2, A2) := A1=A2(Pairs(P1, N1, A1, P2, N2, A2)) Answer(Name1, Name2) := N1,N2 (Matched(P1, N1, A1, P2, N2, A2)) Prakash 2017 VT CS

73 Queries Involving SELECT A, B FROM R, S WHERE C; Nested loops: for each tuple t1 in R for each tuple t2 in S if the abributes in t1 and t2 sa0sfy C output the tuples involving abributes A and B Prakash 2017 VT CS

74 Queries Involving SELECT A, B FROM R, S WHERE C; Conversion to rela0onal algebra: A,B ( C (R S)) Compute R X S Apply selec0on operator σ() to R X S Project the result tuples to abributes A and B Prakash 2017 VT CS