CS317 File and Database Systems

Transcription

1 CS317 File and Database Systems Lecture 3 Relational Calculus and Algebra Part-2 September 7, 2018 Sam Siewert

2 RDBMS Fundamental Theory Relational Algebra and Calculus Part-2 (Basis for SQL Transform-oriented) Sam Siewert 2

3 Note on STRUCTURED vs UNSTRUCTURED Data UNSTRUCTURED - Data with no data model (ER, Relational Schema, Hierarchy, etc.). Can include block storage of any type (e.g. Hard Disk Drive, Nand flash, etc.) as well as binary and simple text files in flat file systems when compared to an R-DBMS. File system directories an naming help, but files are still just named bags of bytes perhaps with a file type (binary, text, etc.) and file extension (.doc,.txt,.c,.cpp). STRUCTURED - Data with a specific data model (OO, Relational, Network, Hierarchical, etc.) An R-DBMS is definitely structured in that every byte (bit) is in a domain (table column) as a named attribute of a record or tuple (table row) uniquely identified by a primary key. SEMI-STRUCTURED - Some file systems could be considered to meet this criteria because they have a namespace (directories and file names) for collections of bytes (otherwise unstructured text or binary data), i-nodes to index those bytes, and perhaps with text that is self-describing (e.g. XML, JSON, WSDL, etc.). Sam Siewert 3

4 Physical Storage Challenge 1 TB SSD (e.g. Samsung EVO 860 MSATA V-Nand) 8TB HDD (e.g. Seagate 8TB IronWolf, 6Gb/s 256MB Cache 3.5- Inch Internal) 1 TB Optical (e.g. bandwidth 99.7% C in 96 channel DWDM SMF, N spools of circumference X at 1 Terabit/sec) 1TB cost $240 QD32 read IOPs 97,000 QD32 write IOPs 88,000 Seq Read MB/s 550 Seq Write MB/s 520 8TB cost $248 QD? read IOPs QD? write IOPs seq Read MB/s 215 seq Write MB/s TB cost $37,000,000 QD96 read IOPs QD96 write IOPs seq Read MB/s seq Write MB/s Samsung Spec UserBenchmark Sam s Optical Model Sam Siewert 4

5 Physical Storage Calculations = 1 Kilo x 1024 = 1 Mega x 1 Mega = 1 Giga x 1 Giga = 1 Tera x 1 Tera = 1 Peta x 1 Peta = 1 Exa x 1 Exa = 1 Zetta x 1 Zetta = 1 Yotta = x 2 10 = 2 20 = 1 Mega x 2 20 = 2 30 = 1 Giga x 2 30 = 1 Tera x 2 40 = 1 Peta x 2 50 = 1 Exa x 2 60 = 1 Zetta x 2 70 = 1 Yotta 2 32 = 4 Giga, So 32-bit address for bytes, addresses 4 Gigabytes 2 64 = 16 Exa, So 64-bit address for bytes, addresses 16 Exabytes 2 32 x 4K blocks of bytes = 4 Giga-blocks, or 16 Terabytes (Large HDD) 2 64 x 4K blocks of bytes = 16 Exa-blocks, or 64 Zettabytes (Large RAID Volume) 4K block, requires 12 bits minimum to find a particular byte in block Sam Siewert 5

6 Example Physical Storage Questions How small can a file be for 1-level i-node access if 64K file block is used? [about 60K] How large can a file be for 3-level i-node access if 64K file block is used? [about 60GB] How many I/Os per second can a HDD do if seek + ½ rotation latency is 5 milliseconds? [1000/5 =200] What is ½ rotation latency for 5000 RPM HDD? [83.33 RPS, so 1/ = 12 milliseconds, 6 milliseconds, 1000/6, so < 167 IOPs] What is the ½ rotation latency for 15,000 RPM HDD? [2 milliseconds, 1000/2, so < 500 IOPs] 32-bit byte addressable memory = 2 10 x 2 10 x 2 10 x 2 2 = 4 Gigabytes of capacity, what about 64-bit? Sam Siewert 6

7 For Discussion Read E.F. Codd Paper in More Detail Now Uploaded on Canvas Read Connolly-Begg Chapters 4 & 5, then DBS Chapter 5 for SQL practice and Examples with Dream Home, etc. Assignment #2 Q&A? Focus on Relational Data, Algebra and Calculus Problems Read and Summarize E.F. Codd Paper OR Compare SQL DBMS Options Will tie RA and TC into SQL More in Assignment #3 I Will Do my Best NOT to Hold More than One Assignment in any Class [Violation of my Protocol!] Sam Siewert 7

8 Recall, Goals Relational Algebra or Views Language with Operations that work on Relations (tables) to Define new Relation (no change to Base Relations) Relational Base (Schema) Define Data Model Relational Calculus What Should be Retrieved (Predicate Calculus True/False Assertions) rather than How Today, Focus is to Compare Relational Algebra to Calculus and Understand Sam Siewert 8

9 Connolly-Begg Chapter 5 (See Canvas) Relational Algebra and Calculus Sam Siewert 9

10 Introduction Relational algebra and relational calculus are formal languages associated with the relational model. Informally, relational algebra is a (high-level) procedural language and relational calculus a non-procedural language. However, formally both are equivalent to one another. A language that produces a relation that can be derived using relational calculus is relationally complete. 10

11 Relational Algebra Relational algebra operations work on one or more relations to define another relation without changing the original relations. Both operands and results are relations, so output from one operation can become input to another operation. Allows expressions to be nested, just as in arithmetic. This property is called closure. [Important Concept in CS332, Organization of Programming Languages] 11

12 Relational Algebra Operations [Look for these in E.F. Codd] Intersection can be composed as R (R S) 12

13 Summary of Relational Algebra Page 132, Ch. 5, Conolly-Begg Recall 5 Necessary Operations Recall 3 that Require Unioncompatible Recall Operations Derived from 5 fundamental and extensions Sam Siewert 13

14 Relational Algebra Operations 14

15 Join Cheat Sheet Recall that all SQL JOIN operations can be derived from set theory and relational algebra (convenient features) Sam Siewert 15

16 Examples Load up DreamHome v1.0 (updated) siewertsdvh1 Browse with Adminer Create INNER JOIN QUERY select * from Branch B inner join Property_For_Rent Prop on B.bno = Prop.bno select B.bno,B.street from Branch B inner join Property_For_Rent Prop on B.bno = Prop.bno Sam Siewert 16

17 DreamHome Schema Page 112, Figure 4.3, Ch. 4, Connolly-Begg Exercise to Upgrade Dream Home v1.0 Create table(s) for schema loaded on Canvas with tab delimited data Can be Used to Verify Examples in Chapter 5 and future SQL in notes Sam Siewert 17

18 Operations on Relations Necessary [Connolly-Begg] 1. Selection 2. Projection 3. Cartesian Product Note that Join can be composed of Cartesian Product with Selection 4. Union 5. Set Difference Note that intersection can be composed as R (R S) Codd s Operations 1. Permutation 2. Restriction 3. Projection 4. Join 5. Composition No Set Difference Codd s Redundancy Concerns 1. Strong redundant attributes in a relation 2. Weak redundant attributes in a database Sam Siewert 18

19 Book Examples Connolly-Begg Chapter 5 [Summary Table on Page 132] Sam Siewert 19

20 Selection (or Restriction) σ predicate (R) Works on a single relation R and defines a relation that contains only those tuples (rows) of R that satisfy the specified condition (predicate). 20

21 Example - Selection (or Restriction) List all staff with a salary greater than 10,000. σ salary > (Staff) 21

22 Projection Π col1,..., coln (R) Works on a single relation R and defines a relation that contains a vertical subset of R, extracting the values of specified attributes and eliminating duplicates. 22

23 Example - Projection Produce a list of salaries for all staff, showing only staffno, fname, lname, and salary details. Π staffno, fname, lname, salary (Staff) 23

24 Union R S Union of two relations R and S defines a relation that contains all the tuples of R, or S, or both R and S, duplicate tuples being eliminated. R and S must be union-compatible. If R and S have I and J tuples, respectively, union is obtained by concatenating them into one relation with a maximum of (I + J) tuples. 24

25 Example - Union List all cities where there is either a branch office or a property for rent. Π city (Branch) Π city (PropertyForRent) 25

26 Set Difference R S Defines a relation consisting of the tuples that are in relation R, but not in S. R and S must be union-compatible. 26

27 Example - Set Difference List all cities where there is a branch office but no properties for rent. Π city (Branch) Π city (PropertyForRent) 27

28 Intersection R S Defines a relation consisting of the set of all tuples that are in both R and S. R and S must be union-compatible. Expressed using basic operations: R S = R (R S) 28

29 Example - Intersection List all cities where there is both a branch office and at least one property for rent. Π city (Branch) Π city (PropertyForRent) 29

30 Cartesian product R X S Defines a relation that is the concatenation of every tuple of relation R with every tuple of relation S. 30

31 Example - Cartesian product List the names and comments of all clients who have viewed a property for rent. (Π clientno, fname, lname (Client)) X (Π clientno, propertyno, comment (Viewing)) 31

32 Example - Cartesian product and Selection Use selection operation to extract those tuples where Client.clientNo = Viewing.clientNo. σ Client.clientNo = Viewing.clientNo (( clientno, fname, lname (Client)) Χ ( clientno, propertyno, comment (Viewing))) Cartesian product and Selection can be reduced to a single operation called a Join. 32

33 Join Operations Join is a derivative of Cartesian product. Equivalent to performing a Selection, using join predicate as selection formula, over Cartesian product of the two operand relations. One of the most difficult operations to implement efficiently in an RDBMS and one reason why RDBMSs have intrinsic performance problems. 33

34 Join Operations Various forms of join operation Theta join Equijoin (a particular type of Theta join) Natural join Outer join Semijoin 34

35 Theta join (θ-join) R F S Defines a relation that contains tuples satisfying the predicate F from the Cartesian product of R and S. The predicate F is of the form R.a i θ S.b i where θ may be one of the comparison operators (<,, >,, =, ). 35

36 Theta join (θ-join) Can rewrite Theta join using basic Selection and Cartesian product operations. R F S = σ F (R Χ S) Degree of a Theta join is sum of degrees of the operand relations R and S. If predicate F contains only equality (=), the term Equijoin is used. 36

37 Example Equi-join List the names and comments of all clients who have viewed a property for rent. (Π clientno, fname, lname (Client)) Client.clientNo = Viewing.clientNo (Π clientno, propertyno, comment(viewing)) The = is the F 37

38 Natural join R S An Equijoin of the two relations R and S over all common attributes x. One occurrence of each common attribute is eliminated from the result. 38

39 Example - Natural join List the names and comments of all clients who have viewed a property for rent. (Π clientno, fname, lname (Client)) (Π clientno, propertyno, comment (Viewing)) 39

40 (Left) Outer join To display rows in the result that do not have matching values in the join column, use Outer join. R S (Left) outer join is join in which tuples from R that do not have matching values in common columns of S are also included in result relation. 40

41 Example - Left Outer join Produce a status report on property viewings. Π propertyno, street, city (PropertyForRent) Viewing 41

42 Semijoin R F S Defines a relation that contains the tuples of R that participate in the join of R with S. Can rewrite Semijoin using Projection and Join: R F S = Π A (R F S) 42

43 Example - Semijoin List complete details of all staff who work at the branch in Glasgow. Staff Staff.branchNo=Branch.branchNo(σ city= Glasgow (Branch)) The = is the F 43

44 Division R S Defines a relation over the attributes C that consists of set of tuples from R that match combination of every tuple in S. Expressed using basic operations: T 1 Π C (R) T 2 Π C ((S X T 1 ) R) T T 1 T 2 44

45 Example - Division Identify all clients who have viewed all properties with three rooms. (Π clientno, propertyno (Viewing)) (Π propertyno (σ rooms = 3 (PropertyForRent))) 45

46 Aggregate Operations I AL (R) Applies aggregate function list, AL, to R to define a relation over the aggregate list. AL contains one or more (<aggregate_function>, <attribute>) pairs. Main aggregate functions are: COUNT, SUM, AVG, MIN, and MAX. 46

47 Example Aggregate Operations How many properties cost more than 350 per month to rent? ρ R (mycount) I COUNT propertyno (σ rent > 350 (PropertyForRent)) 47

48 Grouping Operation GAI AL (R) Groups tuples of R by grouping attributes, GA, and then applies aggregate function list, AL, to define a new relation. AL contains one or more (<aggregate_function>, <attribute>) pairs. Resulting relation contains the grouping attributes, GA, along with results of each of the aggregate functions. 48

49 Example Grouping Operation Find the number of staff working in each branch and the sum of their salaries. ρ R (branchno, mycount, mysum) branchno I COUNT staffno, SUM salary (Staff) 49

50 Summary Value of Theory Sam Siewert 50

51 Relationally Complete Necessary Expressive Power for DBMS Avoids Redundancy in Relational Base (Except for Foreign Keys) Results in Minimal Physical Resource Requirements Results in Maximal Views Possible for Applications Built on Shared Data Model Provides Common Standard Core SQL Sam Siewert 51

52 Extensions DMLs Most Often Add Operators for Calculations, Summary, Ordering (Sort) See Full set of Relational Algebra Operators (Page 132) Algebra Has more than Necessary Relational Calculus has Just What s Necessary DMLs Have Much More than Necessary for Convenience (SQL:2011) with Trend to Add Procedural Features PL/SQL Sam Siewert 52

53 Relational Calculus - Complete Existential Qualifier There Exists an X such that Universal Qualifier For all X Same Operators as Boolean Algebra, but Applied to Predicates rather than Binary Predicate is set of all X such that P is true for x in {X P(X} Conjunction ^ is AND Disjunction v is OR Negation ~ is NOT In Tuple Expressions, Free Variables are Left of the Note De Morgan s Laws (Page 134) Sam Siewert 53

54 Tuple Relational Calculus - Example To find details of all staff earning more than 10,000: {S Staff(S) S.salary > 10000} To find a particular attribute, such as salary, write: {S.salary Staff(S) S.salary > 10000} 54

55 Tuple Relational Calculus Existential quantifier used in formulae that must be true for at least one instance, such as: Staff(S) ( B)(Branch(B) (B.branchNo = S.branchNo) B.city = London ) Means There exists a Branch tuple with same branchno as the branchno of the current Staff tuple, S, and is located in London. 55

56 Tuple Relational Calculus Universal quantifier is used in statements about every instance, such as: ( B) (B.city Paris ) Means For all Branch tuples, the address is not in Paris. Can also use ~( B) (B.city = Paris ) which means There are no branches with an address in Paris. 56

57 Example - Tuple Relational Calculus List the names of all managers who earn more than 25,000. {S.fName, S.lName Staff(S) S.position = Manager S.salary > 25000} List the staff who manage properties for rent in Glasgow. {S Staff(S) ( P) (PropertyForRent(P) (P.staffNo = S.staffNo) P.city = Glasgow )} 57

58 Tuple Relational Calculus Expressions can generate an infinite set. For example: {S ~Staff(S)} To avoid this, add restriction that all values in result must be values in the domain of the expression. 58

59 Domain Relational Calculus IGNORE THIS FOR OUR CLASS READ CHAPTER 6, SQL DML Back to Hands-on with MySQL in Assignment #3 Relational Algebra, Tuple Calculus form Basis for Core SQL Sam Siewert 59

60 More Background on Predicate Calculus Propositional Logic Calculator - Lacks the for-all Universal and there-exists Existential Qualifiers Leslie Lamport Logic Calculator Project Sam Siewert 60