Uncertainty in Databases. Lecture 2: Essential Database Foundations

Transcription

1 Uncertainty in Databases Lecture 2: Essential Database Foundations

2 Table of Contents

3 Table of Contents Codd s Vision Codd Catches On Top Academic Recognition Selected Publication Venues

4 Codd s Vision Codd s Vision Codd Catches On Top Academic Recognition Selected Publication Venues 1970: Edgar F. Codd invents the Relational Database model Data stored as a collection fo relations that conform to a schema and can be accessed through a query language over the schema Separation between the logical and phisical layers Work done in IBM San Jose, which is now IBM Almaden E. F. Codd: A Relational Model of Data for Large Shared Data Banks. In Communications of the ACM 13(6): (1970) 1972: Codd introduced the relational algebra and the relational calculus (logical view of database querying), and proved their equal expressive power [Cod72]

5 Codd Catches On Codd s Vision Codd Catches On Top Academic Recognition Selected Publication Venues 1973: Michael Stonebraker and Eugene Wong implement Codd s vision in INGRES, which commercialised in 1983 and evolved to Postgres (now PostgreSQL) in : A group from the IBM San Jose lab implements Codd s vision in System R, which evolved to DB2 in 1983 SQL initially developed at IBM by Donald D. Chamberlin and Raymond F. Boyce [CB74] 1977: Influenced by Codd, Larry Ellison founds Software Development Laboratories, which becomes Relational Software in 1979, which becomes Oracle Systems Corporation in 1982, named after its flagship product Oracle database

6 Top Academic Recognition Codd s Vision Codd Catches On Top Academic Recognition Selected Publication Venues 1981: Codd receives the ACM Turing Award

7 Breaking News Codd s Vision Codd Catches On Top Academic Recognition Selected Publication Venues Last week: ACM announces Michael Stonebraker as the 2014 Turing Award winner for fundamental contributions to the concepts and practices underlying modern database systems

8 Selected Publication Venues Codd s Vision Codd Catches On Top Academic Recognition Selected Publication Venues Conferences: SIGMOD: ACM Special Interest Group on Management of Data (since 1975) PODS: ACM Symp. on Principles of Database Systems (since 1982) VLDB: Intl. Conf. on Very Large Databases (since 1975) ICDE: IEEE Intl. Conf. on Data Engineering (since 1984) ICDT: Intl. Conference on Database Theory (since 1986) EDBT: Intl. Conference on Extending Database Technology (since 1988) Journals: TODS: ACM Transactions on Database Systems (since 1976) VLDBJ: The VLDB Journal (since 1992) SIGMOD REC: ACM SIGMOD Record (since 1969)

9 Table of Contents Relations Database Schemas Logical Viewpoint

10 Relations Relations Database Schemas Logical Viewpoint A relation r consists of a heading (A 1,..., A m ), which is a sequence (A 1,..., A m ) of distinct attributes, and a body, which is a finite collection of tuples t = (a 1,..., a m ) of values We assume some infinite domain of values (or constants) By a convenient abuse of notation, a relation is often identified with its body (e.g., t r means that t is a tuple in the body of r) We may refer to the ith value in a tuple t R as t.a i or t[i]

11 Database Schemas Relations Database Schemas Logical Viewpoint A relation schema has a relation name (or relation symbol) R and a heading (A 1,..., A n ), which is again a sequence of attributes; it is denoted by R(A 1,..., A m ) Or simply R if the attributes are not important or clear from the context The arity of R(A 1,..., A m ) is m, and is denoted by ar(r) Sometimes the attributes are not important, and we may use just R/m to specify that R is a relation name of arity m A schema is a pair S = (R, Σ), where R is a set of relation schemas with distinct names, and Σ is a set of constraints over R R is often called a signature We later discuss languages of constraints

12 Database Instance Relations Database Schemas Logical Viewpoint A relation r is said to be over a relation schema R if r and R have the same heading A database instance (or just instance) I over a schema S = (R, Σ) associates with every relation name R a relation R I over R, such that all the constraints in Σ are satisfied We denote by Inst(S) the set of all the instances over S

13 Logical Viewpoint Relations Database Schemas Logical Viewpoint It is convenient and common to view the database as a logical system (first-order of higher-order logic) vocabulary=schema+built-in predicates (e.g., <, >, =), and structure=instance Database queries are viewed as logical formulas ϕ(x) over the database: Q(I) = {a I = ϕ(a)} But there are some significant restrictions of the logic: We usually have only relation (no function) symbols We usually consider only finite structures (cf. finite-model theory) should be independent of the domain outside the database (cf. relational calculus)

14 On Finite Structures Relations Database Schemas Logical Viewpoint Consider the set F = {ϕ i i = 1, 2,... } of sentences where ϕ i is the sentence R has at least i distinct tuples Each ϕ i is expressible in first-order logic Every finite subset of F has a finite model, but there is no finite model for F Hence, the compactness theorem no longer holds in the finite

15 Table of Contents Relational Algebra SQL Conjunctive

16 Relational Algebra SQL Conjunctive The queries we will consider are such that produce a relation from the database Formally, a query Q over a schema S is associated with a heading (A 1,..., A k ), and it maps every instance I Inst(S) into a relation with that heading

17 Example Relational Algebra SQL Conjunctive Takes student cno Ahuva 1 Alon 1 Ahuva 2 Course cno cname 1 AI 2 DB 3 PL Find all pairs (s, c) of students and courses, such that there exists a course number x where both Takes(s, x) and Course(x, c) hold student Ahuva Alon Ahuva cname AI AI DB

18 Boolean Relational Algebra SQL Conjunctive A special case is where k = 0, and then the result either contains the empty tuple or is empty; in this case we say that the query is Boolean Example: Is it the case that for some x, both Takes( Ahuva, x) and Course(x, DB ) hold? We often denote Q(I) = {()} by Q(I) = true or I = Q Q(I) = by Q(I) = false or I = Q Boolean queries are very important in the analysis query languages (expressiveness, complexity, optimization and equivalence, etc.)

19 Relational Algebra Relational Algebra SQL Conjunctive Introduced by Codd [Cod72] Used by existing database systems, mainly for internal query-plan optimization A collection of operations over relations Unary: r t; binary: (r, s) t via: 1 Applying the operators to the database relations 2 Composition

20 Algebraic Operators Relational Algebra SQL Conjunctive Union ( ), difference ( ) R S and R S allowed if R and S are union compatible, that is, the have the same heading Cartesian product ( ) R S allowed only when R and S have disjoint headings Projection (π) π A 1,...,A (R) allowed if A k 1,..., A k are distinct attributes of R Selection (σ) σ ϕ (R) allowed if ϕ is a condition over the attributes of R (e.g., A 1 = A 2 or A 1 A 2 ) Renaming (ρ) ρ A B (R) allowed if A is an attribute of R and B is not an attribute of R

21 Example Relational Algebra SQL Conjunctive Takes student cno Ahuva 1 Alon 1 Ahuva 2 Course cno cname 1 AI 2 DB 3 PL Takes ρ cno c Course student cno c cname Ahuva 1 1 AI Alon 1 1 AI Ahuva 2 2 DB Ahuva 1 2 DB Alon 1 3 PL Ahuva 2 3 PL

22 Example Relational Algebra SQL Conjunctive Takes student cno Ahuva 1 Alon 1 Ahuva 2 Course cno cname 1 AI 2 DB 3 PL σ cno=c (Takes ρ cno c Course) student cno c cname Ahuva 1 1 AI Alon 1 1 AI Ahuva 2 2 DB

23 Example Relational Algebra SQL Conjunctive Takes student cno Ahuva 1 Alon 1 Ahuva 2 Course cno cname 1 AI 2 DB 3 PL π student,cno,cname ( σcno=c (Takes ρ cno c Course) ) student cno cname Ahuva 1 AI Alon 1 AI Ahuva 2 DB In short: Takes Course (natural join)

24 Example Relational Algebra SQL Conjunctive Takes student cno Ahuva 1 Alon 1 Ahuva 2 Course cno cname 1 AI 2 DB 3 PL π student,cname ( σcno=c (Takes ρ cno c Course) ) student Ahuva Alon Ahuva cname AI AI DB

25 SQL Relational Algebra SQL Conjunctive SQL (Structured Query Language) is natural language to express relational algebra SELECT (projection)... AS (rename) FROM (Cartesian product) WHERE (selection) UNION MINUS And much more, e.g., aggregate operators (e.g., COUNT, SUM), clustering operators (e.g., GROUP BY, HAVING), ranking (e.g., ORDER BY, LIMIT), and more

26 The Example in SQL Relational Algebra SQL Conjunctive Takes student cno Ahuva 1 Alon 1 Ahuva 2 Course cno cname 1 AI 2 DB 3 PL Find all pairs (s, c) of students and courses, such that there exists a course number x where both Takes(s, x) and Course(x, c) hold SELECT S.student, C.cname FROM Takes T, Course C WHERE T.cno = C.cno

27 Conjunctive Relational Algebra SQL Conjunctive Conjunctive (CQs) are SELECT-FROM-WHERE queries (no MINUS, UNION, etc.) such that all the WHERE conditions are equalities among attributes CQs are typically represented in the following FOL notation: Q(x) : y [ ϕ 1 (x, y) ϕ k (x, y) ] where: x and y are disjoint sequences of variables Each ϕ i (x, y) is a an atomic formula of the form R(τ 1,..., τ m ) where R is an m-ary relation in the schema and each τ j is either a variable in x, a variable in y, or a constant value (e.g., 7 or Ahuva ) Every variable in x occurs at least once on the right hand side

28 CQ Terminology and Notation Relational Algebra SQL Conjunctive Q(x) : y [ ϕ 1 (x, y) ϕ k (x, y) ] For simplification, quantification and conjunction are omitted:

29 CQ Terminology and Notation Relational Algebra SQL Conjunctive Q(x) : y [ ϕ 1 (x, y) ϕ k (x, y) ] For simplification, quantification and conjunction are omitted: Q(x) : ϕ 1 (x, y),, ϕ k (x, y) A variables in x is called a free or head variable, and a variable in y is called an existential variable

30 CQ Terminology and Notation Relational Algebra SQL Conjunctive Q(x) : y [ ϕ 1 (x, y) ϕ k (x, y) ] For simplification, quantification and conjunction are omitted: Q(x) }{{} head : atom {}}{ ϕ 1 (x, y),, ϕ k (x, y) }{{} body A variables in x is called a free or head variable, and a variable in y is called an existential variable

31 The Example as CQ Relational Algebra SQL Conjunctive Takes student cno Ahuva 1 Alon 1 Ahuva 2 Course cno cname 1 AI 2 DB 3 PL Find all pairs (s, c) of students and courses, such that there exists a course number x where both Takes(s, x) and Course(x, c) hold Q(s, c) : Takes(s, x) Course(x, c)

32 Why Are CQs Interesting? Relational Algebra SQL Conjunctive This is the class of all the queries that can be phrased in RA when using only: Selection with equality predicate Projection Join For that reason, CQs are often called SPJ queries CQs are the building block of expressive query languages, such as Datalog Useful queries that are simple enough to perform deep investigation for various database problems As we shall see, there are significant deep insights and algorithms that apply only to conjunctive queries

33 Table of Contents Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies

34 Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Why Do We Care about? Allow to enforce database coherence and avoid bugs Allow to formally determine what inconsistency means Very relevant to us May have dramatic effect on algorithms and complexity By focusing on specific classes of constraint languages, we allow for nontrivial analysis

35 Functional Dependencies Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 A lab belongs to just one faculty (i.e., name is a key for Lab) A specific room in a specific building belongs to only one lab A lab may have multiple rooms, but all in the same building

36 Formal Definition Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Let S be a schema A Functional Dependency (FD) over S is an expression of the form R : U V, where U and V are sets of attributes of R An instance I over S satisfies the FD R : U V if for every two tuples t 1 and t 2 of R I : t 1 and t 2 agree on U t 1 and t 2 agree on V By agree on W we mean that t 1 and t 2 have the same value in every position that corresponds to an attribute of W If U and V cover all the attributes of R, then R : U V is a key constraint and U is said to be a key for R As a simplified notation, we write U and V by simply listing their attributes (no set notation)

37 Example Revisited Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200

38 Example Revisited Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 A lab belongs to just one faculty (i.e., lab is a key for Lab)

39 Example Revisited Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 A lab belongs to just one faculty (i.e., lab is a key for Lab) Lab : name faculty

40 Example Revisited Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 A specific room in a specific building belongs to only one lab

41 Example Revisited Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 A specific room in a specific building belongs to only one lab LabRoom : building room lab

42 Example Revisited Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 A lab may have multiple rooms, but all in the same building

43 Example Revisited Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 A lab may have multiple rooms, but all in the same building LabRoom : lab building

44 Generalizing FDs Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies There are various formalisms that naturally extend FDs to cross-relation dependencies Following are two popular examples: Equality-Generating Dependencies (EGDs) Denial (DCs)

45 EGDs Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies The FD Lab : name faculty can be phrased in FOL as x, y, z [ Lab(x, y) Lab(x, z) y = z ] An EGD is an expression of the form x [ ϕ(x) y 1 = y 2 ] ϕ(x) is a conjunction of atomic formulas y 1 and y 2 are variables in x Example: Lab(l 1, f 1 ), Lab(l 2, f 2 ), LabRoom(l 1, b, r 1 ), LabRoom(l 2, b, r 2 ) f 1 = f 2

46 DCs Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies The FD Lab : name faculty can be phrased in FOL as x, y, z ( Lab(x, y) Lab(x, z) y z ) A DC is an expression of the form x ( ϕ(x) ψ(x) ) x = (x 1,..., x n ) is a sequence of variables Each ϕ(x) is a conjunction of atomic formulas Each γ(x) is a conjunction of comparisons between two variables in x (e.g., x 1 x 2, x 1 < x 2, x 1 x 2, etc.)

47 Inclusion Dependencies Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 Every lab in LabRoom should be listed in the Lab relation (foreign key)

48 Formal Defintion Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Let S be a schema An INclusion Dependency (IND) over S is an expression δ of the form R[A 1,..., A m ] S[B 1,..., B m ] where: R and S are relation name in S R and S may be equal A 1,..., A m are distinct attributes of R B 1,..., B m are distinct attributes of S An instance I over S satisfies δ if π A1,...,A m (R I ) π B1,...,B m (S I )

49 Example Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Consider the relation Friend[person 1, person 2 ] Friend[person 1, person 2 ] Friend[person 2, person 1 ] means that friendship is symmetric

50 Another Example Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Lab name faculty SIPL LCL SSDL STAT EE CS CS IE LabRoom lab building room SIPL Meyer 100 SIPL Meyer 101 LCL Taub 100 SSDL Taub 200 Every lab should be listed in the Lab relation (foreign key) LabRoom[lab] Lab[name]

51 Tuple-Generating Dependencies Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies The IND LabRoom[lab] Lab[name] can be phrased in FOL as x, y, z [ LabRoom(x, y, z) w[lab(x, w)] ] A Tuple-Generating Dependency (TGD) is an expression of the form x [ ϕ(x) yψ(x, y) ] where ϕ(x) and ψ(x, y) are conjunctions of atomic formulas Example: Researcher(p, l), LabRoom(l, b, r) r [PersonRoom(p, b, r )]

52 Question Functional Dependencies Generalizing FDs Inclusion Dependencies Tuple-Generating Dependencies Can we express that Friends is transitive with TGDs? Can we express that Friends has no triangles with TGDs?

53 Table of Contents Data and Combined Other Yardsticks of Efficiency

54 Types of Database Data and Combined Other Yardsticks of Efficiency We consider computational problems that involve one or more of the following components: Schema S A set Σ of constraints A query Q A database instance I Common complexity measures designed to distinguish one input from another (e.g., instances are far bigger than schemas/queries) Combined complexity: everyting is given as input Data complexity: I is given as input, everything else is fixed Formally, we consider infinitely many computational problems P S,Σ,Q, one per combination of S, Σ and Q

55 Data and Combined Other Yardsticks of Efficiency Example: of CQ Answering Problem Def. (Boolean CQ Evaluation) Given a schema S, a Boolean CQ Q over S and an instance I over S, determine whether Q(I) = true. We will show that this problem is NP-complete under combined complexity, by reduction from the Clique problem. Problem Def. (Clique) Given a graph G = (V, E) and a number k, determine whether G contains a clique of size k, that is, a subset U of V such that U = k and every two nodes in U are neighbours.

56 Reduction Data and Combined Other Yardsticks of Efficiency Given G = (V, E) with V = {1,..., n}, and k, construct: S = {R E /2} I G = {R E (i, j) {i, j} E and i < j} Q k is a CQ with existential variables X 1,..., X k, and an atom R E (X i, X j ) for every i and j with 1 i < j k For example, suppose that G is the following graph: I G = R E Q 3 : R E (X 1, X 2 ), R E (X 1, X 3 ), R(X 2, X 3 )

57 Correctness Data and Combined Other Yardsticks of Efficiency The reduction is correct since the following two are equivalent: 1 G has a clique of size at least k 2 Q k (I G ) = true Hence, determining whether Q(I) = true, given S, Q and I, is NP-hard Membership in NP is straightforward, hence, the problem is NP-complete Note: The schema S does not depend on the input (G, k), but the size of Q is quadratic in k

58 Data Data and Combined Other Yardsticks of Efficiency What is the data complexity of answering a query in RA? We consider the problem P S,Q of computing the answers for a query Q in RA (Relational Algebra) over a given input instance I over S The naive way of straightforwardly executing Q runs in polynomial time! As a special case, CQ evaluation is in polynomial time under data complexity Note that data complexity is insensitive to the representation of the query

59 Other Yardsticks of Efficiency Data and Combined Other Yardsticks of Efficiency Other yardsticks of efficiency are often used in database complexity; here are two examples:

60 Parameterized Data and Combined Other Yardsticks of Efficiency Parameterized complexity is between data complexity and combined complexity; Query evaluation is Fixed Parameter Tractable if it can be evaluated in time O(f(Q) p(i)), where f is any (computable) function of the query in p(i) is polynomial in the size of I Our reduction from clique shows that CQ evaluation is not likely to be FPT

61 Input-Output Data and Combined Other Yardsticks of Efficiency A query (e.g., CQ) may be required to output an exponential number of results Hence, it makes no sense to require evaluation in polynomial time Input-output complexity measures the time as a function of both the input and the output Polynomial total time: the running time is polynomial in the combined size of the input and the output Polynomial delay: the answers are produced one by one, where the delay between every two answers is polynomial in the input only

62 Table of Contents

63 I Donald D. Chamberlin and Raymond F. Boyce, SEQUEL: A structured english query language, SIGMOD, ACM, 1974, pp E. F. Codd, A relational model of data for large shared data banks, Commun. ACM 13 (1970), no. 6, , Relational completeness of data base sublanguages, Database Systems (1972),

64 End of lecture 2 Essential Database Foundations