Lyublena Antova, Christoph Koch, and Dan Olteanu Saarland University Database Group Saarbr ucken, Germany Presented By: Rana Daud

Transcription

1 Lyublena Antova, Christoph Koch, and Dan Olteanu Saarland University Database Group Saarbr ucken, Germany Presented By: Rana Daud

2 Introduction Application Scenarios I-SQL World-Set Algebra Algebraic Equivalences Conclusion & Future work 2

3 INTRODUCTION SID CID GradeA GradeB NULL NULL NULL NULL 3

4 There is no agreement in the literature on the semantics of null values in relational databases: One of the reasons why it is difficult to agree on a semantics is that a null value can be interpreted as an unknown, inapplicable, etc. 4

5 Since each occurrence of a null value can substituted by a non null value, the relation containing nulls can be seen as a shorthand for a set of relations, each obtained by different substitutions. This will be our basic semantic assumption: An incomplete relation represents a set of (complete) relation. 5

6 Incomplete information arises naturally in numerous data management applications like data integration, data cleaning, and data exchange. Recently, research community has shown a vivid interest in efficiently managing incomplete information viewed as a set of possible worlds. A significant amount of research has attempted to find the right balance between the succinctness of world-set representations and the efficiency of query evaluation on top of them. However there is a lack of expressive query languages which are well tailored for sets of possible worlds. 6

7 A query language for incomplete information should at least the following demands Generic Expressive Conservative Efficient evaluation To the date of publication this article, no proposal for a query language for incomplete information has been made that satisfies all of them SQL lacks explicit constructs for dealing with uncertainty, though there are queries on incomplete information that can be expressed as SQL queries on relational representations of incomplete databases with complicated nesting and aggregations. Extensions of RA or SQL with limited constructs (such as certain or top-k) are not expressive enough, as they do not allow for the convenient construction of new worlds. 7

8 APPLICATION SCENARIOS 8

9 Example 1: Business decision support Company_Emp CID EID ACME e1 ACME e2 HAL e3 HAL e4 HAL e5 Emp_Skills EID Skills e1 Web e2 Web e3 Java e3 Web e4 SQL e5 Java 9

10 SELECT * FROM Company Emp choice of CID; U1 U 2 CID EID CID ACME e1 HAL ACME e2 HAL HAL EID e3 e4 e5 10

11 SELECT R1.CID, R1.EID FROM Company_Emp R1, (select * from U choice of EID) R2 WHERE R1.CID = R2.CID and R1.EID!=R2.EID; 11

12 CID V 1.1 EID CID V 1.2 EID ACME e1 ACME e2 CID V 2.1 EID CID V 2.2 EID CID V 2.3 EID HAL e3 HAL e3 HAL e4 HAL e4 HAL e5 HAL e5 12

13 Emp_Skills EID Skills e1 Web e2 Web e3 Java e3 Web e4 SQL e5 Java SELECT certain CID, Skill FROM V, Emp_Skill WHERE V.EID = Emp_Skill.EID Group worlds by (SELECT CID FROM V); 1.* 2. * W CID Skill ACME Web CID HAL W Skill Java 13

14 SELECT possible CID FROM W WHERE Skill= Web ; CID ACME 14

15 Example 2: Trip Planning Flights(Fid,Dep,Arr,Dtime,Atime) Hometowns(City) Flights Dep Arr FRA BCN FRA ATL PAR ATL PAR BCN PHL ATL HomeTowns City FRA PAR PHL... 15

16 create view HFlights as select * from Flights where Dep in Hometowns; select certain Arr from HFlights choice of Dep; Assuming the exsistence of a division operator in SQL: select Arr from (select Arr, Dep from HFlights) as F1 divide by (select Dep from HFlights) as F2 on F1.Dep = F2.Dep; 16

17 REMINDER- DIVISION: R = A B C D S= B 1 2 C 1 2 R S = A 2 D

18 Note: Division can be simulated in SQL using a nested sub-query with two not-exists constructs: select Arr from HFlights F1 where not exists (select * from HFlights F2 where not exists (select * from HFlights F3 where F3.Dep = F2.Dep and F3.Arr = F1.Arr)); This shows that at least in certain cases, I-SQL allows to phrase decision support queries more concisely than plain SQL. 18

19 o We will treat I-SQL informally, mostly in examples. o The structure of an I-SQL query: 19

20 Main motivation is to find a natural extension of RA and SQL to the context of incomplete information. We next detail on the syntax and semantics of the Constructs separated to four groups. Standard SQL constructs Merging worlds Splitting up worlds Data manipulation 20

21 BACK TO FLIGHTS Flights Dep Arr FRA BCN FRA ATL PAR ATL PAR BCN PHL ATL 21

22 Standard SQL constructs: a query is evaluated in each world independently and the result is added as a new relation to that world. Example: SELECT * FROM Flights WHERE Arr = BCN 22

23 Merging worlds: constructs that goes across When the query is a world borders to collect information that appears in other worlds as well. projection on a set of attributes, we will write the set of attributes Possible and certain: compute the tuples that appear in some, respectively all worlds. The result is then group-by in SQL added to each world of the input world-set. directly as is done in the Group-worlds-by: used in combination with possible and certain and allows specifying a condition on which the worlds are grouped. The condition is given in form of an SQL query; worlds that produce the same result of that query are then put into the same group. Then, possible or certain respectively, are computes within each of the created groups. 23

24 A Flights Dep FRA FRA Arr BCN ATL B Flights Dep PAR Arr ATL PAR BCN C Flights Dep PHL Arr ATL Example: SELECT certain Arr FROM Flights Result: F A Arr ATL F B Arr ATL F C Arr ATL 24

25 Note: Even though we used the closing construct certain, the result is again the set of three input worlds, where each of them is extended with a new relation F. Only if the input is a single world, or if one is interested only in the result of the operation and not in the input relations, will a possible or certain construct produce a single world. 25

26 Splitting up worlds: creation of new worlds using the operations: choice-of: freezing the values of the given set of attributes and create separate world for every combination. repair-by-key: Generates the possible repairs that violates a uniqueness constraint for the values of a given set of attributes. Generates possible configurations of items where each configuration contains only one item of a type. naturally fits Data cleaning scenarios ( For example: De-duplication based on keys constraints). 26

27 Flights Dep Arr FRA BCN FRA ATL PAR ATL PAR BCN PHL ATL Example: SELECT * FROM Flights choice of Dep; Result: A Flights Dep Arr B Flights Dep Arr C Flights Dep Arr FRA BCN PAR ATL PHL ATL 27 FRA ATL PAR BCN

28 REPAIR-BY-KEY EXAMPLE: Census(SSN, Name, POB, POW) social security number place of birth place of work Functional Dependency: SSN Name, POB, POW 28

29 SELECT * FROM Census repair by key SSN all possible relations that are consistent with regard to the functional dependency and are contained in the relation Census. Note: This query can produce exponentially many worlds, and is thus not expressible in SQL (or RA). In fact, NP-hard problems can be expressed as queries with repair-by-key. 29

30 Data Manipulation: insert update delete The query is executed in each world of the world-set independently. In case that inserting or updating the tuple violates a constraint in some worlds, the update is discarded in all worlds. Example: DELETE FROM Flights WHERE Arr = ATL Result: A Flights B Flights C Flights Dep Arr Dep Arr Dep Arr 30 FRA BCN PAR BCN

31 Order of evaluation: (1) Computing the product of the relations produced by the sub-queries in the from-clause. (2) Applying the conditions of the where-clause on top. (3) If any of the new operators choice-of, repair-by-key and groupworlds-by are specified, they are applied in the order given by structure of the query in I-SQL : (3.1) choice-of to create a world for each combination of values for the specified attributes. (3.2) repair-by-key in each of the created worlds. (3.3) group-worlds-by operation is applied on the world-set created after the repair-by-key. (4) Projecting on the attributes given in the select list, and if possible or certain are present we union, respectively intersect, the tuples in that projection. 31

32 WORLD-SET ALGEBRA Now we will focus on World-set Algebra in the formal treatment. It is for the fragment of I-SQL without SQL grouping and aggregation constructs. World-set Algebra is an extension of RA with new constructs. It is generic: the semantics of a query is independent of the world-set representation. This is fundamental property. 32

33 Syntax and Semantics: Selection Projection Cartesian Product Union Difference Renaming Intersect Division Base operators r s R\S (r) \ R\S (( R\S (r) s) \ r) 33

34 New constructs: poss cert choice-of U possible group-worlds-by certain group-worlds-by V p U V c U 34

35 SEMANTICS OF THE OPERATORS: World-set A contain worlds over schema R, 1 R2,..., R k Apply a query q World-set contain worlds over schema R Relation that represents the answer to q in each world,..., R k Rk 1,, 1 35

36 Semantics of world-set algebra defines as a function mapping between world-sets оif q is the identity on a relation (i.e., of the form R i ), we add a copy of that relation to each world. Unary operator Evaluate q f f R k1,, in each world is evaluated on and the answer replaces Rk1 R k 1 36

37 Binary operators (,,,, ) Evaluate the operands two world-sets A and A Perform the binary operation in those combinations of one world from A and one world from A that agree on the relation. R,..., 1 R k Forbid operations between relations that occur in different worlds in the original world-set 37

38 choice-of U creates a new world for each choice of the values in the U R k 1 projection on in each world. R k1 The relation is then replaced in each of the new worlds by the subset of R consisting of those tuples that agree on the values Each k1 of U. newly Thus there created are world no two new worlds created from the also same contains world with the relations the same R values of U. 1,..., R k of the world from which it was derived. When applied to the empty This relation, assure choice-of compositionality. produces an empty relation. 38

39 Auxiliary definitions: condition 39

40 group-worlds-by: & The group-worlds-by operators and group worlds in a world-set such that all worlds in a group agree on. We then replace by in each world. In the case of, in each world B is replaced by the union of the relations with B. p U V p U R k1 V R k1 R k1 V c U ( R ) V k1 from the group of worlds associated V c U Analogously, in the case, the new relation in a world p V U V c U R k1 ( R ) U k1 B becomes the intersection of the relations R k1 from the group of worlds associated with B. 40

41 41

42 poss: R k1 is replaced by the union of all its instances across all worlds cert: R k1 is replaced by the intersection of all its instances across all worlds. 42

43 The first query asking for possible acquisition targets can be expressed in world set algebra as: poss( CID ( Skill ' W eb' 1. CID2. CID1. EID2. EID * ( c CID ( 1. CID,1. EID ( CID, EID ( Company _ Emp ) Company _ Emp Emp _ Skills)))) 43

44 GENERICITY Genericity is a fundamental property of query languages. It guarantees that query results are independent from the representation of the data and interpretation of domain values. RA and SQL are generic. World-set algebra is generic: its semantics does not depend on the world-set representation. 44

45 FROM WORLD-SET ALGEBRA TO RA Any world-set algebra query can be efficiently translated to an equivalent relational algebra query over a complete representation of the input world-set. Propose the inlined representation, where the tuples of a relation over all worlds are represented in one table that has special attributes to denote the identifier of the world each tuple belongs to. 45

46 Main contributes of this section: World-set algebra is conservative over RA. This means that any world-set algebra query that maps from a complete database to a complete database (a complete-to-complete query) is equivalent to a RA query An efficient algorithm for effecting this translation. It follows that complete-to-complete world-set algebra queries have the same low data complexity as RA. 46

47 ALGEBRAIC EQUIVALENCES The goal of equivalence is optimization. They defined two classes of equivalences: Commute rules: covers pairs of operators that commute. Reduce rules: covers simplifications of operator compositions. 47

48 Commute Rules: Pushing down of the new operators poss and cert even across projection and selection where this is possible. This usually bears even greater potential for optimization. Some pairs of operators do not commute, for example: Selection & Choice-of Product & poss 48

49 Commute rules 49

50 Reduce rules-examples: Equivalence (11): the operator poss eliminates choice-of operator, because choice-of distributes tuples into a set of disjoint worlds, which latter flattened by the operator poss. Equivalence(15): poss can undo world grouping. Equivalence(20)+(21): in the presence of choice-of operators, the group-worlds-by operators are reduced to simple projections in case the choice attributes occur as both grouping and projecting attributes. Equivalence(22)+(23): redundant poss or cert operations. 50

51 Reduce rules 51

52 EXAMPLE: Consider a possibly incomplete version of our HFlights database from example 2, where additionally we have information on Hotels. HFlights Hotels Dep Arr Name City Price * q1 cert( City( ArrCity( Dep( Dep, City( HFlights Hotels))))) 52

53 53 ))))) ( ( ( ( (, * Hotels HFlights cert City Dep Dep City Arr City ))))) ( ( ( ( ( * Hotels HFlights cert Dep City Arr City q 1 )))) ) ( ( ( ( ( * Hotels HFlights cert Dep City Arr City )) ) ( ( ( Hotels HFlights cert City Arr Dep City q

54 q1 q 1 54

55 CONCLUSION & FUTURE WORK Two application scenarios to motivate I-SQL. I-SQL, an analog to SQL for the case of incomplete information. World-set algebra Genericity Conservativity over RA Expressive Set of equivalences in world-set algebra, which produce more efficient queries. Efficient evaluation 55

56 Future work: generalization to bag semantics implementation of I-SQL on top of a relational engine. To implement I-SQL on top of an existing representation system for finite world-sets, like data bases with lineage and uncertainty. 56

57 Thank you & Good luck 57