CompSci Understanding Data: Theory and Applica>ons

Transcription

1 CompSci Understanding Data: Theory and Applica>ons Lecture 8 Why- Not Queries (Query- based) Instructor: Sudeepa Roy sudeepa@cs.duke.edu 1

2 Today s Paper(s) Why Not? Chapman- Jagadish SIGMOD 2009 (Printouts in class for examples and discussions) 2

3 Provenance vs. Why- Not We learnt about provenance in Lecture 7 Explains the history of an exis>ng output tuple in terms of input tuple What if the output tuple was expected but is missing? How will we explain it? Why- Not = Nega>ve Provenance 3

4 Why- Not ques>ons Why did the chicken not cross the road can t answer this but can answer the following Why did Travelocity not show me the Drake Hotel as an op>on in Chicago? Why did a direct ATL- LAX flight not appear on Kayak? Why did IMDB not return my favorite movie in the search? Why did this tuple not appear in the dataset? 4

5 User may not be able to change query Some>mes a user can explore herself SELECT name FROM employee WHERE salary > 100k no answer try the next SELECT name FROM employee WHERE salary > 70k Many applica>ons do not allow this how to search for missing flight in travelocity Many users are not db users 5

6 An Example The user must specify what she is seeking using key or airibute values Trace the progress of missing answer (Hrotsvit, Basilius) through the manipula>ons on the posi>ve answer (Euripides, Medea) Any manipula>on where they do not behave similarly is a possible answer to Why- Not 6

7 Data Items We will consider tuples in a rela>onal table can be elements (XML), obejects (OODB) can include, overlap with, or be disjoint from another data item has a set of aiributes can be single or mul>- valued Dataset = a set of data items 7

8 Manipula>ons A basic unit of processing in the workflow M(D1, D2, ) = D o takes datasets D1, D2 as input produces D o can also apply to data items d1, d2,.. within dataset D1, D2, In a query, each operator can be a manipula>on In a more complex workflow, each query can be a manipula>on 8

9 Workflows and Query Evalua>on Plan 9

10 Manipula>on Example 10

11 Inputs to the Why- not ques>on The query Q query or workflow against database D divided into manipula>ons The result set R = Q[D] D contains data items (= tuples) R contains result items (may not be in D) items in D or R have a set of aiributes The ques>on Why does R not contain any result sa>sfying a predicate S S is defined over a subset of aiributes A of D (AND, OR) no nega>on in S (double nega>on in why- not) 11

12 Sa>sfac>on- Compa>bility of a Predicate Weaker no>on than compa>bility Consider predicate S as a tree AND/OR at internal nodes atomic predicate at leaves On a data item d some atomic predicates evaluate to TRUE some to FALSE some UNDEFINED Treat UNDEFINED as TRUE not the same as three- valued logic with NULL 12

13 Lineage (again) Given a [manipula>on] instance τ(i) = O an output item o O we call the actual set I I of input data items that contributed to o s deriva>on the lineage of o, and we denote it as I = τ (o,i) Cui- Widom 01 i.e. the set of input tuples that have influenced the data item in the result set 13

14 Lineage Example and Nota>on Find books with price < The Odyssey Lineage of (Sophocles, An>gone ) = (Sophocles, An>gone ) (Homer Odyssey) See Nota>on Lineage through manipula>on m 14

15 Unpicked Data Items 15

16 Unpicked Data Items 16

17 Unpicked Data Items Every data item is unpicked if the user ques>on is Why not anything Why is the result set empty 17

18 Successor Lineage only applied to items in result set used in unpicked but refers to items in result set But why- not data items are not in the result set do not have a lineage cannot refer to their unpicked items Need successor to trace forward lineage: output to input successor: input to output 18

19 Successor Unpicked data item does not exist in result set by defini>on But can use this defini>on to to watch how it moves in a workflow Successor depends on lineage, not on airibute value i.e. Airibute preserva>on is not required Why not $61? even if price does not appear in the output the output (Sophocles, An>gone ) is associated with (Sophocles, An>gone, $61) in the input 19

20 Picky manipula>on 20

21 Picky manipula>on 21

22 Picky manipula>on We can iden>fy one or more picky manipula>on for each unpicked data item The union is the set of picky manipula>on But this set may be large each of which is filtering something out user may be overwhelmed Instead, return the highest level of picky manipula>ons top- most in the query tree or latest in the workflow If desired, user can dig deeper in the tree 22

23 Fron>er Picky Manipula>on 23

24 Fron>er Picky Manipula>on NOTE: they depend on the set of unpicked items specified U workflow is DAG in general, not a path 24

25 Fron>er Picky Manipula>on 25

26 Why- not answer 26

27 Why- not answer depends on query 27

28 Algorithm: Boiom- Up Check the output of every manipula>on start at the DAG sources proceed in topologically sorted order Whenever an unpicked successor at the output of a manipula>on is found.. gives a picky manipula>on To find a Fron>er Picky Manipula>on con>nue through the DAG make sure that unpicked successors do not appear later in the DAG (in an alterna>ve path) 28

29 Algorithm: Top- down Work top- down in the DAG In reverse topological order Start from the result set Look for unpicked successor As soon as an unpicked successor is sighted back up one step output the iden>fied Fron>er Picky Manipula>on 29

30 top- down vs. boiom up Asympto>c complexity iden>cal Which one to choose? Depends on where the fron>er is whether materialized intermediates exist Top down finds fron>er close to the output fast Boiom up finds fron>er close to the input fast Choices store intermediate more space, less >me can use either topdown & boiom up start with ini>al data and re- run more >me, less space have to use boiom up 30

31 In the paper Successor visibility if we can find which input contributed to an output in O(1) >me for an operator Evalua>on why- not qualita>ve answer performance 31

32 Why- not approaches Query based this paper: find out fron>er query operator op>onal reading Tran- Chan 10: find out changes to query operator that returns missing answer Later: Data based find out changes in data that will get the missing answers back Huang et al. 08, Herschel- Hernandez 10 32