Materialized Views. March 28, 2018

Transcription

1 Materialized Views March 28,

2 CREATE VIEW salessincelastmonth AS SELECT l.* FROM lineitem l, orders o WHERE l.orderkey = o.orderkey AND o.orderdate > DATE( ) 2

3 CREATE VIEW salessincelastmonth AS SELECT l.* FROM lineitem l, orders o WHERE l.orderkey = o.orderkey AND o.orderdate > DATE( ) SELECT partkey FROM salessincelastmonth ORDER BY shipdate DESC LIMIT 10; 2

4 CREATE VIEW salessincelastmonth AS SELECT l.* FROM lineitem l, orders o WHERE l.orderkey = o.orderkey AND o.orderdate > DATE( ) SELECT partkey FROM salessincelastmonth ORDER BY shipdate DESC LIMIT 10; SELECT suppkey, COUNT(*) FROM salessincelastmonth GROUP BY suppkey; 2

5 CREATE VIEW salessincelastmonth AS SELECT l.* FROM lineitem l, orders o WHERE l.orderkey = o.orderkey AND o.orderdate > DATE( ) SELECT partkey FROM salessincelastmonth ORDER BY shipdate DESC LIMIT 10; SELECT suppkey, COUNT(*) FROM salessincelastmonth GROUP BY suppkey; SELECT partkey, COUNT(*) FROM salessincelastmonth GROUP BY partkey; 2

6 Materialization Views exist to be queried frequently Pre-compute and save the view s contents! (like an index) 3

7 Materialized Views 4

8 Materialized Views Q( ) 4

9 Materialized Views Q( ) 4

10 Materialized Views Q( ) When the base data changes, the view needs to be updated 4

11 View Maintenance VIEW Q(D) 5

12 View Maintenance WHEN D D+ΔD DO: VIEW Q(D+ΔD) 6

13 View Maintenance WHEN D D+ΔD DO: VIEW Q(D+ΔD) Re-evaluating the query from scratch is expensive! 6

14 View Maintenance WHEN D D+ΔD DO: VIEW VIEW+ΔQ(D,ΔD) 7

15 View Maintenance (ideally) Smaller & Faster Query WHEN D D+ΔD DO: VIEW VIEW+ΔQ(D,ΔD) 7

16 View Maintenance (ideally) Smaller & Faster Query WHEN D D+ΔD DO: VIEW VIEW+ΔQ(D,ΔD) (ideally) Fast merge operation. 7

17 Intuition D = {1, 2, 3, 4} ΔD = {5} Q(D) = SUM(D) 8

18 Intuition D = {1, 2, 3, 4} ΔD = {5} Q(D) = SUM(D) Q(D+ΔD) ~ O( D + ΔD ) 8

19 Intuition D = {1, 2, 3, 4} ΔD = {5} Q(D) = SUM(D) Q(D+ΔD) ~ O( D + ΔD ) VIEW + SUM(ΔD) ~ O( ΔD ) 8

20 Intuition R = {1, 2, 3}, S ={5,6} ΔR = {4} Q(R,S) = COUNT(R x S) 9

21 Intuition R = {1, 2, 3}, S ={5,6} ΔR = {4} Q(R,S) = COUNT(R x S) Q(R+ΔR,S) ~ O( ( R + ΔR ) * S ) 9

22 Intuition R = {1, 2, 3}, S ={5,6} ΔR = {4} Q(R,S) = COUNT(R x S) Q(R+ΔR,S) ~ O( ( R + ΔR ) * S ) VIEW + COUNT( ΔR * S ) ~ O( ΔR * S ) 9

23 Incremental View Maintenance WHEN D D+ΔD DO: VIEW VIEW+ΔQ(D,ΔD) 10

24 Incremental View Maintenance WHEN D D+ΔD DO: VIEW VIEW+ΔQ(D,ΔD) Basic Challenges of IVM What does ΔR represent? 10

25 Incremental View Maintenance WHEN D D+ΔD DO: VIEW VIEW+ΔQ(D,ΔD) Basic Challenges of IVM What does ΔR represent? How to interpret R + ΔR? 10

26 Incremental View Maintenance WHEN D D+ΔD DO: VIEW VIEW+ΔQ(D,ΔD) Basic Challenges of IVM What does ΔR represent? How to interpret R + ΔR? How to compute ΔQ? 10

27 What is ΔR? 11

28 What is ΔR? What does it need to represent? 11

29 What is ΔR? What does it need to represent? Insertions Deletions Updates 11

30 What is ΔR? What does it need to represent? Insertions Deletions Updates (Delete Old Record & Insert Updated Record) 11

31 What is ΔR? 12

32 What is ΔR? A Set/Bag of Insertions A Set/Bag of Deletions 12

33 What is +? R A Set/Bag ΔR A Set/Bag of Insertions A Set/Bag of Deletions 13

34 What is +? R A Set/Bag + + ΔR A Set/Bag of Insertions A Set/Bag of Deletions 13

35 What is +? R A Set/Bag + + ΔR A Set/Bag of Insertions A Set/Bag of Deletions R ΔRinserted - ΔRdeleted 13

36 What is +? R A Set/Bag + + ΔR A Set/Bag of Insertions A Set/Bag of Deletions R ΔRinserted - ΔRdeleted But this breaks closure of +! 13

37 Incremental View Maintenance VIEW VIEW+ΔQ(D,ΔD) 14

38 Incremental View Maintenance VIEW VIEW+ΔQ(D,ΔD) 14

39 Incremental View Maintenance VIEW VIEW+ΔQ(D,ΔD) Given Q(R,S, ) Construct ΔQ(R,ΔR,S,ΔS, ) 14

40 Delta Queries σ R 15

41 Delta Queries σ R R ΔR Original R Inserted Tuples of R 15

42 Delta Queries σ σ R R ΔR Original R Inserted Tuples of R 15

43 Delta Queries σ σ R R ΔR Original R Inserted Tuples of R 15

44 Delta Queries σ σ R R ΔR Original R Inserted Does this work for deleted tuples? 15 Tuples of R

45 Delta Queries π π R R ΔR 16

46 Delta Queries π π R R ΔR Does this work (completely) under set semantics? 16

47 Delta Queries U R1 R2 R1 ΔR1 R2 ΔR2 17

48 Delta Queries U R1 R2 R1 ΔR1 R2 ΔR2 17

49 Delta Queries U R1 R2 R1 ΔR1 R2 ΔR2 17

50 Delta Queries x R S R ΔR S 18

51 Delta Queries R : { 1, 2, 3 } S : { 5, 6} 19

52 Delta Queries R : { 1, 2, 3 } S : { 5, 6} R x S = { <1,5>, <1, 6>, <2,5>, <2,6>, <3,5>, <3,6> } 19

53 Delta Queries R : { 1, 2, 3 } S : { 5, 6} R x S = { <1,5>, <1, 6>, <2,5>, <2,6>, <3,5>, <3,6> } ΔRinserted = { 4 } ΔRdeleted = { 3,2 } 19

54 Delta Queries R : { 1, 2, 3 } S : { 5, 6} R x S = { <1,5>, <1, 6>, <2,5>, <2,6>, <3,5>, <3,6> } ΔRinserted = { 4 } ΔRdeleted = { 3,2 } (R+ΔR) x S = { <1,5>, <1, 6>, <4,5>, <4,6> } 19

55 Delta Queries R : { 1, 2, 3 } S : { 5, 6} R x S = { <1,5>, <1, 6>, <2,5>, <2,6>, <3,5>, <3,6> } ΔRinserted = { 4 } ΔRdeleted = { 3,2 } (R+ΔR) x S = { <1,5>, <1, 6>, <4,5>, <4,6> } Δinserted(R x S) = ΔRinserted x S Δdeleted(R x S) = ΔRdeleted x S 19

56 Delta Queries R : { 1, 2, 3 } S : { 5, 6} R x S = { <1,5>, <1, 6>, <2,5>, <2,6>, <3,5>, <3,6> } ΔRinserted = { 4 } ΔRdeleted = { 3,2 } (R+ΔR) x S = { <1,5>, <1, 6>, <4,5>, <4,6> } Δinserted(R x S) = ΔRinserted x S Δdeleted(R x S) = ΔRdeleted x S What if R and S both change? 19

57 Delta Queries Computing a Delta Query 20

58 Delta Queries 21

59 Delta Queries 21

60 Delta Queries The original query 21

61 Delta Queries The original query The delta query 21

62 How about an example 22

63 Delta Queries LINEITEM CUSTOMER ORDERS Let s say you have an insertion into LINEITEM 23

64 Delta Queries LINEITEM CUSTOMER ORDERS 24

65 Delta Queries LINEITEM CUSTOMER ORDERS 25

66 Delta Queries LINEITEM CUSTOMER ORDERS = ø 25

67 Delta Queries LINEITEM CUSTOMER ORDERS 26

68 Delta Queries LINEITEM CUSTOMER ORDERS 27

69 Delta Queries SELECT * FROM CUSTOMER C, ORDERS O, DELTA_LINEITEM DL WHERE C.custkey = O.custkey AND DL.orderkey = O.orderkey AND C.mktsegment = AND O.orderdate = AND DL.shipdate = 28

70 Multisets { 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 4, 4, 4, 4, 4, 4, 5 } (not compact) { 1 x3, 2 x5, 3 x2, 4 x6, 5 x1 } Multiset representation: Tuple # of occurrences 29

71 Multisets { 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 4, 4, 4, 4, 4, 4, 5 } (not compact) { 1 x3, 2 x5, 3 x2, 4 x6, 5 x1 } Multiset representation: Tuple # of occurrences multiplicity 29

72 Multiset Deltas Insertions = Positive Multiplicity Deletions = Negative Multiplicity + = Bag/Multiset Union 30