FDB: A Query Engine for Factorised Relational Databases

Size: px

Start display at page:

Download "FDB: A Query Engine for Factorised Relational Databases"

Charles Park
5 years ago
Views:

1 FDB: A Query Engine for Factorised Relational Databases Nurzhan Bakibayev, Dan Olteanu, and Jakub Zavodny Oxford CS Christan Grant cgrant@cise.ufl.edu University of Florida November 1, 2013 cgrant (UF) FDB: Factorised RDBMS November 1, / 39

2 Contents 1 Intro 2 F-Representation 3 F-Trees 4 Query Evaluation Normalisation Operator Swap Operator Cartesian Product Operator Selection Operator Merge Selection Operator Absorb Selection Operator Selection with Constant Operator Projection Operator 5 Query Optimization Cost of an F-Plan Exhaustive Search Greedy Heuristic 6 Experiments Experiment #1 Experiment #2 Experiment #2 Experiment #3 Experiment #4 7 Conclusion 8 Thanks cgrant (UF) FDB: Factorised RDBMS November 1, / 39

3 Intro Intro A succinct representation for large relations. It is used in Google s F1 data management system for ML. Used for large incomplete/uncertain relations. FDB can speed up expected queries. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

4 Intro Intro A succinct representation for large relations. It is used in Google s F1 data management system for ML. Used for large incomplete/uncertain relations. FDB can speed up expected queries. This paper discusses basics FDB system. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

5 TL;DR Intro Given a query: Q 1 = Order item Store location Disp cgrant (UF) FDB: Factorised RDBMS November 1, / 39

6 TL;DR Intro Given a query: Q 1 = Order item Store location Disp And a result: cgrant (UF) FDB: Factorised RDBMS November 1, / 39

7 TL;DR Intro Given a query: Q 1 = Order item Store location Disp And a result: We can represent it as this: cgrant (UF) FDB: Factorised RDBMS November 1, / 39

8 TL;DR Intro Given a query: Q 1 = Order item Store location Disp And a result: We can represent it as this: Or a tree: cgrant (UF) FDB: Factorised RDBMS November 1, / 39

9 Intro TL;DR Given a query: Q 1 = Order item Store location Disp And a result: We can represent it as this: Or a tree: In exponentially -space and +speed for SPJ queries. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

10 Intro Contents 1 Intro 2 F-Representation 3 F-Trees 4 Query Evaluation Normalisation Operator Swap Operator Cartesian Product Operator Selection Operator Projection Operator 5 Query Optimization Cost of an F-Plan Exhaustive Search Greedy Heuristic 6 Experiments Experiment #1 Experiment #2 Experiment #2 Experiment #3 Experiment #4 7 Conclusion 8 Thanks cgrant (UF) FDB: Factorised RDBMS November 1, / 39

11 Example Schema Intro cgrant (UF) FDB: Factorised RDBMS November 1, / 39

12 F-Representation Factorized Representations Relational algebra expressions. Constructed with singleton relations v, the union and product operators. exponentially more succinct than relations they encode. fast (constant-delay) enumeration of tuples cgrant (UF) FDB: Factorised RDBMS November 1, / 39

13 F-Representation Factorized Representations Definition 1 A factorised representation E, or f-representation for short, over a set S of attributes and domain D is a relational algebra expression of the form, empty relation over S;, the relation consisting of the nullary tuplem if S = ; A : a, the unary relation with a single tuple with value a, if S = {A} and a is a value in the domain D; (E), where E is an f-representation over S; E 1 E n, where each E i is an f-representation over S; E 1 E n, where each E i is an f-representation over S i and S is the disjoint union over all S i. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

14 F-Representation Factorized Representations Singleton A : a is a singleton E the size is the number of singletons A single system may have many different f-representations. The tuples of a given f-representation can be enumerated with O( S ) additional space and delay. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

15 F-Representation Factorized Representations Singleton A : a is a singleton E the size is the number of singletons A single system may have many different f-representations. The tuples of a given f-representation can be enumerated with O( S ) additional space and delay. Factorized Representation Compact Factorized Representation cgrant (UF) FDB: Factorised RDBMS November 1, / 39

16 F-Trees F-Trees Definition An f-tree for short, over a schema S of attributes is an unordered rooted forest with each node labelled by a non-empty subset of S such that each Q1 F-Tree Q1 Alternate F-Tree attribute of S labels exactly one node. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

17 F-Trees F-Trees Definition An f-tree for short, over a schema S of attributes is an unordered rooted forest with each node labelled by a non-empty subset of S such that each Q1 F-Tree Q1 Alternate F-Tree attribute of S labels exactly one node. Tree created in O( S R log R ). We can go back to R in O( S R ). cgrant (UF) FDB: Factorised RDBMS November 1, / 39

18 F-Trees F-Trees Queries Given a query Q = π P σ ϕ (R 1 R n ) Given a query Q, an f-tree T is an f-tree of Q if and only if it satisfies the path constraint. all dependent attributes can only label nodes along a same root-to-leaf path. Nodes of a join are only dependent if they are in the P list or the same R i. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

19 F-Trees F-Trees Size Bounds Let st be the maximal root-to-leaf path edge cover T. For and database D and f-tree T the result size is at most O( P D s(t ) ) cgrant (UF) FDB: Factorised RDBMS November 1, / 39

20 F-Trees F-Trees Size Bounds Let st be the maximal root-to-leaf path edge cover T. For and database D and f-tree T the result size is at most O( P D s(t ) ) If we let s(q) be the minimal s(t ) over all f-trees. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

21 F-Trees F-Trees Size Bounds Let st be the maximal root-to-leaf path edge cover T. For and database D and f-tree T the result size is at most O( P D s(t ) ) If we let s(q) be the minimal s(t ) over all f-trees. P D s(q) Q(D) This can mean asymptotically smaller, exponential size and time savings cgrant (UF) FDB: Factorised RDBMS November 1, / 39

22 Query Evaluation Contents 1 Intro 2 F-Representation 3 F-Trees 4 Query Evaluation Normalisation Operator Swap Operator Cartesian Product Operator Selection Operator Projection Operator 5 Query Optimization Cost of an F-Plan Exhaustive Search Greedy Heuristic 6 Experiments Experiment #1 Experiment #2 Experiment #2 Experiment #3 Experiment #4 7 Conclusion 8 Thanks cgrant (UF) FDB: Factorised RDBMS November 1, / 39

23 Query Evaluation F-plans Any select-project-join query can be evaluated by a sequential composition of operators called an f-plan. f-trees uniquely identify f-representations operators are in tree form. The time complexity of each f-plan operator is O( T 2 N log N), where N is the sum of input and output f-representations and T is the input f-tree. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

24 Query Evaluation Normalisation Operator Normalisation Operator 1 push-up operator ψ B normalization η(t ) A tree is normalized if the push up operator can not be applied. The push-up operator is applied bottom up. ψ B η(t ) Normalization 1 All other operators expect normalized inputs cgrant (UF) FDB: Factorised RDBMS November 1, / 39

25 Query Evaluation Swap Operator Swap Operator X A,B Exchanges B with its parent A and promotes children. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

26 Query Evaluation Cartesian Product Operator Cartesian Product Operator T = T T cgrant (UF) FDB: Factorised RDBMS November 1, / 39

27 Query Evaluation Selection Operator Merge Selection Operator µ A,B If A and B are siblings cgrant (UF) FDB: Factorised RDBMS November 1, / 39

28 Query Evaluation Selection Operator Merge Selection Operator µ A,B If A and B are siblings µ(, ) = cgrant (UF) FDB: Factorised RDBMS November 1, / 39

29 Query Evaluation Selection Operator Absorb Selection Operator α A,B If A is an ancestor of B Example: cgrant (UF) FDB: Factorised RDBMS November 1, / 39

30 Query Evaluation Selection Operator Selection with Constant Operator σ Aθc Filters out all the values in A : a where a θc cgrant (UF) FDB: Factorised RDBMS November 1, / 39

31 Query Evaluation Projection Operator Projection Operator π A Removes a node from the tree. Only permitted if A is a leaf node or with other attributes. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

32 Query Optimization Contents 1 Intro 2 F-Representation 3 F-Trees 4 Query Evaluation Normalisation Operator Swap Operator Cartesian Product Operator Selection Operator Projection Operator 5 Query Optimization Cost of an F-Plan Exhaustive Search Greedy Heuristic 6 Experiments Experiment #1 Experiment #2 Experiment #2 Experiment #3 Experiment #4 7 Conclusion 8 Thanks cgrant (UF) FDB: Factorised RDBMS November 1, / 39

33 Query Optimization Query Optimization Query optimization has two objectives: minimizing the cost of computing a factorised query result; minimizing the size of this output representation. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

34 Query Optimization Query Optimization Optimize the f-tree then the f-representation. Products and Selections are always evaluated first (cheapest). Selection (A = B) can only be done if they are on the same path or siblings cgrant (UF) FDB: Factorised RDBMS November 1, / 39

35 Cost of an F-Plan Query Optimization Cost of an F-Plan use asymptotic bounds of s(t ). with an evaluation time of O( D s(f ) log D ), where s(f ) = max(s(t 0,... s(t k )). cgrant (UF) FDB: Factorised RDBMS November 1, / 39

36 Query Optimization Cost of an F-Plan Cost of an F-Plan use asymptotic bounds of s(t ). with an evaluation time of O( D s(f ) log D ), where s(f ) = max(s(t 0,... s(t k ))., or, use cardinality of estimate from intermediate f-trees using RDBMS style techniques cgrant (UF) FDB: Factorised RDBMS November 1, / 39

37 Query Optimization Exhaustive Search Exhaustive Search Enumerate plans and the one with the shortest path (Dijkstra) between T init to T final is least cost. Search space is O(n 4n ) with n p attributes in the projection list cgrant (UF) FDB: Factorised RDBMS November 1, / 39

38 Query Optimization Greedy Heuristic Greedy Heuristic Only restructures nodes that appear in selections and projections. In projection ordering chooses the least cost at each step. Computing the tree take O( T 2 ). cgrant (UF) FDB: Factorised RDBMS November 1, / 39

39 Experiments Contents 1 Intro 2 F-Representation 3 F-Trees 4 Query Evaluation Normalisation Operator Swap Operator Cartesian Product Operator Selection Operator Projection Operator 5 Query Optimization Cost of an F-Plan Exhaustive Search Greedy Heuristic 6 Experiments Experiment #1 Experiment #2 Experiment #2 Experiment #3 Experiment #4 7 Conclusion 8 Thanks cgrant (UF) FDB: Factorised RDBMS November 1, / 39

40 Experiments Implementation Compare SQLite and PostgreSQL with FDB and RDB for SQLite and PostgreSQL tried to disable expensive writing (try to make it in-memory) Implemented in C++ Optimal f-representation computed with multi-way merge-sort join algorithm. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

41 Experiments Experiment Design Generate random data for queries Repeat a number of times for optimizations time, execution time, representation sizes and f-plan quality Tuples generated independently with a uniform or Zipf distribution. Queries are equality joins cgrant (UF) FDB: Factorised RDBMS November 1, / 39

42 Experiments Experiment #1 Experiment #1 Average time for optimising a query on flat data. A = 40 Attributes, R = 1,..., 8 relations, K = 1,..., 9 selections cgrant (UF) FDB: Factorised RDBMS November 1, / 39

43 Experiments Experiment #1 Experiment #1 Average time for optimising a query on flat data. A = 40 Attributes, R = 1,..., 8 relations, K = 1,..., 9 selections cgrant (UF) FDB: Factorised RDBMS November 1, / 39

44 Experiments Experiment #1 Experiment #1 Average time for optimising a query on flat data. A = 40 Attributes, R = 1,..., 8 relations, K = 1,..., 9 selections cgrant (UF) FDB: Factorised RDBMS November 1, / 39

45 Experiment #2 Experiments Experiment #2 cgrant (UF) FDB: Factorised RDBMS November 1, / 39

46 Experiment #2 Experiments Experiment #2 cgrant (UF) FDB: Factorised RDBMS November 1, / 39

47 Experiment #3 Experiments Experiment #3 Compare performance of FDB, RDB, SQLite and PostgreSQL Results size Evaluation Time cgrant (UF) FDB: Factorised RDBMS November 1, / 39

Each tuple size decreases results size by 20 Results

48 Experiments Experiment #3 Experiment #3 Compare performance of FDB, RDB, SQLite and PostgreSQL. Each tuple size decreases results size by 20 Results size Evaluation time cgrant (UF) FDB: Factorised RDBMS November 1, / 39

49 Experiments Experiment #3 Experiment #3 Compare performance of FDB, RDB, SQLite and PostgreSQL. In Q A is a many-many query. This is highly compressible so the factorized result is small. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

50 Experiment #4 Experiment over Factorized data Experiments Experiment #4 Results size Evaluation time cgrant (UF) FDB: Factorised RDBMS November 1, / 39

51 Conclusion Conclusion Factorizing Relational Data prevents the explosion of data size for large joins. This paper described techniques and experiments to decrease this size and improving query performance. For MLNs and other graph DBs that require many joins this can significantly improve query performance. cgrant (UF) FDB: Factorised RDBMS November 1, / 39

52 Thank you Thanks Questions? cgrant (UF) FDB: Factorised RDBMS November 1, / 39

FDB: A Query Engine for Factorised Relational Databases

FDB: A Query Engine for Factorised Relational Databases Nurzhan Bakibayev, Dan Olteanu, and Jakub Závodný Department of Computer Science, University of Oxford, OX1 3QD, UK {nurzhan.bakibayev, dan.olteanu,