AMAχOS Abstract Machine for Xcerpt

Transcription

1 AMAχOS Abstract Machine for Xcerpt Principles Architecture François Bry, Tim Furche, Benedikt Linse PPSWR 06, Budva, Montenegro, June 11th, 2006

2 Abstract Machine(s) Definition and Variants abstract machine := interpreter for low-level code according to machine model (representation, instruct. set) abstract machine ~ virtual machine why abstract machines? thought models hardware or OS-level virtualization AMs for high-level (programming) languages 2

3 Virtualization everywhere Carbon and Mac OS X As shown in the following figure, Carbon is one of several application environments available on Mac OS X. OS-level C O V E R F E A T U R E Intel Virtualization Technology Once confined to specialized server and mainframe systems, virtualization is now supported in off-the-shelf systems based on Intel architecture hardware. Intel Virtualization Technology provides hardware support for processor virtualization, enabling simplifications of virtual machine monitor software. Resulting VMMs can support a wider range of legacy and future operating systems while maintaining high performance. High-level languages ECMA rd Edition / June 2005 Common Language Infrastructure (CLI) Partitions I to VI V Hardware 3

4 AMAχOS AM for Web Querying Operational Semantics for Xcerpt abstract machine ~ instruction set + machine model just like algebra ~ operators + data model both: precise query semantics on a logical level on the operational/physical level optimizability different combinations of instructions with equivalent overall result but different performance characteristics 4

5 AMAχOS AM for Web Querying Vision language neutral starting with a bias towards Xcerpt already: query core applicable for {Xcerpt, XQuery, XSLT, SPARQL} focus: in-memory processing of distributed data no (guaranteed) control over storage and indexing of data ad-hoc index creation (like XSLT key) and data model selection +: distributed evaluation (if additional query nodes known) distribute compiled code over nodes acc. cost estimation 5

6 AMAχOS AM for Web Querying We have principles Are we alone in this? not quite: XLSTVM now part of Oracle DB centralized query processing very specialized instruction set for XLST 1.0/Oracle algebra vs. abstract machine very similar idea: operators and their semantics but: usually tightly integrated (at least on physical layer) algebra for XML querying hot research issue 6

7 Enough vision already on to the details

8 var Y a[[ b { var X }, Example var Y desc c { var X } ]] How to evaluate this? one eval operator? splitting it into base relations conjunctive queries root(v 0 ) child(v 0, v 1 ) label(v 1, a ) child(v 1, v 2 ) label(v 2, b ) much better for optimization (move tough decisions to compile-time) but: naively done exponential compromise path, twig operators: root(v 0 ) path(v 0, a.b, v 1 ) split at join and result variable 8

9 AMAχOS Core Data Model several variants but common principles: basic type: node with properties element (structured) vs. content (atomic) nodes semi-structured data model with node identity differs from previous Xcerpt DM (infinite regular trees) memory model: memoization matrix non-1-normal-form table of operator results non-redundant (polynomial) store of query results 9

10 AMAχOS Core bib d 1 conference 1 d 2 conference Child + v 5 Root Child paper posters d 4 2 d 7 title author Wax Tablets 1 paper d 3 d 5 1 author Cicero d 6 d 8 paper author pc d 9 d 11 member d 12 name d 14 Storage Media member d 13 Cicero Hirtius d 10 Variable Node Sub-Matrix paper author Child + v 1 v 4 Child + name member v 3 v 2 v 5 d 2 Variable Node Sub-Matrix v 4 d 3 Variable Node Sub-Matrix v 1 d 6 v 1 d 7 v 4 d 5 Variable Node Sub-Matrix v 3 d 11 v 2 d 13 10

11 Operators Three phase algorithm matrix population evaluates only a spanning tree T of operators from query Q directed semi-joins polynomial evaluation expansion of non-tree joins (similar to OO DBS case) worst-case exponential in time and space matrix consumption construction in the flavor of complex value algebra 11

12 Operators Matrix population (spanning tree T of query Q) unary relations (property filters) binary & ternary relations (structural assembly) basic relations (child, desc), (reg.) path operators, twig operators Non-tree join expansion value, identity, and (direct) structural join Matrix consumption basic constructors for each node type grouping, aggregation, order, 12

13 Optimizability Lot s of freedom at compilation how to distribute operators between phase (1) and (2) matrix population: semi-joins, but only acyclic CQ join expansion: arbitrary shape, but exponential cover areas for join variables to reduce exponent hypertree/query decomposition choosing the right operator conjunction of base relations vs. twig operator supportive indices and DM variants e.g., set-based vs. streaming (time vs. space) 13

14 AMAχOS Execution Complexity The core of the core: the evaluation algorithm tree query graph query tree data O(q v 2 + o) O(v q ) graph data O(q v e + o) O(v q ) Table 1: Overview of Combined Time Complexity (q: number of query variables; e, v number of edges, vertices resp., in the data; o: size of output) 14

15 The core of the core: the evaluation algorithm time (msec, logarithmic) query size (variables) without memoization with memoization data size fixed 15

16 The core of the core: the evaluation algorithm time (msec) data size (MB) top-down query size fixed (~ 20 nodes) 16

17 Query Network Application control API (Java) AMAχOS Code Hint Segment Dependency Segment Code Segment Local Data Source e.g. document e.g. database Local Data Source e.g. document e.g. database rule 1 rule 2 AMAχOS Node Xcerpt Node rule n rule 1 Application Web Service API Application command-line interface Query Compiler Xcerpt Program rule 1: c 1 q 1,1 q 1,2 q 1,k1 rule 2: c 2 q 2,1 q 2,2 q 2,k2 rule 3: c 3 q 3,1 q 3,2 q 3,k3 query conjunct q 1,2 AMAχOS Node query conjunct q 1,1 rule 2 AMAχOS Node query conjunct q 2,2 AMAχOS Node AMAχOS Node query conjunct q 2,1 AMAχOS Node Remote Data Source e.g. Web service Remote Data Source e.g. Web service Local Data Source e.g. document e.g. database 17

18 AMAχOS Architecture And a way to realize them Control Plane Compilation API simple observation and control API compilation strategies Execution & Answer API control, observation, parameterization OO & Web Service API Program Plane Parsing & Validation Layer program parsing and validation multi-parser, normalization, modules Compilation Layer unsatisfiable, tautological parts extensive query optimization Execution Layer (AMAχOS) pattern matching engine rule dispatcher and engine Data Plane Schema Access Layer provides access to schema of data type checking for compilation Data Access Layer incremental data access storage and indexing engine Serialization Layer incremental answer creation versatile Web format support 18

19 AMAχOS Architecture core layer: execution or AMAχOS proper Query Compilation Dependency Hints Abstract Machine AMAχOS Rule Engine Rule Dispatch Function Call Pattern Matching Engine Rule Call (Recursion) Static Function Library Construction Engine Answer API Abstract Machine Code Code Scheduler Memoization Matrix Variable Node Sub-Matrix v 5 d 2 Variable Node Sub-Matrix Substitution Sets Answer Construction In-Memory Answer AM Code v 4 d 3 Variable Node Sub-Matrix v 1 d 6 v 1 d 7 Hint Segment Dependency Segment Code Segment rule 1 rule 2 Storage & Index Hints Storage Manager v 4 d 5 Variable Node Sub-Matrix v 3 d 11 v 2 d 13 Runtime Data Access Layer 19

20 AMAχOS Architecture The other core: optimization and compilation Typed AST Query Compilation Logical Optimization Algebraic Optimization Translation logical algebra patterns: annotated conjunctive queries over semi-structured graphs rules: unfolding into complex value or object algebra where possible Translator Canonic Logical Query Plan x Child + s v Root Child y Child Rewriting system elimination of dead and tautological query parts join placement optimization query compaction (common subexpressions) Rewriting System r w Child + z x w π x! s(x,w) s π w! r(y,z) y w z Optimized Logical QP Physical Plan Generation Query Classification determines class of query, e.g., to choose efficient alg. for sub-languages Operator Algorithm Selection determines realization of operators Index and Storage Model Selection selects in-memory representation and indices for data access Physical Query Plan Code Generation generate AM-code direct representation of physical query plan platform-independent motion of invariant code dead-code elimination Code Generator AM Code 20

21 The end (of the talk) novel approach to query execution uniform platform for distributed evaluation separation of querying and compilation lots of open issues, e.g., data structures compilation & evaluation of high-level language constructs Compile once Execute anywhere Optimize all the time 21