ScalableTrigger Processing

Transcription

1 ScalableTrigger Processing February 11 Eric Hanson, Chris Carnes, Lan Huang, Mohan Konyala, Lloyd Noronha, Sashi Parthasarathy, J. B. Park and Albert Vernon

2 References: Decomposition-Based Approach To Join Trigger Processing In TriggerMan Sashi Kumar Parthasarasy Scalable Trigger Processing in TriggerMan.

3 Outline: Motivations Introduction to TriggerMan Architecture Expression Signatures Selection Predicate Indexing Conclusions

4 Motivations: Trigger Scalability How well a solution to some problem will work when the size of the problem increases. Current Databases systems have limited trigger scalability. Streams Coping with large numbers of queries Coping with large numbers of triggers

5 Introduction Commercial Database System Integrity Constraints Limited Trigger Scalability Synchronous Processing TriggerMan Scalable Trigger System Asynchronous Trigger Processing Sophisticated Predicate Index Extension Module for Object Related DB

6 Command Language: Keyword-Delimited SQL-like Syntax Single Default Connection to Informix Database Multiple Data Sources (correspond to tables) Macro Substitution Clauses: From, on, when, group by, having

7 Example Trigger: Consider a database containing a movie table, where you want to look up and change the value of the profit a movie made. create trigger updatematrix2 from movie on update(movie.profit) when movie.name= Matrix2 do execsql update movie set profit=:new.movie.profit where movie.name= HarryPotter

8 General Trigger Structure: create trigger <triggername> [in setname] [optionalflags] from fromlist [on eventspec] [when condition] [group by attributelist] [having groupconnection] do action

9 General Trigger Condition: FROM clause: the data sources ON clause: event condition For at most one data source in from clause WHEN clause: boolean-valued expression Canonical representation (CNF) Number of data sources: Selection predicate, join predicate, hyper-join predicate

10 System Architecture Visual TriggerMan Host DBMS: Informix w/ Universal Data Option Data Source Trigger Driver Data Source App Update Queue Table Console Data Source App Update Queue in Shared Memory TriggerMan DataBlade Client App Client App

11 System Architecture: TriggerMan DataBlade - lives inside Informix DataSources - local or remote tables TriggerMan client applications - create, drop triggers and register for events TriggerMan driver - invokes TmanTest(), allowing condition testing and action execution TriggerMan console - lets the user interact with system Update Queue - stores updated tuples

12 Scalable Predicate Indexing: Tuple variable - symbol, in FROM clause that corresponds to usage of data source in the trigger Data Source ID - per data source (assumed) Expression Signature - Triple of the form <datasourceid, operation code, generalized expression> Defines equivalence class Syntax tree - representation of generalized expression in expression signature.

13 Condition Representation Expression Signature Syntax Tree On Insert to Movie When movie.profit > DataSourse: Movie Event: Insert > movie.profit CONSTANT

14 Trigger Data Structures Updated at Trigger Creation Trigger Cache - located in Main Memory, contains the set of recently accessed triggers Predicate Index - located in Main Memory, takes an update descriptor and identifies predicates that match it

15 Predicate Index Structure Predicate Index Root Data Source Predicate Indexes Expression Signature List Trigger ID Set Constant Table

16 Creating a Trigger: Update the trigger system catalogs and main memory data structures Parse and validate Convert WHEN to CNF, group by tuple variables Form Trigger condition graph Build A-TREAT (discrimination network) for the rule Check the signatures Integrity constraints

17 Expression Signatures and Selection Predicates: Expression Signature - defines equivalence class Alternative Indexing for expression signatures main memory list or index, non-indexed or indexed database table Selection Predicate sub-expression elimination enhances performance, reduced response time, achieved by normalizing predicate index structure Processing Update descriptors (tokens) using predicate index

18 Recall Telegraph: Group Indexing Group Indexing > < =

19 Concurrent Processing: Improve Trigger Scalability Concurrent tasks execute in parallel Improves throughput, response time Four Types: Token-level - selection predicate index join condition-testing networks Condition-level - testing against single token Rule action - processing multiple rules at the same time Data-level - data values processed by parallel queries

20 Recall Niagara: Continuous queries with same structure.

21 Trigger Application Design: Just because you can create a large number of triggers, does not always mean it is the best idea. May be best to create single trigger and modify it for data if trigger structure very similar Scalable Trigger Processing in TriggerMan by E. Hanson

22 Related Work: Large work on Active Databases Little on Trigger Scalability The Postgres rule system ECA - event-condition-action interval skip-list data COND relations

23 Conclusion and Future Research: Truly Scalable Trigger System capable to accept large number of triggers handles high rate of data updates reduces the response time using asynchronous processing Future Research Temporal trigger processing Aggregates in trigger conditions Main-memory / disk-based structures to organize constant sets

24 What to remember Stream connection - stream of data in active DBMS Overview of TriggerMan

25 Discussion: Did the problems triggers generated cause development of streaming databases? Does lack of support for handcrafted triggers call for trigger wizards?

26 Discussion: Streams: Stream to Stream Operators Faster Response Time? Better Scalability? Triggers: Table to Table Operators Maintains a table, allows access to "history" More grounded in traditional relational research

27 Discussion: Proving the Scalability of the TriggerMan system How valid is their claim? How would you go about proving it or changing it?