The Reactor Model: An Experiment in Declarative Concurrent Programming

Size: px

Start display at page:

Download "The Reactor Model: An Experiment in Declarative Concurrent Programming"

Richard Hines
6 years ago
Views:

1 The Reactor Model: An Experiment in Declarative Concurrent Programming *John Field (IBM Research) Maria-Cristina Marinescu (U. Carlos III, Madrid) Christian Stefansen (Google) Datalog 2.0, Oxford 17 March 2010

2 Do these apps have anything in common? cloud-based web 2.0 embedded network real-time data analysis

3 Yes. collection of distributed, concurrent components components are loosely coupled by messages, persistent data irregular concurrency, driven by realworld data ( reactive ) high data volumes fault-tolerance important

4 Why this is hard: an AJAX parable Zip Database City ZIP code Submit State Zip Lookup Servlet Merchant Credit Server Credit Card Number Submit Form + JScript Code Form Submission Servlet User Credit Servers

Web programming: state of the art requires extensive rewriting when moving applications between tiers or updating functionalty; doesn t compose, scale well babble of languages same logical data; many

dtd"> var xmlhttp; <html xmlns="http://www.w3.org/1999/xhtml" > <?

$param="; */ // The server-side script } catch (e) { function handlehttpresponse() { function db_connect() { try { if (http.$ $readystate == 4) { $database_name = 'mysql'; // Set this to your Database Name xmlhttp = new ActiveXObject("Microsoft.XMLHTTP"); if (http.responsetext.$ to your MySQL password xmlhttp = false; var xmldocument = http.

to your MySQL password xmlhttp = false; var xmldocument = http.

firstchild.data; # if (!$result) return false; } var state = xmldocument.getelementsbytagname('state').item(0).firstchild.data; if (!mysql_select_db($database_name)) return false; @else document.

5 Web programming: state of the art requires extensive rewriting when moving applications between tiers or updating functionalty; doesn t compose, scale well babble of languages same logical data; many different physical representations <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" " function gethttpobject() { xhtml1/dtd/xhtml1-strict.dtd"> var xmlhttp; <html xmlns=" > <?php /*@cc_on <head> (@_jscript_version >= 5) <title>zip Code to City and State * using Connects XmlHttpRequest</title> to the database. try { <script language="javascript" type="text/javascript"> * Return false if connection failed. xmlhttp = new ActiveXObject("Msxml2.XMLHTTP"); var url = "getcitystate.php?param="; */ // The server-side script } catch (e) { function handlehttpresponse() { function db_connect() { try { if (http.readystate == 4) { $database_name = 'mysql'; // Set this to your Database Name xmlhttp = new ActiveXObject("Microsoft.XMLHTTP"); if (http.responsetext.indexof('invalid') == -1) { $database_username = 'root'; // Set this to your MySQL username } catch (E) { // Use the XML DOM to unpack the city and state data # $database_password = ''; // Set this to your MySQL password xmlhttp = false; var xmldocument = http.responsexml; # Table structure for table `zipcodes` $result = mysql_pconnect('localhost',$database_username, $database_password); } var city = xmldocument.getelementsbytagname('city').item(0).firstchild.data; # if (!$result) return false; } var state = xmldocument.getelementsbytagname('state').item(0).firstchild.data; if (!mysql_select_db($database_name)) return document.getelementbyid('city').value = city; CREATE TABLE `zipcodes` ( return $result; xmlhttp = false; document.getelementbyid('state').value = state; `zipcode` mediumint(9) NOT '0', } isworking = false; $conn = db_connect(); // Connect to `city` database tinytext NOT NULL, if (!xmlhttp && typeof XMLHttpRequest!= 'undefined') { } if ($conn) { `state` char(2) NOT NULL default '', try { } $zipcode = $_GET['param']; // The `areacode` parameter smallint(6) passed to us NOT NULL default xmlhttp '0', = new XMLHttpRequest(); } $query = "select * from zipcodes PRIMARY where zipcode KEY (`zipcode`), = '$zipcode'"; xmlhttp.overridemimetype("text/xml"); var isworking = false; $result = mysql_query($query,$conn); UNIQUE KEY `zipcode_2` (`zipcode`), } catch (e) { function updatecitystate() { $count = mysql_num_rows($result); KEY `zipcode` (`zipcode`) xmlhttp = false; if (!isworking && http) { if ($count > 0) { ) TYPE=MyISAM; } var zipvalue = document.getelementbyid("zip").value; $city = mysql_result($result,0,'city'); } http.open("get", url + escape(zipvalue), true); $state = mysql_result($result,0,'state'); return xmlhttp; http.onreadystatechange = handlehttpresponse; } } isworking no = true; notion of } var http = gethttpobject(); // We create the HTTP Object http.send(null); if (isset($city) && isset($state)) { </script> } component // $return_value = $city. ",". $state; </head> } $return_value = '<?xml version="1.0" standalone="yes"?><zip><city>'.$city.'</city><state>'.$state.'</state></zip>'; <body> } <form action="post"> interface for fragile plumbing else { <p> $return_value = "invalid".",".$_get['param']; // Include Zip for debugging ZIP code: purposes composition } <input type="text" size="5" name="zip" id="zip" onblur="updatecitystate();" /> header('content-type: text/xml'); </p> echo $return_value; // This will become the response value for the XMLHttpRequest City: object?>6 <input type="text" name="city" id="city" /> State: <input type="text" size="2" name="state" id="state" /> </form> </body> </html> required to compose

6 A prettier picture: app composed from encapsulated, distributed components FormController ZipController DB InputWidget (city) InputWidget (card) SessionController Acct (merchant) InputWidget (state) ButtonWidget (next) InputWidget (zip) Acct (user) Acct ButtonWidget (submit) Acct

7 Web apps just one instance of reactive distributed applications have distributed control integrate front-end, back-end, and database data typically combine presentation logic, business logic, data query and access functionality can evolve and migrate between tiers dynamic instantiation of logical processes complex failure modes

8 Our (long-term) goals develop programming models for reactive distributed applications which have same model for presentation, business logic, data access simplify composition, evolution, and application maintenance target applications: web applications, distributed business applications, software supply chains this talk: initial experiments on a highly declarative model for this class of application some ideas have been incorporated in more recent work on a new scripting language for distributed applications (Thorn)

9 Reactors: key attributes Synthesis of ideas from synchronous languages: event handling, synchronization datalog: expressive data query language NB: here we are using datalog for computation, not just query actor model: simple model for dynamic creation of processes, asynchronous process interaction transactions: system evolves (only) from consistent state to consistent state concurrent computations isolated from one another XForms: declarative specification of web forms

10 Basic reactor: database with external updates Δ + : (1, 5667, 2) Δ - : (1, 2344, 4) Δ + : (1, 5667, 2) Δ + : (5, 9364, 2) (1, 1277, 3) Δ - : (1, 2344, 4) (1, 1277, 3) Δ + : (1, 5667, 2) (1, 1277, 3) update bundle Δ + : additions Δ - : deletions (3, 7623, 7) (1, 2344, 4) s 0 (3, 7623, 7) (1, 2344, 4) (5, 9634, 2) s 1 (3, 7623, 7) (5, 9634, 2) s 2 time reactor encapsulates state in the form of fixed set of relations (sets of tuples) each update is a bundle of additions (Δ + ) and deletions (Δ - ), applied atomically Δ + and Δ - must be disjoint state updates arrive asynchronously, and are queued in most basic form of reaction, updates simply applied to previous state (s i-1 ) to yield new state (s i )

11 Inside a reaction: stimulus and response stimulus state response state (1, 1277, 3) Δ + : (5, 9364, 2) (1, 1277, 3) (1, 1277, 3) (3, 7623, 7) (3, 7623, 7) (3, 7623, 7) (1, 2344, 4) (1, 2344, 4) (1, 2344, 4) (5, 9634, 2) (5, 9634, 2) S i-1 ^s i s i ^s i+1 Mi-1 Mi Mi+1 time s i (response state): observable state of reaction i ^s i (stimulus state): result of applying update to s i -1 ( pre-state ) in the absence of any computational rules, s i = ^s i

12 Adding computation via rules -r ^r r ^s 0 s 0 ^s 1 s 1 ^s 2 s 2 rules...-r......^r......r... reaction s computation defined using datalog-style rules rules define value of each relation r, s,... as function of r, -s,... (pre-state values: values of r, s,... in previous reaction) ^r, ^s,... (stimulus values: result of applying update bundle to r, -s,...) r, s,... (can appear recursively in rules) reactor computation (reaction) is atomic: intermediate states of rule evaluation not visible to other reactors

13 only public relations can be updated from outside Reactor rule evaluation public orders: (int, int, int). log: (int, int, int). public ephemeral updatelog: (). contents of ephemeral relations do not persist between reactions public ephemeral relations can be used to model events log(orderid, itemid, qty) <- orders(orderid, itemid, qty). qty), updatelog(). not log(orderid, itemid, qty) <- not orders(orderid, itemid, qty). LHS negation denotes deletion rules evaluated whenever state is updated no change made to relation unless necessary to satisfy a rule semantics ensures unique solution rule declaration order irrelevant easy to compose rules to yield new functionality without concern for data or control dependencies ( aspect -like) extension of standard datalog semantics as usual, negation requires care

14 Multiple reactors R 1 : R 2 : R 3 : reactor s computation can generate new state updates for other reactors or the same reactor and can dynamically instantiate new reactors

15 Reactor references DB rdatabase: (ref DB).... rdatabase(new) <-... db.order(...) <- rdatabase(db), relations may contain references to other reactors reactors may be instantiated dynamically rules may refer to relations of remote reactors via references

16 Asynchronous interaction via future state (r^) -r ^r r r^ ^s 0 s 0 ^s 1 s 1 ^s 2 s 2...-r......^r......r......r^... future state value: r^ r^ defines asynchronous state updates (to this or other reactors via references) rules compute r^ as function of r, -r, ^r

17 Running example: order entry application asynchronous interaction synchronous interaction City ZIP code Next State Zip Lookup Servlet Zip Database synchronous interaction Merchant Credit Server Form + JScript Code Credit Card Number Submit synchronous interaction Form + JScript Code asynchronous interaction Form Submission Servlet User Credit Servers

18 Running example: order entry application City State ZIP code Submit Form + JScript Code Form Submission Servlet

19 Order entry: validation and FormController submission reactor references for composition InputWidget (city) Δ + SessionController InputWidget (state) Δ + Δ + Δ + InputWidget (zip) asynchronous request from FormController Δ + ButtonWidget (submit) asynchronous user inputs

references to UI widgets and session server FormController Order entry: validation and rcity: (ref InputWidget). rstate: (ref InputWidget). rzip: (ref InputWidget). rsubmit: (ref ButtonWidget).

20 references to UI widgets and session server FormController Order entry: validation and rcity: (ref InputWidget). rstate: (ref InputWidget). rzip: (ref InputWidget). rsubmit: (ref ButtonWidget). rsession: (ref SessionController). submission (private) ephemeral relation holds result of validation for current reaction only ButtonWidget (submit) nullary ephemeral relation used to model UI events ephemeral validated: (string, string, string). public ephemeral buttonpressed: () validated(z, c, s) <- zw.val(z), cw.val(c), sw.val(s), z <> "", c <> "", s <> "", rzip(zw), rcity(cw), rstate(sw). sc.commitorder^(z, c, s) <- buttonpressed(sb), validated(z, c, s), rsubmit(sb), rsession(sc). SessionController... public ephemeral commitorder: (string, string, string).... references to relations of remote reactors set of listeners that are notified when button pressed reference to future state of remote reference: spawns asynchronous reaction public label: (string). public listeners: (ref FormController) public ephemeral pressed: (). rf,buttonpressed(self) <- listeners(rf), pressed(). InputWidget (zip) public label: (string). public val: label: (string). (string). public val: label: (string). (string). public val: (string). public ephemeral filled: (). public ephemeral filled: (). public ephemeral filled: ()

21 Running example: order entry application Zip Database City ZIP code Next State Zip Lookup Servlet Form + JScript Code Form Submission Servlet

22 Order entry: adding AJAX -style asynchronous response from AjaxController asynchronous request from FormController zip lookup database accessed synchronously with receipt of request Δ + FormController Δ + AjaxController DB InputWidget (city) SessionController InputWidget (state) InputWidget (zip) ButtonWidget (submit) widgets updated synchronously with response

23 Order entry: adding AJAX -style FormController... rajax: (ref AjaxController). zip lookup public ephemeral formfilled: () public ephemeral lookupresult: (string, string). zc.lookupreq^(z, self) <- formfilled(zw), zw.val(z), rzip(zw), rajax(ac). cw.val(c), sw.val(s) <- loookupresult(c, s), rcity(cw), rstate(sw).... InputWidget (zip) when zip field is filled, asynchronous lookup public label: (string). request made public to val: label: (string). (string). AjaxController public val: label: (string). (string). public val: (string). public ephemeral filled: (). public ephemeral filled: (). public ephemeral filled: () AjaxController rdatabase: (ref DB). public ephemeral lookupreq: (string, ref FormController). when response received, widget fields are updated fc.lookupresult^(c, s) <- lookupreq(z, fc), db(z, c, s), rdatabase(db). DB response sent asynchronously after DB lookup public zipcitystate: (string, string, string).

24 Running example: order entry application Zip Database City ZIP code Next State Zip Lookup Servlet Merchant Credit Server Credit Card Number Submit Form + JScript Code Form Submission Servlet User Credit Servers

25 Order entry: dynamic widget instantiation FormController ZipController DB InputWidget (city) InputWidget (card) SessionController InputWidget (state) ButtonWidget (next) InputWidget (zip) ButtonWidget (submit) dynamically instantiate new widgets when next button pressed

26 Order entry: dynamic widget instantiation FormController new widgets instantiated when next button pressed... rnext: (ref ButtonWidget) rcard: (ref InputWidget). rsubmit: (ref ButtonWidget). old widget reference removed... City State rcard(new), rsubmit(new), not rnext(_) <- buttonpressed(nb), validated(_,_,_), -rnext(nb). sc.commitorder^(z, c, s, cc) <- buttonpressed(sb), rsubmit(sb) validated(z, c, s), cw.val(cc), cc <> "", rcard(cw) rsession(sc). NB: must refer to prestate to ensure stratification ZIP code Next Credit Card Number Submit Form + JScript Code... commitorder now validates and sends credit card info

27 Order entry: adding synchronous / atomic account transfer FormController ZipController DB InputWidget (city) InputWidget (card) SessionController Acct (merchant) InputWidget (state) ButtonWidget (next) InputWidget (zip) ButtonWidget (submit) transfer between user and merchant accts must occur synchronously and atomically Acct (user) Acct Acct

28 Order entry: adding synchronous / atomic account transfer SessionController Acct (merchant)... cardaccts: (ref Acct, string). rmerchantacct: (ref Acct). ordertotal: (int). ephemeral ruseracct: (ref Acct). ruseracct(ua) <- commitorder(_,_,_,cc), cardaccts(ua, cc). ua.balance(b - amt) := ua.-balance(b), ordertotal(amt), commitorder(_,_,_,_), ruseracct(ua). ma.balance(b + amt) := ma.-balance(b), ordertotal(amt), commitorder(_,_,_,_), rmerchantacct(ma). syntactic sugar for asserting new value; retracting old value for singleton cell predicate FAIL is syntactic sugar for a predefined inconsistency, hence can be used to define consistency constraints public balance: (int). FAIL <- balance(x), balance(y), x <> y. FAIL <- balance(x), x < 0. Acct (user) Acct (user) examples of inconsistent rules: r(...) <-. not r(...) <-. such rules are mutually unsatisfiable public balance: (int). public balance: (int). public balance: (int). FAIL <- balance(x), balance(y), FAIL x <> <- y. balance(x), balance(y), FAIL FAIL x <- <> balance(x), <- y. balance(x), x < balance(y), 0. FAIL x <- <> balance(x), y. x < 0. FAIL <- balance(x), x < 0. reactions that yield inconsistency roll back to pre-state

29 Reactor semantics: some issues (I) Handling reactor references: given reference of form c.r(x) <-... rewrite as r (c, x) <-... Head negation: compute positive and negative information separately explicitly check for consistency before committing result translate rules to datalog with goal (rhs) negation only Rules required to be stratified to ensure unique solution stratification check must account for remote references

30 Reactor semantics: some issues (II) Scope extrusion semantics for composite synchronous reactions defines which reactor instances are affected scope of reaction extrudes to reactor R whenever another reaction attempts to write to response state of relation of R Optimistic concurrency control composite reactions operate on snapshots of reactor state reaction rolls back, update re-queued if states of reactors have changed prior to commitment Reactor instantiation: in each reaction... for each new instance in rule head......generate a globally unique reference for each combination of values bound to variables in the rules modeled using functors

31 Related work actor model transactional languages (Argus, Camelot) synchronous languages (Esterel, Lustre, Signal,...) concurrent constraint programming process calculi (Pi calculus, Join calculus, Ambient calculus,...) tierless web programming models (Links, Hop, Google Web Toolkit...) stream-oriented web programming models (Flapjax,...) modern relational databases (triggers, constraints,...) various web services standards event-condition-action rules Axml <your missing reference goes here>

32 Review data, rather than ports or channels as interface to process expressive data query and transformation declarative, data-driven, compositional specification of functionality in an ``aspect-like'' manner constraints/assertions expressible in core language synchronous and asynchronous interaction in same model simple transactional model dynamic process creation

33 Details Theoretical Computer Science 09

34 Questions?

35 Head Negation public orders: (int, int, int). log: (int, int, int) log(id, itemid, qty) <- orders(id, itemid, qty). not log(id, itemid, qty) <- not orders(id, itemid, qty). transforms to (roughly) public orders: (int, int, int). log: (int, int, int) ephemeral ordersδ + : (int, int, int). ephemeral ordersδ - : (int, int, int). ephemeral logδ + : (int, int, int). ephemeral logδ - : (int, int, int). logδ + (id, itemid, qty) <- ^orders(id, itemid, qty), ordersδ + (id, itemid, qty). logδ - (id, itemid, qty) <- not ^orders(id, itemid, qty), ^log(id, itemid, qty), not ordersδ + (id, itemid, qty). if (logδ + logδ = ) then log := ^log logδ + \ logδ +

36 Reactor syntax REACTOR ::= def reactor-type-name = { {DECL.}* } ENUM ::= enum enum-type-name = { {atom-name,}+ } DECL ::= ( RELATION-DECL RULE-DECL ). RELATION-DECL ::= [public public write public read] [ephemeral] rel-name : ( {TYPE,}* ) RULE-DECL ::= HEAD-CLAUSE <- {BODY-CLAUSE,}+ HEAD-CLAUSE ::= [not] [var-name.]rel-name[^] ( {(_ var-name),}* ) BODY-CLAUSE ::= [not][var-name.][^ -]rel-name ( {(_ var-name),}* ) BASIC-PREDICATE BASIC-PREDICATE ::= EXP (< > <> =) EXP TYPE ::= int string enum-type-name ref reactor-type-name EXP ::= var-name NUMERIC-LITERAL STRING-LITERAL EXP (+ - * / %) EXP new reactor-name self

Thorn: From Scripting to Robust Concurrent Components

Thorn: From Scripting to Robust Concurrent Components IBM Research Bard Bloom John Field* Nate Nystrom Purdue Brian Burg Johan Östlund Gregor Richards Jan Vitek Tobias Wrigstad Cambridge Rok Strniša JAOO