Long running and distributed transactions. TJTSE54 spring 2009 Ville Seppänen

Transcription

1 Long running and distributed transactions TJTSE54 spring 2009 Ville Seppänen

2 Forthcoming lecture dates? For the next two meetings, the course page says 21 th and 28 th which are Tuesdays. Tuesday 21 th is not ok with me. Should we change to Thursday 23 th and 30 th or something else? Info on hands-ons next week.

3 Agenda Transactions Long-running and distributed 2PC Nested Saga Transactional processes

4 Transaction Single unit of work composed of two or more tasks Transaction is a series of logically interconnected operations that change a state of a system according to a set of rules Transaction T = operations t 1,, t n

5 Transaction T = Transfer 100 from an account A to an account B t 1 read the balance from account A to AB t 2 writen AB = AB 100 t 3 read the balance from account B to BB t 4 write BB = BB T = Selling something via online store t 1 Receive a purchase order t 2 Order items from the supplier T 3 Wrap up the products T 4 Prepare an invoice t n

6 ACID Theo Häerder and Andreas Reuter 1983, Principles of Transaction- Orientated Database Recovery Ideally, all kinds of transactions would follow the database transactions ACID properties (atomicity, consistency, isolation, durability) In practice, this is not always reasonable or feasible

7 Atomicity Changes that transaction (T) directs to the state of a system are committed only when each operation of T is successfully executed. Otherwise T is aborted Should there be a failure, all changes already performed are cancelled (rollback, backward error recovery) This means that the original state of a system is restored E.g., operations are logged

8 Consistency Changes to the state of a system must correspond to the (actual) changes that a system represents Further, changes must happen in accordance with the integrity constraints of the system Integrity constraints define what states are allowed and what are not During most transactions, a system will temporarily be in an inconsistent state. Therefore, integrity is usually checked only when T requests commit (deferred integrity constraints)

9 Consistency Consiste nt state n ACID Consiste nt state n+1 ACID Consiste nt state n+2 Read(A) Write(A, (A-10)) Read(B) Write(B, (B+10)) Temporary inconsistence

10 Isolation Simultaneously running transactions must be isolated from each others This means that they must not see temporarily inconsistent states of other transactions Would lead to faulty results and cascading rollback This is guaranteed with serializable execution schedule The results are comparable to those of serial execution There are several ways to implement the isolation, e.g., time-stamping, locking, optimistic approach One should select a suitable means according to the application

11 Isolation S erial execution Read(A) Write(A, (A+10)) Commit Read(A) Write(A, (A+10)) Commit Update loss Read(A) Read(A) Write(A, (A+10)) Commit Write(A, (A+10)) Commit Dirty read Read(A) Write(A, (A+10)) Read(A) Write(A, (A+10)) Commit Abort

12 Durability Once a transaction is notified of success (and is thus committed) it will persist i.e., it is not undone due to, for example, failed integrity and cascading rollback or due to technical failure

13 What has changed since then? Traditionally, distributed ACID transactions have been executed using connectionoriented and synchronous protocols and typically the running time spans from microseconds to seconds Nowadays, transactions are executed over connectionless protocols (e.g., HTTP) using asynchronous communications (e.g., MOM) and may run hours or days (e.g., a business process)

14 Long living (running) transaction Has a long duration compared to the majority of other transactions either because It accesses many database objects It has lengthy computations It pauses for inputs from the users (or processes) Or a combination of these factors About the length of time of execution, not size of a transaction

15 Distributed transaction management Subtransactions withdraw_part return_part stock_level DB tx_begin; order_part; withdraw_part; payment; tx_commit; Transaction coordinator Inventory application Local applications Resource managers order_part billing Billing application Site 1 DBMS DB Site 2

16 2-Phase Commit protocol Atomic commitment protocol An algorithm that ensures that all the processes involved in a distributed transaction either commit or abort Coordinator process Responsible for controlling the overall atomicity Controls the 'voting A number of participating processes Execute parts of a distributed transaction

17 2-Phase Commit protocol Voting procedure phase one The coordinator sends vote requests to all participants When a participant receives the vote request it replies by voting either Yes or No, according to whether it is able to carry out the task Participants that voted Yes start waiting for a confirmation message from the coordinator Participants that voted No can unilaterally abort

18 2-Phase Commit protocol Voting procedure phase two The coordinator collects all the vote messages If all the participants voted Yes the coordinator decides to commit and sends C ommit messages to all participants Otherwise, the coordinator decides to abort and sends Abort messages to all participants that voted Yes According to the received message, participants decide to commit or abort

19 2-Phase Commit protocol It is possible that messages may not arrive due to a failure and processes may be waiting forever: a timeout mechanism must be able to interrupt the waiting period In addition, the coordinator may attach the list of participants to the vote request message and thereby let the participants to know each other: Cooperative termination protocol

20 Cooperative Termination protocol If participant p comes across the timeout while waiting for the Commit or Abort message from the coordinator, it can requrest this from the participant q If q has already decided to commit (or abort) it sends a C ommit (or Abort) to p In the case that q has not voted yet it can decide to abort and send Abort to p If q has voted Yes but has not received the final Commit or Abort request from the coordinator it cannot help p in making the decision

21 Nested transactions A mechanism to facilitate transactions in distributed systems A tree-like model with parents, children, top- level (root), and leaves T 0 Top-level Parent T 1 T 2 Child Leaf T 3 T 4

22 Nested transactions Rules A parent can spawn any number of children Any number of children may be active concurrently Parent can't access data when its children are alive A child can inherit a lock held by any ancestor On child commit, its lock are inherited (anti-inheritance) by the parent

23 Nested transactions Rules, continued Commit dependency: parent can commit only after all its children terminate (commit/abort) Abort dependency: on parent abort, updates of committed children are undone Updates persist only if all ancestors commit

24 Nested transactions void send_all_salaries() { Transaction tx = new Transaction(); tx.begin(); for(int i=0; i<emplcount; i++ ) { send_salary(employeraccount, employeeaccount[i], amount[i]); } tx.commit(); } void send_salary(from, to, amount) { Transaction tx = new Transaction(); tx.begin(); from.remove(amount); to.add(amount); tx.commit(); }

25 Nested transactions Intra-transactional parallelism Safe concurrency Reduced response time Intra-transactional recovery control Finer control on error handling Uncommitted subtransactions can be rolled back without affecting the other subtransactions Improves availability System modularity Composition of separately developed modules

26 Save-point A check point in a transaction that forces system to save the state of the running application and return a save-point identifier for future references Instead of removing the entire transaction after a failure of single operation (sub-process) backward recovery can return the last valid state of the transaction saved in the save-point reference

27 Saga transactions Garcia-Molina & Salem, 1987 Long-living and distributed transactions are not reasonable to implement as single ACID transactions Keep resources locked for the long periods of time, which can significantly delay the commit of other simultaneously executed transactions Deadlock frequency grows with the fourth power of the transaction size

28 Saga transactions Atomic commitment protocols ease the implementation of ACID transactions in distributed systems but further transactions to become long living Backward recovery is expensive and difficult to implement in distributed environments The idea of compensating transactions (or operations, processes) was introduced to simulate the transactional properties in such applications

29 Saga transactions In Saga transaction model a long living transaction is broken up into a collection of subtransactions that interleave with other transactions The results of a subtransaction can be made immediately visible once it has committed

30 Saga transactions Saga subtransactions can be executed independently but they must finally form an atomic unit Should any of subtransactions fail the failed transaction is aborted and already committed subtransactions are undone Removing a committed transaction means that results of subsequent transactions may become inconsistent and cause cascading rollbacks Instead of removing the results of failed transaction by using rollback, the idea of compensating transaction is introduced

31 Saga transactions Each substransaction is a real transaction in the sense that it preserves the consistency However, they are related to each other and any partial executions of the saga are undesirable; if such occurs the compensation takes place

32 Saga transactions Each saga substrabsaction T i is provided with a compensating transaction C i If the compensation is not needed saga T 1, T 2,..., T n will executed as a sequence of transactions The compensating transaction undoes, from a semantic point of view, any of the actions performed by the transaction but doesn't necessarily return the system to the state that existed when the execution of transaction began

33 Saga transactions For the transaction T 1, T 2,..., T n the compensating transaction C 1, C 2, C n-1 is defined In case of compensation the sequence executed is T 1, T 2,..., T j, C j,..., C 2, C 1 ; where 0 <= j < n

34 Saga transactions Contrary to traditional backward recovery mechanism, which is based on compensating operations automatically deduced from the schedule, semantically compensating transactions may not be automatically generated Even simple compensating actions (such as + & - operations) must follow the integrity constraints Everything cannot be undone For example, 'real actions' with 'real consequences': impossible to undo but it may be possible to override the effect with a new action Committed acceptable & aborted acceptable termination states: 'business considerations'

35 Forward recovery Combining the save-points, compensation and recovery of failed transaction Transaction that caused the failure is aborted using the conventional rollback Committed transactions are undone in reversed order using their compensating counterparts until the save-point is found (backward recovery) Finally, the transaction is restarted from the location of save-point (forward recovery)

36 Forward recovery C F G A B D E H I Execution graph (arrow) and compensation graph (dashed arrow)

37 Forward recovery When recovering, it's not always feasible or even possible to complete the transaction by re- executing the original steps again However, it's possible that the same objective can be achieved with different ways. Alternative methods of forward recovery should be considered during the process design

38 Transactional processes cf. Grefen et al. 1997, 1999 Typical scenario: a top-level process consists of subcomponents that are logically connected to the higher level process but execute virtually independently A global transaction A collage of all separate transactions that are included in the process Often long-living and distributed Each global transaction can be divided into a set of local transactions Can be represented as a tree of hierarchically ordered tasks (or a graph), which may also include manual operations

39 Transactional processes G lobal trans action s tructure (top-level proces s) globally ACID (atomic commitment), distributed, long-living Sales Book Invoice Payment Cancel Prepare documents Send documents Write booking Booking app Transportation Accommodation Booking app S ubproces s es Local trans actions, locally ACID

40 Transactional processes Apply save-points on top-level process workflow. Provide with compensating processes. Globally ACID Sales Book Invoice Payment Cancel Prepare documents Send documents ubprocesses may xecute nested, Saga, r some other model epending on he task Local transactions Atomic commitment Booking app Booking app DB-level transactions managed by DB

41 Long running transaction, a real-life example Day 1. (Saturday) Made an order on Hand Cannon (a revolver look-a-like game controller for Wii) on Play.com webstore. Used VISA for payment. Day 1. Received a purchase confirmation via . Day 3. Order posted, expected delivery time 3-5 days. Day 15. Haven't got the cannon yet. Went to Play.com and clicked I have not received my purchase. Play.com told me to allow 21 days for delayed delivery. Day 23. Clicked I have not received my purchase again. Play.com asked me would I like to have a replacing product shipped to me or a reimbursement of price to my VISA. I chose the first alternative. Day 23. Got home, checked mail and found a notice of package arrival. Day 24. Received a polite and regretful from Play.com in which they told me that Hand Cannon was a limited edition product, which was now sold out, so they had reimbursed my VISA instead. Day 24. Went to the post office to pick up the package and it contained my Hand Cannon (The package had a large red sticker that said Do not fly! The screening probably revealed that the package contained something that resembled a gun and therefore it was not allowed the air-delivery? That explained protracted delivery.) I got the product AND my money back. How would the transaction continue or would it?

42 Tradeoff between process integrity and implementation cost Level of integrity 2PC/TPM in homogeneous environment 2PC/TPM in heterogeneous environment SAGAs with complex compensation graphs BPM engine with simple compensation graphs Transactional steps Custom-build distributed log-infrastructure DB-Log Analyzer & System Management Platform Eyes closed and praying Krafzig, Banke & Slama 2005 Implementation cost