Rigorous Design of Moving Sequencer Atomic Broadcast with Unicast Broadcast

Transcription

1 Proc. of Int. Conf. on Advances in Computer Science, AETACS Rigorous Design of Moving Sequencer Atomic Broadcast with Unicast Broadcast Prateek Srivastava a*, Kamaljit I. Lakhtaria b and Amit Jain c a, b, c, Department of Computer Science and Engineering, Sir Padampat Singhania University,Udaipur, Rajasthan, , India prateekabhi2004@gmail.com Abstract This article investigates a novel mechanism of atomic broadcast in distributed systems. Various mechanisms are already given for moving sequencer based atomic broadcast like RMP [1], DTP [2] and Pin Wheel [3]. Since all these mechanisms are built on broadcast broadcast (BB) variant [4] of fixed sequencer atomic broadcast hence introduce a very large number of messages (2*(n-1), where n is total number of correct processes in system). Here we propose a novel mechanism that works on unicast broadcast (UB) invariant of fixed sequencer atomic broadcast in order to build moving sequencer atomic broadcast. Since the proposed mechanism relies on unicast broadcast hence it will introduce very less number of messages (n messages, where n is total number of correct processes in system) in comparison to previous mechanisms. The proposed work will reduce the load on sequencer and also provide an opportunity to each process to be a sequencer in a cyclic manner. We have used Event B [5] as formal technique for development of our model. Event B uses set theory as a modeling notation, refinements to represent system at different abstraction level. We have used Pro B [19] model checker and animator for constraint based checking, discover errors due to invariant violation and deadlocks, thereby, validating the specifications. We present an abstract model specifying atomic broadcast with a fixed sequencer and introducing moving sequencer in refined version. Index Terms Broadcast, Atomic Broadcast, Total Order, Unicast, Sequencer, Deliver, Model Checking, Event B I. INTRODUCTION Atomic broadcast (also known as total order broadcast) is an important abstraction in fault tolerant distributed computing [6]. It ensures that messages broadcasted by different processes are delivered by all destination processes in same order [7]. So, who will be responsible for ordering on messages? There are various approaches to solve this query and a lot of papers have been published in recent 25 years; The fixed sequencer atomic broadcast is one and simplest ordering mechanism. Sequencer is the site that is responsible for assigning sequence on messages before their delivery to destinations. But at high load, sequencer became overloaded and hence the performance of system degrades. Even in case of sequencer failure whole system may suffer. Hence it motivates scholars to think about alternates. Token based mechanism provides a best option as an alternate; where any process will acquire privilege before broadcasting and sequencing. Token Elsevier, 2013

2 based mechanism further characterized into two ways [4]: (i) Moving sequencer based (ii) Privilege based. For example mechanisms reported by Jia [1], Kim and Kim [2] and Flaviu et al [3] fall into first category. And mechanism reported by Richard et al [6] falls into second category. II. WHY UNICAST BROADCAST VARIANT BASED MECHANISM? There are three variants [4] of fixed sequencer total order broadcast: (i) Unicast broadcast (UB), where any process sends (unicast) its message to sequencer then sequencer put some appropriate sequence number (or order) on it and broadcast message. (ii) Broadcast broadcast (BB), where any process broadcast its message subsequently sequencer will put some sequence number (or order) on this message and broadcast. Hence there are two broadcasts for a single message. Any process can receive message only after receiving the sequence (or order) from sequencer. (iii) Unicast unicast broadcast (UUB), In this case, Any process sends (unicast) its message to sequencer then sequencer will put some appropriate sequence number (or order) on message and then return back (unicast) to sender so that the original sender can broadcast its message with a proper sequence number(or order). Any variant can be used to achieve moving sequencer total order broadcast. On scholarly observation of these moving sequencer mechanisms [1, 2, 3], it has found that all of these fall into BB category. If n is total number of correct processes then for a single message broadcast, BB generates 2*(n-1) messages; hence it leads to a very high traffic in system. Subsequently, it will degrade the performance of system. While UB based mechanism generates (n) messages that results very low traffic congestion is system and hence the performance will be maintained. It is therefore interesting to try to design atomic broadcast that rely on unicast broadcast (UB) in order to combine the advantage of UB and moving sequencer that provides, good stability time, fast delivery time and less message transmission in system. III. CONTRIBUTIONS OF THE PAPER The paper presents the first model of moving sequencer atomic broadcast that uses unicast broadcast (UB) variant of fixed sequencer atomic broadcast. The Event B; formal method is used to verify the correctness of this model. The results are obtained in sequential steps. Section 4 presents the system model for our proposed work. Section 5 concentrates on related works that already have been done by different authors; Section 6 presents the description of proposed model; this section starts with the discussion of architecture of our proposed work, then defines abstract model where fixed sequencer is used to build atomic broadcast subsequently, moving sequencer is introduced in refined model. IV. SYSTEM MODEL We assume an asynchronous system composed of n correct processes belongs to a set π = {P 1, P 2... P n }. For the simplicity we have considered a set of three processes as: Process belongs to π and Process = {P 0, P 1, P 2 }. The processes communicate by message passing over reliable channels. Message is a set of messages, for simplicity we have considered three message as: Message = {M 1, M 2, M 3 }, seq(process) is used to create a sequence on process and we have chosen [P 1, P 2, P 3 ] a sequence list for our model. Since processes are in sequence hence we denote (i) first ([P 1, P 2, P 3 ]) = {P 1 }. We also assume that there is no failure in the system. A. Agreement problem The agreement problem considered in this paper is presented below. Atomic broadcast : Atomic broadcast problem is defined by primitive [6] a_broadcast and a_delivers, the processes have to agree on a common order on a set of messages. Formally we can define atomic broadcast (uniform) by four properties [4]; (i) Validity: if a correct process a_broadcast message m then it eventually a_delivers m, (ii) Uniform agreement: If a process a_delivers m then all the correct processes a_deliver m. (iii) Uniform integrity: For any message m, every process p, a_delivers m at most once and only if m was previously a_broadcast. And (iv) Uniform total order: If some process, a_delivers m before m' then every process a_delivers m' only after it has a_delivered m. Sequencer based algorithms: The sequencer based atomic broadcast [3] is simplest one and provides best delivery time (in absence of failure) while the protocols based on privilege provide best stability time in system where logical ring is formed and message is passed along with token. In our proposed work we present a sequencer based approach, where sequencer will move in a logical ring. 485

3 V. RELATED WORK There is lot of work have been done since 25 years in area of privilege based atomic broadcast. On demand protocol [12], Train [13], Totem [14], TPM [15], Gopal and Toueg mechanism [16], RTCAST [17], MARS [18] and [6] are based on privilege. These algorithms provide a concept of logical ring in which message can travel along with token. On the other hand moving sequencer algorithms are simpler than privilege based. RMP [1], DTP [2] and Pin Wheel [3] are based on moving sequencer atomic broadcast. Our mechanism seems similar to Pin Wheel [3] but it is different in many ways. Pin Wheel [3] utilizes broadcast broadcast (BB) variant of fixed sequencer that introduces 2*(n-1) messages in system while our proposed model utilizes unicast broadcast (UB) variant of fixed sequencer that will introduce (n) messages in system which is very less in comparison to 2*(n-1). Hence, to achieve best delivery time and less message transmission in system; we have tried to present a mechanism that utilizes the advantages of moving sequencer along with unicast broadcast (UB) variant of fixed sequencer. VI. DESCRIPTION OF MODEL We have used incremental approach to design a model of total order. In first step we have developed abstract model which describes the basic functionality of total order with fixed sequencer. In refined version we have introduced moving sequencer to build total order. In most traditional moving sequencer based algorithms, broadcast broadcast (BB) variant of fixed sequencer is used [1, 2, 3]. In BB, if any process wants to broadcast their updates (messages) then it can broadcast it to all, and then sequencer process will broadcast the sequence numbers for these updates (messages). This approach (BB) introduces high message congestion into the system. A. Architecture of proposed work Fig. 1 represents the working environment of proposed work. The proposed model try to present a new approach that relies on unicast broadcast (UB) variant of fixed sequencer. In UB, if any process needs to broadcast any update (messages) then it will unicast that update to sequencer process, and then sequencer will put a unique sequence number on it and then broadcast (message with sequence number) to all. After broadcast of message, any process can deliver that message according to increasing sequence number. Before any new broadcast our model elects a new sequencer process (in refined version); which will be just next process to current sequencer. In this way it will form a logical ring. It will also reduce the load on a dedicated process that is responsible for sequencing along with its normal tasks. This approach introduces very less message (n messages) in comparison to BB (2*(n-1) messages) variant. Event B [8, 9, 10, 11] is a formal technique to develop formal models of distributed system through various refinement steps. Table 1 represents the various Event B symbols used in our model. TABLE I EVENT B SYMBOL USED IN MODEL B symbols Description : Belongs to set /: Not belongs to set <: Subset! For every X Cartesian Product POW Power Set <-> Relation PRE Pre-Condition BOOL Boolean Seq Sequence on set We have used Pro B [19] model checker and animator for B Specifications. The Pro B tool supports automatic consistency checking of B machines via model checking. It allows fully automatic animation of B specification for errors. B. Abstract model In abstract model, unicast broadcast (UB) variant of fixed sequencer is chosen to build total order. Fig. 2 summarizes all the machine variables with their corresponding initial values and constraints (or invariant) on them. 486

4 Sequencer Re selection (Next process to current sequencer will elect as new sequencer) Deliver Message (All processes will receive broadcasted messages in total order) Ordering and Broadcast (Sequencer will create an order and broadcast with message) Any new unica Unicast Message (Any process can unicast its message to sequencer) Sequencer Selection (Initially first process will be sequencer) Fig. 1. Architecture of proposed work Process and Message are two sets representing processes and messages correspondingly. selected_sequencer is variable and defined on Process set that represents sequencer process. In this model, we have assumed that, initially first process will be sequencer. sequencer_selection is a boolean flag that represents an information that any process has been elected as sequencer (if TRUE). unicast_message represents a relation between process and corresponding unicasted messages, temporary_receive represents the message delivered by current sequencer to its local memory but it is not actual delivery, follow, represents a relationship between natural numbers where (2 -> 1) represents 2 is following 1, sent is a relationship among processes, messages and sequence numbers. Sequencer will always broadcast any message with a unique sequence number. When sequencer broadcast any message it will append message with a unique sequence number. This relationship shows message and its original sender with assigned sequence number, seq_no is natural number that represents a unique sequence number provided by sequencer, receive is a relationship among processes, messages and sequence numbers. When any process receives any message then appends this information to receive set, message_with_seq_no is a relationship between message and sequencer number. This relation is maintained by sequencer to show which message is broadcasted with which sequence number. Events: Events will get enabled if all PRE conditions are true. We have designed different events that satisfy a particular purpose. Sequencer selection event: The select_sequencer event (see fig. 3) will choose a process as sequencer. Initially first process of Process set will be elected as sequencer. Unicast event: If any process wants to broadcast any message then at first it will use Unicast event (see fig. 4) to unicast its message to current sequencer. Broadcast event: Broadcast event (see fig. 5) will be used by sequencer to broadcast message with proper sequence number to all the destination processes. Hence it will broadcast (n-1) messages. Deliver event: The Deliver event (see fig. 6) will occur at every process to deliver the message. The PRE condition 487

5 MACHINE Abstract1 SETS Process= {P1, P2, P3}; Message={M1, M2, M3} VARIABLES selected_sequencer, sequencer_selection, unicast_message, temporary_receive, follow, sent, seq_no, receive, msg_with_seq_no INVARIANT selected_sequencer : POW(Process) sequencer_selection : BOOL unicast_message : Process <-> Message temporary_receive : Process <-> Message follow : NATURAL1<->NATURAL1 sent : (Process<->Message)<->NATURAL1 seq_no : NATURAL1 receive : (Process <-> Message)<-> NATURAL1 msg_with_seq_no :Message<-> NATURAL1 INITIALISATION selected_sequencer :={} sequencer_selection :=FALSE unicast_message :={} temporary_receive :={} follow :={} sent :={} seq_no :=1 receive :={} msg_with_seq_no :={} Fig. 2. Abstract Machine Sets, Variables and their initial value select_sequencer (p) = PRE p : Process & sequencer_selection = FALSE selected_sequencer := {p} sequencer_selection := TRUE Fig. 3. Sequencer Selection Event! (n, mm). (n: NATURAL1 & mm: dom (msg_with_seq_no) & (num ->n) : follow & {mm -> n} <: msg_with_seq_no=> ({p ->mm} ->n): receive) will make sure that all processes will deliver messages in total order. 488

6 Unicast (p, m) = PRE p : Process & p /: selected_sequencer & m : Message & m /: ran(unicast_message) & sequencer_selection = TRUE unicast_message := unicast_message \/ {p -> m} Fig. 4.Unicast Event Broadcast (p, m, number) = PRE p : selected_sequencer & m : ran(unicast_message) & m /: ran(temporary_receive) & number : NATURAL1 & number= seq_no temporary_receive:=temporary_receive\/{p ->m} sent := sent \/ {{p -> m} -> number} follow :=follow \/{number} X ran(sent) msg_with_seq_no :=msg_with_seq_no \/ {m -> number} seq_no :=seq_no+1 deliver(p, m, num) = Fig. 5. Broadcast Event PRE p : Process & m : dom(msg_with_seq_no) & num : ran(sent) & (m -> num) : msg_with_seq_no & ({p -> m} -> num) /: receive &! (n,mm).(n : NATURAL1 & mm : dom(msg_with_seq_no) & (num -> n) : follow & {mm -> n}<: msg_with_seq_no =>({p -> mm} - >n):receive) receive: = receive\/{{p -> m} -> num} Fig. 6. Deliver Event 489

7 C. Refined model Fig. 2 to fig. 6 presents the abstract machine where a fixed process (we have assumed first process) will work as sequencer and broadcast messages with correct and unique sequence number. In refined version (see fig. 7) of this machine we introduce the moving nature of sequencer; where next process to current sequencer will be elected as new sequencer. For this purpose we have introduced some new invariants and some new events. Variable select_new_sequencer_flag is Boolean. select_new_sequencer_flag : BOOL, If it is true, indicating that new sequencer has to be elected first. Variable sequencer_list is a sequenced list of all processes. sequencer_list : seq(process). At start of system first process will be elected as sequencer then next and so on in a cyclic manner. And process_id variable represents process s identification number that starts from 1 and increases by one. Each process has a unique identification number. process_id: NATURAL Since refined version is based on its previous version (abstract model) hence all the functionalities given in abstract model should also be available here with some new functionality that is introduced in refined version. Events: The event presented in refined version will work together with the events that are already given in abstract model. New sequencer selection event: The new event select_new_sequencer (see fig. 8) is introduced in refined to select new sequencer. It will select a process as a sequencer that is just next to current sequencer. REFINEMENT refine1 REFINES Abstract1 VARIABLES select_new_sequencer_flag, sequencer_list, process_id INVARIANT select_new_sequencer_flag : BOOL & sequencer_list : seq(process) & process_id : NATURAL INITIALISATION select_new_sequencer_flag := FALSE sequencer_list := [p1,p2,p3] process_id := 1 Fig. 7. Variables, Invariants and their initial value in refined version select_new_sequencer(p)= PRE p : Process & process_id <= size(sequencer_list) & p = sequencer_list(process_id+1) & select_new_sequencer_flag = TRUE selected_sequencer := {p} select_new_sequencer_flag := FALSE IF process_id = size(sequencer_list) - 1 process_id :=0 ELSE process_id := process_id + 1 Fig. 8. New Sequencer Selection Event 490

8 VII. CONCLUSION This paper presents a novel approach to build moving sequencer atomic broadcast that relies on unicast broadcast variant of fixed sequencer. It generates very less message (1 message for unicast + (n-1) message for broadcast = n messages) in comparison to broadcast broadcast variant (BB) of fixed sequencer. Each process will be a sequencer in a cyclic manner. So in operational phase not a single process will be overloaded continuously for sequencing. In case of message loss we can also use negative and positive acknowledgement [12] to recover it. Pro B [19], is a model checker and animator tool is used for modelling and step by step checking. We checked our model for invariant violation or for any deadlock occurrence. There is no case of invariant violation or deadlock. The model works fine with any number of processes and messages. REFERENCES [1] Jia, W., Kaiser, J., and Nett, E., RMP: Fault Tolerant GroupCommunication, Micro, IEEE, (1996); 16(2); [2] Kim, J., and Kim C., A total ordering protocol using a dynamic token-passing scheme, Distributed System Engineering, (1997), 4(2); [3] Cristian, F., Mishra, S., and Alvarez, G., High performance asynchronous atomic broadcast, Distributed System Engineering, (1997), 4(2); [4] D efago, X., Schiper, A., and Urb an, P., Total order broadcast and multicast algorithms: Taxonomy and survey, ACM Comput. Surv., (2004), 36 (4); [5] Abrial, J., R., Modeling in Event-B: System and Software Engineering, [6] Ekwall, R., and Schiper, A., A Fault-Tolerant Token-Based Atomic Broadcast Algorithm, Dependable and Secure Computing, IEEE Transactions, (2011), 8(5); [7] Hadzilacos, V., and Toueg, S., Fault-Tolerant Broadcasts and Related Problems, Distributed systems (2nd Ed.), ACM Press/ Addison- Wesley Publishing Co., (1993), [8] Yadav, D., Butler, M., Application of Event B to Global Causal Ordering for Fault Tolerant Transactions. In: REFT: Workshop on Rigorous Engineering of Fault Tolerant Systems, Newcastle upon Tyne, (2005), [9] Butler, M., Yadav, D., an Incremental Development of the Mondex System in Event-B. Formal Aspects of Computing, Springer-Verlag, (2008), 20(1); [10] Metayer, C., Abrial, J.R., Voison, L., Event-B language. Technical Report, Deliverables 3.2, EU Project IST RODIN, (2005), [11] Abrial, J.R.: A System Development Process with Event-B and the Rodin Platform. In ICFEM, (2007), Springer, LNCS, 4789; 1 3. [12] Chang, J. M., and Maxemchuk, N. F., Reliable broadcast protocols, ACM Trans. Comput. Syst., (1984), 2(3); [13] Cristian, F., Asynchronous atomic broadcast. IBM Technical Disclosure Bulletin, (1991), 33(9); [14] Amir, Y., Moser, L. E., Melliar-Smith, P. M., Agarwal, D. A., Ciarfella, P., The Totem single-ring ordering and membership protocol. ACM Trans. Comput. Syst., (1995), 13(4); [15] Rajgopalan, B., Mckinley, P., A token-based protocol for reliable, ordered multicast communication. In Proc. 8th Symp. On Reliable Distributed Systems (SRDS), (1989), [16] Gopal, A., Toueg, S., Reliable broadcast in synchronous and asynchronous Environment, In Proc. 3rd Intl. Workshop on Distributed Algorithms (WDAG). Springer-Verlag, (1989), [17] Abdelzaher, T., Shaikh, A., Jahanian, F., Shin, K., RTCAST: Lightweight multicast for real-time process groups. In IEEE Real- Time Technology and Applications Symposium, (1996), [18] Kopetz, H., Gr unsteidl, G., Reisinger, J., Fault-tolerant membership service in a synchronous distributed real-time system. In Dependable Computing and fault tolerant systems, Springer- Vienna, (1991), 4; [19] Leuschel, M., Butler, M.: Pro B: A model checker for B. In: Araki, K., Gnesi, S., Mandrioli, D. (eds.) FME, Springer,Heidelberg, (2003), LNCS, 2805;