Towards Generic and Middleware-independent Support for Replicated, Distributed Objects

Transcription

1 Towards Generic and Middleware-independent Support for Replicated, Distributed s Jörg Domaschka, Hans P. Reiser, Franz J. Hauck Distributed Systems Lab, Faculty of Computer Science Ulm University Germany Aspectix Research Group 20 March 2007 J. Domaschka (Ulm University) Generic Replication Support 1 / 19

2 Motivation Why replication? -based distributed systems have single point of failure Replication provides fault-tolerance J. Domaschka (Ulm University) Generic Replication Support 2 / 19

3 Motivation Why replication? -based distributed systems have single point of failure Replication provides fault-tolerance What are the drawbacks? Implementing replication support is non-trivial High development costs Bigger middleware core J. Domaschka (Ulm University) Generic Replication Support 2 / 19

4 Motivation Why replication? -based distributed systems have single point of failure Replication provides fault-tolerance What are the drawbacks? Implementing replication support is non-trivial High development costs Bigger middleware core State of the art Many middleware systems provide replication support All of them support group communication, replica management Barely re-use of existing solutions J. Domaschka (Ulm University) Generic Replication Support 2 / 19

5 Goal Goal Provide a single replication framwork for multiple middleware systems! Problems to solve What interface does the framework have to provide? How to conceptually integrate the framework into the middleware systems? J. Domaschka (Ulm University) Generic Replication Support 3 / 19

6 Outline 1 Architecture for Replication Support 2 Separation in Practice 3 Evaluation 4 Conclusion J. Domaschka (Ulm University) Generic Replication Support 4 / 19

7 Background Middleware for Distributed s Application Impl. Stub Communication Communication Dispatcher J. Domaschka (Ulm University) Generic Replication Support 5 / 19

8 Background Middleware for Distributed s Application Impl. Stub Communication Communication Dispatcher Client accesses a single remote object Client-server interaction scheme Request followed by response Rather simple communication J. Domaschka (Ulm University) Generic Replication Support 5 / 19

9 Background Replication Support in Middleware Application Dispatcher Impl. Communication Stub Communication Communication Dispatcher Impl. J. Domaschka (Ulm University) Generic Replication Support 6 / 19

10 Background Replication Support in Middleware Application Dispatcher Impl. Communication Stub Communication Communication Dispatcher Impl. Client accesses a group of identical remote objects Replica group may change Determinism by totally ordered multicast Complex communication logic J. Domaschka (Ulm University) Generic Replication Support 6 / 19

11 Background Replication support: Separation of concerns Application Dispatcher Impl. Communication Stub Communication Communication Dispatcher Impl. Replication Framework J. Domaschka (Ulm University) Generic Replication Support 7 / 19

12 Background Replication support: Separation of concerns Application Dispatcher Impl. Communication Stub Communication Communication Dispatcher Impl. Replication Framework Responsibility of a replication framework Ensure determinism and consistency Follow changes of the replica group Ensure replication transparency J. Domaschka (Ulm University) Generic Replication Support 7 / 19

13 An Architecture for Replication Support Initial step Application Dispatcher Impl. Communication Stub Communication Communication Dispatcher Impl. Replication Framework J. Domaschka (Ulm University) Generic Replication Support 8 / 19

14 An Architecture for Replication Support Initial step Application Dispatcher Impl. Communication Stub Communication Communication Dispatcher Impl. Replication Framework Generic replication framework: Transports any data Handles data in an opaque way J. Domaschka (Ulm University) Generic Replication Support 8 / 19

15 An Architecture for Replication Support Initial step Application Dispatcher Impl. Communication Stub Communication Communication Dispatcher Impl. Replication Framework Generic replication framework: Transports any data Handles data in an opaque way Interface supports only byte[] J. Domaschka (Ulm University) Generic Replication Support 8 / 19

16 An Architecture for Replication Support Initial step Application Dispatcher Impl. Communication Stub Communication Communication Dispatcher Impl. Replication Framework Generic replication framework: Transports any data Handles data in an opaque way Interface supports only byte[] Integration into middleware Not all stubs support byte[] Redirecting calls to replication framework necessary J. Domaschka (Ulm University) Generic Replication Support 8 / 19

17 An Architecture for Replication Support Step II: The Application Obj Dispatcher Impl. Replication Server Stub Replication Client Replication Server Rep lica tion Fra m e work Obj Dispatcher Impl. J. Domaschka (Ulm University) Generic Replication Support 9 / 19

18 An Architecture for Replication Support Step II: The Application Obj Dispatcher Impl. Replication Server Stub Replication Client Replication Server Rep lica tion Fra m e work Obj Dispatcher Impl. characteristics Conceptually part of the middleware Initialises replication framework Redirects calls to replication framework J. Domaschka (Ulm University) Generic Replication Support 9 / 19

19 An Architecture for Replication Support Step II: The Application Obj Dispatcher Impl. Replication Server Stub Replication Client Replication Server Rep lica tion Fra m e work Obj Dispatcher Impl. characteristics Conceptually part of the middleware Initialises replication framework Redirects calls to replication framework Does not solve all problems J. Domaschka (Ulm University) Generic Replication Support 9 / 19

20 An Architecture for Replication Support The binding problem Application Remote Ref Middleware Runtime Remote Ref (A,B) Replica A Replica B J. Domaschka (Ulm University) Generic Replication Support 10 / 19

21 An Architecture for Replication Support The binding problem Application Remote Ref Stub Middleware Runtime Remote Ref (A,B) Replica A Replica B J. Domaschka (Ulm University) Generic Replication Support 10 / 19

22 An Architecture for Replication Support The binding problem Application Stub Middleware Runtime Remote Ref (A,B) (A, B) Replication Client Contact Info (A,B) Replica A Replica B J. Domaschka (Ulm University) Generic Replication Support 10 / 19

23 An Architecture for Replication Support The binding problem Application Stub Middleware Runtime Remote Ref (A,B) Replication Client Contact Info (C,D) Replica A Replica C Replica D Replica B J. Domaschka (Ulm University) Generic Replication Support 10 / 19

24 An Architecture for Replication Support The binding problem Application Remote Ref Application Stub Stub Middleware Runtime Middleware Runtime Remote Ref (A,B) Remote Ref (A,B) Replication Client Contact Info (C,D)? Replication Client Contact Info (A,B) Replica A Replica C Replica D Replica B J. Domaschka (Ulm University) Generic Replication Support 10 / 19

25 Solving the Binding Problem Remote Reference should reflect lastest known state J. Domaschka (Ulm University) Generic Replication Support 11 / 19

26 Solving the Binding Problem Remote Reference should reflect lastest known state Middleware system has to get the state J. Domaschka (Ulm University) Generic Replication Support 11 / 19

27 Solving the Binding Problem Remote Reference should reflect lastest known state Middleware system has to get the state Middleware pulls information from adapter/stub Not supported by all middleware systems J. Domaschka (Ulm University) Generic Replication Support 11 / 19

28 Solving the Binding Problem Remote Reference should reflect lastest known state Middleware system has to get the state Middleware pulls information from adapter/stub Not supported by all middleware systems Replication framework pushes state into middleware No generic middleware interface J. Domaschka (Ulm University) Generic Replication Support 11 / 19

29 Solving the Binding Problem Remote Reference should reflect lastest known state Middleware system has to get the state Middleware pulls information from adapter/stub Not supported by all middleware systems Replication framework pushes state into middleware No generic middleware interface Conversion step required Converter as callback handler J. Domaschka (Ulm University) Generic Replication Support 11 / 19

30 An Architecture for Replication Support Step III: The Converter Application Stub Middleware Runtime Remote Ref (A,B) updatereference Replication Client Contact Info (B,C) Replica A Replica C Replica B J. Domaschka (Ulm University) Generic Replication Support 12 / 19

31 An Architecture for Replication Support Step III: The Converter Application Stub Middleware Runtime Remote Ref (A,B) Converter Replication Client Contact Info (B,C) Replica A Replica C Replica B J. Domaschka (Ulm University) Generic Replication Support 12 / 19

32 An Architecture for Replication Support Step III: The Converter Application Stub Middleware Runtime Remote Ref (B,C) Converter Replication Client Contact Info (B,C) Replica A Replica C Replica B J. Domaschka (Ulm University) Generic Replication Support 12 / 19

33 Separation in Practice Starting Point: FTflex replication framework integrated into Aspectix middleware J. Domaschka (Ulm University) Generic Replication Support 13 / 19

34 Separation in Practice Starting Point: FTflex replication framework integrated into Aspectix middleware Coupling: IOR Aspectix: Remote reference, binding FTflex: Store state of replica group Coupling code: Code in framework unrelated to replication J. Domaschka (Ulm University) Generic Replication Support 13 / 19

35 Separation in Practice Starting Point: FTflex replication framework integrated into Aspectix middleware Coupling: IOR Aspectix: Remote reference, binding FTflex: Store state of replica group Coupling code: Code in framework unrelated to replication Separation IOR Used for binding only Extended FTflex with a replication state management facility Converter translates between both systems Coupling Code: Moved to adapter J. Domaschka (Ulm University) Generic Replication Support 13 / 19

36 Support for Arbitrary Middleware Architectures and converter heavily depend on middleware In general four approaches to include the adapter Integration: modify the middleware source code Interception: intercept messages at OS level Service approach: application calls replication services Customised stub: All further stubs are copies of the first one Existing adapters Aspectix-FTflex adapter Java RMI-FTflex J. Domaschka (Ulm University) Generic Replication Support 14 / 19

37 Evaluation Replication: Java RMI 81.7 µs RMI-FTflex µs Costs 55 % Separation: FTflex integrated µs FTflex adapter µs Costs 4 % Set-up: One client, one replica On a single physical machine Different virtual machines J. Domaschka (Ulm University) Generic Replication Support 15 / 19

38 Conclusion Conclusion: Middleware-independent replication support is feasible Genericity by the use byte[] at interface Handle data in an opaque way Light-weight Integration into middleware and Converter Overhead introduced by replication logic Ongoing work: for JacORB Comparison with FT-CORBA implementation J. Domaschka (Ulm University) Generic Replication Support 16 / 19

39 Separation of Concerns reference, Interface definition, adapter, Binding, Code generation, Consistency management, Replication management J. Domaschka (Ulm University) Generic Replication Support 17 / 19

40 Separation of Concerns Interface definition, adapter, Binding, Code generation, Consistency management, Replication management reference Middleware concerns Replication concerns J. Domaschka (Ulm University) Generic Replication Support 17 / 19

41 Separation of Concerns Interface definition, adapter, Binding, Code generation, Consistency management, Replication management Middleware concerns reference Replication concerns J. Domaschka (Ulm University) Generic Replication Support 17 / 19

42 Separation of Concerns adapter, Binding, Code generation, Consistency management, Replication management Interface definition Middleware concerns reference Replication concerns J. Domaschka (Ulm University) Generic Replication Support 17 / 19

43 Separation of Concerns adapter, Binding, Code generation, Consistency management, Replication management Middleware concerns reference Interface definition Replication concerns J. Domaschka (Ulm University) Generic Replication Support 17 / 19

44 Separation of Concerns Binding, Code generation, Consistency management, Replication management adapter Middleware concerns reference Interface definition Replication concerns J. Domaschka (Ulm University) Generic Replication Support 17 / 19

45 Separation of Concerns Binding, Code generation, Consistency management, Replication management Middleware concerns reference Interface definition adapter Replication concerns adapter J. Domaschka (Ulm University) Generic Replication Support 17 / 19

46 Separation of Concerns Code generation, Consistency management, Replication management Binding Middleware concerns reference Interface definition adapter Replication concerns adapter J. Domaschka (Ulm University) Generic Replication Support 17 / 19

47 Separation of Concerns Code generation, Consistency management, Replication management Middleware concerns reference Interface definition adapter Binding Replication concerns adapter J. Domaschka (Ulm University) Generic Replication Support 17 / 19

48 Separation of Concerns Consistency management, Replication management Code generation Middleware concerns reference Interface definition adapter Binding Replication concerns adapter J. Domaschka (Ulm University) Generic Replication Support 17 / 19

49 Separation of Concerns Consistency management, Replication management Middleware concerns reference Interface definition adapter Binding Code generation Dispatching Marshalling in Stub and Replication concerns adapter J. Domaschka (Ulm University) Generic Replication Support 17 / 19

50 Separation of Concerns Replication management Consistency management Middleware concerns reference Interface definition adapter Binding Code generation Dispatching Marshalling in Stub and Replication concerns adapter J. Domaschka (Ulm University) Generic Replication Support 17 / 19

51 Separation of Concerns Replication management Middleware concerns reference Interface definition adapter Binding Code generation Dispatching Marshalling in Stub and Consistency management Calling semantics Replication concerns adapter Consistency management Message ordering Determinism J. Domaschka (Ulm University) Generic Replication Support 17 / 19

52 Separation of Concerns Replication management Middleware concerns reference Interface definition adapter Binding Code generation Dispatching Marshalling in Stub and Consistency management Calling semantics Replication concerns adapter Consistency management Message ordering Determinism J. Domaschka (Ulm University) Generic Replication Support 17 / 19

53 Separation of Concerns Middleware concerns reference Interface definition adapter Binding Code generation Dispatching Marshalling in Stub and Consistency management Calling semantics Replication concerns adapter Consistency management Message ordering Determinism Replication Management Monitoring of replicas Start new replicas migrate replicas J. Domaschka (Ulm University) Generic Replication Support 17 / 19

54 Aspectix CORBA-based Supports fragmented objects An object consits of multiple fragments Fragments are arbitrarily distributed over multiple address spaces Distribution of state and functionality is arbitrary Communication is an object-internal issue Fragment 1 Fragment 2 Fragment 3 Client J. Domaschka (Ulm University) Generic Replication Support 18 / 19

55 Aspectix CORBA-based Supports fragmented objects An object consits of multiple fragments Fragments are arbitrarily distributed over multiple address spaces Distribution of state and functionality is arbitrary Communication is an object-internal issue Features Implicit binding Interface Definition Code generation but arbitrary distribution of state and functionality disables generation of marshalling and dispatching J. Domaschka (Ulm University) Generic Replication Support 18 / 19

56 FTflex Replication infrastructure based on Aspectix Fragmented objects Has a more concrete object structure Access fragments Replica fragments Code generation results in marshalling and dispatching code Replica Fragment Replica Fragment Replica Fragment Access Fragment Client J. Domaschka (Ulm University) Generic Replication Support 19 / 19

57 FTflex Replication infrastructure based on Aspectix Fragmented objects Has a more concrete object structure Access fragments Replica fragments Code generation results in marshalling and dispatching code adapter part of replica fragment Message Management Dispatching Error handling Own group communication system J. Domaschka (Ulm University) Generic Replication Support 19 / 19