On Hierarchical, parallel and distributed components for Grid programming

Transcription

1 On Hierarchical, parallel and distributed components for Grid programming Francoise Baude, Denis Caromel, Matthieu Morel OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis Distributed Components vs. A component over several machines! 1. Context & Motivation 2. Distributed Objects in ProActive, and Component in Fractal 3. Parallel, Distributed, Hierarchical Components 4. Deployment, Implementation, and runtime 1

2 1. Context & Motivation 2

3 Component interface A CORBA Component OFFERED Facets Event sinks My Business Component Receptacles Event sources REQUIRED Attributes Courtesy of Philippe Merle, Lille, OpenCCM platform 3

4 Building CCM Applications = Assembling CORBA Component Instances Provide + Use, but flat assembly 4

5 Component Orientation Level 1: Instantiate - Deploy - Configure Simple Pattern Meta-information (file, XML, etc.) JavaBeans, EJB Level 2: Assembly (flat) Server and client interfaces CCM, CCA Level 3a: Parallel Parallel SPMD codes, collective invocations (MxN) Level 3b: Hierarchic : ease design of complex applications Composite, inclusion, reconfiguration(for adaptation) GridCCM, DCA Fractal Level 4: Parallel & Distributed, Hierarchical Assembly+Reconfiguration Location, single/collective remote invocations ProActive components This talk 5

6 2. ProActive and Fractal 6

7 ProActive: A Java model + API for parallel and distributed programming Parallel, Distributed, Mobile, Secured Activities, across the world! Desktop SMP LAN Clusters Remote Mobile Objects Asynchronous RMI with synchro: automatic Futures Group Communications (support for object oriented SPMD) Migration (of computations) Secured communications Well-defined semantic of the computing model In Consortium Working on: remote nonfunctional exceptions SOAP vs RMI MPI code wrapping 7

8 Creating AO and Groups A ag = newactivegroup ( A, [ ], VirtualNode) V v = ag.foo(param); //remote method call of foo v.bar(); A V Typed Group Java or Active Object Group and Asynchrony are crucial for Components 8

9 ProActive: model + tools for deployment on the grid Abstract away machines, creation, registry, lookup protocols VirtualNodes: Use only Virtual Nodes in the source code Describe mapping of virtual nodes in XML Deployment Descriptors Interfaced with various protocols for creation/lookup: rsh,ssh,jini, LSF,PBS,Globus,OGSA... Unification of various deployment systems through ProActive runtimes Dispatcher <RegisterIn RMIregistry, Globus, Grid Service, > RendererSet Mapping: Dispatcher --> DispatcherJVM RendererSet --> JVMset JVMs: DispatcherJVM = <javapath /> <classpath /> sshprocess JVMset = <javapath /> <classpath /> GlobusProcess Processes: sshprocess = <hostname di.unice.fr/> <login../> GlobusProcess = <hostname cluster.inria.fr/> 9 <gram port 2119/> <node 10/>

10 IC2D: Interactive Control and Debugging of Distribution 10

11 Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore ProActive IC2D: Width of links proportional to the number of communications 11

12 The Fractal model: Hierarchical Component Defined by E. Bruneton, T. Coupaye, J.B. Stefani, INRIA & FT 12

13 Component = runtime entity Controller Content 13

14 Interface = access point Controller Content 14

15 Hierarchical model : composites encapsulate primitives, which encapsulates Java code Controller Content 15

16 Binding = interaction Controller Content 16

17 Binding = interaction Controller Content 17

18 Controllers : non-functional properties Component Identity Binding Controller LifeCycle Controller Content Controller Controller Content Reconfiguration is possible and transparent for the client 18

19 3. Parallel, Distributed, Hierarchical Components 19

20 ProActive Components for Grid programming An activity, a process, a MPI or OO SPMD pgm potentially in its own JVM or JVM s set 1. Primitive component 2. Composite component Composite: Hierarchical, and Distributed over machines 3. Parallel and composite Parallel: Composite component + Broadcast (group) 20

21 Components vs. Activities and JVMs A B1 Activity JVM Component Cp. are rather orthogonal to activities and JVMs: contain activities, span across several JVMs B2 B3 Hierarchical components: a way to globally manipulate distributed, and running activities on the grid C 21

22 ProActive Component Definition A component is: Formed from one (or several) Active Object Executing on one (or several) JVM Provides a set of server ports: Java Interfaces Uses a set of client ports: Java Attributes Point-to-point or Group method invocation between components Hierarchical: Primitive component: defined with Java code Composite component: composition of primitive + composite Parallel component: multicast of calls in composites Descriptor: XML definition of primitive and composite (ADL) Virtual nodes capture the deployment capacities and needs Virtual Node is a very important abstraction for GRID components 22

23 Groups in Components : collective interfaces Broadcast or Scatter of in parameters B C A parallel component P A A B D Gather or Reduce of Group out proxyparameters Group proxy C Multicast due to binding, on client interface Due to composition, on composite inner server interface 23

24 Composing collective ports (MxN like problem) P1 Toolkit: Future for parameters and result Use additional underlying active objects Specific behaviour in the binding P2 controller Gather interface of method calls Group proxy Group proxy 24

25 4. Composition, Deployment and Runtime 25

26 Component Definition: ADL+package Primitive-component "cd-player implementation = "CdPlayer // Java class with functional code Provides interface "input Requires VirtualNode = VNa Composite-component stereo Provides Requires // Virtual Node name Primitive-component cd-player with VirtualNode = VNa Parallel-component Parallel-speakers with VirtualNode = VNb, cyclic... Bindings bind "cd-player.output" to Parallel-speakers.input" Merging VNa, VNb, ---> VNb // Co-allocation in that case 26

27 VNa Two different Virtual Nodes compositions VNb VNc = VN(a,b) C A B C A B Separate or Co-allocation 27

28 Deployment Descriptor: Initial Mapping of Virtual Nodes Example of an XML deployment file (application dependant) : VirtualNodes: VNa <RegisterIn RMIregistry > //or Globus, Grid Service, VNb Mapping: VNa --> playerjvm VNb --> speakersset JVMs: playerjvm = Current // (the current JVM) speakersset=//clustersophia.inria.fr/ <Protocol Globus 10 > // or rsh, ssh, LSF, 28

29 Migration Capability of composites Migrate sets of components, including composites 29

30 Migration Capability of composites Migrate sets of components, including composites Initial mapping of components can be easily modified 30

31 Co-allocation, Re-distribution e.g. upon communication intensive phase 31

32 Co-allocation, Re-distribution e.g. upon communication intensive phase 32

33 Co-allocation, Re-distribution e.g. upon communication intensive phase 33

34 Conclusion Summary: Distributed and composite components A single component executing on several machines Parallel components: multicast of calls at Composition Group proxy interconnection: multicast of calls at Deployment Virtual Nodes: ability to compose virtual nodes On Going/Future work: Publish components as web services (and orchestrate with foreign web services) Optimization of composite (binding controller) but problem for Reconfiguration Compose / Deploy / Monitor / Control at the component level: Interactive Graphical Composition using the Fractal GUI tool Merging Component + Activity / JVM Views 34

35 Interactive Composition in IC2D Composition View Content Instead of XML ADL Composite-componen Provides Requir Primitive-compone VirtualNode = VNa Parallel-component VirtualNode = Bindings bind "cd-p speakers.input" Merging VNa, VNb, allocation in that cas 35

36 Merging Component + Activity / JVM Views: Component Monitoring Dynamic View Content 36

37 Merging Component + Activity / JVM Views: Component Monitoring Dynamic View Content 37

38 Conclusion Summary: Distributed and composite components A single component executing on several machines Parallel components: multicast of calls at Composition Group proxy interconnection: multicast of calls at Deployment Virtual Nodes: ability to compose virtual nodes On Going/Future work: Publish components as web services (and orchestrate with foreign web services) Optimization of composite (binding controller) but problem for Reconfiguration Compose / Deploy / Monitor / Control at the component level: Interactive Graphical Composition using the Fractal GUI tool Merging Component + Activity / JVM Views Available with ProActive in The freedom your applications deserve! 38

39 39

40 Some code! // CREATE THE COMPONENTS ComponentIdentity speakers = ProActive.newActiveComponent(speakers_parameters); // OR USE THE COMPONENTS LOADER // ComponentIdentity speakers = ComponentsLoader.getComponent(«speakers»); // BIND THE COMPONENTS ((BindingController)cd_player.getFcInterface(BindingController.BINDING_CONTROLLER)).bindFc(«output», speakers.getfcinterface(«input»); // START THE LIFE CYCLE OF THE COMPONENTS (ENABLE THE COMPONENTS) ((LifeCycleController)speakers.getFcInterface(LifeCycleController.LIFECYCLE_CONTROLLER)).startFc(); // INVOKE SOME ACTIONS ON FUNCTIONAL INTERFACES ((Input)speakers.getFcInterface(«input»)).newMusic (music.mp3); ((PlayerFacade)cd_player.getFcInterface(«control»)).play(); 40

41 VirtualNodes // names of the virtual nodes VirtualNode name= «Node-facade» VirtualNode name =«Node-speaker» - cyclic Deployment // what is behind the names of the nodes mapping // correspondance between the names of the VNs and the JVMs Node-facade --> JVM1 Node-speaker --> {JVM2, JVM3, JVM4} // 1 VN can be mapped onto a set of JVMs JVMs JVM1 created by process «linuxjvm» JVM2 created by process «rsh-computer1» JVM3 created by process «rsh-computer2» JVM4 created by process «rsh-computer3» Infrastructure // how and where the JVMs specified above are created process-definition «linuxjvm» // this process creates a JVM on the current host JVMProcess class= JVMNodeProcess process-definition «rsh-computer1» // this process establishes an rsh connection and starts a JVM on the remote host rshprocess class = RSHProcess host = «computer1» processreference = «linuxjvm» process-definition «rsh-computer2» rshprocess class = RSHProcess host = «computer2» processreference = «linuxjvm process-definition «rsh-computer2» rshprocess class = RSHProcess host = «computer3» processreference = «linuxjvm» 41

42 Implementation: MOP and Meta-Objects VM1 VM2 mobility component representative asynchronism component meta - objects proxy body Stub_a b a 42

43 Implementation: MOP and Meta-Objects VM1 VM2 mobility component representative asynchronism component meta - objects proxy method calls are reified body b a 43

44 Implementation: MOP and Meta-Objects VM1 VM2 mobility component representative asynchronism component meta - objects proxy Component calls also reified body b a 44