From Grids to Clouds to PaaS: new research challenges

Transcription

1 From Grids to Clouds to PaaS: new research challenges AOC Team Christophe Cérin 1 1 Université de Paris XIII, CNRS UMR 7030, France Inria - universités tunisiennes workshop 1 Christophe Cérin Sous thème grille Équipe AOC

2 Table of contents 2 / Objectives 2 Desktop Grids History and Challenges BonjourGrid 3 Towards PaaS and Clouds PaaSoordinated (under reviewing) The coordination and data exchange layer Technologies (ex.) 4 Conclusion 2 Christophe Cérin Sous thème grille Équipe AOC

3 Objectives 3 / Motivate research projets in Grids & Clouds ; 2. Starting from recent advances in Desktop Grid Middleware: BonjourGrid (orchestration of multiple instances of DG middleware) and PastryGrid (fully distributed execution of applications) Joint works with UTIC lab., Tunisia (Heithem Abbes and Mohamed Jemni) 3. Before keeping innovative ideas to reuse in Cloud Architectures / Systems: decentralized architectures and services; large scale systems (FT); interoperability of services (the client is not a prisoner, or if it is, he can choose his prison(s)!) 3 Christophe Cérin Sous thème grille Équipe AOC

4 Desktop Grid Architectures 4 / 113 Desktop Grid ")*&+',#--)*.#'*/+,!#$#%(0,1$&(2)'(0!"!"#$%&'()&*#+,"%(+%-#(!"#$%&'()"*+&%,-($",$.%" 3&(2)'( "//$4*+#'/$1 3&(/2$.&,5*(.0!#$#%&'&$(!" 9(%":&'';<6= 3 /0#0'1$-(2."+&%,-($",$.%" 45"%+3+6*7(#+(#$"%8&," 6>>'(,&$(0#?,-"*.'"% =&5@+3+A&$&+3+<"$+ B?+3+?&#*C0D E%0$0,0'5 Key Points Federation of thousand of nodes; Internet as the communication layer: no trust! Volatility; local IP; Firewall! "#$%&!'()*+,-!-)(./ 0 4 Christophe Cérin Sous thème grille Équipe AOC

5 Desktop Grid Architectures 5 / 113 Desktop Grid ")*&+', #--)*.#'*/+,!#$#%(0,1 $&(2)'(0!"#$%&'("%')*#+,-"#-.*"!"#$%&'()"*+&%,-($",$.%" /01'($+$&0203*&$&+45#$6 7#$"%+#8*"+,8409 "//$4*+#'/$1 5.6&42)&$,78#(9(:!#$#%&'&$( 3&(2)'(!" :8#8';$-(<."+&%,-($",$.%" ;#'#,<#+#=&$ 5.6&42)&$,78#(9(:! I(%"J&''3D?B "#$%&!'()*+,-!-)(./ & = >0"%+=+?*4(#+(#$"%@&,"?11'(,&$(8# A,-"*.'"% B&02+=+C&$&+=+D"$+ EA+=+A&#*F8G H%8$8,8'0 Future Generation (in 2006) Distributed Architecture Architecture with modularity: every component is configurable : scheduler, storage, transport protocole Direct communications between peers; Security; Applications coming from any sciences (e-science applications) 5 Christophe Cérin Sous thème grille Équipe AOC

6 In search of distributed architecture 6 / 113 First line: publish/subscribe system to notify and coordinate services and multiple DG without a central broker BonjourGrid; Second line: approach based on structured overlay network to discover (on the fly) the next node executing the next task PastryGrid; (main contributions of Heithem Abbes in his PhD) 6 Christophe Cérin Sous thème grille Équipe AOC

7 Main objectives of BonjourGrid 7 / 113 Count on existing distributed tools for services discovering (publish/subscribe paradigm) for coordinating any available Desktop Grid middleware user can participate to any Boinc, Condor, XtremWeb project (enhanced scalability ); 7 Christophe Cérin Sous thème grille Équipe AOC

8 Main objectives of BonjourGrid 8 / 113 Count on existing distributed tools for services discovering (publish/subscribe paradigm) for coordinating any available Desktop Grid middleware user can participate to any Boinc, Condor, XtremWeb project (enhanced scalability ); Design and implement a platform able to manage multiple instances of DG middleware; Reduce as much as possible the use of any central element; 8 Christophe Cérin Sous thème grille Équipe AOC

9 Main objectives of BonjourGrid 9 / 113 Count on existing distributed tools for services discovering (publish/subscribe paradigm) for coordinating any available Desktop Grid middleware user can participate to any Boinc, Condor, XtremWeb project (enhanced scalability ); Design and implement a platform able to manage multiple instances of DG middleware; Reduce as much as possible the use of any central element; Create a coordinator, on the fly, without any system administrator intervention; From a vision with a single coordinator towards a vision with multiple coordinators. 9 Christophe Cérin Sous thème grille Équipe AOC

10 10 / 113 Main objectives of BonjourGrid Count on existing distributed tools for services discovering (publish/subscribe paradigm) for coordinating any available Desktop Grid middleware user can participate to any Boinc, Condor, XtremWeb project (enhanced scalability ); Design and implement a platform able to manage multiple instances of DG middleware; Reduce as much as possible the use of any central element; Create a coordinator, on the fly, without any system administrator intervention; From a vision with a single coordinator towards a vision with multiple coordinators. Each coordinator searches, in a concurrent way, participants (idle machines) 10 Christophe Cérin Sous thème grille Équipe AOC

11 11 / 113 How BonjourGrid works 11 Christophe Cérin Sous thème grille Équipe AOC

12 12 / 113 How BonjourGrid works 12 Christophe Cérin Sous thème grille Équipe AOC

13 13 / 113 How BonjourGrid works 13 Christophe Cérin Sous thème grille Équipe AOC

14 14 / 113 How BonjourGrid works 14 Christophe Cérin Sous thème grille Équipe AOC

15 15 / 113 How BonjourGrid works 15 Christophe Cérin Sous thème grille Équipe AOC

16 16 / 113 How BonjourGrid works 16 Christophe Cérin Sous thème grille Équipe AOC

17 17 / 113 How BonjourGrid works 17 Christophe Cérin Sous thème grille Équipe AOC

18 18 / 113 How BonjourGrid works 18 Christophe Cérin Sous thème grille Équipe AOC

19 19 / 113 How BonjourGrid works 19 Christophe Cérin Sous thème grille Équipe AOC

20 20 / 113 How BonjourGrid works 20 Christophe Cérin Sous thème grille Équipe AOC

21 21 / 113 How BonjourGrid works 21 Christophe Cérin Sous thème grille Équipe AOC

22 22 / 113 How BonjourGrid works 22 Christophe Cérin Sous thème grille Équipe AOC

23 23 / 113 How BonjourGrid works 23 Christophe Cérin Sous thème grille Équipe AOC

24 24 / 113 How BonjourGrid works 24 Christophe Cérin Sous thème grille Équipe AOC

25 25 / 113 How BonjourGrid works 25 Christophe Cérin Sous thème grille Équipe AOC

26 26 / 113 BonjourGrid vision The user requests for computation; 26 Christophe Cérin Sous thème grille Équipe AOC

27 27 / 113 BonjourGrid vision The user requests for computation; The user provides the control flow graph, binaries, input data; 27 Christophe Cérin Sous thème grille Équipe AOC

28 28 / 113 BonjourGrid vision The user requests for computation; The user provides the control flow graph, binaries, input data; The user deploys locally a coordinator and requests for participants; We support XtremWeb, Condor, Boinc. 28 Christophe Cérin Sous thème grille Équipe AOC

29 29 / 113 BonjourGrid vision The user requests for computation; The user provides the control flow graph, binaries, input data; The user deploys locally a coordinator and requests for participants; We support XtremWeb, Condor, Boinc. The coordinator selects a set of machines (criteria: RAM, CPU, costs... ) 29 Christophe Cérin Sous thème grille Équipe AOC

30 30 / 113 BonjourGrid vision The user requests for computation; The user provides the control flow graph, binaries, input data; The user deploys locally a coordinator and requests for participants; We support XtremWeb, Condor, Boinc. The coordinator selects a set of machines (criteria: RAM, CPU, costs... ) Upon completion, the coordinator returns to the idle state, slaves are freed and the coordination protocol: 30 Christophe Cérin Sous thème grille Équipe AOC

31 31 / 113 BonjourGrid vision The user requests for computation; The user provides the control flow graph, binaries, input data; The user deploys locally a coordinator and requests for participants; We support XtremWeb, Condor, Boinc. The coordinator selects a set of machines (criteria: RAM, CPU, costs... ) Upon completion, the coordinator returns to the idle state, slaves are freed and the coordination protocol: manages and controls resources, services and computing elements; does not depend on any specific machine nor any central element. 31 Christophe Cérin Sous thème grille Équipe AOC

32 32 / 113 BonjourGrid vision The user requests for computation; The user provides the control flow graph, binaries, input data; The user deploys locally a coordinator and requests for participants; We support XtremWeb, Condor, Boinc. The coordinator selects a set of machines (criteria: RAM, CPU, costs... ) Upon completion, the coordinator returns to the idle state, slaves are freed and the coordination protocol: manages and controls resources, services and computing elements; does not depend on any specific machine nor any central element. 32 Christophe Cérin Sous thème grille Équipe AOC

33 33 / 113 The protocol for resources discovering Based on Bonjour from Apple; 33 Christophe Cérin Sous thème grille Équipe AOC

34 34 / 113 The protocol for resources discovering Based on Bonjour from Apple; A publish/subscribe system is easy to use (toolbox = publish(), subscribe(), browse()) 34 Christophe Cérin Sous thème grille Équipe AOC

35 35 / 113 The protocol for resources discovering Based on Bonjour from Apple; A publish/subscribe system is easy to use (toolbox = publish(), subscribe(), browse()) Bonjour is dedicated to LAN (wide area Bonjour? We need some experiments) 35 Christophe Cérin Sous thème grille Équipe AOC

36 36 / 113 The protocol for resources discovering Based on Bonjour from Apple; A publish/subscribe system is easy to use (toolbox = publish(), subscribe(), browse()) Bonjour is dedicated to LAN (wide area Bonjour? We need some experiments) Concerning the Wide Area implementation: we can also think about Apache Kandula ( or even cisco Jabber protocol ( 36 Christophe Cérin Sous thème grille Équipe AOC

37 The protocol for resources discovering 37 / 113 Based on Bonjour from Apple; A publish/subscribe system is easy to use (toolbox = publish(), subscribe(), browse()) Bonjour is dedicated to LAN (wide area Bonjour? We need some experiments) Concerning the Wide Area implementation: we can also think about Apache Kandula ( or even cisco Jabber protocol ( Extensible Messaging and Presence Protocol (XMPP) (formerly named Jabber) is an open, XML-based protocol originally aimed at near-real-time, extensible instant messaging (IM) and presence information, but now expanded into the broader realm of message-oriented middleware 37 Christophe Cérin Sous thème grille Équipe AOC

38 The protocol for resources discovering 38 / 113 Based on Bonjour from Apple; A publish/subscribe system is easy to use (toolbox = publish(), subscribe(), browse()) Bonjour is dedicated to LAN (wide area Bonjour? We need some experiments) Concerning the Wide Area implementation: we can also think about Apache Kandula ( or even cisco Jabber protocol ( Extensible Messaging and Presence Protocol (XMPP) (formerly named Jabber) is an open, XML-based protocol originally aimed at near-real-time, extensible instant messaging (IM) and presence information, but now expanded into the broader realm of message-oriented middleware The current protocol has been developed/specified with ad-hoc methods we need to consolidate the trust (ongoing project to verify it, based on Colored Petri Nets) 38 Christophe Cérin Sous thème grille Équipe AOC

39 39 / 113 Fault Tolerance with BonjourGrid Intrinsic property of any large scale system; 39 Christophe Cérin Sous thème grille Équipe AOC

40 40 / 113 Fault Tolerance with BonjourGrid Intrinsic property of any large scale system; We assume that any coordinator is responsible for its FT (it manages the volatility of attached slaves) 40 Christophe Cérin Sous thème grille Équipe AOC

41 41 / 113 Fault Tolerance with BonjourGrid Intrinsic property of any large scale system; We assume that any coordinator is responsible for its FT (it manages the volatility of attached slaves) Our solution: tolerate the failure of coordinators 41 Christophe Cérin Sous thème grille Équipe AOC

42 42 / 113 Fault Tolerance with BonjourGrid Intrinsic property of any large scale system; We assume that any coordinator is responsible for its FT (it manages the volatility of attached slaves) Our solution: tolerate the failure of coordinators For any application we create and manage dynamically copies of the coordinator; We manage k copies; based on passive replication. When a service disappears: we added a special status flag to distinguish between end of the application / failure slaves can redirect the communication to a copy. 42 Christophe Cérin Sous thème grille Équipe AOC

43 43 / 113 Intensive Experiments BonjourGrid has been tested intensively: stressed scenario to more relaxing scenario 43 Christophe Cérin Sous thème grille Équipe AOC

44 44 / 113 Intensive Experiments BonjourGrid has been tested intensively: stressed scenario to more relaxing scenario in terms of #coordinator versus #nodes in terms of using virtual machines to reach 1000 nodes; in terms of comparing Boinc, Condor, XtremWeb over our protocol; in terms of robustness in supporting FT; 44 Christophe Cérin Sous thème grille Équipe AOC

45 45 / 113 Intensive Experiments BonjourGrid has been tested intensively: stressed scenario to more relaxing scenario in terms of #coordinator versus #nodes in terms of using virtual machines to reach 1000 nodes; in terms of comparing Boinc, Condor, XtremWeb over our protocol; in terms of robustness in supporting FT; Example Condor: 130 applications (2 to 128 // tasks), 200 nodes, application task: 1s to 500s. Result: with BonjourGrid, 35% of applications generate a delay of about 30s. 45 Christophe Cérin Sous thème grille Équipe AOC

46 46 / 113 Experiments: one example 46 Christophe Cérin Sous thème grille Équipe AOC

47 47 / 113 Publish-subscribe Unusual for programming a FT protocol 47 Christophe Cérin Sous thème grille Équipe AOC

48 48 / 113 Publish-subscribe Unusual for programming a FT protocol But the benefit: integrated code for both the control of the coordination and the FT management; 48 Christophe Cérin Sous thème grille Équipe AOC

49 49 / 113 Publish-subscribe Unusual for programming a FT protocol But the benefit: integrated code for both the control of the coordination and the FT management; Basic ideas/properties ot Event-based (pub-sub) systems: 49 Christophe Cérin Sous thème grille Équipe AOC

50 50 / 113 Publish-subscribe Unusual for programming a FT protocol But the benefit: integrated code for both the control of the coordination and the FT management; Basic ideas/properties ot Event-based (pub-sub) systems: Decomposing distributed work activities into a sequence of events; event notifications can decouple sources and sinks in space, time, and topology; provide a defined interface for the separation of concerns between different handlers; 50 Christophe Cérin Sous thème grille Équipe AOC

51 51 / 113 Publish-subscribe Unusual for programming a FT protocol But the benefit: integrated code for both the control of the coordination and the FT management; Basic ideas/properties ot Event-based (pub-sub) systems: Decomposing distributed work activities into a sequence of events; event notifications can decouple sources and sinks in space, time, and topology; provide a defined interface for the separation of concerns between different handlers; it is not sufficient that a user sits down at her terminal for an event to elicit reaction necessarily (event, notification) 51 Christophe Cérin Sous thème grille Équipe AOC

52 52 / 113 Publish-subscribe - event-based systems To afford a loosening of space and time considerations (for example, pick up a piece of luggage when it is seen accommodates a wide variety of conveyor belt speeds and bag sizes) 52 Christophe Cérin Sous thème grille Équipe AOC

53 53 / 113 Publish-subscribe - event-based systems To afford a loosening of space and time considerations (for example, pick up a piece of luggage when it is seen accommodates a wide variety of conveyor belt speeds and bag sizes) a loosening of semantic meanings (for example, buy this stock under any of a set best-buy criteria permits the decentralized definition of a set of action triggers such as price-to-earnings ratio or performance compared with an index fund) 53 Christophe Cérin Sous thème grille Équipe AOC

54 54 / 113 Publish-subscribe - event-based systems To afford a loosening of space and time considerations (for example, pick up a piece of luggage when it is seen accommodates a wide variety of conveyor belt speeds and bag sizes) a loosening of semantic meanings (for example, buy this stock under any of a set best-buy criteria permits the decentralized definition of a set of action triggers such as price-to-earnings ratio or performance compared with an index fund) a loosening of participant constraints (for example, any available authorized broker can purchase allows a pool of providers to enhance the recipients qualities of service) 54 Christophe Cérin Sous thème grille Équipe AOC

55 55 / 113 Publish-subscribe - event-based systems To afford a loosening of space and time considerations (for example, pick up a piece of luggage when it is seen accommodates a wide variety of conveyor belt speeds and bag sizes) a loosening of semantic meanings (for example, buy this stock under any of a set best-buy criteria permits the decentralized definition of a set of action triggers such as price-to-earnings ratio or performance compared with an index fund) a loosening of participant constraints (for example, any available authorized broker can purchase allows a pool of providers to enhance the recipients qualities of service) a loosening of the coordination between subsystems (for example, a decoupled billing module could know to deduct a calculated commission whenever it is notified of a stock-trade against an account) 55 Christophe Cérin Sous thème grille Équipe AOC

56 56 / 113 Publish-subscribe - event-based systems To afford a loosening of space and time considerations (for example, pick up a piece of luggage when it is seen accommodates a wide variety of conveyor belt speeds and bag sizes) a loosening of semantic meanings (for example, buy this stock under any of a set best-buy criteria permits the decentralized definition of a set of action triggers such as price-to-earnings ratio or performance compared with an index fund) a loosening of participant constraints (for example, any available authorized broker can purchase allows a pool of providers to enhance the recipients qualities of service) a loosening of the coordination between subsystems (for example, a decoupled billing module could know to deduct a calculated commission whenever it is notified of a stock-trade against an account) 56 Christophe Cérin Sous thème grille Équipe AOC

57 57 / 113 Publish-subscribe - event-based systems Such mechanisms reduce the cost of system evolution, because with loosely coupled components, it is possible to integrate new components without affecting the components that implicitly invoke the new components [Garlan and Notkin, 1991] 57 Christophe Cérin Sous thème grille Équipe AOC

58 58 / 113 Publish-subscribe - event-based systems Such mechanisms reduce the cost of system evolution, because with loosely coupled components, it is possible to integrate new components without affecting the components that implicitly invoke the new components [Garlan and Notkin, 1991] 58 Christophe Cérin Sous thème grille Équipe AOC

59 59 / 113 Publish-subscribe - event-based systems Such mechanisms reduce the cost of system evolution, because with loosely coupled components, it is possible to integrate new components without affecting the components that implicitly invoke the new components [Garlan and Notkin, 1991] Today needs: Messaging, Presence, Conferencing, Simulation/graphics, Tool integration (compose software components by orchestrating tool invocations and data flow between participants: JavaScript only?) 59 Christophe Cérin Sous thème grille Équipe AOC

60 60 / 113 Publish-subscribe - event-based systems Such mechanisms reduce the cost of system evolution, because with loosely coupled components, it is possible to integrate new components without affecting the components that implicitly invoke the new components [Garlan and Notkin, 1991] Today needs: Messaging, Presence, Conferencing, Simulation/graphics, Tool integration (compose software components by orchestrating tool invocations and data flow between participants: JavaScript only?) 60 Christophe Cérin Sous thème grille Équipe AOC

61 61 / 113 Publish-subscribe - event-based systems Such mechanisms reduce the cost of system evolution, because with loosely coupled components, it is possible to integrate new components without affecting the components that implicitly invoke the new components [Garlan and Notkin, 1991] Today needs: Messaging, Presence, Conferencing, Simulation/graphics, Tool integration (compose software components by orchestrating tool invocations and data flow between participants: JavaScript only?) If there is an intermediator, there is a tradeoff between single-hop and creating a spanning tree of mail, news, or chat servers. (see the naming model) 61 Christophe Cérin Sous thème grille Équipe AOC

62 62 / 113 Publish-subscribe - event-based systems Such mechanisms reduce the cost of system evolution, because with loosely coupled components, it is possible to integrate new components without affecting the components that implicitly invoke the new components [Garlan and Notkin, 1991] Today needs: Messaging, Presence, Conferencing, Simulation/graphics, Tool integration (compose software components by orchestrating tool invocations and data flow between participants: JavaScript only?) If there is an intermediator, there is a tradeoff between single-hop and creating a spanning tree of mail, news, or chat servers. (see the naming model) 62 Christophe Cérin Sous thème grille Équipe AOC

63 63 / 113 Publish-subscribe - event-based systems Such mechanisms reduce the cost of system evolution, because with loosely coupled components, it is possible to integrate new components without affecting the components that implicitly invoke the new components [Garlan and Notkin, 1991] Today needs: Messaging, Presence, Conferencing, Simulation/graphics, Tool integration (compose software components by orchestrating tool invocations and data flow between participants: JavaScript only?) If there is an intermediator, there is a tradeoff between single-hop and creating a spanning tree of mail, news, or chat servers. (see the naming model) Today new Needs: with ramifications for issues such as ontology, security, mobility, and reliability(?) 63 Christophe Cérin Sous thème grille Équipe AOC

64 64 / 113 Publish-subscribe - event-based systems Such mechanisms reduce the cost of system evolution, because with loosely coupled components, it is possible to integrate new components without affecting the components that implicitly invoke the new components [Garlan and Notkin, 1991] Today needs: Messaging, Presence, Conferencing, Simulation/graphics, Tool integration (compose software components by orchestrating tool invocations and data flow between participants: JavaScript only?) If there is an intermediator, there is a tradeoff between single-hop and creating a spanning tree of mail, news, or chat servers. (see the naming model) Today new Needs: with ramifications for issues such as ontology, security, mobility, and reliability(?) 64 Christophe Cérin Sous thème grille Équipe AOC

65 65 / 113 Publish-subscribe - event-based systems The Many Faces of Publish/Subscribe [EUGSTER, FELBER, GUERRAOUI, KERMARREC]: the common denominator of flavors is the full decoupling of the communicating entities in time, space, and synchronization between publishers and subscribers. 65 Christophe Cérin Sous thème grille Équipe AOC

66 66 / 113 Publish-subscribe - event-based systems The Many Faces of Publish/Subscribe [EUGSTER, FELBER, GUERRAOUI, KERMARREC]: the common denominator of flavors is the full decoupling of the communicating entities in time, space, and synchronization between publishers and subscribers. 66 Christophe Cérin Sous thème grille Équipe AOC

67 67 / 113 Publish-subscribe - event-based systems The Many Faces of Publish/Subscribe [EUGSTER, FELBER, GUERRAOUI, KERMARREC]: the common denominator of flavors is the full decoupling of the communicating entities in time, space, and synchronization between publishers and subscribers. Decoupling production and consumption = increases scalability (less dependencies between participants) 67 Christophe Cérin Sous thème grille Équipe AOC

68 68 / 113 Publish-subscribe VS msg passing, RPC, shared spaces RPC (remote invocation, 1990 Java RMI, CORBA) 68 Christophe Cérin Sous thème grille Équipe AOC

69 69 / 113 Publish-subscribe VS msg passing, RPC, shared spaces RPC (remote invocation, 1990 Java RMI, CORBA) 69 Christophe Cérin Sous thème grille Équipe AOC

70 70 / 113 Publish-subscribe VS msg passing, RPC, shared spaces RPC (remote invocation, 1990 Java RMI, CORBA) The synchronous nature of RPC introduces a strong time, synchronization (on the consumer side1), and also space coupling (since an invoking object holds a remote reference to each of its invokees). 70 Christophe Cérin Sous thème grille Équipe AOC

71 71 / 113 Publish-subscribe VS msg passing, RPC, shared spaces RPC (remote invocation, 1990 Java RMI, CORBA) The synchronous nature of RPC introduces a strong time, synchronization (on the consumer side1), and also space coupling (since an invoking object holds a remote reference to each of its invokees). SOA = RPC not good for scalability, not good for Clouds (main idea) 71 Christophe Cérin Sous thème grille Équipe AOC

72 72 / 113 Towards PaaS and Clouds The new context: Platform as a Service and Cloud Outsourcing of software resources (Google word/spreadsheet online) and hardware resources (Amazon EC2); 72 Christophe Cérin Sous thème grille Équipe AOC

73 73 / 113 Towards PaaS and Clouds The new context: Platform as a Service and Cloud Outsourcing of software resources (Google word/spreadsheet online) and hardware resources (Amazon EC2); A variant: Platform as a Service (PaaS) where users also inherit from a complete development infrastructure (based on Javascript for the future?); 73 Christophe Cérin Sous thème grille Équipe AOC

74 74 / 113 Towards PaaS and Clouds The new context: Platform as a Service and Cloud Outsourcing of software resources (Google word/spreadsheet online) and hardware resources (Amazon EC2); A variant: Platform as a Service (PaaS) where users also inherit from a complete development infrastructure (based on Javascript for the future?); No hosting problem for the user; 74 Christophe Cérin Sous thème grille Équipe AOC

75 75 / 113 Towards PaaS and Clouds The new context: Platform as a Service and Cloud Outsourcing of software resources (Google word/spreadsheet online) and hardware resources (Amazon EC2); A variant: Platform as a Service (PaaS) where users also inherit from a complete development infrastructure (based on Javascript for the future?); No hosting problem for the user; No update problem for the user (he always uses the last release); 75 Christophe Cérin Sous thème grille Équipe AOC

76 76 / 113 Towards PaaS and Clouds The new context: Platform as a Service and Cloud Outsourcing of software resources (Google word/spreadsheet online) and hardware resources (Amazon EC2); A variant: Platform as a Service (PaaS) where users also inherit from a complete development infrastructure (based on Javascript for the future?); No hosting problem for the user; No update problem for the user (he always uses the last release); No maintenance, no local storage. 76 Christophe Cérin Sous thème grille Équipe AOC

77 77 / 113 Towards PaaS and Clouds The new context: Platform as a Service and Cloud Outsourcing of software resources (Google word/spreadsheet online) and hardware resources (Amazon EC2); A variant: Platform as a Service (PaaS) where users also inherit from a complete development infrastructure (based on Javascript for the future?); No hosting problem for the user; No update problem for the user (he always uses the last release); No maintenance, no local storage. We have started an initiative for defining and designing a general purpose PaaS based on distributed protocols for coordination and data exchange. 77 Christophe Cérin Sous thème grille Équipe AOC

78 78 / 113 Architecture overview of the PaaSoordinated project 78 Christophe Cérin Sous thème grille Équipe AOC

79 79 / 113 Key points regarding Philosophy Different instances (say, of a database) want to exchange data temporally an open protocol does not capture the user 79 Christophe Cérin Sous thème grille Équipe AOC

80 80 / 113 Key points regarding Philosophy Different instances (say, of a database) want to exchange data temporally an open protocol does not capture the user Different instances (say, of an ERP) do public announcements to search for providers, then explore HTML links (interrogate different Clouds able to answer) an open protocol do not capture the user, again 80 Christophe Cérin Sous thème grille Équipe AOC

81 81 / 113 Key points regarding Philosophy Different instances (say, of a database) want to exchange data temporally an open protocol does not capture the user Different instances (say, of an ERP) do public announcements to search for providers, then explore HTML links (interrogate different Clouds able to answer) an open protocol do not capture the user, again So, we want open protocols to coordinate and to exchange data! 81 Christophe Cérin Sous thème grille Équipe AOC

82 82 / 113 Key points regarding Philosophy Different instances (say, of a database) want to exchange data temporally an open protocol does not capture the user Different instances (say, of an ERP) do public announcements to search for providers, then explore HTML links (interrogate different Clouds able to answer) an open protocol do not capture the user, again So, we want open protocols to coordinate and to exchange data! MORE FREEDOM 82 Christophe Cérin Sous thème grille Équipe AOC

83 83 / 113 Some Challenges Where to insert the different connectors in the PaaS software stack to get an open infrastructure? Figure: Software stack in PaaS (source: Coghead) 83 Christophe Cérin Sous thème grille Équipe AOC

84 84 / 113 Research opportunities Above the Clouds: A Berkeley View of Cloud Computing Availability of a service mastering FT redundancy; 84 Christophe Cérin Sous thème grille Équipe AOC

85 85 / 113 Research opportunities Above the Clouds: A Berkeley View of Cloud Computing Availability of a service mastering FT redundancy; Data Lock-In: API must not be proprietary but should rely on open standards; 85 Christophe Cérin Sous thème grille Équipe AOC

86 86 / 113 Research opportunities Above the Clouds: A Berkeley View of Cloud Computing Availability of a service mastering FT redundancy; Data Lock-In: API must not be proprietary but should rely on open standards; Data Transfer Bottlenecks use P2P techniques; 86 Christophe Cérin Sous thème grille Équipe AOC

87 87 / 113 Research opportunities Above the Clouds: A Berkeley View of Cloud Computing Availability of a service mastering FT redundancy; Data Lock-In: API must not be proprietary but should rely on open standards; Data Transfer Bottlenecks use P2P techniques; Bugs in Large-Scale Distributed Systems use Formal Methods to specify and to analyse architecture and protocols; 87 Christophe Cérin Sous thème grille Équipe AOC

88 88 / 113 Research opportunities Above the Clouds: A Berkeley View of Cloud Computing Availability of a service mastering FT redundancy; Data Lock-In: API must not be proprietary but should rely on open standards; Data Transfer Bottlenecks use P2P techniques; Bugs in Large-Scale Distributed Systems use Formal Methods to specify and to analyse architecture and protocols; Scaling Quickly instanciate new PaaS on the fly define a model for cooperation and interaction; 88 Christophe Cérin Sous thème grille Équipe AOC

89 89 / 113 Research opportunities Above the Clouds: A Berkeley View of Cloud Computing Availability of a service mastering FT redundancy; Data Lock-In: API must not be proprietary but should rely on open standards; Data Transfer Bottlenecks use P2P techniques; Bugs in Large-Scale Distributed Systems use Formal Methods to specify and to analyse architecture and protocols; Scaling Quickly instanciate new PaaS on the fly define a model for cooperation and interaction; 89 Christophe Cérin Sous thème grille Équipe AOC

90 90 / 113 State of the Art Similar projects Vertebra ( Service-Oriented-Architecture for the cloud, an application deployment platform based on Ruby and Erlang. The project moves to On-demand deployment and management of your Ruby on Rails applications with Engine Yard Cloud One-click code deploys, application cloning, data automation Christophe Cérin Sous thème grille Équipe AOC

91 91 / 113 State of the Art Similar projects Vertebra ( Service-Oriented-Architecture for the cloud, an application deployment platform based on Ruby and Erlang. The project moves to On-demand deployment and management of your Ruby on Rails applications with Engine Yard Cloud One-click code deploys, application cloning, data automation... Kandula ( Presently Kandula implements WS-Coordination and WS-AtomicTransaction protocols. WS-BusinessActivity protocol will be available in the near future. Kandula project has 2 branches. Kandula1 branch runs on Apache Axis 1.x. Kandula2 branch (new) runs on Apache Axis2 91 Christophe Cérin Sous thème grille Équipe AOC

92 92 / 113 State of the Art Similar projects Vertebra ( Service-Oriented-Architecture for the cloud, an application deployment platform based on Ruby and Erlang. The project moves to On-demand deployment and management of your Ruby on Rails applications with Engine Yard Cloud One-click code deploys, application cloning, data automation... Kandula ( Presently Kandula implements WS-Coordination and WS-AtomicTransaction protocols. WS-BusinessActivity protocol will be available in the near future. Kandula project has 2 branches. Kandula1 branch runs on Apache Axis 1.x. Kandula2 branch (new) runs on Apache Axis2 92 Christophe Cérin Sous thème grille Équipe AOC

93 93 / 113 State of the Art Similar projects Axis ( Axis2 is a Web Services / SOAP / WSDL engine, the successor to the widely used Apache Axis SOAP stack. There are two implementations of the Apache Axis2 Web services engine - Apache Axis2/Java and Apache Axis2/C 93 Christophe Cérin Sous thème grille Équipe AOC

94 94 / 113 State of the Art Similar projects Axis ( Axis2 is a Web Services / SOAP / WSDL engine, the successor to the widely used Apache Axis SOAP stack. There are two implementations of the Apache Axis2 Web services engine - Apache Axis2/Java and Apache Axis2/C Corona: A High Performance Publish-Subscribe System for the World ( Topic based publish subscribe system for the Web. URLs of Web content serve as topics or channels. Any Web object identifiable by an URL can be monitored with Corona 94 Christophe Cérin Sous thème grille Équipe AOC

95 95 / 113 State of the Art Similar projects Axis ( Axis2 is a Web Services / SOAP / WSDL engine, the successor to the widely used Apache Axis SOAP stack. There are two implementations of the Apache Axis2 Web services engine - Apache Axis2/Java and Apache Axis2/C Corona: A High Performance Publish-Subscribe System for the World ( Topic based publish subscribe system for the Web. URLs of Web content serve as topics or channels. Any Web object identifiable by an URL can be monitored with Corona UDDI, XMMP Christophe Cérin Sous thème grille Équipe AOC

96 96 / 113 State of the Art Similar projects Axis ( Axis2 is a Web Services / SOAP / WSDL engine, the successor to the widely used Apache Axis SOAP stack. There are two implementations of the Apache Axis2 Web services engine - Apache Axis2/Java and Apache Axis2/C Corona: A High Performance Publish-Subscribe System for the World ( Topic based publish subscribe system for the Web. URLs of Web content serve as topics or channels. Any Web object identifiable by an URL can be monitored with Corona UDDI, XMMP Christophe Cérin Sous thème grille Équipe AOC

97 97 / 113 State of the Art Pieces of the maze (not exhaustive) Data exchange: Bitdew (no comment). Comment: have a look to SyncML Protocol ( This open standard seeks to drive data mobility by establishing a common language for communications among devices, applications, and networks. 97 Christophe Cérin Sous thème grille Équipe AOC

98 98 / 113 State of the Art Pieces of the maze (not exhaustive) Data exchange: Bitdew (no comment). Comment: have a look to SyncML Protocol ( This open standard seeks to drive data mobility by establishing a common language for communications among devices, applications, and networks. TioLive ( Open source by Nexedi Corp. for Communication: , telephone, chat, Backoffice: contacts, documents, accounting, ERP, CRM, e-business: web site, e-commerce: 98 Christophe Cérin Sous thème grille Équipe AOC

99 99 / 113 State of the Art Pieces of the maze (not exhaustive) Data exchange: Bitdew (no comment). Comment: have a look to SyncML Protocol ( This open standard seeks to drive data mobility by establishing a common language for communications among devices, applications, and networks. TioLive ( Open source by Nexedi Corp. for Communication: , telephone, chat, Backoffice: contacts, documents, accounting, ERP, CRM, e-business: web site, e-commerce: 99 Christophe Cérin Sous thème grille Équipe AOC

100 100 / 113 State of the Art Pieces of the maze (not exhaustive) Programming: To.Become.ERP5.Developer/view OSOE course: TioLive tutorial: Documentation for developers: section/developer/ section/developer/developer-technology/view section/developer/ enterprise-high..performance.zope/view 100 Christophe Cérin Sous thème grille Équipe AOC

101 101 / 113 The Free Cloud Alliance A French innitiative by small/medium size companies involved in Open Source Projects; 101 Christophe Cérin Sous thème grille Équipe AOC

102 102 / 113 The Free Cloud Alliance A French innitiative by small/medium size companies involved in Open Source Projects; Who and What: BS2 storage, IaaS, Data center, Internet connectivity Xtreem Storage Server, GNU Linux (server, desktop, netpc); TioLive (SaaS/PaaS), NEO Transactionnal cloud Storage, SLAP (Accounting and Provisionning), ERP5 ERP/CRM/KM SaaS/KM PaaS 102 Christophe Cérin Sous thème grille Équipe AOC

103 103 / 113 Nexedi projects Special focus on SLAP Simple Languages for Accounting and Provisioning (SLAP) leverages and extends buildout technology to install any kind of self contained Cloud Computing containers on any operating system. Buildout technology ( Python-based build system for creating, assembling and deploying applications from multiple parts, some of which may be non-python-based. It lets you create a buildout configuration and reproduce the same software later. SLAP Cloud Engine leverages open source ERP5 to provide accounting and billing of allocated Cloud resources; 103 Christophe Cérin Sous thème grille Équipe AOC

104 104 / 113 Nexedi projects Special focus on SLAP Key Applications: Automated Datacenter Management; Cloud Computing Resource Allocation, Optimization and Exchange; Accounting and Billing. Strength: Compatible with any software technology and any OS; Built-in CRM and ERP; 104 Christophe Cérin Sous thème grille Équipe AOC

105 105 / 113 Nexedi SlapGrid software component of TioLive Grid or grid.tiolive.com role: protocole for the coordination of VM Slapgrid Cloud Enginer combines desktop grid computing and cloud computing to distribute Cloud Compputing resources over multiple suppliers of raw computing power, either in Data Centers or at home. Slapgrid also includes billing and provisionning features. 105 Christophe Cérin Sous thème grille Équipe AOC

106 Conclusion Hope 106 / 113 DG has proved to be relevant for resource sharing transpose this success story to the Cloud and PaaS universes offer a technical alternate to Google, Salesforce, Amazon big farm of servers 106 Christophe Cérin Sous thème grille Équipe AOC

107 Conclusion Hope 107 / 113 DG has proved to be relevant for resource sharing transpose this success story to the Cloud and PaaS universes offer a technical alternate to Google, Salesforce, Amazon big farm of servers Our approaches are based on emerging open source Cloud solution. From an economic point of view: if it is less expensive to host services locally and if it offers more advantages (we are not dependant on a technology no prison, more potential partners), then small/medium size companies will adopt our approaches; 107 Christophe Cérin Sous thème grille Équipe AOC

108 Conclusion Hope 108 / 113 DG has proved to be relevant for resource sharing transpose this success story to the Cloud and PaaS universes offer a technical alternate to Google, Salesforce, Amazon big farm of servers Our approaches are based on emerging open source Cloud solution. From an economic point of view: if it is less expensive to host services locally and if it offers more advantages (we are not dependant on a technology no prison, more potential partners), then small/medium size companies will adopt our approaches; Christophe Cérin Sous thème grille Équipe AOC

109 Conclusion Hope 109 / 113 DG has proved to be relevant for resource sharing transpose this success story to the Cloud and PaaS universes offer a technical alternate to Google, Salesforce, Amazon big farm of servers Our approaches are based on emerging open source Cloud solution. From an economic point of view: if it is less expensive to host services locally and if it offers more advantages (we are not dependant on a technology no prison, more potential partners), then small/medium size companies will adopt our approaches; Main change: accept to manage redundancy, scaling the server (even for temporary needs), synchronisation coordination with grid technology (BonjourGrid, PastryGrid?); 109 Christophe Cérin Sous thème grille Équipe AOC

110 110 / 113 Hope Conclusion Benefit: less expensive (comparing to Amazone EC2) because you control your data 110 Christophe Cérin Sous thème grille Équipe AOC

111 111 / 113 Hope Conclusion Benefit: less expensive (comparing to Amazone EC2) because you control your data Could also be implemented and deployed with a centralized Web Services for each application and for each Cloud type, you need a specific coordination protocol single point of failure. 111 Christophe Cérin Sous thème grille Équipe AOC

112 112 / 113 Hope Conclusion Benefit: less expensive (comparing to Amazone EC2) because you control your data Could also be implemented and deployed with a centralized Web Services for each application and for each Cloud type, you need a specific coordination protocol single point of failure. Ex: a company wants to install the Virtual Desktop EyeOS and the TioLive/ERP5 PaaS. During the night, the company rents different services: one (company) to many many (services) to many (companies) = new abilities, new business! demonstrate that a single coordination protocol is better than configuring as many middlewares than we have software! 112 Christophe Cérin Sous thème grille Équipe AOC

113 113 / 113 From Grids to Clouds to PaaS: new research challenges AOC Team Christophe Cérin 1 1 Université de Paris XIII, CNRS UMR 7030, France Inria - universités tunisiennes workshop 113 Christophe Cérin Sous thème grille Équipe AOC