Decentralized Control of Large-Scale Distributed System

Transcription

1 Decentralized Control of Large-Scale Distributed System Mario Südholt Département Informatique, Mines Nantes IMT Colloquium; March 26, 2014

2 Outline Large-scale software systems 1 Large-scale software systems 2 Improving distributed control in the Cloud The Discovery initiative Capacity planning Advanced choreographies for service compositions Protocols and distributed property enforcement Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 2 / 24

3 Large-scale software systems Networks and systems Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 3 / 24

4 Large-scale software systems Networks and systems Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 3 / 24

5 Large-scale software systems Networks and systems Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 3 / 24

6 Large-scale software systems Networks and systems Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 3 / 24

7 Large-scale software systems Networks and systems Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 3 / 24

8 Large-scale software systems Large-scale software systems Large-scale software systems (we are interested in) The Web The Cloud Major questions How are they built and coordinated? New architectures and implementation mechanisms? How to ensure availability, correctness and security? Handle cross-domain functionalities (across technical and policy domains) Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 4 / 24

9 The Web Large-scale software systems Basic model: distributed coordination of services Loose coupling (Some) Well-defined, standardized interfaces REST interfaces Centralization often arises: popular services, service orchestration... Correctness, security Standards, protocols for low-level properties But: frequent violation of high-level properties Ex.: social cross-site forgery (S-CSRF) attacks Support for distributed property enforcement? Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 5 / 24

10 The Cloud Large-scale software systems Mutualize resources required by many users Many types Public (Cloudwatt, Numergy, Amazon, Google, Microsoft,...) Private, community, hybrid clouds Different service levels: IaaS, PaaS, SaaS Homogeneous environment Hardware: datacenters (up to hundreds of thousands of servers) Software: virtual environments Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 6 / 24

11 Cloud federations Large-scale software systems Mutualize resources among one or several providers Scale cloud services over geographic regions Significant centralization Problems Availability Connectivity Energy consumption ("ice clouds") Legal issues (data privacy) Alice Duke Paula Users energy footprint Bob Internet backbone Tom Dan Rob Charles Sam Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 7 / 24

12 Large-scale software systems Issues with centralized control Recap: centralized control in large-scale infrastructures Cloud: significant centralization Web: access to special servers, service orchestration Scaling issues, legal issues Issues on the system architecture level Ex.: datacenters at the edge of the backbone Implementation-level issues Ex.: centralized capacity planning in datacenters Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 8 / 24

13 Outline Improving distributed control in the Cloud 1 Large-scale software systems 2 Improving distributed control in the Cloud The Discovery initiative Capacity planning Advanced choreographies for service compositions Protocols and distributed property enforcement Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 9 / 24

14 Improving distributed control in the Cloud The Discovery initiative 1. The Discovery initiative: architecture A new architectural principle: from federated clouds... Bob Dan Alice Charles Duke Internet backbone Sam Paula Users energy footprint Tom Rob Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 10 / 24

15 DISCOVERY Network DISCOVERY Network DISCOVERY Network DISCOVERY Network DISCOVERY Network Improving distributed control in the Cloud The Discovery initiative 1. The Discovery initiative: architecture A new architectural principle:...to cooperative clouds Bob Dan Alice Charles Duke DISCOVERY Network DISCOVERY Network Sam - - Paula Users energy footprint Tom Rob Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 10 / 24

16 Improving distributed control in the Cloud Main characteristics The Discovery initiative Cooperative and autonomous management of virtual environments Manipulate virtual environments like processes in traditional OSes Localization of data and computations Key to efficiency and sustainability Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 11 / 24

17 Improving distributed control in the Cloud The Discovery initiative Locality in backbones (ex. Renater) Network state on 17 May 13 Underutilized links Redundancy Evolves in terms of points-of-presence (PoP) Potential for "close" Clouds Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 12 / 24

18 Improving distributed control in the Cloud The Discovery initiative Distributed cooperative clouds (ex. Renater) 12 A.Le bre et al. Close deployment to network infrastructure hal , version 1-26 Aug 2013 Extend network hubs with servers Dedicated to VM hosting Proportional to PoP s size Mario Südholt (IMT, Mines Nantes) Figure 2: Overlay local groups on top of the RENATER platform. in terms of number of bridge nodes to go through. Hence, the system will be able to route quickly between close groups. The routing of requests between far groups will be based on a random decision when no information are available, but oriented by the aim of going away from the request s source. This overlay will provide the basic building block of the platform, on which will which 26/3/14 are described in13 the/ 24 Decentralized control...rely higher level overlays and functionalities, IMT Coll.,

19 Improving distributed control in the Cloud Capacity planning 2. Capacity planning: virtual machines Virtual machines: software emulation of a computer Advantages Isolation Snapshotting Suspend/resume Fast live migration in a datacenter Downtime: ca. 60ms But: migration plans for large sets of VMs are costly Crucial for handling over-/underutilization Migration across datacenters? Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 14 / 24

20 VM scheduling Improving distributed control in the Cloud Capacity planning Objective: autonomously manage millions of VMs on tens of thousands of machines Limitations of current approaches because of centralization Reactivity and scalability Fault-tolerance (single point of failure) Service node Worker node Communication between nodes 1. Monitoring 3. Applying schedule 2. Computing schedule Discovery also needs new VM scheduling strategy Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 15 / 24

21 Improving distributed control in the Cloud Distributed VM scheduling Capacity planning DVMS alg.: first fully decentralized algorithm Nodes have a local view of the system Cooperation between direct neighbors to solve scheduling events Validation [Quesnel et al.: CCPE 12]} In vivo (on Grid5000): ca. 500 physical machines, 4500 VMs Simulation (using Simgrid): ca. 10K PMs, 80K VMs Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 16 / 24

22 Improving distributed control in the Cloud Advanced choreographies for service compositions 3. Management of service compositions Service compositions (e.g., for business processes) Composition programs (not manageable on large-scale) Declarative definitions: orchestrations, choreographies Service orchestration (e.g., using BPEL) Central chef d orchestra Subject to scalability issues (availability, lack of autonomy,...) S B:Item Service choreography No central orchestrator Correct implementation? Properties? B S:Sale B C:Purchase Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 17 / 24

23 Improving distributed control in the Cloud Advanced choreographies for service compositions Service choreographies with session types Session types: type-based fully distributed choreographies Global types define an interaction as a whole Projection: compilation to correct decentralized implementation Guarantee correctness properties No messages send at wrong times to wrong receiver No deadlocks From 1998 (researchers from Imperial College L., U Lisbon) Multi-party session types [POPL 08] Session-types with roles [POPL 12] Extension by security properties [Concur 12] Problems Forbidden functionalities: no race conditions Extensive rewrites for adding functionalities Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 18 / 24

24 Improving distributed control in the Cloud Aspectual session types Advanced choreographies for service compositions Extension [Tabareau et al.: Modularity 14] Larger set of functionalities (admit some race conditions) Simple and declarative adding B S:* + B C:* S B:Item + S B:Item proceed B S:* + B C:* B A:Auth A B:Retry B S:Sale B C:Purchase + + C B:Counter B C:Offer proceed + A B:Ok B L:LogData proceed a) Trade session b) Negotiation aspect c) Logging aspect d) Authentication aspect Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 19 / 24

25 1. Display UA Improving distributed control in the Cloud 4. Protocol adaptation Protocols and distributed property enforcement Ex. OAuth 2.0 Framework for the authorization of resource accesses Access by third parties without original credentials Used by all major Web, Cloud and software editors companies Facebook, Google, Microsoft, SAP... Third-Party Client (TC) User Agent (UA) 5. Grant Access 2. Fill authentication credential 4. Grant Access 3. Request Access Alice OAuth 2.0 Provider (OP) Main OAuth protocol flow Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 20 / 24

26 1. Display UA 11. Alice's Authz Code Improving distributed control in the Cloud OAuth single sign-on attacks Protocols and distributed property enforcement Use access token to access data Cloud Storage/ Identity Server (RS) Third-Party App (TA) 5. Alice's Authz Code 2. Fill authentication credential Alice Holds Alice's Authz Code from TA Chuck Initially already authenticated 7. Fill authentication credential 10. Rewrite Chuck's Authz Code with Alice's Authz Code Another Third-Party App (ATA) 6. Display UA User Agent (UA) 3. Request Access OAuth 2.0 Provider (OP) 8. Request Access User Agent (UA) 4. Alice's Authz Code 9. Chuck's Authz Code (a) Alice Authorization/Authentication with OAuth (b) Alice Session Swapping while OAuth Authentication New types of distributed attacks Single sign-on (SSO), social cross-site request forgery (S-CSRF) May involve one instance of an OAuth protocol May include several instances Problem: OAuth is a framework not a protocol Right usage has to be enforced Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 21 / 24

27 Improving distributed control in the Cloud Protocols and distributed property enforcement Distributed transformation of protocols Modifications to the protocol flows needed Dynamic modifications Over different steps/different instances of the protocols Over different levels of the software stack Ex.: session identification, state introduction Approach [Cherrueau et al.: CoudCom 13] Domain-specific framework/protocol transformation language Invasive but controled transformation of service compositions and implementations Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 22 / 24

28 Conclusion Improving distributed control in the Cloud Protocols and distributed property enforcement Centralized control (still) common and problematic Cloud architectures, capacity management, service orchestrations Discovery initiative for a cloud architecture Cooperative Clouds close to users Interest by large players: Renater, Orange... New distributed algorithms and tools for VM scheduling, service choreographies, protocol manipulations Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 23 / 24

29 Improving distributed control in the Cloud Thank you for your attention! Protocols and distributed property enforcement Questions? Further information: Ascola research team: Discovery initiative: (PI) Mario Südholt (IMT, Mines Nantes) Decentralized control... IMT Coll., 26/3/14 24 / 24