COMP4442. Service and Cloud Computing. Lecture 04: OpenStack. Prof. George Baciu PQ838.

Transcription

1 COMP4442 Service and Cloud Computing Lecture 04: OpenStack Prof. George Baciu PQ838 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

2 OpenStack 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

3 Contents What is OpenStack? A brief history The OpenStack Foundation What is a Design Guide? Guide methodology How to use it Cloud Architecture Customization 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

4 Recall the Cloud Stack Model Application Data Runtime Middleware OS Virtualization Servers Storage Networking Hardware 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

5 What is OpenStack? OpenStack is a cloud operating system that controls large pools of compute, storage and networking resources throughout that gives administrators control while empowering their users to provision resources through a web interface. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

6 What is OpenStack? 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

7 A Brief History In Oct Rackspace invests $15M to acquire two companies: Slicehost (Virtual Machine provider) JungleDisk (on line backup service) Rackspace then develops Cloud Files 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

8 A Brief History In July 2010 Rackspace and NASA merged two projects: 1. Swift object storage (based on Rackspace's Cloud Files platform) 2. Nova based on Nebula compute platform (NASA) Fastest growing Open Source project to date! 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

9 A Brief History In Sep OpenStack Foundation is established to promote the architecture design and implementation More than 500 companies have joined AppFormix, Arista Networks, AT&T, AMD, Intel, Juniper Avaya, Brocade, Canonical, Cisco, Citrix, EMC Ericsson, Fujitsu, Go Daddy, Google, HP, Huawei, IBM 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

10 Strong Community Support Contributions 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

11 OpenStack Versions OpenStack versions: A new version every 6 months! Most recent Jan X X X X X X X X X X X X Queens Pike Okata Newton Mitaka Liberty Kilo Juno Icehouse Havana Grizzly Folsom Essex Cactus Bexar Austin First release Nov /5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

12 OpenStack by Numbers Over 1202 contributors Top 20 users: Samsung, Yahoo Over 350 features Top 20 companies committing code: Red Hat, IBM, HP, RackSpace, Mirantis, Suse OpenStack Foundation: enovance, VMWare, Intel 16 Languages Over 3,000 bug fixes 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

13 OpenStack Foundation An Open Source Foundation Serves over 30,000 individual members in over 170 countries Goal: provide a set of shared resources to grow public and private OpenStack clouds 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

14 OpenStack Foundation Technical Committee OpenStack Technical Committee members define and steward the technical direction of OpenStack software, including crossprogram issues. The committee of 13 is fully elected by the project s Active Technical Contributors 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

15 OpenStack Foundation Board of Directors Provides strategic and financial oversight of Foundation resources and staff. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

16 OpenStack Foundation Board of Directors Alan Clark, Director of Industry Initiatives, Emerging Standards and Open Source at SUSE, was elected Chairman of the Board 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

17 OpenStack Foundation Committees Technical Committee User Committee 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

18 OpenStack Foundation Home of the OpenStack Project Uses the Apache licensing model Serves developers, users, and the entire ecosystem by providing: Shared resources Enable technology vendors targeting the platform Individual membership is free and accessible to anyone 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

19 OpenStack DefCore Committee working on OpenStack standardization and shared coding library Agile documentation Agile software Large testing group 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

20 Conceptual Architecture OpenStack architecture is designed to: deliver massively scalable cloud OS make constituent services work together in order to provide a complete IaaS solution Integration is facilitated through public application programming interfaces (APIs) 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

21 High Level Conceptual Architecture Compute Storage Networking 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

22 Main Service Modules Service Name Function Dashboard Horizon Provides a web front end Compute Nova Stores and retrieves virtual disks ("images") and associated metadata in Image ("Glance") Network Neutron Provides virtual networking for Compute. Block Storage Cinder Provides storage volumes for Compute. Image Glance Can store the actual virtual disk files in the Object Store("Swift") Identity Keystone All the services authenticate with Identity ("Keystone") 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

23 2/5/

24 cinder database 2/5/

25 Horizon (Dashboard) Horizon is the canonical implementation of OpenStack s Dashboard Provides a web-based user interface to OpenStack services: Nova, Swift, Keystone etc. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

26 Nova(Compute) Nova is the project name for OpenStack Compute A cloud computing fabric controller The main part of an IaaS system. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

27 Swift (Object Store) Swift is a distributed, consistent object/blob store. Organizations can use Swift to store lots of data efficiently, safely and cheaply. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

28 Glance (Image Registry) The Glance project provides services for discovering, registering & retrieving virtual machine images. Glance has a RESTful API that allows querying of VM image metadata as well as retrieval of the actual image. REST = Representational State Transfer 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

29 Neutron (Networking) Neutron is an OpenStack project to provide network connectivity as a service Between interface devices (e.g. VNICs) managed by other OpenStack services (e.g. nova). VNIC = Virtual Network Interface Card 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

30 Icehouse Official Projects Integrated Projects (Icehouse release) OpenStack Compute (Nova): OpenStack Object Storage (Swift): OpenStack Image Service (Glance): 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

31 Icehouse Official Projects Integrated Projects (Icehouse release) OpenStack Identity (Keystone): OpenStack Dashboard (Horizon): OpenStack Networking (Neutron): 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

32 Icehouse Official Projects Integrated Projects (Icehouse release) OpenStack Block Storage service (Cinder): OpenStack Monitoring (Ceilometer): OpenStack Orchestration (Heat): 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

33 OpenStack Icehouse Roadmap Highlights Database Service (Trove) Scalable and reliable Cloud Database as a Service provisioning functionality for both relational and nonrelational database engines Bare Metal (Ironic) Provides an API for management and provisioning of physical machines. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

34 OpenStack Icehouse Roadmap Highlights Queue Service (Zaqar) Previously: Marconi Message queueing API and service for distributed application messaging Data Processing (Savannah) Hadoop on OpenStack 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

35 Cloud Architecture Customization 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

36 Different Architectures for Different Purposes (1) General purpose: A cloud built with common components that should address 80% of common use cases. Compute focused: A cloud designed to address compute intensive workloads such as high performance computing (HPC). 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

37 Different Architectures for Different Purposes (2) Storage focused: A cloud focused on storage intensive workloads such as data analytics with parallel file systems. Network focused: A cloud depending on high performance and reliable networking, such as a Content Delivery Network (CDN). 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

38 Different Architectures for Different Purposes (3) Multi-site: A cloud built with multiple sites available for application deployments for geographical, reliability or data locality reasons. Hybrid cloud: An architecture where multiple disparate clouds are connected either for failover, hybrid cloud bursting, or availability. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

39 Different Architectures for Different Purposes (4) Massively scalable: An architecture that is intended for cloud service providers or other extremely large installations. Other specialized cases 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

40 Examples 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

41 General Purpose Most common Base for all other specialized architectures No predefined use Need to support all potential cases Balanced loads across all Cloud components 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

42 General Purpose 2/5/

43 Compute Focused High performance computing (HPC) Big data analytics using Hadoop or other distributed data stores Continuous integration/continuous deployment (CI/CD) Platform-as-a-Service (PaaS) Signal processing for network function virtualization (NFV) 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

44 Compute Focused 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

45 2/5/ END

46 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

47 Appendix More Examples 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

48 Storage Focused Active archive Backups and hierarchical storage management. General content storage and synchronization. Data analytics with parallel file systems. Unstructured data store for services. Persistent block storage. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

49 Storage Focused Operating system and application image store. Media streaming. Databases. Content distribution. Cloud storage peering. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

50 Storage Focused 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu HDD Pool

51 2/5/

52 Network Focused Content delivery network Network service offerings High speed and high volume transactional systems Virtual desktop infrastructure (VDI) Voice over IP (VoIP) Video Conference or web conference High performance computing (HPC) 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

53 Network Focused 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

54 2/5/

55 Multi-site An organization with a diverse geographic footprint. Geo-location sensitive data. Data locality, in which specific data or functionality should be close to users. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

56 Multi-site 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

57 Hybrid Cloud Bursting from a private cloud to a public cloud Disaster recovery Development and testing Federated cloud, enabling users to choose resources from multiple providers Hybrid clouds built to support legacy systems as they transition to cloud 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

58 Hybrid Cloud 2/5/

59 Massively scalable 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

60 2/5/

61 Specialized cases Specialized Networking: This describes running networking-oriented software that may involve reading packets directly from the wire or participating in routing protocols. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

62 Specialized cases Software-defined networking (SDN): This user case details both running an SDN controller from within OpenStack as well as participating in a software-defined network. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

63 Specialized cases Desktop-as-a-Service: This is for organizations that want to run a virtualized desktop environment on a cloud. This can apply to private or public clouds. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

64 Specialized cases OpenStack on OpenStack: Some organizations are finding that it makes technical sense to build a multi-tiered cloud by running OpenStack on top of an OpenStack installation. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

65 Specialized cases Specialized hardware: Some highly specialized situations will require the use of specialized hardware devices from within the OpenStack environment. 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

66 Specialized Multi-Hypervisor 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

67 Specialized Software-defined networking SDN 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

68 Specialized Desktop as a Service Boot storms: What happens when hundreds or thousands of users log in during shift changes, affects the storage design. The performance of the applications running in these virtual desktops Operating system and compatibility with the OpenStack hypervisor 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

69 Specialized Desktop as a Service 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

70 Specialized OpenStack on OpenStack 2/5/

71 Specialized Hardware 2/5/

72 2/5/

73 What is RESTful API? Think about a mobile user requesting data from a server: Request Response XML/JASON Mobile Client Cloud Server 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

74 Problem 1 How to request the data and/or modify data? Request Response Mobile Client Cloud Server 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

75 Problem 2 How to return the state of the data: JASON, XML, How to tell the client the format and the status of the data, i.e. the state of the data? Request Response Mobile Client Cloud Server 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

76 What is RESTful API? RESTful API is an application program interface (API) that uses HTTP requests to perform the following operations on data: GET PUT POST DELETE 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

77 What is REST? REpresentational State Transfer Provide access to resources (not logic) Resources can be single documents or complex objects Resources are not a direct map of data store Client-Server development (2005) Server does not maintain the state for the client No cookies 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

78 What is REST? Predictable regardless of domain Discoverable standard verbs and routing structure Simple know the domain, know the system Client-Server development (2005) Incorporates multiple W3 standards 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

79 Standard REST Data Transfer Objects to represent resources Routing to locate resources HTTP verbs to perform actions on resources Negotiate content type via headers Accept application/json Respond with HTTP status codes 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

80 Standard REST Data Transfer Objects to represent resources Routing to locate resources HTTP verbs to perform actions on resources Negotiate content type via headers Accept application/json Respond with HTTP status codes 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

81 Data Transfer Objects A data model representing a resource Mask the actual data store Represent data that makes sense for the application Handle common relations Ex. Customer and Address Tables include an address object as a child of the customer 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

82 Routing to access res. Routes represent paths to resources api/customer A route can represent the location of a unique resource api/customer/{id}/order Routing structure can imply relationships api/customer/{id}/order Routes can represent a version API api/v1/customer 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

83 HTTP Verbs GET POST PUT PATCH DELETE one or more resources to create a new resource to update a resource to partially update a resource to delete a resource 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

84 HTTP Verbs Safe Methods the only action taken is to retrieve data GET Idempotent any N > 0 actions is the same as a single action PUT, PATCH, DELETE PUT vs POST, typically: PUT is to update POST is to create 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

85 HTTP Status Codes 2xx Success 200 OK 201 Created 3xx Redirection Most likely don t need to use these 4xx Client Error Client did something wrong 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

86 HTTP Status Codes 4xx Client Error 401 Unauthorized The request is not authorized to access or perform actions on a type of resource 403 Forbidden Client is forbidden to access the server 404 Not found The request is not authorized to access a specific resource 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu

87 HTTP Status Codes 5xx Server Error 500 Internal Server Error Consider returning a stack trace when not in production 501 Not implemented Not implemented yet, but in planning To learn more: 2/5/2018 COMP4442 Cloud Computing Lecture 04 George Baciu