OpenStack Technology Review & Demo Egan Ford IBM Distinguished Engineer egan@us.ibm.com
PPT s and Videos: http://xmission.com/~egan/cloud/! 2
Agenda IBM SmartCloud and OpenStack Cloud Taxonomy Some OpenStack Public Use Cases What is OpenStack OpenStack Resources IBM Resources/Solutions for OpenStack Available Today OpenStack (Video) Demo 3
Cloud capabilities that are built upon a common platform, with a commitment to open standards Business Process as a Service Software as a Service Platform as a Service Infrastructure as a Service Design Deploy Consume Foundation Services Solutions Private & Hybrid Clouds Cloud Enablement Technologies Managed Cloud Services Infrastructure and Platform as a Service Cloud Business Solutions Software and Business Process as a Service Commitment to open standards and a broad ecosystem 4
Managed with cloud technologies How to manage inflexible, siloed systems and business processes to improve business agility 1 Integrate Build an efficient IT infrastructure 2 Automate Improve speed and dexterity 3 Orchestrate Deliver IT without boundaries IBM Systems Director, VMControl BladeCenter Foundation for Cloud IBM SAN Volume Controller IBM SmartCloud Entry IBM SmartCloud Provisioning zenterprise Starter Edition for Cloud IBM SONAS & Storwize V7000 with Active Cloud Engine Tivoli Service Automation Manager IBM Service Delivery Manager System z Solution Edition for Cloud Technology client reduced rack space by 94% and energy costs by 97% Telecom client brought new applications to market 35x faster Communications client reduced datacenters from 200 to 5 5
Cloud Taxonomy 6 Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
Cloud Value Proposition and Positioning Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/ 7
How You (Provider) Build These Clouds Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/ 8
What You (Consumer) Get with These Clouds: Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/ 9
Policy-based Clouds and Design-for-fail Clouds are purpose optimized Infrastructure Management solutions Policy-based Clouds Design-for-fail Clouds Purpose optimized for longer-lived virtual machines managed by Server Administrator Centralizes enterprise server virtualization administration tasks High degree of flexibility designed to accommodate virtualization all workloads Significant focus on managing availability and QoS for long-lived workloads with level of isolation Characteristics derived from exploiting enterprise class hardware Legacy applications Purpose optimized for shorter-term virtual machines managed via end-user or automated process Decentralized control, embraces eventual consistency, focus on making good enough decisions High degree of standardization Significant focus on ensuring availability of control plane Characteristics driven by software New applications 10
Some OpenStack Public Use Cases Internap http://www.internap.com/press-release/internap-announces-world%e2%80%99s-firstcommercially-available-openstack-cloud-compute-service/ Rackspace Cloud Servers, Powered by OpenStack http://www.rackspace.com/blog/rackspace-cloud-servers-powered-by-openstack-beta/ Deutsche Telekom http://www.telekom.com/media/media-kits/104982 AT&T http://arstechnica.com/business/news/2012/01/att-joins-openstack-as-it-launches-cloudfor-developers.ars MercadoLibre http://openstack.org/user-stories/mercadolibre-inc/mercadolibre-s-bid-for-cloudautomation/ NeCTAR http://nectar.org.au/ San Diego Supercomputing Center http://openstack.org/user-stories/sdsc/ 11
OpenStack design tenets focus on delivering essential infrastructure on an available, scalable, elastic control plane Basic Design Tenets 1) Scalability and elasticity are our main goals 2) Any feature that limits our main goals must be optional OpenStack Leadership's vision statement essential Infrastructure, support platform 3) Everything should be asynchronous. If you can't do something asynchronously, see #2 4) All required components must be horizontally scalable 5) Always use shared nothing architecture (SN) or sharding. If you can't Share nothing/shard, see #2 6) Distribute everything. Especially logic. Move logic to where state naturally exists. 7) Accept eventual consistency and use it where it is appropriate. 8) Test everything. We require tests with submitted code. (We will help you if you need it) Sources: http://www.openstack.org/downloads/openstack-compute-datasheet.pdf http://wiki.openstack.org/basicdesigntenets 12
OpenStack 13 Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
OpenStack is comprised of seven core projects that form a complete IaaS solution IaaS Compute (Nova) Storage (Cinder) Network (Quantum) Provision and manage virtual resources Dashboard (Horizon) Self-service portal Image (Glance) Catalog and manage server images Identity (Keystone) Unified authentication, integrates with existing systems Object Storage (Swift) petabytes of secure, reliable object storage IaaS 14 Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
Compute delivers a fully featured, redundant, and scalable cloud computing platform Key Capabilities: Architecture Manage virtualized server resources CPU/Memory/Disk/Network Interfaces API with rate limiting and authentication Distributed and asynchronous architecture Massively scalable and highly available system Live guest migration Move running guests between physical hosts Live VM management (Instance) Run, reboot, suspend, resize, terminate instances Security Groups Role Based Access Control (RBAC) Ensure security by user, role and project Projects & Quotas VNC Proxy through web browser Sources: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/ http://openstack.org/projects/compute/ 15
Compute management stack control plane is built on queue and database Key Capabilities: Responsible for providing communications hub and managing data persistence RabbitMQ is default queue, MySQL DB Documented HA methods ZeroMQ implementation available to decentralize queue Single cell (1 Queue, 1 Database) typically scales from 500 1000 physical machines Cells can be rolled up to support larger deployments Communications route through queue API requests are validated and placed on queue Workers listen to queues based on role or role + hostname Responses are dispatched back through queue 16
nova-compute manages individual hypervisors and compute nodes Key Capabilities: Responsible for managing all interactions with individual endpoints providing compute resource, e.g. -- Attach iscsi volume to phsyical host, map to guest as additional HDD Implementations direct to native hypervisor APIs Avoids abstraction layers that bring least common denomination support Enables easier exploitation of hypervisor differentiators Service instance runs on every physical compute node, helps to minimize failure domain Support for security groups that define firewall rules Support for KVM LXC VMware ESX/ESXi (4.1 update 1) Xen (XenServer 5.5, Xen Cloud Platform) Hyper V 17
nova-scheduler allocates virtual resources to physical hardware Key Capabilities: Determines which physical hardware to allocate to a virtual resource Default scheduler uses a series of filters to reduce set of applicable hosts and uses costing functions to provide Weight Not a focus point for OpenStack Default implementation finds first fit Shorter the workload lifespan, less critical the placement decision If default does not work, often deployers have specific requirements and develop custom 18
nova-api supports multiple API implementations and is the entry point into the cloud Key Capabilities: APIs supported OpenStack Compute API (REST-based) Similar to RackSpace APIs EC2 API (subset) Can be excluded Admin API (nova-manage) Robust extensions mechanism to add new capabilities 19
Network automates management of networks and attachments (network connectivity as a service) Key Capabilities: Architecture Responsible for managing networks, ports, and attachments on infrastructure for virtual resources Create/delete tenant-specific L2 networks L3 support (Floating IPs, DHCP, routing) Moving to L4 and above in Grizzly Attach / Detach host to network Similar to dynamic VLAN support Support for Open vswitch OpenFlow (NEC & Floodlight controllers) Cisco Nexus Niciria 20
Cinder manages block-based storage, enables persistent storage Architecture Key Capabilities: Responsible for managing lifecycle of volumes and exposing for attachment Structure is a copy of Compute (Nova), sharing same characteristics and structure in API server, scheduler, etc. Enables additional attached persistent block storage to virtual machines Support for booting virtual machines from nova-volume backed storage Allows multiple volumes to be attached per virtual machine 21 Supports following ISCSI RADOS block devices (e.g. Ceph distributed file system) Sheepdog Zadara
Identity service offers unified, project-wide identity, token, service catalog, and policy service designed to integrate with existing systems Key Capabilities: Identity service provides auth credential validation and data about Users, Tenants and Roles Token service validates and manages tokens used to authenticate requests after initial credential verification Catalog service provides an endpoint registry used for endpoint discovery. Policy service provides a rule-based authorization engine and the associated rule management interface. Each service configured to serve data from pluggable backend Key-Value, SQL, PAM, LDAP, PAM, Templates REST-based APIs 22
Image service provides basic discovery, registration, and delivery services for virtual disk images Key Capabilities: Think Image Registry, not Image Repository REST-based APIs Query for information on public and private disk images Register new disk images Disk images can be stored in and delivered from a variety of stores (e.g. SoNFS, Swift) Supported formats Raw Machine (a.k.a. AMI) VHD (Hyper-V) VDI (VirtualBox) qcow2 (Qemu/KVM) 23 References http://openstack.org/projects/image-service/ VMDK (VMWare) OVF (VMWare, others)
Dashboard enables administrators and users to access and provision cloud-based resources through a self-service portal Key Capabilities: Thin wrapper over APIs, no local state Registration pattern for applications to hook into Ships with three central dashboards, a User Dashboard, a System Dashboard, and a Settings Out-of-the-box support for all core OpenStack projects Nova, Glace, Switch, Quantum Anyone can add a new component as a first-class citizen. Follow design and style guide. Visual and interaction paradigms are maintained throughout. References http://horizon.openstack.org/intro.html Console Access 24
OpenStack Resources Forums http://forums.openstack.org/ Wiki http://wiki.openstack.org/ Documentation http://docs.openstack.org/ Mailing Lists http://wiki.openstack.org/mailinglists OpenStack Project Management https://launchpad.net/openstack Blogs http://planet.openstack.org Real-time chat room #openstack and #openstack-dev on irc://freenode.net (443 users currently logged in) Rackspace Reference Architectures http://www.referencearchitecture.org/ Easy Install http://www.hastexo.com/resources/docs/installing-openstack-essex-20121-ubuntu-1204-precisepangolin 25
IBM Resources/Solutions for OpenStack Available Today developerworks https://www.ibm.com/developerworks/mydeveloperworks/wikis/home?lang=en#/wiki/ OpenStack Google: openstack IBM developerworks xcat (FOSS) for 0-day deployment xcat OpenStack Paper (CATStack) Automated qcow2 image creation for Glance HW control Bare-metal discovery and bring up Firmware, Base OS, etc IBM Intelligent Cluster Solutions (see Matt Ziegler's PPT) Preconfigured Switches Rack and stacked and ready to go Lab Services for 0-day 26
IBM Resources/Solutions for OpenStack Available Today All IBM System Software and Tools can coexist with OpenStack. Director, ASU, lflash, etc SoNAS for shared file (NFS, SMB) XIV for block storage (Nova Volume) idpx for scale-out Nova Compute and Swift BNT switches for OpenFlow and Quantum GPFS for iscsi/block (Nova Volume) or file. 27
OpenStack Demo Setup Private Networks: eth0: 172.20.249/24 vm: 172.20.250/24! Control Nodes Compute Nodes 172.20.249.10! 172.20.249.11! 172.20.249.12! 172.20.249.13! 172.20.249.X! compute! compute! VM compute! VM compute! compute! network! network! network! network! network! scheduler! scheduler! VM VM volume! volume! console! console! glance! glance! api! api! os-essex0! os-essex1! os-essex2! os-essex3! os-essexx! 10.0.9.10! 10.0.9.11! 10.0.9.12! 10.0.9.13! 10.0.9.X! VM Firewall HA Active/Passive Scale Out Public Networks: eth1: 10.0.9.0/25 vm: 10.0.9.128/25! 28
PPT s and Videos: http://xmission.com/~egan/cloud/! 29