WHAT S NEW IN OPENSTACK LIBERTY

Transcription

1 WHAT S NEW IN OPENSTACK LIBERTY Sean Cohen Principal Product Manager October 1st, 2015 Stephen Gordon Sr. Technical Product Manager

2 INTRODUCTION 2 Sean Cohen Principal Product Manager Stephen Gordon Sr. Technical

3 AGENDA Liberty Overview Project Updates Q&A

4 LIBERTY OVERVIEW

5 RELEASE TIMELINE April 2015 Kilo Sèrves, France is home to the Kilogram. October 2015 TBD 2016 O - Barcelona Summit Mitaka - is a city located in Tokyo Metropolis, Japan. TBD TBD Liberty - a village in the Canadian province of Saskatchewan. N - Austin Summit 5

6 POLL QUESTION Which OpenStack release are you currently using, if any? None Grizzly Havana Icehouse Kilo Juno Liberty (master)

7 STATISTICS OpenStack Liberty 27,000+ commits 163,000+ reviews ~160 corporate contributors ~2100 individual contributors

8 GOVERNANCE UPDATES

9 THE OPENSTACK BIG TENT Previously projects were either in the integrated release or not, which meant different things to different people and potentially excluding projects that were just as much a part of the OpenStack community. Changes toward a big tent model endeavor to: More clearly define what it means to be an OpenStack project. Be more broadly inclusive that projects that meet this definition, even those which compete with each other. Provide a more precise taxonomy for navigating the ecosystem of OpenStack projects.

10 DEFINING THE BIG TENT Project aligns with the OpenStack Mission Project follows the OpenStack way ( 4 opens ): Open Source Open Community Open Development Open Design Project ensures basic interoperability with the rest of OpenStack Project has an active team of one or more contributors Project meets any policies the TC requires all projects to meet

11 TAGGING THE BIG TENT Tags introduced to help operators navigate all the projects that make up OpenStack. Tag types: Technical Committee Managed Team Description Project Assertions Release Management Vulnerability Management E.g: Starter kit, approved release Team diversity Follows deprecation policy Has stable branches, cycle-wit Vulnerability team managed

12 COMPUTE

13 API MICRO-VERSIONING Compute API v2 has been in place for some time, expanded by adding extensions, lots of them. Determined that implementing new major version of API (v3) would be too difficult: User impact Developer overhead Micro-versions aim to: Make it possible to evolve the API incrementally Provided backwards compatibility to REST API users Improve code cleanliness to make doing the right thing easier

14 API MICRO-VERSIONING (cont.) Use a single monotonic counter of the form where X.Y where: Client will specify version it supports, e.g.: X will only be changed due to a significant backwards incompatible API change being made. Expected to rarely never be incremented. Y will change when making any change to the API. X-OpenStack-Nova-API-Version: In Kilo, v2.0 API code was still used to serve v2.0 requests, v2.0 API was frozen In Liberty, v2.1 API code handles all requests - implementing both the v2.0 and v2.1 API endpoints

15 EXPERIMENTAL ONLINE SCHEMA CHANGES Endeavours to make database schema changes occur online (either with no service outage or a minimized one) Instead of using migration scripts: Compares running schema against the model defined in Nova Generates DDL statements to close the gap Optionally, executes the DDL to make the running schema match the model Schema changes that aren t in use today or are difficult to automate are not catered to in the initial online schema migration implementation. Currently extremely experimental, requires further work in subsequent releases. Not yet a recommended path to upgrading.

16 INSTANCE HIGH AVAILABILITY Provided using external tools such as Pacemaker (previously demonstrated at Red Hat Summit with Intel, Dell). Liberty allows for cutting down on the time to evacuate guests after detecting node failure: Mark host down / force down API allows external tooling to notify Nova that a host is down before periodic tasks would normally detect it. As soon as host has been explicitly marked down evacuation can commence, triggered by the external tool (e.g. Pacemaker fence agent).

17 VIRTIO NETWORKING PERFORMANCE Default behavior provides a single VirtIO-net queue: Multiple queues provide greater throughput when: Restricts scale of guest protocol stack Does not scale with # of vcpus Guests cannot transmit or receive in parallel (only one TX/RX queue) Packets are relatively large Guest is active on many connections at the same time Number of queues == number of vcpus Enabling: hw_vif_multiqueue_enabled=true false (default false) Nova will match number of queues to number of vcpus

18 IMPROVED BAREMETAL DRIVER SUPPORT Maturing and increased driver support: Dell, HP, Open CloudServer, SeaMicro, Cisco UCS, Fujitsu, AMT Generic and virtual driver coverage Wake-on-LAN power driver SSH driver support for VirtualBox, Parallels, Virsh Specific driver enhancements: HP (ilo) UEFI secure boot Fujitsu Primergy booting from virtual media using NFS or CIFS supported

19 NETWORKING

20 QUALITY OF SERVICE QoS is a general term referring to capability of a network to provide better service to select network traffic. Various tools exist and may be applied in different places. Examples include traffic classification, policing, marking, and shaping. OpenStack tenant administrator may want to offer different service levels based on application needs and available bandwidth. In Liberty OpenStack Networking ( Neutron ) provides a new extensible API for defining QoS policies.

21 QUALITY OF SERVICE (cont.) Policies can be applied: Initial support allows rate limiting the traffic a virtual machine can send - packets beyond the specified rate are dropped: Per port - a specific Neutron port has the policy applied. Per network - all ports plugged on the Neutron network inherit the policy. Virtual interfaces using OVS Virtual Functions (VFs) using SR-IOV Policy can be applied, modified, or removed dynamically.

22 QUALITY OF SERVICE (cont.) # neutron qos-policy-create bw-limiter # neutron qos-bandwidth-limit-rule-create bw-limiter \ --max_kbps max_burst_kbps 300 # neutron qos-rule-show 9be535c3-daa2-4d7b-88ea-e8de Field Value id 9be535c3-daa2-4d7b-88ea-e8de16 rule_type bandwidth_limit description max_kbps 3000 max_burst_kbps

23 PLUGGABLE IP ADDRESS MANAGEMENT Deployers have a need to integrate OpenStack with their existing infrastructure that uses external IPAM solutions. Historically most OpenStack Networking ( Neutron ) plug-ins leverage an embedded IPAM implementation. While DHCP provider could be changed, allocation logic could not. Liberty introduces pluggable IPAM framework into Neutron. Reference implementation remains unchanged but there is now a plugin system to support external IPAM offerings.

24 ROLE-BASED ACCESS CONTROL Network sharing in Neutron has historically been binary, a network is shared with every tenant or not shared at all. Liberty introduces a RBAC table that allows finer grained sharing of Neutron networks between tenants. Allow only specific tenants to attach virtual machines to the network. Eliminate the ability for some tenants to create networks at all instead only allow attachment to pre-created networks corresponding to their project.

25 IDENTITY

26 IDENTITY Support for tokenless authorization with X.509 SSL client certificate Per identity provider Web Single Sign-On (WebSSO) support. Additional attributes added to SAML assertion in order to map user and project domains.

27 STORAGE

28 GENERIC VOLUME MIGRATION Volume migration allows you to migrate a volume with its data between two backends in a manner that is transparent to users and workloads. The original API implementation assumed that both source and destination volumes can be attached locally via iscsi. The new generic API now allows more drivers such as Ceph RBD to perform the migration functionality.

29 IMPORT/EXPORT SNAPSHOTS Complementing feature for the import/export volume function. Allows to import volumes snapshot from one Cinder to another. The imported snapshots can be used as volume templates. Allows to import "non" openstack snapshots already on a backenddevice. Where, export snapshots should work the same way as export volumes.

30 VOLUME REPLICATION (v2) Provides a new implementation for replication of volumes between backends that leaves the bulk of the work to the driver. Provides basic administrative API methods to perform actions such as enable/disable and fail-over. First iteration of this new API supports only a single remote device, with limited tenant visibility. Allowing to introduce new replication drivers in the Mitaka cycle.

31 NON-DISRUPTIVE BACKUP Allows OpenStack administrators to perform a non-disruptive volume backup while the volume is still attached to a guest. By allowing the backup driver to: Take a temporary snapshot Attach the temporary snapshot Do backup from the temporary snapshot Detach the temporary snapshot Cleanup temporary snapshot (Until now, volumes had to be detached first before they could be backed up)

32 GENERIC IMAGE CACHE With this new feature backends are able to use cached glance images when creating volumes from images. If enabled, and the cinder internal tenant is configured, it will create a minimal sized clone of a volume the first time it is downloaded from glance. The original will then be extended to full size. Any following requests to create a volume from an image will be able to do a volume clone from the image-volume cache entry instead of downloading the image data from glance again.

33 IMAGE SIGNING AND ENCRYPTION Protect image integrity by ensuring no modifications occur after the image is uploaded by a user. Supports both signing of the image and signature validation of bootable images when used.

34 OBJECT STORAGE Increasing ring partition power A new swift-ring-tool has been introduced to increase or decrease the partition power of an OpenStack Swift ring without the need to copy all data to a new cluster. Symbolic links Swift has gained the notion of a symbolic link ( symlink ) object allowing to do aliases or multiple names to the same object. This object will reference another object, where GET, HEAD, POST, and OPTIONS requests for a symlink object will operate on the referenced object.

35 DATA PROCESSING Spark now available on YARN based (generation 2) distributions Storm elastic data processing (EDP) support EDP jobs can now be suspended and resumed Can now launch multiple clusters in parallel Hadoop generation 1 support has been deprecated Manila as a runtime data source

36 DATABASE AS A SERVICE DB Cluster Scaling - Adds cluster grow and shrink functionality Expanding Backup/Restore capabilities to MongoDB Adding MariaDB support Guestagent configuration manager functionality - provides ability to allow custom configuration overrides Floating IP for Vertica management master node - provides external access to Vertica cluster management console Database log file downloads - create API for end user to access log files stored in swift containers

37 ORCHESTRATION

38 CONVERGENCE UPDATES Initial steps in longer term trend towards supporting a convergence model for Heat stack updates. Liberty introduces an (optional) mode to persist per-resource state during stack updates. Improved fault tolerance, including recovery from a failure of the Heat engine. Improved scalability, potential to spread work over engine workers in a more granular way than was previously possible. Lays groundwork for auto-healing style features in future.

39 STACK TAGS Allow categorization of Heat stacks when they are created. Includes ability to mark a stack as hidden based on a tag. E.g.: Cloud provider wants to use a heat stack for behind the scenes orchestration such as automating networking configuration when launching an instance

40 BROADER PROJECT SUPPORT New Heat resources for emerging projects including: Barbican Mistral Manila Monasca Designate Zaqar integration for SoftwareConfig SoftwareDeployment resources now support providing metadata and signalling via Zaqar as an alternative to existing transports.

41 MONITORING

42 CEILOMETER SPLIT UP Lack of clarity around how to best use ceilometer for different use cases in the past. Split into multiple sub-projects, each aimed at one high level use and using its own storage backend: Collectors Metrics ( Gnocchi ) Alarms ( AODH ) Events

43 CEILOMETER TIME SERIES METRICS Ceilometer s poor performances on large scale metrics API requests was one of the biggest issues Gnocchi (Time Series Database as a Service) project was created to address this under the Ceilometer program umbrella.

44 EMERGING PROJECTS

45 SHARED FILE SYSTEMS SERVICE (MANILA) Share Extend & Shrink New REST API for extending and shrinking shares Share migration Share Migration allows a share to be migrated from one host pool to another hostpool, as well as allowing to perform migration between different backends. Big data (Sahara) support for Manila as a data source

46 CONTAINERS SERVICE (MAGNUM) API service developed by the OpenStack Containers Team. Aims to make container orchestration engines (COE) first class resources in OpenStack. Uses Heat to orchestrate deploying and configuring an OS image which contains Docker and a COE on virtual machines or bare-metal. Liberty enhancements: Smart(er) scale down for Kubernetes clusters. High availability (2N) for Kubernetes master node. Ability to act as a CA (stepping stone to TLS support etc.)

47 MESSAGE SERVICE (ZAQAR) Zaqar is a multi-tenant cloud messaging service for web developers. The service enables cloud developers to send messages between various components. Liberty enhancements include: Pre-Signed URLs: auth-less URLs to access resources Persistent transport (websocket): Persistent connections and streamed API calls. push-notifications: Notify users over on messages for certain topics.

48 POLL QUESTION Which projects in the OpenStack big tent would you like to hear more about in the future? Key management (Barbican) DNS as a Service (Designate) Application catalog (Murano) Governance service (Congress) Workflow service (Mistral) Key-value store as a Service (MagnetoDB) Deployment (TripleO) Benchmark service (Rally)

49 OPENSTACK SUMMIT TOKYO 49

50 OPENSTACK SUMMIT TOKYO 50

51 OPENSTACK SUMMIT TOKYO 51

52 THANK YOU plus.google.com/+redhat facebook.com/redhatinc linkedin.com/company/red-hat twitter.com/redhatnews youtube.com/user/redhatvideos redhatstack.com