Oracle Ravello Cloud Service

Oracle Ravello Cloud Service Technology Overview ORACLE WHITE PAPER SEPTEMBER 2016

Introduction If hybrid cloud environments were a reality, enterprises would design their internal data centers for average load, and simply rent additional capacity on the cloud to accommodate peaks. They would be able to take their multi-virtual machine (VM) applications as-is and run them on any cloud, without changes. They would continue with their existing VMs, networking, and storage configurations and use the same processes and tools. To realize this vision of the hybrid cloud environment, you need an infrastructure that normalizes the public cloud and the data center from the application s perspective. This infrastructure would run on the cloud and expose the same computing, networking, and storage services to the multi-vm enterprise application as if it were running internally, on premises. To accomplish this, Oracle Ravello Cloud Service is built upon a distributed hypervisor infrastructure called HVX. It enables an enterprise to completely encapsulate a multi-vm application and run it on any cloud including Oracle Public Cloud, Amazon Web Services (AWS), and Google Cloud without any changes whatsoever. This white paper provides an overview of the technology components that comprise Oracle Ravello Cloud Service. HVX: Virtual Infrastructure for the Cloud The HVX infrastructure underlying Oracle Ravello Cloud Service consists of three technology components and a management layer, wrapped up and offered as a software-as-a-service (SaaS). The technology components are» a high-performance nested virtualization engine also called a nested hypervisor» a software-defined network» a storage overlay The management layer manages the technology components, provides the user interface and API, handles image management, and enables monitoring all as a SaaS service.

Figure 1. Need caption HVX enables enterprises and software vendors to replicate multi-vm application environments with complex networking in self-contained capsules in the public cloud.» Development and testing (DevTest) environments: Nested virtualization technology enables enterprises to encapsulate VMWare- or KVM-based, multi-tier applications and deploy them on Oracle Public Cloud, AWS, or Google Cloud without making any changes. Enterprises can then spin up as many isolated environments as they need for testing, on-demand.» Virtual training environments: Training managers can automate the provisioning of complex environments for students with one click. Each student receives an isolated environment for learning, on-demand. Environments can be provisioned in any Oracle Public Cloud, AWS, or Google Cloud region in the world for an ideal training experience. HVX: High-Performance Nested Virtualization An integral part of HVX is a high-performance nested hypervisor or virtual machine manager (VMM) capable of running unmodified guests on top of already virtualized hardware. Conventional hypervisors such as VMware ESX, KVM, and Xen are designed to run on physical x86 hardware and use virtualization extensions offered by modern CPUs such as Intel VT and AMD SVM to virtualize the Intel architecture. HVX, on the other hand is a nested hypervisor that runs inside a virtual machine where these hardware extensions are not normally available. Instead, HVX employs a technology called binary translation to implement high-performance virtualization that does not require these virtualization extensions.

Figure 2. Need caption Virtualization on the x86 Architecture The job of the hypervisor is to provide the illusion that guest operating systems running below it are running on their own hardware while, in fact, they are not. The hardware is shared with the hypervisor itself and any other VMs running on the same host. When virtualization extensions are available, the easiest way to implement the illusion is by using trap and emulate. Trap and emulate works as follows. The hypervisor configures the processor so that any instruction that can potentially break the illusion that is, access the memory of the hypervisor itself--will generate a trap. This trap will interrupt the guest and will transfer control to the hypervisor. The hypervisor then examines the offending instruction, emulates it in a safe way, and then allows the guest to continue executing. The trap and emulate approach is well understood and delivers good performance, but it depends critically on virtualization extensions. Without the extensions, the Intel architecture is not able to generate all the necessary traps. So in the cloud, where these extensions are not available, it cannot be used to implement nested virtualization. Nested Virtualization Using Dynamic Binary Translation HVX, the hypervisor in Oracle Ravello Cloud Service, uses a technology called binary translation. Unlike the trapand-emulate method, binary translation does work when virtualization extensions are not available. Binary translation was first described by Digital Equipment Corporation (DEC) in the early 90s. DEC used it to run programs written for the VAX computer on the Alpha AXP processor. The binary translation software would examine the instructions that make up a VAX program, translate them into equivalent Alpha instructions, and then run the translated instructions directly on the Alpha processor. When the translation can be done ahead of time for an entire program, the process is called Static Binary Translation (SBT). When a few instructions are translated at a time while executing, the process is called Dynamic Binary Translation (DBT).

Oracle Ravello Cloud Service uses DBT for virtualization. The concept is the same as in the VAX-to-Alpha example, but instead of translating instructions from one CPU to the other, HVX uses DBT to locate the illusion-breaking instructions. Those instructions are then translated into safe equivalents. HVX Nested Virtualization: Performance HVX has been extensively optimized to run inside a VM. Many of the optimizations are novel and patent pending. Two basic optimizations that are implemented in Oracle Ravello Cloud Service are described below. First, modern operating systems prevent applications from meddling with their internal code. All the illusion-breaking instructions that an application could possibly execute are also unsafe for the operating system (OS) and already intercepted by it using trap-and-emulate. (In this case, the Intel architecture is able to generate sufficient traps.) This means that Oracle Ravello Cloud Service only needs to translate the OS itself (called ring 0), and not any of the applications running inside it (called ring 3). This greatly reduces the amount of code that needs to be translated. The second optimization is based on the fact that executable code does not normally change. This means that Oracle Ravello Cloud Service can translate a block of code once, save it, and re-use it if it is executed again later. The entire guest OS is translated very quickly and no more translation occurs. In addition, a block of translated code can be generated in such a way that, when it is done, it will directly call into another translated block. This is called block chaining. HVX Nested Virtualization: Foundational Technology The base functionality of HVX nested virtualization is to virtualize the Intel architecture and run unmodified guest operating systems. But because HVX fully controls the execution of its guest VMs, it is also a foundation that enables many advanced features. Below are some examples of current features enabled by HVX:» Ability to intercept all network traffic and enable implementation of software defined network.» Ability to intercept all storage access, making storage overlay possible.» Ability to run multiple VMs inside a single cloud VM. This is called consolidation. Consolidation allows a much higher utilization and in some cases, improves performance.» Ability to virtualize console access so that a graphical console is available for each VM. HVX Nested virtualization: Running VMware or KVM VMs Unmodified on Leading Public Clouds In addition to the core nested hypervisor functionality, HVX exposes VMware or KVM devices to the VM running on top. This enables enterprises to run their existing VMware VMs unmodified on top of Oracle Ravello Cloud Service on top of leading public clouds. Everything about the VM OS, paravirtualized drivers such as VMXNet3 network drivers or PVSCSI storage drivers, application settings, network settings, VMware tools stays the same. HVX Nested^2 Virtualization: Run ESXi or KVM on Leading Public Clouds Not only can HVX run VMware or KVM virtual machines, but it also implements in software virtualization hardware extensions (Intel VT and AMD V) functionality inside HVX. HVX can expose a true x86 platform type to the VM running on top. This allows enterprises to run hypervisors like KVM (and soon ESXi) on top of Oracle Ravello Cloud Service on top of leading public clouds. From an implementation perspective, Oracle has adapted its binary translation so that it recognizes the double-nesting, and effectively removes one layer of nesting and runs the guest directly on top of HVX. As a result, the performance overhead is relatively low. In addition, we have also implemented nested pages support inside HVX which will make running a hypervisor on top of HVX even more efficient.

Figure 3. Need caption In addition, HVX enables some really interesting future use cases that Ravello is actively investigating. For example, it supports advanced use cases that today are only available in the datacenter, such as live migration, agentless backups and hot-plugging of CPU and memory. HVX: Software-Defined Networking (Overlay Network) Today, all but the simplest applications consist of multiple virtual machines. These machines communicate with each other over an IP network and provide a service together. The configuration of the network that includes host names, IP address and firewall rules is an essential part of an application s configuration. Change the network, and your application stops working. In the public cloud however, the network is very different from what it is in the datacenter. For example, pure L2 access is difficult and VLANs, span ports, broadcast, and multicasting usually do not work. Oracle Ravello Cloud Service allows you to run existing multi-vm applications unmodified in the cloud, not just single virtual machines. To make this possible, Oracle provides a software-defined network that virtualizes the connectivity between the virtual machines in an application. The virtual network is completely user-defined and can include multiple subnets, routers, and supplemental services such as dynamic host configuration protocol (DHCP), domain name service (DNS) servers, and firewalls. The virtual network can be made to look exactly like a datacenter network. You can even create your own network elements by uploading virtual appliances that implement the desired function. For example, this way you can implement load balancing appliances and L2/L3 site-to-site or access virtual private network (VPN) endpoints.

Figure 4. Need caption The data plane of the virtual network is formed by a fully distributed virtual switch and virtual router software component that resides within HVX. Network packets that are sent by a VM are intercepted and injected into the switch. The switch operates similarly to a regular network switch. For each virtual network device, the virtual switch creates a virtual port that handles incoming and outgoing packets from the connected virtual network interface card (NIC) device. The virtual port learns media access control (MAC) addresses of incoming packets and builds a forwarding table based on this. For broadcast frames, the virtual port floods the packet to all other distributed virtual ports in the same broadcast domain. The transport between ports is done using either a local transport (in case the ports are on the same switch), or using P2P tunnels over user datagram protocol (UDP). The control plane of our software-defined network includes a distributed router and also distributed DNS and DHCP servers. Each service is located at every HVX node. HVX: Storage Overlay The storage overlay solution is the third technology component of the HVX stack within Oracle Ravello Cloud Service. Storage is very different from its compute and network counterparts. Like compute and network virtualization, storage virtualization has its transient and run-time behavior. However, it also has a very different aspect of persistency over time, even after an application is shut down. Abstracts Cloud Storage and Presents Local Block Devices to Guests Oracle Ravello Cloud Service s storage overlay solution focuses on performance, persistence and security. It abstracts native cloud storage primitives such as object storage and various types of block devices into local block devices exposed directly to the guest VMs. Everything from the device type and controller type to the location on the PCI bus remains the same. Hence it appears to the guest as-if it was running in its original data center infrastructure. This allows the guest VM to run exactly as is with its storage configuration as if it was running on premises.

Figure 5. Need caption Caches Images for Improved Performance Oracle Ravello Cloud Service s HVX storage overlay solution acts as a global overlay on top of a distributed object store that contains the VM images. In many cases, the overlay would cache image parts locally to the cloud provider s local block storage solution for best performance. This caching process is adaptive and uses multiple strategies in order to get the best performance from purely on-demand copying, to various read-ahead strategies, and prefetching the entire volume. Network-Based Protocols Cloud storage abstraction (and presentation as a local block device), coupled with the HVX overlay networking capabilities allows for running various network-attached storage (NAS) appliances and their consumption over network-based protocols such as internet small computer system interface (iscsi), network file system (NFS), common internet file system (CIFS) and server message block (SMB). These block devices are backed by a high performance copy-on-write file system which allows Oracle to implement a multi-vm incremental snapshot feature. Virtual CD-ROM As any device type can be exposed to the guest VM, the Oracle Ravello Cloud Service HVX storage overlay even natively supports the attachment of CD-ROM drives to guest VMs with inserting and ejecting media that was uploaded to the image store in Oracle Ravello Cloud Service. This enables the unique capability of installing an OS from scratch in the cloud. Built in RAID 0 for Large Disk Sizes In order to support high performance storage, as well as large volumes, the Oracle Ravello Cloud Service HVX storage overlay, together with the Oracle Ravello Cloud Service management service, implements an algorithm to use various disk primitives such as magnetic, provisioned IOPS (I/O operations) and solid-state drive (SSD) based in the different cloud providers and constructs an ad-hoc RAID 0 transparently to the guest.

Oracle Ravello Cloud Service s Management Service Oracle Ravello Cloud Service s management service is built from multiple components, as shown in the figure below. The main two components are HVX the unique infrastructure layer that virtualizes cloud compute, networking, and storage and the management service which is described below. Management Service Overview Oracle Ravello Cloud Service s management service is a highly available enterprise-grade system deployed in the public cloud. It uses industry best practices to isolate different tenants and eliminate any risk of data leak or data corruption. Figure 6. Need caption The management service was developed with scalability, high availability, and security as key architectural guidelines, while focusing on a clear and simple user experience of its rich, RESTful API and web-based user interface (UI). To accomplish this, the management service uses state-of-the-art technologies such as a mix of relational and non-relational databases, a distributed in-memory data grid, and persistent queues for asynchronous processing, while employing cloud deployment best-practices for scale and availability. All communication channels between the management system and the HVXs, as well as between the end-user and the management system (either using the web-based UI or the RESTful API) are authenticated and encrypted using standard secure sockets layer (SSL)-based mechanisms.

Oracle Ravello Cloud Service s management service is responsible for all aspects of defining, running, and controlling an application running on HVX, such as the ones described in Figure 7. All of this functionality is accessible using a web-based UI, a rich RESTful API, as well as various SDKs (Python SDK, Ruby SDK) and plugins for different ecosystem components (Jenkins plugin, Maven plugin). Figure 7. Need caption Oracle Ravello Cloud Service s Image Service Another key additional component of Oracle Ravello Cloud Service is the image service. This service is used to securely store and access all images and snapshots that are being used in the different VMs running in Oracle Ravello Cloud Service. Data is written to the image store by a specialized utility that allows secure uploading of images as well as by the HVX directly when writing snapshots. This lightweight and cross-platform utility also supports the extraction of VMs directly from leading on-premises virtualization systems such as VMWare vcenter, and their streaming-upload to Oracle Ravello Cloud Service, while parsing the images for various metadata information. Images of different tenants are isolated in the image store, and cannot be accessed in any way by the non-owning tenants. SaaS Operations Overview Oracle Ravello Cloud Service s deep technology is wrapped up and presented to users as an easy-to-use service. Consider Oracle Ravello Cloud Service as an overlay cloud provider sitting on top of Tier-1 clouds such as Oracle Public Cloud, AWS, and Google Cloud. Enterprises simply log into Oracle Ravello Cloud Service, import their workloads, and deploy them to any supported cloud region.

Figure 8. Need caption Global Footprint Oracle does not own and operate its own capital-intensive data centers for Oracle Ravello Cloud Service, but rather, operates a cloud service, powered by its unique HVX software infrastructure on top of Tier-1 clouds such as Oracle Public Cloud, AWS, and Google Cloud. Oracle leverages the best-in-class hardware, scale, economics, and operational practices of these leading cloud providers to provide a robust enterprise grade cloud service to customers. Through Oracle Ravello Cloud Service, your enterprise can deploy its VMware or KVM workloads on any Oracle Public Cloud, AWS, or Google Cloud region (without requiring migration) and benefit from low-latency, highthroughput access to you application environments. Uptime and Service Availability A big element of uptime is the architecture of the software used to provide the service. As discussed in this white paper, Oracle Ravello Cloud Service s management service is built using industry best practices and includes redundancy and high availability at all layers in the overall stack. In addition to the architecture, Oracle has implemented industry best practices to ensure maximum service uptime. Oracle monitors and manages its entire service offering system, storage, network, databases, and customer usage 24 hours a day, 7 days a week, 365 days a year. This monitoring is done using internal and external engines from various locations around the world in a distributed and scalable manner. Oracle Ravello Cloud Service s redundant software architecture and robust operations result in very low levels of unplanned downtime. Oracle Ravello Cloud Service s software architecture also minimizes planned downtime. Most software updates and upgrades to the management system do not cause any downtime. In exceptional cases, the management system is unavailable for a maximum of 10 minutes. It is also important to note, that Oracle Ravello Cloud Service is designed so that even if the management service were to suffer downtime, enterprise workloads that were already deployed will continue to run uninterrupted. Given Oracle Ravello Cloud Service s unique capability of enabling enterprises to deploy their workloads to either Oracle Public Cloud, AWS, or Google Cloud, enterprises can achieve very high availability levels for application

environments. For example, if there is a storm on the US eastern seaboard, enterprises can, with a single click, deploy that application environment to the US central region with zero downtime. Continuous Improvement Oracle Ravello Cloud Service s development and operations team function as one. The product architecture and the operational practices are built around rolling out new features rapidly while simultaneously maintaining high quality and uptime for the service. Oracle rolls out dozens of software enhancements, updates, and bug fixes every quarter. The company has implemented robust release and change management methodologies, procedures, and tools to ensure that this level of agility is provided safely to its customers. One risk mitigation approaches involves gradually rolling out new features and key core components to subsets of users. Capacity Management Since Oracle Ravello Cloud Service does not operate its own servers, networking, and storage instead leveraging Tier-1 clouds such as Oracle Public Cloud, AWS, and Google Cloud theoretically the service s capacity is the sum of the capacity of the underlying clouds. That is, basically, a lot of capacity. Despite that, the Oracle Ravello Cloud Service operations team carefully monitors and manages underlying cloud resources and quotas to ensure that customers always have enough capacity even for peak workloads. Operational Transparency Transparency is a guiding tenet of Oracle Ravello Cloud Service s operations team. Oracle provides a systems health page outlining the status of all service components (status.ravellosystems.com). Customers and prospective customers can always reach the operations team via the support portal. The operations team proactively sends out regular updates to customers when specific actions are required. For example, if Oracle is notified that instances belonging to certain customers are being retired by AWS, the Oracle operations team reaches out to the customer and ensures that there is a transition plan for that workload to move to a different instance or even a different cloud provider. Storm warnings that have the potential to impact service operations are similarly communicated in advance, as are cloud provider rolling upgrades.

Oracle Corporation, World Headquarters Worldwide Inquiries 500 Oracle Parkway Phone: +1.650.506.7000 Redwood Shores, CA 94065, USA Fax: +1.650.506.7200 CONNECT WITH US blogs.oracle.com/oracle facebook.com/oracle twitter.com/oracle oracle.com Copyright 2016, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0116 Oracle Ravello Cloud Service: Technology Overview September 2016