Composable Infrastructure The Foundation for a Future- Proof Software-Defined Data Center

Transcription

1 Composable Infrastructure The Foundation for a Future- Proof Software-Defined Data Center Key steps to get you started with Wiwynn & Intel Rack Scale Design White Paper

2 Contents 1. Overview Enabling Composable Infrastructure for SDDC with Intel RSD How to get started planning your RSD system evaluation Start with a Minimum Set Proof of Concept (POC) Evaluate from four perspectives Options for POC Phase in RSD gradually How does it work deploy your workloads on RSD OpenStack Auto-Scaling Hypervisor Network integration Storage integration NVMeoF for VMs Kubernetes Auto-Scaling Compute Nodes Storage integration Hadoop/Spark Custom Clusters Custom Cluster based on Wiwynn Cluster Manager and RSD Conclusion Composable Infrastructure The Foundation for a Future-Proof SDDC

3 1. Overview Thanks to the admirable evolution of cloud computing and virtualization technologies, innovative business models and services can be facilitated agilely with a lower barrier to entry on capital expenditures. It fosters many cloud services and boosts the development of new technologies and applications, such as NFV, IoT, edge computing, and AI. However, new services and applications also bring challenges to data centers. Some workloads require hardware that are not compatible with the virtualization software or need to run on bare metal for performance reasons. Virtualization is not a panacea on its own any more. Cloud and communication service providers need to future-proof their data centers. Every type of hardware resource needs to be software definable. As discussed in the previous whitepaper, Wiwynn Cluster Manager with Intel Rack Scale Design is an integrated software solution to address problems that virtualization alone cannot solve while reducing Total Cost of Operation (TCO) for modern data centers as well. Intel Rack Scale Design (Intel RSD) is a logical architecture (Figure 1) that disaggregates hardware from preconfigured servers and deploys them in sharable resource pools. Wiwynn Cluster Manager is a system software that makes data centers easier to manage with features such as resource planning, massive firmware and OS deployment, and rack level, visual monitoring in real-time. This integrated solution empowers cloud service providers, telecoms, and enterprises to build their own agile, efficient, software-defined data centers (SDDC) to meet dynamic workload changes while expanding their global services and emerging technologies. Figure 1. Architecture of an RSD-based software-defined datacenter In this whitepaper, we will introduce the entire implementation of Wiwynn Cluster Manager with Intel RSD, including proof-of-concept (POC), pilot-run, large-scale phase-in, and popular workloads deployment. This is to help potential adopters evaluate the benefits and feasibility of this solution and then transfer their workloads smoothly. Composable Infrastructure The Foundation for a Future-Proof SDDC 3

4 2. Enabling Composable Infrastructure for SDDC with Intel RSD Intel RSD[1] provides a more efficient infrastructure and realizes a fully software defined datacenter with features including: Flexible and dynamic resource allocation Hardware resources are disaggregated from preconfigured servers and deployed in sharable resource pools. They can be dynamically allocated to logical nodes based on workload requirements to minimize hardware resource underutilization. Resource sharing across multiple orchestrating clusters All clusters share the same resource pools instead of maintaining spare nodes by individual clusters. Multiple workloads, such as OpenStack, Kubernetes, and Hadoop, owned by different subsidiaries or external tenants in your datacenter can leverage the data center-wide resource pooling available via RSD. Software-to-software communication No manual efforts are needed for routine operations, such as cluster commissioning, growth/shrinking, and decommissioning. The software on its own communicates with RSD to acquire or return resources. Intel RSD is also an evolving architecture for advanced features. In 2017, Intel launched RSD 2.1, featuring high throughput and low latency NVMe SSD pooling [2]. Wiwynn is one of the first OEMs in the industry to deliver Intel RSD version 2.1. In 2018, RSD 2.2 and 2.3 were released with features of telemetry and NVMe over Fabrics (NVMeoF), respectively. The latest Intel RSD 2.3 with NVMeoF realizes NVMe resource pooling at scale through Ethernet protocol without the rack boundary and enables advantages as following: Larger scale pooling and provisioning NVMe storage can be disaggregated and allocated to any compute server under the same L2 or even L3 domain. More flexible space allocation an NVMe SSD can be partitioned and allocated to different compute servers with NVMeoF. Easier to be integrated with Infrastructure as a Service (IaaS) platforms Virtual Machines (VMs) or containers may migrate between compute nodes. In contrast to direct-attached NVMe, NVMeoF can be re-attached as VMs or containers. 4 Composable Infrastructure The Foundation for a Future-Proof SDDC

5 3. How to get started planning your RSD system evaluation Wiwynn is one of the leading Intel RSD OEM partners and has deep experience in RSD-based solution development and deployment. We developed the integrated software solution, Wiwynn Cluster Manager with Intel RSD, and data center products that are compliant with Intel RSD specifications. In the rest of the chapter, we ll share how to evaluate and phase in an RSD system as well as what support is offered by Wiwynn Start with a Minimum Set Proof of Concept (POC) To evaluate the system, a minimum POC rack is required as the first step. As shown in Figure 2, the pod can only contain one or two compute blades, a data switch, a management switch, a storage server, and an NVMe JBOF. PODM Management network iscsi storage pool NVMe over PCIe storage pool Compute Compute Ethernet Fabrics PCIe cable Ethernet cable for production networks Ethernet cable for management networks NVMe over Fabrics storage pool 3.2. Evaluate from four perspectives Figure 2. Logical view of a minimum set of POC Effectiveness of asset management and resource pooling Users can add/remove a compute blade to or from your RSD rack, add/remove an SSD to or from your JBOF, and check if the resource pools reflect the changes in a timely manner to assess effectiveness. You can evaluate whether the reported asset data and topology/location detection mechanisms are satisfactory. Effectiveness and benefits of dynamic resource provisioning and node composition User can analyze node composition and decomposition and dynamic resources allocation for a composed node. By using Wiwynn Cluster Manager (CM), you can Composable Infrastructure The Foundation for a Future-Proof SDDC 5

6 also evaluate the integrated OS deployment for composed nodes, either booting from an iscsi volume or local disk drives. Feasibility of integration with existing management software Users can evaluate whether their existing infrastructure management can be integrated with RSD. Both RSD and Wiwynn CM are API-based and users can verify the details of these APIs by calling them with a RESTful API client. Feasibility of migrating workloads to RSD pods Users can verify if their workloads work with RSD. They can check the performance of their applications running on RSD-ready hardware models Options for POC Wiwynn provides a variety of RSD POC options for customers. Experience RSD at Wiwynn Certified Labs Four POC sites are available at Wiwynn HQ in Taipei, Taiwan, Facebook Disaggregate Lab in Menlo Park, US, Wiwynn Manufacturing Site in Brno, Czech Republic, and CTC Lab in Tokyo, Japan. Customers can log-in to Wiwynn s onsite POC system to experience Wiwynn Cluster Manager with Intel RSD system. They can visit the POC sites to see how to build the environment and participate in training. Experience RSD at customers sites with Wiwynn professional support Wiwynn provides POC service at customers sites. We will offer a POC rack, engineer on-site support and POC closing reports. There is no hardware procurement cost, and only a service charge is required. Build your own environment Customers can buy a POC rack to evaluate in their own environment with their own workloads. Wiwynn can support free licenses and training Phase in RSD gradually Once the customers decide to adopt RSD after evaluation, they can phase in RSD gradually while purchasing RSD-ready products to fulfill increasing demands or replace retired machines without needing to make an all-or-nothing decision. Upgrade to new RSD-ready hardware Customers can purchase new equipment compliant with Intel RSD specifications directly. Wiwynn provides various options (website) including compute nodes containing Intel Xeon Scalable processor and high density NVMe JBOF (Just a bunch of flash) for customers to build their RSD pods. Upgrade existing hardware to support RSD For customers whose existing devices are eligible to be upgraded to support RSD, they can just upgrade the BIOS/BMC firmware, accommodate per-rack software instances and adjust network configurations. 6 Composable Infrastructure The Foundation for a Future-Proof SDDC

7 Customers can contact Wiwynn representatives for Wiwynn s experienced technical support. Expand your resource pools as demand grows As customers expand their resource pools to keep up with growing demand, they can simply scale out the pool by sliding additional resources into their RSD racks. Once the new chassis is turned-on, it reports to RSD s POD Manager (PODM) automatically and can be provisioned without any manual configuration. Migrate your workloads one by one If customers want to migrate workloads to RSD pods gradually, the following are some suggested workloads to be moved first. 1. Applications that run on bare-metal and only run occasionally. For example, financial analysis may only run once a month. 2. Applications that run on bare-metal and need to scale in/out frequently. For example, an online ticket booking system may need to respond to overwhelming traffic for popular concerts or sports games but fall back to off-peak hours after these events. 3. Clusters and cloud platforms that need to scale in/out their nodes to address dynamic demands. We will discuss how to migrate OpenStack, Kubernetes, Spark, and Hadoop to RSD pods in the subsequent sections. Composable Infrastructure The Foundation for a Future-Proof SDDC 7

8 4. How does it work deploy your workloads on RSD In this chapter, we will explain how to deploy popular platforms and clusters, such as OpenStack, Kubernetes, Hadoop, Spark, and custom clusters, to RSD pods. The required steps, which APIs were used, and Wiwynn s own implementation experience will be shared. Customers can receive further support by contacting Wiwynn representatives OpenStack There are a few initiatives trying to integrate OpenStack with RSD globally. In this section, we will introduce how we deploy OpenStack through Wiwynn Cluster Manager with Intel RSD integration, covering compute, network, storage, and NVMeoF Auto-Scaling Hypervisor The total number of VMs running on your OpenStack cloud varies from time to time. Ideally, OpenStack acquires compute nodes through RSD when it has to increase hypervisors and returns surplus nodes to RSD, if any. OpenStack can automatically scale up and out through RSD with the following steps: Monitor Hypervisor Loading We implemented an auto-scaler (Figure 3), which regularly collects the loading status of hypervisors via Nova and RSD. It increases hypervisors to keep the expected service level while the total loading exceeds thresholds. Conversely, it reduces hypervisors to save energy while the total loading is too low. Increase Hypervisors A preliminary step is to prepare a system image of the OpenStack compute node. The image includes all necessary packages, such as hypervisor and Nova Compute. The Nova Compute needs to know where the Nova Controller is. A simple way to specify the URL of Nova Controller is using Nova Compute s /etc/nova/nova.conf. A more flexible way is to assign Nova Controller at runtime with a configuration manager, such as Chief or Puppet. To add a hypervisor to the OpenStack, the auto-scaler calls RSD to compose a node and deploy the system image to the composed node. After the composed node is prepared, the Nova Compute connects with the specified Nova Controller to join the OpenStack cluster. Wiwynn Cluster Manager integrates RSD node composition, RSD storage services, and OS deployment functions so that you can compose nodes and deploy the OS with a single API call to simplify the auto-scaling process. It not only supports iscsi boot (diskless server) but also local boot. 8 Composable Infrastructure The Foundation for a Future-Proof SDDC

9 Reduce Hypervisors To remove a hypervisor from an OpenStack cluster, the auto-scaler migrates running VMs to other hypervisors and then calls the RSD-based system to decompose the node. RSD releases the hardware resources back to the pools and then closes the node. Nova Controller Nova Compute list CPU/memory allocation Scale in Auto-scaler Nova Nova Controller Compute Nova Compute t Scale out CPU/mem/IO utilization of Nova Computes Node composition/decomposition RSD Figure 3. OpenStack auto-scaling mechanism working with RSD Network integration There are several possible ways to integrate OpenStack networking with RSD. In this implementation, we have verified the following models: RSD manages 802.1q VLAN tags while OpenStack provides tenant networks with VxLAN. RSD manages 802.1ad service provider virtual networks while OpenStack provides tenant networks with either VLAN or VxLAN Storage integration OpenStack can leverage the software-defined storage managed by RSD. In actual implementation, we integrate Ceph with RSD to provide iscsi storage services. Cinder links to the same Ceph as its backend so that our OpenStack shares the same storage pool with other clusters and platforms. Composable Infrastructure The Foundation for a Future-Proof SDDC 9

10 NVMeoF for VMs 4.2. Kubernetes Wiwynn developed a driver for Cinder to communicate with RSD for NVMeoF volume creation, query, and deletion. Besides the iscsi volumes, you can provision a VM with NVMeoF volumes to boost up its I/O performance. This section discusses how to integrate Kubernetes with RSD to automatically scale out/in nodes, as shown in Figure 4. Master API Server Scheduler Autoscaler RSD Node Node Kubelet POD POD POD Container Container Container Proxy Kubelet POD POD POD Container Container Container Proxy Figure 4. Kubernetes auto-scaling with RSD Auto-Scaling Compute Nodes The loading of current nodes should be monitored to automatically scale in or out Kubernetes nodes when the workload demand changes as described in details below. Monitor utilization of compute node A daemon to gather data periodically is required to monitor the status and utilization of your compute nodes. It can run in the same host with Kubernetes Master or anywhere else within the same management network. In Wiwynn s implementation, we collect data from the following sources: Node Status API[3] Several data points are worth watching in the Conditions field, such as OutOfDisk, MemoryPressure, PIDPressure, DiskPressure, etc. 10 Composable Infrastructure The Foundation for a Future-Proof SDDC

11 RSD Telemetry With the RSD Telemetry function, CPU, memory, and disk I/O utilization of compute nodes can be monitored without installing an agent in the host OS. By continuously analyzing the streams of monitoring data, you can identify nodes utilization status and trigger auto-scaling actions accordingly. Monitor new requests Another key configuration concern is that new requests may fail due to insufficient resources. Some plug-ins at the API layer can be used to listen to new requests and proactively trigger a scale-out action on the fly. Alternatively, the auto-scaling agent can keep querying the pod API to check if there are pending pods waiting for execution and request RSD to compose additional nodes to expand your Kubernetes cluster. Scale out nodes To automatically scale out nodes, the system image should be prepared in advance. The following ingredients are required in the base OS. Kubelet, Kube proxy, and container runtime A run-once startup script to register itself with Master Run commands where API_SERVER_IP and an authentication token could be either delivered by a configuration manager or queried online. kubelet --api_servers= --v=2 -- enable_server --allow-privileged kube-proxy --master= --v=2 The script will run and register at the first time the composed node starts up. When you determine to add a node to the Kubernetes cluster, call the RSD API to compose a node, deploy the system image to the node, and boot it up. Wiwynn Cluster Manager with Intel RSD provides an integrated node composition function which simplifies the scale out steps by requiring only one API call. Scale in nodes When the node utilization is too low, you can scale in nodes as well. The autoscaling mechanism just needs to identify which node to remove, label it as NoExecution, and call RSD to return it at the proper time (for example, after Descheduler[4] migrates running pods to other nodes, if it s adopted). RSD will decompose the Node, return the resources to pools, and then power it off. Composable Infrastructure The Foundation for a Future-Proof SDDC 11

12 Storage integration 4.3. Hadoop/Spark Kubernetes permanent volumes can be served by RSD storage services as well. Containers in Kubernetes can attach NVMeoF volumes provided by RSD to boost the I/O performance, just like the OpenStack use cases discussed in the previous section. The architecture of Hadoop and Spark clusters working with RSD is shown as Figure 5. Hadoop Distributed File System (HDFS) is composed of a Name Node and a number of Data Nodes while Apache Hadoop YARN acts as a compute resource manager which comprises a Resource Manager and a Node Manager in each compute node. In this configuration, Resource Manager is hosted in Name Node, and a Node Manager resides in each Data Node. MapReduce or Spark clients submit applications to YARN for resources allocation. MapReduce Spark YARN HDFS Figure 5. YARN-based Hadoop and Spark Monitor Workloads As shown in Figure 6, Resource Manager accepts job requests from application clients and maintains multiple queues of to-do jobs with different priority to store. If jobs cannot be executed in a timely basis, auto-scaler will add data nodes. On the other hand, if the queues are empty for a while it removes partially idle nodes. Scale Data Nodes To scale data nodes, the first step is to prepare a system image of the Data Node that includes Hadoop packages plus the Name Node s SSH key. Alternatively, the SSH key can be delivered by a configuration manager at runtime. When the auto-scaler determines to add a node, it calls RSD to compose a node, deploys the system image to the node, and adds the IP address of the node to the Name Node s 12 Composable Infrastructure The Foundation for a Future-Proof SDDC

13 /etc/hosts. Conversely, the auto-scaler calls RSD to return a node and removes its IP address from the Name Node s /etc/hosts, when there is underutilization. Clients Submit jobs Resource Manager Name Node Autoscaler RSD Node Manager Data Node Node Manager Data Node Figure 6. Hadoop auto-scaling design 4.4. Custom Clusters Customers may want to run self-defined clusters on top of RSD to provide services agilely. Customized scripts are required to deal with RSD in the entire workload life cycle (Figure 7). The required features at each phase are described as below. Creation The script calls RSD to compose nodes with required resources, deploys OS, and gets your applications up and running. Monitoring/profiling The script/daemon keeps monitoring the composed nodes via RSD and/or IPMI. By subscribing RSD Telemetry data on CPU/memory/IO bandwidth utilization continuously, you can profile your nodes to see if there are performance bottlenecks. It is of great help for deciding to scale up or scale out. Scale in/out The script may call RSD to adjust in-node resources or add/remove nodes to/from your cluster as your demand changes. End After the workloads/clusters are finished, the script needs to call RSD to decompose all the provisioned resources. This way, customers can control resource allocation autonomously through software and without human interference. Composable Infrastructure The Foundation for a Future-Proof SDDC 13

14 Deployment Creation Call RSD to get resources Running with auto-scaling Monitoring Call RSD to get Telemetry data Scaling in/out Call RSD to shrink or grow Decommission End Call RSD to decommission resources Figure 7. Life cycle of an auto-scalable application on top of RSD Custom Cluster based on Wiwynn Cluster Manager and RSD Customers can also leverage Wiwynn Cluster Manager (CM) to implement autoscaling for the self-defined cluster easily (Figure 8). Monitoring Wiwynn Cluster Manager provides real-time rack level visual monitoring function. CM Monitoring can regularly retrieve all in-band, out-of-band, and RSD Telemetry monitoring data from all composed nodes and store them in its database. You can set up thresholds for each monitored item. CM Monitoring generates logs if any of the thresholds is hit. You can also subscribe the data streams with CM Monitoring and conduct more sophisticated examinations. Log correlation CM Event Manager collects logs from all variety of sources and correlates them to find out important events or incidents that need immediate healing actions or administrators attention. You can set proper correlation policies to detect over- or under-utilization of the clusters and set an associated auto-healing policy to trigger the auto-scaling function. 14 Composable Infrastructure The Foundation for a Future-Proof SDDC

15 Auto-healing CM Auto-Healing is a framework that triggers auto-healing scripts on an event that meets the criteria of an associated correlation policy. The pre-defined script will then call CM Orchestrator for resource adjustment. To increase nodes, CM Orchestrator calls RSD to compose nodes and deploys your OS image to them at once. RSD Telemetry & sensor readings RSD-based Orchestrator User-uploaded scrips for autoscaling Managed Cluster In-band data Monitoring Event Correlator Wiwynn Cluster Manager (CM) Auto- Healing Figure 8. Auto-Scaling based on Wiwynn Cluster Manager and RSD Composable Infrastructure The Foundation for a Future-Proof SDDC 15

16 5. Conclusion Composable infrastructure sets compute, storage and networking resources free from their physical locations and enables them to be dynamically combined to meet the unique needs of any application. It offers agility and efficiency of IT hardware utilization and is the foundation of future-proof software-defined data centers. Intel RSD is an open, extensible, and interoperable industry standard that defines an achievable architecture of composable infrastructure. The evolving standard release specifications of RSD included features for Telemetry and NVMeoF in In the future, more types of disaggregated resources, such as FPGA, GPU, and NIC, to compute nodes will be supported. As the ecosystem continues to grow in the coming years, more and more platforms and applications, such as OpenStack, Kubernetes, and some commercial products, will work with RSD. Wiwynn is one of the leading Intel RSD OEM partners. We integrated Intel RSD into our Wiwynn Cluster Manager and developed compliant data center products. The integrated solution provides a friendly interface to manage legacy and Intel RSD enabled pods. It also offers rich management features such as resource planning, auto-healing, and real-time rack level visual monitoring. We are eager to assist customers to transform to real composable and softwaredefined data centers through our deep experience in RSD development and deployment. We can assist customers with POC projects, large scale deployment, and workload migration. Remote demonstrations and four Wiwynn Certified POC sites in Taiwan, US, Japan and Czech are ready for customer testing. POC rack procurement as well as on-site engineering support and training are also available. We provide a wide range of compute and storage solutions and can customize suitable configurations based on customers needs. Furthermore, our software expertise in RSD APIs, OpenStack, Kubernetes, Spark and Hadoop can help customers migrate workloads and platforms to RSD pods smoothly. Evolving your data center into a true SDDC is not a difficult task anymore. Engage with Wiwynn and embrace the power of composable infrastructure by gradually phasing in RSD to your data center. You will get the benefits of the unified and simplified management, experience increased efficiency of resource provisioning, and be prepared for new RSD features in the near future. 16 Composable Infrastructure The Foundation for a Future-Proof SDDC

17 Reference [1] Intel RSD official website at [2] Available at [3] Online documentation at [4] Github at Learn More About Wiwynn Cluster Manager with Intel Rack Scale Design Wiwynn website: Wiwynn Cluster Manager: White Paper: Datasheet: %20CM_ pdf YouTube: Need Additional Information? Technical support: Composable Infrastructure The Foundation for a Future-Proof SDDC 17