Huawei FusionCloud Desktop Solution 5.1 Resource Reuse Technical White Paper HUAWEI TECHNOLOGIES CO., LTD. Issue 01.

Transcription

1 Huawei FusionCloud Desktop Solution 5.1 Resource Reuse Technical White Paper Issue 01 Date HUAWEI TECHNOLOGIES CO., LTD.

2 2014. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademarks and Permissions and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders. Notice The purchased products, services and features are stipulated by the contract made between Huawei and the customer. All or part of the products, services and features described in this document may not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, either express or implied. The information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute a warranty of any kind, express or implied. Huawei Technologies Co., Ltd. Address: Website: Huawei Industrial Base Bantian, Longgang Shenzhen People's Republic of China ii

3 Contents Contents 1 Resource Reuse Overview Background Customer Benefits Key Technologies VM Live Migration Memory Overcommitment VM QoS DRS DPM Resource Reuse Solution Resource Reuse Architecture Components of the Resource Reuse Solution Resource Reuse Among Services Time Division Multiplexing of VDI Services Energy Conservation and Emission Reduction Examples Examples About Resource Reuse Among Services Examples About Energy Conservation and Emission Reduction iii

4 1 Resource Reuse Overview 1 Resource Reuse Overview 1.1 Background After cloud platforms and virtual desktops are deployed, enterprise services, especially virtual desktop services and other services (such as the CI, automation test, supercomputing, rendering, and batch processing), need to make full use of computing resources. This meets time division multiplexing requirements. If no service reuses these computing resources and these computing resources are still idle, energy conservation and emission reduction will be considered. For example, users use virtual desktops in the daytime, and unused computing resources are released at night to run other services (such as the CI, automation test, supercomputing, rendering, and batch processing). Before users come to the office the next day, the system reclaims these computing resources for virtual desktop use, improving the resource reuse rate. 1.2 Customer Benefits Resource reuse allows idle computing resources to be efficiently used. This improves resource utilization, supports energy conservation and emission reduction, and reduces investment costs. Resources are shared by multiple applications, saving costs. Flexible application scalability is supported. The flexible resource reuse policy meets higher scale-out requirements. Automated resource adjustment is supported, which strengths automation capabilities of management systems. Energy conservation and emission reduction are supported, saving investment costs. 4

5 2.1 Key Technologies VM Live Migration Figure 2-1 VM live migration rules Migration domain Migration domain General storage server Volume Volume Volume Volume Volume Volume Host 1 Host 2 Host 1 Host 2 Virtual machine (VM) live migration allows a VM to be migrated from a physical server to another physical server without interrupting services. The migration is not perceived by VMs. Live migration enables memory information to be exchanged and converted. The VM hypervisor quickly copies the memory data and migrates the memory data to the destination host without interrupting services. VM live migration allows VMs to be managed across multiple physical servers. VM live migration benefits customers in the following aspects: Reduces service running costs. Servers are idle in some time periods. If services running on multiple physical servers are migrated to one or a few number of physical servers and physical servers on which no services are running are stopped, service running costs can be reduced. At the same time, energy is saved and emission is reduced. Balances the load of physical servers. If the load of a physical server is heavy, some VMs running on the physical server can be migrated to other physical servers 5

6 dynamically. This lessens the load of the physical server and balances the load of all physical servers in the entire cluster. Ensures high system availability. If a physical server is running abnormally, services running on the physical server can be migrated to another physical server that is running properly. This provides a high availability system. Supports online hardware upgrading without interrupting services. When physical server hardware needs to be upgraded, all VMs running on the physical server can be migrated to another physical server, and then the upgrading is performed for the physical server. After the upgrading is complete, these VMs are migrated back to the upgraded physical server. This implements online hardware upgrading without interrupting services Memory Overcommitment Figure 2-2 Memory overcommitment Memory sharing Memory ballooning Intelligent memory overcommitment policy Memory swapping Memory overcommitment allows the physical memory of servers to be allocated to VMs at different times by using specified policies (such as memory ballooning, memory sharing, and memory swapping). Memory resources on idle VMs are released for other VMs, and memory overcommitment is balanced on VMs. In this way, the available memory resources for VMs are greater than the memory resources provided by the physical server, improving the VM density on the server. Memory overcommitment has the following advantages: Improves memory utilization. Memory overcommitment allows the hypervisor to reclaim free memory resources from light-loaded VMs and assign these memory resources to heavy-loaded VMs. Improves the consolidation ratio. Memory overcommitment enables VMs to consume less host memory. As a result, more VMs with high performance can be created on hosts. The following describes memory overcommitment technologies: 6

7 Memory Ballooning Figure 2-3 Memory ballooning Memory Sharing With memory ballooning, memory that was allocated for a VM can be given to another VM without shutting down the machine. Memory ballooning is a lightweight memory overcommitment technology. Memory resources are occupied by perceiving the operating system (OS), which has little impact on the performance. This technology is perceived by VM users. (Actual VM memory decreases.) Memory ballooning relies on free memory resources of the OS. When the OS memory load increases, memory cannot be returned automatically. Figure 2-4 Memory sharing VMs on the unified virtualization platform (UVP), a Huawei-developed hypervisor, are identified based on blktap2 linked clone to read data in the same disk, achieving memory sharing. In desktop cloud scenarios, a large number of same memory pages exist on the VMs that have the same OS. Memory sharing enables repeated memory pages to be reduced, which is not perceived by VMs. 7

8 Memory Swapping Enabling and disabling memory sharing bring extra costs. Figure 2-5 Memory swapping VM QoS Memory page data of the Guest OS are swapped to disks by the hypervisor to release memory. Memory swapping is not perceived by VMs and supports measurable reuse (memory ballooning and memory sharing do not support measurable reuse.). That is, when memory is insufficient, memory swapping ensures that VMs or physical machines are running properly. VM memory is swapped to disks so that the read and write speed is reduced and VM performance is greatly affected. The VM quality of service (QoS) consists of the CPU QoS and memory QoS, as shown in Figure 2-6. Figure 2-6 VM QoS 8

9 The CPU QoS functions are designed to enhance user experience. The CPU QoS allows different VMs to have different resource usage priorities. VMs with a high priority can be allocated with resources first, ensuring the high service level agreement (SLA). In addition, the upper limit, share, and reservation of CPU computing resources can be set to ensure the marketability and availability of computing resources. CPU QoS parameters are as follows: Limit: Set this parameter when resources are isolated strictly. VM resources are isolated to ensure good user experience. Reserved: Set this parameter when resources need to be evaluated. Minimum resources are provided to ensure resource marketability and service quality. CPU quota: Set this parameter when resources are reused. Resources are allocated based on user levels. Memory overcommitment allows memory resources on idle VMs to be released for other VMs. To meet specific memory requirements and improve user experience, the memory QoS allows different VMs to set different memory resource usage priorities. VMs with a high priority can be allocated with resources first, ensuring the high SLA. In addition, the share and reservation of memory resources can be set to ensure the marketability and availability of memory resources. Memory QoS parameters are as follows: Reserved: Set this parameter when resources need to be evaluated. Minimum resources are provided to ensure resource marketability and service quality. Memory quota: Set this parameter when resources are reused. Resources are allocated based on user levels DRS The dynamic resource scheduler (DRS) regularly checks load status of physical hosts in the same resource cluster and migrates VMs between different physical hosts by using intelligent load balancing algorithms to achieve load balancing. The DRS regularly monitors the usage of CPU and memory resources of all physical hosts and VMs in the same cluster. A comparison is made between the current load status of physical hosts and the ideal load status of physical hosts (configured threshold of the DRS policy). If 50% of sampling values are greater than the threshold or meet association rules, VM migration policies will be performed. Figure 2-7 shows the VM distribution view before the migration. 9

10 Figure 2-7 VM distribution view before the migration Resource cluster 1 Figure 2-8 shows the VM distribution view after the migration: Figure 2-8 VM distribution view after the migration Resource cluster DPM The dynamic power management (DPM) feature allows load status of physical hosts in resource clusters to be checked regularly. When loads of physical hosts are heavy, new physical hosts are powered on to share loads of other servers by using the DPM module. When loads of physical hosts are low, VMs running on a low-load physical host can be migrated, and then the low-load physical host is powered off so that the VMs run on other physical hosts. This reduces power consumption of the entire resource clusters. As shown the following figures, in the same resource cluster, the load of physical host 2 is low before the DPM scheduling is enabled. The system regularly schedules the DPM algorithm (600s by default) to migrate VM 21 running on physical host 2 to physical host 1 and power off physical host 2 to reduce power consumption. 10

11 Figure 2-9 shows the VM distribution view before the migration. Figure 2-9 VM distribution view before the migration Resource cluster 1 Figure 2-10 shows the VM distribution view after the migration and shutdown. Figure 2-10 VM distribution view after the migration and shutdown Resource cluster 1 Shutdown 2.2 Resource Reuse Solution Resource Reuse Architecture Figure 2-11 shows the resource reuse architecture. 11

12 Figure 2-11 Resource reuse architecture Non-VDI VMs VDI VMs The VDI proactively releases free resources and has a priority to obtain resources. CI VMs The AME schedules other non-vdi services to reuse released free resources. The VRM dynamically schedules resources in resource clusters to achieve load balancing. Computing resource pool of resource clusters The resource reuse solution has the following features: VMs with virtual desktop infrastructure (VDI) services are deployed on the same resource cluster as VMs with other services. VMs with VDI services run on the same host as VMs with other services. Host CPU and memory resources can be reused. The FusionCompute supports key technologies for resource reuse. The FusionManager supports the QoS to configure the service priority and computing resource threshold. In addition, the FusionManager provides application scalability capabilities to schedule and manage resources for VMs with non-vdi services. The FusionAccess supports QoS configuration of VMs on which VDI services are running to achieve good user experience. In addition, scheduled tasks are implemented to schedule and manage these VMs, so that resources of these VMs that meet resource reuse policies can be applied and released regularly. The FusionCompute, FusionManager, and FusionAccess can be configured properly based on plans to achieve resource reuse and meet various application scenarios Components of the Resource Reuse Solution Figure 2-12 shows the components of the resource reuse solution. 12

13 Figure 2-12 Components of the resource reuse solution The FusionCompute supports computing resource scheduling. (Enabling this function is recommended.) Resource reuse among services The FusionManager supports application scalability. (The function is mandatory when multiple services exist.) The FusionAccess supports scheduled task management (scalability of VDI services). Time division multiplexing of VDI services The FusionCompute supports computing resource scheduling. (Enabling this function is recommended.) The FusionAccess supports scheduled task management (time division multiplexing of VDI services). Energy conservation and emission reduction The FusionCompute supports the DPM. (Enabling this function is mandatory.) The FusionAccess supports scheduled task management (scalability of VDI services). The resource reuse solution contains the FusionCompute, FusionManager, and FusionAccess components, whose functions are as follows: The FusionCompute supports computing resource scheduling, that is, the DRS function, which needs to be enabled for resource clusters on the FusionCompute portal. Policies, such as the automation level, measurement factor, and migration threshold, need to be configured as required to achieve load balancing of physical hosts in resource clusters. Figure 2-13 Computing resource scheduling configuration The FusionCompute also supports the DPM, that is, the power management automation function, which needs to be enabled for resource clusters on the FusionCompute portal. Policies, such as the power management threshold, need to be configured as required to achieve light traffic integration and heavy traffic separation for physical hosts in resource clusters. Physical hosts can be powered on and powered off based on load status to reduce power consumption. 13

14 Figure 2-14 Power management configuration The FusionManager supports application scalability, that is, the automatic resource sharing function. When multiple service groups exist in a resource cluster, different policies are configured to differentiate priorities of service groups, and free resources in a service group can be released for other service groups. Resources in the same service group are also scalable in different periods by using configured policies. When resources are insufficient, resources can be reclaimed and re-allocated based on priorities and weight of service groups to ensure resource availability for high-priority and high-weight services. The FusionAccess supports scheduled task management, that is, computing resource scalability of VDI services. Administrators can perform the following operations to manage VDI VMs: Create scheduled tasks: include starting, stopping, hibernating, waking up, and restarting VMs. Specify triggering time: triggered immediately, in a specific period, by day, by week, and by month, and supporting multiple times of triggering within a day. Specify VMs as required: VMs and VMs in desktop groups. Select proper policy configurations Resource Reuse Among Services Figure 2-15 shows the application scenarios of resource reuse among services. 14

15 Figure 2-15 Resource reuse among services Local desktop development Submit tasks. Perform HLT tests. Automation factory (HLT) Code library VDI desktop CI Master Local desktop development 2 Distribute tasks. Code Center Update codes. Archive results. VDI desktop Cloud computing platform: one RH2285 server = MSX CI environment = CI Master + Code Center + Seven Windows servers + Seven Linux servers Implement tasks. VDI desktop VDI desktop cloud Static check resource pool (Windows 2003) Linux compiling resource pool (SUSE) Packaging resource Packaging resource pool (Windows 2003) pool (SUSE) CI services Time division multiplexing: Users use virtual desktops in the daytime, and unused computing resources are released at night to run CI services (as a CI agent node). Before users come to the office the next day, CI services are disabled. VDI users can use virtual desktops continuously. Resource reuse 24/7 hours: When free VDI resources exist in the daytime, these resources can be released for CI services. When VDI user resources are insufficient, CI resources are released automatically for VDI services Time Division Multiplexing of VDI Services Multiple groups of VDI VMs share same computing resources and reuse different groups of resources in different periods. For example, a group of VDI VMs are used to access virtual desktops in the daytime, and another group of VDI VMs are used to access virtual desktops at night. To reuse computing resources, scheduled startup and shutdown tasks need to be created on the task management page of the FusionAccess Energy Conservation and Emission Reduction Figure 2-16 shows the application scenarios of energy conservation and emission reduction. 15

16 Figure 2-16 Energy conservation and emission reduction Light traffic integration Heavy traffic separation The CPU usage of VMs is low at night, and some VMs are stopped. In this case, system energy policies are used, and free physical hosts are powered off to save energy. When the system load is heavy, some physical hosts are powered on, and VMs are migrated to the new physical hosts to ensure good user experience. The system determines which physical host is to be powered on or off to reduce the number of VMs that need to be migrated. To ensure quick response, the system ensures that only a few physical hosts are hibernated. 2.3 Examples Examples About Resource Reuse Among Services An enterprise needs to deploy VDI and CI services in the same resource cluster. There are about 300 VDI VMs and several CI VMs. During working hours in the daytime (08:00 18:00), computing resources need to be used to run VDI services. In other periods (19:00 07:00), free resources are used to run CI services. Specific configurations are as follows: On the FusionCompute, enable the DRS of the resource cluster, and configure parameters about thresholds as required. On the FusionManager, configure application scalability. 1. Add intra-group policies for CI services as required, such as capacity expansion and reduction. 16

17 Figure 2-17 Adding intra-group policies 2. Add inter-group policies as required. Figure 2-18 Adding inter-group policies Parameters about the policies are described as follows: Reserved CPU and Reserved memory: specify resources reserved for a virtual data center (VDC) in a resource cluster. (A VDC is a logical computing resource pool created by an administrator. It is smaller than a logical cluster.) If the remaining number of resources in the cluster where the VDC is located is less than the parameter values, resources will be reclaimed. Max. CPU Usage and Max. Memory Usage: specify the maximum number of available resources in a scaling group. If the actual number of CPU and memory resources is greater than the two values, resources will be reclaimed. 17

18 Reserved CPU Usage and Reserved Memory Usage: specify the minimum number of available resources in a scaling group. Priority: specifies the priority (high, medium, or low) of services in a scaling group. A higher priority indicates a lower probability to reclaim the corresponding resources. The status of resource reclaiming triggered by inter-group policies can be queried from policy logs. 3. Create tasks as required (possible to collaborate with FusionAccess scheduled tasks). Figure 2-19 Creating tasks Parameters about the tasks are described as follows: Triggering Mode: The options are scheduled triggering and periodic triggering. Scheduled triggering: Triggered only once according to the configured date and time. Periodic triggering: Triggered by day or week. Associate Inter-Group Policy: Only one inter-group policy can be selected. Associate Intra-Group Policy: Multiple intra-group policies can be selected. On the FusionAccess, create scheduled tasks, such as stopping, hibernating, starting, or waking up VMs. The specific procedure is as follows: 1. Specify triggering time as required. 18

19 Figure 2-20 Specifying triggering time 2. Specify VMs as required. Figure 2-21 Specifying VMs 3. Select proper policy configurations. 19

20 Figure 2-22 Selecting proper policy configurations Examples About Energy Conservation and Emission Reduction An enterprise needs to deploy VDI services in the same resource cluster. There are about 300 VDI VMs. During working hours in the daytime (08:00 18:00), computing resources need to be used to run VDI services. In other periods (19:00 07:00), computing resources are idle. Specific configurations are as follows: On the FusionCompute, enable the DPM of the resource cluster, and configure parameters about thresholds as required. On the FusionAccess, create scheduled tasks, such as stopping, hibernating, starting, or waking up VMs. The specific procedure is as follows: 1. Specify triggering time as required. Figure 2-23 Specifying triggering time 2. Specify VMs as required. 20

21 Figure 2-24 Specifying VMs 3. Select proper policy configurations. Figure 2-25 Selecting proper policy configurations 21