POUX: Performance Optimization Strategy for Cloud Platforms based on User Experience

Transcription

1 POUX: Performance Optimization Strategy for Cloud Platforms based on User Experience Zhijin Qiu, Zhongwen Guo, Yanan Sun, Yingjian Liu and Yu Wang Ocean University of China, Qingdao, Shandong, China University of North Carolina at Charlotte, Charlotte, North Carolina, USA Alcatel-Lucent Shanghai Bell, Shanghai, China Taiyuan University of Technology, Taiyuan, Shanxi, China Abstract Cloud computing has been widely used in almost every areas of our life. System virtualization is one of the key technologies in cloud computing. However, it is a great challenge to achieve a balance among the virtual machines (s) in different physical machines (PMs) by migrating overloaded s to underloaded PMs while minimizing the number of migrations. Most of existing performance optimization strategies only concern about the hardware parameters (i.e., CPU, memory and I/O, etc.) of s, but the important user experience parameters (i.e., response time and through rate, etc.) have been ignored. We propose a novel Performance Optimization strategy based on User experience (POUX) for the cloud platform. To obtain the parameters of user experience, we design a management architecture of the cloud platform, and define the standardized interfaces between s and management center. Due to the unique characteristics of load balancing in the cloud platform, performance optimization problems are often NP-hard. Therefore, we propose a heuristic for automatic performance optimization, which plays a tradeoff among hardware utilization, user experience and the number of migrations. We use the CloudSim simulator and our deployed small-scale real-world testbed to evaluate the performance of POUX. Various experimental results have indicated that our cloud-based management architecture and performance optimization strategy not only significantly reduce the number of migrations, but also ensure a better user experience. I. INTRODUCTION Cloud computing has been emerging as the dominating Internet-based computing approach. It is flexible since the shared resources of hardware and software are provided to the remote users. Infrastructure as a Service (IaaS) is an important service type in cloud computing, which provides server, storage, and network hardware for users. To supply users with stable, efficient and economical computing resources, IaaS uses the virtualization technology to consolidate the underlying hardware resources, which can realize the dynamic scaling of virtualized resources. Virtualization polymerizes the computational power of a large number of independent computers, and smartly allocates them to improve the service flexibility of the cloud platform. Providers dynamic allocate computing resources to meet user needs, and avoid waste of computing resources. At present, the typical virtualization technologies mainly include ESX, Hyper-V, Xen, and K [1] [3]. Different from the traditional computer system architecture, virtualization technology adds the virtual machine monitor (M) between the operating system and hardware to deploy and manage s resources, which creates loose couplings between and PM. Multiple s can be deployed in the same PM. Each is assigned to independent CPU, memory and I/O resources from PM as separate computing nodes. Through M, s can be flexibly migrated among different PMs, and without impacting the user experience. When a provider leases services to a user, service level agreement (SLA) is required to ensure that the provider can provide services corresponding to the user s expense. It is a great challenge that providers are not only compliance with SLA, but also reduce the waste of cloud computing resources as far as possible. The requirements of the application running on the are constantly changing. Therefore, it is necessary to dynamically adjust the configuration properties of the according to the load and user experience of the on the cloud platform. Even more, the overloaded s need to be migrated on different PM, so that the entire cloud platform achieves balancing loads, reduces costs, and meets SLAs. Overall, how to effectively manage the cloud platform based on virtualization technology to achieve the high performance is critical but challenging. The current load balancing algorithms usually only concern about the hardware parameters of s and PMs, but ignore user experience, which is a more direct fact for performances. Another issue about current performance optimization solutions is that their evaluation functions are usually simple summations of different monitoring parameters. However, many overloaded s have only one hardware parameter out of the threshold, which may not affect user experience. Thus, the existing evaluation functions may be unfair. To solve these problems, this paper tries to perform the optimization over the cloud platform based on user experience. We put forward a performance optimization strategy, POUX, to ensure high performance of the cloud platform. This strategy includes methods of evaluating overload and PM, selecting s to migrate, and finding the most suitable destination PMs. It realizes the load balancing not only among different s of the same PM, but also among different PMs. This proposed heuristic strategy is based on user experience to

2 avoid only concerning the hardware parameters. To implement POUX based on the Xen system architecture, we also design a management architecture of the cloud platform to acquire, analyze and manage the performance of PMs and s on the cloud platform. The contributions of this paper are summarized as follows. We have designed a novel management architecture of the cloud platform, and developed a standardized communication interface, which can not only obtain each PM and s utilization of CPU, Memory and I/ O through the Xen s XAPI, but also derive the response time and throughput rate of applications in each through the UAPI interface of our development. Evaluation indicators of the proposed POUX include both parameters of hardware and user experience. So even if we increase the threshold of hardware parameters, the user experience is not affected and the number of migrations can be reduced. POUX uses a heuristic method to perform the migration strategy, which considers the different resource intensities of the overloaded and the destination PM, respectively. The efficiency of performance optimization, and the robustness of the cloud platform are improved due to the reduction in the number of migrations. In the process of migration, POUX improves the rate of migration by selecting the migrated that is greater than and closest to the overloaded resource size. The rest of this paper is organized as follows. Section 2 briefly describes the related work. In Section 3, the management architecture of the cloud platform is proposed, and the performance optimization problem is defined. We put forward a heuristic strategy POUX in Section 4. Section 5 implements and validates the management architecture and POUX. Finally, Section 6 summarizes the paper with remarks on possible future work. II. RELATED WORK With the development of virtualization technology, the flexibility of computing service makes the computer performance been fully utilized, and promotes the popularization of cloud computing. At present, the major virtualization technologies include Ware, K, Hyper-V and Xen [1] [3]. Even though different technologies adopt different methods and strategies in aspect of performance management, they all can divide the resources at the server and dynamically manage them according to load fluctuation. Ware is more mature virtualization product on the market, but it is expensive, non-open source, and relevant low performance. Kernel-based Virtual Machine (K) is a full virtualization solution for Linux on x86 hardware and open-source software, but it consumes more resources at the host. Hyper-V, formerly known as Windows Server Virtualization, is a native hypervisor. It can create s on x86-64 systems running Windows, but its applicability is poor in the Linux server. Xen Project is a hypervisor using a microkernel design, which provides services that allow multiple computer operating systems to execute on the same computer hardware concurrently. The Xen Project community develops and maintains Xen Project as free and open-source software. It also supports full virtualization and paravirtualization technology, of which is its unique advantages. Citrix XenServer based on Xen virtualization technology is a virtualization operating system, rather than the software deployed on the operating system. It does not require the support of the underlying native operating system compared with traditional software, so it can take full advantage of server performance. When the PM downtime or the load is unbalanced, the needs to be migrated. To improve the convenience of migration, iscsi [4], [5] and Network File System (NFS) [6], [7] are used in the cloud platform. When performing migrations, we only need to copy a small amount of data such as memory data, instead of migrating large amount of hard disk storage data. Currently, the methods of migration include static migration and live migration. Static migration method needs to pause the before migrating, then copies the data of system state to the destination PM, and finally rebuilds the s state on the destination PM. It is simple, but it requires to shut down when is migrated [8]. Clark et al. [9] and Sun et al. [1] propose live migration, which uses a memory pre-copy strategy to migrate s between different PMs without disrupting tasks on s. The downtime is reduced to milliseconds. However, live migration leads to performance loss and energy overhead that cannot be ignored in the modern cloud platforms, especially in the case with limited network bandwidth [11]. To achieve the load balance among PMs and maintain the highest performance of PMs, many methods have been proposed to deal with PM overloading problem through migrations [12], [13]. Wood et al. [14] present Sandpiper, a system that automates the task of monitoring and detecting hotspots, determining a new mapping of physical to virtual resources, resizing s to their new allocations, and initiating any necessary migrations. Sandpiper tries to move load from the most overloaded PMs to the most underloaded PMs. It defines volume for s and PMs: volume = (1/(1 u cpu )) (1/(1 u men )) (1/(1 u net )), where u is resource utilization. It also defines a volume-to-size ratio (VSR) for each : V SR = volume/size, where size is the memory footprint of the. It then migrates the with the maximum VSR to the PM with the least volume. Weng et al. [15] propose a management framework for the virtualized cluster system, and present an automatic performance tuning strategy to balance the workload in the virtualized cluster system. They adopt an exponential function for the cost of a with a given load. The cost of j running on node i is defined as: cost[i][j] = cost c [i][j]+cost m [i][j]+cost n [i][j], where cost c, cost m, cost n represent CPU cost, memory cost and network cost, respectively. However, all previous methods statically assume equal importance for different resources, which may not be correct due to the different time-varying demands on different resources in each PM. Chne et al.

3 Xen M Xen M XAPI UAPI Xen M Management PM Acquisition Processing Execute Monitoring Engine Dom- v1,1 v1,m1 Monitoring Engine Dom- v2,1 v2,m2 Node PM p1 Node PM p2 Machine Monitor (M), Dom- and multiple s. M is located between the operating system and the hardware, and in full control of allocating the physical resources to the [15]. Dom- has special privileges, like being able to access the hardware directly. We can create, configure, and destroy s through the Domain management and control tools located in Dom-. In particular, M enables dynamic reclamation of idle resources with more s. Each node PM contains multiple s. The application and the Monitoring Engine are deployed in each. Management PM manages the cluster through the XAPI interface, and collects the application information, which is obtained by the Monitoring Engine through the UAPI interface. Monitoring Engine Dom- vn,1 vn,mn Fig. 1. Management architecture of the cloud platform. [16] propose a Resource Intensity Aware Load balancing method (RIAL), which dynamically assigns different weights to different resources according to the utilization intensity of each in the PM. Wood et al. [14] and Weng et al. [15] propose a pre-load allocation based on the prediction of the future performance requirements. However, the above methods only focus on the utilization intensity of hardware resources, but ignores the user experience in different loads. In this paper, we aim to take the user experience into migrations. A. Architecture Design III. ARCHITECTURE OVERVIEW To implement PM migration algorithm, Wood et al. [14] and Weng et al. [15] put forward their own management architecture of the cloud platform. Similarly, in order to realize the performance optimization strategy based on user experience, as shown in Figure 1, we propose a management architecture of the cloud platform based on Xen. Management PM has three functions, which are acquisition of s status information (acquisition module), analysis of the status information (processing module) and scheduling (execute module). Acquisition module obtains each s CPU, Memory and I/O information through the official XAPI interface, and the running information of the application system in each through the our custom UAPI interface, such as the system response time, the number of task requests and the number of task responses. Processing module analyzes the information of status, finds overloaded s, and selects the migrant and destination PM according to the load balancing policy. Execute module performs a migration through the XAPI interface. Node PM is a PM as a computing node installed with the operating system of XenServer, which consists of Xen Virtual Node PM pn B. Architecture Description Each the cloud platform includes multiple PMs, each containing multiple s. The PM set is defined as P = {p 1, p2, p n }. The set for each PM is defined as V = {v i,1, v i,2, v i,mi }, where m i represents the number of s that p i contains. Ri,j,k v represents the capacity of each s hardware parameters. h v i,j,k (t) represents the utilization intensity of each s each hardware resources at time t, such as h v i,j,k (t) = {v i,j.cpu(t), v i,j.mem(t), v i,j.io(t)}, where k is the type of hardware parameter. For convenience, we always use hardware parameters to represent the hardware utilization intensity parameters. Tk v is the threshold for each hardware parameter of the, usually Tk v = 7%. Simultaneously, the hardware parameters of each PM are expressed as h p (t) at time t. R p and T p k represent the capacity and threshold of the hardware parameters of each PM, respectively. The sum of the hardware parameters of the in each PM is the amount of PM s hardware parameters, namely h p (t) = mi j=1 (hv i,j,k (t) Rv i,j,k )/Rp, where i = 1, 2, n. However, hardware parameters of the PM can be directly obtained through the XAPI interface, instead of using this equation, and. At the same time, we define u v i,j,k (t) to represent the user experience parameters of the v i,j at time t, including the response time u v i,j,1 = v i,j (t).t and throughput rate u v i,j,2 = v i,j(t).r of the application system. That is v i,j (t).t = Task.getFinishTime(t) - Task.getSubmissionTime(t) and v i,j (t).r = Task.getFinishingNumber(t) - Task.getRequestNumber(t). Tk u indicates the threshold of the user experience parameters. With regard to the threshold of system response time setting, as shown in Table 1, the 2/5/1s principle is widely accepted [17]. The threshold of the throughput rate is typically set to greater than 85%. When the throughput rate is TABLE I RESPONSE TIME AND CORRESPONDING USER EXPERIENCE. Response time Within 2s 2s-5s 5s-1s More than 1s User experience Speedy Accepted Anxiety, some user try to re-click Believe system had crashed

4 low, the system will try to request the task again, which will increase the system response time. C. Problem Formulation: Performance Optimization When the hardware parameters or the user experience parameters exceed their thresholds at certain, we need to migrate applications from that to eliminate the hotspot. However, migration can also cause performance degradation, and frequent migration may cause the crash of system and the awful of user experience. So the objective of our performance optimization is to reduce the number of migrations. We use O(t) to represent overloaded s at time t, and assume that under migration strategy M, there are M(O(t)) migrations. Then the performance optimization problem based on the user experience can be described as follows. T min M(O(t)) t=1 s.t. h v i,j,k(t ) T v k, v i,j V, t h p (t ) T p k, p i P, t u v i,j,k(t ) T u k, v i,j V, t. Here to represent the number of the migrations, T is the total time period, and t is the time after migration M(O(t)) but before t + 1. The first two constraints represent that the hardware parameters thresholds are satisfied at each and PM, respectively. The third constraint is to satisfy the threshold requirements of the user experience parameters. The objective function is to minimize the number of migrations. A simplified version of this problem can be related to the problem of multiple knapsack, which is NP-complete [18]. While several works suggest that the approximate solution can be obtained on the simplified problem only with constraints on hardware parameters, we are considering migrations based on user experience. IV. PERFORMANCE OPTIMIZATION STRATEGY In this section, we propose a new heuristics, POUX, which is based on the user experience to optimize the performance of the cloud platform. The description of POUX includes the definition of overloading, local optimization strategy and global optimization strategy. We will first provide each of them in detail and then present the overall POUX strategy. A. Defining Overloading In POUX, whether the is overloaded depends on h v i,j,k (t), uv i,j,k (t), T k v and T k u. Since hv i,j,k (t) and uv i,j,k (t) are often fluctuate, we use the single moving average (SMA) to calculate arithmetic moving averages of them. For example, h v i,j,k(σ) = h v i,j,k(t) = x=t x=t η+1 hv i,j,k (x). (1) η Hereafter, we use h v i,j,k (σ) to represent the arithmetic moving average of h v i,j,k (t) at time t. η is the window size for the moving average. Similarly, h p i,j,k (t) and uv i,j,k (t) can be converted to h p i,j,k (σ) and uv i,j,k (σ), respectively. The purpose of our performance optimization is to enhance the user experience, therefore, we also take user experience parameters u v i,j,k (σ) into consideration. We define the overloading based on the following two scenarios. Scenario 1: When there are user experience parameters that exceed their threshold, i.e. n k=1 (uv i,j,k (σ) > T k u) = 1 (regardless of any value of h v i,j,k (σ)), this is overloaded. Scenario 2: When there are hardware parameters that exceed their threshold (i.e., n k=1 (hv i,j,k (σ) > T k v ) = 1) but no user experience parameters that exceed their threshold (i.e., all n k=1 (uv i,j,k (σ) > T k u ) = ), this is also overloaded. Traditionally, in order to ensure a positive user experience, Tk v is set to a relatively low value, such as 7%. However, such low threshold significantly increases the number of migrations. Since our method has a separate constraint on user experience in Scenario 1, we can set Tk v to a higher value, such as 85%. Doing so not only can guarantee a good user experience, but also reduce the number of migrations. Consequently, the overloading of a is defined as follows. overloaded = { 1 h v i,j,k (σ) > T v k or uv i,j,k (σ) > T u k otherwise. B. Selecting Optimization Strategy When s are overloaded, performance optimization (i.e., migration) is required. We define two optimization s- trategies: local optimization and global optimization. When an overloading occurs, it is necessary to select one of these optimization strategies based on the decision condition. Whether the remaining resources of current PM can satisfy the overloaded s additional requirements for resources can be defined by the following equation. (R p T p k hp (σ)) (hv i,j,k(σ) R v i,j,k T v k ) > δ. (3) Here, δ is a tolerance value. If δ =, it indicates that the remaining resources of PM just satisfy with the additional resources needed by overloaded, but this PM may be unstable after migration due to fully usage of resources at PM. Thus, the stability of the system can be ensured by adjusting the value of δ. The selection of optimization strategies under the two overloaded scenarios is described as follows. In Scenario 1 ( n k=1 (uv i,j,k (σ) > T k u ) = 1), we analyze each of the hardware parameters h v i,j,k of this overloaded and choose the type of the most intensive utilization parameter among the k parameters. Here, only one hardware parameter is picked. If this parameter of the PM where this overloaded is located satisfies Eq. (3), local optimization is selected. Otherwise, global optimization is performed. In Scenario 2 ( n k=1 (hv i,j,k (σ) > T k v ) = 1), we analyze the hardware parameters that exceed the threshold. Here, multiple hardware parameters may be picked. If the parameter(s) of the PM where this overloaded is located still satisfies Eq. (3), (2)

5 Start Get the information of hardware and user experience Y Local Optimization Select the types of intensive resource Overloaded Y Selsct Strategy (Eq.3) Perform resource allocation or migration N N Global Optimization Select migrating and destination PM CPU Mem I/O (a) CPU Mem I/O (b) CPU Mem (c) Fig. 3. Resource allocation: (a) The overloaded, where the red part represents the capacity of overloaded resources; (b) Resource allocation 1 [(a) to (b)] where multiple resources are considered at the same time. When allocating the capacity for overloaded resources, it also expands unnecessary capacity for other underloaded resources. (c) Resource allocation 2 [(a) to (c)] which only consider the specific overload resources. Only the overload resource is increased and the other resources remain unchanged. When eliminating overloaded s, the latter method migrates less resources. I/O End Fig. 2. Execution process of POUX. local optimization is selected. Otherwise global optimization is performed. In summary, as shown in Figure 2, when the is overloaded the decision of which optimization strategy is selected is based on the following rule. { local, if overloaded & Eq.(3) holds strategy = global, C. Local Optimization R v,extend i,j,k = if overloaded & Eq.(3) does not hold. (4) When the is overloaded and Eq. (3) is satisfied (i.e., PM s idle resources can satisfy the needs of overloaded resource), local optimization is selected to eliminate the overloaded s. In different overloaded scenarios, the method of optimizing resources is different. In Scenario 1, the different types of hardware parameters of the overloaded are sorted according to n k=1 (uv i,j,k (σ) > T k u ) = 1 (i.e., ordering the hardware parameters which may cause user experience overloading), then the resource type k of the most intensive hardware parameters is selected as overloaded resource. In Scenario 2 where n k=1 (hv i,j,k (σ) > T k v ) = 1, the type of overload resource k is determined. Note that it could be multiple k overloaded. After determining the type of overloaded resource, the amount of resources R v,extend that i,j,k needs to be expanded is calculated. R v ( 1 1 i,j,k T u k u v i,j,k (σ)), Scenario 1 R v ( 1 1 i,j,k T v k h v i,j,k (σ) ), Scenario 2. (5) The Management PM commands M dynamically adds the k -type resources with the amount of R v,extend to the i,j,k overloaded. In this way, the overloaded is eliminated locally. As shown in Figure 3, local optimization uses resource redistribution for specific overloaded resources, so the number of dynamically added resources is less compared with when the multi-resource parameters are considered at the same time (a) (b) (c) Fig. 4. migration: (a) The overloaded 1 and other two s on the same PM; (b) Migration strategy 1 [(a) to (b)] shows that the overload 1 is migrated to eliminate overloaded 1. The amount of resources that are migrated is Rvm1 v hv vm1 (σ); (c) Migration strategy 2 [(a) to (c)] shows that the underloaded 2 is migrated to eliminate overloaded 1. The amount of resources that are migrated is Rvm2 v hv vm2 (σ). Obviously, Rvm2 v hv vm2 (σ) < Rv vm1 hv vm1 (σ). Therefore, the migration strategy 2 is more efficient, and has lower migration costs. In addition, local optimization eliminates overloaded s without migrations among PMs, which saves migration costs. D. Global Optimization When the local resources cannot satisfy Eq. (3), the global optimization strategy is executed. Global optimization is more complex than local optimization, mainly including selecting to migrate and selecting destination PM. As shown in Figure 4, on the PM where the is overloaded, it should choose with the minimum amount of resources to migrate, which can effectively reduce migration costs. The amount of resource R mig that needs to be migrated is defined as follows. Ri,j,k v ( 1 1 T u k u v i,j,k (σ) ) Rp 1 ( h p (σ) 1 ) + δ, T p k R mig = Scenario 1 R v i,j,k ( 1 T v k 1 h v i,j,k (σ) ) Rp ( 1 h p (σ) 1 T p k ) + δ, Scenario 2. (6) Similarly to local optimization, we first find the overloaded resource s type k, and then obtain s v satisfied i,j whose k source satisfies R v > i,j,k Rmig. These s belong to

6 Number of migrations POUX Non POUX Sandpier Time(hr) Fig. 5. Number of migrations in CloudSim. the same PM as the overloaded. v satisfied i,j is sorted in ascending order by R v to obtain i,j,k vsort i,j, which include all candidates for migration s. We search for the other PMs, in turn, and find PMs p satisfied i, which meet the k resources needs (i.e, (R p T p k hp (σ)) δ > Rmig, i). Next, i p satisfied i is sorted in ascending order according to R p to obtain p sort i as the candidates for destination PMs. Meanwhile, the remaining resources R p,idle = (R p T p k hp (σ)) δ, k k of each PM are recorded. Then we try to match a in vi,j sort to a PM in p sort i in order (from the beginning to the end). If R p,idle R mig, k is satisfied (i.e. a match k is found), then the to migrate v mig and the destination PM p des are obtained. After determining to be migrated and destination PM, Management PM commands M to perform migration through XAPI. If no match is found at the end, we have to add a new PM to meet overloaded s requirements for the resources. E. Overall POUX Strategy The detail algorithm of POUX is described as Algorithm 1. POUX is executed in the processing module of Management PM. As shown in Figure 2, after POUX obtains R v,extend i,j,k, v mig and p des, the execute module of Management PM commands M module to executes local optimization or global optimization through the XAPI. V. PERFORMANCE EVALUATION In this section, we use both the CloudSim simulator and a small-scale real-world testbed deployed by us to evaluate the performance of POUX. The implementation of POUX is based on XenServer. A. Simulation Evaluation In CloudSim, each PM has 1GHz 2-core CPU, 2GB memory, and 1GB/s network bandwidth. Each has 5Hz CPU, 512MB memory, and 1Mbit/s bandwidth. Initially, we simulate 4 s over 1 PMs, and four s are allocated on each PM. Later, according to the load, the resources of s will be dynamically adjusted. We will test migration Algorithm 1: POUX Strategy Input: PM-related parameters R p, hp (t), T p k ; -related parameters R v i,j,k, hv i,j,k (t), T v k ; and user experience-related parameters u v i,j,k (t), T u k. Output: The migrating strategy of the overloaded : v mig p des 1 while overloaded = 1 in Eq. (2) do 2 if Eq. (3) is satisfied then 3 Get R v,extend in Eq. (5); i,j,k 4 Perform resource allocation; 5 else 6 if n k=1 (hv i,j,k (σ) > T k v) = 1 then 7 Get the type k of overloaded resource; 8 end 9 if n k=1 (uv i,j,k (σ) > T k u) = 1 then 1 Choose the resource type k with the highest utilization rate; 11 end 12 Compute R mig in Eq. (6); 13 Get v satisfied i,j that satisfies R v i,j,k > Rmig 14 Sort v satisfied i,j R v, and obtain vsort i,j,k i,j ; 15 Get PM p satisfied i ; in ascending order according to that satisfies (R p T p k hp (σ)) δ > Rmig 16 Sort p satisfied i R p 17 Match v sort i,j, and obtain psort v mig p des ; 18 end 19 end, i; i in ascending order according to i ; in sequence with p sort i, obtain methods of POUX, Non-POUX and Sandpier [14]. Here, Non- POUX is a simplified version of POUX without considering user experience, and Sandpier is a strategy considering multiple parameters jointly. To facilitate the test, we only verify the CPU-intensive resources, by setting the utilization intensity of Memory and I/O below the threshold. The CPU resource consumption of each in the test instance uses the realtime load data in [16]. The CPU resource s threshold is set to 7% of non-user-experience-based strategy (Non-POUX) and 9% in POUX strategy. The threshold of the response time and the throughput rate (the ratio of the amount of requests and response times) are set to 3s and 8%, respectively. The evaluation parameter acquisition interval is set to 5 minutes, and the window size of the single moving average is set to 3. The M of Xen dynamic reclaims resources for less than 3%. The experiment runs the set of CPU resource consumption instances repeatedly within 12 hours. Number of Migrations: Figure 5 compares the number of migrations for POUX, Non-POUX and Sandpier [14] at the time 4h, 8h, and 12h. We can see that the number of migrations of Sandpier is more than those of POUX and Non-

7 25 2 POUX Non POUX Number of migrations 15 1 Response Time(s) Time(min) CPU Utilization(%) Fig. 6. Number of migrations in our testbed. Fig. 7. Response time of applications. POUX, since Sandpier considers multiple parameters at the same time. Non-POUX only considers the hardware parameters, resulting in unnecessary migrations, so the number of migrations is more than that of POUX. POUX considers both hardware parameters and user experience parameters at the same time, and aims to balance for specific overloaded resources, so fewer s are migrated. Ratio Hardware Overload User Experience Overload B. Testbed Evaluation To evaluate the management architecture and proposed POUX strategy on a real testbed, we built a small cluster, which includes six identical Dell PMs. XenServer 7. is deployed on each PM. All s run on Ubuntu server 16.1, which is based on the Linux release series 4.8. Each deploys jre-8u121, Apache Tomcat and Web Services, which implement the UAPI interface. In order to achieve the rapid migration s, a server acts as NFS. A notebook with Windows 1 OS acts as Management PM. We have developed a CPU-intensive web application, which is deployed in each. Meanwhile, a web client generates continuous requests of web access for each based on Httperf [19], [2]. Number of Migrations: Figure 6 shows the accumulated number of migrations over time. As POUX takes the user experience as the leading role in the migration and the hardware parameter s threshold is set to a higher value, the number of migrations of Non-POUX is much more than that of POUX as time increases. Response Time: On a single, we make the utilizations of CPU are 2%, 25%, up to 9% by increasing the amount of requests. As shown in Figure 7, the relationship between the CPU utilization and the application s response time is obtained, through calculating the average value of the application response time. The response time is stable in its early stage, but with the increase in CPU utilization, the response time grows rapidly. From 2% to 75%, the response time only increases by about 5ms, while up to 2s form 75% to 9%. Under the condition of neglecting Throughput Rate as the migration condition, Figure 8 compares the ratio of the number of migrations caused by overloading of user Response Time(s) Fig. 8. Ratio of the number of migrations caused by overloading of user experience and hardware. experience and hardware under the response times of 1s, 3s and 6s, respectively. Throughput Rate: On a single, we also increase the amount of CPU utilization from 2% to 9% by increasing the amount of requests. Figure 9 shows that the relationship between CPU utilization and throughput rate. clearly with more utilization the throughput rate goes down. When the CPU utilization is 7%, 8% and 9%, throughput rate is 83%, 7% and 3%, respectively. Throughput Rate CPU Utilization(%) Fig. 9. Throughput rate of applications.

8 VI. CONCLUSION Virtualization technology plays a key role in improving the performance of the cloud platform and saving energy through migrations. User experience is often overlooked when s are migrated. We propose a performance optimization strategy, POUX, for the cloud platform based on user experience. To implement this strategy, we design s management architecture of the cloud platform, obtain the information of hardware parameters through the official XAPI, and define the UAPI data interface for obtaining the information of user experience parameters. The user experience parameters refer to the response time and throughput rate of the application systems. Since user experience parameters are more direct description of the user s cloud platform requirements, POUX not only considers the hardware parameters, but also includes and focuses on the user experience parameters. In the POUX strategy, when the user experience parameters exceed the threshold, we analyze the hardware parameters, and find the resource constraints of the hardware parameters as conditions of migrations. Performance optimization is then performed for specific overloaded hardware parameters. Therefore, POUX can quickly eliminate overloaded s, and reduce the number of migrations. Evaluations via both the CloudSim simulator and our deployed small-scale real-world testbed show that our management architecture of the cloud platform and performance optimization strategy POUX are efficient and feasible. We plan to further study the relationship between the number of migrations and threshold setting of user experience parameters and hardware parameters to achieve optimal tradeoffs between overheads and effectiveness. We will also perform experiments with the simultaneous presence of multiple tasks with different loads, and further improve the performance optimization strategy. ACKNOWLEDGMENT Zhijin Qiu is supported by the fellowship from the China Scholarship Council (CSC) under No This work is also partially supported by the National Natural Science Foundation of China under Nos , and [6] A. Muthitacharoen, B. Chen, and D. Mazieres, A low-bandwidth network file system, in ACM SIGOPS Operating Systems Review, vol. 35, no. 5. ACM, 21, pp [7] P. Sahni and A. Batra, Network file system, International Journal of Research, vol. 2, no. 4, pp , 215. [8] T. S. Kang, M. Tsugawa, A. Matsunaga, T. Hirofuchi, and J. A. Fortes, Design and implementation of middleware for cloud disaster recovery via virtual machine migration management, in Proceedings of the 214 IEEE/ACM 7th International Conference on Utility and Cloud Computing. IEEE Computer Society, 214, pp [9] C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield, Live migration of virtual machines, in Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation-Volume 2. USENIX Association, 25, pp [1] G. Sun, D. Liao, V. Anand, D. Zhao, and H. Yu, A new technique for efficient live migration of multiple virtual machines, Future Generation Computer Systems, vol. 55, pp , 216. [11] A. Strunk, Costs of virtual machine live migration: A survey, in Proc. of 212 IEEE Eighth World Congress on Services (SERVICES),. IEEE, 212, pp [12] N. J. Kansal and I. Chana, Energy-aware virtual machine migration for cloud computing-a firefly optimization approach, Journal of Grid Computing, vol. 14, no. 2, pp , 216. [13] D. Sun, J. Zhang, W. Fan, T. Wang, C. Liu, and W. Huang, SPLM: security protection of live virtual machine migration in cloud computing, in Proceedings of the 4th ACM International Workshop on Security in Cloud Computing. ACM, 216, pp [14] T. Wood, P. Shenoy, A. Venkataramani, and M. Yousif, Sandpiper: Black-box and gray-box resource management for virtual machines, Computer Networks, vol. 53, no. 17, pp , 29. [15] C. Weng, M. Li, Z. Wang, and X. Lu, Automatic performance tuning for the virtualized cluster system, in Proc. of 29th IEEE International Conference on Distributed Computing Systems, ICDCS 9. IEEE, 29, pp [16] L. Chen, H. Shen, and K. Sapra, RIAL: Resource intensity aware load balancing in clouds, in Proceedings of IEEE INFOCOM, 214. IEEE, 214, pp [17] J. A. Hoxmeier and C. DiCesare, System response time and user satisfaction: An experimental study of browser-based applications, Proceedings of AMCIS 2, p. 347, 2. [18] M. J. Magazine and M.-S. Chern, A note on approximation schemes for multidimensional knapsack problems, Mathematics of Operations Research, vol. 9, no. 2, pp , [19] D. Mosberger and T. Jin, httperf: a tool for measuring web server performance, ACM SIGMETRICS Performance Evaluation Review, vol. 26, no. 3, pp , [2] S. Rajasekaran, Z. Ni, H. S. Chawla, N. Shah, T. Wood, and E. Berger, Scalable cloud security via asynchronous virtual machine introspection, in Proc. of 8th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 16), vol. 29. USENIX Association, 216, pp REFERENCES [1] P. Padala, X. Zhu, Z. Wang, S. Singhal, K. G. Shin et al., Performance evaluation of virtualization technologies for server consolidation, HP Labs Tec. Report, 27. [2] A. J. Younge, R. Henschel, J. T. Brown, G. Von Laszewski, J. Qiu, and G. C. Fox, Analysis of virtualization technologies for high performance computing environments, in Proc. of 211 IEEE International Conference on Cloud Computing (CLOUD). IEEE, 211, pp [3] N. Regola and J.-C. Ducom, Recommendations for virtualization technologies in high performance computing, in Cloud Computing Technology and Science (CloudCom), 21 IEEE Second International Conference on. IEEE, 21, pp [4] J. Satran, K. Meth et al., Internet small computer systems interface (iscsi), IETF RFC372, 24. [5] S. Aiken, D. Grunwald, A. R. Pleszkun, and J. Willeke, A performance analysis of the iscsi protocol, in Proceedings of 2th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, (MSST 23). IEEE, 23, pp