APPENDIX C ADDITIONAL THEORETICAL RESULTS

Transcription

1 PPENDIX MORE DISCUSSION ON THE TOMIC VM SIZING SSUMPTION Readers may be concerned that different jobs will have varying resource requirements that cannot be accommodated by the simple atomic size assumption. For example, hadoop and data analytics jobs have stringent requirement on CPU and bisection bandwidth, while database applications require more I/O and memory. We notice that in reality, public clouds have some special VM configurations that do not follow the atomic sizing assumption for these jobs. For example mazon EC provides high-memory, high-cpu, high I/O, and cluster compute instances specialized for certain applications. It is highly likely that they are managed separately with a different infrastructure. Measurement results [] corroborate our arguments while public information is yet to be made available on the specifics of the mazon infrastructure. We thus take the liberty to adopt this assumption, and believe that this represents a close resemblance of reality while keeping the problem analytically tractable. PPENDIX B DDITIONL EXMPLES FOR SEC. ND B. n example where the classical D algorithm may fail for job-machine stable matching Fig. shows an example similar to Fig.. Say we first let jobs a, b, c propose until they cannot. The result is (c),b (b), and a is rejected by. t this point, only d can propose to, and holds the offer. The final result is (c, d),b (b). This is clearly type- blocked by (b, ), as prefers b to d and b prefers to B. p(j) j s(j) c(m) m p(m), B B, a b c d B c, a, b, d b, c Fig.. n example where a possible execution of the D algorithm produces a type- blocking pair (b, ). On the other hand, if we let d propose to first before a and b, and keep the rest of the execution sequence unchanged, the result becomes (c),b (b), which is weakly stable. This example demonstrates two problems when applying the classical D algorithm here. First, the execution sequence is no longer immaterial to the. Without the atomic sizing or any other assumption to reduce the dimensionality of resources, the problem is essentially a multiple knapsack problem which is known to be NP-hard []. Multi-resource allocation is an open problem for which there has been some progress made recently in understanding the problem []. outcome. Second, it may yield an unstable matching. This creates considerable difficulties since we cannot determine which proposing sequence yields a weakly stable matching for an arbitrary problem. B. n example where running Revised D multiple times may result in an unstable matching To give an example, let us take a look at the problem in Fig.. We now run Revised D over this example. The result will then be (d),b (e),c (a). Clearly there are two type- blocking pairs, (a, ) and (b, C). Say we fix this by letting a propose to and b propose to C. Then we have a new type- blocking pair (c, C) due to the removal of job a from C, where c prefers C to being unassigned, and C prefers c to b and by rejecting b it has enough capacity to admit c. This is a result of C wrongly accepting b when it actually has more capacity after a leaves. p(j) j s(j) c(m) m p(m), B, C B, B a b c d e B C d, c, b, a e, d, c, b, a a, c, b Fig.. n example where simply running the revised D algorithm multiple times will produce a new type- blocking pair (c, C). PPENDIX C DDITIONL THEORETICL RESULTS C. Proof of Lemma Proof: This is a direct result of the algorithm design, since in t + every proposing job proposes to machines that have previously rejected it. If any of these machines accepted it, µ t+ (j) j µ t (j). If none of these machines accepted it, it will for sure be able to propose to its previous machine µ t (j) since c pre t+ (µ t(j)) must be no smaller than s(j). µ t (j) will for sure accept j at t +, because it will only receive offers from jobs that it previously rejected, possibly also from other jobs that it previously accepted if they propose to other machines and are rejected in t +. j remains favorable to m, even when all of m s accepted jobs in t proposed to m in t + again. C. Proof of Theorem 7 To prove Theorem 7 we need the following lemmas.

2 Lemma C.: If a particular job j participates in stage t +, it must have participated in stage t. Proof: It is a direct result of our algorithm design. When a job stops participating in t +, it does not form any blocking pair with any of the machines no matter how the rest of jobs participated in t are matched. Lemma C.: If a set of jobs do not participate in a particular stage t in Multi-stage D, then they are assigned to their best possible machine in all weakly stable matchings after t. Proof: Let us call a machine possible for a given job if there is a weakly stable matching that assigns it there. We can prove this lemma by induction. It is clear that when we pick up all the jobs that can participate, it is equivalent to pick up those that definitely cannot participate. We can easily do so by first finding jobs that cannot participate unless some jobs that did not participate in the previous stage participate, marking them, and finding the rest that cannot participate unless some jobs that were marked in this stage or previous stages participate, until there is no such job. ssume that up to a certain point of Multi-stage D, the lemma holds, i.e. no marked job was permanently rejected by a possible machine (since it cannot propose any more). Now suppose we mark job j, which is (permanently) rejected by a possible machine m in a hypothetical matching µ. Since j is marked, m must have accepted a set of jobs J m that are preferable than j and marked before. Then, in µ, at least one of the jobs from J m must be sent to a less desirable machine, since all machines preferable than m is impossible for it by assumption. This clearly forms a type- blocking pair in µ, and contradicts with the assumption. Thus the proof. This shows that our algorithm only permanently rejects jobs from machines that are impossible to accept them in all weakly stable matchings, when the jobs cannot participate any further. The resulting assignment is therefore optimal. Lemma C.: If µ t = µ t in Multi-stage D, then the set of participating jobs at t are assigned their best possible machine in all weakly stable matchings. Proof: ssume that at a given point of the algorithm execution at t, every job matched to its previous machine µ t (j) is given its best possible machine. Suppose now, j is rejected by all machines better than µ t (m), while there is a hypothetical weakly stable matching µ that sends j to a better machine m. Thus j must have proposed to and been rejected by m. m rejected j because it accepted j,j,, each of which is preferable than j. If j i did not participate in t, by Lemma C. m is its best possible machine. If j i participated in t, by assumption m is impossible for it. Thus in µ, for m to have j, at least one of j i has to be sent to a less desirable machine, which causes a type- blocking pair (j i,m) in µ, which contradicts the assumption. Now we can prove Theorem 7. Proof of Theorem 7: There are two possible cases when Multi-stage D terminates. If the algorithm terminates when there is no type- blocking pair, i.e. no job can, or wishes to if it can, participate by proposing to a better machine. Then by Lemma C., all these jobs are assigned to their best possible machine. Therefore the resulting matching is job-optimal. If the algorithm terminates at stage t where µ t = µ t, then by Lemma C. all jobs that did not participate in t are assigned the best machine. By Lemma C. all jobs that participated in t are also sent to their best machine. Hence the matching is also the job-optimal weakly stable matching. C. Proof of Theorem 8 Proof: This can be easily proved by contradiction. ssume that the matching produced when the algorithm terminates with no job proposing is not strongly stable. Thus, we must be able to find a type- blocking pair, say (j, m), as implied by Theorem. m will participate in the next stage, and j will be willing to propose to m, and our algorithm will continue to run, rather than terminating. This contradicts with our assumption. We conjecture that when the algorithm terminates with type- blocking pairs, the problem does not admit a strongly stable solution. The proof is, however, not immediate and left for future work. PPENDIX D N ONLINELGORITHM D. lgorithm design In this section, we address the technical challenge of devising an online algorithm. We extend the static problem setting to a dynamic environment, where the set of jobs J arrive online instead of offline. We observe that our Revised D algorithm can be readily used in the online setting. The high-level idea is that, we fix the previous matching and improve upon it by applying Revised D only on the set of new jobs. The reason to fix the previous matching is to avoid the potential overhead of and downtime caused by VM migration that may occur if we allow the previous jobs to participate with the new jobs. Since most resource management policies for clients, as we shown in Sec.., depend on system state variables that change during the placement process, the preferences of new VMs need to be configured according to the most updated state variables. Let us now present the details of the online version of Revised D called as shown in Table. First of all, the execution never stops. fter a matching is found, it waits for new requests. When a new set of jobs J arrive, their preferences are configured according to the current system state variables. The server preferences. Presumably, the set of machines is fixed for a long period of time.

3 are configured with regards to the new jobs only. Then the Revised D algorithm runs with the new jobs J and all the servers M. Note that in step 7 c(m) is the current server capacity, since it is always updated during the running of Revised D. TBLE : Input: c(m), 8m M : Initialize all m Mto free : while true do : Wait until new jobs J arrive : Configure preferences of new jobs j J p(j) according to the current system state. : Configure preferences of all servers p(m) with regards to new jobs J. 7: Run Revised D with c(m),p(m), 8m M, s(j),p(j), 8j J. 8: Output the current matching 9: end while D. Discussions Readers may wonder if our Multi-stage D algorithm also can be revised to be an online algorithm. We choose not to use Multi-stage D because it is iterative and takes much longer to converge when the problem scale is large, which may not be acceptable in practice. This is experimentally demonstrated in Sec... Thus Multi-stage D is only to be used as an offline algorithm for small to medium scale static problems, while the simple Revised D and can be used in offline and online fashion, for even large-scale problems. We thus consider Revised D and Online D more suited for practical use of large-scale dynamic cloud systems, while Multi-stage D more of theoretical value in exploring the intricacy of the job-machine matching problem with size heterogeneity. PPENDIX E DDITIONL POLICY EXMPLES The versatility of the preference concept can be further shown in the following examples. Colocation/anti-colocation. In practice, some clients do or do not want their VMs to be placed with VMs of some other tenants. nchor supports this policy using the PI call colocate(tenants, i, g_c), where tenants is a list of tenants, i is a boolean variable to indicate whether the caller wishes to colocate or anticolocate itself with clients in tenants, and g_c is an optional argument indicating the common policy group the colocating tenants belong to. In case a client wishes to colocate with a set of clients B, it makes a call to colocate(b, true, g). Since colocation is a mutual agreement, client and clients in B must share a common policy group g which is assumed to be agreed upon beforehand. The provider then bundles client s VM with VMs of clients in B as virtual VMs, where each virtual VM consists of an individual VM of and an individual VM of each client in B. Such a virtual VM is then treated as a single VM to join the policy group g and participate in nchor s stable matching algorithms. In case a client wishes to anti-colocate with a set of clients B, it can indicate so with colocate(b, false). The provider partitions the entire set of servers into two non-overlapping sets X and Y. It creates for a new policy group g_ containing servers in X only, and for each client in B a new policy group g_b containing servers in Y only. Client and clients in B can then configure their own policy groups for their preferences. It is guaranteed that client and clients in B propose to nonoverlapping sets of servers and will not be colocated. Note that anti-colocation is not a mutual agreement, and thus clients involved can have distinct preferences and policy groups. Tiered service. It is a common practice to implement tiered services in an operational cloud, by associating each VM with a priority class. This can be done in nchor with a call to conf(g_o, priority). Incomplete preferences. It is possible that some VMs can only be placed on a subset of servers, due to hardware constraints for instance. Such placement limitations can be accommodated in nchor, as the list of preference does not need to contain the entire set of servers. The policy engine in nchor supports an additional PI call limit(g_c, servers) so that VM preferences in a client policy group g_c only include servers specified in servers. Combination of policies. Besides individual policies, nchor also supports combinations of policies, which is also common in practice. The PI call conf(g, factor, factor,...) is designed for this purpose, where different policies are input following a decreasing priority order. multi-pass sorting procedure is then performed, first based on the least important factor, then the second least, and so on, to produce preferences that adhere to this combination of policies. E. Discussion Most preference examples shown above for clients are based on state variables that change during the placement process. However, we emphasize that the preferences do not change during the placement process, for otherwise the concept of stability cannot be well defined with respect to the constantly changing preferences. Our algorithms thus produce a stable matching with respect to the preferences and system state given before the placement process. This results in sub-optimal client performance compared to other algorithms that keep track of state variables during the process. This is an inherent limitation of the stable matching framework when applied to cloud resource management, because many practical policies rely critically on system state variables. We believe that it is an important and difficult

4 problem, and would like to study it further as our future work. PPENDIX F DDITIONL EVLUTION RESULTS F. Implementation details The resource monitor is implemented as a Python application that maintains resource statistics of servers and VMs using SQLite, a lightweight database engine. The sqlite Python bundle is utilized to update records in the database. The resource monitor periodically listens for usage reports once per second from a daemon in each server, which we have implemented for the sole purpose of collecting measurements of CPU and memory usage, via the VirtualBox management PI (VBoxManage metrics). Our daemon utilizes the iptraf tool to detect the available bandwidth of each server, since VirtualBox does not provide suitable PIs for this purpose. The policy manager also utilizes SQLite to manage policy groups for both the operator and cloud clients, and updates its databases upon receiving an nchor PI call. For efficiency, it maintains all the preferences in memory. When a placement request arrives, it obtains necessary information from the resources monitor s database, and sends sorted preferences to the matching engine. The matching engine implements the Revised D and Multi-stage D in the offline case, as well as the in the online case in Python. We preprocess server preferences (with a complexity of O(n)) to obtain an inverse of the list indexed by VM ID. Each subsequent rank comparison can thus be performed in constant time. For maximum efficiency, we use numpy in Python to implement the data structure of preferences. F. Small-scale experimental evaluation of nchor s effectiveness This experiment uses both consolidation and load balancing policies for VM placement to demonstrate the effectiveness of nchor in realizing resource management policies. We assume that clients follow the operator s default policy in the experiment here, so there is no conflicting interest involved. Occupancy existing VMs new VMs Server Fig.. Consolidation. Occupancy 7 existing VMs new VMs Server Fig.. Load Balancing. The experiment is to allocate VMs to servers, the first four of which are configured with an occupancy of,,,, respectively. Fig. shows the result of using the consolidation policy, where VM preference is ranked in descending order of server occupancy, and server preference is ranked in descending order of VM size. We observe that all the VMs are packed into the first five servers, whose occupancy is thus maximized. On the other hand, the load balancing policy distributes VMs across the idle servers, resulting in a more balanced server load as shown in Fig.. Multi-stage D takes iterations to converge to the strongly stable matching of Fig. for the former case, and iterations for the latter. F. Evaluation results on We conduct large-scale trace-driven simulations to evaluate nchor in a dynamic environment where VM placement requests arrive dynamically. We again use the RICC workload trace as the input to our algorithm. The trace contains each task s arrival time, finish time, and amount of resources requested. In our simulation, we process each task scheduling request in the trace according to its arrival time as VM placement request(s) of various sizes, and use to determine a matching for them. When the task finishes, we remove its VMs from the corresponding servers, so capacity is freed to accommodate future requests. In our simulations, the servers are configured to use the consolidation policy. The VMs are again randomly chosen to join the default, the CPU-bound, and the memory-bound policy group that uses the consolidation policy, the CPU bound and the memory-bound resource hunting policy, respectively. The free CPU and memory of a server is simply calculated as the total CPU and memory minus the amount allocated to its active VMs. We emphasize that VM preferences in this case are configured upon arrival using the most current system state variables as discussed above. Large tasks that require more than one server is translated into multiple smaller requests. We simulate the exact birth-death history of the first tasks in the RICC trace, amounting to VM placement requests that cover a time period of 79 seconds, or roughly. days. We compare with the online version of the first fit algorithm that first sorts the VMs according to their size (recall servers use a consolidation policy), and then tries to place a VM to the best machine according to its preference that has sufficient capacity to take it upon its arrival. The number of servers is fixed at. VM Happiness (%) Online first fit 8 8 VM request arrival () Fig.. VM happiness in a dynamic setting. Server Happiness (%) 8 Online first fit 8 VM request arrival () Fig.. Server happiness in a dynamic setting.

5 The results are shown in Fig. and. We can see that in terms of VM happiness, only slightly outperforms Online first fit by about 7%. However in terms of server happiness, enjoys a significant improvement where servers obtain their top % of the VMs while Online first fit is only able to match servers with their top % VMs. The reason of the performance discrepancy is again due to the elegant propose-reject design of stable matching algorithms. s discussed in Sec.., the first fit algorithm will not match a VM to a server whose capacity is insufficient, while will if this VM is preferable than some of the server s VMs during its execution. This improves the happiness of both VMs and servers, since VMs can get better machines even when they are occupied, and servers can also get their favorable VMs. We also evaluate the complexity of. Fig. 7 shows the running time of, including processing preferences, for the entire course of simulation. Observe that it takes less than seconds in any case to find a matching for new VMs, making it responsive to dynamic situations. Fig. 8 further shows that the algorithm usually terminates in less than iterations. Time (s) 8 VM request arrival () Fig. 7. Running times of. Iterations 8 VM request arrival () Fig. 8. Number of iterations of. Result: The algorithm is responsive and efficient to handle dynamic VM placement requests. Its performance is significantly better than existing methods based on the first fit algorithm. F. Deployment in Real Cloud Environment Here we discuss some potential challenges related to the real-world deployment of nchor in a cloud environment. The virtualized cloud infrastructure does not need any change to deploy nchor. ll the major virtualization solutions, including VMware, Xen, KVM, etc., collect resource utilization statistics through the hypervisor and provide PI to access the information. nchor can directly utilize the hypervisor PIs for its resource monitor. The cloud s client PI needs to be augmented to include nchor s policy interface as shown in Table of the main file for clients and the operator to express their policy goals. Given that the cloud operator has total control over its own client PI and interface, this is not difficult to implement in reality. In terms of scalability, nchor s Revised D and Online D algorithms are demonstrated to be efficient in handling large-scale VM placement problems, while Multistage D may only be suitable for small-scale problems in Sec.. For large-scale problems, the possible limitation is from the network interconnect, which may become the bottleneck for transmitting the bulky OS image files for initializing VMs. This has started to gain attention in some recent work []. PPENDIX G RELTED WORK ON JOB SCHEDULING There have been tremendous efforts on job scheduling in a computer cluster or grid. [], [] represent several early works that focus on minimizing the expected completion time of all jobs by strategically choosing their execution sequence. [7], [8] discuss the online algorithm design problem for grid computing. The focus is on the completion times of jobs, which in our case are not known a prior to the scheduler of a cloud since users may use VMs for as long as they wish. For more complete coverage of the literature readers are directed to [9], []. [] present some recent efforts of developing scheduling algorithms taking into account the practical constraints of a grid, including the trade-off between cost and time in a grid resource market, the QoS, etc., while [] analyzes the performance of datacenters using queueing theory. The distinction between this line of work and ours is clear: in cloud computing with VMs, the major difficulty of job scheduling comes from the inability to unify the different factors that operators and clients may consider as a single utility function, and from the packing constraint represented by the size heterogeneity of jobs. Our use of stable matching theory resolves the problem by adopting the stability concept, and new algorithms are developed to specifically tackle the size heterogeneity of jobs. REFERENCES [] G. Wang and T. Ng, The impact of virtualization on network performance of mazon EC data center, in Proc. IEEE INFOCOM,. [] M. Korupolu,. Singh, and B. Bamba, Coupled placement in modern data centers, in Proc. IPTPS, 9. [] C. Joe-Wang, S. Sen, T. Lan, and M. Chiang, Multi-resource allocation: Fairness-efficiency tradeoffs in a unifying framework, in Proc. IEEE INFOCOM,. [] C. Peng, M. Kim, Z. Zhang, and H. Lei, VDN: Virtual machine image distribution network for cloud data centers, in Proc. IEEE INFOCOM,. [] O. H. Ibarra and C. E. Kim, Heuristic algorithms for scheduling independent tasks on nonidentical processors, J. CM, vol., no., pp. 8 89, pril 977. [] S. K. Sahni, lgorithms for scheduling independent tasks, J. CM, vol., no., pp. 7, January 97. [7] W. Leinberger, G. Karypis, and V. Kumar, Job scheduling in the presence of multiple resource requirements, in Proc. CM/IEEE Conference on Supercomputing, 999. [8] S. Muthukrishnan, R. Rajaraman,. Shaheen, and J. Gehrke, Online scheduling to minimize average stretch, in Proc. IEEE FOCS, 999.

6 [9] V. K. Naik, M. S. Squillante, and S. K. Setia, Performance analysis of job scheduling policies in parallel supercomputing environments, in Proc. CM/IEEE Conference on Supercomputing, 99. [] T. D. Braun, H. J. Siegel, N. Beck, L. L. Bölöni, M. Maheswaran,. I. Reuther, J. P. Robertson, M. D. Theys, B. Yao, D. Hensgen, and R. F. Freund, comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems, Elsevier Journal of Parallel and Distributed Computing, vol., no., pp. 8 87, June. [] M.. Salehi, B. Javadi, and R. Buyya, QoS and preemption aware scheduling in federated and virtualized grid computing environments, Journal of Parallel and Distributed Computing (JPDC), vol. 7, no., pp.,. [] H. Khazaei, J. Mi sić, and V. Mi sić, Performance analysis of cloud computing centers using M/G/m/m + r queueing systems, IEEE Trans. Parallel Distrib. Syst., vol.,, to appear.