Design and Analysis of an Adjustable and Configurable Bio-inspired Heuristic Scheduling Technique for Cloud Based Systems.

Transcription

1 Design and Analysis of an Adjustable and Configurable Bio-inspired Heuristic Scheduling Technique for Cloud Based Systems by Ali Al Buhussain Thesis submitted to the Faculty of Graduate and Postdoctoral Studies In partial fulfillment of the requirements For the M.A.Sc. degree in Electrical and Computer Engineering School of Electrical Engineering and Computer Science Faculty of Engineering University of Ottawa c Ali Al Buhussain, Ottawa, Canada, 2016

2 Abstract Cloud computing environments mainly focus on the delivery of resources, platforms, and infrastructure as services to users over the Internet. More specifically, Cloud promises user access to a scalable amount of resources, making use of the elasticity on the provisioning of recourses by scaling them up and down depending on the demand. The cloud technology has gained popularity in recent years as the next big step in the IT industry. The number of users of Cloud services has been increasing steadily, so the need for efficient task scheduling is crucial for improving and maintaining performance. Moreover, those users have different SLAs that imposes different demands on the cloud system. In this particular case, a scheduler is responsible for assigning tasks to virtual machines in an effective and efficient matter to meet with the QoS promised to users. The scheduler needs to adapt to changes in the cloud environment along with defined demand requirements. Hence, an Adjustable and Configurable bio-inspired scheduling heuristic for cloud based systems (ACBH) is suggested. We also present an extensively comparative performance study on bio-inspired scheduling algorithms namely Ant Colony Optimization (ACO) and Honey Bee Optimization (HBO). Furthermore, a networking scheduling algorithm is also evaluated, which comprises Random Biased Sampling (RBS). The study of bio-inspired techniques concluded that all the bio-inspired algorithms follow the same flow that was later used in the development of (ACBH). The experimental results have shown that ACBH has a 90% better execution time that it closest rival which is ACO. ACBH has a better performance in terms of the fairness between execution time differences between tasks. HBO shows better scheduling when the objective consists mainly of costs. However, when there is multiple optimization objectives ACBH performs the best due to its configurability and adaptability. ii

3 Acknowledgements I am very thankful to Allah for blessing me with the chance to pursue an important step in my life. Also, I am very grateful to my supervisor professor Azzedine Boukerche for his guidance, his advice, and encouragement during my Master s journey. I would like to extend my gratitude to Dr. Robson Eduardo De Grande for his advice and his kind approach throughout my research. For the PARADISE lab members, thank you all for attending all my presentations and advising me about my work. My Father and mother back home, words cannot begin to express how thankful I am to you both for teaching the importance of hard work and for trusting and believing in my abilities. Last but not least, to my lovely wife Sara, thank you for your patience throughout this journey, thank you for your constant help and encouragement. Finally, I would like to thank the Saudi Cultural bureau for funding my research and making my dream a reality. iii

4 Glossary VM Virtual Machines Cloudlet Cloud Tasks QoS SaaS PaaS IaaS Quality of Service Software as a Service Platform as a Service Infrastructure as a Service PBSH Population Based Scheduling Heuristics ACO HBO RBS NID WIL Ant Colony Optimization Honey Bee Optimization Random Biased Sampling Node In Degree Walk In Length MOT Multi-Objective Theory ACBH Adjustable and Configurable Bio-inspired Scheduling Heuristic W c ACBH Cost Factor Weight W cp ACBH Computation Factor Weight W ld ACBH Load Factor Weight iv

5 ACBH ALL ACBH with All Factors considered ACBH CC ACBH with Cost and Computation Factors Considered ACBH CL ACBH with Cost and Load Factors Considered ACBH COMP ACBH with Only Computation Factor Considered ACBH LOAD ACBH with Only Load Factor Considered v

6 Contents 1 Introduction Thesis statement and motivation Thesis Objectives Contributions Thesis Outline Cloud Computing Cloud Types and Service Models Importance of Scheduling to Cloud Computing Related Work Introduction to Scheduling in Cloud Environments Scheduling Objectives Resource Scheduling in the cloud Population Based Scheduling Heuristics (PBSH) Overview Most Common PBSH Scheduling In Different Cloud Layers Scheduling in the SaaS Layer vi

7 3.3.2 Scheduling in the PaaS Layer Scheduling in the IaaS Layer Discussion Adjustable and Configurable Bio-inspired Heuristic Scheduling (ACBH) 27 5 Population Based Scheduling Heuristics (PBSH) Honey Bee Optimization (HBO) Ant Colony Optimization (ACO) Random Biased Sampling (RBS) Performance Evaluation Simulation Scenarios CloudSim CloudSim Architecture Base Test Performance Metrics Experimental Results Homogeneous Scenario Heterogeneous Scenario ACBH Variations Conclusion and Future Work Conclusion Future Work vii

8 List of Tables 4.1 ACBH Cost Factor Equations VM Characteristics in the Homogeneous Setup Cloudlet Characteristics in the Homogeneous Setup VM Characteristics in the Heterogeneous Setup Cloudlet Characteristics in the Heterogeneous Setup Datacenter Characteristics in the Heterogeneous Setup ACBH Factors viii

9 List of Figures 3.1 PBSH Cloud Service Models ACBH Architecture Honey Bee Structure Ant Colony Architecture Random Biased Sampling Match-Making CloudSim Architecture [37] Simulation Time of the Homogeneous Scenarios Scheduling Time in the Homogeneous Scenarios Execution Time of the Heterogeneous Scenarios Scheduling Time for the Heterogeneous Scenarios Degree of Time Imbalance in the Heterogeneous Scenarios Processing Costs for Heterogeneous Scenarios ACBH Variations Cost ACBH Variations Execution Time ACBH Variations Scheduling Time ACBH Variations Time Degree Imbalance ix

10 Chapter 1 Introduction This chapter introduces the purpose of my research and the contributions that this thesis are adding to the scheduling in the cloud computing field. A summery of the objective that my thesis are satisfying is provided. The outline of the thesis is shown at the end of this chapter. 1.1 Thesis statement and motivation Cloud computing most important aspect is the elasticity of resources provided to users. This Elasticity can only be possible if the cloud has a scheduling system that can adapt to the changes imposed on demand for the cloud resources by the user. Cloud Computing is seen by many nowadays as the future of the IT industry. Cloud environments as any computing environment suffer from drawbacks that held it from reaching its maximum potential. Therefore, those drawbacks must be first identified and then resolved in accordance with the certain requirements provided by either users, providers, or both. One of those drawbacks is scheduling the use of resources (physical or virtual). Managing the cloud resources (i.e. scheduling) plays a crucial role in helping the cloud to effectively 1

11 Introduction 2 and efficiently utilize those resources and hence, scheduling helps the cloud reach its maximum potential. 1.2 Thesis Objectives The objectives of this thesis can be summarized as follows: To contextualize the scheduling of cloud resources. This aids in finding or suggesting new or modified algorithms to solve the cloud scheduling problem. To perform an extensive analysis on bio-inspired algorithms adapted in the literature to be able to compare them against each other. The study will aid in spotting drawbacks of those algorithms and hence aids in enhancing or inventing new ways to approach the scheduling problem of cloud resources. To develop an enhanced bio-inspired scheduler that is able to overcome the rigidity of the traditional bio-inspired techniques used in the related work. 1.3 Contributions The main contribution of this thesis is the design and implementation of an adjustable and configurable bio-inspired Based Scheduling Algorithms (ACBH). ACBH is able to adapt to the QoS preferences of the user and schedule based on the current demands on the cloud virtual machines. Moreover, ACBH is elastic enough to work as a stand-alone scheduler or as a meta-heuristic. This makes ACBH a centralized scheduler or partially distributed if chosen to work as a meta-heuristic. Also, an extensive performance analysis of bio-inspired scheduling heuristics and suggesting future directions that can potentially be perused.

12 Introduction Thesis Outline This thesis is structured as follows. Section 2 introduces the cloud environment, it s characteristics, and various service modules. Section 3 provides a brief description of the related works. Section 4 explains the suggested ACBH in details in terms of architecture and functionality. Section 5.1 provides details of the architecture and functioning of the HoneyBee Optimization algorithm. Section 5.2 thoroughly describes AntColony algorithm. Section 5.3 presents Random Biased Sampling algorithm. Section 6.1 introduces the experimental scenario and discusses the results obtained. Finally, Section 7.1 concludes the paper and provides directions for the future work.

13 Chapter 2 Cloud Computing This chapter introduces the necessary characteristics of the cloud environment. Also, different service layers of the cloud are presented and discussed. These layers are relevant to the research topic as can be seen in Chapter 3. Furthermore, the importance of the scheduling in the cloud is presented. Cloud computing provides an environment where the needs for hardware and software is minimized through flexible, agile provisioning. The Cloud enables the access to its services through the network or an on-line simple interface. NIST defined the cloud as a model for providing a set of ubiquitous, convenient, on-demand network access to a shared pool of configurable resources over the Internet [77]. Those resources should be provisioned and managed by the provider of the cloud services. The cloud computing environment is seen as the next step in the IT industry. As a result, Cloud computing has been given a lot of attention and effort by many large IT corporations, such as Amazon EC2, Microsoft Azure, and Google Cloud Platform. According to [77] any cloud environment has the following essential characteristics: On-demand self-service: The Cloud user must have control and ability to fetch the 4

14 Cloud Computing 5 resources needed (e.g. server time, storage) without any interaction with the cloud service provider. Broad network access: Users of the cloud services should be able to access the capabilities of the cloud through the Internet using any thin or thick client (smartphones, tablets, laptops, and workstations). Resource Pooling: The cloud service provider must provide the requested physical and virtual resources by the cloud users according to their demand. Those resources must be available at all times regardless of the location of the cloud user. Rapid Elasticity: The resources (physical and virtual) must be Elastically provisioned either manually or automatically to match the current demand of the cloud user. This capability must be available at all times. Measured Service: Cloud service provider must have a monitoring tool that is able to monitor, control, and report resources usage. This monitor provides transparency for both the cloud user and cloud service provider. 2.1 Cloud Types and Service Models The cloud environment has three main types. The first type is the public cloud in which the access of its resources is open to everyone interested in subscribing to the services provided. For example, Amazon EC2, Microsoft Azure. Private clouds are similar to the public cloud but all of its capabilities are dedicated to a single organization. The hybrid as the name suggests it falls in between the public and the private cloud [82, 77, 55]. Cloud environment offers three main service models. The first service model offered is Software as a service (SaaS). The SaaS layer is the top most layer in which the cloud

15 Cloud Computing 6 providers use to provide software services to cloud users [82, 77, 55]. The users can access those services remotely. SaaS enables the users to access to software cheaper than buying the license and worrying about all the updates. Furthermore, remote hosting of those software means that the organizations using SaaS will also save on infrastructure (i.e. Hardware). The second service model is a platform as a service (PaaS). In PaaS as the name suggest provides cloud users with a platform to host their software. This model offers programming languages, libraries, and services and tools required to the cloud service consumers applications [82, 77, 55]. Finally, the infrastructure as a service (IaaS) is the last model offered in the cloud. In IaaS, the cloud provides complete infrastructure to deploy and run users applications. The cloud providers offer a range of resources such as datacenters, virtual machines, network, and storage [82, 77, 55]. Furthermore, the management of those resources in terms of scheduling, routing, and integration is offered by the cloud providers. All of the mentioned cloud service models as based on a pay-asyou-go model. In other words, users will pay for the resource they use. 2.2 Importance of Scheduling to Cloud Computing Cloud computing services aim to provide resources for high availability. Those resources are scalable depending on the need of users. Due to its features, flexibility, and scale, a Cloud is a highly complex distributed environment, and all of its characteristics imposes tremendous challenges on allowing centralized governance. Therefore, there is an increasing need to identify distributed solutions that are able to govern the Cloud environment through local knowledge. A distributed governance system is expected to be self-organized and able to manage itself [90]. In the Cloud, the demands for resources change dynamically, and a Cloud provider is expected to be able to accommodate and react to these changes on the demand to

16 Cloud Computing 7 meet performance requirements, which involves time constants for real-time applications, money, which corresponds mostly to re-sizing resources according to dynamic application demands, and SLA agreement. Virtualization of physical resources provides the necessary dynamics to manage the resources in the cloud platform [62]. Therefore, virtualization plays a major role in the effective use of physical resources in the Cloud to meet the SLA agreement. Consequently, scheduling the assigned tasks to the virtual machines directly impacts on the performance of the Cloud and aids in balancing the distribution of load among the different physical servers. Thus, finding an effective scheduling scheme is very crucial. Several scheduling algorithms have been proposed for allocating resources in distributed systems. However, the context of resource management systems for Cloud computing restricts the use of such algorithms to a narrower class of solutions. Bioinspired algorithms, more specifically swarm or gang scheduling algorithms, have shown very suitable for Cloud environments as per results presented in some works. However, the analysis performed in previous have restricted themselves to limited scenarios.

17 Chapter 3 Related Work This chapter presents an extensive survey of the existing cloud scheduling solutions that are implemented in each layer of the cloud environment. In this chapter, the existing solutions are categorized based on the layer of the cloud in which they operate. Moreover, a brief discussion of the the related work is added. A table summarizing all the techniques used in the literature is available as well. 3.1 Introduction to Scheduling in Cloud Environments According to [117]. [58] there are three scheduling levels on the cloud environments. At the application layer (SaaS), virtualization layer (PaaS), and deployment layer (IaaS). The application layer scheduling is needed to map user tasks to the virtual machines. In the virtualization layer, virtual machines are deployed in the datacenters to achieve certain objectives such as load balancing and power saving. In terms of the deployment layer, a scheduler is needed to schedule services and data routing. 8

18 Related Work 9 The cloud scheduling of task is an NP-Hard problem due to the size and the heterogeneity of the system. It is very important to find solutions to the scheduling problem in the cloud to meet the SLAs. In other words, for the cloud to be elastic, profitable, energy efficient, cost effective, available, and balanced in terms of the load; a good scheduler or group of schedulers must be used to achieve the cloud maximum potentials. To formalize the scheduling problem in the cloud as illustrated in 3.2, we want to assign task i to the best fitted VM j that is deployed to the best suited according to certain criteria PM k. The scheduling in the cloud is a multi-objective problem in which we are trying to find the optimal mapping of Tasks T = T1,T2,,Tn to VMs (V = V1,V2,,Vm). Then VMs are deployed in Datacenters D = D1,D2,,Dk to achieve (Maximize or Minimize) a goal set we have G = G1,G2,,Gs. In other words, we are allocating resources to achieve a desired level of cost, energy level, time constraint, scalability, and so on. The existing algorithms are mainly focusing on one section of the scheduling problem to find the best solution to that particular section. In other words, the developed schedulers are focusing on mainly solving the SaaS, PaaS, or IaaS scheduling problems separately. However, to have a more effective scheduler, we need to have interaction between the cloud layers to achieve better results Scheduling Objectives Cloud scheduling objectives will differ from one layer to the next in the cloud and also will depend on the SLA between the user and provider. This variation in demands on the cloud require the scheduler to be adaptable. In order for the scheduler to be adaptable, it has to be self-managed, self-organized and distributed [91]. However, there exist common objectives to cloud schedulers in which those schedulers are developed to satisfy a selected few depending on the set of goals agreed upon in the SLA [117][58].

19 Related Work 10 The scheduling objectives can be summarized as follow: Performance: Computing resources utilization measurements. Response Time: The amount of time to finish scheduling tasks. Throughput: A number of jobs finished per unit time. Makespan: The total execution time of the tasks. Reliability: trustworthiness of the algorithm. Bandwidth: The amount of bandwidth used to process tasks. Load balance: The amount of work that is assigned to the computing entity (Virtual Machine or Datacenter). Cost: The amount of money the will be consumed to execute the given tasks. Cost of Operation (Provider): How much is that task in costing the cloud provider to execute it. Price of use (user): How much does the cloud user will pay for that task to be executed. Eco: environment related measurements. Energy consumption: The amount of energy consumed to execute users tasks. CO2 Emission: The amount of CO2 emissions from datacenters while executing uses tasks. Others: Security.

20 Related Work Resource Scheduling in the cloud There have been some novice works to try and solve the scheduling problem in the cloud. Those initial work required the exhaustive search. Those works can be applied in a limited size environment, as they require a long time to make scheduling decision that will hinder the cloud capabilities in terms of availability and scalability. Others try to adopt the transitional (grid computing) scheduling schemes to the cloud but still those schedulers can achieve a single goal at a time. At the beginning, there was some straightforward approach where developed like [91] suggested in amazon EC2 a rule-based scheduler was used. As the cloud grew and more resources were added, also did the rule set. This rule set was huge and taken long time to evaluate that hinders the cloud performance. Also, the rules have intersections in which the execution of one rule might cause another to trigger resulting in a cascade of rules. Thus, this solution is time consuming, centralized, not elastic to be implemented to a huge and heterogeneous cloud environment.other approaches as seen in [43]. This work used developed an improved version of the Max-Min algorithm for task scheduling. The mechanism depended on not finding the maximum execution and assigned it to the least loaded computation node but also achieved lower make-span. In the work of [99] introduced a priority-based scheduling algorithm. The proposed scheduler evaluates a priority based on the cost of the execution of the task among other things. Then the tasks are divided into three groups of high, moderate, and low priority. The scheduler achieved task execution with lower cost. Furthermore, first come first serve (FIFO) is also was used to approach the scheduling problem and also used as a base test to compare purposed solutions as in [52]. Moreover, some other networking algorithms are also used to schedule the cloud tasks. Those heuristics were developed to challenge the scheduling problem in the cloud. Some

21 Related Work 12 of these heuristics are driven from networking problems (e.g. Random Biased Sampling, Active Clustering). Those algorithms found some success in solving the scheduling problem but the scalability and randomness make them unpredictable. Also, the number of scheduling objective achieved is limited [91]. Based on new constraints, scheduling in clouds is faced as an NP-Hard problem [54, 115, 117, 58], requiring self-regulations for balancing load on entities. This selforganization is possible with the implementation of distributed algorithms that rely on local knowledge. In recent years, nature has influenced many works on seeking out solutions to the increasing scale and complexity of the Cloud system [6]. There are two general directions that are typically followed to solve this issue. The most powerful heuristics used to solve NP-Hard problems are population-based algorithms, or evolutionary optimization, such as fuzzy and neural controllers. Due to its complexity, the scheduling in the cloud is best tackled by the previously mentioned heuristics [93, 100], including genetic algorithms (GA), Particle Swarm optimization (PSO), Ant Colony optimization (ACO), and Honey Bee optimization (HBO). There are major population heuristics approaches to schedule task in the cloud different layers [117, 58]. Genetic algorithms (GA) are a common approach that is being used recently as the survey illustrates later. 3.1 shows the general follow diagram of any GA. Moreover, Particle Swarm optimization (PSO) is also used to schedule task in the cloud. 3.1 explains the general step taken in PSO algorithms to make scheduling decisions. Ant Colony optimization (ACO) is heavily researched and is well defined. Hence, ACO is used as a scheduler in the cloud environments because it is easily extended to accommodate scheduling objectives as seen in the works bellow. Furthermore, Honey Bee optimization (HBO) is a viable scheduler that has been successfully used recently. Based on the importance of the population based schedulers on the cloud environ-

22 Related Work 13 ments, it is interesting to go over how these algorithms work, some objectives, and characteristics. In the next two sections, the algorithms will be explained and other schedulers used in the cloud will be mentioned. In addition, a table summing the objective, simulation used, test size, and other objectives of the schedulers. 3.2 Population Based Scheduling Heuristics (PBSH) Overview The PBSH follow the same step in their evolutionary process in order to make a decision. Those Algorithms have the following steps: I. Fitness evaluation. II. Candidate selection. III. Trial variation. During the Fitness evaluation, the algorithm finds the best sources that it can exploit for benefits are usually defined in terms of the optimization objectives of the scheduling algorithms such as, cost and deadline. After that step is done, we will have a set of candidates solution that we can choose from to achieve the scheduling objective(s). Those candidates have different things to offer and some will help reach the scheduling objectives faster or cheaper than others. Thus, we have to choose the best out of the candidates to maximize the chances of reaching the scheduling objectives. Finally, trial variation is choosing when the scheduling algorithm stops and thus giving the best scheduling solution possible. The GA, PSO, and ACO are non-deterministic in nature and require no prior guidance. Therefore, those schedulers represent a good for a complex and multi-objective environment like the cloud environment [117, 58]; however, in [117] there was no mention of one of the most studied and well-developed optimization technique that is HBO. The next section will have a brief introduction on the most common PBSH.

23 Related Work 14 GA ACO PSO HBO Start Start Start Start Initialize Population Initialize Population Initialize Population Initialize Population Fitness Evaluation For each ant (1<n<k) Fitness Evaluation Datacenters Evaluation by Foragers Population Selection Population Crossover Population Mutation Assign Task (1:n) for resources (1:m) Update Local Pheromone Update Global Phermonoe Update Velocity Update Position update local and global best Scouts Recruiting Update Datacenter Profit No No No No Termination? Termination? Termination? Termination? Yes Yes Yes Yes Finish Finish Finish Finish Figure 3.1: PBSH Most Common PBSH (a)genetic Algorithm (GA): In any genetic algorithms, any cloudlet (task) is considered to be a gene. Each task is coupled with a resource to form a chromosome. They construct a set of possible solutions to be chosen in the next steps of the GA. This step is done randomly to have an initial population of chromosomes. Then, GA starts the optimization process. An evaluation of the constructed chromosomes is done by evaluation their fitness. This evaluation depends on the objectives of the GA desired. Then, the chromosomes with the best fitness value are chosen to survive as they represent the best solutions thus far. After that, the set of best solutions is then crossed over and mutated to generate the new candidates for the next iteration. The GA evolutionary cycle will continue until a termination condition is reached (i.e. number of iterations set by the user). (b)particle Swarm Optimization (PSO): PSO works as GA when initializing the pop-

24 Related Work 15 ulation as it creates a particle. It contains a pair made with a task and a value. This value represents the virtual machine id the particle will run on. The optimization in PSO happens when evaluating the velocity and position of the population pairs. The velocity and position updates guide the particle to fly towards the target. PSO terminates when reaching a certain number of iterations or the solution cannot be enhanced [68]. (c)ant Colony Optimization (ACO): ACO initialization of the population means creating the desired number of ants then placing those ants randomly on the resources available. In this initial step, ants assign tasks to the resources they forage to randomly. Each ant will update the pheromone value by a constant value. Then each ant will choose the next resource for the task according to the pheromone and heuristic value. The local and global pheromone values are updated in every iteration. ACO stops when the maximum number of iterations that is specified by the user is reached or the ants completed their tours. (d)honey Bee Optimization (HBO): In HBO, the population is split into two types foragers and scouts. At the start, the foragers scour for resources that are most fitted to perform the task. Then the foragers will recruit scouts to use the most fitted source to execute the tasks. 3.3 Scheduling In Different Cloud Layers The scheduling in the SaaS layer as seen in 3.2 means assign the task to the virtual machines to satisfy the scheduling objectives. Moreover, in the PaaS layer, scheduling means finding the best datacenter to deploy a virtual machine on. The deployment is done in a way to achieve the scheduling goals. Finally in the SaaS layer, the cloud service provider must have datacenters deployed in many geographical locations. Those datacenters must be managed effectively in terms of energy, load balancing, and cost.

25 Related Work 16 Cloudlet1 Cloudlet2 Cloudlet3 Cloudlet4 Cloudlet5 Cloudlet6 Cloudlet7 Cloudlet8 Cloudletn-1 Cloudletn SaaS VM1 VM2 VM3 VM4 VM5 VM6 VM7 VM8 VM9 VMm-2 VMm-1 VMm PaaS IaaS IaaS Datacenter1 Datacenter2 Datatacenter3 Figure 3.2: Cloud Service Models Scheduling in the SaaS Layer According to [117, 58] the objectives of scheduling in this layer is mainly focused on performance (e.g. makespan ) and cost of running the application. Cloud service providers also aim to schedule the resources efficiently. In other words, the schedulers not only satisfies the QoS for the user but also take into account maximizing the providers objectives such as saving carbon cost or energy consumed by running virtual machines on datacenters [36, 67]. In this layer, the scheduler is responsible for the assignment of tasks to virtual machines according to the scheduling objectives specified by the user and provider.

26 Related Work 17 Scheduling in SaaS Using GA The use of GA was suggested by [120] in The works suggest that GA was used and being successful in approaching the NP-Hard problem and thus cloud be used in scheduling the cloud resources. The GA used was simple as its encoding for chromosomes was straightforward. The main objective of this algorithm was to meet deadlines. In [64] the encoding technique is similar to the work in [120] but the objectives considered are not only deadlines but also processing cost, throughput, and VMs utilization. The test performance was good; however, it was only compared to other simple schedulers such as round robin. [56] used similar encoding as in [120] combined with simulated annealing. The scheduler considered makespan, bandwidth, cost, distance, and reliability in making decisions. Some other works cluster several objectives such as the one in [76], which has been developed using a multi-objective scheduling algorithm. Reputation has also been used to introduce memory in the scheduling, such as work in [87]. Grouping based on memory consumption and computation requirements of tasks as in [120]. A GA scheduler that scans the entire job queue to make scheduling decision based was purposed in [53]. It aims to minimize the makespan of the tasks only. Thus, distributed, self-organized, and self-managed algorithms are needed to solve the cloud-scheduling problem. [69] combined the GA scheduler with Min-Min and Max-Min to enhance the initial population and hence a better scheduling solution can be found. Makespan was the focus of the previous algorithm. In [40] GA was scheduler goes through two optimization stages. At first, the deadline is used as the optimizer to find a feasible solution(s). After that, the set of feasible solutions is optimized further by using the cost. There are other GA algorithms that are designed to optimize different objectives. Load balancing, energy consumption, utilization, and SLA are among those objectives. These extend the previous objectives that focus on QoS. Using load balancing as the

27 Related Work 18 main optimization in GA as in [123], a multi-agent GA (MAGA) was purposed. The authors reduced the search space (number of tasks) by grouping tasks based on certain criteria. Also, a binary code is used to encode the chromosomes, not just an integer value. Scheduling in SaaS Using ACO The Ant Colony Optimization (ACO) uses the behavior of real ants in foraging for food to implement a solution for the cloud task scheduling. The ants leave the nest to search for food sources (VMs) in random. Then they evaluate the quality of the food source and carry it back to its nest. The ants leave a chemical trail on the ground. The strength of that chemical trail depends on the quality of the food source found [44]. Researchers used ACO to solve NP-hard problems such as traveling salesman problem, graph coloring problem, vehicle routing problem, and scheduling problem. In the context of Cloud computing, ACO is used to find the optimal way to schedule tasks to VMs [44, 75]. ACO differs from in the implementation of the transition rule. This rule has the heuristics information and pheromone update factors [44]. In [5] added to the transition rule another factor called execution matrix. It contains the expected execution time of task Ti on resource Rj. Both [75, 104] defined the ACO in terms of cloud entities. The heuristics were defined as the expected execution and transfer time for any given task. The pheromone update depends on the best current solution found in the previous iteration. Classification of tasks based on QoS parameters was used in the work of [124]. Tasks were grouped on the passes of bandwidth, completion time, system reliability, and cost. After that, ACO is used to schedule subgroups of the tasks on the cloud resources. Others modified ACO to detect load imbalances between nodes as in works of [83]. The ants will start at the nodes that have many neighboring nodes. As the ants travel

28 Related Work 19 they will detect the heavy loaded nodes and redistribute some of the load to the lightly loaded nodes. This process is two steps. The first step is to leave trail indicating how loaded is the node. The other step is to leave a trail to lightly loaded nodes. The work of [72] implemented ACO as in the [75, 104] but added a factor in the transition function to take into account the number of tasks assigned to a certain virtual machine to balance the load. In other words, this factor is added to ensure fairness when distributing tasks to virtual machines. ACO can also be merged with other algorithms such as PSO as in [111]. At first, ACO finds the best candidates (i.e. virtual machines) and then PSO performs crossover to avoid prematurity. Scheduling in SaaS Using PSO particle swarm optimization (PSO) is simple and fast as its only uses only two encodings. The first is the velocity and the second is the position. PSO is the fastest algorithm to converge when compared to GA and ACO [117]. The encoding in the work of [85] was similar to the GA in which that each task is associated with an integer holding the resource id. The PSO takes into account only the cost, both data transmission and computational, as optimization objective. This work is similar to [113] in terms of the optimization objective but they differ in form [85] as the PSO is discrete not continuous [117]. The optimization factors were then further extended in [39] to include the makespan, cost, reliability. The only problem with [39] is that there is no dependency between the optimization factors. The work done by [94] is encoding the scheduling by providing a rounded integer specifying the index of the resource assign to each task. The optimization objective was to reduce cost and meet deadlines. The index does not represent enough guidance or does not show the real features of the resources chosen [117]. Hence, [71] purposed to redefine what the index of resources should mean in PSO

29 Related Work 20 by using a renumbering based on the price of the resource and the ability to reorder. Also [70] extended the [71] by using a include many other scheduling objectives. Scheduling in SaaS Using HBO Honey bee optimization algorithm, like the PSO, is simple. It is also fast to converge since its only has one encoding scheme. It chooses the most profitable source to utilize. The optimization can be extended to hold many scheduling objectives such as cost, computation time, load balancing, and so on [97]. In the work of [91], the authors used HBO to balance the loads in the physical servers. Others like [80] used cost as an optimization factor to derive the profit of the HBO. Although HBO is not researched as much as GA, ACO, and PSO, It represents a valid optimization solution to the cloud scheduling as HBO can be extended to accommodate many optimization objectives and is also fast to converge. HBO has been found to have the ability to solve a set of multiobjective optimization problems [86, 106]. Compared with the optimization algorithms GA, PSO, and ACO, the performance of HBO has shown a great competitiveness [116]. Other HBO like [57] built a fault tolerant a load-aware algorithm. In this HBO, the authors consider fault and load awareness to be essential for the cloud performance. The table bellow summarizes the approaches used in the SaaS layers and shows that those approaches lack distribution and adaptability in comparison with ACBH.

30 Related Work 21 Type PBSH Objectives Distribution Configurable GA Deadline-GA [64] Deadline NO NO MGA [120] Deadline NO NO Processing Cost Throughput VMs utilization Makespan-GA [53] Makespan NO NO MinMax-GA [69] Makespan Partially Dis NO Rep-GA [87] Memory NO NO Makespan LocSearch-GA [56] Makespan Partially Dis NO Bandwidth Cost reliability Load-GA [123] Load Balance (CPU and Partially Dis NO Memory) ACO ACO [5] Makespan NO NO ACO [75] Makespan NO NO ACO [104] Makespan NO NO Fairness Task-ACO [124] Bandwidth Partially Dis NO Completion time Reliability Cost LoadBalance-ACO [83] Load balance (VMs) NO NO PSO-ACO [111] Maximize VMs utilization Partially Dis NO PSO MPSO[85] Cost NO NO Bandwidth Makespan MPSO [113] Cost NO NO Bandwidth Makespan MCR-PSO [39] Makespan NO NO Cost Reliability RU-PSO[94] Cost NO NO Deadline ReNum-PSO [70] Cost NO NO ReNum-PSO [71] Cost Partially Dis NO HBO Load-HBO[91] Load Balance NO NO Profit-HBO [80] Cost NO NO Fault-HBO [57] Fault NO NO Load Balance ACBH Flexible-ACBH Cost Bandwidth Makespan Fairness Memory Load Partially Dis/No Yes Scheduling in the PaaS Layer In this layer, the scheduler objective is to satisfy the or meet the requirement cloud provider, the user, or both. This layer is responsible for the virtualization management. In other words, assigning virtual machines to physical machines (i.e. datacenters) in a way to meet the specified requirements in the SLA between cloud user and provider

31 Related Work 22 [117, 58]. The works in this area have similar scheduling objectives used to optimized the PBSH as in the SaaS Layer; however, instead of assigning tasks to virtual machines, we are going one step deeper and assigning virtual machines to datacenters as illustrated in 3.2. The vitalization enables the utilization or use of the physical resources over the Internet in an effective matter over the Internet [114]. The objectives of scheduling in this layer are similar to the objectives in the SaaS layer. However, the focus of schedulers in this layer is more towards load-balancing, maximizing the utilization of physical resources, and energy saving [117, 58]. Others also considered cost and makespan. The algorithms used in IaaS layer are GA, ACO, PSO, and HBO which is also similar to the previous layer with a change of perspective. The next subsections summarize the works related to this layer and their optimization goals. Scheduling in the PaaS Layer Using GA MostoftheGAdevelopedtohavethesamecodingtechniqueasin3.1. Thechromosomes represent the number of virtual machines available (live). Then, each gene is an integer. This integer is the id of the datacenter that the virtual machine is scheduled to run on. This encoding can be seen in the work of [121]. The optimization objectives included CPU usage, memory needed to by the virtual machine, and the bandwidth. This work used NSGA-II to solve the multi-objective problem; however, the test size is very small (10 VMs, 6 datacenters). Another GA is more focused on reducing the migration of virtual machines between datacenters along with having balanced datacenters such as the GA purposed by [63]. The encoding is enhanced to achieve the optimization objectives as by using a tree structure that changes as the number of virtual machines assign to datacenters change (e.g. migration, destroying). This encoding helps balance the system s load and reduce migrations.

32 Related Work 23 Other GA algorithms are used to schedule the cloud tasks with the optimization objective focused on energy conservation. The purposed GA in [122] has a similar encoding representation as in 3.1. The GA schedule the virtual machines on the datacenters in such a way to maximize utilization. The authors represented the problem as an unbalanced assignment with the objective being maximizing the utilization of the datacenters. Hence, the GA purposed in [122] can use fewer datacenters to accomplish tasks and, therefore, saves energy. This GA only called initially and when the virtual machines number changes. However, in the work done by [61] work both offline (i.e. initially) and online to reconstruct the system state when the environment is not changing. The work done by [79] added predictor with the classical GA approach as the previously mentioned GAs. This predictor is added to make the GA run fast as the classical GA in known to be slow. The optimization objective is to maximize utilization in the datacenters to reduce energy consumption. The [79] used binary encoding. It represents the datacenter assign to that virtual machine along with the average request number of that datacenter is also represented in that encoding. Also, in the purposed hybrid GA as seen in [102]. The HGA optimization objectives are reducing the number of datacenters used and the amount of communication between virtual machines. Scheduling in the PaaS Layer Using ACO The ACO used to place virtual machines in datacenters has a similar flow as seen in 3.2. The ants will go on trips and assign the virtual machines to certain datacenters based on a specific optimization objectives. The ACO used by [73] is the same as the ACO is shown in 3.1.The ACO has only one optimization objective, Finding the nearest underloaded datacenter. This datacenter is used to deploy new virtual machines or to migrate virtual machines to from heavy loaded datacenters.

33 Related Work 24 Other ACO algorithm like the works of both [105, 51] in which the ants will place the virtual machines in their hosts based on multi-optimization objectives. Those objectives are to minimize both CPU and memory resources wastage. The ACO in [105, 51] maximizes the utilization of the cloud resources by efficiently utilizing available CPU and memory. The ACO of [49] represented the virtual machines placement problem on the least possible number of datacenters as the well known multi-dimensional bin packing (MDBP) problem. The optimization objective in the previous ACO of [49] was to reduce the number of datacenters used by placing as many virtual machines as each datacenter can handle to save energy. Some other ACO algorithms save energy but they consider different optimization objectives. For example, depended on the resources wastage and Scheduling in the IaaS Layer In this section, the PBSH are also used to solve problem arises in this layer. Most of the solution are based on GA and ACO algorithms. The importance of resources management in this layer is to manage the datacenters of the cloud providers in order to be able to deliver promised services to customers. Those datacenters can be located in one location, across country, or across continents to serve worldwide users. The issues encountered are placement of services, routing, and interoperability (Partner Federation). Service placement means deployment of the cloud service user tasks and their assigned virtual machines on the datacenters based on certain optimization objectives using GA or ACO. Routing means finding the desired datacenter to execute or run virtual machines based on provided optimization objectives of both cloud user and cloud provider.interoperability means collaborating between different cloud provider(i.e. sharing datacenters) to provide services to users.

34 Related Work Discussion The previous works done to find a solution to the scheduling problem in the cloud in any layer are nature driven. Those PBSH found success in solving specific issue only (i.e. the PBSH are tailored for specific scheduling optimizations). There exists a theory called multi-objective theory (MOT). MOT dose does not give a specific solution to the optimization problem but rather it finds the set of feasible solution to optimization objectives and constraints given (Perto set) [42].Some of the PBSH above uses MOT to reduce the search space as seen in [50, 65, 66, 107, 101, 60, 59, 78, 103]. However, the PBSH in the previous works is lacking simulations. Most of the testing on PBSH are done in math. Math is a good formalization of those algorithms but the simulation is necessary to really test PBSH under a cloud like setup. Moreover, simulations were done, however, few, are using a small test environment (i.e. few task, virtual machines, and datacenters). It is necessary to stress the PBSH under more realistic tests to see how they will perform in a real cloud environment. Furthermore, all the previous works are missing Elasticity. In the cloud environments, the requirements of cloud users in terms of budget, and time changes all the time. In addition, the cloud service provider emphasis on the cloud infrastructure changes. Both changes must be reflected in the cloud scheduler in all the cloud service models to accommodate those changes and get the most out of the cloud. However, using the previously mentioned scheduling (PBSH) is limited as they are only good for what they are specifically designed for. Moreover, the previous related work is not considering the interaction between cloud service models when it comes to scheduling. All the mentioned previous works consider only one cloud service model. This consideration is limiting cloud performance because the scheduling decisions are done in the SaaS layer, for example, has effects on the

35 Related Work 26 placement of the assigned virtual machine in datacenters (PaaS). Developing a scheduler that is able to schedule through cloud service models can boost cloud performance. Mobile cloud is an emerging computation platform that employs wireless networks technologies to bring cloud computing to mobile users such as cars and cellphones. The mobile cloud presents new challenges to meet its performance expectations[96]. Scheduling is one of the challenges of mobile cloud but there are other issues to be faced especially with networking due to the mobility of the nodes[96]. There are some works that has been done to accommodate to the new challenges of mobile cloud such as [110, 15, 41, 92] where the routing techniques are light, secure, and reliable. Other works such as [1, 11, 12, 24, 47, 23, 9, 20, 48, 119, 8]provided networking solution for mobile cloud which is an emerging computation platform that will use wireless networks to bring cloud computing to mobile users such as cars and cellphones. Some works are providing solutions to data transport and distribution across moving nodes (cars) as in [13, 118, 14, 31, 22], neighbor localization that are error aware will improve mobile cloud performance as seen in the works of [29, 33, 26, 25, 16, 30, 2, 84], and allocation of available nodes dynamically as demonstrated in [18]. The green computing is important in the cloud as it saves money and the environment and thus some research as been conducted to implement an energy aware cloud and mobile cloud as seen in [4, 21, 95, 10]. One main advantage of the mobile cloud is to bring live entertainment (i.e. streaming) to end user as in [19, 28, 27, 109] and safety message propagation between cars as in [17, 10, 108]. An important aspect of the cloud is to meet the service level agreements between the service users and providers. Therefore, not only the scheduling has to be QoS aware, the routing must be aware of the QoS too as seen in [8].

36 Chapter 4 Adjustable and Configurable Bio-inspired Heuristic Scheduling (ACBH) One of the most important aspects of the cloud environments is to be able to scale and adapt to changes in demands by the different key players providing or using the cloud resources. The work suggested here takes a step forward into providing an adjustable scheduler that is able to accommodate and adapt to the change of requirements (QoS) in the cloud. The ACBH suggested by this thesis is detailed in this chapter. In this chapter, the idea behind the ACBH factors that represents the heart of the algorithm and it s vetoing system are explained. The significance of the ACBH is also mentioned. Itisextremelyimportanttohaveaschedulerthatcanprovidethiskindofflexibilityin the cloud because of the clouds high elasticity which differentiates it from grid computing. The cloud must be able to rapidly increase the amount of resources that users need and those resources must be utilized efficiently. Moreover, the cloud based system is service driven (i.e. the SLA agreements must be met because customers are paying for their 27

37 Adjustable and Configurable Bio-inspired Scheduling Heuristic (ACBH) 28 level of service). In other words, every customer has a different service level agreement that imposes a variety of demands on the cloud scheduler. For example, SLA1 will finish the tasks in 1 minute and costs 20$ and SLA2 will execute the same set of tasks in 1.5 minutes and will cost 10$. The customer has the choice to choose which service level best meets his/her requirements. Thus, it is crucial that the scheduler be easily adjustable and configurable to be able to insure that different SLAs are met [35, 112, 3, 34]. The notion of configurability is somewhat new to the cloud scheduling based systems as seen in section 3; however, it has been introduced in grid computing as seen in [32, 7, 46]. The authors of those works have combined existing techniques to reach a new scheme that is able to enhance the rigidity grid data distribution manager or to yield a more dynamic and efficient load balancing algorithm. The authors were able to enhance the quality of their algorithms by adding adaptability and dynamicity in a grid based system which is not as elastic as the cloud based systems. Thus, to enhance the performance in the cloud based scheduling the suggested scheduler is adding adjustability and configurability to be able to provide a better service quality to cloud customers. As seen in the previous chapter in Fig 3.1, the PBSH schedulers are following a similar process in to make a scheduling decision as demonstrated in Fig 4.1.Following that same way of decision making, a scheduler that is bio-inspired is developed that is also able to be adjusted and configured. The adjustability and reconfigurability of the implemented scheduler helps in enhancing the rigidity of the PBSHs. The idea behind ACBH is to create a scheduler that is able to change preferences or importance depending on the change in the requirements of the cloud service subscriber and the expectations of the cloud service provider as seen in Figure 4.1. Similar to the PBSH heuristics, the ACBH has three main evolutionary steps to make the scheduling decision. The first step is called weight initialization in which

38 Adjustable and Configurable Bio-inspired Scheduling Heuristic (ACBH) 29 ACBH Start Select (Wc,Wcp,Wld) Evaluate VMs Cost Evaluate VMs Comutation Time Evaluate Vms Load Get Cost Vote Get Computaiton Vote Get Load Vote Select VMi With the Highest Vote No Termination? Yes Finish Figure 4.1: ACBH Architecture. the algorithm chooses the current appropriate preference in terms of the optimization objectives. For example, the current preference could be 50% computation, 70% load, and 10% cost. The idea those weight is to be able to tweak the performance focus of the scheduling algorithm by only adjusting those three controlling weights. The second steps is called the candidate selection in which the ACBH evaluate the existing virtual machines using three deciding factors, they are explained thoroughly in later. After the evaluations are done, the values from each factor converted into relative values (i.e.

39 Adjustable and Configurable Bio-inspired Scheduling Heuristic (ACBH) 30 votes) in the voting system. Then those votes are to be adjusted using the weight chosen in step one of ACBH which will translate the objective of the scheduling to the voting system. At the end the adjusted votes are added to each virtual machine and the one with the maximum vote will be chosen to execute the given task. The optimization objectives of ACBH are cost, computation. Moreover, the cloud service provider has a similar the computation and load. The meaning of those objectives are as follows: Cost: ThecostofusingVM i toexecutecloudlet j intermsofram,mips,storage, Bandwidth. Computation: The time VM i will take to execute Cloudlet j. Load: The number of Cloudlets assigned to VM i. The ACBH has works as a voting system in which each optimization objective is adjusted to the weight desired. Then the virtual machine with the most or the highest vote at time T i will be assigned the current cloudlet. This property makes EBSH standalone scheduler because of the ability to choose the best virtual machine for the current task. Moreover, EBSH can be used as a fixable meta-heuristic since it can be extended to arrange the virtual machines in order of highest votes and then use any PBSH to assign the highest voted virtual machines to a set of tasks. The Elasticity of ACBH is to be able to work either as a heuristic and meta-heuristic enable it to be both centralized and partially distributed. The steps followed by EBSH are:

40 Adjustable and Configurable Bio-inspired Scheduling Heuristic (ACBH) 31 Algorithm 1 ACBH VM Assignment Algorithm Require: Cloudlet list,vm list,datacenter list,w cost,w computation,w load 1: Size(m) getsize(cloudlet list ) 2: Size(n) getsize(vm list ) 3: for j = 1 to m do 4: for i = 1 to n do 5: CostVote getcostvote(vm i,vm i,cloudlet j ) // as in 4.1 6: AdjuCostVote adjustvote(w c,cost V ote) 7: ComputationVote getcomputationvote(vm i,cloudlet j ) // as in 4.5 8: AdjuComputation V ote adjustvote(w cp,computationvote) 9: LoadVote getloadvote(vm i ) // as in : AdjuLoadVote adjustvote(w ld,loadvote) 11: SumAdjustedV otesv Mi sumofv otes(adjucostv ote, AdjuComputationV ote, AdjuLoadV ote) 12: end for 13: V MT obeassigned getmaximumv otev M(SumAdjustedV otesv Mi) 14: Assign(Cloudlet j,vmtobeassigned) 15: end for 1. The estimation of the cost for executing Cloudlet j on VMi where i(1..n) is done. The cost is estimated based on the RAM, Storage, Bandwidth, and MIPS required by Cloudlet j from VMi. 2. The execution time of Cloudlet j on VMi where i (1..n) is estimated. This estimation is based on the worst case to ensure that we have the virtual machines with the least execution time. 3. The load on each virtual machine is calculated by the number of cloudlets assigned to it. this is also based on the worst case to ensure we get the least loaded virtual machine. 4. If stand-alone scheduler Convert all the optimization objectives from VM i i (0..1). Add all the votes for every virtual machine and then find the one with the maximum vote. Finally, assign that virtual machine to the cloudlet to execute. 5. if meta-heuristic Convert all the optimization objectives from VM i i (0..1).

41 Adjustable and Configurable Bio-inspired Scheduling Heuristic (ACBH) 32 Make a subset of virtual machines according to a threshold of the minimum number of votes. Then give the subset of virtual machines to any PBSH to assign tasks to that subset. Table 4.1: ACBH Cost Factor Equations Parameter Values T CLj CPSVM i DC CPS size VMi CPRVM i DC CPR RAM VMi CPBVM i DC CPB Bw VMi NOCVM i MIPSVM i The clength of the Cloudlet j The cost of storage used by Vm i The Cost of storage of Datacenter i The storage required by VM i The cost of RAM to execute Cloudlet j by VM i Cost of RAM for executing Cloudlet j by VM i The RAM required by VM i Cost of Bandwidth for executing Cloudlet j by VM i Datacnter i cost per bandwidth. The needed bandwidth consumed by VM i The number of cloudlets assigned to VM i The MIPS of VM i Cost i,j V ote = (CPSVM i +CPRVM i +CPBVM i ) (T CLj ),i = 1...N,j = 1...M (4.1) Where, CPSVM i = DC CPS size V Mi,i = 1...N (4.2) CPRVM i = DC CPR RAM V Mi,i = 1...N (4.3) CPBVM i = DC CPB Bw V Mi,i = 1...N (4.4) The cost estimation functions are similar to the cost estimation followed by HBO.

42 Adjustable and Configurable Bio-inspired Scheduling Heuristic (ACBH) 33 Computation i,j V ote = (T CL j NOCVM i ) MIPSVM i,i = 1...N,j = 1...M (4.5) Load i,j V ote = NOCVM i,i = 1...N,j = 1...M (4.6) All of the above equations are then used to get a vote for their optimization objective. This can be done by finding the minimum value calculated. Then this value will get a vote of 1. The others are divided by the minimum value given us the vote that is between (0 and 1). The importance of converting the optimization objectives to votes because the calculations will give different values. They are not close and adding them to gather affects the choice of virtual machines. Thus, converting them to votes (i.e. relative values) ensure fairness. Moreover, the voting system is necessary for the meta-heuristic mode.

43 Chapter 5 Population Based Scheduling Heuristics (PBSH) This chapter of the thesis presents the chosen PBSH in details. The descriptions, flow charts, and pseudocodes of the PBSH are shown. Also, the details of the Random Biased Sampling algorithm is displayed. 5.1 Honey Bee Optimization (HBO) The basic elements in HBO scheduling algorithm is a bee. Those bees find the most profitable source and exploit it. Foragers are also considered by shifting in quality or profit in the nectar sources [98]. HBO provides a self-managed and self-organized solution, by essence decentralized, to the scheduling problem [81]. Such characteristics fit it as a strong candidate for Cloud scheduling. HBO scheduling algorithms are basically divided into two parts, as delimited in Algorithm 2. The first part consists of the foraging behavior of the bees as they look for food sources. The second part is the scouting where the bees start their search for the best 34

44 Population Based Scheduling Heuristics (PBSH) 35 Algorithm 2 HBO VM Assignment Algorithm Require: Cloudlet list,vm list,datacenter list,fac LB 1: Groups(q) divide(cloudlet list ) 2: for i = 1 to q do 3: length i lenghtofgroup K (Groups i ) 4: end for 5: for k = 1 to q do 6: CloudLet L max(groups k 7: while Groups k {Groups i i = 1..q and i k} do 8: for s = 1 to n do 9: Datacenter s select(datacenter list ) // as in Eq : if fac LB VMsAssigned(Datacenter) then 11: assign(cloudlet L,Datacenter s(vm leastload )) 12: else 13: assign(cloudlet L,Datacenter i s (VM leastload )) 14: end if 15: decrement(length k ) 16: end for 17: end while 18: end for food source brought by the foragers. The HoneyBee procedure is described in essence according to the following elements: 1. The number of foraging VMs (n) is equal to the number of Datacenters (DCs) available. The choice of foraging VMs placement in DCs is random; 2. The DCs have its own characteristics (dch). Those characteristics (RAM, Storage, Bandwidth) help the foraging VMs to find the best food source for a cloudlet with a specific execution time; 3. The above-mentioned dch and the execution time are used to evaluate the fitness function. The outcome of the fitness value will then be the deciding factor by which we chose to run the cloudlet on which DC; 4. The DC with the highest fitness value, or the lowest cost as defined in Equation 5.1, receives a percentage of the tasks. The remaining subset of the tasks will be executed on the other DCs by repeating step 4 and excluding the chosen DCBest.

45 Population Based Scheduling Heuristics (PBSH) 36 Figure 5.1: Honey Bee Structure. DC i,j Cost = (Size i +M i +Bw i ) (T CLj ),i = 1...N,j = 1...M (5.1) Size i = dch CPS size V Mi,i = 1...N (5.2) M i = dch CPR RAM V Mi,i = 1...N (5.3) BW i = dch CPB Bw V Mi,i = 1...N (5.4) To simulate the honey bee for a Cloud environment, Figure 5.1 shows that Cloudlets aresplitintotwogroupsformingafood source. Then, thevmsaresplitintoforagersand scouts. Each forager is assigned to a cloudlet group where it is responsible for recruiting scouts VM to help execute the Tasks. Each scout chooses a task with certain probability

46 Population Based Scheduling Heuristics (PBSH) 37 as delimited in Equation 5.1. This equation is formed by a sum of the size, quantity of memory, and bandwidth, respectively represented by Equations 5.2, 5.3, and 5.4 where the following parameters delimit them: Parameter Values T CLj Size i dch CPS size VMi M i dch CPR RAM VMi BW i dch CPB Bw VMi The clength of the Cloudlet j The cost of storage used by Vm i The Cost of storage of Datacenter i The storage required by VM i The cost of RAM to execute Cloudlet j by VM i Cost of RAM for executing Cloudlet j by VM i The RAM required by VM i Cost of Bandwidth for executing Cloudlet j by VM i Datacnter i cost per bandwidth. The needed bandwidth consumed by VM i 5.2 Ant Colony Optimization (ACO) The Ant Colony Optimization (ACO) uses the behavior of real ants in foraging for food to implement a solution for the cloud task scheduling. The ants leave the nest to search for food sources (VMs) in random. Then they evaluate the quality of the food source and carry it back to its nest. The ants leave a chemical trail on the ground. The strength of that chemical trail depends on the quality of the food source found [45]. Researchers used ACO to solve NP-hard problems such as traveling salesman problem, graph coloring problem, vehicle routing problem, and scheduling problem. In the context of Cloud computing, ACO is used to find the optimal way to schedule tasks to VMs [45, 74]. In ACO when ants search for food first, ants search randomly for food sources and once one is found, an ant leaves a chemical trail called pheromone leading to that food source. Then, other ants are attracted to that specific food source by following the

47 Population Based Scheduling Heuristics (PBSH) 38 pheromone trail. This process continues until ants find the shortest path leading to a specific food source by accumulating huge amounts of pheromone on the shortest path leading to the food source [45]. The ACO can be applied to solve complex combination problems if the following elements are properly delimited for the algorithm: Problem statement: In this algorithm, ants find the optimal solution to the cloudscheduling problem by moving from town to town (VM to VM) to choose the best VM to the Cloudlets. Ants move from At the start of the simulations, ants are placed randomly at different VMs with initial pheromone trail τ(0) as in Equation 5.5. Heuristic desirability η: the inverse of the expected execution time is used. Constraint satisfaction method: each ant is only allowed to visit a VM once to minimize scheduling time Pheromone-updating rule: each ant deposit a pheromone the depends on the quality of the solution. The pheromone is updated according to Equations Probabilistic transition rule: the ant related to tabu k is updated by adding the visited VM in the initial step. Afterwards, Ant k selects the next VM j to execute Cloudlet i based on the probability defined in Equation 5.5. ( ) [τ i,j (t) α ] [η i,j ] β ρ k s i,j = Allowed k[τ i,s ](t) α [η i,s,jǫallowed ] β k 0, Otherwise (5.5) τ i,j (t) : is the pheromone concentration between task i and VM j. allowed k : keeps track of the VMs that Ant k can use at all times.

48 Population Based Scheduling Heuristics (PBSH) 39 η i,j : is the heuristic value calculated in Eq(3) where η i,j = 1 d i,j. ( TL Task j d i,j = + InFileSize ) Pe num j Pe mips j VM bw j (5.6) α and β : to choose the relative weight of between the pheromone concentration and the heuristic value. Equations 5.7, 5.8, 5.9, 5.10, and 5.11 are used to update the pheromone concentration. Initially each ant is placed on a VM randomly and pheromone value of τ(0) is given to all edges, as described in Algorithm 3. ( ) Q τi,j(t) k L k,i,jǫt (t) k (t) = 0, Otherwise (5.7) L k (t) = argmax jǫj sum jǫij (d ij ) (5.8) τ k ij(t) = (1 ρ)τ k ij(t)+ τ k ij(t) (5.9) τ k ij(t) = m τij k (5.10) k=1 τ k ij(t) = τ k ij(t)+ ( ) Q if(i,j)ǫt k (t) (5.11) L k (t) L k (t) is the length of the current best tour done by the ants in the current iteration as in Eq(8). The local Pheromone value is updated by Eq(9) where ρ is the decay factor of the pheromone deposited before. Finally, the global pheromone value is updated by Eq(11).Multiple values were tested and the best were chosen and the are as follow:

49 Population Based Scheduling Heuristics (PBSH) 40 ACO Parameter Values Ants 50 α 0.01 β 0.99 ρ 0.4 Q 100 Figure 5.2: Ant Colony Architecture. The diagram represented in Figure 5.2 illustrates in general terms the whole process of combining Cloudlets to VMs, which is described in Algorithm 3. The scheduler firstly creates the ants and then distribute the Cloudlets to each ant. Then, the ants evaluate the VMs based on the previously mentioned equations to choose the best VM. Finally, the tabu of each ant is updated with the chosen VM and the algorithm is repeated in the following iteration. 5.3 Random Biased Sampling (RBS) The Random Biased Sampling (RBS) algorithm provides a way to construct a network of resources (VMs). Those networked resources are then divided into groups. The groups are given a degree. This degree on an average of all resource groups must be the same

50 Population Based Scheduling Heuristics (PBSH) 41 Algorithm 3 AntColony VM Assignment Algorithm Require: α,β,max iterations,cloudlet list,vm list 1: for i in Cloudlet list and k in VM list do 2: pair Cloudleti,V Mk τ i,j (0) = C // pheromone(c) 3: end for 4: VM k Ant j randompick(ant pool ) 5: Ant tabu j add(vm k ) 6: while NOT done do 7: for k = 1 to m do 8: VM s select(ant k,vm list,cloudlet list ) // as in Eq 5.5 9: Ant tabu j add(vm s) 10: end for 11: for k = 1 to m do 12: L k calculate() // as in Eq : end for 14: τ i,j update() // as in Eq : pheromone global update() // as in Eq : increment(iterations) 17: end while achieving balanced scheduling. The created network of resources is self-organized and able to distribute the tasks based on only local knowledge. Thus, the Random Sampling algorithm represents a viable solution to the cloud task-scheduling problem [89]. RBS constructs a graph of resource each with a node in degree (NID) and a walk length threshold (υ). The tasks coming into the servers have an associated walk in length (ω) that is used to schedule those tasks to the appropriate resources. The NID value varies depending on the number of free resources on a given node. When a task i comes into the servers, node k will only executes a task if the execution test is fulfilled (ω υ). If the previous condition is not satisfied, ω is incremented by one and sent to the other nodes and the execution test is applied again. Algorithm 4 represent the general procedure of RBS in a cloud environment. The following steps shortly describe the functioning of the algorithm: Step1: The VMs are split into n groups. Each group contains an equal number of VMs; Step2: Each Group is assigned walk length threshold (υ) and an NID. The NID is equal to the number of free VMs in that group;

51 Population Based Scheduling Heuristics (PBSH) 42 Algorithm 4 Random Biased Sampling VM Assignment Algorithm Require: Cloudlet list andvm list 1: for k = 1 to n do 2: Groups(q) divide(vm list,groupsize(number(r))) 3: end for 4: for k = 1 to q do 5: Group k ascending(wil) 6: end for 7: for all Cloudlet i in Cloudlet list do 8: Cloudlet i random(wil) 9: end for 10: for k = 1 to m and i to q do 11: if Cloudlet k.wil Group i.wil then 12: Group i Cloudlet k 13: else 14: increment(clouldet k.wil,1) 15: Group i+1 Cloudlet k 16: end if 17: end for Step3: Each Cloudlet is given a random ω; Step4: The execution test is performed to assign Cloudlet i to group j ; Step5: The NID of groupj is reduced by one; Step6: The assignment inside the VMs groups is done in a cyclic way; Step7: Repeat starting from step 3.

52 Population Based Scheduling Heuristics (PBSH) 43 Figure 5.3: Random Biased Sampling Match-Making. Figure5.3depictsthematch-makingofCloudletstoVMSofthealgorithmonasimple diagram. In the figure, the RBS scheduler divides the VMs available into two groups. Then, the created VM groups are assigned a walk in length (WIL) values (WIL = 1.. n). Following this step, the Cloudlets are given a random WIL value between the available WIL values assigned to the VM groups. The RBS assigns a Cloudlet to a VM group if the Cloudlet WIL is equal or greater to the WIL of the VM group. If this condition is not applied, then RBS scheduler will increase the Cloudlet WIL in incremented by one and the algorithm is run again.

53 Chapter 6 Performance Evaluation The performance evaluation chapter presents the results of the simulation. In the first part of this chapter, the homogeneous simulations scenario is introduced and it s simulation results and explanations are presented. In the second part of the experiments, the heterogeneous simulations are detailed and presented along with their results, which are discussed in details. Finally, the simulations of ACBH, as well as its variations, are also displayed. 6.1 Simulation Scenarios Two scenarios have been used to analyze the bio-inspired schedulers fully and thoroughly. The first scenario comprises a homogeneous setup of physical resources that receives a homogeneous load, Cloudlets that impose the same amount of workload. The use of this scenario aims at testing the suggested algorithms against a base test, which is expected to be the optimum solution in this type of homogeneous setup. On the other hand, the second scenario represents a more realistic Cloud environment, in which heterogeneous resources and load are most likely to exist. 44

54 Experiments 45 The homogeneous environment scenario was conducted to show that even in the worst case conditions in which no scheduler is needed, the bio-inspired algorithms are expected to converge to the optimal scheduling performance, and the only difference in this particular case resides on the time each scheduler takes to produce the assignment of load, which shows that the bio-inspired scheduling requires significantly more time than the base test. Tables 6.1 and 6.2 show the experimental setup used in the homogeneous scenario respectively for the environment, VMs, and the load, Cloudlets. The experiment was conducted with the number of VMs ranging from 1000 to and Cloudlets. There is a focus on computational performance while conducting this analysis, so negligible network delay is observed on the execution of the Cloudlets during the simulation; thus, the default network topology provided by CloudSim was used in running the experiments. Several different aspects have been considered in the analysis in order to determine effectiveness and efficiency of the algorithms. The first measurement observed in the experiments consists in the scheduling time of the selected scheduling algorithm. The second measurement criteria involve the execution time that the Cloudlets generated in order to complete their work. The third measurement considers the execution time imbalance which gives a sense of the balancing ratio of the load during its execution in the simulator. Since all elements in this scenario present the same characteristics and capabilities, it represents the worst case for the scheduling algorithms, and the base test is necessarily the optimal solution for any time to scheduling setup due to equally assign Cloudlets to the virtual resources in a cyclic fashion. The heterogeneous environment scenario was implemented to mimic real cloud environment where task and virtual machines are not similar. In other words, different

55 Experiments 46 Table 6.1: VM Characteristics in the Homogeneous Setup Where: VM characteristics Values vmmips 1000 vmsize 5000 vmram 512 vmbw 500 vmpesnumber 1 vmmips: million instructions per second. vmsize: the size of the virtual machine in MB. vmram: the RAM of the virtual machine. vmbw: the bandwidth of the virtual machine. vmpesnumber: the number of processing elements of the virtual machine. Table 6.2: Cloudlet Characteristics in the Homogeneous Setup Where: Cloudlet characteristics Values clength 250 cfilesize 300 coutputsize 300 cpesnumber 1 clength: The required MIPS for the cloudlet. cfilesize: The required memory for the cloudlet. coutputsize: The size of the output file of the cloudlet. cpesnumber: The required processing elements number for the cloudlet. workloads were submitted to virtual machines that also have different capabilities. For the setup on this scenario, a different range of parameters was used, restricting to smaller dimensions; however, such restrictions did not impact on the results obtained. Thus, the

56 Experiments 47 number of virtual machines was reduced to 50 and the number of Cloudlets was reduced to Tables 6.3, 6.5, and 6.4 list the characteristics of the VMs, Datacenters, and the tasks, respectively. Table 6.3: VM Characteristics in the Heterogeneous Setup Heterogeneous VM characteristics Values vmmips vmsize 5000 vmram 512 vmbw 500 vmpesnumber 1 Table 6.4: Cloudlet Characteristics in the Heterogeneous Setup Heterogeneous Cloudlet characteristics Values clength cfilesize 300 coutputsize 300 cpesnumber 1 Table 6.5: Datacenter Characteristics in the Heterogeneous Setup Datacenter characteristics Values CostPerMemeory CostPerStorage CostPerBandwidth CostPerPrcessing 3 All the simulations below were executed for ten times and the results displayed in the figures are the averages of those runs. Moreover, the graphs have confidence intervals

57 Experiments 48 shown as well. The simulation parameters (i.e. Cloudlet, VM, Datacenter characteristics) used are based on what has been extensively used in the literature. These parameters have to be distinct so that the results shows the differences in performance of the scheduling algorithms as suppose to using similar simulations parameters that will not show performance differences. 6.2 CloudSim CloudSim emerged from the need to have a cloud simulation environment that would extend the traditional distributed systems simulators (Grid and Network) [37]. Moreover, testing on real cloud environment is costly and imposes several challenges, which considerably increase the complexity in conducting large-scale experiments; several factors drive such challenges, such as (i) variation in the clouds demand, supply pat tern, system size, and resources; (ii) the heterogeneous characteristics of user and QoS requirements in these dynamic environments; and (iii) variations in applications performance, dynamic, and scaling requirements. Moreover, a simulator is crucial in providing controlled scenarios in which results are reproducible over the most diverse combination of set up parameters [88]. Furthermore, aside from money and time when using commercial clouds, testing the scheduling and allocation of resources are challenging and not practical due to many factors. The first factor is variation in the clouds demand, supply pattern, system size, and resources. Secondly, the heterogeneousness of user and QoS requirements is due to the cloud being a dynamic environment. The third factor is the variations in applications performance, dynamic, and scaling requirements. Thus, the use of real cloud environments, such as Amazon EC2 and Microsoft Azure, cannot be used to test scheduling and allocations of resources (benchmarking) under varying conditions because of the rigidity

58 Experiments 49 imposed by infrastructure. Thus, repeating test cases and getting reliable performance results is an extremely hard task because of the need to reconfigure benchmark across the cloud platform for multiple runs. Therefore, the need arises for a more controllable and developer-friendly cloud simulation environment to perform benchmarks and get reliable results. Hence, CloudSim was created to fulfill this need. The need for a simulation tool that is able to provide control to the developer is very crucial in testing the hypothesis. A tool where it is easy to reproduce results will benefit researchers and IT companies greatly. Among the benefits of using a simulation based cloud environment are the controllability over the environment and experimenting with a verity of workloads and resources for developing and testing new scheduling and provisioning algorithms [88]. CloudSim: a new, generalized, and extensible simulation framework that allows seamless modeling, simulation, and experimentation of emerging Cloud computing infrastructures and application services. This simulation tool facilitate develops to configure the test environment with ease and build test scenarios. CloudSim provides test results and can be enhanced and added to by the developers. Furthermore, CloudSim saves time because of the ease of use and very flexible because a developer can build and test environment and scheduling algorithms for testing their applications in a heterogeneous cloud environment was developed to simulate real cloud environment and more specifically for testing the scheduling of tasks, Cloudlets, to virtual machines (VMs). Moreover, CloudSim is also capable of testing the virtual machines migration from one physical server to another to balance the loads on those servers.

59 Experiments 50 Figure 6.1: CloudSim Architecture [37] CloudSim Architecture The Figure 6.1 shows that CloudSim is a multi-layered Simulation environment. This environment was based on SimJava but then changed to be able to support advanced operations that are not supported by it. CloudSim provides the core functionalities, such as queuing and event processing. Also, the creation of the cloud core entities (services, host, data center, broker, and VMs) is handled by CloudSim [38]. The main classes that are needed to be extended are listed below. Those classes are essential to be overwritten in order to implement our own scheduling algorithms. DatacenterBroker: this class mediates between the user and services. In this class, the user adds the scheduling algorithms for assigning the Cloudlets to VMs. Thus, DatacenterBroker class must be extended by researchers and developers to simulate the scheduling algorithms.