Scheduling Algorithms in Large Scale Distributed Systems

Scheduling Algorithms in Large Scale Distributed Systems Prof.dr.ing. Florin Pop University Politehnica of Bucharest, Faculty of Automatic Control and Computers (ACS-UPB) National Institute for Research and Development in Informatics (ICI), Bucharest Emails: florin.pop@cs.pub.ro,florin.pop@ici.ro 21.04.2018 Scheduling Algorithms in LSDS 1

21.04.2018 Scheduling Algorithms in LSDS 2

Historically, the most common Scheduler is the user... 21.04.2018 Metode şi Algoritmi de Planificare Curs 1 3

Introduction to scheduling Scheduling problem definition Classification of scheduling problems Complexity of scheduling problem DAG Scheduling 21.04.2018 Scheduling Algorithms in LSDS 4

General Scheduling Problem (1/3) Resource Constrained Project Scheduling Problem (RCPSP) We have: Tasks j = 1,..., n with processing times p j Resources k = 1,..., r A constant amount of R k units of resource k is available at any time. During processing, task j occupies r jk units of resource k. n and r are finite. Precedence constrains i j between some activities i, j with the meaning that activity j cannot start before i is finished. 21.04.2018 Scheduling Algorithms in LSDS 5

General Scheduling Problem (2/3) The objective is to determine starting times Sj for all tasks j in such a way that: at each time t the total demand for resource k is not greater than the availability R k the given precedence constraints are fulfilled, i. e. S i + p i S j if i j some objective function f(c 1,..., C n ) is minimized where C j = S j + p j is the completion time of activity j. The fact that activities j start at time S j and finish at time S j + p j implies that the activities j are not preempted. We may relax this condition by allowing preemption (activity splitting). 21.04.2018 Scheduling Algorithms in LSDS 6

General Scheduling Problem (3/3) If preemption is not allowed the vector S = (S j ) defines a schedule. S is called feasible if all resource and precedence constraints are fulfilled. One has to find a feasible schedule which minimizes the objective function f(c 1,..., C n ). In project planning f(c 1,..., C n ) is often replaced by the makespan C max which is the maximum of all C j values. The constraints S i + p i S j may be replaced by S i + d ij S j, were d ij represents the deadline for task j on resource i. 21.04.2018 Scheduling Algorithms in LSDS 7

Scheduling example Let s consider: n = 4, r = 2 (R 1 = 5, R 2 = 7) Precedence constrain 2 3, and: j 1 2 3 4 p j 4 3 5 8 r j1 2 1 2 2 r j2 3 5 2 4 21.04.2018 Scheduling Algorithms in LSDS 8

Example of Scheduling Problems Task scheduling for computer machine Production scheduling Robotic cell scheduling Computer processor scheduling Timetabling Personnel scheduling Railway scheduling Air traffic control Scheduling of Big Data workflows 21.04.2018 Scheduling Algorithms in LSDS 9

Machine Scheduling Problems Here we will consider single machine problems, parallel machine problems, and shop scheduling problems 21.04.2018 Scheduling Algorithms in LSDS 10

Single machine problem We have n tasks to be processed on a single machine (r = 1). Additionally precedence constraints between the tasks may be given. This problem can be modeled with r = 1, R 1 = 1, and r j1 = 1 for all tasks j. 21.04.2018 Scheduling Algorithms in LSDS 11

Parallel Machine Problem We have tasks j as before and m identical machines M 1,..., M m. The processing time for j is the same on each machine. One has to assign the tasks to the machines and to schedule them on the assigned machines. This problem correspond with r = 1, R 1 = m, and r j1 = 1 for all tasks j. 21.04.2018 Scheduling Algorithms in LSDS 12

Parallel Machine Problem For unrelated machines the processing time p jk depends on the machine M k on which j is processed. The machines are called uniform if p jk = p j /r k. In a problem with multi-purpose machines a set of machines m j is associated with each task j indicating that j can be processed on one machine in m j only. 21.04.2018 Scheduling Algorithms in LSDS 13

Shop Scheduling Problem In a general shop scheduling problem we have m machines M 1,..., M m and n tasks. Job j consists of n(j) operations O 1j, O 2j,..., O n(j)j where O ij must be processed for p ij time units on a dedicated machine m ij {M 1,..., M m }. Two operations of the same job cannot be processed at the same time. Precedence constraints are given between the operations. To model the general shop scheduling problem r = n + m resources k = 1,..., n + m with R k = 1 for all k. While resources k = 1,..., m correspond to the machines, resources m + j (j = 1,..., n) are needed to model that different operations of the same job cannot be scheduled at the same time. n(1) + n(2) +... + n(n) activities O ij where operation O ij needs one unit of machine resource m ij and one unit of the job resource m + j. 21.04.2018 Scheduling Algorithms in LSDS 14

Shop Scheduling Problem A job-shop problem is a general shop scheduling problem with chain precedence constraints of the form: O 1j O 2j... O n(j)j. A flow-shop problem is a special job-shop problem with n(j) = m operations for j = 1,..., n and m ij = M i for i = 1,..., m and j = 1,..., n. In a permutation flow-shop problem the jobs have to be processed in the same order on all machines. 21.04.2018 Scheduling Algorithms in LSDS 15

Shop Scheduling Problem An open-shop problem is like a flow-shop problem but without precedence constraints between the operations. 21.04.2018 Scheduling Algorithms in LSDS 16

Classification of Scheduling Problems Classes of scheduling problems can be specified in terms of the three-field classification a b g where: a specifies the machine environment b specifies the task characteristics, and g describes the objective function(s). 21.04.2018 Scheduling Algorithms in LSDS 17

a - Machine Environment To describe the machine environment the following symbols are used: 1: single machine P: parallel identical machines Q: uniform machines R: unrelated machines MPM: multipurpose machines J: job-shop F: flow-shop O: open-shop The above symbols are used if the number of machines is part of the input. If the number of machines is fixed to m we write P m, Q m, R m, MPM m, J m, F m, O m. 21.04.2018 Scheduling Algorithms in LSDS 18

b - Task Characteristics pmtn: preemption r j : release times d j : deadlines p j = 1 or p j = p or p j {1,2}: restricted processing times Prec: arbitrary precedence constraints intree (outtree): intree (or outtree) precedences chains: chain precedences series-parallel: a series-parallel precedence graph 21.04.2018 Scheduling Algorithms in LSDS 19

g - Objective Functions Two types of objective functions are most common: bottleneck objective functions max {f j (C j ) j=1,..., n} sum objective functions S f j (C j ) = f 1 (C 1 ) + f 2 (C 2 ) +... + f n (C n ). C max and L max symbolize the bottleneck objective functions with f j (C j ) = C j (makespan) and f j (C j ) = C j - d j (maximum lateness), respectively. 21.04.2018 Scheduling Algorithms in LSDS 20

g - Objective Functions Common sum objective functions are: SC j (mean flow-time) and Sw j C j (weighted flow-time) SU j (number of late jobs) and Sw j U j (weighted number of late jobs) where U j = 1 if C j >d j and U j = 0 otherwise. ST j (sum of tardiness) and Sw j T j (weighted sum of tardiness) where the tardiness of job j is given by T j = max { 0, C j - d j }. 21.04.2018 Scheduling Algorithms in LSDS 21

Scheduling problem - Examples 1 prec; p j = 1 Sw j C j P 2 C max P p j = 1; r j Sw j U j R 2 pmtn; intree C max J 3 C max F p ij = 1; outtree; r j S C j O m p j = 1 S T j The scheduling zoo: A searchable bibliography on scheduling Peter Brucker and Sigrid Knust http://schedulingzoo.lip6.fr 21.04.2018 Scheduling Algorithms in LSDS 22

Complexity Theory Polynomial algorithms Classes P and NP NP-complete and NP-hard problems 21.04.2018 Scheduling Algorithms in LSDS 23

Polynomial algorithms A problem is called polynomially solvable if it can be solved by a polynomial algorithm. Example: 1 Sw j C j can be solved by scheduling the jobs in an ordering of non-increasing w j /p j values. Complexity: O(nlogn) If we replace the binary encoding by an unary encoding we get the concept of a pseudo-polynomial algorithm. Example: An algorithm for a scheduling problem with computational effort O(Sp j ) is pseudo-polynomial. 21.04.2018 Scheduling Algorithms in LSDS 24

Classes P and NP A problem is called a decision problem if the output range is {yes, no}. We may associate with each scheduling problem a decision problem by defining a threshold k for the objective function f. The decision problem is: Does a feasible schedule S exist satisfying f(s) k? P is the class of decision problems which are polynomially solvable. NP is the class of decision problems with the property that for each yes -answer a certificate exists which can be used to verify the yes -answer in polynomial time. Decision versions of scheduling problems belong to NP (a yes -answer is certified by a feasible schedule S with f(s) k). P NP holds. It is open whether P = NP. 21.04.2018 Scheduling Algorithms in LSDS 25

NP- complete and NP- hard problems For two decision problems P and Q, we say that P reduces to Q (denoted by P a Q) if there exists a polynomial-time computable function g that transforms inputs for P into inputs for Q such that x is a yes -input for P if and only if g(x) is a yes -input for Q. Properties of polynomial reductions: Let P, Q be decision problems. If P a Q then Q P implies P P (and, equivalently, P P implies Q P). Let P, Q, R be decision problems. If P a Q and Q a R, then P a R. A decision problem Q is called NP - complete if Q NP and, for all other decision problems P NP, we have P a Q. If any single NP-complete decision problem Q could be solved in polynomial time then we would have P = NP. 21.04.2018 Scheduling Algorithms in LSDS 26

NP- complete and NP- hard problems To prove that a decision problem P is NP - complete it is sufficient to prove the following two properties: P NP, and there exist an NP- complete problem Q with Q a P. An optimization problem is NP- hard if its decision version is NP- complete. Cook [1971] has shown that the satisfiability problem from Boolean logic is NP- complete. Using this result he used reduction to prove that other combinatorial problems are NPcomplete as well. 21.04.2018 Scheduling Algorithms in LSDS 27

Complexity of machine scheduling problems http://www.mathematik.uni-osnabrueck.de/research/or/class/ Elementary reductions 21.04.2018 Scheduling Algorithms in LSDS 28

Polynomially solvable single machine problems 21.04.2018 Scheduling Algorithms in LSDS 29

NP - hard single machine scheduling problems 21.04.2018 Scheduling Algorithms in LSDS 30

Minimal and maximal open problems 21.04.2018 Scheduling Algorithms in LSDS 31

NP- hard parallel machine problems without preemption 21.04.2018 Scheduling Algorithms in LSDS 32

Taxonomy General presentation Distributed systems layer Scheduling Taxonomy for Distributed Systems (Grid) Scheduling models Taxonomy of Systems Taxonomy of Resources Taxonomy of Applications 19.10.2010 Metode si Algoritmi de Planificare Curs 3 33

Scheduling in distributed systems The scheduling problem Allocate a group of different tasks to available resources NP-Complete problem Optimizations of approximate algorithms are required Dynamic (available) resource allocation Tasks With dependences Without dependencies With preemption Without preemption With specific constrains Environment: Computational and data GRID Heterogeneous resources with different processing capacities Shared resources can be used by multiple users 19.10.2010 Metode si Algoritmi de Planificare Curs 3 34

Scheduling in distributed systems Objectives Optimize the application execution Minimize the total execution time of the tasks Efficient use of computing resources Successful completion of tasks Deadline restrictions Resource requirements Optimize the scheduling process R (P, Q, J, F, O) r j ; d j ; p j ; pmtn; prec C max (Sw j C j ) 19.10.2010 Metode si Algoritmi de Planificare Curs 3 35

Scheduling in distributed systems In distributed computing, sites can participate in by offering resources, provided that certain conditions such as security requirements are met. The interaction between distributed resources during the execution of a job requires a scheduling layer that uses a different scheduling paradigm. Scheduling process requires interaction to remote sites and their local scheduling systems => the use of several scheduling layers for the Grid. Different level for scheduling process: lower-level scheduling instances, used for the local scheduling of resources, and higher-level scheduling instances, used for interaction in coordinated scheduling in a distributed system. Solution: Decentralized scheduling Use of monitoring as feedback 19.10.2010 Metode si Algoritmi de Planificare Curs 3 36

Taxonomy 19.10.2010 Metode si Algoritmi de Planificare Curs 3 37

Taxonomy 19.10.2010 Metode si Algoritmi de Planificare Curs 3 38

Taxonomy 19.10.2010 Metode si Algoritmi de Planificare Curs 3 39

Taxonomy 19.10.2010 Metode si Algoritmi de Planificare Curs 3 40

Taxonomy 19.10.2010 Metode si Algoritmi de Planificare Curs 3 41

Scheduling Taxonomy 19.10.2010 Metode si Algoritmi de Planificare Curs 3 42

Scheduling Taxonomy 19.10.2010 Metode si Algoritmi de Planificare Curs 3 43

Push Model Scheduling methods Manager pushes jobs from Q to a resource. Used in Clusters, Grids Pull Model P2P Agent request for a job for processing from jobpool Commonly used in P2P systems such as SETI@Home Hybrid Model (both push and pull) Broker deploys an agent on resources, which pulls jobs from a resource. May use in Grid (e.g., Nimrod-G system). Broker also pulls data from user host or separate data host (distributed datasets) (e.g., Gridbus Broker) 19.10.2010 Metode si Algoritmi de Planificare Curs 3 44

Taxonomy of Systems Computational Grid: distributed supercomputing (parallel application execution on multiple machines) high throughput (stream of jobs) Data Grid: provides the way to solve large scale data management problems Service Grid: systems that provide services that are not provided by any single local machine. on demand: aggregate resources to enable new services Collaborative: connect users and applications via a virtual workspace Multimedia: infrastructure for real-time multimedia applications 19.10.2010 Metode si Algoritmi de Planificare Curs 3 45

Taxonomy of Resources β classification: 1, P, Q, MPM, R, J, F, O Time-shared vs. Non time-shared Dedicated vs. Non-dedicated Preemptive vs. Non-preemptive 19.10.2010 Metode si Algoritmi de Planificare Curs 3 46

Taxonomy of Applications Distributed supercomputing consume CPU cycles and memory. High-Throughput Computing unused processor cycles On-Demand Computing meet short-term requirements for resources that cannot be cost-effectively or conveniently located locally. Data-Intensive Computing: e.g. satellite image processing Collaborative Computing enabling and enhancing human-to-human interactions (e.g.: CAVE5D system supports remote, collaborative exploration of large geophysical data sets and the models that generated them). Big Data Workloads???? 19.10.2010 Metode si Algoritmi de Planificare Curs 3 47

Characteristics of Cluster Scheduling P r j ; d j ; p j ; pmtn; prec C max (Sw j C j ) Q r j ; d j ; p j ; pmtn; prec C max (Sw j C j ) Homogeneity of resource and application Dedicated resource Centralized scheduling architecture High-speed interconnection network Monotonic performance goal 19.10.2010 Metode si Algoritmi de Planificare Curs 3 48

Building a Schedule using a hyper-heuristic Advantages of hyperheuristics for clusters Cheap and fast to implement & Produce solutions of good quality Require limited domain-specific knowledge Robustness: can be effectively applied to a wide range of problems and problem instances Initial solution Objective value Hyper-heuristic algorithm Set of low-level heuristics CPU time Selected lowlevel heuristic Objective value Perturbed solution Objective value Current solution (according to acceptance criterion) 19.10.2010 Metode si Algoritmi de Planificare Curs 3 49

Description of task dependencies A task graph is a directed acyclic graph G(V, E, c, τ) where: V is a set of nodes (tasks); E is a set of directed edge (dependencies); c is a function that associates a weight c(u) to each node; represent the execution time of the task T u, which is represented by the node u in V; τ is a function that associates a weight to a directed edge; if u and v are two nodes in V then τ denotes the inter-tasks communication time between T u and T v. st(u) is start time and ft(u) is finish time Makespan = max{ft(u)} 26.10.2010 Metode si Algoritmi de Planificare Curs 4 50

DAG Directed Acyclic Graph 1: a = 2 1 2: u = a + 2 3: v = a * 7 2 3 4: x = u + v Scheduling examples: 4 P1 P2 P1 P2 1 1 2 3 2 3 4 4 time time Metode si Algoritmi de Planificare Curs 4 26.10.2010 51

Assigning priority: Example of DAG Task tlevel (top-level) - for a node u is the weight of the longest path from the source node to u. blevel (bottom-level) - for a node u is the weight of the longest path from u to an exit node. Critical path (CP) - the longest path in a graph. ALAP (As Late As Possible): ALAP(u) =CP blevel(u) 1 2 4 1 1 1 2 3 5 5 10 4 4 3 3 1 7 1 1 6 4 7 4 8 4 5 6 5 9 1 26.10.2010 Metode si Algoritmi de Planificare Curs 4 52

Task Dependencies Model and DAG Scheduling ALAP( u) CP - blevel( u) Node b-level ALAP 1 22 0 2 3 19 3 5 17 4 1 21 26.10.2010 Metode si Algoritmi de Planificare Curs 4 53

t-level b-level 26.10.2010 Metode si Algoritmi de Planificare Curs 4 54

Algorithms for DAG Scheduling A taxonomy of the DAG scheduling problem Taxonomy of task dependency scheduling algorithms in Grid environments 26.10.2010 Metode si Algoritmi de Planificare Curs 4 55

The MCP (Modified Critical Path) algorithm Step 1. Compute the ALAP time of each node. Step 2. 2.1 For each node, create a list which consists of the ALAP times of the node itself and all its children in a descending order. 2.2 Sort these lists in an ascending lexicographical order. 2.3 Create a node list according to this order. Step 3. Repeat 3.1 Schedule the first node in the node list to a processor that allows the earliest execution, using the insertion approach. 3.2 Remove the node from the node list Until the node list is empty. 26.10.2010 Metode si Algoritmi de Planificare Curs 4 56

Metrics for DAG Adaptive Scheduling Granularity is a measure of the communication to computation ratio DAG can be: coarse grained (the computation dominates the communication) fine grained (the communication dominates the computation) CCR (Communication to Computation Ratio) CCR < 1 - coarse grained graph CCR = 1 - mixed CCR > 1 - fine grained graph min{ ( p)} g( G) max{ c( p, q)} RCCR (Resources Communication to Computation Ratio) CCR represents the necessary ratio for communication to computation RCCR represents the available ration for communication to computation 26.10.2010 Metode si Algoritmi de Planificare Curs 4 57

DAGs generation Random Leveled with link complexity Balanced DAGs of many real world workflow applications Laplace differential equation solving Stencil linear and non-linear image processing, seismic reconstruction FFT signal and image processing, electro-acoustic music and audio signal processing, medical imaging, image processing, pattern recognition, computational chemistry LU solving linear equations, systems, matrix inversion, FPGA programming 26.10.2010 Metode si Algoritmi de Planificare Curs 4 58

How to Live with NP - hard Scheduling Problems Small sized problems can be solved by Mixed integer linear programming Dynamic programming Branch and bound methods To solve problems of larger size one has to apply Approximation algorithms Heuristics 26.10.2010 Metode si Algoritmi de Planificare Curs 4 59

Other Types of Scheduling Problems Due-date scheduling Batching problems Multiprocessor task scheduling Cyclic scheduling Scheduling with controllable data Shop problems with buffers Inverse scheduling No-idle time scheduling Multi-criteria scheduling Scheduling with no-available constraints Scheduling problems are also discussed in connection with other areas: Scheduling and transportation Scheduling and game theory Scheduling and location problems Scheduling and supply chains 26.10.2010 Metode si Algoritmi de Planificare Curs 4 60

The Human Face of Big Data Full movie: https://www.youtube.com/watch?v=3g9lrjcqxbk Big Data will impact every part of your life: https://www.youtube.com/watch?v=0q3srsuymys What Exactly Is Big Data? https://www.forbes.com/video/4857597029001 25.09.2017 Introduction to Big Data 61

Thank you! for scheduling this seminar! I am interested in the following research problem Finding Pure Nash Equilibrium for the Resource-Constrained Project Scheduling Problem Energy-aware task scheduling Power consumption optimization 21.04.2018 Scheduling Algorithms in LSDS 62