Scheduling Monday, November 22, :22 AM

Transcription

1 Scheduling Page 1 Scheduling Monday, November 22, :22 AM The scheduling problem (Chapter 9) Decide which processes are allowed to run when. Optimize throughput, response time, etc. Subject to constraints including context switch time, swap time, etc. Kinds of scheduling: Short term: which process, already in memory, gets a slice of time next. Medium-term: which processes are loaded partially or fully into memory. Long term: which processes are "admitted" for later processing. AKA "admission control". I/O: which pending I/O request should be handled first?

2 Scheduling Page 2

3 Scheduling Page 3 Long-term scheduling Also known as "admission control". A big research topic in grid computing right now. Questions asked by a long-term scheduler: How many concurrent processes can the system handle? How much memory is needed for each? How important is response time versus throughput versus efficiency?

4 Scheduling Page 4 Long-term scheduling factors Response time: how long until one gets an answer to a problem? Example: weather forecasting must be finished before the 7pm news. Throughput: how much of the cpu can one use by latency hiding? Example: How fast can one finish n predictions of geological substructure, where n is large?

5 Scheduling Page 5 Typical admission control algorithm Predict resource utilization. Predict overlap and latency hiding. Admit process only if within bounds. Otherwise, defer process until there are enough resources to assure completion. (banker's algorithm) Application: grid computing Process: a parallel program that runs on a grid. Resource utilization: # of nodes required, amount of memory on each node (I/O bandwidth to each node) Resources must be known in advance (not a problem for grid applications)

6 Scheduling Page 6 Medium term scheduling Monday, November 22, :56 AM Medium-term scheduling Essentially, the swapping algorithm Most common strategy: least recently used (LRU). Keep track of time since each page was accessed by a process last. Always swap out page that was least recently accessed. Less common strategies: lookahead. Try to predict cpu access pattern. Preload pages that might be accessed next.

7 Scheduling Page 7 Short-term scheduling Monday, November 22, :59 AM Short-term scheduling Deals with processes that are not blocked waiting for swap-out, swap-in. Behavioral aspects: Throughput: how much work is performed. Response time: how quickly are units of work completed, once requested. All scheduling algorithms choose a balance between these criteria.

8 Scheduling Page 8 Priorities Priority: a number than indicates how important a process is, in terms of response time. Low priority: response time unimportant. High priority: response time very important. Key to our discussion: running high priority processes is not enough; lowpriority processes still need to satisfy response-time limits. Note: the game here is to balance: round-robin scheduling (equal slices) optimal for response time over homogeneous client base. run to block: keep going until you can't. optimal for batch: no context switches.

9 Scheduling Page 9 Decision modes: Non preemptive: once process runs, it stays running till blocked or exit. Preemptive: once a process is scheduled, it is still pre-empted later in order to allow others to run. Scheduling algorithms First-come, first-serve (FCFS): queue unblocked processes, run each until blocked. non-preemptive minimum overhead. Round-robin: execute each process for a little of the time available, grant slice times relative to priority. Preemptive Fair treatment Shortest Process Next (SPN): priority queued, allow the process with the shortest "expected" time to run next. Non-preemptive. Must predict runtime Works best with lots of little processes or phases: average runtime is good predictor. used in RTOS's Shortest Remaining Time (SRT): priority queued, the process with the shortest predicted remaining time goes first. Non-preemptive Must predict runtime. Highest Response Ratio Next (HRRN): choose process with highest response ratio. Non-preemptive. Response ratio = (w+s)/s w=time spent waiting for processor s=expected computation time Feedback: establish several queues, each with lower priority than preceeding; bubble down processes in the queues. Preemptive. Queue position = priority.

10 Queue position = priority. Priority changes with time spent computing. Scheduling Page 10

11 Scheduling Page 11

12 Scheduling Page 12 Queueing theory Monday, November 22, :16 PM Problem: which one of these works best in my environment. Solution: can analyze the response of systems with queueing theory. History Early machines were batch processors. Job time was often predictable: Accounts payable: time # checks to write! Compilation: time program length Etc. Question: how to predict throughput based upon what we know about a system. Answer: queueing theory and simulation Queueing theory Mathematical model of batch processing. Overly precise models that almost model reality. Precise answers to a number of specific questions. No information in many realistic cases. Simulation Messy replication of processing operations. Results as accurate as the detail with which it is simulated. Sensitive to sloppiness in coding simulator behaviors. Queueing theory assumptions: Poisson arrival of jobs. Exponential service times. Whoa there! What is going on here?

13 Scheduling Page 13 Poisson processes Monday, November 22, :30 PM Poisson process: Models arrival time as a set of completely independent trials. Let t be the time of arrival of the next event, as a difference between now and then. Then: Prob(arrival time t q) = 1-e -λq λ: the rate of the process; a measure of how fast things happen. Small λ: events are spaced far apart. Large λ: events are spaced close together.

14 Scheduling Page 14

15 Scheduling Page 15 Facts about Poisson distributions Mean = 1/λ Standard deviation = 1/λ Memoryless: Prob(T x+t T>t)=Prob(T x) In other words, waiting longer (T>t) doesn't change things. Suppose X,Y,Z are independent Poission processes with parameters λx,λy,λz. Then the union of X,Y,Z is Poisson with parameter λx+λy+λz In other words, the union of a set of Poisson arrivals is Poisson, with the obvious parameter.

16 Scheduling Page 16 Exponential distribution Monday, November 22, :13 PM Exponential distribution: Prob(Processing time t) = 1-e -μt μ = parameter of jobs. Mean = 1/μ Standard deviation = 1/μ Memoryless: Prob(T x+t T>t)=Prob(T x)

17 Scheduling Page 17 What's going on? Monday, November 22, :27 PM Same equation, different meanings: A Poisson process is one in which the inter-event arrival times are exponential. An exponential variable is one in which the value of the variable follows an exponential distribution. Same equation, two meanings. Prob(next event before time t) = 1-e -λt Prob(processing time less than t) = 1-e -μt This symmetry makes analysis a lot easier than if these were different!

18 Scheduling Page 18

19 Poisson postulates a. Prob(1 arrival in Δt) = λδt b. Prob(>1 arrival in Δt) = o(δt) and negligible. c. Arrival time is independent of other arrivals and last arrival. Scheduling Page 19

20 Scheduling Page 20 Simple Poisson Properties Monday, November 22, :29 PM Combination/multiplexing: Splitting/demultiplexing: π 1 +π 2 + +π n =1.0 0 π i 1.0 π i is a constant probability of taking branch i.

21 Simple queueing Monday, November 22, :36 PM Scheduling Page 21

22 Scheduling Page 22 Kinds of queues Kendall notation: A/B/c/k/m/Z A=arrival process; default is M (Poisson) B=service process; default is M (Exponential) c=number of servers; default is 1 k=maximum queue size; default is infinite m=customer population; default is infinite Z=name of queueing discipline, default is FCFS Previous queue is M/M/1 Here's M/M/3:

23 Scheduling Page 23 Some startling mathematical results: For M/M/1 queue, Queue reaches steady state only if λ/μ<1 Prob(queue length is n) = p n (1-p) where p=λ/μ The mean queue length (enqueued jobs) is n=p/(1-p). The mean waiting-line length (jobs not being served) is w=p 2 /(1-p) (this includes jobs in process of being queued and dequeued) Little's laws: for M/M/1 queue: Mean time in system = n/λ Mean wait time before service = w/λ Key to proofs: principle of balance: input rate = output rate if system is in balance (lambda<mu)

24 Scheduling Page 24 The M/M/c queue Monday, November 22, :09 PM Generalization: c processors: Steady state: λ/cμ<1 S0=Prob(queue empty)= Mean time prior to service=

25 Scheduling Page 25 The M/M/ Queue Monday, November 22, :14 PM Infinite consumers, Poisson arrival (monkeys on keyboards) Waiting line length = w = 0 Mean time in system = 1/μ Mean queue length = λ/μ

26 Scheduling Page 26 Networks of Queues Monday, November 22, :18 PM Networks of queues Can connect queueing systems together into networks Output of one system is input to another. Surprising result: can compute the parameters of the network provided one knows the distribution parameters of inputs and service parameters of the queues. Key concept: "stability" is simpler to analyze than dynamic state. To achieve stability, system must come into "equilibrium" subject to "balance equations". Key equilibrium: input=output. In absence of this balance, system cannot be analyzed easily. When equilibrium is present, analysis is easy!

27 First case: no feedback Scheduling Page 27

28 Second case: simple feedback. Scheduling Page 28

29 Scheduling Page 29 Case 2: feedback in open networks (input from outside, output to outside) Queues 1-q with server constants µi, c i servers. Each queue gets input from all other queues, and input from outside world. Prob(queue i gets input from queue j) = πji Prob(queue i output exits network) is ψi Each queue's view of the world: Note: λk, ψk, π jk μk, ck constant. Variables are e1,, eq q equations in q unknowns unique solution. If this solution exists, it is the equilibrium point! ("Jackson's algorithm")

30 Scheduling Page 30 Closed systems Tuesday, November 30, :29 PM Alas, some systems are "closed": Example: fixed number of users typing on keyboards and receiving responses. Every input gets a response. Every response generates an input. Can model this as a "closed" system. Closed-world situation Fixed number of jobs N circulate within the system. No input from outside the system. At any time, the sum of jobs in all queues is N: If ni is the number of jobs in the queue for server i, then i ni = N Unfortunately, the simple Jackson method doesn't work in closed systems in which requests circulate and no request is ever deleted. Linear system is underdetermined. Too many solutions! Problem description: N requests. K server queues. A state of the system is a vector (n1,, nk) describing how many requests are queued for each server. Let vi represent the input rate for queue i. Let pkm represent the probability that the output of queue k will be assigned to the input of queue m. Jackson's equations:

31 Scheduling Page 31 Jackson's equations: vm = pkmvk are underdetermined. Eureka! All we're interested in is a steady state of the system. If such a state exists, it has some extremely simple properties. In particular, output=input (Jackson's equations) But, more specifically, The probability of a specific steady state must be independent of any prior state. Method: normalization constants: Compute probability of each state in a steady-state solution = Prob(n1,,nK) = (probability that queue 1 has n1 elements and queue 2 has n2 elements and queue K has nk elements) A really dirty trick: we know it's independent, so that it's proportional to (probability that queue i has ni elements) (product of all of these) = pi n i (pi n i is probability that queue i has ni elements) = (vi/µi) n i (by previous discussion of M/M/1 queues) Key: Assume the existence of a proportionality constant G(N) so that Prob (state n1,,nk) = 1/G(N) (vi/µi) n i

32 Scheduling Page 32 1/G(N) (vi/µi) n i So that Prob(state n1,,nk) = 1.0 If we can compute G(N), we know the probabilities for states, which in turn means that we know the expected aggregate state E= (n1,,nk) Prob(n1,,nK) Which predicts how long each queue is during steady-state behavior. With this, we have all we need to compute any throughput parameter. Big problem: combinatorial explosion To compute G(N), must sum over all states. There are lots of states! How to cope? Solution: Buzen's algorithm Build network incrementally Base new normalization constant on constants for smaller networks. Implementing Buzen's algorithm gk(n) = normalization constant for K servers, N jobs. gn(0) = 1 for all n g1(n)= probability that queue 1 has length n = (v1/µ1) n gm(n)=gm-1(n)+(vm/µm)gm(n-1) In short, build the network one server at a time, compensate for that server's behavior. Not all is won: Note: at end of Buzen's algorithm, we have GK(N) in terms of vi But we still don't know vi Must still apply Jackson's system, using the

33 extra equation from here. Scheduling Page 33

34 Scheduling Page 34 Beyond simple queueing theory Tuesday, November 30, :24 PM A whole course is possible (used to be on the qualifier for a Ph.D. in computer science) (fortunately, no longer!) What we understand Queueing of load-independent servers (tasks in queue don't affect response time) Queueing of load-dependent servers (tasks in queue affect response time) Distribution-independent results for very simple networks. What we don't understand Models other than FCFS. Non-linear response. Non-equilibrium behavior. Etc.

35 Scheduling Page 35 Why queueing theory fails Tuesday, November 23, :19 PM Good news: queueing theory explains a lot of simple cases and illustrates basic properties Distribution parameter Processing parameter Queue length Time in system Etc. Applicable to: Typing at keyboard I/o requests coming from programs. Service requests coming from network. Disk performance. Bad news: not realistic for many real systems λ is a variable over time. All that is known only works for special cases. Accuracy of conclusions very sensitive to assumptions made. No closed-form solution to the general problem. Not applicable to: Situations in which parameters change over time. Bursty behavior, e.g., hacking or denial of service. Situations in which division of labor changes (πjk not constant) To study unconstrained systems, must use simulation.

36 Scheduling Page 36 Simulating Poisson Processes Tuesday, November 23, :21 AM Given: Poisson constant λ And: uniformly distributed random number generator. Compute: time between adjacent arrivals. Basics: Probability density function (PDF): integral over interval is probability: Cumulative distribution function (CDF) integral from minimum boundary to some number a:

37 Uniform Distribution: Scheduling Page 37

38 Poisson Distribution: for one arrival time: Scheduling Page 38

39 Idea: generate uniformly distributed random numbers, "twist" them into following a different distribution. Start with CDF Generate uniform random numbers on the Y axis. Project each value onto the X axis. Result: X axis values follow the CDF. Scheduling Page 39

40 Reason this works: Scheduling Page 40

41 Scheduling Page 41 Implementing Poisson simulation So -(ln(1-drand48())/lambda has a Poisson distribution (because drand48() has a uniform distribution over [0,1))

42 Scheduling Page 42 Inside a simulator Tuesday, November 23, :29 PM Inside a simulator Some concept of timebase, e.g., a global clock. A state variable for each server showing what state it is in. State transition logic for servers at job initiation/completion. Queue data structures for each required queue. Simulator structures: Initialize all data structures Main loop: Compute server input events. Compute state transitions on servers. Record server output events. Increment time End main loop. Report performance. Computing server input events Compute delay time While delay is active, do nothing except delay--. When delay==0, inject event and compute new delay. Computing state transitions When starting a server on a job record starting time. Compute job length from distribution When computing a job on a server, Advance time while doing nothing, until job length has passed. When past, load next job from queue into server.

43 p j q server. Scheduling Page 43

44 Scheduling Page 44 Other important distributions Tuesday, November 23, :38 PM Pareto: describes distribution of sizes of disk files on disk, or in a user home directory. PDF(x) = ab a /x a+1 CDF(x) = 1-(b/x) a a: rate factor; determines ratio of small-to-large b: starting (smallest) size ( 0) Pareto distribution attributes Not memoryless; what files you have does determine what files you create! Starts at non-zero boundary Polynomial increase over time, not exponential. Pareto distribution is a reasonably good model for: Sizes of files on disk. User disk usage (sample=user) CPU utilization of a user over some time period.