10. Multiprocessor Scheduling (Advanced)

Transcription

1 10. Multirocessor Scheduling (Advanced) Oerating System: Three Easy Pieces 1

2 Multirocessor Scheduling The rise of the multicore rocessor is the source of multirocessorscheduling roliferation. w Multicore: Multile CPU cores are acked onto a single chi. Adding more CPUs does not make that single alication run faster. à You ll have to rewrite alication to run in arallel, using threads. How to schedule jobs on Multile CPUs? AOS@UC 2

3 Single CPU with cache CPU Cache Cache Small, fast memories Hold coies of oular data that is found in the main memory. Utilize temoral and satial locality Memory Main Memory Holds all of the data Access to main memory is slower than cache. By keeing data in cache, the system can make slow memory aear to be a fast one AOS@UC 3

4 Cache coherence Coherence of shared resource data stored in multile caches. 0. Two CPUs with caches sharing memory 1. CPU0 reads a data at address 1. CPU 0 CPU 1 CPU 0 CPU 1 Cache Cache Cache! Cache Bus Bus! Memory ! Memory AOS@UC 4

5 Cache coherence (Cont.) 2.! is udated and CPU1 is scheduled. 3. CPU1 re-reads the value at address A CPU 0 CPU 1 CPU 0 CPU 1 Cache Cache!!! Cache Cache Bus Bus! Memory ! Memory CPU1 gets the old value # instead of the correct value #. AOS@UC 5

6 Cache coherence solution Bus snooing w Each cache ays attention to memory udates by observing the bus. w When a CPU sees an udate for a data item it holds in its cache, it will notice the change and either invalidate its coy or udate it. AOS@UC 6

7 Don t forget synchronization When accessing shared data across CPUs, mutual exclusion rimitives should likely be used to guarantee correctness. 1 tyedef struct Node_t { 2 int value; 3 struct Node_t *next; 4 } Node_t; 5 6 int List_Po() { 7 Node_t *tm = head; // remember old head... 8 int value = head->value; //... and its value 9 head = head->next; // advance head to next ointer 10 free(tm); // free old head 11 return value; // return value at head 12 } Simle List Delete Code AOS@UC 7

8 Don t forget synchronization (Cont.) Solution 1 thread_mtuex_t m; 2 tyedef struct Node_t { 3 int value; 4 struct Node_t *next; 5 } Node_t; 6 7 int List_Po() { 8 lock(&m) 9 Node_t *tm = head; // remember old head int value = head->value; //... and its value 11 head = head->next; // advance head to next ointer 12 free(tm); // free old head 13 unlock(&m) 14 return value; // return value at head 15 } Simle List Delete Code with lock AOS@UC 8

9 Cache Affinity Kee a rocess on the same CPU if at all ossible w A rocess builds u a fair bit of state in the cache of a CPU. w The next time the rocess run, it will run faster if some of its state is already resent in the cache on that CPU. A multirocessor scheduler should consider cache affinity when making its scheduling decision. AOS@UC 9

10 Single queue Multirocessor Scheduling (SQMS) Put all jobs that need to be scheduled into a single queue. w Each CPU simly icks the next job from the globally shared queue. w Cons: Some form of locking have to be inserted à Lack of scalability Cache affinity Examle: Queue A B C D E NULL Possible job scheduler across CPUs: CPU0 A E D C B CPU1 B A E D C CPU2 C B A E D CPU3 D C B A E (reeat) (reeat) (reeat) (reeat) AOS@UC 10

11 Scheduling Examle with Cache affinity Queue A B C D E NULL CPU0 A E A A A CPU1 B B E B B CPU2 C C C E C CPU3 D D D D E (reeat) (reeat) (reeat) (reeat) w Preserving affinity for most Jobs A through D are not moved across rocessors. Only job e Migrating from CPU to CPU. w Imlementing such a scheme can be comlex. AOS@UC 11

12 Multi-queue Multirocessor Scheduling (MQMS) MQMS consists of multile scheduling queues. w Each queue will follow a articular scheduling disciline. w When a job enters the system, it is laced on exactly one scheduling queue. w Avoid the roblems of information sharing and synchronization. AOS@UC 12

13 MQMS Examle With round robin, the system might roduce a schedule that looks like this: Q0 A C Q1 B D CPU0 CPU1 A A C C A A C C A A C C B B D D B B D D B B D D MQMS rovides more scalability and cache affinity. AOS@UC 13

14 Load Imbalance issue of MQMS After job C in Q0 finishes: CPU0 CPU1 Q0 A Q1 B D A A A A A A A A A A A A B B D D B B D D B B D D A gets twice as much CPU as B and D. After job A in Q0 finishes: Q0 Q1 B D CPU0 CPU1 B B D D B B D D B B D D CPU0 will be left idle! AOS@UC 14

15 How to deal with load imbalance? The answer is to move jobs (Migration). w Examle: Q0 Q1 B D The OS moves one of B or D to CPU 0 Q0 D Q1 B Or Q0 B Q1 D AOS@UC 15

16 How to deal with load imbalance? (Cont.) A more tricky case: Q0 A Q1 B D A ossible migration attern: w Kee switching jobs CPU0 CPU1 A A A A B A B A B B B B B D B D D D D D A D A D Migrate B to CPU0 Migrate A to CPU1 AOS@UC 16

17 Work Stealing Move jobs between queues w Imlementation: w Cons: A source queue that is low on jobs is icked. The source queue occasionally eeks at another target queue. If the target queue is more full than the source queue, the source will steal one or more jobs from the target queue. High overhead and trouble scaling AOS@UC 17

18 Linux Multirocessor Schedulers O(1) w A Priority-based scheduler w Use Multile queues w Change a rocess s riority over time w Schedule those with highest riority w Interactivity is a articular focus Comletely Fair Scheduler (CFS) (current mainline) w Deterministic roortional-share aroach w Based on Staircase Deadline (fairness is the focus) w Red-black tree for scalability AOS@UC 18

19 Linux Multirocessor Schedulers (Cont.) BF Scheduler (BFS) (Not in the mainline) w A single queue aroach w Proortional-share w Based on Earliest Eligible Virtual Deadline First (EEVDF) w Focus on interactive (not scale well with cores). Suerseded by MuQSS to fix that The battle of schedulers : Kolivas (SD) vs Molnar (CFS) AOS@UC 19

20 And you have to realize that there are not very many things that have a ged as well as the scheduler. Which is just another roof that scheduling is easy. Linus Torvalds, 2001 [43] Scheduling is not easy!, E.g: The Linux Scheduler: a Decade of Wasted Cores htt:// final29.df AOS@UC 20

21 Disclaimer: Disclaimer: This lecture slide set is used in AOS course at University of Cantabria. Was initially develoed for Oerating System course in Comuter Science Det. at Hanyang University. This lecture slide set is for OSTEP book written by Remzi and Andrea Araci-Dusseau (at University of Wisconsin) 21