Operating Systems 16 - CS 323 Assignment #2

Operating Systems 16 - CS 323 Assignment #2 Scheduler March 18, 2016 1 Objectives 1. Learn about scheduling in the Linux kernel 2. Understand the tradeoffs involved in scheduling 3. Work on the codebase of an existing operating system Section 2 of this document gives a high level overview of the assignment. Sections 3 and 4 give a detailed overview of how to add a scheduler to the Linux kernel and how your scheduler interacts with the rest of the kernel. 2 Your assignment Your task is to add a new, dummy, scheduler to the Linux kernel. You scheduler is supposed to schedule tasks whose kernel priority is in the range [131-135](which translates to the nice priority range [11-15]). You are provided a scheduler which implements a first come first serve scheduling policy (FCFS). Your assignment is to modify the provided scheduler and add the following functionalities: 1. Priority scheduling with support for 5 priority levels 2. Preemption of tasks when a task with higher priority becomes available 3. Preemption of tasks once its timeslice expires 4. A mechanism to prevent the starvation of processes with lower priority The CPU always belongs to the task with the highest priority. This means that if a task of priority higher than the priority of the task being executed becomes available, you should preempt the running task. Tasks with higher priorities can become available for a variety of reasons - a new task is activated, a blocked task is woken up, a lower priority task had its priority changed, etc. In addition to preemption, your scheduler should handle the case when a process gives up the CPU by yielding. In case there are multiple processes with the same priority, they should be scheduled in a round robin (RR) manner. 1

In a preemptive system, depending on the number of processes with a high priority, these processes might never get the CPU long enough to finish. This problem is known as starvation. In order to prevent this, you should implement a technique called priority ageing. The priority of tasks waiting for the CPU should be periodically increased proportionally to the time they were waiting. The amount of CPU time they get afterwards should be proportional to their priority. Once such a task gets the CPU, its priority is restored to the initial one. The age threshold parameter in the dummy scheduler code defines how long a task should wait in the runqueue before its priority gets increased by one level. This parameter is also accessible through the /proc filesystem at /proc/sys/kernel/sched dummy age threshold. The priority of a task should never change to a value outside the [11-15] range due to ageing. The scheduler is required to work only for the single CPU case. You do not need to worry about SMT issues like per-cpu runqueues, locking and task migration 3 The Linux Scheduler Linux and most modern operating systems implement preemptive scheduling by assigning priorities to processes based on different criteria (timeslice, waiting time, importance to the system etc.). 3.1 Process priorities in the Linux Scheduler In the Linux kernel, processes with higher priorities run before those with a smaller priority and processes with the same priority run in a round-robin. In order to improve its scheduling decisions, Linux allows for dynamic priority-scheduling. Each process starts with an initial priority. The scheduler can then increase or decrease the priority in order to follow a certain scheduling policy. There are two separate priority ranges. One range is called real-time priority normally ranging from 0-99. A smaller number means that a process has higher priority. All real-time processes have higher priorities than any other process in the system. Among themselves they are scheduled using the real-time scheduler (RT) in Linux. Usually, the programs you run on your machine are outside this range. Normal processes are mapped to real-time priorities 100-139. These processes have corresponding priorities in the second priority range called the nice value. A process can have a nice value from -20 to 19. The higher the value, the lower the priority. Internal kernel representation of the process priority scale and macros for mapping user-space to kernel-space priority values can be found in include/linux/sched/prio.h The reason we are using the specified priorities for your dummy scheduler is to avoid interference with system processes in other ranges and thus prevent them from being scheduled by the dummy scheduler on boot. Inside the task struct structure (that represents a process) there are several fields 2

representing priorities: int prio, static_prio, normal_prio; unsigned int rt_priority; static prio is the priority assigned by a user. Another field that will be of interest for this assignment is prio which is the processes dynamic priority. This priority is changed by the scheduler depending on the scheduling policy. normal prio is the same as the static priority for normal processes but for real time processes it is mapped to their value in the real-time range. Note. task struct is defined in include/linux/sched.h. 3.2 How to implement a new scheduler in Linux We have provided you with a patch file (dummy.patch) that will add a dummy scheduler skeleton into the Linux kernel source code. All files related to the kernel scheduler are located in the kernel/sched directory and the include/linux/sched.h header file (which gives an overview of the scheduling interface). The dummy patch adds a simple scheduler to the kernel by: 1. Creating a new scheduling class 2. Adding a runqueue for all processes that will be scheduled by the dummy runqueue. 3. Adding a scheduling entity corresponding to the dummy scheduler into the task struct. 4. Implementing all functions from the sched class. Even if they are left empty, they have to be defined as such. The rest of the document will give you a step by step overview of how these things were done and what is the role of each of these objects and functions we added. We start from the sched class struct defined in kernel/sched/sched.h. It is a struct that encapsulates a scheduler and the functionalities it should have. Each new scheduler is of type sched class and re-implementes some or all of the functions defined in sched class. The Linux kernel already has a fair(cfs) and a real-time(rt) scheduler whose implementations are in the kernel/sched/fair.c and kernel/sched/rt.c files respectively. To add a new scheduler we add a file called dummy.c. This is where you should implement most of your solution. Note that depending on how you choose to implement the required scheduling policy, some functions can remain unimplemented. Here we declare and define the struct dummy sched class of the type sched class. Our new struct needs to identify which functions in its implementation correspond to the default scheduling functions defined by default in sched class. Here is the declaration of the dummy scheduler: 3

const struct sched_class dummy_sched_class = {.next = &idle_sched_class,.enqueue_task = enqueue_task_dummy,.dequeue_task = dequeue_task_dummy,.yield_task = yield_task_dummy,.check_preempt_curr = check_preempt_curr_dummy,.pick_next_task.put_prev_task.set_curr_task.task_tick.switched_from.switched_to.prio_changed = pick_next_task_dummy, = put_prev_task_dummy, = set_curr_task_dummy, = task_tick_dummy, = switched_from_dummy, = switched_to_dummy, = prio_changed_dummy,.get_rr_interval = get_rr_interval_dummy,.update_curr = update_curr_dummy, }; Except for the first one, all members of this struct are function pointers which are used by the scheduler skeleton to call the corresponding policy implementation hook. All existing scheduling classes in the kernel are in a list which is ordered by the priority of the scheduling class. The first member in the struct called next is a pointer to the next scheduling class with a lower priority in that list. The list is used to prioritise tasks of different types before others. Currently the list of schedulers in the kernel is organized in the following way: rt sched class fair sched class idle sched class Since we want our dummy scheduler to have a lower priority than the real-time and CFS sheduler, we edit the next pointer of the CFS class so that it points to the dummy scheduler and the list now looks like this: rt sched class fair sched class dummy sched class idle sched class 3.2.1 The runqueue data structure This is one of the central data structures for scheduling. It holds the list of all processes runnable on a CPU and is defined in the kernel/sched/sched.h header file. Among others it contains the total number of running processes as well as pointers to runqueues of different schedulers. Hence we need to define a dummy runqueue and add a pointer to it in this datastructure: struct dummy_rq { struct list_head queue; }; struct rq { 4

... task_struct* curr; // pointer to the currently running task unsigned int nr_running; struct cfs_rq cfs; struct rt_rq rt;... struct dummy_rq dummy;... } If a scheduler implements priority scheduling, it has one runqueue per priority. The dummy scheduler skeleton implements a FCFS scheduling policy and hence (at the moment) has only one list. How would you change the dummy rq data structure to support the 5 priority levels? In order to initialize the runqueues, schedulers implement an initialization function called from the sched init function. The sched init function calls init functions of all schedulers on boot (which means your code might disable the machine from booting if you do something wrong here). Here is the initialization function for the dummy scheduler that you will find in the dummy.c file: void init_dummy_rq(struct dummy_rq *dummy_rq, struct rq *rq) { INIT_LIST_HEAD(&dummy_rq->queue); } 3.2.2 The scheduling entity A scheduling entity of a scheduler is an object that is on the runqueue of a particular scheduler - all tasks that belong to the runqueue of the dummy scheduler are instances of dummy scheduling entities. But where is this entity defined and where is it referenced from? We first define the following structure in include/linux/sched.h struct sched_dummy_entity { struct list_head run_list; }; This is our scheduling entity. Any task that is in the dummy runqueue will be of this type and contain the variables defined in this structure. If you feel like you need additional variables for your solution, feel free to add them. Currently the entity contains a pointer to its current runqueue. Fields of this structure are initialized in include/linux/init task.h. Feel free to look at the implementations of scheduling entities for the RT and CFS schedulers. A scheduling entity is added as a member to the task struct structure. 3.2.3 Assigning schedulers to processes The task struct structure contains a pointer to the scheduling class responsible for scheduling it : 5

struct task_struct {... const struct sched_class *sched_class; struct sched_entity se; struct sched_rt_entity rt; struct sched_dummy_entity dummy_se;... }; This field is set anytime a priority of a process is changed or set. You can change a processes priority by using the setpriority(2) system call or by running your program with the nice(1) utility. On system boot, all processes are assigned a scheduler within the sched init function: if (unlikely(current->prio >= MIN_DUMMY_PRIO && current->prio <= MAX_DUMMY_PRIO)) current->sched_class = &dummy_sched_class; else current->sched_class = &fair_sched_class; 4 What scheduling functions does a scheduler define 1. enqueue task: called when a task enters a runnable state. It puts the scheduling entity (task) into the run queue and increments the nr running variable; 2. dequeue task: when a task is no longer runnable, this function is called to keep the corresponding scheduling entity out of the run queue. It also decrements the nr running variable; 3. yield task: called when a task wants to voluntarily give up CPU, but not going out of runnable state. 4. check preempt curr: this function checks if a task that entered runnable state should preempt the currently running task 5. pick next task: this function chooses the most appropriate task eligible to run next according to the scheduling policy you want to implement. Note, that this is not the same as enqueuing and dequeuing tasks. It simply picks the next task, but the kernel has to schedule it. 6. set curr task: this function is called when a running task changes its scheduling class. 7. task tick: mostly called from time tick functions; it might lead to process switch. There is a timeslice parameter in the dummy scheduler code that defines how long a task can execute in a singe burst before being preempted. This parameter is in jiffies (a jiffy is one tick of the interrupt timer). The parameter can be set through the /proc filesystem at proc/sys/kernel/sched dummy timeslice. 6

8. switched to/from: if a task changes its scheduling class the kernel calls these functions so that the scheduler can do any bookkeeping if necessary. Most of these functions receive a pointer to the runqueue data structure from which they access the dummy runqueue. The entry point into the scheduler is the schedule() function defined in kernel/sched/core.c. Along with this file you can look for further useful functions within kernel/sched/sched.h file. Within these two files, you will find the core scheduling functionalities implemented. They are automatically called. For example on a timer event a default task tick function is called which automatically calls the task tick function of the scheduling class assigned to the currently running task. This is also why you can leave some of the functions unimplemented if they do not contribute to your solution. 5 Deliverables The assignment is due on April 14, 11:59pm. As for the previous assignment, you need to submit a ZIP file via VMChecker containing the following: 1. a.patch file that contains your solution 2. a README file in which you explain your solution. This file is obligatory and we will take away points for those who did not submit it. You are supposed to implement the features stated in the handout by using the template generated in class dummy.c. You may not need to implement all the functions and you may not need to use all the arguments of the function for your implementation. The template follows the default template of sched class and it has to provide all these functions that are mapped to it. And while the template provides function skeletons, you can see that some variables are defined in other places. It is up to you to determine which variables you will need and where you are supposed to define and declare them. 6 How to start the assignment? On Moodle you will find an archive containing all the resources that are mentioned below - Assignment2 resources.zip. 6.1 Change the config file of your kernel You need to update the.config file found in /usr/src/linux. You need to replace the.config in /usr/src/linux/ with the.config provided in the archive. In order for this to remain permanent in your system you need to do the following: 1. cd /usr/src/linux 2. git checkout 7b312549 3. git branch assignment2 7

4. cp /home/oslab/config.txt.config - Note. The.txt extension is left out on purpose. 5. git add.config 6. git commit 7. git log -n 1 8. get the first seven numbers from the commit part of the output 9. sudo edit the /usr/src/create patch.sh file and change the commit version at the top with what you wrote down Note. Step 2 will checkout the original version of the kernel (without the changes you made for the previous assignment). Therefore, if you need this code, create a snapshot or start from a clean VM before performing this step. 6.2 Apply the Dummy patch After this step you apply the provided dummy.patch as you would any other. Notice that, since this patch creates a new file (dummy.c), before generating your solution patch you need to run git add kernel/sched/dummy.c. Otherwise the script will not detect the new file and your submission will not contain this file. Note. A patch can be applied the following way: 1. cd /usr/src/linux 2. patch -p1 </home/oslab/dummy.patch. You can also try patch -p1 dry-run </home/oslab/dummy.patch. This will not apply the patch but will generate messages as if the patch was applied. 6.3 Recompile the kernel to test the dummy skeleton Type the following commands to recompile the kernel. 1. make clean 2. make -j2 bzimage 3. sudo cp /usr/src/linux/arch/x86/boot/bzimage /boot/vmlinuz-3.18.3+ 4. make modules 5. sudo make modules install 6. sudo update-initramfs -k 3.18.3+ -u After this you need to reboot your machine. Note. In case you do not see your shared folder after booting, run the following command: sudo vmware-config-tools.pl -d -m. VMWare tools have an issue if the kernel.config file changes and need to be reinstalled. 8

7 Tests We have provided 5 tests for you to apply to the solution. The tests are in Moodle. If you followed the previous steps correctly you should be able to run them on the dummy scheduler as is but the output will be incorrect. Once you implement your scheduler, their output should be roughly as the output described in the README files. Unzip the scheduler tests.zip file, enter the extracted directory and run make. You can then run the tests buy running the bash scripts provided as sudo. If you look at the tests you can see the system calls and commands used to change the priority of a process. 8 Recommendations 1. We highly recommend that you use the provided VM with the provided kernel version for this assignment. The scheduler patch might fail to apply otherwise. The reason is that the patch interferes with kernel functions whose signature and functionality might have changed between different kernel versions. Assignment 1 would not be impacted by this change but this assignment will be highly sensitive to the kernel version. Even if the provided patch applies successfully, you might have problems creating a solution that will work on the VM in VM Checker. 2. When printing debug messages inside scheduler code, do not use printk. Instead, you can use printk deferred. 3. Always make a snapshot before rebooting since your machine might not boot if there is a mistake in the code. If you forget but still want to get your code - boot by selecting the second option in the boot menu. 9