Comparison of soft real-time CPU scheduling in Linux kernel 2.6 series with Solaris 10

Transcription

1 Comparison of soft real-time CPU scheduling in Linux kernel 2.6 series with Solaris 10 Kristoffer Eriksson Philip Frising Department of Computer and Information Science Linköping University 1(15)

2 1. About the authors 1.1. Kristoffer Eriksson Kristoffer Eriksson is a student at Linköping University. He studies Computer Science and Engineering. Kristoffer can be reached via krier843@student.liu.se 1.2. Philip Frising Philip Frising is a student at Linköping University. He studies Computer Science and Engineering. Philip can be reached via phifr311@student.liu.se 2(15)

3 2. Abstract This report is a comparison between the soft real-time scheduling in Solaris 10 and the Linux kernel The report first present the systems and their schedulers, and then tries to compare them. In conclusion the both schedulers are very similar in many ways. The scope of this report is unfortunately not enough to make a realistic choice between them. However the report might still be interesting as an introduction to the internals of the two schedulers. 3(15)

4 Table of Contents 1. About the authors Kristoffer Eriksson Philip Frising Abstract Introduction Linux scheduler Priority levels Run queues Real-time scheduling policies SCHED_FIFO SCHED_RR Preemptive kernel System calls Solaris scheduler Priority levels Global priorities Class specific priorities Priority inversion Dispatch queue Real-time scheduling policies The Real-Time Class Preemptive kernel System calls Comparison of Linux and Solaris Priority levels Scheduling algorithms Real-time scheduling policies Preemptive kernel System calls Conclusion References Books Internet Source code (15)

5 3. Introduction The aim of this report is to compare soft real-time CPU scheduling in Linux version with Solaris version 10. The report will look at priority levels, run queue data structures, scheduling algorithms and policies, preemption and real-time related system calls, in Linux and Solaris A soft real-time system is supposed to make certain computations in a given time. However there is no guarantee that this actually will happen. Soft real-time systems are used in contexts where the realtime constraint need not be absolute. For example a soft real-time system might be used in a live audio-video system where a missed deadline may result in loss of quality but the system can still continue to operate. First soft-real time scheduling in Linux and Solaris is investigated in chapter 4 and 5 respectively. After that Linux and Solaris is compared in chapter 6. The report ends with a conclusion in chapter 7. 5(15)

6 4. Linux scheduler All information in this chapter is based on [Love] unless else stated. The Linux kernel is run on many different systems with handhelds, desktop computers and supercomputers as examples. On embedded systems responsiveness is more important than throughput, while on a supercomputer the situation is vice versa. This fact makes it particular hard to design the scheduler since it's supposed to work well on any type of targeting system. If the Linux scheduler would be improved for embedded systems then it would probably perform worse on a supercomputer. The 2.6 series of the Linux kernel introduced a completely new scheduler called Ο(1) scheduler. The Ο(1) scheduler is designed to work well many kinds of systems including real-time systems. The 2.6 series also introduced better real-time support with kernel preemption and improved synchronisation. Linux makes no difference between processes and thread when it comes to scheduling. Processes and threads are abstracted to tasks. Tasks can be created to share data like threads, to not share any data as processes or something in between. The scheduler schedules tasks and don't care about what type of task it is. The Linux kernel is improved rapidly. This report is based on the version of the Linux kernel. However, most of the report is valid for the whole 2.6 series 4.1. Priority levels Linux uses 140 priority levels that range from zero to 139. The highest priority that a task can have is 0 and the lowest is 139. The first 100 levels are used for real-time tasks, that is from zero to 99. The next 40 levels from 100 to 139 is used for standard tasks. The well known nice values maps on the 40 levels for standard tasks. Linux uses nice values from -20 to 19 as most other UNIX systems. A task with a high nice value say 19 is nicer to other tasks that have lower nice values Run queues Each CPU in a Linux 2.6 system has two priority arrays. At a given time one of the arrays is used to represent the active run queue while the other array represents the expired run queue. Two pointers are used to control which one of the arrays that are used as the active respectively the expired priority array. The priority arrays are data structures that contains three data types: the number of tasks active tasks [unsigned int] a bitmap [array of words to form at least 140 bits] an array of lists containing task [array of 140 lists] The first data structure is quite obvious, it stores the number of active tasks in the priority array. The second structure is much more exciting. The bitmap has one bit for each priority level. If a bit is on then there is a task in the corresponding list queue. The third data structure is an array of lists with runnable tasks. The array has 140 elements and each element maps onto the priority levels. So for example at index 115 there is a list with all tasks that have the priority 115. The scheduler will always choose the task with the highest priority in the active priority array first. That means that the scheduler will try to find the task with the lowest numerical priority value. 6(15)

7 So why is there a bitmap? Wouldn't it be enough to just look at the array of lists? Linux uses the bitmap to make the search for the next runnable task as fast as possible. Finding the first marked bit is a extremely fast operation and is supported by many architectures. Since the number of priority levels is fixed the search for the first bit is performed in constant time. That means that the search time can never exceed a certain level no matter how many tasks there are in the priority array. The best case is of course when the first bit for priority zero is marked and the worst case is when only the bit for priority 139 is marked. When the first marked bit is found the scheduler looks in the corresponding place in the array of lists with runnable tasks. The scheduler then knows the pointer to the list that contains the runnable tasks with the highest priority in the active priority array. The first task in the list is then scheduled for running. When a task has used up its time slice it will normally be inserted into the expired priority array. However real-time tasks and highly interactive tasks will be reinserted into the active priority array. When a task is reinserted in a priority it is added to the end of the list at the given priority level. When all tasks has used up their time slices and the active priority array is empty the active priority array is swapped with the expired array. This is simply done by swapping the pointers to the active and expired arrays. So after this is done the old expired priority array is the new active array and vice versa. The arrays can also be swapped without all tasks having used up their time slices. Tasks that sleeps are not in the priority arrays. Therefore the active priority can get empty without all tasks have used up their time slices. So the priority arrays are swapped when the active array is empty Real-time scheduling policies As already stated the first 100 priority levels are reserved for real-time tasks. These tasks will always be chosen before any other task. In fact the the real-time tasks will also preempt any other task with a lower priority. Linux offers four different scheduling policies. Two of those are for real-time tasks and are called SCHED_FIFO and SCHED_RR. The third policy is called SCHED_NORMAL and is only used for non real-time tasks. There is also a fourth policy called SCHED_BATCH for batch tasks. Let's look into the real-time scheduling policies SCHED_FIFO and SCHED_RR. [sched.h] SCHED_FIFO SCHED_FIFO is the simplest algorithm. For a given list of runnable task it simply starts by scheduling the first task and waits for it to finish or sleep. Then it schedules the second one, waits for it to finish or sleep, and so on. A task has no time slice and will run either until it finishes, until it sleeps, or until it's preempted by task with higher priority. A SCHED_FIFO task will preempt any other task that has a lower priority and will run as long as it wish or until it gets preempted SCHED_RR SCHED_RR is very similar to the SCHED_FIFO algorithm. It simply just uses time slices as well. All runnable tasks on the same priority levels are scheduled as the SCHED_FIFO tasks, but will only run until they have consumed their time slice or until they are preempted by a higher priority task. Tasks at the same priority level are scheduled round-robin. So when all tasks have consumed their time slices they will get new once and start all over if they are runnable. That means that a task with a lower priority can never be run until a SCHED_RR task has either finished or yield. This ensures that real-time tasks will always run before any other tasks. 7(15)

8 The time slices for tasks that is scheduled with the SCHED_RR policy is based on the priority of the task. Higher priority means higher time slice. [sched.c] 4.4. Preemptive kernel Much work during the development of the 2.6-series kernel has been focused on making the Linux kernel preemptive when running in kernel-space. The Linux kernel has as most modern operative systems for long been able to preempt tasks in user-space. It's not possible to preempt the kernel in any time. Some parts of the kernel code has to run in one shot. That is when the kernel is holding some kind of lock. A kernel being preemptive is an important feature for real-time systems. The big advantage with a preemptive kernel is of course less latency. The kernel can preempt a normal task even in kernel mode in favour of a real-time task. So when a real-time task get runnable the scheduler can preempt another task with lower priority that is executing kernel code System calls Linux developers use the shorter name syscall instead of system call. In table 1 all real-time related scheduling syscalls are shown and explained. Syscall Action sched_setscheduler() sched_getscheduler() sched_setparam() sched_getparam() sched_get_priority_max() sched_get_priority_min() sched_rr_get_interval() sched_yield() Table 1: Real-time related scheduling syscalls This syscall sets a tasks scheduling policy. For real-time tasks SCHED_FIFO and SCHED_RR are available. SCHED_NORMAL or SCHED_BATCH is used for any other task. Use this syscall to check a tasks scheduling policy. This syscall is used to set a tasks real-time priority. The possible priority range is zero to 99. Get the real-time priority for a task. These syscalls are used to get the min and max priority for a real-time task. The default min is zero and the default max is 99. However these settings can easily be modified in the schedulers header file. This syscall is used to get the time slice of a task. If the scheduling policy is SCHED_FIFO then the return value will be zero since that represent a infinite long time slice. For all tasks with other policies the return value will be greater than zero. Use this syscall to temporarily yield the CPU. Information sources for syscalls are both [Love] and [sched.h] 8(15)

9 5. Solaris scheduler All information in this chapter is based on [Sun] unless else stated. SunOS 5.10, or Solaris 10 as it s brand name is, is a soft real-time operating system released by the Sun corporation. Most of the then current source code was released in 2005 under an open source licence under the community project opensolaris.org and the source and binaries can be downloaded freely from their site, Sun has also said that future releases of SunOS will be derived from the progress made by opensolaris. This report however will try to focus on the scheduler that is used by the operating system and in particular the real-time support given by that scheduler. SunOS has supported real-time processes since its release and has a fully preemptive core. The scheduler works not on the processes themselves but rather the threads which are contained in LWP:s, light weight processes. From now on when the word process is used associated with the Solaris kernel it refers to a LWP unless otherwise specified Priority levels Global priorities The system has got global priorities ranging from zero to 169 in which certain ranges are allocated for special process types. The SunOS kernel ranges the high level properties to be most critical Hardware interrupts Real-time processes 0 99 Other time-share and system processes The highest priorities are given to hardware interrupts which cannot be controlled by software. The other priorities are given to user processes and the highest are given to real-time processes. As long as there is a real-time process ready to run the other user processes will not be allowed to run, unless some priority inversion is active (see section about priority inversion). The other 100 process levels are given to normal time-sharing and system processes Class specific priorities Apart from the global priorities each process is given by the user not a direct mapping to the global priorities but a class specific value which is then mapped to a global priority. The time-share processes are given values in the range -20 to 20 which correspond to the global priorities zero to 40, with temporary assignments as high as 99. The RT-processes are given priorities ranging from zero to 59 which are mapped to 100 to 159 in the global priorities list. The kernel s class independent code runs the process with the highest global priority on the queue. 9(15)

10 Priority inversion In some situation a high priority process might be forced to wait for a lower priority process in order to continue execution. This situation must be resolved by the scheduler to prevent deadlocks, this is referred to as priority inversion. In the SunOS kernel a process that is blocking another process with higher priority will inherit that process priority while the block is active Dispatch queue The SunOS kernel keeps a dispatch queue for each processor which contains the processes to be dispatched to that processor. When a new process is allocated to a process, from a state of sleep or if it was just created, the process is inserted into the dispatch queue at a point depending on its priority. If a process is placed first in the queue and it has a higher priority than the current running process the current process will be preempted for the new process Real-time scheduling policies The scheduler sorts process into one of six priority classes, which decides with what policy that process, or thread, will be scheduled. The six classes are: the real-time class the system class the interactive class (IA) the fixed-priority class (FX) the fair-share class (FSS) the time-sharing class (TS) However this report will only focus on the real-time class and it s scheduling policies The Real-Time Class The real-time class handles the processes that has been classified as real-time and will use a policy that tries to uphold the constraints usually associated with real-time processes. Each process is scheduled according to a static priority which each process is given at initiation by the user and which it also passes on to any eventual children. The scheduler will always give the runnable real-time process with the highest priority control of the CPU (or a CPU in case of a multi-core system). The real-time scheduler can be set to operate with a FIFO-strategy or a Round-Robin strategy depending on the users needs. The FIFO-strategy lets the process with highest priority run until it s finished or if it yields itself, it will however still preempt the current process if a higher priority becomes runnable. The round-robin strategy uses a process specific time-quantum to decide how long a process can run, and it will run until that time-quantum runs out or if it yields or finishes. But just as in the FIFO-strategy it will still preempt the current process if a higher priority process becomes runnable. The time-quantum is dependent on the priority of the process. The higher priority the smaller time-quantum. 10(15)

11 5.4. Preemptive kernel In order to guarantee response time the Solaris kernel will preempt any process if a higher priority real-time process becomes dispatchable. This means the current process will be immediately switched out through a context switch for the new process. This is a basic feature required to support a realtime system System calls In order to control the processes the user uses a number of system calls to get info and set parameters associated with the scheduler and the processes. priocntl() Syscall sched_setparam() sched_getparam() sched_get_priority_max() sched_get_priority_min() sched_rr_get_interval() sched_yield() Table 2: Real-time related system calls Action The syscall that controls scheduling parameters of processes and classes. Sets the parameters of a given process. Gets the parameters of a given process. Returns the maximum value for the specified policy. Returns the minimum value for the specified policy. Takes a process id and an interval and sets the time-quantum of the specified process to that interval. Forces the running thread to relinquish the processor until the process again becomes scheduled. 11(15)

12 6. Comparison of Linux and Solaris Linux and Solaris are very similar in many ways as they both implement standards that are common to UNIX systems. There are still some slight differences between the systems that will be presented in this chapter Priority levels The main difference in priorities in the two systems is the fact that Solaris considers gives the high priority processes a high numerical priority value while the Linux give tasks with a high priority a low numerical priority value. Linux also allocates 100 priority levels for real-time tasks while Solaris chooses to allocate 60. This means that Linux give the user more power to differentiate between real-time processes than Solaris. However the priority levels is purely a design decision and doesn t really affect the systems performance unless the user has more real-time processes than the OS allocates levels for. Although most of the priority levels can be configured in both the systems Scheduling algorithms Somehow the critical criteria for a real-time system is how fast it responds when a real-time process wants the CPU and therefore it might be interesting to compare the performance of the scheduling algorithms. The Linux scheduler uses a simple and effective algorithm to obtain a O(1) complexity. Unfortunately there isn t very much reliable information available free either on the Internet in it s whole or in through official SunOS channels on the internals of the scheduler algorithm. From what we gathered it uses a priority queue and it presumably runs with the same complexity as the Linux kernel. Given this we can at least say we are certain that both the schedulers are suited for running real-time applications in a useful way. The complexity of a scheduling algorithms doesn't necessarily define how well a scheduler performs in a real-time system. For example if a system has no more than five processes running a O(n) algorithm may be faster than a O(1) since n is small. So, it isn't really the complexity that determines the speed of the scheduler when there is a small number of processes running. However if a system has many processes running which of only few are real-time the complexity of the scheduling algorithm will highly influence the speed of the scheduler. To really compare the scheduling algorithms it would be necessary to perform extensive testing on both system running on the same hardware. Unfortunately that is beyond the scope of this investigation Real-time scheduling policies The scheduling policies in Linux and Solaris are very similar. They both implement the POSIX standard real-time process scheduling policies FIFO and round-robin. However when using the roundrobin policy Linux gives largest time slice to the process with the highest priority while Solaris give the largest time slice to the process with the lowest priority. 12(15)

13 6.4. Preemptive kernel Both Linux and Solaris are preemptable in kernel space. In Linux there is some code that can't be preempted in kernel space. However, this report has not manages to present whether Linux or Solaris is the most preemptable in kernel space System calls Linux and Solaris supports almost the same system calls, again because they implement common standards. This makes it easier to port applications between the two systems. 13(15)

14 7. Conclusion In conclusion the both systems are very similar and features the same functionality when it comes to scheduling of real-time processes. So trying to choose between them based on the results of this report is difficult and we would recommend that one would investigate other aspects of the two systems before making a decision or testing the schedulers in real life. 14(15)

15 8. References 8.1. Books [Love] Love, Robert (2005). Linux Kernel Development. Second edition. Indianapolis, Indiana, USA: Novell press Internet [Sun] Programming Interfaces Guide [www] < Retrieved 16 th November Source code [sched.h] linux /include/linux/sched.h. [sched.c] linux /kernel/sched.c 15(15)