The modularity requirement The obvious complexity of an OS and the inherent difficulty of its design lead to quite a few problems: an OS is often not completed on time; It often comes with quite a few bugs; Performance is often not as good as expected. An OS has to be modular so that it is easier to implement and the scope of error checking and correction will be limited. The modules have to have clearly defined, and simple, interfaces to each other. This will not only ease the task of a programming team, but also limit the impact to other modules when one of them has to be changed. 1
System structures For a large system such as OS, it is also necessary to put these modules into a clearly defined hierarchical structure to separate various functions by their complexity, the respective time scale, and their level of abstraction. When put into such a hierarchy, each level takes care of a bunch of functions required of the OS at that level, and relies on the previous level to provide necessary services, without having to know the implementation details. At the same time, this level also provides services to the higher levels, using the functions defined within. The whole idea is that the changes made at one level does not lead to changes at other levels. This way, we can decompose a complicated issue to a collection of more manageable ones. 2
Different time scales Generally speaking, lower levels deal with a much shorter time scale. For example, those functions dealing with hardware parts have to work very briefly. At the other end of the spectrum, it takes a while for the OS to communicate with the users. Taking into account the time it takes to go through the levels, this hierarchical structure fits very well into the reality. Let s consider an example of such a hierarchical structure, similar to the structure of a network system. 3
One such structure Level 1 deals with the hardware parts, such as registers, memory cells, and logic gates. The functions provided at this level include clearing registers or reading memory cells. Level 2 provides the instruction set, such as add, subtract, branch, load, store, etc.. Level 3 adds in the concept of a procedure or a subroutine, plus the call/return operations. Level 4 introduces interrupts, which cause the processor to save the current context and calls an interrupt-handling function, a procedure. Level 5 introduces the concept of process, and the ability to suspend and resume processes. Moreover, in the case of multiple programming, some mechanism of process synchronization has to be added. 4
Level 6 deals with the secondary storage devices, and has to provide such functions as positioning the r/w heads, and to support the actual transfer of data blocks. This level relies on level 5 to schedule operations and to inform a requesting process of completion of an operation, and provides related services to higher levels. Level 7 creates a logical address space for processes. Particularly, it organizes the virtual address space into blocks that can be transferred between main memory and secondary memory. When a piece of data can t be found in the space, it calls Level 6 services to load the page. 5
Level 8 deals with the communication of information and messages between processes. This level provides a powerful tool, namely, pipe, a logical channel for the flow of data between processes. It can also be used to link processes and external devices. Level 9 supports the long-term storage of named files, where files are viewed as variable-length entities, instead of tracks, sectors, and fixedsize blocks. Level 10 provides access to external devices using standard interfaces. 6
Level 11 maintains a mapping between the external and internal identifiers of various resources and objects. The external id is a name an application can use, such as LPT1; and the internal one is a memory location that can be used by lower levels to locate and/or control such resources(objects). Level 12 provides the full support to processes so that process management is possible. This includes the virtual address space for the processes, a list of processes and other objects a process might have to interact, together with the restriction of such interaction, also the parameters that have to be passed in when the process is created. Level 13 provides an interface to an OS for the users. It is often referred to as a shell, since it separates the services an OS provides from the associated technical details. 7
Microkernel architecture Earlier OS features a single large kernel. Most of the functionality is provided in this large kernel, including scheduling, file systems, networking, device drivers, memory management, etc.. Typically, such a kernel is implemented as a single process, with all elements share the same address space. In contrast, a microkernel architecture assigns only those essentials, such as space addressing, interprocess communication, and basic scheduling, into a much smaller kernel; and provide all the other services via various processes that run in users mode. Such an approach takes less space, simplifies implementation, and fits a distributed environment really well. 8
Multi threading In such a system, a process, when executing an application, is divided into threads that can run concurrently. A thread runs sequentially, and is interruptible so that threads of the other processes can also be run. In contrast, a process is a collection of one or more threads and associated resources, and is more closely associated with a program in execution. By breaking a process into a bunch of concurrently running threads, a programmer has better control over the modularity of the application, and the associated timing. Also, the efficiency will go up as well. The key here is how to break a process into a bunch of independent units of work that can be independently processed. 9
Symmetric multiprocessing A symmetric multiprocessor is a stand-alone computer in which there are multiple symmetric processors, sharing the same main memory and I/O facilities. Those processors can perform same kinds of functions, and can be interconnected by a bus, or other connection schema. The following chart shows the advantage of following such an approach. 10
Why SMP? Besides increased performance, an SMP also provides other potential benefits. For example, since all the involved processors have the same ability, even if one or few processors fail, the whole system will still function, at reduced performance. Moreover, a user can enhance the system performance by adding additional processors, when there is such a need. Finally, vendors can offer a collection of products with different price range and performance indicator based on the number of processors in such systems. 11
Window overview Windows, starting with Windows 2000, continuing with Vista, and going to Windows 7, is a significant example of a new wave in the new generation OS for microcomputers, in the sense that it is still intended for a single user, but it supports multitasking, in terms of 70+ processes, and hundreds of threads. The services and resources available to users become more powerful and varied, particularly with the increase of speed and memory capacity, support for virtual memory, applications have become more complex and related. A document may be generated with several applications working together, WP, CG, Spreadsheet, calculator, etc.. 12
Another motivation for the multitasking development is due to the growth of client/server computing, where a client system, coupled with a server system, work together, in doing, e.g., database applications. To provide the needed real-time responsiveness, the OS needs to support sophisticated real-time communication activities, while supporting an ongoing user session. Besides supporting multitasking, the server version of Windows XP certainly also supports the multiuser environment. Homework: Read through the History of Window systems, page 80 through 82. 13
Window 2K architecture W2K consists of application-oriented software and operating-system software. The latter, running in the kernel mode, includes the Executive, the microkernel, various device drivers, and the hardware abstraction layer. They have access to system data and to the hardware. In contrast, the application oriented software does not have such access. Among others, HAL, the hardware abstraction layer, provides a mapping between generic hardware commands and responses and those specific to a unique platform, thus isolating the OS from specific hardware pieces, and making them all look the same to the kernel. Homework: Check through the Windows architecture as demonstrated in Figure 2.13. 14
User processes Windows provides four basic types of user processes: Special system support processes: This includes services not provided by the W2K, such as the login process, and the session manager. Server processes: This refers to other W2K services such as the event logger, which collects various records of all kinds of events. Environment subsystems: Provide other environment, such as OS/2, Win32, etc., that convert the user application calls to W2K calls, to maintain a compatibility. User applications: With the support by the environment systems, W2K supports user applications in one of five types: Win32, Posix, OS/2, Windows 3.1, and MS-DOS. 15
Client/Server model Many parts, particularly, the Executive, the protected subsystems, and the applications, are structured following the client/server model. Each environment subsystem and executive service subsystem is implemented as one or more processes. Each process waits for a request from one client, e.g., a user program, or other process, for its services such as memory management, process scheduling, etc., via a message, which is routed by the Executive to a proper server. The server returns the result to the requester via another message, which is routed by the Executive back to the client. 16
Advantages It simplifies the construction of the Executive, and make it more flexible, and less redundant. It improves reliability, since each executive service process runs with its own virtual space, thus avoiding interference from other processes. Also, a user process is not allowed to get to the hardware or the data space used by an executive process. It provides a uniform means for applications to communicate with the Executive. More specifically, the message passing process is hidden from the calling process via private data holders. Finally, it provides a suitable base for distributed computing. 17
Window objects Windows also depend heavily on the objectoriented design, which focus on the sharing of resources and data among processes and the protection of resources from unauthorized access. Below are the familiar concepts. 1. Encapsulation: An object consists of data and one or more methods that we can use to work with those data. 2. Object class and instances: An object class is a template of data that shares the same attributes and methods. 18
3. Inheritance: This feature is not supported at the user level but is somewhat supported at the Executive level. Thus, for example, all the files as contained in a folder inherits its access rights, except specified otherwise. 4. Polymorphism: Windows uses a bunch of API to work with objects of any type. But, we do notice that Windows have many type specific APIs. Windows is not a complete object-oriented system, but it does demonstrate the power of such an approach. 19
UNIX description The structure of the traditional UNIX system, given as follows, is pretty straightforward: Hardware is wrapped around by a kernel, which supports the system call via an interface. Sandwiched in between is basically a collection of UNIX commands and libraries, organized as a shell, and components of the C compiler. The out-most layer consists of the user applications and the user interface to the C compiler 20
A closer look of the kernel User programs can invoke OS services either directly or through library programs. The lower system call interface separates the user and allows high-level software to get access to specific kernel functions. At the bottom, the kernel contains primitive routines that deal directly with the hardware layer. Between these two ends, the kernel is divided into two parts: one dealing with the process control; and the other dealing with file management and I/O service. The process part takes care of memory management, the scheduling and dispatching of process, the synchronization and interprocess communication. The file system exchanges data between places, using a variety of device drivers, sharing the same user interface. 21
A picture 22
Modern UNIX system As there is a need to produce a new implementation that incorporates many of the important innovations, attractive design features of other systems, and produces a more modular system, a variety of UNIX implementation have come up, more or less sharing the following architecture. 23
SVR4 and company Some of the new features included in System V Release 4 (SVR4) include real-time processing, needed for critical system control; process scheduling classes, dynamically allocated data structures such as linked lists, virtual memory management, virtual file system, and a preemptive kernel which can take back already committed resources. It is running on many platforms, ranging from 32 bit machine to super computers. 24
Other stuff Solaris is Sun s SVR4-based UNIX system. It further adds on some more advanced features such as a fully preemptive, multi threaded kernel, full SMP support and an object-oriented interface to file systems. BSD has played a key role in developing OS design theory, with 4.4 being its final version. 25
Linux: a modern UNIX system Most UNIX kernels are large body of codes. Any change made to any portion of such a system will have a chain reaction. As a result, any modification of such a system, e.g., adding a new device driver, is difficult: the whole system has to reconnect, reinstall, and then reboot. To address this problem, Linux is organized as a collection of relatively independent loadable modules, which provides dynamic linkage, and stackable modules. 26
Why the two features? With dynamic linkage, a kernel module can be loaded, either by a user program or the kernel itself, and linked into the kernel while the kernel is already in execution, and can be removed at any time without affecting the kernel operation. Stackable modules can be organized into a hierarchy, thus dependency can be defined on them. Particularly, code common to a set of similar modules can be pulled out and moved into a single module. Also, this way, the kernel can use a prerequisite relation to decide if the needed modules are present, and load in those that are not. 27
An example Below shows the structure Linux uses to manage modules. Each module is defined by two tables, the module table and the symbol table. Particularly, the next pointer indicates the next module in the dependency hierarchy, and ref tells what modules depend on it. Below shows that VFAT is loaded after FAT, thus depending on the latter. 28
Homework Check out the Kernel Components subsection beginning at pp. 98 to have at least some ideas as the general flow of in between processes and the hardware at the bottom layer. 29