Introduction to Operating Systems Part I

Transcription

1 TDDE25 Introduction to Operating Systems Part I Christoph Kessler IDA, Linköping University Copyright Notice: The slides partly use material from Silberschatz s, Galvin s and Gagne s book ( Operating System Concepts, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. Christoph Kessler, IDA, Linköpings universitet. Why should I be interested in Operating Systems? Mainframe OS/360 IBM 360 Midframe TOPS-10 DEC-20 MiniComputers Unix Workstations MS-DOS Windows Linux MacOSX IBM PC? Sun Workstations 2 1

2 A Whole New World! Smart Phones Wearable Devices Portability Connectivity Networks Sensors 3 A Whole New Future World! The current 7- Series BMW and S-class Mercedes boast about 100 processors a piece. A relatively lowprofile Volvo still has 50 to 60 baby processors on board. Smart Vehicles Smart Highways Smart Cities Smart Homes / Smart El-Grids 4 2

3 Increase in Hardware Complexity Moore s Law (prediction 1965/75): #Transistors / mm 2 doubles every ~2 years. Holds until ~2015 and, with somewhat lower exponential slope, still today. Source: Intel Need for new types of interfacing to hardware Multiprocessor servers Hardware multithreading Multicore CPUs Many-Core CPUs Accelerators e.g. GPGPU Hybrid computing Clusters Cloud servers Nvidia Tesla GPU NSC s cluster supercomputer Triolith (TDDC78) 5 Increase in Software Complexity Need for new types of OS Architectures MacOS X 10.4: 86M LOC (2005) Linux kernel 2001 Linux kernel

4 Agenda Lecture I: Computer Systems Overview Building and Executing Programs Operating System Basics Lecture II: Interrupts and I/O System Calls CPU Management Memory Management File System Protection and Security 7 Computer Systems Overview Copyright Notice: The slides partly use material from Silberschatz s, Galvin s and Gagne s book ( Operating System Concepts, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. Christoph Kessler, IDA, Linköpings universitet. 4

5 Computer Systems and Environments Stand-alone desktop computer Client-server systems Parallel systems Clustered systems Distributed systems Real-time systems Handheld systems 9 Stand-alone desktop computer PC, workstation: Computer system dedicated mainly to a single user. I/O devices: keyboards, mice, screens, small printers. Requirement: User convenience and responsiveness. Can adopt OS technology developed for larger computer systems May run several different types of operating systems (Windows, MacOS, UNIX, Linux) 10 5

6 Client-Server Computing Servers respond to requests by clients Remote procedure call also across machine boundaries via network Client/Server are mainly software roles, but machines can be dedicated to one or few such roles, e.g. Compute server: compute an action requested by the client File server: Interface to file system (read, create, update, delete) 11 Parallel Systems (1) Multiprocessor systems with more than one CPU in close communication. Today the default, even for desktop machines (multi-core) Tightly coupled system (aka. shared-memory system, multiprocessor) processors share memory and a clock; communication usually takes place through the shared memory. Advantages of parallel systems: Increased throughput Economical Scalability of performance Multiprocessor system vs multiple single-processor system (reduction of hardware such as disks, controllers etc) Increased reliability graceful degradation (fault tolerance, ) fail-soft systems (replication, ) 12 6

7 Parallel Systems (2) Processor Processor Processor Symmetric multiprocessing (SMP) One OS controls all processors and the shared memory Many processes can run at once without performance deterioration. Most modern operating systems support SMP Asymmetric multiprocessing Each processor is assigned a specific task; a master processor schedules and allocates work to slave processors. More common in special-purpose systems (e.g., embedded MP-SoC, very common in mobile/smartphones) Remark: the notion of processor is relative: A traditional PC usually has one CPU, but it usually also has a graphics processor, a communication processor etc., and this was not considered a multi-processing system. Today, though, almost all computers have multicore CPUs that could be used for parallel computing, and can thus be considered multiprocessors. 13 SMP architecture Hardware Multithreading, Multi-Core T0 T1 T2 T3 P0 P1 L1$ D1$ L1$ D1$ L2$ Memory Ctrl 2 hardware threads per core 2 cores per CPU chip Appears to the OS like 4 standard processors Main memory Intel Xeon Dualcore(2005) More in TDDD56 Multicore and GPU Programming 14 7

8 Parallel Systems (3) Multicomputer ( distributed memory parallel computer ) Several stand-alone servers, each one a SMP, loosely coupled by an interconnection network for internal data exchange More scalable than shared-memory SMPs But more cumbersome to program (message passing) NSC Tetralith, LiU, 2018 More in TDDC78 Programming parallel computers 15 Parallel Systems (4) Speed-up of a single application by parallel processing? Requires parallelisation / restructuring of the program using special parallel programming languages / systems SMP: Multithreading TDDB68, TDDC78 HPC clusters: MPI (Message-Passing Interface) TDDC78 Data centers: Mapreduce / Hadoop 732A54/TDDE31 Or new, explicitly parallel algorithms First used in High-Performance Computing for numerically intensive applications (weather forecast, simulations,...) Now ubiquitous problem due to switch to multicore/manycore CPUs 16 8

9 Parallel Computing Systems 17 Clustered Systems Like multiprocessor systems, but multiple systems (servers) working together Often using a high-speed interconnect between servers (e.g. Infiniband) Usually sharing external mass storage via a storage-area network (SAN) NSC Tetralith, LiU, 2018 Some clusters are used for high-performance computing (HPC) For data centers and critical applications: Provides a high-availability/reliability service which survives failures Asymmetric clustering - one server runs the application while other servers are in hot-standby mode Symmetric clustering - has multiple nodes running the same application, monitoring each other 18 9

10 Distributed Systems (1) Loosely coupled system Networked stand-alone computers (nodes) each node has its own local memory Nodes communicate with one another by message passing through various communications lines, such as high-speed buses, telephone lines or wireless connections (LAN, WAN, MAN, Bluetooth, ) Clustered systems are a special case, with stronger network Distribute the computation among several physical computers Advantages of distributed systems: Resource sharing Computation speed up Adaptivity: load sharing (migration of jobs) Fault tolerance May be either client-server or peer-to-peer systems 19 Distributed Systems (2) OS support for distributed systems Network Operating System provides file sharing, e.g., NFS - Network File System provides communication scheme runs independently from other computers on the network Distributed Operating System less autonomy between computers gives the impression that there is a single operating system controlling the network. More about this in TDDD25 Distributed Systems 20 10

11 Distributed Computing Systems 21 Real-Time Systems Often used as a control device in a dedicated application/environment, such as controlling scientific experiments, medical devices, avionics / automotive control, industrial control systems, and some display systems. Well-defined tasks with fixed time constraints. Hard real-time systems. Deadlines must be met under all circumstances Conflicts with time-sharing systems, not supported by general-purpose OSs. Soft real-time systems Missed deadline degraded quality of service Limited utility in industrial control or robotics Useful in applications (multimedia, virtual reality) requiring advanced OS features. More in TDDD07 Real-time Systems 22 11

12 Handheld Systems Personal Digital Assistants (PDAs), ipads etc. Cellular telephones, smartphones Issues: Limited memory Slow processors Small display screens Limited battery lifetime Iphone 8 Shipping with 2 GB RAM, 64GB Flash memory 6 core CPU + embedded GPU +... up to 2.39GHz 4.7 (but 1334 x 750 pixels) 23 Before talking about Operating Systems Copyright Notice: The slides partly use material from Silberschatz s, Galvin s and Gagne s book ( Operating System Concepts, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. Christoph Kessler, IDA, Linköpings universitet. 12

13 Before talking about OS, a short note on programming languages suitable for system programming Early operating systems were implemented in assembly language (and some low-level parts of modern OS still are). Necessary for direct access to hardware devices Since the 1970s the dominating language for system programming is C. Also used a lot in high-performance and embedded computing Today, no new microprocessor ships without a C compiler. Many high-level programming languages are compiled to C. You will need C for programming labs in later courses Operating systems, Compiler construction, Parallel programming, Multicore/GPU programming, Realtime systems, Embedded systems, 25 Java vs. C Java C For application programming only Design goals: Programmer productivity Safety Hardware completely hidden Comfortable E.g. automatic memory management Protection (to some degree) Slow E.g., array bound checking Time-unpredictable For system programming mainly Design goals: Direct control of hardware High performance / real-time Minimalistic design Less comfortable Little protection low-level 26 13

14 A Short History of C C was developed in the early 1970 s by Dennis Ritchie at Bell Labs Objective: structured but flexible programming language e.g. for writing device drivers or operating systems Used for implementing the Unix OS Book 1978 by Brian Kernighan and Dennis Ritchie ( K&R-C ) ANSI-C 1989 standard by ANSI ( C89 ) The C standard for many programmers (and compilers...) Became the basis for standard C++ Java borrowed much of its syntax The GNU C compiler (gcc) implemented a superset ( GNU-C ) C99 standard by ISO, only minor changes C11 (ISO) multithreading support added 27 How to build and execute programs on a real computer Copyright Notice: The slides partly use material from Silberschatz s, Galvin s and Gagne s book ( Operating System Concepts, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. Christoph Kessler, IDA, Linköpings universitet. 14

15 The Compilation Workflow /* file hello.c */ #include <stdio.h> int main() { printf("hello, world\n"); }.file "hello.c".section.rodata.str1.1,"ams",@progbits,1.lc0:.string "hello, world".text.p2align 4,,15.globl main.type main: pushl %ebp movl %esp, %ebp subl $8, %esp andl $-16, %esp subl $16, %esp movl $.LC0, (%esp) call puts leave xorl %eax, %eax ret.size main,.-main 29 Preprocessing Phase C Preprocessor (cpp): modifies the original C source program according to directives that begin with the # character. #include <stdio.h> tells the processor to read the contents of the system header file stdio.h and insert it directly into the program text. Output is another C source program, typically with the.i suffix

16 Compilation Phase Compilation Phase (cc1): Translate the text file hello.i into the text file hello.s. hello.s contains an assembly-language program Each statement in an assembly language program exactly describes one low-level machine language instruction in a standard text form. C compiler Assembly language Fortran compiler Assembly language provides a common output language for different compilers main: pushl %ebp movl %esp, %ebp subl $8, %esp andl $-16, %esp subl $16, %esp movl $.LC0, (%esp) call puts leave xorl %eax, %eax ret 31 hello.s (in x86 assembler language).file "hello.c".section.rodata.str1.1,"ams",@progbits,1.lc0:.string "hello, world".text.p2align 4,,15.globl main.type main: pushl %ebp movl %esp, %ebp subl $8, %esp andl $-16, %esp subl $16, %esp movl $.LC0, (%esp) call puts leave xorl%eax, %eax ret.size main,.-main.section.note.gnu-stack,"",@progbits.ident "GCC: (GNU) (Red Hat fc3)" 32 16

17 Assembly Phase Assembler (as): translates the text-based assembly program into machine language instructions, packages them in a form known as a relocatable object program, and stores the result in the object file hello.o hello.o is a binary file (object file) whose bytes encode machine instructions and -data rather than characters. Binary files use a system-specific binary file format, e.g. ELF, COFF. a4 f0 dd 07 7c 65 a6 b2 06 0f c3 dd a2 ff bd Linking Phase Linker (ld): merges pre-compiled object files to a single one. The result is an executable object file that is ready to be loaded into memory (using the OS loader) and executed by the system. The hello program uses the printf function, which is part of the standard C library. This function resides in a separate precompiled object file (e.g. printf.o or in libc.a) that has to be merged with hello.o. Also some additional code and data (program startup code, C runtime system, etc., in libc.a) is added by the linker

18 Calling the entire toolchain E.g., gcc hello.c o hello (here, for the GNU C compiler gcc) calls cpp, cc1, as, ld for single-module program hello.c For automatizing the build process of multi-module programs, building-tools like make or IDEs like ECLIPSE are convenient. 35 Compiling and Linking for Multi-Module C Programs xy.h glob.h stdio.h #include #include #include abc.c mymain.c def.c preprocess compile + asm cc abc.c cc mymain.c cc def.c abc.o mymain.o def.o link libc.a a.out (executable) C run-time library is linked with the user code 36 18

19 Running an Executable Program /* hello.c */ #include <stdio.h> int main() { printf("hello, world\n"); } hello a4 f0 dd 07 7c 65 a6 b2 06 0f c3 dd a2 ff bd 87 Preprocess Compile Assemble Link Executable Object File (binary) In a Unix system, a shell is a program which is a command line interpreter The shell prints a prompt, waits for the user to type in a command and then performs the command. unix>./hello hello, world unix> Input Output If it is not a built-in shell command, then the shell assumes it is an executable file and that it should load and run it. 37 Running the hello program shell As the characters./hello are typed at the keyboard, the shell program reads each one into a register and then it is stored in memory Reading the hello command from the keyboard 38 19

20 Running the hello program Loading the executable from disk into main memory a4 f0 dd 07 7c 65 a6 b2 06 0f c3 dd a2 ff bd 87 a4 f0 dd 07 7c 65 a6 b2 06 0f c3 dd a2 ff bd 87 When the enter key is hit on the keyboard, the shell knows that we have finished typing the command. The shell then calls the OS to allocate memory and load the executable file. The loader copies the code and data in the hello object file from disk to main memory. Using a technique called Direct Memory Access (DMA) the data travels directly from the disk to the main memory without passing through the processor. 39 Running the hello program Writing the output string from memory to the display Once the code and data in the hello object file are loaded into memory, the processor begins executing the machine instructions in the hello program s main routine main: pushl %ebp movl %esp, %ebp subl $8, %esp andl $-16, %esp subl $16, %esp movl $.LC0, (%esp) call puts leave xorl %eax, %eax ret These instructions copy the bytes in the hello, world string from memory to the register file and from there to the display device where they are displayed on the screen 40 20

21 Some questions How can we achieve that the shell remains active while hello is running? e.g. to accept input from the keyboard for aborting the started program, or for starting further programs How do we make sure to re-gain control over the computer e.g. if a started program hangs in an endless loop? How can multiple programs (or multiple copies of the same program) run concurrently (execution overlap in time) even if there is only one processor? How can such processes communicate with each other? How do we make sure that processes use system resources fairly? This (and more) is solved by using an operating system. 41 OS Basics Copyright Notice: The slides partly use material from Silberschatz s, Galvin s and Gagne s book ( Operating System Concepts, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. Christoph Kessler, IDA, Linköpings universitet. 21

22 What is an Operating System (OS)? A program that acts as an intermediary between a user of a computer and the computer hardware. Operating system goals: Execute user programs in a well-defined environment. Make application software portable and not tied to a specific machine hiding low-level and implementational detail Make the computer system convenient to use. Administrate system resources. Improves overall system reliability error confinement, fault tolerance, reconfiguration... Enable efficient use of the computer hardware support parallel activity, avoid wasted CPU cycles... Provides protection mechanisms for user programs and files and security against possible threats from within and outside the system 43 What is an Operating System (OS)? An operating system provides an environment within which other programs can do useful work; the OS does not perform any useful function itself

23 Where are OSs found? Embedded systems Microprocessor market shares in % 1% Real-time systems Operating Systems (A Selection) General purpose operating systems Unix incl. variants, such as Sun/Oracle Solaris, HP-UX Linux Windows 95/98/2000, NT/XP/Vista/7/8/10/ Mac OS X Application specific operating systems, e.g. mobile OS, real-time OS Android ios Symbian Windows CE, Mobile, Phone Embedded Linux RT-Linux VxWorks OSE QNX

24 Operating System Definition OS is a resource allocator Manages all resources of a computer system Decides between conflicting requests for efficient and fair resource use OS is a control program Controls execution of programs to prevent errors and improper use of the computer No universally accepted definition The one program running at all times on the computer is called the kernel. Everything else is either a system program (ships with the operating system) or an application program. 47 Computer System Structure Computer system can be divided into 4 components: Hardware provides basic computing resources CPU, memory, I/O devices Operating system controls and coordinates use of hardware among various applications and users Application programs define the ways in which the system resources are used to solve the computing problems of the users Word processors, compilers, web browsers, database systems, games Users People, machines, other computers 48 24

25 Agenda Lecture I: Computer Systems Overview Building and Executing Programs Operating System Basics Lecture II: Interrupts and I/O System Calls CPU Management Memory Management File System Protection and Security 49 25