Operating Systems Notes

Size: px

Start display at page:

Download "Operating Systems Notes"

Denis Perry
5 years ago
Views:

1 Operating Systems Ntes Here are are sme rugh ntes I put tgether as part f revisin fr a uni curse. They are heavily based n the curse lecture ntes by Kevin Elphinstne and Lenid Ryzhyk. All diagramsare surced frm thse lecture ntes, sme f which are in turn frm the text bk, A. Tannenbaum, Mdern Operating Systems. OS Overview The OS hides the details f the hardware and prvides an abstractin fr user cde. The OS is respnsible fr allcating resurces (cpu, disk, memry ) t users and prcesses. It must ensure n starvatin, prgress, and that the allcatin is efficient. The kernel is the prtin f the OS running in privileged mde (hardware must have sme supprt fr privileged mde fr the OS t be able t enfrce the user and kernel mde distinctin). When the hardware is in privileged mde all instructins and registers are available. When the hardware is in user mde nly a limited sets f instructins, registers and memry lcatins are available. Fr instance when in user mde, it is nt pssible t disable interrupts because therwise a user culd ensure that the OS never gets cntrl again.. A prcess is a instance f a prgram in executin. A prcess in memry usually has at least three segments. The text segment fr the prgram cde (instructins), a data segment called the heap fr glbal variables and dynamically allcated memry, and a stack segment used fr functin calls. A prcess als has an executin cntext which includes registers in use, prgram cunter, stack pinter, etc.

2 In a Mnlithic OS different parts f the OS like prcessr allcatin, memry management, devices and file systems are nt strictly separated int separate layers but rather intertwined, althugh usually sme degree f gruping and separatin f cmpnents is achieved. System Calls The system call interface represents the abstract machine prvided by the perating system. man syscalls Syscall numbers hardware syscall instructin Prcesses and Threads Prcesses can cntain ne r mre threads. These threads are units f executin, and always belng t a prcess. A prcess can terminate by, nrmal exit (vluntary), usually returning EXIT_SUCCESS errr exit (vluntary), usually returning EXIT_FAILURE fatal errr (invluntary), eg. segfault, divide by zer errr killed by anther prcess (invluntary), eg. pkill (sending the SIGTERM signal) Thread states Running t ready culd be frm a vluntary yield, r end f time allcated by the CPU, s the scheduler mves the thread t the Ready queue t give anther prcess a g. Running t blcked culd be frm a calling a syscall and waiting fr the respnse, waiting n a device, waiting fr a timer, waiting fr sme resurce t becme available, etc. The scheduler (als called the dispatcher) decides which thread t pick frm the read queue t run. The idea is that a prcess with multiple threads can still make prcess when a single thread blcks, as if ne thread blcks, the thers can still be wrking. In mre recent times, it allws a prcess t have threads running n multiple CPU s n mdern multi-cre machines.

The ther mdel is finite-state machine where syscall s dn t blck but instead allw the prgram t cntinue running and then send the prcess an interrupt when the syscall has been dealt with.

3 The ther mdel is finite-state machine where syscall s dn t blck but instead allw the prgram t cntinue running and then send the prcess an interrupt when the syscall has been dealt with. This is nt as easy t prgram thugh. During the system initialisatin backgrund prcesses (called daemn s n linux, service s n windws) and fregrund prcesses can be started. Threads Implementatin Kernel-threads vs. User-level threads. User-level threads Thread management is handled by the prcess (usually in a runtime supprt library) The advantages are, switching threads at user-level is much cheaper than the OS ding a cntext switch yu can tune yur user-level thread scheduler, and nt just use the OS prvided ne n OS supprt fr threads needed The disadvantages are, yur threads must yield(), yu can t really n an interrupt t give yur scheduler cntrl again (this is knwn as c-perative multithreading) cannt take advantage f multiple CPU s if ne f yur user-level threads gets blcks by the OS, yur whle prcess gets blcked (because the OS nly sees ne thread running)

Kernel-level Threads Thread management is dne by the kernel thrugh system calls The dwnside is thread management (creatin, destructin, blcking and unblcking threads) requires kernel entry and exit,

4 Kernel-level Threads Thread management is dne by the kernel thrugh system calls The dwnside is thread management (creatin, destructin, blcking and unblcking threads) requires kernel entry and exit, which is expensive The upside is we nw have preemptive multithreading (the OS just takes cntrl when it wants t schedule anther prcess, s n need t rely n the thread t yield), can take advantage f a multiprcessr, and individual threads can have islated blcking. A Thread switch can happen in between executin f any tw instructins, at the same time it must be transparent t the threads invlved in the switch. S the OS must save the state f the thread such as the prgram cunter (instructin pinter), registers, etc. as the thread cntext. The switch between threads by the OS is called a cntext switch. Cncurrency A race cnditin ccurs when the cmputatin depends n the relative speed f tw r mre prcesses. Dealing with race cnditins, Mutual exclusin Identify the shared variables, identify the cde sectins which access these variables (critical regin) and use sme kind f mutual exclusin (such as a lck) t ensure that at mst nly ne prcess can enter a critical regin at a time. Lck-free data structures Allw the cncurrent access t shared variables, but design data structures t be designed fr cncurrent access Message-based cmmunicatin Eliminate shared variables and instead use cmmunicatin and synchrnisatin between prcesses.

5 Mutual Exclusin enter_regin() and leave_regin(). Hardware usually prvides sme mutual exclusin supprt by prviding an atmic test and set instructin. A uniprcessr system runs ne thread at a time, s cncurrency arises frm preemptive scheduling. But with multiprcessr machines cncurrency als arises frm running cde in parallel using shared memry. The prblem with the apprach t mutual exclusin s far is that when prcess B is blcked, it just sits in a lp waiting fr the critical regin t becme available. Can fix this by using sleep and wakeup. We use a mutex lck fr this. Blcking lcks can nly be implemented in the kernel, but can be accessed by user-level prcesses by system calls. A mutex allws at mst ne prcess t use a resurce, a semaphre allws at mst N prcesses. Prducer-cnsumer prblem Prducer can sleep when buffer is full, and wake up when space available. Cnsumer can sleep when buffer is empty and wake up when sme items available. Can d using semaphres r cnditin variables. Mnitrs are set up t nly allw ne prcess/thread t perate inside it at nce, with extra requests put in a queue. Implemented by the cmpiler. Cnditin variables. cv_create, cv_destry, cv_wait, cv_signal, cv_bradcast Dining philsphers prblem. Need t prevent deadlck. Deadlck A set f prcesses is deadlcked if each prcess in the set is waiting fr an event that nly anther prcess in the set can cause. Fur cnditins fr deadlck: Mutual exclusin cnditin (device nt shareable amngst threads) Hld and wait cnditin (a resurce can be held, but then blck awaiting mre resurces) Can attack this by requiring all prcess ask fr all resurces at the start, and nly start if they are granted. N preemptin cnditin previusly granted resurces cannt be frcibly taken away Circular wait cnditin Always request resurces in the same rder Dealing with deadlck: ignre the prblem detect and recver

dynamic avidance (careful resurce allcatin) we require sme infrmatin in advance like which resurces and hw many will be needed befre prcess starts.

6 dynamic avidance (careful resurce allcatin) we require sme infrmatin in advance like which resurces and hw many will be needed befre prcess starts. Bankers algrithm preventin (negate ne f the fur cnditins fr deadlck) Starvatin is slightly different t deadlck as the system can be making prgress, but there are sme prcesses which never make prgress. File Systems File systems prvide an abstractin f the physical disk. They allw yu t stre files in a structured manner n a strage device. The file system abstractin, Architecture f the OS strage stack, The applicatin interacts with the system call interface which n linux prvides creat, pen, read, write, etc.

In the case f mdern hard drives, The disk cntrller (between device driver and physical disk) hides disk gemetry and expses a linear sequence f blcks.

7 In the case f mdern hard drives, The disk cntrller (between device driver and physical disk) hides disk gemetry and expses a linear sequence f blcks. The device driver hides the device specific prtcl t cmmunicate with the disk. The disk scheduler takes in requests cming frm different prcesses and schedules them t send ne by ne t the device driver. The buffer cache keeps blcks in memry t imprve read and write times fr frequently accessed blcks. The file system (FS) hides the blck numbers and instead expses the directry hierarchy, files, file permissins, etc. T d this it must knw which blcks relate t which file, and which part f the file. It must als manage hw t stre this n the blcks f the linear address space f the disk, keeping track f which blcks are in use. See Allcatin Strategies The virtual file system (VFS) prvides an interface s that different file systems which are suitable fr VFS (ie. they have the cncept f files, directries, etc..) can be accessed unifrmly. The pen file table (OF table) and file descriptr table (FD table) keep track f files pened by userlevel prcesses. There are many ppular file systems in use tday. FAT16, FAT32 are gd fr embedded devices; Ext2, Ext3, Ext4 are designed fr magnetic disks, ISO9660 is designed fr CD-ROM s, JFFS2 is ptimised fr flash memry as yu dn t want t write t the same lcatin t many times as it wears ut. Others include NTFS, ReiserFS, XFS, HFS+, UFS2, ZFS, JFS, OCFS, Btrfs, ExFAT, UBIFS. Perfrmance issues with standard magnetic disks T access a blck n a disk the head must mve t the required track (seek time), then we have t wait until the required blck n the track reaches the head (rtatinal delay). We als have sme mstly fixed verhead fr the data t be passed between the device and the driver (transfer time). Ttal average access time,. Where r is the rtatinal speed, b is the number f bytes t transfer, N is the number f bytes per track and is the seek time. Seek time is the mst expensive, s we want t ensure that ur disk arm scheduler gives gd perfrmance. First-in, First-ut (n starvatin) Shrtest seek time first (gd perfrmance, but can lead t starvatin) Elevatr (SCAN) (head scans back and frth ver the disk. n great starvatin, sequential reads are pr in ne f the directins) Circular SCAN (head scans ver the disk in ne directin nly) Blck Allcatin Strategies Cntiguus Allcatin Gives gd perfrmance fr sequential peratins as all the blcks fr a file are tgether in rder thugh yu need t knw the maximum file size at creatin as files are deleted, free space is fragmented (external fragmentatin)

8 Dynamic Allcatin Blcks allcated as needed, hence the blcks fr a file culd be all ver the place n the disk need t keep track f where all the blcks are and the rder fr each file External fragmentatin - space wasted external t the allcated regins, this space becmes unusable as its nt cntiguus (s yu have lts f small spaces but yu need a larger space t fit a whle file in) Internal fragmentatin space wasted internal t the allcated memry regins, eg. yu get a 4K blck allcated and nly use 1K, but the OS can t give the leftver 3K t smene else. Keeping track f file blcks File allcatin table (used fr FAT16 and FAT32) inde based FS; keep an index nde fr each file which stres all the blcks f the file. Directries are stred as nrmal files but the FS gives these file special meaning. A directry file stres a list f directry entries, where each entry cntaining the file name (because Ext2/3/4 stre these with the directry file, nt the inde fr the file), attributes and the file i-nde number. Disk blcks (sectrs) have a hardware set size, and the file system has a filesystem set blck size which is sectr size * 2^N, fr sme N. A larger N means large files need less space fr the inde, but smaller blcks waste less space fr lts f small files.

The single indirect is a blck numbers t a blck which cntains mre blck numbers.

9 Ext2 indes The tp bunch f data is file attributes. Blck cunt is the number f disk blcks the file uses. The direct blcks area stres index s fr the first 12 blcks used by the file. The single indirect is a blck numbers t a blck which cntains mre blck numbers.w System call interface At the system call level a file descriptr is used t keep track f pen files. The file descriptr is assciated with the FS inde, a file pinter f where t read r write next and the mde the file was pened as like read-nly.

10 Mst Unix OS s maintain a per-prcess fd table with a glbal pen file table. VFS Prvides a single system call interface fr many file systems. the applicatin can write file access cde that desn t depend n the lw level file system device can be hard disk, cd-rm, netwrk drive, an interface t devices (/dev), an interface t kernel data structures (/prc)

Jurnaling file system keeps a jurnal (r lg) f FS actins which allws fr the FS t recver if it was interrupted when perfrming an actin that is nt atmic, but needs t be.

11 Jurnaling file system keeps a jurnal (r lg) f FS actins which allws fr the FS t recver if it was interrupted when perfrming an actin that is nt atmic, but needs t be. Memry Management and Virtual Memry The OS needs t keep track f physical memry, what s in use and which prcess is it allcated t. It must als prvide applicatins with a view f virtual memry that they are free t use.

12 Swapping (smetimes called paging) is where memry is transferred between RAM and disk t allw fr mre data in memry than we have space fr in physical RAM. On base-limit MMU s swapping nly allws wh prgrams memry allcatin t be swapped t disk, rather than just pages at a time as with virtual memry, hence yu can t use swapping here t allw prgrams larger than memry t run. If we are nly running ne prgram we can give it all f memry (and either run the in part f it, r in sme ther ROM). Hwever many systems need mre than ne prcess t be running at the same time. Multiprgramming als means that we can be utilising the CPU even when a prcess is waiting n I/O (ie. give the CPU t the ther prcess). But t supprt multiprgramming we need t divide memry up. We culd divide memry up int fixed size partitins and allcate these t prcesses, but this creates internal fragmentatin (wasted space inside the partitin allcated). Using dynamic partitining we give each prcess exactly hw much it needs, thugh this assumes we knw hw much memry we need befre the prcess is started. This apprach als leads t external fragmentatin where we have lts f little unallcated gaps, but these are t small t be used individually. Appraches t dynamic partitining include first-fit, next-fit, best-fit and wrst-fit. Binding addresses in prgrams Cmpile time Lad time Run time Prtectin we nly want each prcess t be able t access memry assigned t that prcess, nt ther prcess Base and Limit Registers - set by the kernel n a cntext switch, the hardware handles the rest Virtual Memry - tw variants Paging Partitin physical memry int small equal sized chunks called frames. Divide each prcess s virtual (lgical) address space int same size chunks calledpages. S a virtual memry address cnsists f a page number and an ffset within that page. OS maintains a page table which stres the frame number fr each allcated virtual page.

N external fragmentatin, small internal fragmentatin. Allws sharing f memry Prvides a nice abstractin fr the prgrammer Implemented with the aid f the MMU (memry management unit).

13 N external fragmentatin, small internal fragmentatin. Allws sharing f memry Prvides a nice abstractin fr the prgrammer Implemented with the aid f the MMU (memry management unit). Segmentatin A prgram s memry can be divided up int segments (stack, symbl table, main prgram ) Segmentatin prvides a mapping fr each f these segments using a base and limit. Virtual Memry If a prgram attempts t access a memry address which is nt mapped (ie. is an invalid page) a page fault is triggered by the MMU which the OS handles. Tw kinds f page faults, Illegal Address (prtectin errr) signal r kill the prcess Page nt resident get an empty frame r lad the page frm disk, update the page table, and restart the faulting instructin. Each entry in the page table nt nly has the crrespnding frame number but als a cllectin f bits like prtectin bits, caching disabled bit, mdified bit, etc. Page tables are implemented as a data structure in main memry. Mst prcesses dn t use the full 4GB address space s we need a data structure that desn t waste space. The tw level page table is cmmnly used,

14 An alternative is the inverted page table, The IPT grws with size f RAM, nt virtual address space like the tw level page table. Unlike tw level page table which is required fr each prcess, yu nly need ne IPT fr the whle machine. The dwnside is sharing pages is hard. Accessing the page table creates an extra verhead t memry access. A cache fr page table entries is used called a translatin lk-aside buffer (TLB) which cntains frequently used page table entries.

15 TLB can be hardware r sftware laded. TLB entries are prcess specific s we can either flush the TLB n a cntext switch r give entries an address-space ID s that we can have multiple prcesses entries in the TLB at the same time. Principle f Lcality (90/10 rule) Tempral Lcality is the principle that accesses clse tgether in terms f time are likely t be t the same small set f resurces (fr instance memry lcatins). Spatial lcality is the principle that subsequent memry accesses are ging t be clse tgether (in terms f their address) rather than randm. array lp example The pages r segments required by an applicatin in a time windw ( ) is called its memrywrking set. Thrashing, T recver frm thrashing, just suspend sme prcesses until it eases. Fetch plicy when shuld a page be brught int memry? n demand, r pre-fetch? Replacement plicy defines which page t evict frm physical memry when its full and yu start swapping t disk. Optimal FIFO prblem is that age f a page isn t necessarily related t its usage Least Recently Used wrks well but need t timestamp each page when referenced. Clck (aka. Secnd Chance) an apprximatin f LRU. Uses a usage r reference bit in the frame table Resident Set Size hw many frames shuld each prcess have? Fixed allcatin Variable allcatin give enugh frames t result in an acceptable fault rate Input Output Prgrammed I/O (plling r busy waiting) Interrupt Driven I/O Prcessr is interrupted when I/O mdule (cntrller) is ready t exchange data Cnsumes prcessr time because every read and write ges thrugh the prcessr Direct Memry Access (DMA) Prcessr sent interrupt at start and end, but is free t wrk n ther things while the device is transferring data directly t memry.

16 Scheduling The scheduler decides which task t run next. This is used in multiple cntexts, multi-prgrammed systems (threads r prcesses n a ready queue), batch system (deciding which jb t run next), multi-user system (which user gets privilege?). Applicatin behaviur CPU bund cmpletin time largely determined by received CPU time I/O bund cmpletin time largely determined by I/O request time Preemptive (requires timer interrupt, ensures ruge prcesses can t mnplise the system) v nnpreemptive scheduling. Scheduling Algrithms can be categrised int three types f systems, Batch systems (n users waiting fr the muse t mve) Interactive systems (users waiting fr results) Rund Rbin each prcess is run and if it is still running after sme timeslice t, the scheduler preempts it, putting it n the end f the ready queue, and scheduling the prcess n the head f the ready queue. If the timeslice is t large the system becmes sluggy and unrespnsive, if it is t small t much time is wasted n ding the cntext switch. The traditinal UNIX scheduler uses a pririty-based rund rbin scheduler t allw I/O bund jbs t be favured ver lng-running CPU-bund jbs. Realtime systems (jbs have deadlines that must be meet) Realtime systems are nt necessarily fast, they are predictable. T schedule realtime tasks we must knw, its arrival time, maximum executin time (service time), deadline ( ). tasks culd be peridic r spradic. slack time is time when the CPU is nt being used, we are waiting fr the next task t becme available. tw scheduling algrithms, Rate Mntnic pririty driven where pririties are based n the perid f each task. ie. the shrter the perid the higher the pririty Earliest Deadline First guaranteed t wrk if set f tasks are schedulable. The earlier the deadline the higher the pririty. Multiprcessr Systems Fr this sectin we lk at shared-memry multiprcessrs. There are unifrm memry accesses multiprcessrs and nn-unifrm memry access multiprcessrs which access sme parts f memry slwer than ther parts. We fcus n the UMA MP s. Spinlcking n a uniprcessr is useless, as anther thread n the same prcessr needs t release it, s blcking asap is desirable. On a multiprcessr, the thread hlding the lck may be presently active n anther prcessr, and it culd release the lck at any time. On a multiprcessr, spin-lcking can be wrthwhile if the average time spent spinning is less than the average verheads f cntext switch away frm, and back t, the lck requester.

17 Thread affinity refers t a thread having sme preferred prcessr t run n an a multiprcessr machine. Fr instance if thread A started n cpu0 it may want t be scheduled again n cpu0 rather than cpu1 as parts f the cache may still be intact. Multiprcessr systems have multiple ready queues, as just ne ready queue wuld be a shared resurce which intrduces lck cntentin. Surce:

CSE 3320 Operating Systems Synchronization Jia Rao

CSE 3320 Operating Systems Synchrnizatin Jia Ra Department f Cmputer Science and Engineering http://ranger.uta.edu/~jra Recap f the Last Class Multiprcessr scheduling Tw implementatins f the ready queue