Overview: Concurrent Architectures - Unix: Forks and Pipes Other Matters: TuteLab-5 solutions and the proof of Peterson s Algorithm Ref: [Coulouris&al Ch 4] history architecture: monolithic vs microkernels, hypervisors processes: creation, synchronization, sleeping shells and command line interpretation file descriptors and I/O redirection pipes (based on lecture by Alistair Rendell, CDS 2009; architecture diagrams from Coulouris et al, Distributed Systems) COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 1
Unix - History origins - the Multics Project Bell AT&T, MIT and GE Ken Thomson and Dennis Ritchie the C language PDP 11 features hierarchical file system compatible I/O (file, device & inter-process) command language choice portability standards - POSIX descendants Version 7, BSD 4.2, SystemV, Solaris, XINU, Minix, Linux, MacOSX specification history and timeline the first Unix Port (1977) COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 2
Operating System Architecture (a) system layers ([Coulouris&al Fig 7.1]) (b) core OS functionality ([Coulouris&al Fig 7.2]) COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 3
Monolithic vs Micro- Kernels monolithic: all OS services within the kernel hard to maintain, port, ensure reliability; may be efficient e.g. most old OS: Windows (pre NT), MacOS (pre 9) monolithic and modular: high-level services as platform-independent modules better! e.g. most modern Unix implementations monolithic and layered: services are layered from high to- low level better except possibly reduced efficiency (e.g. Solaris) microkernel: only implements low-level process, memory, interrupt and message handling easiest to port, maintain, make reliable may be inefficient due to increased messaging e.g. MacOSX: Mach µkernel + BSD Unix, L4 can support multiple OS on 1 host (usually on a lower privilege level) COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 4
Monolithic vs Microkernels: Operation (a) process memory image ([Coulouris&al Fig 7.15])... Microkernel Hardware (b) µkernel supports middleware via subsystems ([Coulouris&al Fig 7.16]) COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 5
Hypervisors: Microkernels at a Higher Privilege Level virtualization provides the facility to abstract an OS from its physical resources guest OS obtains microkernel services from a hypervisor. e.g. VMWare, Xen this forms an essential technology for cloud computing Xen Architecture: ([Coulouris&al Fig 7.16]) communication goes through domain0 issue: mitigating the overheads of this opportunity: take advantage of guest OS migration save energy, better match of guest to H/W COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 6
Unix process creation is by cloning Unix Process Reproduction the standard Unix C library contains the function ÓÖ µ typical usage: Ô ÓÖ µ ÓÖ µ spawns a duplicate process (same memory, registers) except the return value is different 0 to the child; pid of child to the parent (never 0!) -1 if it fails example: forkexec.c ÓÖ µ ¼µ {»» Á Ñ Ø Ð " / b i n / g r e p " " g r e p " ÝÛÓÖ ÆÍÄÄ µ Ü ÐÔ µ ÓÖØ Ð } { }»» Ö Ø Ó Ô Ö ÒØ ÔÖÓ Ö Ñ what is the child? what is concurrent? why does grep appear twice? why might it abort? COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 7
([Coulouris&al Fig 7.4]) Process Creation and Copy-on-write Semantics COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 8
Ô Ö ÓÖÑ Ð ³ Ø»» ¼µ Ü Ø Ô Ö ÒØ ³ Ø»» ² Ð ËØ ØÙ µ Û Ø co-ordination can be a problem: Process Requiem! a child was spawned to do something in parallel (concurrently) at some point the parent might need to be sure it is complete the ÒØ Û Ø ÒØ Ø ØÙ µ system call solves this waits for termination of a (any) child returns the pid of the child that has died argument gives status information (including exit code) for the child example ÓÖ µ ¼µ{ } Ð { ÒØ Ð ËØ ØÙ } concurrency? COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 9
Unix Processes Dozing why wait for time to pass? it may want to do something periodically it can make sure other processes get a chance other reasons? the ÒØ Ð Ô ÒØ µ system call enables this the supplied argument is the duration (in seconds) may return early if woken up (by a signal) return value indicates the sleep-time remaining use of Ð Ô ¼µ COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 10
Unix Shells a job control language pre-unix JCLs were proprietary and idiosyncratic not part of the Operating System (but must reflect its facilities) can have one (or more) shells per user e.g. sh - Bourne shell,csh - C Shell, tcsh - extended C-shell, bash - Bourne-again shell processes and process combinations are supported by shells ps emacs Bakery.java & ps -aux grep $USER java -ea Bakery 4 2 bakeryanalyzer can write executable programs in shell code (a script) (e.g. BakeryTraceViews) interpreted, not compiled first line of file indicates the appropriate interpreter COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 11
Command-Line Interpretation Fork() Fork() wait(&status) exec(cmd arglist) Fork() exec(cmd2 args2) exec(cmd1 args1) exit(status) exit(status) wait(&status) exit(status)?wait(&status)? (See Zombies in Lab7) cmd arglist cmd1 args1 & cmd2 args2 COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 12
UNIX File Descriptors non-negative integers denoting an open I/O channel they are indices (0, 1, 2,... ) the File Descriptor Table (FDT) associated with the current process the corresponding table entries only mean something to the operating system! three descriptors are standard: Ø Ò: 0, Ø ÓÙØ: 1, Ø ÖÖ: 2 (by default, associated with keyboard & current command window) the FDT is preserved across Ü µ and copied during 0 1 2 3 4 5 0 1 2 3 4 5 ÓÖ µ COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 13
File Descriptor Table Operation 0 1 2 3 4 5 opening a file first free entry in the FDT is chosen closing a file entry in FDT is marked free opposite to open! 0 1 Open 4 5 0 1 2 3 Close Redirecting I/O, for example Ø ÓÙØ: user cannot change the FDT directly the 0 1 2 3 4 5 ÒØ ÙÔ¾ ÒØ ÓÐ ÒØ Ò Û µ system call can do this: ÙÔ¾ ½µ Xterm outfile 0 1 2 3 4 5 outfile 0 1 2 3 4 5 outfile outfile 0 1 2 3 4 5 outfile Close dup (copy fd 5 to fd 1) Close COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 14
UNIX Pipes consider ps -aux grep 666 ps writes to Ø ÓÙØ grep reads from Ø Ò I/O has been re-directed by the shell mental model of a pipe: it s a buffer for characters handled by I/O sub-system of Unix may only be one-way (Solaris has two-way) pipe dependencies: P2 will wait for production P1 will not wait for consumption there may also be buffering in P1 the importance of timely flushing! what does this mean? (COMP230 Lab9 2010) P 1 P P 2 COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 15
Creating UNIX Pipes opening a pipe: ÚÓ Ô Ô ÒØ Ô Ô ¾ µ P 1 Ô Ô ¼ is set to file descriptor of the input end of the pipe P 2 Ô Ô ½ is set to file descriptor of the output end of the pipe the result does not connect two processes! connecting two processes by a pipe fork the process, but do it after opening a pipe close unwanted descriptors P 1 P 2 COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 16
Command Line I/O Redirection cmd1 args1 > file Fork() cmd1 cmd2 create pipe open(file); make it stdout Fork() Fork() There is a mistake what is it? wait(&status) exec(cmd1 args1) wait(&status) redirect o/p exec(cmd2 ) redirect i/p exec(cmd1 ) exit(status) exit(status) exit(status) wait(&status) close pipe Q1: what is the mistake? (1) redirects need to be swapped (2) commands need to be swapped (3) either of the above (4) both of (1) and (2) (5) none of the above COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 17
Pipelines for Sub-processes and Shared Processes sub-process pipeline: desired state make pipes (before forking) fork two times close pipes in parent each child closes the descriptors that it won t use each child does an Ü µ if appropriate red state parent pipes can be shared integrity of messages a pipe is just a stream of chars no interleaving BUT care required with buffering messages must be less than 2KB Producer 1 child Consumer Producer Filter Consumer child child child Producer 2 child child COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 18
Putting it all Together: Parent Pipes its Input to its Child Q2: on which side is the code for the child process? piperpwc.c Ô ¾ Ö ÒØ Ô Ô Ôµ ½µ { Ð { } Ô ÖÖÓÖ " n o p i p e " µ Ü Ø ½µ ÓÖ µ ¼µ { ÒØ Ð ËØ ØÙ Ô ¼ µ ÐÓ Ø Ö µµ Ç µ Û Ð Ô ½ µ ÐÓ Ö Ö Ô ¼ ² ½µµ Û Ð ¼µ > ÛÖ Ø Ô ½ ² ½µ ½µ " p i p e b r o k e n " µ Ô ÖÖÓÖ µ ÔÙØ Ö Ö ½µ { " p i p e b r o k e n " µ Ô ÖÖÓÖ Ô ¼ µ ÐÓ } } Ô ½ µ ÐÓ ½µ Ü Ø } Ü Ø ½µ } Ô ½ µ ÐÓ ² Ð ËØ ØÙ µ Û Ø Ö ØÙÖÒ ¼µ Ô ¼ µ ÐÓ ¼µ Ü Ø Q3: is the 2nd ÐÓ Ô ¼ µ on the left good style? needed? Q4: and the 2nd ÐÓ Ô ½ µ on the right? COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 19
Ô ½ µ ÐÓ»» ½ µ ººº Ô ¼ µ ÐÓ»» ¾ µ ººº Ø Ö µµ Ç µ Û Ð µ ÔÙØ Ö ËÌ ÇÍÌ ÁÄ ÆÇ µ ÐÓ ² Ð ËØ ØÙ µ Û Ø Parent Pipes its Input to its Child by I/O Redirection Use the ÙÔ¾ ÒØ ÓÐ ÒØ Ò Û µ call to redirect the pipe to Ø ÓÙØ (1) of parent and Ø Ò (0) of child: Ô ¾ Ö ÒØ Ô Ô Ôµ ½µ { Ð { } Ð ËØ ØÙ ÒØ Ô ÖÖÓÖ " n o p i p e " µ Ü Ø ½µ»» µ ººº ÓÖ µ ¼µ { Ø Ö µµ Ç µ Û Ð µ ÔÙØ Ö } Ö ØÙÖÒ ¼µ Q5: should the ÙÔ¾ µ calls be at (A), or (B1) and (B2)? Ü Ø ¼µ Q6:: the appropriate calls are: ÙÔ¾ Ô ¼ ½µ (1) for child, ÙÔ¾ Ô ½ ¼µ and for parent ÙÔ¾ Ô ¼ ¼µ (2) for child, ÙÔ¾ Ô ½ ½µ and for parent ÙÔ¾ Ô ½ ½µ (3) for child, ÙÔ¾ Ô ¼ ¼µ and for parent ÙÔ¾ Ô ½ ¼µ (4) for child, ÙÔ¾ Ô ¼ ½µ and for parent COMP310 Lecture 20: Operating Systems: Unix Forks and Pipes2014 20