1. Systems Programming using C (File Subsystem)

Transcription

1 Intended Schedule 1. Systems Programming using C (File Subsystem) Date Lecture Hand out Submission Introduction to Operating Systems Course registration Systems Programming using C (File Subsystem) 1. Assignment Systems Programming using C (Process Control) 2. Assignment 1. Assignment Processes Scheduling 3. Assignment 2. Assignment Process Synchronization 4. Assignment 3. Assignment Inter Process Communication 5. Assignment 4. Assignment Pfingstmontag 6. Assignment 5. Assignment Input / Output 7. Assignment 6. Assignment Memory Management 8. Assignment 7. Assignment Assignment 8. Assignment Filesystems Assignment 9. Assignment Special subject: Transactional Memory 10. Assignment Special subject: XQuery your Filesystem Wrap up session First examination date Second examination date Schedule for today Last lecture: All operating system provide services for programs they run. execute new program, open a file, read a file, allocate a region of memory, get the current time,... Today: Systems Programming using C Practical approach to the UNIX System Interface Use system calls for unbuffered file I/O. Construct higher level (standard) routines from system calls. Use library functions for standard I/O. From Programs to Processes. Tomorrow: Introduction to the C Programming Language Focus on differences compared to JAVA UNIX System Interface UNIX provides its services through a set of system calls. System calls: Functions within the OS that may be called by user programs. (A) Black Box Benutzerapplikation, die API anspricht POSIX.1 System Call Interface (B) White Box Benutzerapplikation, die API anspricht POSIX.1 System Call Interface Betriebssystemkern (operating system kernel) 48 49

2 System calls System calls determine a direct erface to the kernel. Employed for maximum efficiency. Access some facility not implemented in the libraries. The calls available in the erface vary from OS to OS. Underlying concepts, however, tend to be similar. System calls accessed by C functions Definition of the system call erface is in the C language * A standard technique on UNIX is for each system call to have a function of the same name encapsulated in a C library. Those functions invoke the apt kernel service, using whatever technique is required on the system. The function may put one or more of the C arguments o general registers and then execute some machine instruction that generates a software errupt in the kernel. We can consider the system calls as being C functions. * Regardless of the actual implementation technique used to invoke a call UNIX system architecture (System V) -."#&'"("%!"#$"%&'"("%!"+5#<"$=$;6$1))" :$;6$1))&%&A%;#?"D.('#")15E$5E4.>?+%##'#"AA"-8.3.4"5&8*%%&0$4"#/*8"9 Documentation of system calls The system call erface has traditionally been documented in Section 2 of the UNIX Programmer s Manual. 71#"%4.5&'('#")-8/0%"& "59 B+#"$4:$;<"''4 C;))5+%D1#%;+ 8;<=9 Files :5EE"$ //"#&8*89"9 :$;<"'''#"5"$5+6'4.5&'('#")-86#78".. 87$4#7%& "59 :$;<"'';$4 <5#"%A "+1%0$:9 Processes F"%>?"+-889*#*84"#9!A;>D-82%78>9 85"57#3 5*$*:"5"$49 01$231$"4.#"5"$5+6!"#$"%&'"("% )*#+,*#"&'"("% 01$231$" 52 53

3 Documentation of library calls Section 3 of the UNIX Programmer s Manual defines general purpose functions available to the programmer. These functions are not entry pos o the kernel. They may use kernel s system calls, however. prf(3): May invoke write(2) to perform output. atoi(3) (convert ASCII string to eger): no kernel at all. Implementor s view (kernel programming): Distinction between system call vs. library function is fundamental. User s perspective (systems programming): Not as critical, both exist to provide services for application programs. System Interface Standardization Standardization of UNIX During the 1980s the proliferation of UNIX versions and differences between them led many large users (such as the U.S. government) to call for standardization. Among others ANSI C and the IEEE POSIX emerged. American National Standards Institute Institute of Electrical and Electronics Engineers Portable Operating System Interface for UNIX Overview UNIX versions and systems BSD BSD BSD BSD BSD Lite BSD Lite BSD 1986 A/UX 1989 Acorn RISC ix 1988 Acorn RISC Unix 1990 AIX 2000 AIX 5L 1989 AIX PS/ AIX/ AIX/ AIX/ESA 1986 AIX/RT 1990 AMiX 1995 AOS Lite 1992 AOS Reno 2007 AppleTV 1994 ArchBSD 1991 ASV 1989 Atari Unix 1989 BOS 1979 BRL Unix 1988 BSD Net/ BSD Net/ BSD/ BSD/OS 1978 CB Unix 1986 Chorus 1988 Chorus/MiX 1983 Coherent 1987 CTIX 1984 CXOS 1999 Darwin 2000 Debian GNU/Hurd 1995 DEC OSF/1 ACP 1989 Dell Unix 2005 DesktopBSD 1995 Digital Unix 2003 DragonFly BSD 1984 Dynix 1993 Dynix/ptx 2003 ekkobsd 1977 Eunice 2004 FireFly BSD 1993 FreeBSD 2006 FreeDarwin 1986 GNU 2001 GNU-Darwin 2005 Gnuppix GNU/Hurd-L HPBSD 1983 HP-UX 2000 HP-UX 11i 1991 HP-UX BLS 1988 IBM AOS 1985 IBM IX/ Interactive 386/ix 1978 Interactive IS 2007 iphone OS X 2007 ipod OS X 1983 IRIS GL IRIX 1991 Linux 1994 Lites 1977 LSX 1999 Mac OS X 1999 Mac OS X Server 1985 Mach 1974 MERT 2002 MicroBSD 1977 Mini Unix 1984 Minix 2005 Minix Minix-VMD 1985 MIPS OS RISC/os 2002 MirBSD 1996 Mk Linux 1998 Monterey 1988 more/bsd 1983 mt Xinu 1993 MVS/ESA OpenEdition 1993 NetBSD 1988 NeXTSTEP 1987 NonStop-UX 1994 Open Desktop 2001 Open UNIX OpenBSD 2003 OpenDarwin 1995 OpenServer OpenSolaris 1996 OPENSTEP 1996 OS/390 OpenEdition 1997 OS/390 Unix 1990 OSF/ PC-BSD 1982 PC/IX 1986 Plan Plurix 2007 PureDarwin 1977 PWB 1974 PWB/UNIX 1984 QNX 2001 QNX RTOS 1996 QNX/Neutrino 1981 QUNIX 1997 ReliantUnix 1997 Rhapsody 1991 RISC ix 1977 RT 1994 SCO UNIX 2002 SCO UnixWare SCO Xenix 1987 SCO Xenix System V/ Security-Enhanced Linux 2004 Silver OS 1983 Sinix 1995 Sinix ReliantUnix 1990 Solaris Solaris SPIX 1982 SunOS 2004 Triance OS 1999 Tru64 Unix 1995 Trusted IRIX/B 1998 Trusted Solaris 1991 Trusted Xenix 1977 TS 1981 Tunis 1980 UCLA Locus 1979 UCLA Secure Unix 1988 Ultrix 1984 Ultrix 32M 1982 Ultrix Unicos 1996 Unicos/mk 2002 Unicos/mp 1993 Unicox-max 1969 UNICS 1981 UniSoft UniPlus 1979 UNIX 32V 1991 UNIX Interactive 1981 UNIX System III 1982 UNIX System IV 1983 UNIX System V 1984 UNIX System V Release UNIX System V Release UNIX System V Release UNIX System V/ UNIX System V/ UNIX Time-Sharing System 1993 UnixWare 1998 UnixWare UNSW 1977 USG 1982 Venix 1980 Xenix OS 1984 Xinu 1998 xmach 2001 z/os Unix System Services The POSIX standard POSIX refers to a family of related standards originally used as synonym for IEEE Std POSIX.1 emerged as a preferred term The latest version of POSIX.1 was published on April, 30th 04. It is called IEEE Std , 2004 Edition (POSIX.1) It is also known as Single UNIX Specification Version 3 (SUSv3)

4 Single UNIX Specification The latest version POSIX.1 has been joly developed by the IEEE and The Open Group 5. As such it is both an IEEE & Open Group Technical Standard: IEEE Std , 2004 Edition The Open Group Technical Standard Base Specifications, Issue 6 It is also an ernational standard ISO/IEC 9945:2003 The standard is published free of charge on the web as The Single UNIX Specification, Version 3, 2004 Edition Tight coupling of C and standards The Single UNIX Specification, Version 3, 2004 Edition: Conceptually, this standard describes a set of fundamental services needed for the efficient construction of application programs. Access to these services has been provided by defining an erface, using the C programming language, a command erpreter, and common utility programs that establish standard semantics and syntax. Readers are expected to be experienced C language programmers The UNIX system erface for File I/O File Subsystem ISO C Standard Library provides an I/O erface that is uniform across operating systems. On any particular system the routines of the standard library have to be written in terms of the facilities provided by the host system. Next we will describe some UNIX system calls for input and output, and show an example how parts of the standard library can be implemented with them

5 UNIX system architecture (System V)!"+5#<"$=$;6$1))" Hello World and related system calls -."#&'"("%!"#$"%&'"("% :$;6$1))&%&A%;#?"D.('#")15E$5E4.>?+%##'#"AA"-8.3.4"5&8*%%&0$4"#/*8"9 71#"%4.5&'('#")-8/0%"& "59 B+#"$4:$;<"''4 C;))5+%D1#%;+ 8;<=9 Files :5EE"$ //"#&8*89"9 :$;<"'''#"5"$5+6'4.5&'('#")-86#78".. 87$4#7%& "59 :$;<"'';$4 <5#"%A "+1%0$:9 Processes 85"57#3 5*$*:"5"$49 01$231$"4.#"5"$5+6!"#$"%&'"("% )*#+,*#"&'"("% 01$231$" I/O system calls - Tracing hello world strace(1): trace system calls & signals Unbuffered I/O 64 65

6 I/O system calls Regarding file I/O it all boils down to five system calls: open, read, write, lseek, and close We speak of unbuffered I/O in contrast to the standard I/O. Unbuffered I/O because each read or write invokes a system call which is immediately executed and not cached inside a buffer (as is using the standard I/O). Unbuffered I/O is not part of ISO C, but POSIX.1 and SUSv3. Standard I/O is provided by the standard C library (ISO C). Further file related system calls: Everything is a file unlink(2), mkdir(2), rmdir(2), link(2), symlink(2), chmod(2), stat(2), umask(2), chown(2), chflags(2), utimes(2), Everything is a file File Descriptor In the UNIX operating system, all input and output is done by reading or writing files. All peripheral devices, even keyboard and screen, are files in the filesystem. This means that a homogeneous erface handles all communication between a program and peripheral devices. In the most general case, before we read or write a file, we must inform the system of our ent to do so, a process called opening the file. When we open an existing file or create a new file, the kernel returns a file descriptor to the process. To the kernel all open files are referred to by file descriptors. A file descriptor is a small non-negative eger. Whenever I/O is to be done on the file, the file descriptor is used instead of the name to identify the file. Each process has a fixed size descriptor table, which is guaranteed to have at least n slots. The call getdtablesize(3) returns the size of this table

7 Special File Descriptors Special File Descriptors A file descriptor is analogous to the FILE poer used by the standard library, or the file object used in JAVA. All info about an open file is maained by the system. Since I/O involving keyboard and screen is so common, special arrangements exist to make this convenient. When the command erpreter (the shell) runs a program, three files are open, with file descriptors*: 0 standard input (stdin) line buffered 1 standard output (stdout) line buffered 2 standard error (stderr) unbuffered * POSIX.1 replaces the magic numbers 0, 1, and 2 with STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO (unistd.h) 70 The user can redirect I/O to and from files with < and >: $./a.out <infile >outfile In this case, the shell changes the default assignments for file descriptors 0 and 1 to the named files. File descriptor 2 normally remains attached to the screen to display error messages. In all cases, the file assignments are changed by the shell, not by the program. The program does not known where its input comes from nor where its output goes, as long as it uses file 0 for input and 1 and 2 for output. 71 Open a file When opening a file the system checks for, e.g., the existence of the file our permissions to access the file If we are going to write on a file it may also be necessary to create it first or discard its previous contents. read(2)ing and write(2)ing a file Read and Write a File Input and output uses the read and write system calls. Those are accessed from C programs through two identically named functions: read(2) and write(2). For both the first argument is a file descriptor. Second argument is a poer to a buffer in the program where the data is to go to or come from. Third argument is the number of bytes to be transferred. # include <sys / types.h> # include <unistd.h> /* Returns : number of bytes read, 0 is EOF, -1 on error ( errno is set ) */ ssize_t read ( d, void * buf, size_t nbytes ); /* Returns : number of bytes written if OK -1 on error ( errno is set ) */ ssize_t write ( d, const void * buf, size_t nbytes );

8 Higher-level routines build on syscalls Higher-level routines build on syscalls It is instructive to see how system calls, such as read and write can be used to construct higher-level routines defined in the standard library, such as getchar(3), putchar(3) words of wisdom Computer Science is largely concerned with an understanding of how low-level details make it possible to achieve highlevel goals. --- Donald E. Knuth, TAOCP *, p1.3 Unbuffered I/O routines * The Art of Computer Programming, Volume 1, Fascicle 1, 2005, Addison-Wesley 76 77

9 Open A File Open(2)ing a file Flags to open(2) a file Flags For The open(2) Function A file is opened by calling the open(2) function: # include <fcntl.h> open ( const char * path, flags, mode_t mode ); flags is an that specifies how the file is to be opened: O RDONLY Open for reading only. O WRONLY Open for writing only. O RDWR Open for reading and writing. path is the name of the file to open or create. Table: One and only one of these three constants must be specified. 29 flags specifies a multitude of options (formed by ORing together one or more constants from fcntl.h (next slide)). fd = open ( name, O_RDONLY, 0); mode holds permission information associated with a file. If successful, open(2) returns a file descriptor. Otherwise, a value of -1 is returned and errno is set to indicate the error ( errno.h). e.g. ENAMETOOLONG: A component of a path name exceeded MAXNAMLEN, or an entire path name exceeded MAXPATHLEN In principle, it s an error to try to open a non-existing file. The system call creat (sic!) is provided to create new files. But Most implementations define O RDONLY as 0, O WRONLY as 1, and O RDWR as 2, for compatibility with older programs Creation of new files Creating New Files More flags to open(2) a file Even More Flags For The open(2) Function # include <sys / types.h> # include <sys / stat.h> # include <fcntl.h> creat ( const char * path, mode_t mode ); However, creat is made obsolete by open: Historically, in early version of the UNIX system, the second argument to open could be only 0, 1, or 2. There was no way to open a file that didn t exist. Therefore, a separate system call, creat, was needed to create new files. With th O CREAT and O TRUNC options now provided by open, a separate creat function is no longer needed: open ( path, O_CREAT O_TRUNC O_WRONLY, mode ); Following constants are optional flags to open(2): O APPEND Append on each write. O CREAT Create file if it does not exist (requires mode argument). O EXCL Error if O CREAT and file exists. O TRUNC Truncate file length to 0. O NONBLOCK Do not block on open or for data to become available. O SYNC Have each write wait for I/O to complete (incl. file attributes). O RSYNC Let read wait until pending writes to same area are complete. O DSYNC Have each write wait for I/O to complete (excl. file attributes). Table: Short description of some more POSIX.1 flags to open. Consult your system manual for further information & implementation details

10 Close A File Close(2)ing time lseek(2): Reposition file offset lseek - Reposition Read/Write File Offset close(2) - delete a descriptor # include <unistd.h> close ( d); # include <unistd.h> off_t lseek ( fildes, off_t offset, whence ); An open file is closed by calling the close function. Releases any record locks the process may have on the file. May be used to not run out of active descriptors per process. When a process exits, all associated file descriptors are freed. returns 0 on success, -1 on failure and sets global errno. close will fail if: Argument is not an active descriptor (EBADF) An errupt was received (EINTR) Every open file has a current file offset Normally, a non-negative eger. Measures the number of bytes from the beginning of file. read and write normally start at the current file offset. They increment the offset by number of bytes read or written. By default the offset is initialized to 0 when a file is opened. Open with O APPEND is an exception Offset erpretation Offset Interpretation Errors & determine current offset Error Indication And Current Offset # include <unistd.h> off_t lseek ( fildes, off_t offset, whence ); Interpretation of offset depends on whence argument: whence offset (re-)position SEEK SET offset is set to offset bytes from the beginning of file. SEEK CUR file s offset is set to its current value plus offset. 30 SEEK END file s offset is set to the size of the file plus offset. 30 A successful call to lseek returns the new file offset. Otherwise, a value of -1 is returned and errno is set. lseek will fail and the file poer will remain unchanged if: EBADF fd is not an open file descriptor. ESPIPE fd is associated with a pipe, socket, or FIFO. EINVAL whence is not a proper value or the resulting offset would be negative on a filesystem or special device that does not allow negative offsets to be used. To determine the current offset, we can seek with zero offset: off_t pos ; pos = lseek ( fd, 0, SEEK_CUR ); 30 The offset can be negative or positive

11 Determine seeking capabilities Seeking Capability Same goes to determine if a file is capable of seeking: main ( void) { if ( lseek ( STDIN_FILENO, 0, SEEK_SET ) == -1) err ( errno, " can not seek [%d].", errno ); else prf (" seek OK.\n"); return (0); } read(2)ing and write(2)ing a file Read and Write a File Input and output uses the read and write system calls. Those are accessed from C programs through two identically named functions: read(2) and write(2). For both the first argument is a file descriptor. Second argument is a poer to a buffer in the program where the data is to go to or come from. Third argument is the number of bytes to be transferred. # include <sys / types.h> # include <unistd.h> $./a. out / etc / motd seek OK. $ cat / etc / motd./a. out a. out : can not seek [29].: Illegal seek /* Returns : number of bytes read, 0 is EOF, -1 on error ( errno is set ) */ ssize_t read ( d, void * buf, size_t nbytes ); $ grep 29 / usr / include / sys / errno.h # define ESPIPE 29 /* Illegal seek */ /* Returns : number of bytes written if OK -1 on error ( errno is set ) */ ssize_t write ( d, const void * buf, size_t nbytes ); File Copying, v4 Example: Low-Level File Copy /* cp: copy f1 to f2 ( file copy) */ main ( argc, char * argv []) { f1, f2; ssize_t n; char buf [ BUFSIZ ]; } if ( argc!= 3) error (" Usage : cp from to"); if (( f1 = open ( argv [1], O_RDONLY, 0)) == -1) error (" can t open %s", argv [1]); if (( f2 = creat ( argv [2], PERMS )) == -1) error (" can t create %s, mode %03 o", argv [2], PERMS ); while (( n = read ( f1, buf, BUFSIZ )) > 0) if ( write (f2, buf, n)!= n) error (" write error on file % s", argv [2]); return (0); Standard I/O

12 UNIX system architecture (System V)!"+5#<"$=$;6$1))" Unbuffered vs. standard I/O Streams and FILE Objects Files -."#&'"("%!"#$"%&'"("% 71#"%4.5&'('#")-8/0%"& "59 F"%>?"+-889*#*84"#9!"#$"%&'"("% )*#+,*#"&'"("%.('#")15E$5E4.>?+%##'#"AA"-8.3.4"5&8*%%&0$4"#/*8"9 :5EE"$ //"#&8*89"9!A;>D-82%78>9 01$231$"4.#"5"$5+6 01$231$" :$;6$1))&%&A%;#?"D :$;<"'''#"5"$5+6'4.5&'('#")-86#78".. 87$4#7%& "59 B+#"$4:$;<"''4 C;))5+%D1#%;+ 8;<=9 :$;<"'';$4 <5#"%A5+6 85"57#3 5*$*:"5"$49 Processes Unbuffered I/O File descriptors So far I/O centered around file descriptors. When a file is opened a file descriptor is returned. It was used for all subsequent I/O operations. Standard I/O Library Streams Standard I/O centers around streams. When opening or creating a file we say that we associate a stream with the file (fopen(3) returns a poer to FILE). FILE contains all the information required by the standard I/O library to manage the stream Unbuffered vs. standard I/O The Standard I/O Library The FILE object The FILE Object Input and output functionality of the ISO C standard library Specified by the ISO C standard. Has been implemented on many OSs other than UNIX. Additional erfaces defined as extensions by SUSv3. Handles details such as buffer allocation and performing I/O in optimal-sized chunks (no need to worry about using the correct buffer size). Ease of use. Initially written by Dennis Ritchie around Typical members of the FILE structure The file descriptor used for actual I/O. A poer to a buffer for the stream. The size of the buffer. Count of the number of characters currently in the buffer. An error flag. Incidental Remark In general there is no need to examine a FILE object, just pass the poer as an argument to each standard I/O function. A poer with type FILE * is referred to as a file poer

13 Buffering Automatic Buffering The standard I/O library provides buffering Goal is to minimize the number of read and write calls. Buffering is tried to be automatically associated to streams. Applications should not worry about it. Different buffering modes can lead to confusions. Buffering Three types of buffering provided by the standard I/O library Fully buffered Line buffered Unbuffered Fully (block) buffered I/O Fully (block) buffered I/O Line buffered I/O Line buffered I/O The fully buffered I/O provided by the standard I/O library: Line buffered I/O provided by the standard I/O library: Actual I/O takes place when the standard I/O buffer is filled. Files residing on disk are normally fully buffered by the library. The buffer is obtained by one of the I/O functions. Usually by calling malloc(3) the first time I/O takes place. The term flush describes the writing of a standard I/O buffer. A buffer can be flushed automatically by the standard I/O routines such as when a buffer fills. Explicitly, by using the function fflush(3). Actual I/O takes place, when a newline character is encountered on input or output. This allows us to output a single character at a time (e.g., with fputc(3)), knowing that actual I/O will take place only when we finish writing each line. Line buffering is typically used on a stream when it refers to a terminal (e.g., standard input and standard output). However, the size of the buffer is fixed, so I/O might take place if the buffer is filled before a newline is seen

14 Unbuffered I/O Unbuffered I/O ISO C buffering requirements ISO C Buffering Requirements ISO C requires the following buffering characteristics: Unbuffered I/O: The standard I/O library does not buffer the characters. When an output stream is unbuffered, information appears on the destination file/terminal as soon as written write(2). Standard error stream is normally unbuffered. Any error messages are displayed as quickly as possible (regardless whether they contain a newline or not). Standard input and output are fully buffered, if and only if they do not refer to an eractive device. Standard error is never fully buffered. Should standard input and output be unbuffered or line buffered, if they refer to an eractive device? Should standard error be line buffered or unbuffered? System dependent (for instance OpenBSD) If stdin and stdout refer to a terminal they are line buffered. Standard error is initially unbuffered Turn buffering on and off Turn Buffering On and Off Altering buffer behaviour Alter Buffering Behaviour void setbuf ( FILE * stream, char * buf ); setbuf turns buffering on or off. It may be implemented similar to: /* 0 if OK else EOF ( but stream is still functional ) */ setvbuf ( FILE * stream, char * buf, mode, size_t size ); setvbuf is used to alter the buffering behavior of a stream. May only be used after sucessful open and before first I/O. mode must be one of the following three macros: /* / usr / src / lib / libc/ stdio / setbuf.c */ void setbuf ( FILE *fp, char * buf ) { ( void) setvbuf (fp, buf, buf? _IOFBF : _IONBF, BUFSIZ ); } # define _IOFBF 0 /* setvbuf should set fully buffered */ # define _IOLBF 1 /* setvbuf should set line buffered */ # define _IONBF 2 /* setvbuf should set unbuffered */ For an unbuffered stream, buf and size are ignored. For line or fully buffered streams buf and size can optionally specify a buffer and its size. If buf is NULL the system chooses an apt size System-dependent: BUFSIZE (stdio.h), st blksize (stat.h)

15 Buffer options overview The setbuf and setvbuf functions and their options: Function mode buf Buffer & length Type of buffering setbuf setvbuf nonnull user buf of length BUFSIZ fully buffered or line buffered NULL (no buffer) unbuffered IOFBF nonnull user buf of length size NULL system buffer of apt length fully buffered IOLBF nonnull user buf of length size NULL system buffer of apt length line buffered IONBF (ignored) (no buffer) unbuffered Standard I/O routines fopen(3) a stream Opening a Stream Modes to fopen(3) a stream Modes to Open a Standard I/O Stream FILE * fopen ( const char *path, const char * mode ); FILE * freopen ( const char * path, const char * mode, FILE * stream ); FILE * /* all: fpoer if OK, NULL on failure with errno */ fdopen ( fildes, const char * mode ); fopen opens a specified file freopen opens a specified file on a specified stream. The original stream (if it exists) is always closed. Change the file associated with stderr, stdin, stdout. fdopen is part of Posix.1 not ISO C, as standard I/O does not deal with file descriptors. ISO C specifies 15 values for opening a standard I/O stream: mode Description r or rb open for reading w or wb truncate to 0 length or create for writing a or ab append; open for writing at end of file, or create for writing r+ or r+b or rb+ open for reading and writing w+ or w+b or wb+ truncate to 0 length or create for reading and writing a+ or a+b or ab+ open or create for reading and writing at end of file Using b allows to differentiate between text and binary files. UNIX kernels do not differentiate between these types of files it has no effect. With fdopen, the meaning of mode differs slightly

16 Remarks on fopen(3) Final Remarks on Opening a Stream fflush(3) a stream Six different ways to open a standard I/O stream: Restriction r w a r+ w+ a+ file must already exist x x previous contents of file discarded x x stream can be read x x x x stream can be written x x x x x stream can be written only at end x x Creating a new file with mode w or a, there is no way to specify file s access permission bits, as with open(2). Any created files will have mode S IRUSR S IWUSR S IRGRP S IWGRP S IROTH S IWOTH (0666). With a file opened for reading and writing (+ sign in mode) reads and writes cannot be arbitrarily ermixed. At any time, a stream can be flushed: /* 0 if OK, EOF on failure and errno set */ fflush ( FILE * stream ); Any unwritten data for the stream is passed to the kernel. If stream is NULL, all output streams are flushed. Output shall not be directly followed by input without an ervening fflush. Input shall not be followed by output without repositioning fclose(3) a stream Closing a stream with fclose(3) Reading and writing a stream Reading and Writing a Stream Once opened a stream there are three different types of I/O: /* 0 if OK, else EOF / errno ( no further access) */ fclose ( FILE * stream ); Character-at-a-time I/O. Read and write one character at a time, with the standard I/O functions handling all the buffering (if the stream is buffered). An open stream is closed by calling fclose. Any buffered output data is flushed before the file is closed. Any buffered input data is discarded. Any automatically allocated buffers are released. When a process terminates normally (calling exit or returning from main), all open standard I/O streams are closed. Line-at-a-time I/O. To read or write a line at a time, we use fgets(3) and fputs(3). Each line is terminated with a newline character, and we have to specify the maximum line length we can handle. Direct I/O 36. Provided by fread(3) and fwrite(3). For each operation we read or write some number of objects, where each object is of specified size. These types of I/O are refered to as unformatted I/O. Formatted I/O is done by functions, such as prf or scanf. 36 aka binary I/O, object-at-a-time I/O, record/structure-oriented I/O

17 Character-at-a-time input functions Character-at-a-time Input Functions Character-at-a-time I/O fgetc ( FILE * stream ); getc ( FILE * stream ); /* equivalent to getc() with the argument stdin. */ getchar ( void); Return the next requested object from the stream. Next character as an unsigned char converted to. The input functions return the same value* whether an error occurs or EOF (feof and ferror are used to distinguish). *EOF Push back characters Push-Back Characters Character-at-a-time output functions Character-at-a-time Output Functions /* c if OK, EOF on failure */ ungetc ( c, FILE * stream ); fputc ( c, FILE * stream ); Characters pushed back return by subsequent reads on the stream in reverse order of their pushing (FILO). One character of push-back is guaranteed, but as long as there is sufficient memory, an effectively infinite amount of pushback is allowed. If a character is successfully pushed-back, the end-of-file indicator for the stream is cleared. Pushing back EOF will fail and the stream remains unchanged. Pushed characters don t get written back to file or device. They are kept incore. putc ( c, FILE * stream ); /* All : c if OK, EOF / errno on failure */ putchar ( c); The functions write the character c (converted to an unsigned char) to the output stream. EOF is returned if a write error occures, or if an attempt is made to write a read-only stream

18 Line-at-a-time input functions Line-at-a-time Input Functions char * fgets ( char *str, size, FILE * stream ); Line-at-a-time I/O char * /* should NEVER be used->unknown buffer size */ gets ( char * str ); /* Both return str if OK, NULL on EOF or error */ str specifies the address of the buffer to read the line o. gets reads from stdin and fgets from stream. With fgets the size of the buffer is specified. The buffer is always null-terminated, i.e., at most size 1 is read. If the line is longer, a partial line is returned. The next call will read what follows Line-at-a-time output functions Line-at-a-time Output Functions /* 0 on success and EOF on error */ fputs ( const char *str, FILE * stream ); puts ( const char * str ); /* >=0 on success and EOF or error */ Direct I/O fputs writes the string poed to by str to the stream poed to by stream. puts writes the string str, and a terminating newline character, to the stream stdout

19 Binary I/O Binary I/O Reading and writing using binary I/O Binary I/O Functions Motivation for binary I/O size_t fread ( void *ptr, size_t size, size_t nmemb, FILE * stream ); Read or write an entire structure at a time. With character-at-a-time functions, such as getc or putc we have to loop through an entire structure. Line-at-a-time functions will not work. fputs stops writing when it hits a null byte. fgets won t work right on input with null or newline bytes. size_t fwrite ( const void * ptr, size_t size, size_t nmemb, FILE * stream ); /* Return number of objects read or written */ fread reads nmemb objects, each size bytes long. Input is taken from stream and stored at the location ptr. Both return number of objects read or written. For read it can be less than nmemb if error occurs or EOF. 37 For write an error has occured if it is not equal to nmemb. 37 ferror and feof must be called to determine Writing an array Binary I/O Write an Array Writing a structure Binary I/O Write a Structure The functions have two common cases: Read or write a binary array float data [10]; if ( fwrite (& data [2], sizeof( float ), 4, fp)!= 4) err (1, " fwrite error."); size as the size of each element of the array. Read or write a structure struct tuple { unsigned size ; unsigned level ; enum kind kind ; void * cnt ; } tup ; if ( fwrite (& tup, sizeof( tup ), 1, fp)!= 1) err (1, " fwrite error."); nmemb as the number of elements. size as the size of structure. nmemb as one (the number of objects to write)

20 Problems with binary I/O Fundamental Problems with Binary I/O Positioning a stream Positioning a Stream Binary formats change between compilers and architectures Binary formats used to store multibyte egers and floating-po values differ among machine architectures. The offset of a member within a structure can differ between compilers and systems. Even on a single system, the binary layout of a structure can differ, depending on compiler options. To exchanging binary data among different systems a higher-level protocol is probably the better choice. There are three ways to position a standard I/O stream: ftell and fseek. File position stored as long. (Historic) ftello and fseeko. File position stored as off t. (SUSv3) fgetpos and fsetpos. File position stored as fpos t. (ISO C) They work similar to lseek(2) and the whence options (SEEK SET etc.) are the same Check stream status Check Stream Status Obtaining a file descriptor Obtaining a File Descriptor /* non- zero if it is set */ feof ( FILE * stream ); /* non- zero if it is set */ ferror ( FILE * stream ); /* file descriptor assoc. with the stream */ fileno ( FILE * stream ); clearerr ( FILE * stream ); On UNIX, the standard I/O library ends up calling the low-level I/O routines. Each standard I/O stream has an associated file descriptor. Most implementations have two flags for each stream in FILE. fileno can obtain the descriptor (SUSv3, not ISO C). An error flag. An end-of-file flag. Both flags are cleared by clearerr

21 Standard I/O Example Review On slide: I/O system calls - Tracing hello world write(2) is invoked without previous call to open(2). How so? Why does the following code yield exit status 13?* * on some systems, e.g., Linux in the CIP pool 128