Outline. OS Interface to Devices. System Input/Output. CSCI 4061 Introduction to Operating Systems. System I/O and Files. Instructor: Abhishek Chandra

Transcription

1 Outline CSCI 6 Introduction to Operating Systems System I/O and Files File I/O operations File Descriptors and redirection Pipes and FIFOs Instructor: Abhishek Chandra 2 System Input/Output Hardware devices: Keyboard, monitor, printer, touchscreen Hard disk, flash drive, network card Programs access devices through the OS Device drivers: Special kernel programs that control devices Hide complexity of device Protect device from unauthorized access OS Interface to Devices All I/O is basically done through device drivers OS provides system call interface to access device drivers In some Operating Systems, the interface could differ from device to device E.g.: different for keyboard vs. disk Is this a problem?

2 Unix I/O: Files All devices are mapped to files Everything is a file Uniform interface for I/O: A single set of system calls (open, close, read, write) with some additional ones Actual implementation of the system calls depends on device type Benefits: Hide the complexity of actual device properties Programmers need to learn only a few system calls Unix File Types Regular files: Collection of data stored on disk Device files: Devices mapped onto files Pipes and FIFOs: Special files used for datasharing across processes Directories: Collection of files Symbolic Links: Pointer to another file Sockets: Files used for network communication 5 6 Device Files (/dev/*) Block special file: Device from which we can access blocks of data Multiple bytes, random access E.g.: disk (/dev/hda), CD-ROM (/dev/cdrom) Character special file: Device from which we can access stream of data Bytes-in-sequence, sequential access E.g.: Terminals (/dev/tty, /dev/pty) /dev/null: Black hole file (no input/output) Use if want to throw away output E.g.: myprog > /dev/null File I/O Operations Open a file Read/write to/from the file Move around in the file if required Close the file when done 7 8 2

3 File I/O System Calls open read write close lseek (for regular/block device files) There are additional system calls for each file type E.g.: mkdir, chmod for directories E.g.: connect, accept for sockets Opening a File: open int open(char *path, int oflag,...); Opens a file or creates a new file Returns a file descriptor Handle to the file Used by process to identify the file Parameters path: name of file to be opened oflag: Mode of opening the file O_RDONLY, O_WRONLY, O_RDWR Additional flags: Combine using OR operation ( ) 9 Reading from a File: read ssize_t read(int fd, void *buf, size_t nbytes); Reads data from the current offset in the file Parameters fd: file descriptor of file buf: buffer into which data is to be read Should have been allocated DO NOT pass NULL or unallocated buffer nbytes: number of bytes to read read is typically a blocking system call Return value of read ssize_t read(int fd, void *buf, size_t nbytes); Number of actual bytes read Could be less than number of bytes requested End-of-file reached Character devices: line-by-line reading Sockets: Network buffering : End-of-file, could be device-dependent -: Error 2

4 Reading from a File: read char *buf= malloc(nbytes*sizeof(char)); size_t nbytes, bytes_read; bytes_read=read(fd, buf, nbytes); if (bytes_read <= ) /* Error or EOF */ handle_error_or_eof(); else if (bytes_read < nbytes) /* Try to read remaining bytes */ bytes_read=read(fd, buf+bytes_read, nbytes-bytes_read); Writing to a File: write ssize_t write(int fd, void *buf, size_t nbytes); Writes data to the current offset in the file Parameters fd: file descriptor of file buf: buffer from which data is to be written nbytes: number of bytes to write Returns number of bytes written Typically equal to the number of bytes passed Otherwise, possible error (disk full, kernel buffer full, etc.) Closing a File: close int close(int fd); Closes an open file Releases resources associated with the file: file descriptors, other kernel data if last close of file Deletes file if marked for deletion and last close of file Parameter: file descriptor to close Returns: if successful - if error Setting Offset in a File: lseek off_t lseek(int fd, off_t offset, int whence); Current file offset: Position in file where next byte would be read from or written to when file is opened, or end-of-file if opened in append mode Returns: New file offset - if error 5 6

5 Setting Offset in a File: lseek Changing Offset in a File off_t lseek(int fd, off_t offset, int whence); Parameters: fd: file descriptor offset: number of bytes to skip whence: where to skip from SEEK_SET: from beginning of file SEEK_CUR: from current position SEEK_END: from end of file open read(..., ) lseek(..., ) read(..., ) EOF Buffering All reads/writes eventually go to the disk or I/O device What if we write byte at a time? Will have to go to the disk for every byte Highly inefficient Solution: Buffer a certain number of bytes before reading/writing them Can be done in OS or in user space (e.g., through a library) Buffering Policies Depends on device and file type Fully Buffered Data read/written only when whole buffer is filled E.g.: Disk files (buffered in chunks of blocks) Line buffered Data written on newline E.g.: Terminal I/O, stdin, stdout Unbuffered Data read/written immediately E.g.: stderr Device files: Depends on internal device/driver buffer 9 2 5

6 Buffering example printf( A ); printf( B ); fprintf(stderr, Z ); printf( C\n ); fprintf(stderr, Y ); printf( D\n ); fprintf(stderr, X ); In what order do the prints happen? Forced Buffer Cleanup: fflush Forces writing of any unwritten data in a buffer printf( A ); fflush(stdout); printf( B ); fprintf(stderr, Z ); printf( C\n ); fprintf(stderr, Y ); printf( D\n ); fprintf(stderr, X ); 2 22 Buffering: Benefits and Limitations Benefits: Improves I/O efficiency Need fewer data copies, device accesses Limitations: Could have unintended consequences E.g.: Out of order prints Might result in loss of data File Descriptors Identifier returned by the operating system on opening a file All operations performed on file descriptors Each process has a file descriptor table Contains currently open file descriptors Opening a file adds a new entry to the table Closing a file removes its entry from the table 2 2 6

7 File Descriptor Table Standard I/O Close(); fd=open(file,...); fd2=open(file2,...); fd=open(file,...); 2 5 stdin, stdout, stderr File descriptors, and 2 STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO Always open by default for each process Standard I/O is just like reading/writing to a file File Descriptor Implementation System File Table Proc A 2 System File Table In-memory inode-table file2 Shared kernel data structure Has an entry for each active call to open Each entry contains Pointer to inode-table entry for file File offset file Access mode Count of no. of fds pointing to it Proc B

8 Inode Table Opening a File Shared kernel data structure Contains an entry for each open file Each entry contains Information about file type, functions Proc A 2 System File Table In-memory inode-table file2 inode for the file: file information, data location, etc. Count of no. of system table entries pointing to it file Proc B 2 open( file, ) 29 Inheriting File Descriptors File Descriptors after fork After fork(), all file descriptors are copied to the new process Each process has identical file-descriptor table Each process has the same files open The file offset is the same within each file for both processes Each file descriptor points to the same sys-table entry Reads/writes/lseeks are shared by the processes parent 2 child 2 System File Table 2 8

9 File Redirection Redirect the standard input or output to a file Input redirection (<): Takes input from a file instead of keyboard sort < file Output redirection (>): Send output to a file instead of display ls -l > file Duplicating File Descriptors: dup int dup(int fd); Creates a copy of fd Returns new fd where it is copied Lowest available entry in the file descriptor table 2 newfd=dup(); fd table system file table Duplicating File Descriptors: dup2 int dup2(int fd, int fd2); Creates a copy of fd onto fd2 Closes fd2 if open File Redirection How does a process redirect intput/output to a file instead of standard I/O? E.g.: how do we do: sort < file newfd=dup2(,2); 2 fd table system file table 5 6 9

10 File Redirection Example /* Open the desired file*/ fd=open(file,...); /* Duplicate fd onto stdin */ newfd=dup2(fd, STDIN_FILENO); /* Now exec desired command (e.g., sort) */ 2 file Pipes prog prog2 Allow multiple processes to be linked together Connects output of prog to input of prog2 Examples: ls -l more cat foo sort head 7 fd table system file table 8 What are Pipes? Pipes are special files Pipes are a mechanism for inter-process communication (IPC) Allow processes to communicate and share data Other IPC mechanisms: sockets, shared memory, semaphores, etc. A pipe provides a serial data channel between two processes Relatively simple IPC mechanism Channel is half-duplex (one-way) How do Pipes Differ from Regular Files? Sequential read/write Nameless Finite-sized memory buffer No disk I/O involved Sharing has to be done through inheritance Disappear as soon as all their ends are closed 9

11 Using Pipes: Overview Create a pipe: Returns two fds Share between processes using fork() Close one end of the pipe in each process One process would read, the other would write Read and write data through the pipe Close the pipe when done Creating a Pipe: pipe int pipe(int fds[2]); Creates a pipe and returns its two ends in an array of two fds fds[]: for reading fds[]: for writing Output of fds[] is input of fds[] 2 Creating a Pipe: pipe int pipe(int fds[2]); Process fds[] fds[] Sharing a Pipe across Processes The only way to share a pipe is via fork() After fork(), both processes get copies of the pipe fds Parent fork Child fds[] fds[] fds[] fds[] Pipe Pipe

12 Pipe I/O: Initialization Pipe is used to channel data from one process to another One process would read from pipe Another process would write to file Each process closes one end of the pipe Writing process closes read-end Reading process closes write-end Pipe I/O: Initialization Suppose the parent wants to read from the pipe and the child wants to write to the pipe Parent fds[] fds[] x Child fds[] fds[] x Pipe 5 6 Replacing Standard I/O with Pipe Example: ls l sort Use dup2 to replace stdin or stdout Replace stdin with read-end of pipe (fds[]) Replace stdout with write-end of pipe (fds[]) Parent Child fds[] fds[] x x Pipe Pipe I/O: Data Sharing Reading: Done from read-end of pipe Returns whatever data is in the pipe Blocks if pipe is empty Returns EOF if write-end of pipe is closed Writing: Done from write-end of pipe Writes into the pipe buffer Blocks if pipe is full (finite buffer) Receives SIGPIPE signal if read-end of pipe is closed No lseek (treat as character device file) 7 8 2

13 FIFOs Named pipes Unidirectional shared channels like pipes Have name and path like a regular file Persistent even when both ends are closed Benefits: Can link unrelated processes Longer lasting than pipes I/O and Files Summary File I/O operations File descriptors and redirection Pipes and FIFOs 9 5