Chapter 11: File System Implementation File System Structure File System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery Log-Structured File Systems NFS 1
File-System Structure - File system is the most visible part of the OS to the user - Disks provide the bulk of secondary storage to build File System - FS provides an abstract view of the secondary storage physical view File system resides on secondary storage (disks) Provides efficient and convenient access to disk by allowing data to be stored, located and retrieved easily File structure def n : Collection of related information Logical storage unit File system organized into layers (next slide) 2
Layered File System Use file system interface provided by logical file system: create; open; share; read/write MetaData: FCB; protection/security; data Logical file mapto physical blocks (transformation); free-space mng Device drivers (INT handlers): Uses: Disk drive; cylinders; sectors Read blk 123 ---> cntrl registers. OS primitives: Read / Write physical blocks Using: disk block Numbers Physical devices: Hard disk; CD ROM; Floppy disk; DVD 3
FCB: File Control Block contains important information about a file 4
On-Disk / In-Memory File System Structures On disk information:" Boot control block: Boot Block/Sector" Partition control block: Superblock, Master File Table" Directory structure (next slide)" File control block (FCB, Inode): " Blk size Blk count Free blks Ptrs to free blks FCB count Ptrs to FCB In memory information:" Partition table: mounted partitions" Directory structure: accessed directories; directory as partition" System wide open-file table" Per-process open-file table 5
A UNIX directory tree Hierarchical Directory Structure Each dir entry = [path-name component, I-number] I-number: pointer to Inode Inode = FCB File-blocks Directory I-number -> I-node -> file-blocks /usr/jim/ Full Path Name /usr/jim/ File /usr/jim/ - Path name translation needs to access 3 Files (directories) - Cache the found directory and its I-number for future access 6
In-Memory File System Structures Open a file (on disk) (In memory) Read a file 7
Example: file system tables 8
Example: file system table (solution) 9
Virtual File Systems Virtual File Systems (VFS) provide an objectoriented way of implementing file systems. VFS allows the same system call interface (the API) to be used for different types of file systems. The API is to the VFS interface, rather than any specific type of file system. 10
Schematic View of Virtual File System Provides a uniform view of the entire file system consisting of different types of file systems 11
Directory Implementation Selection of directory-allocation and management algorithms significantly affects the efficiency, performance, and reliability of the file system Linear list array of file names with pointer to the file data blocks. Simple to program Time-consuming to execute. Create/delete of files/dirs needs searching the whole list. To reuse the entry: Mark as unused; list of free entries; swap with last entry. Hash Table linear list with hash data structure. Decreases directory search time Collisions situations where two file names hash to the same location Fixed size Linear Probing & Chained Overflow hash tables http://www.cl.cam.ac.uk/~cwc22/hashtable/ 12
Hash tables 67 % 20 = 7 46 % 20 = 6 88 % 20 = 8 91 % 20 = 11 734 % 20 = 14 67 % 10 = 7 46 % 10 = 6 88 % 10 = 8 91 % 10 = 1 734 % 10 = 4 Only Key, Value not shown. (Key, Value) pairs. 13
Hash Table Example of Telephone Directory Hash collision resolved by chaining Hash collision resolved by linear probing (interval = 1) 14
Allocation Methods An allocation method refers to how disk blocks are allocated for files: Contiguous allocation Linked allocation Indexed allocation 15
Contiguous Allocation Each file occupies a set of contiguous blocks on the disk. Simple only starting location (block #) and length (number of blocks) are required. Random access. Wasteful of space (dynamic storage-allocation problem). Files cannot grow. External fragmentation: Regular compaction operation is time consuming 16
Contiguous Allocation of Disk Space 17
Linked Allocation Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk. Starting Block # myfile pointer" Disk block " pointer" pointer" DATA DATA DATA 18
Linked Allocation (Cont.) Simple need only starting address Free-space management system no waste of space No random access Mapping File-allocation table (FAT) disk-space allocation used by MS-DOS and OS/2." 19
Linked Allocation FCB 20
File-Allocation Table (FAT) 21
Indexed Allocation Brings all pointers together into the index block. Logical view. index table" 22
Example of Indexed Allocation FCB 23
Indexed Allocation (Cont.) Need index table Random access Dynamic access without external fragmentation, but have overhead of index block. Mapping from logical to physical in a file of maximum size of 256K words and block size of 512 words. We need only 1 block for index table. Assume each pointer to disk-block requires one word. 256K words 256 (2*512) 512 * 512 words 512 blocks requires 512 pointers = one 1 block for index table 24
Two-level index (maximum file size is 512 3 ). Mapping from logical to physical in a file of unbounded length (block size of 512 words). Linked scheme Link blocks of index table (no limit on size). Indirect index-block Pointer to Index-block " Direct index-block index table" file" 25
Combined Scheme: UNIX (4K bytes per block) 26
Figure below illustrates the block diagram of a file system and the tables system wide open-file table and per-process open-file table. Two processes perform open file commands to open two instances of files foo and bar and perform operations on these two files. Answer to the following questions: a. Which of the above parts are in hard-disk and which ones are in memory? b. What is the purpose for Super block and what are its contents? c. By drawing arrows and labeling the arrows show how the values of four file descriptors fd1, fd2, fd3, and fd4 are determined, and write the values below. d. What are the contents of entries in System-wide open-file table and per-process open-file table? Suppose Process 1 is a producer and Process 2 is a consumer. Specify the sequence of instructions (only those shown above) and use the file descriptors you obtained above such that Process 1 passes 300 bytes to Process 2 (in the figure rc is read-count and wc is write-count). 27
fd1: 4, fd2: 5, fd3: 3, fd4: 4 Process 1: Put 300 bytes from address space of Process 1 into Buffer starting from the first entry of the Buffer. fd2 = open(/../../bar, Read-and-write); // fd2=5 wc = write(fd2, address of first entry of Buffer, 300); // wc should be 300 if wc!= 300 then Error( couldn t write 300 bytes into file bar ) close(fd2) /// this line is optional for this question.. Process 2: fd3 = open(/../../bar, Read-only); // fd3 = 3 rc = read(fd3, address of first entry of Buffer, 300); // rc should be 300 if rc!= 300 then Error ( couldn t read 300 bytes from file bar into Buffer) Get 300 bytes from Buffer starting from the first entry of Buffer and move them into the address space of Process 2. close(fd3); /// this line is optional for this question 1) Process 1 puts 300 bytes from the address space of Process 1 into the Buffer, starting from first entry of the Buffer. 2) The OS allocates 3 free blocks (2 x 128 =256 < 300 < 3 x 128 = 384) to put 300 bytes from Buffer. 3) The OS assigns these three blocks as the file blocks of file bar. Depending on the file allocation mechanism of OS, this can be contiguous allocation, Indexed allocation, or linked allocation. 28
Free-Space Management Disk Space is limited and we need to reuse the space from deleted files for new files Bit vector (n blocks) 0" 1" 2" n-1" " bit[i] =" " 0 block[i] occupied" 1 block[i] free" Block number calculation" (number of bits per word) *" (number of 0-value words) +" offset of first 1 bit" 29
Free-Space Management (Cont.) Bit map requires extra space. Example: block size = 4KB = 2 12 bytes disk size = 2 30 bytes (1 gigabyte) n = 2 30 /2 12 = 2 18 bits (or 32K bytes) Easy to get contiguous files Linked list (free list) Cannot get contiguous space easily No waste of space Grouping Counting 30
Free-Space Management (Cont.) Need to protect: Pointer to free list Bit map Must be kept on disk Copy in memory and disk may differ. Cannot allow for block[i] to have a situation where bit[i] = 0 in memory and bit[i] = 1 on disk. Solution: Set bit[i] = 0 in disk. Allocate block[i] Set bit[i] = 0 in memory 31
Linked Free Space List on Disk 32
Simulated File System Project Disk Device interfaces: The whole file system is a file: simdisk.data 512 disk blocks (0 to 511), Each block is 128 bytes. Put_block(blk#, memloc): copies 128 bytes of data from memory to a disk block. get_block(blk#, memloc): does the reverse operation as put_block(..) Simdisk.data is created automatically (all bits zero) if not exist before. Otherwise, the existing one is used. File System: Unix-like hierarchical directory structure Two types of file: directory and regular Full path name of files/directories: /foo/bar/zam not bar/zam Path-name to file-name translation to get I-node number of the file/directory Use of I-node table and file-descriptor (one per-process table) to open a file several times (at least 4) At least 64 files to exist in system, 4 open files, each file up to 512 byte long. File name component max 6 characters. Appending to a file is by setting start location of the file to -1. This is the only allowable way to increase the length of a file. 33
Interface of Simulated File System (see sfstest.c for instruction on how to compile) sartipi% sfstest o: open a file r: read from a file w: write to a file R: read from a directory c: close a file m: create (make) a new file d: delete a file s: get the size of a file t: get the type of a file i: initialize the file system q: quit - exit this program Command? 34