CSCI 503: Operating Systems File System (Chapters 16 and 17) Fengguang Song Department of Computer & Information Science IUPUI File System (FS) Highlights File system is the most visible part of OS From inception, Unix has been built around two entities: Process and File Users use File System to access data and programs no need to know where and how a file is stored A file system has two distinct parts: Files Directory 2 1
Files and Directories File is a logical storage unit an abstraction for physical storage devices It is simply a linear array of bytes //logically Two common types of files: Data numerical character binary Program Directory is the second abstraction to organize files Each directory entry: <file name, ID or a pointer> Why we need it: Used to organize data Used to permit data sharing among users Provides mechanisms for protection 3 File Attributes Name the only information kept in human-readable form ID A unique number within file system Type needed for some systems that support different types Location pointer to the file s location on device Size current file size Protection controls who can do reading, writing, executing Time, date, and user identification for protection, security, and usage monitoring Information about files are kept in the directory structure, which is stored on disk. Try to run the stat <filename> command yourself 4 2
How to Get File Attributes System Call: fstat(intfd, struct stat* buf); struct stat { dev_t st_dev; /* ID of device containing file */ ino_t st_ino; /* inode number */ mode_t st_mode; /* protection */ nlink_t st_nlink; /* number of hard links */ uid_t st_uid; /* user ID of owner */ gid_t st_gid; /* group ID of owner */ dev_t st_rdev; /* device ID (if special file) */ off_t st_size; /* total size, in bytes */ blksize_t st_blksize; /* blocksize for file system I/O */ blkcnt_t st_blocks; /* number of 512B blocks allocated */ time_t st_atime; /* time of last access */ time_t st_mtime; /* time of last modification */ time_t st_ctime; /* time of last status change */ }; 5 File Access Methods Sequential Access One byte at a time, in order, until the end //e.g., tape Direct Access Ability to position anywhere in the file Position to block # Block number is relative to the beginning of the file Indexed Access A file contains an index to its various data blocks So we can first search the index, and use the pointer to access the file directly. Linux and Windows: Both support direct access methods But internally their file system uses indexing of blocks 6 3
Disk Structure Files and Directories are stored in disks But how they are physically stored? A disk is divided into multiple logical partitions It s OK to use the entire disk as 1 partition Each partition holds an individual file system Which is called volume Each volume tracks its FS info in its device directory 7 Revisiting Disks 8 4
An Example File-System Organization There are 3 partitions from 3 disks: A, B, C File system: directory + files Two volumes created in 1 disk Third volume created over 2 disks 9 What is Directory Directory can be viewed as a symbol table Translates file names to their File Control Blocks File Name i-node Number Directory is a set of nodes (maybe in a certain hierarchy), containing information about files One-level Directory Files F 1 F 2 F 3 F 4 F n Both the directory structure and the files reside on disk 10 5
Tree-Structured Directories 11 Tree-Structured Directories More general than 1-level and 2-level directory structures Each directory can contain either files or sub-directories Each running process has a current directory ($ pwd) There are two types of path name: Relative path Absolute path Path name translation, e.g., /one/two/three Open / (file system knows where it is on disk) Search for one and get its FCB on disk Open one, search it for two and get its FCB on disk Open two, search it for three and get its FCB on disk Open three The OS spends a lot of time walking directory paths This is the reason why one separates open from read/write OS attempts to cache common path prefix Q: Can you share a file/directory between two users? 12 /: <bin, id> <usr, id> <tmp, id> <one, id> 6
Acyclic-Graph Directories Support shared subdirectories and files By creating links 13 File System Mounting A file system is existing on disks (with or without power) However, it must be mounted before it can be accessed Each file system is mounted at a special location, called mount point When given a mount point and a volume, OS then 1) Asks the device driver to read the device directory 2) Checks that the volume does contain a valid file system 3) Makes note of this new file system at the specified mount point i.e., OS keeps a list of mount points Mac OS X: All volumes are mounted in the /volumes/directory also including temporary volumes on USB keys, CDs, etc. UNIX: volumes can be mounted anywhere e.g., on Linux, command mount lists all the mounted volumes 15 7
Mount Point / users bill fred sue jane help prog doc (a) Existing file system (b) Un-mounted volume 16 File Access Mode 3 classes of users on Unix: owner, group, public 3 access rights: read, write, execute RWX a) owner access 7 Þ 1 1 1 RWX b) group access 6 Þ 1 1 0 RWX c) public access 1 Þ 0 0 1 owner group public chmod 761 game Ask manager to create a group, say G and add some users to the group. Attach a group to a file: chgrp G game 17 8
File System Implementation 18 Implementation of File System There are On-disk File System structures: Boot control block Usually the 1st block of a volume (stores bootloader) Contains info needed by system to boot OS from that volume UFS: boot block ; NTFS: partition boot sector ; Volume control block Contains volume details: total # of blocks, # of free blocks, block size, free block pointers, FCBs UFS: superblock ; NTFS: master file table There are In-memory File System structures: A mount table (one entry point per mounted volume) A global open-file table, a per-process open-file table Buffers for holding disk blocks //performance Directory cache for path translation //performance 9
Virtual File System Virtual File Systems (VFS) is an object-oriented way to implement file systems (like C++ virtual methods) //Software Engineering Separate generic operations from implementation details Implementation can be one of many file system types Dispatch operation to appropriate FS implementation routines Allow the same syscall interface to be used by different FS types The API is to the VFS interface, rather than any specific FS 20 Disk Block Allocation Problem: How to allocate disk blocks to files? Simplest one: Contiguous allocation Each file uses a number of contiguous disk blocks Best performance Simple to implement Only remember start-block# and a length Problems? How to find contiguous free space (policies) Too slow (why?) External fragmentation Need defragmentation Know the file size in advance (why?) What if a file grows? Solutions: -Move to a bigger hole -Ask users to specify maximum size -Use a linked list (aka extent) 22 10
Linked Allocation Linked allocation Each file is a linked list of blocks: blocks may be scattered anywhere on the disk Each block contains pointer to next block Ends at a NULL pointer Advantages: No external fragmentation (yeah!) Can grow dynamically Can further improve efficiency by clustering blocks into groups Problems? Overhead of pointers (e.g., 4/512) Reliability If a pointer is damaged, then the entire file is unrecoverable Locating a block can take many I/Os and disk seeks. Great for sequential access, but not good for direct access Note: The block# has typos. 23 Indexed Allocation (1/4) Indexed allocation All the previous block links are brought to a single table Each file has its own index block of pointers (pointing to data blocks) Similar to page table Here it is called Index Table Supports random access, no external fragmentation! Problem: Each block has 512 bytes, how many entries can a single block store?» Then, how to use multiple blocks to store the index table? 24 11
Indexed Allocation (2/4) Mapping from logical to physical in a file of unbounded length 1) Linked scheme Link a number of index tables (no limit on size) e.g., the last word in the first index table is the address of the next index table Q 1 = which index table R 1 is used as follows: LA / (512 x 127) Q 1 R 1 tbl tbl tbl 512B 512B 512B R 1 / 512 Q 2 tbl R 2 Q 2 = displacement into index table R 2 = displacement into block of file: 25 Index Allocation (3/4) 2) Multilevel scheme: Similar to a multi-level page table If we have 512-byte blocks, and 4-byte pointers, then we could store 128 entries (=2 7 ) in a block A 3-level scheme can then allow for 1GB files Why? 7bits + 7 bits + 7 bits --> blocks A 4-level scheme can then allow for 128GB files! outer-index index table file 26 2-level scheme 12
Index Allocation (4/4) 3) Combined scheme: For small to medium-sized file it seems a waste to keep multi-level index table How about keeping all options open? A few pointers to actual disk blocks A pointer to a single-level index A pointer to a two-level index A pointer to a three-level index That way, small files don t use multi-level index table 27 Combined Scheme: UNIX File System Block size = 512 bytes Pointer = 4 bytes Max file size = 12*512 + + 128*512 + 128 2 *512 + 128 3 *512 bytes (12 pointers) Note: The block size on Unix is 4096 bytes, instead of 512 bytes. The Max File Size is # terabytes. i-node 28 13
inode An inode is used to describe both a file and a directory A bit tells whether it s file or directory Each file has an inode (shown in the previous slide) Each directory also has an inode, which points to its blocks Those data blocks are just <name, inode#> pairs Those data blocks will be searched for names when doing pathname resolution e.g., /home/user/hello.c OS keeps an in-memory cache of recent pathname resolutions (for better performance) 29 Network File System (NFS) A client-server approach Servers can share their file systems to other hosts A host can mount a remote file system into its local directory Need the server name and the remote directory name E.g., $ mount master.foo.com:/home /mnt/home Once this network mounting is complete, access to the remote file system is completely transparent Common example: mount NFSServer:/home on all workstations of a LAN so, a user can log in anywhere, and see /home/username/ NFS specifies two protocols: NFS mount protocol (for server) NFS file access protocol (for client) Implemented as RPCs Running NFS daemons on servers that execute RPCs Note: It is Not part of the Kernel 30 14
NFS Mount Protocol Each NFS server maintains an export list A list of directories that can be accessed, and machines that can mount them Stored in a file It also maintains a list of who has mounted file systems So they can be notified of events 31 Schematic View of NFS Architecture 33 15