CSE : Principles of Operating Systems Lecture File Systems February, 6 Before We Begin Read Chapters and (File Systems) Prof. Joe Pasquale Department of Computer Science and Engineering University of California, San Diego 6 by Joseph Pasquale Processes So Far, We Have Considered Abstraction of a running program Processing resource Virtual Memory Next Abstraction of physical short-term memory Memory resource File system: long-term storage Motivation for File Systems Long-term storage is needed for user data: text, graphics, images, audio, video user programs system programs (including kernel) and data Persistent: remains forever Large: unlimited size Sharing: controlled access Security: protecting information 4
Goals File System Support multiple types of storage Disks (different types), remote disks Simple intuitive user interface Single name space, simple naming, access control Request file by name, portion Respond with file contents file File System Archival storage Keep forever, including previous versions File System: a structured/organized collection of files Access control, name space, persistent storage High performance, high reliability, protection, security 5 File: a logical unit of storage Unstructured (e.g., array of bytes) Structured (e.g., tree, records) 6 File Naming Hierarchical Name Spaces Name space Space of possible names Flat: no structure Structured: name has components, organized Distinguish between user names and system names User: user-friendly, space managed by user System: efficient for system, managed by system Name space is organized as a tree Name has components, branches start from root Actually, not strictly a tree: links No size restrictions Intuitive for users to manage Example: UNIX pathnames Absolute: /a/b/c/d Relative: c/d relative to /a d c b e a d c e 7 8
A File has Attributes Basic File Operations Name (one or more) Type (recognized by system or users) Times: when created; last accessed; last modified Sizes: current size; maximum size Structure: linear; sequence of records; indexed records Access control (permissions) open ( filename ) return fd file x read (fd, buf, n) place n bytes in buf Creation: create, delete Prepare for Access: open, close Access: read, write (append) File System file x 9 Other File Operations Search: move to indexed location, find a keyed record Attributes: get, set (e.g., permissions) Mutual exclusion: lock, unlock Name management: rename Typical Usage create (filename, permissions) openfile = open (filename, usage) amountread = read (openfile, buffer, howmuch) amountwritten = write (openfile, buffer, howmuch) close (openfile) remove (filename)
Memory Mapped Files Map file into address space Can be total or partial Can then use memory r/w ops Works best in segmented VM Problems VAS Small File Large File What is the size of a file being written? What if file is to be shared by multiple processes? What if file exceeds size of address space? Access Control How are files shared without giving away total control? Access control Who can access file What operations are allowed User interface must be simple and intuitive Example: UNIX rwx permissions for owner, group, and everyone 4 Accessing System Objects Storage Abstraction for File System System objects need to be accessed by user Peripheral resources such as disks, networks Also OS objects: processes, memory segments Use file system interface Name space Access control Hide complexity of addressing of underlying storage device Model storage as array of randomly accessible blocks File system accesses the storage using simple model read (block_no) write (block_no) block block block block n-... 5 6
File System Structure File System Metadata File System Metadata Info about file system File Metadata File control blocks FSM FM Information about the file system Sizes Files in use, free entries Data blocks in use, free entries FSM FM Data Blocks Free lists (or bitmaps) File contents DB File control blocks Data blocks DB 7 8 File Control Block Allocation of Storage Space for Files Information about a file Name: may or may not be included Attributes: type of file, size (+ max), permissions References to data blocks: FSM FM DB Contiguous blocks Single group of blocks that are contiguous Extents Multiple groups of contiguous blocks Non-contiguous blocks Blocks are randomly located disk block map 9
Data Structures Block Map (UNIX) Start block pointer and size 5 Array of pointers to data blocks Map/Index table of extents table of blocks Linked lists list of blocks list of indexes pointers direct: references data blocks singly-indirect: references n data blocks doubly-indirect: references n data blocks triply-indirect: references n data blocks For data block of 4 bytes pointer requires 4 bytes: n = 56 max: ( + 56 + 56 + 56 ) * 4 = 6 GB Block Map (UNIX) Keeping Track of Free Blocks Data Blocks Free Block Map Compact if lots of free regions of space Block Map Ptrs to Double Indirect Blocks Triple Indirect Block Double Indirect Block Ptrs to Single Indirect Blocks Single Indirect Block Ptrs to Data blocks Direct: x KB = KB Single Indirect: 56 x KB = 56KB Double Indirect: 56 x 56 x KB = 64MB Triple Indirect: 56 x 56 x 56 x KB = 6GB Doubly Linked Easy to keep ordered due to fast inserts and deletes Bit Map Fixed size regardless of fragmentation 4
Implementing Directories where each entry contains name and attributes name and pointer to file control structure Unix (name and pointer) - pre BSD i-node number (), file name (4) Berkeley Unix uses a more complex scheme to support long names Example of Parsing Names in Unix Consider the file pathname: /sports/tennis/agassi root inode business entertain politics science sports 7 5 67 basketball golf inode tennis volleyball 6 5 44 inode 5 agassi chang sampras 88 9 46 inode 88 file data 5 6 The Big Picture File Systems Use Disks for Storage 4 File System Metadata File Metadata. Why disk storage: persistent, random access, cheap But, disks are slow, because they are mechanical sector disk r/w head(s) Block Map 7.. cat platters File Control Blocks ( Inode ) Data Blocks 4 6 contents of dog dog fish 7 track cylinder 8
File System Performance Disk accesses are expensive in time: 5- msec! Rotational latency: -6 msec (54-5 RPM) Seek time: - msec Transfer rate: + MB/sec Reduce accesses by reading multiple blocks in one access (read ahead) maintaining a block cache Cluster related blocks to reduce seek time Block Cache An in-memory cache of recently used blocks Upon read or write Check if block is in cache; if it is, done If not in cache, access disk, place in cache Eventually, cache will fill up; need replacement policy : keep a list ordered by access Remove old block at head of list Put new block at tail of list 9 Searching the Block Cache Searching the Block Cache Read (6) 6%4 = Search list for 6 Cache hit! 8 4 6 5 4 4 4 5 47 Read (6) 6%4 = Search list for 6 Cache hit! 8 4 6 5 4 4 4 5 47 7 59 Update 7 59 list
Searching the Block Cache Searching the Block Cache Read (47) 47%4 = Search list for 47 Cache miss! 8 4 6 5 4 4 4 5 47 Read (47) 47%4 = Search list for 47 Cache miss! 8 4 6 5 4 4 4 5 7 59 Retrieve 47 7 59 47 Update 4 Searching the Block Cache Searching the Block Cache Read (5) 5%4 = Search list for 5 Cache miss! 8 4 6 5 4 4 4 5 Read (4) 4%4 = Search list for 4 Cache miss! 8 4 6 5 4 4 5 7 59 47 Remove 7 59 47 Retrieve 4 5 6
File System Consistency Block cache reduces disk accesses, good performance But if sys crashes and mod blocks not written, data lost To improve file system consistency Write out modified blocks (how often?) Write out critical blocks (write-through) Critical blocks: file system meta-data Directories, i-nodes, free block lists Clustering of Related Blocks Some blocks expected to be accessed within short time Directory and files within that directory The inodes of the directory and the files Strategy Place related blocks close to each other: clustering Minimizes disk head movement: improves seek time 7 8 Caching of Other File System Info File system metadata (keep in memory) File metadata Currently active files Recently used Block maps File names Name to file metadata translations Disk Block Size What should the block size be? The larger the block, the better the throughput The smaller the block, the less wasted space Technology trends Disk density is increasing faster than disk speed Make disk blocks larger: KB -> 8 KB 9 4
Summary A file system is a collection of files A file is the basic unit of storage Disks are good for file system storage, but slow File systems use simple model of disk: array of blocks File system structure File system metadata, file metadata, data blocks Use caching to improve performance 4