CSE 120: Principles of Operating Systems Lecture 10 File Systems February 22, 2006 Prof. Joe Pasquale Department of Computer Science and Engineering University of California, San Diego 2006 by Joseph Pasquale 1
Before We Begin Read Chapters 10 and 11 (File Systems) 2
So Far, We Have Considered Processes Abstraction of a running program Processing resource Virtual Memory Abstraction of physical short-term memory Memory resource Next File system: long-term storage 3
Motivation for File Systems Long-term storage is needed for user data: text, graphics, images, audio, video user programs system programs (including kernel) and data Persistent: remains forever Large: unlimited size Sharing: controlled access Security: protecting information 4
Goals Support multiple types of storage Disks (different types), remote disks Simple intuitive user interface Single name space, simple naming, access control Archival storage Keep forever, including previous versions High performance, high reliability, protection, security 5
File System Request file by name, portion file Respond with file contents File System File System: a structured/organized collection of files Access control, name space, persistent storage File: a logical unit of storage Unstructured (e.g., array of bytes) Structured (e.g., tree, records) 6
File Naming Name space Space of possible names Flat: no structure Structured: name has components, organized Distinguish between user names and system names User: user-friendly, space managed by user System: efficient for system, managed by system 7
Hierarchical Name Spaces Name space is organized as a tree Name has components, branches start from root Actually, not strictly a tree: links No size restrictions a Intuitive for users to manage b c Example: UNIX pathnames Absolute: /a/b/c/d c d e Relative: c/d relative to /a d e 8
A File has Attributes Name (one or more) Type (recognized by system or users) Times: when created; last accessed; last modified Sizes: current size; maximum size Structure: linear; sequence of records; indexed records Access control (permissions) 9
Basic File Operations open ( filename ) return fd read (fd, buf, n) place n bytes in buf file x file x File System Creation: create, delete Prepare for Access: open, close Access: read, write (append) 10
Other File Operations Search: move to indexed location, find a keyed record Attributes: get, set (e.g., permissions) Mutual exclusion: lock, unlock Name management: rename 11
Typical Usage create (filename, permissions) openfile = open (filename, usage) amountread = read (openfile, buffer, howmuch) amountwritten = write (openfile, buffer, howmuch) close (openfile) remove (filename) 12
Memory Mapped Files Map file into address space VAS Can be total or partial Can then use memory r/w ops Works best in segmented VM Problems Small File Large File What is the size of a file being written? What if file is to be shared by multiple processes? What if file exceeds size of address space? 13
Access Control How are files shared without giving away total control? Access control Who can access file What operations are allowed User interface must be simple and intuitive Example: UNIX rwx permissions for owner, group, and everyone 14
Accessing System Objects System objects need to be accessed by user Peripheral resources such as disks, networks Also OS objects: processes, memory segments Use file system interface Name space Access control 15
Storage Abstraction for File System Hide complexity of addressing of underlying storage device Model storage as array of randomly accessible blocks File system accesses the storage using simple model read (block_no) write (block_no) block 0 block 1 block 2 block n-1... 16
File System Structure File System Metadata Info about file system File Metadata File control blocks Data Blocks File contents FSM FM DB 17
File System Metadata Information about the file system Sizes Files in use, free entries Data blocks in use, free entries FSM FM Free lists (or bitmaps) File control blocks Data blocks DB 18
File Control Block Information about a file Name: may or may not be included FSM FM Attributes: type of file, size (+ max), permissions References to data blocks: disk block map DB 19
Allocation of Storage Space for Files Contiguous blocks Single group of blocks that are contiguous Extents Multiple groups of contiguous blocks Non-contiguous blocks Blocks are randomly located 20
Data Structures Start block pointer and size 5 Map/Index table of extents table of blocks Linked lists 2 3 list of blocks list of indexes 21
Block Map (UNIX) Array of pointers to data blocks 13 pointers 10 direct: references 10 data blocks 1 singly-indirect: references n data blocks 1 doubly-indirect: references n 2 data blocks 1 triply-indirect: references n 3 data blocks For data block of 1024 bytes pointer requires 4 bytes: n = 256 max: (10 + 256 + 256 2 + 256 3 ) * 1024 = 16 GB 22
Block Map (UNIX) Data Blocks Block Map Triple Indirect Block Ptrs to Double Indirect Blocks Double Indirect Block Ptrs to Single Indirect Blocks Single Indirect Block Ptrs to Data blocks Direct: 10 x 1KB = 10KB Single Indirect: 256 x 1KB = 256KB Double Indirect: 256 x 256 x 1KB = 64MB Triple Indirect: 256 x 256 x 256 x 1KB = 16GB 23
Keeping Track of Free Blocks Free Block Map Compact if lots of free regions of space Doubly Linked List Easy to keep ordered due to fast inserts and deletes 1 3 2 1 3 2 Bit Map Fixed size regardless of fragmentation 101000100 24
Implementing Directories Table where each entry contains name and attributes name and pointer to file control structure Unix (name and pointer) - pre BSD i-node number (2), file name (14) Berkeley Unix uses a more complex scheme to support long names 25
Example of Parsing Names in Unix Consider the file pathname: /sports/tennis/agassi root inode inode 22 inode 15 inode 88 business entertain politics science sports 12 37 25 67 22 basketball golf tennis volleyball 62 21 15 44 agassi chang sampras 88 92 46 file data 26
The Big Picture 0 1 File System Metadata 2 3 File Metadata 4 2. Block Map 7 32.. cat File Control Blocks ( Inode Table ) Data Blocks 4 16 contents of dog dog fish 27
File Systems Use Disks for Storage Why disk storage: persistent, random access, cheap But, disks are slow, because they are mechanical sector disk r/w head(s) platters track cylinder 28
File System Performance Disk accesses are expensive in time: 5-20 msec! Rotational latency: 2-6 msec (5400-15000 RPM) Seek time: 3-13 msec Transfer rate: 100+ MB/sec Reduce accesses by reading multiple blocks in one access (read ahead) maintaining a block cache Cluster related blocks to reduce seek time 29
Block Cache An in-memory cache of recently used blocks Upon read or write Check if block is in cache; if it is, done If not in cache, access disk, place in cache Eventually, cache will fill up; need replacement policy LRU: keep a list ordered by access Remove old block at head of list Put new block at tail of list 30
Searching the Block Cache Read (36) 36%4 = 0 Search list 0 for 36 Cache hit! LRU List 0 1 2 Head Tail 80 24 36 25 41 13 30 34 42 15 47 3 27 59 Hash Table 31
Searching the Block Cache Read (36) 36%4 = 0 Search list 0 for 36 Cache hit! LRU List 0 1 2 Head Tail 80 24 36 25 41 13 30 34 42 15 47 Update LRU 3 27 59 list Hash Table 32
Searching the Block Cache Read (47) 47%4 = 3 Search list 3 for 47 Cache miss! LRU List 0 1 2 Head Tail 80 24 36 25 41 13 30 34 42 15 47 3 27 59 Hash Table 33
Searching the Block Cache Read (47) 47%4 = 3 Search list 3 for 47 Cache miss! LRU List 0 1 2 Head Tail 80 24 36 25 41 13 30 34 42 15 Retrieve 47 3 27 59 47 Update LRU Hash Table 34
Searching the Block Cache Read (15) 15%4 = 3 Search list 3 for 15 Cache miss! LRU List 0 1 2 Head Tail 80 24 36 25 41 13 30 34 42 15 3 27 59 47 Hash Table 35
Searching the Block Cache Read (14) 14%4 = 2 Search list 2 for 14 Cache miss! LRU List 0 1 2 Head Tail 80 24 36 25 41 13 30 34 15 Remove LRU 3 27 59 47 Retrieve 14 Hash Table 36
File System Consistency Block cache reduces disk accesses, good performance But if sys crashes and mod blocks not written, data lost To improve file system consistency Write out modified blocks (how often?) Write out critical blocks (write-through) Critical blocks: file system meta-data Directories, i-nodes, free block lists 37
Clustering of Related Blocks Some blocks expected to be accessed within short time Directory and files within that directory The inodes of the directory and the files Strategy Place related blocks close to each other: clustering Minimizes disk head movement: improves seek time 38
Caching of Other File System Info File system metadata (keep in memory) File metadata Currently active files Recently used Block maps File names Name to file metadata translations 39
Disk Block Size What should the block size be? The larger the block, the better the throughput The smaller the block, the less wasted space Technology trends Disk density is increasing faster than disk speed Make disk blocks larger: 1KB -> 8 KB 40
Summary A file system is a collection of files A file is the basic unit of storage Disks are good for file system storage, but slow File systems use simple model of disk: array of blocks File system structure File system metadata, file metadata, data blocks Use caching to improve performance 41