Chapter 6. File Systems

Transcription

1 Chapter 6 File Systems 6.1 Files 6.2 Directories 6.3 File system implementation 6.4 Example file systems 350 Long-term Information Storage 1. Must store large amounts of data 2. Information stored must survive the termination of the process using it 3. Multiple processes must be able to access the information concurrently 351 1

2 File Naming Typical file extensions. 352 File Structure Three kinds of files byte sequence record sequence tree 353 2

3 File Types (a) An executable file (b) An archive 354 Sequential access File Access read all bytes/records from the beginning cannot jump around, could rewind or back up convenient when medium was mag tape Random access bytes/records read in any order essential for data base systems read can be move file marker (seek), then read or read and then move file marker 355 3

4 File Attributes Possible file attributes 356 File Operations 1. Create 2. Delete 3. Open 4. Close 5. Read 6. Write 7. Append 8. Seek 9. Get attributes 10.Set Attributes 11.Rename 357 4

5 An Example Program Using File System Calls (1/2) 358 An Example Program Using File System Calls (2/2) 359 5

6 Memory-Mapped Files (a) Segmented process before mapping files into its address space (b) Process after mapping existing file abc into one segment creating new segment for xyz 360 Memory-Mapped Files (2) OS doesn t know how far in main memory the process wrote, i.e., does not know length give hint, or else write back whole frame(s) Consistency: process no longer knows when updated file will be visible (on disk) Size: file size limited to what will fit mem

7 Directories Single-Level Directory Systems A single level directory system contains 4 files owned by 3 different people, A, B, and C 362 Two-level Directory Systems Letters indicate owners of the directories and files 363 7

8 Hierarchical Directory Systems A hierarchical directory system 364 Hierarchical Directory Systems (2) Other organizations are possible: OS knows about application types and groups files accordingly files v. human-readable documents hyperlinked by content associative memory, keywords, relevance measure: search engines chronologic (!), spatial (piles on desks) 365 8

9 Path Names A UNIX directory tree relative, absolute path working directory dot and dotdot filename with w/out path length limits? 366 Directory Operations 1. Create 2. Delete 3. Opendir 4. Closedir 5. Readdir 6. Rename 7. Link 8. Unlink 367 9

10 File System Implementation A possible file system layout 368 File System Layout Master Boot Record a known place (i.e., burned somewhere) points to active partition, first block (boot block) executed, loads OS old lilo: MBR could not address boot block beyond 8GB

11 Implementing Files (1) (a) Contiguous allocation of disk space for 7 files (b) State of the disk after files D and E have been removed 370 Implementing Files (1b) Contiguous allocation enough to know 1st block address and length file may be read in a single disk operation but: over time, fragmentation (again amortized analysis matters to real systems!) Used on Write-Once (CDROMs) media

12 Implementing Files (2) Storing a file as a linked list of disk blocks 372 Implementing Files (3) Linked list allocation using a file allocation table in RAM

13 Implementing Files (3b) Linked list allocation no external fragmentation but, slow to read (need current block read before can issue next one) space for pointer reduces disk block size to a non power of two FAT: Linked list with table in memory speed ok, but now table size is a problem how do tradeoffs (RAM/Disk) affect this? 374 Implementing Files (4) An example i-node

14 Implementing Files (4b) index-nodes, one i-node per file contains file attrs and list of its blocks only in memory while file open i.e., open file operation loads/creates i-node size of i-node table proportional to max number of open files, not to # of blocks in disk last disk address points to block with addresses of the next run of blocks for file this allows each i-node to have fixed size 376 Implementing Directories (1) (a) A simple directory fixed size entries disk addresses and attributes in directory entry (b) Directory in which each entry just refers to an i-node

15 Implementing Directories (2) Two ways of handling long file names in directory (a) In-line (b) In a heap 378 Implementing Directories (3) file name size affects directory structure fixed or variable, long names inline or heap variable size dir entries remove file: fragment dir! compacted in-mem dirs may span pages, page faulting long dirs may need hashing or caching

16 Shared Files (1) File system containing a shared file 380 Shared Files (2) (a) Situation prior to linking (b) After the link is created (c)after the original owner removes the file

17 Shared Files Digression: Security Small example of attack using file ops Slides by Prof. Terran Lane 382 Race Conditions & Security Atomicity failures can sometimes be exploited to break security on multiprocessing systems One of the top 10 classes of exploits since... mid s, at least 100 s (or more) of reported vulnerabilities Independent of language: Java will not save you! Hostile program grabs a shared resource (e.g., file) before it is secured Beware when writing privileged code! N.b.: Sometimes your never-intended-to-be- secure code will be run in privileged context!

18 Basic Race Condition Exploit priv proc 384 Basic Race Condition Exploit priv proc file /tmp/foo open( /tmp/foo, O_RDWR O_CREAT); write() read() close() unlink()

19 Basic Race Condition Exploit priv proc open( /tmp/foo, O_RDWR O_CREAT); write() file /tmp/foo open(...) hostile proc read() read() close() unlink() 386 Basic Race Condition Exploit priv proc hostile proc open( /tmp/foo, O_RDWR O_CREAT); chmod() write() read() file /tmp/foo close() unlink()

20 Basic Race Condition Exploit priv proc open( /tmp/foo, O_RDWR O_CREAT); chmod() file /tmp/foo open(...) hostile proc write() read() close() unlink() 388 Basic Race Condition Exploit priv proc umask() file /tmp/foo hostile proc open( /tmp/foo, O_RDWR O_CREAT); write() read() close() unlink()

21 Basic Race Condition Exploit priv proc umask() file /tmp/foo open(...) hostile proc open( /tmp/foo, O_RDWR O_CREAT); write() read() read() close() unlink() 390 Basic Race Condition Exploit priv proc file /tmp/foo hostile proc umask() symlink( /tmp/foo, /etc/passwd ) open( /tmp/foo, O_RDWR O_CREAT); write() read() close() unlink()

22 Basic Race Condition Exploit priv proc umask() file /tmp/foo stat( /tmp/foo ); if (!exists) { open( /tmp/foo, O_RDWR O_CREAT); } else { error(); } hostile proc write() read() close() unlink() 392 Basic Race Condition Exploit priv proc umask() file /tmp/foo stat( /tmp/foo ); symlink( /tmp/foo, if (!exists) { /etc/passwd ) open( /tmp/foo, O_RDWR O_CREAT); } else { error(); } write() read() hostile proc close() unlink()

23 Preventing FS Race Conditions Could create foo in dir owned/writable only by owner of proc Can be hard to ensure this Still have to watch out for filename collisions Could make file names hard to predict (e.g., picked randomly) Exploit still possible; hard to make fnames really random Ultimate answer: use OS atomicity facilities open( /tmp/foo, O_RDWR O_CREAT O_EXCL) Always be on guard! 394 Disk Space Management (1) Block size Dark line (left hand scale) gives data rate of a disk Dotted line (right hand scale) gives disk space efficiency All files 2KB

24 Disk Space Management Most file systems break files into non-adjacent blocks. Choice of Block size: disk sector, track, cylinder; memory page device-dependent? fragmentation: median Unix file 1KB (?!) multiple blocks per file: more acc. overhead tradeoff: performance versus space utilization UNIX: 1KB typical MS-DOS: max 2 16 blocks (disk size defs block size) 396 Disk Space Management (2) (a) Storing the free list on a linked list (b) A bit map

25 Disk Space Management (2b) need to manage free blocks linked list (often stored in free blocks) 1KB block, 32-bit disk block # = 255 (+1 next) 16GB disk has 2 24 block # ~ 16,794 blocks bitmap 16GB needs 2 24 bits blocks ~ 2048 blocks what would Linus do? 398 Disk Space Management (3) (a) Almost-full block of pointers to free disk blocks in RAM - three blocks of pointers on disk (b) Result of freeing a 3-block file (c) Alternative strategy for handling 3 free blocks - shaded entries are pointers to free disk blocks

26 Disk Space Management (4) Quotas for keeping track of each user s disk use 400 File System Reliability (0) State more important than computing power i.e., files are more important than hardware Backups - for disaster or error; slow, boring. focus on user files (is that enough?) incremental dumps; possibly compress data consistency issues (fuzzy, offline) maintain a broader backup system (offsite, etc.)

27 File System Reliability (1) File that has not changed A file system to be dumped squares are directories, circles are files shaded items, modified since last dump each directory & file labeled by i-node number 402 File System Reliability (1b) Backups: What to copy, and how? dumps: physical block 0) logical: follow directory structure must dump full path to modified files; why? incremental dumps BTW: backup program must be correct (!!)

28 File System Reliability (2) Bit maps used by the logical dumping algorithm 404 File System Reliability (3) File system states (a) consistent (b) missing block (c) duplicate block in free list (d) duplicate data block

29 File System Reliability (3b) Consistency issues: set of disk operations corresponding to one high-level operation do not execute atomically disk crash leaves FS datastructs inconsistent especially: i-node, dir, free list blocks UNIX fsck, Windows scandisk at boot time both block and file consistency checked count block appears in file or in free list via i-nodes check directories: how many dirs contain each file, compare with i-node counts 406 File System Reliability (3c) block inconsistencies block not on either free/used: missing add it to free list block twice in free list: rebuild list (bitmap?) block twice in files: copy and separate directory inconsistencies i-node link count > # dir entries, dec i-node count i-node < # dir, similar to block twice (deletion) other: permissions, setuid, too many files,

30 File System Performance (0) block / buffer cache (hash table) LRU with modifs (block type matters) consistency-critical blocks: write back always i-node, dir, full and being filled data blocks Block read-ahead Reducing disk arm motion: placement of metadata (i-nodes; cylinder groups) 408 File System Performance (1) The block cache data structures

31 File System Performance (2) I-nodes placed at the start of the disk Disk divided into cylinder groups each with its own blocks and i-nodes 410 File System Performance (3) caching - block cache (modified LRU) block likely to be needed soon? block essential to consistency of file system? Unix: sync syscall via update daemon MS-DOS: write-through cache block read ahead reducing disk arm motion contiguous free blocks (bitmap v freelist) placement of i-nodes

32 Log-Structured File Systems With CPUs faster, memory larger disk caches can also be larger increasing number of read requests can come from cache thus, most disk accesses will be writes LFS Strategy structures entire disk as a log have all writes initially buffered in memory periodically write these to the end of the disk log when file opened, locate i-node, then find blocks