Physical and Logical structure. Thursday, December 02, 2004

Transcription

1 Logical_and_physical Page 1 Physical and Logical structure Thursday, December 02, :32 PM Logical and Physical structure Physical structure of a disk: tracks, sectors, cylinders. Logical structure of a disk: numbered blocks. Job of raw disk driver: given an operation on a block, do it to the raw block. Reads: fetch numbered blocks into a page cache. Writes: read targets into cache, update, mark for flush to disk. But meanings of specific blocks are unknown to the raw driver. To impart meaning, we must go to the next level: filesystems.

2 Logical_and_physical Page 2 Layers and meanings Thursday, November 17, :04 AM Layers and meanings: So far, we have one meaning for disk data: blocks. We will build up the concept of files in layers. Each layer Imparts a new form of meaning. That is not known at lower layers. First layer: the raw disk Meaning: a physical location L on the disk contains data. Second layer: the raw device Meaning: maps logical block N => physical location L Third layer: the filesystem Meaning: makes a file out of a set of blocks. Fourth layer: directories: Meaning: gives files human-readable names.

3 Logical_and_physical Page 3 A picture of layers Thursday, November 17, :08 AM A picture of the layers:

4 Logical_and_physical Page 4 Sets and sequences Thursday, November 17, :14 AM Sets and sequences A set is an unordered collection of things. A sequence is an ordered collection. The disk is a construction of sets and sequences: A directory is a set of inodes An inode describes a sequence of blocks. The way that sets and sequences are implemented varies greatly!

5 Logical_and_physical Page 5 Filesystems Thursday, December 02, :32 PM Next level after physical structure: "filesystems" organize and give meaning to sectors. organize sequences of sectors into files. allow naming of files. support directory structure.

6 Logical_and_physical Page 6 Inodes and blocks Thursday, November 17, :52 AM Inodes and blocks Two kinds of storage on a disk: Inodes: "identity nodes": describe an entity such as a file. protections owner, mode, etc. pointer to first block of file (a linked list) Blocks: contain content of a file. linked list elements other than this, content format unspecified. a directory is nothing more than a special kind of file, containing name/inode pairs.

7 Logical_and_physical Page 7 Logical filesystem structure Thursday, November 17, :55 AM Logical filesystem structure A filesystem is a set of inodes. A file is an inode that specifies a sequence of blocks. A directory is a file that maps names to inodes.

8 Logical_and_physical Page 8 Model of a filesystem Thursday, December 02, :32 PM Simplest possible model of a filesystem two sets of inodes, "free" and "used" two sets of blocks, "free" and "used" To create a file, one allocates an inode from the free set, transfers it to the used set, allocates a block from the free set, and moves it to the used set. points the inode at the block. To delete a file, one returns its inode and blocks to the free lists, respectively.

9 Logical_and_physical Page 9 Identity Thursday, November 17, :46 AM Identity The identity of a disk is pair of numbers: the major and minor device numbers. The identity of a file in a disk is a number: the "inode number". The triple <major, minor, inode> uniquely points to a file.

10 Logical_and_physical Page 10 Directories Thursday, November 17, :49 AM The name of a file is a derived thing, based upon the directories in which it appears. Directories are files that associate inodes and names. The top-level directory is named '/' '/foo' is the directory or file 'foo' in the directory '/'. From '/', can get to any file that has a name.

11 Logical_and_physical Page 11 UNIX filesystem structure Thursday, December 02, :32 PM Rough idea of realistic UNIX filesystem structure Three kinds of data: Blocks: of data All content of a file. All same size. Inodes: identity, access rights for data. Super-block: gives identity of things needed to mount a filesystem: Which inode refers to /? Where are free and used lists of inodes, blocks? What is a file? A linked list of blocks; each block contains a header that points to the "next block". Next block address is a tuple (cylinder, sector) Identity maintained in Inode: Owner, group, Protections (user, group, other) Pointer to first block of file. Really difficult to grasp: a file, intrinsically, has no name except

12 Logical_and_physical Page 12 that offered by context. A directory is a list of name/inode associations. Each directory has two special entries. => inode of the directory itself... => inode of parent directory. Special directory /: top level.. =. How a file gets its name: Open() traverses the directory hierarchy looking for each element of the path in turn /foo/bar/cat/dog.html Directory elements point to the inode of the next directory. File inodes end the search, and point to file content. A very counter-intuitive fact: Most expensive OS operation: "pwd" Print working directory. Problem: working directory is stored as an inode number! To get pwd, Start at current inode Traverse tree upward using.. At each level, figure out the path name that pointed to the lower level

13 Logical_and_physical Page 13 required. Stop when inode of. = inode of.. (root)

14 Inode example Thursday, December 02, :32 PM Logical_and_physical Page 14

15 Logical_and_physical Page 15 Mapping inodes to blocks Thursday, December 02, :32 PM

16 Filesystem structure Thursday, December 02, :32 PM Filesystem structure sketch ` Logical_and_physical Page 16

17 Logical_and_physical Page 17 Layout of a UNIX filesystem Thursday, December 02, :32 PM (oversimplified) layout of a UNIX filesystem:

18 Filesystem variations Thursday, December 02, :32 PM a bit deeper: it's not quite a linked list. file systems vary on how they store files. FAT32, EXT2, EXT3,... IDEA that there is a linked list showing sequence is an invariant, the way that list is implemented varies. (treatment so far is oversimplified) Format of realistic block content of a file. Starts with a descriptor block that points to where other blocks are. one block with pointers to other blocks, followed by a next. (This kind of tiered linked-list architecture comes Logical_and_physical Page 18

19 into its own in CD filesystems.) There are TWO blocks in the smallest file you can create! Logical_and_physical Page 19

20 Logical_and_physical Page 20 Managing complexity Thursday, December 02, :52 PM The operating system authors manage complexity of the operating system by: putting boundaries between subsystems and closing boxes; assuming that subsystems do not share information until there is a really good reason. presuming that in the absence of such a reason, no such sharing occurs. The following processes going on within an operating system are orthogonal and completely unaware of one another. File identity management (inodes, directories) Kernel process Utilizes page pool to manage entries. Subject to page pool management for updates. Page pool management (open file, not process!) Kernel process Manage read/write to/from blocks and inodes. Doesn't know the difference between them!

21 Logical_and_physical Page 21 Updates are asynchronous with writes! Page pool is fixed size determined at boot time (kernel tuning parameter) Updating of page pool entries (write) Request that page be memory resident (from page pool manager); get memory address. Update page as given. Mark page updated Leave everything else to page pool manager! Process I/O: Writes to specific page Defers to driver to do this. Driver interacts with page pool manager to load and modify page. Process scheduling Has nothing to do with page pool management! Interacts with virtual memory subsystem. Virtual memory (process, not open file!) Disk image changes in size Requires algorithm that allows changes in image size with demand. Knows nothing about page pool algorithms and sizes. Physical devices Don't "know" whether they're being

22 Logical_and_physical Page 22 treated as "filesystems" or not. Can read them in raw mode! If they're mounted at the time, havoc ensues! Maintaining transparency (closing boxes) When a process wants to write to a disk file It calls fopen, Which allocates write and read buffers, and calls open Which updates a kernel file descriptor table Which contains a pointer to the current read/write position in the file and points a process file descriptor at the kernel table. When a process wants to write to the file: It calls fwrite (or printf, or something else that calls fwrite Which updates its local write buffer and perhaps calls write Which refers to the kernel descriptor Which contains the location to write to And calls the page manager to

23 Logical_and_physical Page 23 load the page being written to And writes the data into the kernel page image. Likewise for read. When a process wants to stop accessing a file It calls fclose Which calls close Which deallocates the kernel descriptor only if no other process has it open (via access counts) Note: when you close a file, its page table remains active, perhaps indefinitely! Case study: /var/adm/pacct: user accounting file. full disk: pacct is 90% of disk rm /var/adm/pacct surprise: disk is still full. oops: the KERNEL has the file open => will not be deleted until the KERNEL closes it proper procedure turn off accounting accton -d delete file turn on accounting accton -a

24 Logical_and_physical Page 24

25 Logical_and_physical Page 25 Why update must be separate Thursday, December 02, :52 PM update and why it has to be separate purpose of update: optimal scheduling of reads and writes, depending upon the disk geometry and speed. Job of update: plan the writing out so that it saves as much time as possible, by grouping outputs.