Unit 3: Storage Management

Transcription

1 Unit 3: Storage Management Unit Overview Unit 3 discusses how memory, file systems, mass storage, and I/O (input/output) are handled in modern computer systems. In a multiprogramming system, a process may change its state from waiting to running several times before completing its task. Clearly, if each process were loaded from secondary storage into the main memory in its entirety, a large fraction of machine time would be wasted on loading and unloading processes to and from the main memory during each section of process execution. To minimize the overhead associated with loading and unloading, operating systems allocate memory to several processes simultaneously. This complex task requires effective memory management schemes and their hardware support (major topics in this unit). The other major topic in this unit is secondary storage, in particular, file systems and disk management. Disks serve as extensions and backup for the main memory, and as economical online storage devices for program and data files. The operating system organizes files in the framework of a file system. Disk access is much slower than access to memory and, in general, several processes compete for disk access. For these reasons, in many computers, disk access is the main bottleneck to improved performance. As such, section 3.4 examines the software and hardware mechanisms employed to improve disk performance. Input-output operations are among the main jobs of a computer system. As various I/O devices vary widely in their function and speed, different methods are needed to control them, and those methods form the I/O subsystem of the kernel. The final section of this unit covers the structure, principles, and performance aspects of I/O hardware and software. Unit 3 is divided into five sections: 3.1 Main Memory 3.2 Virtual Memory 3.3 File-System Interface and Implementation 3.4 Mass-Storage Structure 3.5 I/O Systems Learning Objectives: When you complete Unit 3, you will be able to describe the different strategies that operating systems employ to manage memory, with knowledge of the benefits, limitations, resource overhead (such as fragmentation and data structures), and hardware requirements of each management technique. You will also learn how developers can write applications (program and data structures) that reduce overhead for memory in a shared environment and that use memory more efficiently. In modern application environments, in which virtual memory is becoming increasingly important, it is important that you can explain this concept and its importance, and that you can list and describe several strategies that systems software uses to manage disks and other devices. Finally, you should be able to describe the structure, principles, and performance issues of I/O hardware and software. 3.1 Main Memory Overview This section focuses on memory management in the main system. Memory is an important finite resource that has a significant impact on system throughput. As such, it must be managed carefully. In a multiprogramming environment, the operating system attempts to complete as much work as possible at one time by allowing processes to share memory. To do this, it is critical that the operating system manage memory in such a way that many processes may share it and that the memory space for each process is protected from other processes. This security measure requires that all memory addresses be validated. The overhead resource cost of verifying every memory access makes hardware support necessary.

2 Learning Outcomes After you have completed Section 3.1, you should be able to: Describe address binding at compile, load, and execution time. Explain the different strategies that operating systems use to manage the sharing and allocation of memory among several processes, including contiguous allocation, paging, segmentation, and segmentation with paging. Discuss how relocation, sharing, and protection are considered in memory management. Describe how an operating system may implement shared memory. Reading Assignment Operating system concepts (9th ed.): Chapter 8: Main Memory: 8.1 to 8.7. As you do the assigned reading, focus on the key concepts and Topics outlined below to ensure that you can meet Learning Outcomes 1-4 above. Key Concepts and Topics Address Binding Compile Time Binding Execution Time Binding Load Time Binding Address binding is the process of mapping the program's logical or virtual addresses to corresponding physical or main memory addresses. In other words, a given logical address is mapped by the MMU (Memory Management Unit) to a physical address. Classically, the binding of instructions and data to memory addresses can be done at any step along the way: Compile time. The compiler translates symbolic addresses to absolute addresses. If you know at compile time where the process will reside in memory, then absolute code can be generated (Static) If it is not known at compile time where the process will reside in memory, then the compiler must generate relocatable code (Static). Execution time. If the process can be moved during its execution from one memory segment to another, then binding must be delayed until run time Load time. The compiler translates symbolic addresses to relative (re-locatable) addresses. Logical Address Virtual Address Memory Management Unit Dynamic Loading An address generated by the CPU is commonly referred to as a logical address, whereas an address seen by the memory unit that is, the one loaded into the memory-address register of the memory is commonly referred to as a physical address. A virtual address is a binary number in virtual memory that enables a process to use a location in primary storage (main memory) independently of other processes and to use more space than actually exists in primary storage by temporarily relegating some contents to a hard disk or internal flash drive. A memory management unit (MMU), sometimes called paged memory management unit (PMMU), is a computer hardware unit having all memory references passed through itself, primarily performing the translation of virtual memory addresses to physical addresses. Dynamic loading is a mechanism by which a computer program can, at run time, load a library (or other binary) into memory, retrieve the addresses of functions and variables contained in the library, execute those functions or access those variables, and unload the library from memory. Static and Dynamic Linking Swapping Contiguous Allocation Partitioning Static linking is the process of copying all library modules used in the program into the final executable image. This is performed by the linker and it is done as the last step of the compilation process. The linker combines library routines with the program code in order to resolve external references, and to generate an executable image suitable for loading into memory. When the program is loaded, the operating system places into memory a single file that contains the executable code and data. This statically linked file includes both the calling program and the called program. In dynamic linking the names of the external libraries (shared libraries) are placed in the final executable file while the actual linking takes place at run time when both executable file and libraries are placed in the memory. Dynamic linking lets several programs use a single copy of an executable module.

3 A process must be in memory to be executed. A process, however, can be swapped temporarily out of memory to a backing store and then brought back into memory for continued execution (Figure 8.5). Swapping makes it possible for the total physical address space of all processes to exceed the real physical memory of the system, thus increasing the degree of multiprogramming in a system. Contiguous memory allocation is a classical memory allocation model that assigns a process consecutive memory blocks (that is, memory blocks having consecutive addresses). Contiguous memory allocation is one of the oldest memory allocation schemes. When a process needs to execute, memory is requested by the process. Partitioning is useful for limiting the sizes of individual file systems, putting multiple file-system types on the same device, or leaving part of the device available for other uses, such as swap space or unformatted (raw) disk space. A file system can be created on each of these parts of the disk. Variable-Partition Scheme External Fragmentation Internal Fragmentation Compaction In the variable-partition scheme, the operating system keeps a table indicating which parts of memory are available and which are occupied. Initially, all memory is available for user processes and is considered one large block of available memory, a hole. Eventually, as you will see, memory contains a set of holes of various sizes. As processes are loaded and removed from memory, the free memory space is broken into little pieces. External fragmentation exists when there is enough total memory space to satisfy a request but the available spaces are not contiguous: storage is fragmented into a large number of small holes. This fragmentation problem can be severe. In the worst case, we could have a block of free (or wasted) memory between every two processes. If all these small pieces of memory were in one big free block instead, we might be able to run several more processes. Internal fragmentation is the wasted space within each allocated block because of rounding up from the actual requested allocation to the allocation granularity. External fragmentation is the various free spaced holes that are generated in either your memory or disk space. The goal is to shuffle the memory contents so as to place all free memory together in one large block. Paging Page Frame Page Table Segmentation permits the physical address space of a process to be noncontiguous. Paging is another memory-management scheme that offers this advantage. However, paging avoids external fragmentation and the need for compaction, whereas segmentation does not. It also solves the considerable problem of fitting memory chunks of varying sizes onto the backing store. Most memory-management schemes used before the introduction of paging suffered from this problem. The page number is used as an index into a page table. The page table contains the base address of each page in physical memory. This base address is combined with the page off-set to define the physical memory address that is sent to the memory unit. Frame Table Page-Table Base Register Translation Look-Side Buffer Hit-Ratio Since the operating system is managing physical memory, it must be aware of the allocation details of physical memory which frames are allocated, which frames are available, how many total frames there are and so on. This information is generally kept in a data structure called a frame table. The frame table has one entry for each physical page frame, indicating whether the latter is free or allocated and, if it is allocated, to which page of which process or processes. To support this, a processor that supports virtual memory must have a page table base register that is accessible by the operating system.... The page table base address for the process is stored there. The operating system loads this address into the PTBR whenever a process is dispatched. A translation look aside buffer (TLB) is a memory cache that stores recent translations of virtual memory to physical addresses for faster retrieval. When a virtual memory address is referenced by a program, the search starts in the CPU. First, instruction caches

4 are checked. A Translation look aside buffer (TLB) is also a memory cache that is used to reduce the time taken to access a user memory location. It is a part of the chip's memory-management unit (MMU). The percentage of times that the page number of interest is found in the TLB is called the hit ratio. An 80-percent hit ratio, for example, means that we find the desired page number in the TLB 80 percent of the time. If it takes 100 nanoseconds to access memory, then a mapped-memory access takes 100 nanoseconds when the page number is in the TLB. Frame Protection Multilevel Paging Hash-Page Table Inverted Page Table Memory protection in a paged environment is accomplished by protection bits associated with each frame. Normally, these bits are kept in the page table. One bit can define a page to be read write or read-only. Every reference to memory goes through the page table to find the correct frame number. At the same time that the physical address is being computed, the protection bits can be checked to verify that no writes are being made to a read-only page. An attempt to write to a read-only page causes a hardware trap to the operating system (or memory-protection violation). Multilevel Paging occurs when the page table might be too big to fit in a contiguous space, so we may have a hierarchy with several levels The Hash-Page Table is common in address spaces > 32 bits. The virtual page number is hashed into a page table, this page table contains a chain of elements hashing to the same location because the same hash function can have same value for different page no. Each element contains (1) the virtual page number (2) the value of the mapped page frame (3) a pointer to the next element Virtual page numbers are compared in this chain searching for a match. If a match is found, the corresponding physical frame is extracted. An inverted page table has one entry for each real page (or frame) of memory. Each entry consists of the virtual address of the page stored in that real memory location with information about the process that owns the page. Thus, only one page table is in the system, and it has only one entry for each page of physical memory. Segment Segment Table Segmentation Segmentation with Paging Each of the parts into which something is or may be divided. The segment table is thus essentially an array of base limit register pairs. Segmentation permits the physical address space of a process to be noncontiguous. It is also a memory-management scheme that supports this programmer view of memory (i.e.) a logical address space is a collection of segments. Segments can be of different lengths, so it is harder to find a place for a segment in memory than a page. With segmented virtual memory, we get the benefits of virtual memory but we still have to do dynamic storage allocation of physical memory. In order to avoid this, it is possible to combine segmentation and paging into a two-level virtual memory system. Each segment descriptor points to page table for that segment. This gives some of the advantages of paging (easy placement) with some of the advantages of segments (logical division of the program). Local Descriptor Table / Global Descriptor Table The IA-32 architecture allows a segment to be as large as 4 GB, and the maximum number of segments per process is 16 K. The logical address space of a process is divided into two partitions. The first partition consists of up to 8Ksegments that are private to that process. The second partition consists of up to 8 K segments that are shared among all the processes. Information about the first partition is kept in the local descriptor table (LDT); information about the second partition is kept in the global descriptor table (GDT). Each entry in the LDT and GDT consists of an 8-byte segment descriptor with detailed information about a particular segment, including the base location and limit of that segment.

5 Study Questions 1. What is address binding? Address binding is the process of mapping the program's logical or virtual addresses to corresponding physical or main memory addresses. In other words, a given logical address is mapped by the MMU (Memory Management Unit) to a physical address. 2. What are the main purposes of swapping, paging, and segmentation in memory management? A process must be in memory to be executed. A process, however, can be swapped temporarily out of memory to a backing store and then brought back into memory for continued execution. Swapping makes it possible for the total physical address space of all processes to exceed the real physical memory of the system, thus increasing the degree of multiprogramming in a system. 3. When should an operating system designer consider using hashed paging and inverted paging instead of hierarchical paging? A common approach for handling address spaces larger than 32 bits is to use a hashed page table, with the hash value being the virtual page number. Each entry in the hash table contains a linked list of elements that hash to the same location. Therefore, a single page-table entry can store the mappings for multiple physical-page frames. An inverted page table has one entry for each real page (or frame) of memory. Each entry consists of the virtual address of the page stored in that real memory location, with information about the process that owns the page. Thus, only one page table is in the system, and it has only one entry for each page of physical memory. Inverted page tables often require that an address-space identifier be stored in each entry of the page table, since the table usually contains several different address spaces mapping physical memory. Storing the address-space identifier ensures that a logical page for a particular process is mapped to the corresponding physical page frame. Most modern computer systems support a large logical address space. In such an environment, the page table itself becomes excessively large. Assuming that each entry consists of 4 bytes, each process may need up to 4 MB of physical address space for the page table alone. Clearly, there would be no desire to allocate the page table contiguously in main memory. One simple solution to this problem is to divide the page table into smaller pieces. One way is to use a two-level paging algorithm, in which the page table itself is also paged. By partitioning the page table the operating system can leave partitions unused until a process needs them. Entire sections of virtual address space are frequently unused, and multilevel page tables have no entries for these spaces, greatly decreasing the amount of memory needed to store virtual memory data structures. In addition, there is a prohibitive number of memory accesses to translate each logical address. 4. How does an Intel architecture support both paging and segmentation? 3.2 Virtual Memory Overview For users, the tangible benefit of virtual memory is that it removes the restriction that the size of the computer s memory places on the size of an executable program. The key idea behind virtual memory is easy to state but much harder to implement: keep only those (code and data) parts of a process in the memory that are needed in the current CPU burst of the process. This section examines the software and hardware mechanisms needed to implement virtual memory. Learning Outcomes After you have completed Section 3.2, you should be able to: Explain the concept of virtual memory, and discuss the benefits, implementation, and overhead. Define demand paging and page replacement. Describe the page replacement algorithms and strategies listed below in terms of algorithms, data structures, overhead, and benefits.

6 First-In First-Out (FIFO). Optimal. Least-Recently-Used (LRU) and variations. Page Buffering. Outline the decision problems associated with frame allocation. Explain the phenomenon of thrashing and the strategies and techniques deployed to deal with this problem. Describe how memory mapping files and shared memory operate in the Win32 API. Discuss the following factors that affect the character and performance of a paging system: Pre-paging, Page Size, Program Structure, and I/O Interlock. Reading Assignment Operating system concepts (9th ed.): Chapter 9: Virtual Memory: 9.1 to As you do the assigned reading, focus on the Key Concepts and Topics outlined below to ensure that you can meet Learning Outcomes 1-7 above. Key Concepts and Topics virtual memory demand paging lazy swapper pager Virtual Memory. A technique that allows the execution of a process that is not completely in memory. Demand Paging employs a technique that loads pages into memory as needed. It is commonly used in virtual memory systems. A lazy swapper never swaps a page into memory unless that page will be needed. A swapper manipulates entire processes, whereas a pager is concerned with the individual pages of a process. We thus use pager, rather than swapper, in connection with demand paging. page fault locality of reference page fault rate copy-on-write Access to a page marked invalid causes a page fault. The paging hardware, in translating the address through the page table, will notice that the invalid bit is set, causing a trap to the operating system. This trap is the result of the operating system s failure to bring the desired page into memory. Locality of Reference, also known as the principle of locality, is a term for the phenomenon in which the same values, or related storage locations, are frequently accessed, depending on the memory access pattern. A page fault rate (sometimes called #PF, PF or hard fault) is a type of exception raised by computer hardware when a running program accesses a memory page that is not currently mapped by the memory management unit (MMU) into the virtual address space of a process. Copy-On-Write, works by allowing the parent and child processes initially to share the same pages. These shared pages are marked as Copy-On-Write pages, meaning that if either process writes to a shared page, a copy of the shared page is created. Page Replacement Reference String First-In First-Out Page Replacement Optimal Page Replacement Page Replacement algorithms decide which memory pages to page out, sometimes called swap out, or write to disk, when a page of memory needs to be allocated. We evaluate an algorithm by running it on a particular string of memory references and computing the number of page faults. The string of memory references is called a reference string. The simplest page-replacement algorithm is a first-in, first-out (FIFO) algorithm. A FIFO replacement algorithm associates with each page the time when that page was brought into memory. When a page must be replaced, the oldest page is chosen. Optimal Page Replacement Algorithm is an algorithm that works as follows: when a page needs to be swapped in, the operating system swaps out the page whose next use will occur farthest in the future.

7 Belady s Anomaly Least-Recently-Used Page Replacement Page-Buffering Algorithm Frame Allocation In computer storage, Bélády's Anomaly is the phenomenon in which increasing the number of page frames results in an increase in the number of page faults for certain memory access patterns. This phenomenon is commonly experienced when using the first-in first-out (FIFO) page replacement algorithm. The Least Recently Used (LRU) Page Replacement Algorithm is a good approximation to the optimal algorithm. It is based on the observation that pages that have been heavily used in the last few instructions will probably be heavily used again in the next few instructions. This strategy is called LRU (Least Recently Used) paging. The operating system maintains a pool of free frames. When a page fault occurs, a page is selected for replacement and written into the pool of free frames. The faulty page is swapped out of disk and the page table is modified. A process can select a replacement from among its own frames or the frames of any lower-priority process. This approach allows a high-priority process to increase its frame allocation at the expense of a low-priority process. global page replacement local page replacement thrashing working-set model A global page-replacement algorithm is used; it replaces pages without regard to the process to which they belong. Local vs. global replacement. When a process incurs a page fault, a local page replacement algorithm selects for replacement some page that belongs to that same process (or a group of processes sharing a memory partition). A global replacement algorithm is free to select any page in memory. In computer science, thrashing occurs when a computer's virtual memory subsystem is in a constant state of paging, rapidly exchanging data in memory for data on disk, to the exclusion of most application-level processing. The working-set model assumes that processes execute in localities. The working set is the set of pages in the current locality. Accordingly, each process should be allocated enough frames for its current working set. If a process does not have enough memory for its working set, it will thrash. Memory-Mapped Files and I/O Shared Memory Prepaging I/O Interlock Memory Mapping a file, allows a part of the virtual address space to be logically associated with the file. In computer science, shared memory is memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them or avoid redundant copies. Shared memory is an efficient means of passing data between programs. Prepaging is a technique whereby the operating system in a paging virtual memory multi-tasking environment loads all pages of a process's working set into memory before the process is restarted. Pages must sometimes be locked into memory. Pages that are used for copying a file from a device must be locked from being selected for eviction by a page replacement algorithm. Study Questions 1. Why is virtual memory an important feature of modern operating systems? Virtual memory is a component of most operating systems, such as MAC OS, Windows and Linux. Virtual memory has a very important role in the operating system. It allows us to run more applications on the system than we have enough physical memory to support. Virtual memory is simulated memory that is written to a file on the hard drive. That file is often called page file or swap file. It s used by operating systems to simulate physical RAM by using hard disk space. 2. What is page replacement? How should different page replacement algorithms be chosen?, page replacement algorithms decide which memory pages to page out, sometimes called swap out, or write to disk, when a page of memory needs to be allocated. Page replacement happens when a requested page is not in memory (page fault) and a free page cannot be used to satisfy the allocation, either because there are none, or because the number of free pages is lower than some threshold. When the page that was selected for replacement and paged out is referenced again it has to be paged in (read in from disk), and this involves waiting for I/O completion.

8 3. How are memory-mapped files used for memory sharing? A memory-mapped file is a segment of virtual memory that has been assigned a direct byte-for-byte correlation with some portion of a file or file-like resource. This resource is typically a file that is physically present on disk, but can also be a device, shared memory object, or other resource that the operating system can reference through a file descriptor. Once present, this correlation between the file and the memory space permits applications to treat the mapped portion as if it were primary memory. The memory-mapping system calls can also support copy-on-write functionality, allowing processes to share a file in read-only mode but to have their own copies of any data they modify. So that access to the shared data is coordinated, the processes involved might use one of the mechanisms for achieving mutual exclusion 3.4 Mass-Storage Structure Overview A file system can be viewed logically as consisting of three parts: the interface to the file system (for users and programmers), the internal data structures and algorithms for implementing the interface, and the secondary and tertiary storage structures. The first two parts were covered in Section 3.3; this section addresses the third. Secondary storage structure topics include the physical structure of disks, disk-scheduling algorithms, disk management (disk formatting, boot block, and bad blocks), and swapspace management. Section 3.4 also introduces RAID structure, stable-storage, and tertiary storage structure. Learning Outcomes After you have completed Section 3.4, you should be able to: Describe overall disk structure. Explain and compare several algorithms for scheduling disk requests, including FCFS, SSTF, SCAN, C-SCAN, and LOOK. Describe issues related to disk management such as disk initialization, booting from disk, and bad-block recovery. Describe swap-space management, disk reliability, RAID structure, and tertiary storage structures. Reading Assignment Operating system concepts (9th ed.): Chapter 10: Mass-Storage Structure: 10.1 to As you do the assigned reading, focus on the Key Concepts and Topics outlined below to ensure that you can meet Learning Outcomes 1-4 above. Key Concepts and Topics Magnetic Disk and Tape Logical Block Constant Linear Velocity (CLV) Constant Angular Velocity (CAV) Computers can store information on various storage media, such as magnetic disks, magnetic tapes, and optical disks. So that the computer system will be convenient to use, the operating system provides a uniform logical view of stored information. Modern magnetic disk drives are addressed as large one-dimensional arrays of logical blocks, where the logical block is the smallest unit of transfer. The term logical block is typically used to refer to the host's view of data addressing on a physical device. In optical storage, constant linear velocity is a qualifier for the rated speed of an optical disc drive, and may also be applied to the writing speed of recordable discs. Constant Angular Velocity is a technique for accessing data off of rotating disks. With CAV, the disk rotates at a constant speed regardless of what area of the disk is being accessed. This differs from Constant Linear Velocity (CLV), which rotates the disk faster for inner tracks. Disk drives use CAV, whereas CD-ROMs generally use CLV, though some newer drives use a combination of CAV and CLV.

9 Host-Attached and NAS SCSI Fiber Channel (FC) Storage-Area Network Computers access disk storage in two ways. One way is via I/O ports (or host-attached storage); this is common on small systems. The other way is via a remote host in a distributed file system; this is referred to as network-attached storage. Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, electrical and optical interfaces. Fiber Channel, or FC, is a high-speed network technology primarily used to connect computer data storage to servers. Fiber Channel is mainly used in storage area networks in commercial data centers. A storage-area network (SAN) is a private network (using storage protocols rather than networking protocols) connecting servers and storage units. The power of a SAN lies in its flexibility. Multiple hosts and multiple storage arrays can attach to the same SAN, and storage can be dynamically allocated to hosts. A SAN switch allows or prohibits access between the hosts and the storage. SANs make it possible for clusters of servers to share the same storage and for storage arrays to include multiple direct host connections. SANs typically have more ports as well as more expensive ports than storage arrays. Disk Scheduling Seek Time Rotational Latency Bandwidth In operating systems, seek time is very important. Since all device requests are linked in queues, the seek time is increased causing the system to slow down. Disk Scheduling Algorithms are used to reduce the total seek time of any request. The seek time is the time for the disk arm to move the heads to the cylinder containing the desired sector. The rotational latency is the additional time for the disk to rotate the desired sector to the disk head. Bandwidth is also defined as the amount of data that can be transmitted in a fixed amount of time. For digital devices, the bandwidth is usually expressed in bits per second (bps) or bytes per second. For analog devices, the bandwidth is expressed in cycles per second, or Hertz (Hz). First-come, First-Served (FCFS) Shortest-Seek-Time-First (SSTF) Scan scheduling (SCAN) or Elevator Algorithm First come, first served (FCFS) is an operating system process scheduling algorithm and a network routing management mechanism that automatically executes queued requests and processes by the order of their arrival. With first come, first served, what comes first is handled first; the next request in line will be executed once the one before it is complete. FCFS is also known as first-in, first-out (FIFO) and first come, first choice (FCFC). It is also the simplest form of disk scheduling. Shortest seek first (or shortest seek time first) is a secondary storage scheduling algorithm to determine the motion of the disk's arm and head in servicing read and write requests. The elevator algorithm (also SCAN) is a disk scheduling algorithm to determine the motion of the disk's arm and head in servicing read and write requests. Circular SCAN (C-SCAN) Look Scheduling (LOOK) Disk Formatting Logical Formatting Circular scanning works just like the elevator to some extent. It begins its scan toward the nearest end and works its way all the way to the end of the system. Once it hits the bottom or top it jumps to the other end and moves in the same direction. Keep in mind that the huge jump doesn't count as a head movement. The total head movement for this algorithm is only 187 track, but still this isn't the most sufficient. This is just an enhanced version of C-SCAN. In this the scanning doesn't go past the last request in the direction that it is moving. It too jumps to the other end but not all the way to the end. Just to the furthest request. C-SCAN had a total movement of 187 but this scan (C-LOOK) reduced it down to 157 tracks. Disk formatting is the process of preparing a data storage device such as a hard disk drive, solid-state drive, floppy disk or USB flash drive for initial use. In some cases, the formatting operation may also create one or more new file systems. High-level formatting (logical) is the process of setting up an empty file system on a disk partition or logical volume and, for PCs, installing a boot sector. This is a fast operation, and is sometimes referred to as quick formatting.

10 Physical Formatting Boot Block Master Boot record (MBR) Bad Blocks Physical Format. The lowest level structure. For example, the physical format of a disk is its division into sectors (see low-level format). See record layout. Boot Block (plural boot blocks). A dedicated block usually at the beginning (first block on first track) of a storage medium that holds special data used to start a system. Some systems use a boot block of several physical sectors, while some use only one boot sector. A master boot record is a special type of boot sector at the very beginning of partitioned computer mass storage devices like fixed disks or removable drives intended for use with IBM PC-compatible systems and beyond. Because disks have moving parts and small tolerances, they are prone to failure. Sometimes the failure is complete; in this case, the disk needs to be replaced and its contents restored from backup media to the new disk. More frequently, one or more sectors become defective. Most disks even come from the factory with bad blocks. Depending on the disk and controller in use, these blocks are handled in a variety of ways. On simple disks, such as some disks with IDE controllers, bad blocks are handled manually. One strategy is to scan the disk to find bad blocks while the disk is being formatted. Any bad blocks that are discovered are flagged as unusable so that the file system does not allocate them. If blocks go bad during normal operation, a special program (such as the Linux badblocks command) must be run manually to search for the bad blocks and to lock them away. Data that resided on the bad blocks usually are lost. Swap-Space Management RAID Data Striping Bit-Level striping Swap-space management is another low-level task of the operating system. Virtual memory uses disk space as an extension of main memory. Since disk access is much slower than memory access, using swap space significantly decreases system performance. The main goal for the design and implementation of swap space is to provide the best throughput for the virtual memory system. A swap file (or swap space or, in Windows NT, a page file) is a space on a hard disk used as the virtual memory extension of a computer's real memory (RAM). Having a swap file allows your computer's operating system to pretend that you have more RAM than you actually do. RAID (Redundant Array of Inexpensive / Independent Disks) is a data storage virtualization technology that combines multiple physical disk drive components into a single logical unit for the purposes of data redundancy, performance improvement, or both. In computer data storage, data striping is the technique of segmenting logically sequential data, such as a file, so that consecutive segments are stored on different physical storage devices. Striping is useful when a processing device requests data more quickly than a single storage device can provide it. It also consists of splitting the bits of each byte across multiple disks; such striping is called bit-level striping. Bit-level striping consists of splitting the bits of each byte across multiple disks. Block-Level striping RAID Levels Stable-Storage implementation Tertiary-Storage structure In block-level striping, for instance, blocks of a file are striped across multiple disks; with n disks, block i of a file goes to disk (i mod n) + 1. Other levels of striping, such as bytes of a sector or sectors of a block, also are possible. Block-level striping is the most common. Mirroring provides high reliability, but it is expensive. Striping provides high data-transfer rates, but it does not improve reliability. Numerous schemes to provide redundancy at lower cost by using disk striping combined with parity bits (which we describe shortly) have been proposed. These schemes have different cost performance trade-offs and are classified according to levels called RAID Levels. Most often though, stable storage functionality is achieved by mirroring data on separate disks via RAID technology (level 1 or greater). The RAID controller implements the disk writing algorithms that enable separate disks to act as stable storage. Primary storage refers to computer memory chips; Secondary storage refers to fixed-disk storage systems (hard drives); And Tertiary Storage refers to removable media, such as tape drives, CDs, DVDs, and to a lesser extend floppies, thumb drives, and other detachable devices.

11 Removable Media Removable media is any type of storage device that can be removed from a computer while the system is running. Examples of removable media include CDs, DVDs and Blu-Ray disks, as well as diskettes and USB drives. Removable media makes it easy for a user to move data from one computer to another. Study Questions 1. What is disk attachment? Name several kinds of disk attachment, outline their differences, and outline their differences in usability. Computers access disk storage in two ways. One way is via I/O ports (or host-attached storage); this is common on small systems. The other way is via a remote host in a distributed file system; this is referred to as network-attached storage. Host-attached storage is storage accessed through local I/O ports. These ports use several technologies. The typical desktop PC uses an I/O bus architecture called IDE or ATA. This architecture supports a maximum of two drives per I/O bus. A newer, similar protocol that has simplified cabling is SATA. A network-attached storage (NAS) device is a special-purpose storage system that is accessed remotely over a data network (Figure 10.2). Clients access network-attached storage via a remote-procedure-call interface such as NFS for UNIX systems or CIFS for Windows machines. The remote procedure calls (RPCs) are carried via TCP or UDP over an IP network usually the same local area network (LAN) that carries all data traffic to the clients. Thus, it may be easiest to think of NAS as simply another storage-access protocol. One drawback of network-attached storage systems is that the storage I/O operations consume bandwidth on the data network, thereby increasing the latency of network communication. This problem can be particularly acute in large client server installations the communication between servers and clients competes for bandwidth with the communication among servers and storage devices. A storage-area network (SAN) is a private network (using storage protocols rather than networking protocols) connecting servers and storage units. 2. What are the main disk scheduling algorithms, and how do they work? Given the following queue -- 95, 180, 34, 119, 11, 123, 62, 64 with the Read-write head initially at the track 50 and the tail track being at 199 let us now discuss the different algorithms. 1. First Come -First Serve (FCFS): All incoming requests are placed at the end of the queue. Whatever number that is next in the queue will be the next number served. Using this algorithm doesn't provide the best results. To determine the number of head movements you would simply find the number of tracks it took to move from one request to the next. For this case it went from 50 to 95 to 180 and so on. From 50 to 95 it moved 45 tracks. If you tally up the total number of tracks you will find how many tracks it had to go through before finishing the entire request. In this example, it had a total head movement of 640 tracks. The disadvantage of this algorithm is noted by the oscillation from track 50 to track 180 and then back to track 11 to 123 then to 64. As you will soon see, this is the worse algorithm that one can use. 2. Shortest Seek Time First (SSTF): In this case request is serviced according to next shortest distance. Starting at 50, the next shortest distance would be 62 instead of 34 since it is only 12 tracks away from 62 and 16 tracks away from 34. The process would continue until all the process are taken care of. For example the next case would be to move from 62 to 64 instead of 34 since there are only 2 tracks between them and not 18 if it were to go the other way. Although this seems to be a better service being that it moved a total of 236 tracks, this is not an optimal one. There is a great chance that starvation would take place. The reason for this is if there were a lot of requests close to each other the other requests will never be handled since the distance will always be greater. 3. Elevator (SCAN): This approach works like an elevator does. It scans down towards the nearest end and then when it hits the bottom it scans up servicing the requests that it didn't get going down. If a request comes in after it has been scanned it will not be serviced until the process comes back down or moves back up. This process moved a total of 230 tracks. Once again this is more optimal than the previous algorithm, but it is not the best.

12 4. Circular Scan (C-SCAN): Circular scanning works just like the elevator to some extent. It begins its scan toward the nearest end and works it way all the way to the end of the system. Once it hits the bottom or top it jumps to the other end and moves in the same direction. Keep in mind that the huge jump doesn't count as a head movement. The total head movement for this algorithm is only 187 track, but still this isn't the most sufficient. 5. C-LOOK: This is just an enhanced version of C-SCAN. In this the scanning doesn't go past the last request in the direction that it is moving. It too jumps to the other end but not all the way to the end. Just to the furthest request. C- SCAN had a total movement of 187 but this scan (C-LOOK) reduced it down to 157 tracks. 3. What are the main tasks in disk management? Disk Management is used to manage the drives installed in a computer - like hard disk drives (internal and external), optical disk drives, and flash drives. It can be used to partition drives, format drives, assign drive letters, and much more. 4. What are RAID levels, and how does one determine which level to use for a specific computer system? RAID used to stand for redundant array of inexpensive disks. Today, the term has been updated to redundant array of independent disks. RAID is a way of grouping individual physical drives together to form one bigger drive called a RAID set. The RAID set represents all the smaller physical drives as one logical disk to your server. The logical disk is called a logical unit number, or LUN. Using RAID has two main advantages: better performance and higher availability, which means it goes faster and breaks down less often. The primary benefits of using RAID are performance improvements, resiliency and low costs. Performance is increased because the server has more spindles to read from or write to when data is accessed from a drive. Availability and resiliency are increased because the RAID controller can recreate lost data from parity information. Which RAID array you choose to use depends solely on the need being addressed (i.e.) Level 0: Striping. Level 1: Mirroring, Level 2: Memory Style ECC etc 5. What technologies are used in new tertiary storage devices? SSD (DRAM) CD, DVD, HD DVD Blue-Ray USB Flash Drive. 3.3 File-System Interface and Implementation Overview The main criterion for a good file system is that it provide the user with convenient (user friendly), effective, and secure means of organizing and accessing his or her files. This section examines the file-system concepts familiar to most users, such as file names, file types, directory structures, and file protection. Section 3.3 also discusses some of the practical problems, block-allocation schemes, performance criteria, and backup and recovery schemes that system designers must consider when they implement file systems on disk. Learning Outcomes After you have completed Section 3.3, you should be able to: Explain the function of a file systems and their interfaces. Describe the concepts of file, file attributes, file operations, file organization, and file access. Outline the different directory structures. Describe file system mounting and file sharing. Describe the nature of protection schemes, including the concept of owner, group, and universe (world) categories, and the concept of access lists. Explain the layered approach to file system organization. Describe the implementation of a local file system and directory.

13 Explain contiguous, linked, and indexed disk allocation strategies, and the benefits, and problems associated with each. Discuss free-space management and its possible implementations. Discuss the importance of backup and recovery. Discuss efficiency and performance issues related to file systems and organization. Reading Assignment Operating system concepts (9th ed.): Chapter 11: File-System Interface: 11.1 to As you do the assigned reading, focus on the Key Concepts and Topics outlined below to ensure that you can meet Learning Outcomes 1-5 above. Key Concepts and Topics Open File Table File Organization File Structure Packing The operating system keeps a table, called the open-file table, containing information about all open files. When a file operation is requested, the file is specified via an index into this table, so no searching is required. The file-organization module knows about files and their logical blocks, as well as physical blocks. By knowing the type of file allocation used and the location of the file, the file-organization module can translate logical block addresses to physical block addresses for the basic file system to transfer. Each file s logical blocks are numbered from 0 (or 1) through N. The file system structure is the most basic level of organization in an operating system. Almost all of the ways an operating system interacts with its users, applications, and security model are dependent upon the way it organizes files on storage devices. Variable vs. static files. Internal Fragmentation, Files Access Method Sequential Access Relative Access Internal fragmentation is the wasted space within each allocated block because of rounding up from the actual requested allocation to the allocation granularity. External fragmentation is the various free spaced holes that are generated in either your memory or disk space. An access method is a function of a mainframe operating system that enables access to data on disk, tape or other external devices. They were introduced in 1963 in IBM OS/360 operating system. The simplest access method is sequential access. Information in the file is processed in order, one record after the other. This mode of access is by far the most common; for example, editors and compilers usually access files in this fashion. Another method is direct access (or relative access). Here, a file is made up of fixed-length logical records that allow programs to read and write records rapidly in no particular order. The direct-access method is based on a disk model of a file, since disks allow random access to any file block. Logical Record Random Access Indexed Indexed Sequential Access A logical record may be a byte, a line (of fixed or variable length), or a more complex data item. The process of transferring information to or from memory in which every memory location can be accessed directly rather than being accessed in a fixed sequence. Indexing mechanisms are used to optimize certain accesses to data (records) managed in files. For example, the author catalog in a library is a type of index. Search Key (definition): attribute or combination of attributes used to look up records in a file. Indexed sequential-access method (ISAM) uses a small master index that points to disk blocks of a secondary index. The secondary index blocks point to the actual file blocks. The file is kept sorted on a defined key.