# Disks and I/O Hakan Uraz - File Organization 1

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Disks and I/O 2006 Hakan Uraz - File Organization 1

2 Disk Drive 2006 Hakan Uraz - File Organization 2

3 Tracks and Sectors on Disk Surface 2006 Hakan Uraz - File Organization 3

4 A Set of Cylinders on Disk Drive 2006 Hakan Uraz - File Organization 4

5 Capacities Track capacity = number of sectors per track X bytes per sector Cylinder capacity = number of tracks per cylinder X track capacity Drive capacity = number of cylinders X cylinder capacity 2006 Hakan Uraz - File Organization 5

6 Example We want to store a file with fixedlength data records on a 300-megabyte small disk with the following characteristics: Number of bytes per sector = 512 Number of sectors per track = 40 Number of tracks per cylinder = 11 Number of cylinders = 1331 How many cylinders does the file require if each data record requires 256 bytes? 2006 Hakan Uraz - File Organization 6

7 Answer Since each sector can hold two records, the file requires 20000/2 = sectors. One cylinder can hold 40 X 11 = 440 sectors. So the number of cylinders required is 10000/440 = 22.7 cylinders. It may be that a disk drive with 22.7 cylinders of available space does not have 22.7 physically contiguous cylinders available. The file may be spread of hundreds of cylinders Hakan Uraz - File Organization 7

8 Organizing data on a disk Organizing tracks by sector, or Organizing tracks by user-defined blocks Hakan Uraz - File Organization 8

9 Organizing Tracks by Sector The physical placement of sectors The simplest view is that sectors are adjacent fixed-sized segments of a track that happen to hold a file. It is an adequate way to view a file logically, but it may not be a good way to store sectors physically Hakan Uraz - File Organization 9

10 Organizing Tracks by Sector The physical placement of sectors To read a series of sectors, you often cannot read adjacent sectors. It takes the disk controller some time to process the received information before it is ready to accept more. Consequently, we would be able to read only one sector per revolution of the disk Hakan Uraz - File Organization 10

11 Organizing Tracks by Sector Interleaving Hence sectors are interleaved, leaving an interval of several physical sectors between logically adjacent sectors. In the following figure, interleaving factor is 5. It takes five revolutions to read 32 sectors. An improvement over 32 revolutions. High performance disks can offer 1:1 interleaving Hakan Uraz - File Organization 11

12 Interleaving 2006 Hakan Uraz - File Organization 12

13 Organizing Tracks by Sector Clusters The view maintained by Operating System s file manager. A cluster is a fixed number of contiguous sectors. It is not always physically contiguous; the degree of physical contiguity is determined by the interleaving factor. All sectors in a cluster can be accessed without requiring additional seeks Hakan Uraz - File Organization 13

14 Organizing Tracks by Sector Clusters To view a file as a series of clusters and still maintain the sectored view, the file manager ties logical sectors to the physical clusters that they belong to by using a file allocation table (FAT). The FAT contains a list of all the clusters in a file, ordered according to the logical order of the sectors they contain. With each cluster entry in the FAT is an entry giving the physical location of the cluster Hakan Uraz - File Organization 14

15 Organizing Tracks by Sector 2006 Hakan Uraz - File Organization 15

16 Organizing Tracks by Sector Extents A further attempt to emphasize physical contiguity of sectors in a file, hence minimizing seeking even more. It may be possible to make a file consist entirely of contiguous clusters. Then, we say that the file consists of one extent: All of its sectors, tracks, and (if it is large enough) cylinders form one contiguous whole. If there is not enough contiguous space available to contain an entire file, the file is divided into two or more noncontiguous parts. Each part is an extent Hakan Uraz - File Organization 16

17 Organizing Tracks by Sector 2006 Hakan Uraz - File Organization 17

18 Organizing Tracks by Sector Fragmentation If the size of a sector is 512 bytes and the size of all records in a file is 300 bytes, there is no convenient fit between records and sectors. Solutions: 1. Store only one record per sector. 2. Allow records to span sectors, so the beginning of a record might be found in one sector and the end of it in another Hakan Uraz - File Organization 18

19 Organizing Tracks by Sector 2006 Hakan Uraz - File Organization 19

20 Organizing Tracks by Sector Fragmentation The first option has the advantage that any record can be retrieved by retrieving just one sector, but it has the disadvantage that it might leave an unused space within each sector called internal fragmentation. The second option has the advamtage that it loses no space from internal fragmentation, but it has the disadvantage that some records may be retrieved only by accessing two sectors Hakan Uraz - File Organization 20

21 Organizing Tracks by Sector Fragmentation The use of clusters may also result in internal fragmentation. When the number of bytes in a file is not an exact multiple of the cluster size, there will be internal fragmentation in the last extent of the file. A disk holding smaller files or files that are accessed only randomly would normally be set up with small clusters Large files processed sequentially are usually given large cluster sizes for performance Hakan Uraz - File Organization 21

22 Organizing Tracks by Block Sometimes disk tracks are not divided into sectors, but into integral numbers of user-defined blocks whose size can vary. The amount of data transferred in a single I/O operation can vary depending on the needs of the software designer, not the hardware. Blocks can normally be either fixed or variable in length, depending on the requirements of the file designer. As with sectors, blocks are often referred to as physical records Hakan Uraz - File Organization 22

23 Organizing Tracks by Block 2006 Hakan Uraz - File Organization 23

24 Organizing Tracks by Block A block organization does not present the sectorspanning and fragmentation problems of sectors because blocks can vary in size to fit the logical organization of the data. A block is usually organized to hold an integral number of logical records. The term blocking factor is used to indicate the number of records that are to be stored in each block in a file. Blocks are superior to sectors when it is desirable to have the physical allocation of space for records correspond to their logical organization Hakan Uraz - File Organization 24

25 Organizing Tracks by Block Each block of data is usually accompanied by one or more subblocks containing extra information about the data block. Count subblock contains (among other things) the number of bytes in the data block. There may be a key subblock containing the key for the last record in the data block. When key subblocks are used, a track can be searched by the disk controller for a block or record identified by a given key Hakan Uraz - File Organization 25

26 Organizing Tracks by Block 2006 Hakan Uraz - File Organization 26

27 Nondata Overhead Some of the overhead consists of information that is stored on the disk during preformatting. On sector-addressable disks, preformatting involves storing, at the beginning of each sector, such information as sector address, track address, and condition (whether the sector is usable or defective). Preformatting also involves placing gaps and synchronization marks between fields of information to help the read/write mechanism distinguish between them. These overhead is of no concern to programmer Hakan Uraz - File Organization 27

28 Nondata Overhead On a block-organized disk, some overhead is invisible some is visible to programmer. Since subblocks and interblock gaps have to be provided with every block, there is more overhead with blocks than with sectors. Also, since the number and sizes of blocks can vary from one application to another, the relative amount of space taken up by overhead can vary when block addressing is used Hakan Uraz - File Organization 28

29 Nondata Overhead Suppose we have a block-addressable disk drive with bytes per track, and the amount of space taken up by subblocks and interblock gaps is equivalent to 300 bytes per block. We want to store a file containing 100-byte records on the disk. How many records can be stored per track if the blocking factor is 10, or if it is 60? 2006 Hakan Uraz - File Organization 29

30 Nondata Overhead If there are byte records per block, each block holds 1000 bytes of data and uses or 1300 bytes of track space. The number of blocks which can fit on a byte track can be expressed as floor(20000/1300) = 15 So 15 blocks or 150 records can be stored per track. Note that we take the floor of the result because a block cannot span two tracks Hakan Uraz - File Organization 30

31 Nondata Overhead If there are byte records per block, each block holds 6000 bytes of data and uses 6300 bytes of track space. The number of blocks per track is floor(20000/6300) = 3 So 3 blocks, or 180 records, can be stored per track. Clearly, the larger blocking factor can lead to more efficient use of storage Hakan Uraz - File Organization 31

32 Nondata Overhead Larger blocking factors do not always lead to more efficient storage utilization. Since we can put only an integral number of blocks on a track, we lose some space at the end of the track. Here we have the internal fragmentation problem again within a track. The greater the block size, the greater potential amount of internal track fragmentation Hakan Uraz - File Organization 32

33 Nondata Overhead The use of blocks, rather than sectors introduces flexibility, since it lets the programmer determine how data are to be organized physically. As a disadvantage, this is a requirement for programmer and/or operating system. Also, the flexibility introduced precludes the synchronization of I/O operations with the physical movement of the disk. This means strategies such as sector interleaving cannot be used to improve performance Hakan Uraz - File Organization 33

34 The Cost of a Disk Access Three distinct physical operations: Seek time Rotational delay (rotational latency) Transfer time Hakan Uraz - File Organization 34

35 The Cost of a Disk Access Seek Time The time required to move the access arm to the correct cylinder. Seeking is likely to be more costly in a multiuser environment. Average seek time is the time to traverse 1/3 of the total number of cylinders Hakan Uraz - File Organization 35

36 The Cost of a Disk Access Rotational Delay The time it takes for the disk to rotate so the sector we want is under the read/write head. On average, the rotational delay is half a revolution Hakan Uraz - File Organization 36

37 The Cost of a Disk Access 2006 Hakan Uraz - File Organization 37

38 The Cost of a Disk Access Transfer Time Once the data we want is under the read/write head, it can be transferred. Transfer time = number of bytes transferred number of bytes on a track X rotation time 2006 Hakan Uraz - File Organization 38

39 The Cost of a Disk Access Let s compare the time it takes to access a file in sequence with the time it takes to access all of the records in the file randomly 2006 Hakan Uraz - File Organization 39

40 Specifications of disk drive Minimum seek time 6 msec Average seek time 18 msec Rotational delay 8.3 msec Maximum transfer rate 16.7 msec/track, or 1229 bytes/msec Bytes per sector 512 Sectors per track 40 Tracks per cylinder 11 Tracks per surface 1331 Interleave factor 1 Cluster size 8 sectors Smallest extent size 5 clusters 2006 Hakan Uraz - File Organization 40

41 The Cost of a Disk Access Smallest extent is 5 clusters. Space is allocated in one-track units. Let s read a 2048 Kbyte file that is divided into byte records. A cluster holds 16 records. There are 500 clusters to hold the file in 100 extents occupying 100 tracks Hakan Uraz - File Organization 41

42 To read sector by sector in sequence Average seek 18 msec Rotational delay 8.3 msec Read one track 16.7 msec Total 43 msec (Assume 100 tracks are randomly dispersed) Total time = 100 X 43 msec = 4.3 seconds 2006 Hakan Uraz - File Organization 42

43 To read same records with random access For each record Average seek 18 msec Rotational delay 8.3 msec Read one cluster (1/5 X 16.7) 3.3 msec Total 29.6 msec Total time = 8000 X 29.6 msec = seconds Hakan Uraz - File Organization 43

44 Disk as Bottleneck To speed up system, multiprogramming is used. Another technique is disk striping which involves splitting the parts of a file on several different drives, then letting the separate drives deliver data simultaneously. A RAM disk is a large part of RAM configured to simulate the behavior of a mechanical disk in every respect except speed and volatility Hakan Uraz - File Organization 44

45 Disk as Bottleneck A disk cache is a large block of RAM configured to contain pages of data from a disk. When data is requested from secondary memory, the file manager first looks into the disk cache if it contains the data. Only if cache does not contain, disk is used. Caches improve performance because of the programs locality of references Hakan Uraz - File Organization 45

46 A Journey of a Byte What happens when a program writes a byte to a file on a disk? What happens between the program and the disk? Suppose we want to append a byte representing the character P stored in a character variable c to a file named in the variable TEXT. From the program s point of view, the journey is represented by the statement WRITE(TEXT, c, 1); But the journey is longer than that Hakan Uraz - File Organization 46

47 A Journey of a Byte The WRITE() statement results in a call to the operating system (OS). OS File manager deals with files and I/O. File manager may be thought of layers of procedures. Each layer calls the one below. At the lowest level, the byte is actually written to disk Hakan Uraz - File Organization 47

48 2006 Hakan Uraz - File Organization 48

49 A Journey of a Byte Instead of sending the sector immediately to the disk, the file manager usually waits to see if it can accumulate more bytes going to the same sector before transmitting. Also when the file is closed, the output buffers are flushed to disk Hakan Uraz - File Organization 49

50 A Journey of a Byte The Byte Leaves RAM: The I/O Processor and Disk Controller Because of bottlenecks created by the differences in speed and data-path widths, our byte might have to wait for an external data path to become available. Because of speed differences, the CPU can transmit to several external devices simultaneously Hakan Uraz - File Organization 50

51 The I/O Processor A Journey of a Byte I/O Processor disassembles and assembles groups of bytes for transmission to/from devices. It takes the instructions from the OS but once it begins processing I/O, it runs independently. So, I/O processes and internal computing overlap. It can take data directly from RAM without the involvement of the CPU (direct memory access DMA). The file manager might tell the I/O processor that there is data in the buffer, how much data there is, and where it is to go on the disk. This info might be in the form of a little program that the OS constructs and I/O processor executes Hakan Uraz - File Organization 51

52 A Journey of a Byte The Disk Controller The I/O processor asks the disk controller if the disk drive is available. If not, our byte will have to wait in its buffer. The disk drive is instructed to move its head to the track and sector where to store our byte and its companions (a mechanical movement). Now, the I/O processor (or the controller) can send out the bytes, one at a time, to the drive where they are written on the disk surface Hakan Uraz - File Organization 52

53 Buffer Management Involves working with large chunks of data in RAM, so the number of disk accesses can be reduced. Consider what happens if a program is performing both input and output on one character at a time, and only one I/O buffer is available. To avoid such problems there is the need for several system buffers Hakan Uraz - File Organization 53

54 Buffer Management Even if a program transmits data in only one direction, the use of a single system I/O buffer can slow it down considerably. (The operation of reading a sector from a disk is extremely slow compared to RAM speed). So, we use more than one buffer and have the I/O system filling the next sector or block of data while the CPU is processing the current one Hakan Uraz - File Organization 54

55 Multiple Buffering Suppose that a I/O bound program is only writing to disk. If two buffers are used and I/O-CPU overlapping is permitted, the CPU can be filling one buffer while the contents of the other are being transmitted to disk. When both tasks are finished, the roles of the buffers can be exchanged. This technique is called double buffering. The idea of swapping system buffers to allow processing and I/O to overlap need not be restricted to two buffers Hakan Uraz - File Organization 55

56 Multiple Buffering 2006 Hakan Uraz - File Organization 56

57 Buffer Pooling When a system buffer is needed, it is taken from a pool of available buffers and used. When the system receives a request to read a certain sector or block, it looks to see if one of its buffers already contains that sector or block. If no buffer contains it, then the system finds from its pool of buffers one that is not currently in use and loads the sector or block into it Hakan Uraz - File Organization 57

58 Buffer Pooling Sevaral schemes are used to decide which buffer to take from a buffer pool. One generally effective strategy is to take from the pool that buffer that is least recently used (LRU). When a buffer is accessed, it is put on a least-recentlyused queue, so it is allowed to retain its data until all other less-recently used buffers have been accessed. LRU strategy is based on the assumption that a block of data that has been used recently is more likely to be needed in the near future than one that has been used less recently Hakan Uraz - File Organization 58

59 Buffers Move Mode and Locate Mode When data must always be copied from a system buffer to a program buffer (or vice versa), the amount of time to perform the move can be substantial. This way of handling buffered data is called move mode. There are two ways to avoid move mode. Both techniques are examples of the approach called locate mode. These are: The file manager performing I/O directly between disk and the program s data area. The file manager using system buffers to handle I/O, but provide the program with the locations, through the use of pointer variables, of the system buffers Hakan Uraz - File Organization 59

60 I/O in UNIX The topmost I/O layer in UNIX deals with data primarily in logical terms. This layer consists of processes that impose certain logical views on files. Processes include shell routines, user programs that operate on files and library routines. Below this layer is the UNIX kernel, which incorporates all the rest of the layers. The kernel views all I/O as operating on a sequence of bytes, so once we pass control to the kernel all assumptions about the logical view of a file are gone Hakan Uraz - File Organization 60

61 UNIX-Kernel Let s illustrate the journey of a byte through the kernel. Considers the system call such as write(fd, &c, 1); The routines that let processes communicate directly with the kernel make up the system call interface Hakan Uraz - File Organization 61

62 UNIX-Kernel The kernel I/O system begins by connecting the file descriptor (fd) in your program to some file or device in the filesystem. It does this by proceeding through a series of four tables: A file descriptor table. An open file table, with information about open files A file allocation table, which is part of a structure called an index node. A table of index nodes, with one entry for each file in use Hakan Uraz - File Organization 62

63 UNIX-Kernel The file descriptor table is owned by the process (your program). The open file table and index node tables are owned by the kernel. The index node itself is part of the filesystem Hakan Uraz - File Organization 63

64 UNIX-Kernel Every process has its own descriptor table Hakan Uraz - File Organization 64

65 UNIX-Kernel It is possible for several different processes to refer to the same open file table entry, so one process could read part of a file, another process could read the next part, and so forth. On the other hand, if the same file is opened by two separate open() statements, two separate entries are made in the table, and the two processes operate on the file quite independently Hakan Uraz - File Organization 65

66 UNIX-Kernel The information in the open file table is transitory. The kernel needs more information about the file itself. This information is found in an index node referred to as an inode Hakan Uraz - File Organization 66

67 UNIX-Kernel An inode is more permanent than an open file table s entry (called file structure). A file structure exists only while a file is open for access, but an inode exists as long as its corresponding file exists. So, inode is kept on disk with the file (though not adjacent to the file). When a file is opened, a copy of its inode is loaded into RAM where it is added to the inode table for rapid access Hakan Uraz - File Organization 67

68 UNIX-Linking File Names to Files All references to files begin with a directory. A directory is a small file that contains, for each file, a file name together with a pointer to the file s inode on disk. This pointer is called a hard link. When a file is opened, this hard link is used to bring the inode into RAM and to set up the corresponding entry in the open file table. It is possible for several file names to point to the same inode. A field in the inode tells how many hard links there are to the inode Hakan Uraz - File Organization 68

69 UNIX-Linking File Names to Files Another kind of link, called a soft link (or symbolic link) links a file name to another file name, rather than to an actual file. Instead of being a pointer to an inode, a soft link is a pathname of some file. Since a symbolic link does not point to an actual file, it can refer to a directory or even a file in a different file system Hakan Uraz - File Organization 69

70 UNIX-Normal Files, Special Files, and Sockets Normal files are block I/O files. Special files represent a stream of characters (e.g. the first three file descriptors in the descriptor table) Sockets are abstractions that serve as endpoints for interprocess communication Hakan Uraz - File Organization 70

71 UNIX-Block I/O The UNIX view of a block device is similar to disk. It is a randomly addressable array of fixed blocks. No other organization (such as clusters) was imposed on the placement of files on disk Hakan Uraz - File Organization 71

72 UNIX-Device Drivers They perform the actual I/O between the I/O buffer and the device. For block I/O, the device driver s job is to take a block from a buffer, destined for one of these physical blocks, and see that it gets deposited in the proper physical place on the device Hakan Uraz - File Organization 72

73 UNIX-The Kernel and Filesystems A filesystem includes the directory structure, the directories, ordinary files, and the inodes that describe the files. All parts of a filesystem reside on disk, rather than in RAM. These parts are brought into RAM by the kernel as needed. One advantage of keeping the filesystem and I/O system distinct is that we can have separate filesystems that are organized differently, perhaps on different devices, but are accesible by the same kernel Hakan Uraz - File Organization 73

### 1. What is the difference between primary storage and secondary storage?

1. What is the difference between primary storage and secondary storage? Primary Storage is - Limited - Volatile - Expensive - Fast (May be accessed directly from the CPU) - Retrieving a single character

### Chapter 4 File Systems. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved

Chapter 4 File Systems File Systems The best way to store information: Store all information in virtual memory address space Use ordinary memory read/write to access information Not feasible: no enough

### Chapter 11: Implementing File Systems. Operating System Concepts 8 th Edition,

Chapter 11: Implementing File Systems, Silberschatz, Galvin and Gagne 2009 Chapter 11: Implementing File Systems File-System Structure File-System Implementation Directory Implementation Allocation Methods

### Chapter 11: File System Implementation. Objectives

Chapter 11: File System Implementation Objectives To describe the details of implementing local file systems and directory structures To describe the implementation of remote file systems To discuss block

### Secondary Storage (Chp. 5.4 disk hardware, Chp. 6 File Systems, Tanenbaum)

Secondary Storage (Chp. 5.4 disk hardware, Chp. 6 File Systems, Tanenbaum) Secondary Stora Introduction Secondary storage is the non volatile repository for (both user and system) data and programs. As

### Introduction to OS. File Management. MOS Ch. 4. Mahmoud El-Gayyar. Mahmoud El-Gayyar / Introduction to OS 1

Introduction to OS File Management MOS Ch. 4 Mahmoud El-Gayyar elgayyar@ci.suez.edu.eg Mahmoud El-Gayyar / Introduction to OS 1 File Management Objectives Provide I/O support for a variety of storage device

### CHAPTER 11: IMPLEMENTING FILE SYSTEMS (COMPACT) By I-Chen Lin Textbook: Operating System Concepts 9th Ed.

CHAPTER 11: IMPLEMENTING FILE SYSTEMS (COMPACT) By I-Chen Lin Textbook: Operating System Concepts 9th Ed. File-System Structure File structure Logical storage unit Collection of related information File

### e-pg Pathshala Subject: Computer Science Paper: Operating Systems Module 35: File Allocation Methods Module No: CS/OS/35 Quadrant 1 e-text

e-pg Pathshala Subject: Computer Science Paper: Operating Systems Module 35: File Allocation Methods Module No: CS/OS/35 Quadrant 1 e-text 35.1 Introduction File system is the most visible part of the

### Chapter 8: Virtual Memory. Operating System Concepts

Chapter 8: Virtual Memory Silberschatz, Galvin and Gagne 2009 Chapter 8: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating

### Disks, Memories & Buffer Management

Disks, Memories & Buffer Management The two offices of memory are collection and distribution. - Samuel Johnson CS3223 - Storage 1 What does a DBMS Store? Relations Actual data Indexes Data structures

### File Management. Chapter 12

File Management Chapter 12 Files Used for: input to a program Program output saved for long-term storage Terms Used with Files Field basic element of data contains a single value characterized by its length

### Chapter-6. SUBJECT:- Operating System TOPICS:- I/O Management. Created by : - Sanjay Patel

Chapter-6 SUBJECT:- Operating System TOPICS:- I/O Management Created by : - Sanjay Patel Disk Scheduling Algorithm 1) First-In-First-Out (FIFO) 2) Shortest Service Time First (SSTF) 3) SCAN 4) Circular-SCAN

### Operating System Concepts

Chapter 9: Virtual-Memory Management 9.1 Silberschatz, Galvin and Gagne 2005 Chapter 9: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped

### Mass Storage. 2. What are the difference between Primary storage and secondary storage devices? Primary Storage is Devices. Secondary Storage devices

1. What are the logical organization of a file? Mass Storage 2. What are the difference between Primary storage and secondary storage devices? Primary Storage is Devices Secondary Storage devices - Limited,

### UNIT 4 Device Management

UNIT 4 Device Management (A) Device Function. (B) Device Characteristic. (C) Disk space Management. (D) Allocation and Disk scheduling Methods. [4.1] Device Management Functions The management of I/O devices

### Storing Data: Disks and Files

Storing Data: Disks and Files Chapter 9 CSE 4411: Database Management Systems 1 Disks and Files DBMS stores information on ( 'hard ') disks. This has major implications for DBMS design! READ: transfer

### Memory Management. q Basic memory management q Swapping q Kernel memory allocation q Next Time: Virtual memory

Memory Management q Basic memory management q Swapping q Kernel memory allocation q Next Time: Virtual memory Memory management Ideal memory for a programmer large, fast, nonvolatile and cheap not an option

### Chapter 11: Implementing File

Chapter 11: Implementing File Systems Chapter 11: Implementing File Systems File-System Structure File-System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency

### Operating Systems Memory Management. Mathieu Delalandre University of Tours, Tours city, France

Operating Systems Memory Management Mathieu Delalandre University of Tours, Tours city, France mathieu.delalandre@univ-tours.fr 1 Operating Systems Memory Management 1. Introduction 2. Contiguous memory

### Disk Scheduling COMPSCI 386

Disk Scheduling COMPSCI 386 Topics Disk Structure (9.1 9.2) Disk Scheduling (9.4) Allocation Methods (11.4) Free Space Management (11.5) Hard Disk Platter diameter ranges from 1.8 to 3.5 inches. Both sides

### Chapter 11: Implementing File Systems. Operating System Concepts 9 9h Edition

Chapter 11: Implementing File Systems Operating System Concepts 9 9h Edition Silberschatz, Galvin and Gagne 2013 Chapter 11: Implementing File Systems File-System Structure File-System Implementation Directory

### Chapter 17: Distributed-File Systems. Operating System Concepts 8 th Edition,

Chapter 17: Distributed-File Systems, Silberschatz, Galvin and Gagne 2009 Chapter 17 Distributed-File Systems Background Naming and Transparency Remote File Access Stateful versus Stateless Service File

### Virtual Memory. Chapter 8

Virtual Memory 1 Chapter 8 Characteristics of Paging and Segmentation Memory references are dynamically translated into physical addresses at run time E.g., process may be swapped in and out of main memory

### OPERATING SYSTEM. Chapter 9: Virtual Memory

OPERATING SYSTEM Chapter 9: Virtual Memory Chapter 9: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating Kernel Memory

### Addresses in the source program are generally symbolic. A compiler will typically bind these symbolic addresses to re-locatable addresses.

1 Memory Management Address Binding The normal procedures is to select one of the processes in the input queue and to load that process into memory. As the process executed, it accesses instructions and

### Che-Wei Chang Department of Computer Science and Information Engineering, Chang Gung University

Che-Wei Chang chewei@mail.cgu.edu.tw Department of Computer Science and Information Engineering, Chang Gung University Chapter 10: File System Chapter 11: Implementing File-Systems Chapter 12: Mass-Storage

### CS3600 SYSTEMS AND NETWORKS

CS3600 SYSTEMS AND NETWORKS NORTHEASTERN UNIVERSITY Lecture 11: File System Implementation Prof. Alan Mislove (amislove@ccs.neu.edu) File-System Structure File structure Logical storage unit Collection

### OPERATING SYSTEM. Chapter 12: File System Implementation

OPERATING SYSTEM Chapter 12: File System Implementation Chapter 12: File System Implementation File-System Structure File-System Implementation Directory Implementation Allocation Methods Free-Space Management

### Storing and Retrieving Data. Storing Data: Disks and Files. Solution 1: Techniques for making disks faster. Disks. Why Not Store Everything in Tapes?

Storing and Retrieving Storing : Disks and Files Chapter 9 base Management Systems need to: Store large volumes of data Store data reliably (so that data is not lost!) Retrieve data efficiently Alternatives

### Chapter 12: File System Implementation

Chapter 12: File System Implementation Silberschatz, Galvin and Gagne 2013 Chapter 12: File System Implementation File-System Structure File-System Implementation Allocation Methods Free-Space Management

### MEMORY MANAGEMENT/1 CS 409, FALL 2013

MEMORY MANAGEMENT Requirements: Relocation (to different memory areas) Protection (run time, usually implemented together with relocation) Sharing (and also protection) Logical organization Physical organization

### Chapter 11: Implementing File Systems

Chapter 11: Implementing File Systems Operating System Concepts 99h Edition DM510-14 Chapter 11: Implementing File Systems File-System Structure File-System Implementation Directory Implementation Allocation

### Operating Systems. Lecture File system implementation. Master of Computer Science PUF - Hồ Chí Minh 2016/2017

Operating Systems Lecture 7.2 - File system implementation Adrien Krähenbühl Master of Computer Science PUF - Hồ Chí Minh 2016/2017 Design FAT or indexed allocation? UFS, FFS & Ext2 Journaling with Ext3

### Directory Structure and File Allocation Methods

ISSN:0975-9646 Mandeep Kaur et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 7 (2), 2016, 577-582 Directory Structure and ile Allocation Methods Mandeep Kaur,

### Chapter 13 Disk Storage, Basic File Structures, and Hashing.

Chapter 13 Disk Storage, Basic File Structures, and Hashing. Copyright 2004 Pearson Education, Inc. Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files Ordered Files

### Storing Data: Disks and Files

Storing Data: Disks and Files Chapter 9 Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke Disks

### Memory Management. To do. q Basic memory management q Swapping q Kernel memory allocation q Next Time: Virtual memory

Memory Management To do q Basic memory management q Swapping q Kernel memory allocation q Next Time: Virtual memory Memory management Ideal memory for a programmer large, fast, nonvolatile and cheap not

### CS6401- Operating System QUESTION BANK UNIT-IV

Part-A QUESTION BANK UNIT-IV 1. What is a File? A file is a named collection of related information that is recorded on secondary storage. A file contains either programs or data. A file has certain structure

### Operating Systems CMPSC 473. File System Implementation April 1, Lecture 19 Instructor: Trent Jaeger

Operating Systems CMPSC 473 File System Implementation April 1, 2008 - Lecture 19 Instructor: Trent Jaeger Last class: File System Interface Today: File System Implementation Disks as Secondary Store What

### L9: Storage Manager Physical Data Organization

L9: Storage Manager Physical Data Organization Disks and files Record and file organization Indexing Tree-based index: B+-tree Hash-based index c.f. Fig 1.3 in [RG] and Fig 2.3 in [EN] Functional Components

### 20-EECE-4029 Operating Systems Fall, 2015 John Franco

20-EECE-4029 Operating Systems Fall, 2015 John Franco Final Exam name: Question 1: Processes and Threads (12.5) long count = 0, result = 0; pthread_mutex_t mutex; pthread_cond_t cond; void *P1(void *t)

### Principles of Operating Systems

Principles of Operating Systems Lecture 21-23 - Virtual Memory Ardalan Amiri Sani (ardalan@uci.edu) [lecture slides contains some content adapted from previous slides by Prof. Nalini Venkatasubramanian,

### Virtual Memory. Reading. Sections 5.4, 5.5, 5.6, 5.8, 5.10 (2) Lecture notes from MKP and S. Yalamanchili

Virtual Memory Lecture notes from MKP and S. Yalamanchili Sections 5.4, 5.5, 5.6, 5.8, 5.10 Reading (2) 1 The Memory Hierarchy ALU registers Cache Memory Memory Memory Managed by the compiler Memory Managed

### Chapter 4 - File Systems

Chapter 4 - File Systems Luis Tarrataca luis.tarrataca@gmail.com CEFET-RJ L. Tarrataca Chapter 4 - File Systems 1 / 159 1 Motivation 2 Files File Naming File Structure File Types File Access File Attributes

### Storing Data: Disks and Files. Storing and Retrieving Data. Why Not Store Everything in Main Memory? Chapter 7

Storing : Disks and Files Chapter 7 base Management Systems, R. Ramakrishnan and J. Gehrke 1 Storing and Retrieving base Management Systems need to: Store large volumes of data Store data reliably (so

### Chapter 9: Virtual Memory

Chapter 9: Virtual Memory Silberschatz, Galvin and Gagne 2013 Chapter 9: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating

### File System Implementation

File System Implementation Last modified: 16.05.2017 1 File-System Structure Virtual File System and FUSE Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance. Buffering

### Chapter 9: Virtual Memory. Operating System Concepts 9 th Edition

Chapter 9: Virtual Memory Silberschatz, Galvin and Gagne 2013 Chapter 9: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating

### Principles of Data Management. Lecture #2 (Storing Data: Disks and Files)

Principles of Data Management Lecture #2 (Storing Data: Disks and Files) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Topics v Today

### CS 162 Operating Systems and Systems Programming Professor: Anthony D. Joseph Spring Lecture 18: Naming, Directories, and File Caching

CS 162 Operating Systems and Systems Programming Professor: Anthony D. Joseph Spring 2004 Lecture 18: Naming, Directories, and File Caching 18.0 Main Points How do users name files? What is a name? Lookup:

### Operating Systems Unit 6. Memory Management

Unit 6 Memory Management Structure 6.1 Introduction Objectives 6.2 Logical versus Physical Address Space 6.3 Swapping 6.4 Contiguous Allocation Single partition Allocation Multiple Partition Allocation

### Chapter 9: Virtual Memory. Operating System Concepts 9th Edition

Chapter 9: Virtual Memory Chapter 9: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating Kernel Memory Other Considerations

### File system internals Tanenbaum, Chapter 4. COMP3231 Operating Systems

File system internals Tanenbaum, Chapter 4 COMP3231 Operating Systems Architecture of the OS storage stack Application File system: Hides physical location of data on the disk Exposes: directory hierarchy,

### Chapter 12: File System Implementation

Chapter 12: File System Implementation Chapter 12: File System Implementation File-System Structure File-System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency

### Chapter 10: File System Implementation

Chapter 10: File System Implementation Chapter 10: File System Implementation File-System Structure" File-System Implementation " Directory Implementation" Allocation Methods" Free-Space Management " Efficiency

### Local File Stores. Job of a File Store. Physical Disk Layout CIS657

Local File Stores CIS657 Job of a File Store Recall that the File System is responsible for namespace management, locking, quotas, etc. The File Store s responsbility is to mange the placement of data

### File-System Structure

Chapter 12: File System Implementation File System Structure File System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery Log-Structured

### Final Exam Preparation Questions

EECS 678 Spring 2013 Final Exam Preparation Questions 1 Chapter 6 1. What is a critical section? What are the three conditions to be ensured by any solution to the critical section problem? 2. The following

### Storing Data: Disks and Files

Storing Data: Disks and Files Module 2, Lecture 1 Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Database Management Systems, R. Ramakrishnan 1 Disks and

### File System: Interface and Implmentation

File System: Interface and Implmentation Two Parts Filesystem Interface Interface the user sees Organization of the files as seen by the user Operations defined on files Properties that can be read/modified

### STORING DATA: DISK AND FILES

STORING DATA: DISK AND FILES CS 564- Spring 2018 ACKs: Dan Suciu, Jignesh Patel, AnHai Doan WHAT IS THIS LECTURE ABOUT? How does a DBMS store data? disk, SSD, main memory The Buffer manager controls how

### Da-Wei Chang CSIE.NCKU. Professor Hao-Ren Ke, National Chiao Tung University Professor Hsung-Pin Chang, National Chung Hsing University

Chapter 11 Implementing File System Da-Wei Chang CSIE.NCKU Source: Professor Hao-Ren Ke, National Chiao Tung University Professor Hsung-Pin Chang, National Chung Hsing University Outline File-System Structure

### Disks & Files. Yanlei Diao UMass Amherst. Slides Courtesy of R. Ramakrishnan and J. Gehrke

Disks & Files Yanlei Diao UMass Amherst Slides Courtesy of R. Ramakrishnan and J. Gehrke DBMS Architecture Query Parser Query Rewriter Query Optimizer Query Executor Lock Manager for Concurrency Access

### Chapter 11 I/O Management and Disk Scheduling

Operating Systems: Internals and Design Principles, 6/E William Stallings Chapter 11 I/O Management and Disk Scheduling Patricia Roy Manatee Community College, Venice, FL 2008, Prentice Hall 1 2 Differences

### CSE380 - Operating Systems. Communicating with Devices

CSE380 - Operating Systems Notes for Lecture 15-11/4/04 Matt Blaze (some examples by Insup Lee) Communicating with Devices Modern architectures support convenient communication with devices memory mapped

### Chapter 8: Virtual Memory. Operating System Concepts Essentials 2 nd Edition

Chapter 8: Virtual Memory Silberschatz, Galvin and Gagne 2013 Chapter 8: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating

File Systems Part 1 Operating Systems In Depth XIV 1 Copyright 2018 Thomas W. Doeppner. All rights reserved. Files Memory Disk Disk Operating Systems In Depth XIV 2 Copyright 2018 Thomas W. Doeppner. All

### Chapter 9: Virtual Memory. Operating System Concepts 9 th Edition

Chapter 9: Virtual Memory Silberschatz, Galvin and Gagne 2013 Chapter 9: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating

### Operating Systems (2INC0) 2017/18

Operating Systems (2INC0) 2017/18 Memory Management (09) Dr. Courtesy of Dr. I. Radovanovic, Dr. R. Mak (figures from Bic & Shaw) System Architecture and Networking Group Agenda Reminder: OS & resources

### Input/Output Management

Chapter 11 Input/Output Management This could be the messiest aspect of an operating system. There are just too much stuff involved, it is difficult to develop a uniform and consistent theory to cover

### Computer Systems Laboratory Sungkyunkwan University

File System Internals Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics File system implementation File descriptor table, File table

### CHAPTER 8 - MEMORY MANAGEMENT STRATEGIES

CHAPTER 8 - MEMORY MANAGEMENT STRATEGIES OBJECTIVES Detailed description of various ways of organizing memory hardware Various memory-management techniques, including paging and segmentation To provide

### CS 4284 Systems Capstone

CS 4284 Systems Capstone Disks & File Systems Godmar Back Filesystems Files vs Disks File Abstraction Byte oriented Names Access protection Consistency guarantees Disk Abstraction Block oriented Block

### Chapter 9 Memory Management

Contents 1. Introduction 2. Computer-System Structures 3. Operating-System Structures 4. Processes 5. Threads 6. CPU Scheduling 7. Process Synchronization 8. Deadlocks 9. Memory Management 10. Virtual

### CS510 Operating System Foundations. Jonathan Walpole

CS510 Operating System Foundations Jonathan Walpole File System Performance File System Performance Memory mapped files - Avoid system call overhead Buffer cache - Avoid disk I/O overhead Careful data

### Review 1-- Storing Data: Disks and Files

Review 1-- Storing Data: Disks and Files Chapter 9 [Sections 9.1-9.7: Ramakrishnan & Gehrke (Text)] AND (Chapter 11 [Sections 11.1, 11.3, 11.6, 11.7: Garcia-Molina et al. (R2)] OR Chapter 2 [Sections 2.1,

### CS6401- Operating System UNIT-III STORAGE MANAGEMENT

UNIT-III STORAGE MANAGEMENT Memory Management: Background In general, to rum a program, it must be brought into memory. Input queue collection of processes on the disk that are waiting to be brought into

### Chapter 9: Peripheral Devices: Magnetic Disks

Chapter 9: Peripheral Devices: Magnetic Disks Basic Disk Operation Performance Parameters and History of Improvement Example disks RAID (Redundant Arrays of Inexpensive Disks) Improving Reliability Improving

### Disks and Files. Storage Structures Introduction Chapter 8 (3 rd edition) Why Not Store Everything in Main Memory?

Why Not Store Everything in Main Memory? Storage Structures Introduction Chapter 8 (3 rd edition) Sharma Chakravarthy UT Arlington sharma@cse.uta.edu base Management Systems: Sharma Chakravarthy Costs

### OPERATING SYSTEMS II DPL. ING. CIPRIAN PUNGILĂ, PHD.

OPERATING SYSTEMS II DPL. ING. CIPRIAN PUNGILĂ, PHD. File System Implementation FILES. DIRECTORIES (FOLDERS). FILE SYSTEM PROTECTION. B I B L I O G R A P H Y 1. S I L B E R S C H AT Z, G A L V I N, A N

### Lecture S3: File system data layout, naming

Lecture S3: File system data layout, naming Review -- 1 min Intro to I/O Performance model: Log Disk physical characteristics/desired abstractions Physical reality Desired abstraction disks are slow fast

### Disk scheduling Disk reliability Tertiary storage Swap space management Linux swap space management

Lecture Overview Mass storage devices Disk scheduling Disk reliability Tertiary storage Swap space management Linux swap space management Operating Systems - June 28, 2001 Disk Structure Disk drives are

### Chapter 12: File System Implementation. Operating System Concepts 9 th Edition

Chapter 12: File System Implementation Silberschatz, Galvin and Gagne 2013 Chapter 12: File System Implementation File-System Structure File-System Implementation Directory Implementation Allocation Methods

### General Objective:To understand the basic memory management of operating system. Specific Objectives: At the end of the unit you should be able to:

F2007/Unit6/1 UNIT 6 OBJECTIVES General Objective:To understand the basic memory management of operating system Specific Objectives: At the end of the unit you should be able to: define the memory management

### Chapter 8 Virtual Memory

Chapter 8 Virtual Memory Contents Hardware and control structures Operating system software Unix and Solaris memory management Linux memory management Windows 2000 memory management Characteristics of

### File Management. Ezio Bartocci.

File Management Ezio Bartocci ezio.bartocci@tuwien.ac.at Cyber-Physical Systems Group Institute for Computer Engineering Faculty of Informatics, TU Wien Motivation A process can only contain a limited

### Operating systems. Part 1. Module 11 Main memory introduction. Tami Sorgente 1

Operating systems Module 11 Main memory introduction Part 1 Tami Sorgente 1 MODULE 11 MAIN MEMORY INTRODUCTION Background Swapping Contiguous Memory Allocation Noncontiguous Memory Allocation o Segmentation

### CSCI-GA Database Systems Lecture 8: Physical Schema: Storage

CSCI-GA.2433-001 Database Systems Lecture 8: Physical Schema: Storage Mohamed Zahran (aka Z) mzahran@cs.nyu.edu http://www.mzahran.com View 1 View 2 View 3 Conceptual Schema Physical Schema 1. Create a

### Chapter 11: File System Implementation

Chapter 11: File System Implementation Chapter 11: File System Implementation File-System Structure File-System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency

### There is a general need for long-term and shared data storage: Files meet these requirements The file manager or file system within the OS

Why a file system? Why a file system There is a general need for long-term and shared data storage: need to store large amount of information persistent storage (outlives process and system reboots) concurrent

### Chapter 8 Virtual Memory

Operating Systems: Internals and Design Principles Chapter 8 Virtual Memory Seventh Edition William Stallings Modified by Rana Forsati for CSE 410 Outline Principle of locality Paging - Effect of page

### CHAPTER 3 RESOURCE MANAGEMENT

CHAPTER 3 RESOURCE MANAGEMENT SUBTOPIC Understand Memory Management Understand Processor Management INTRODUCTION Memory management is the act of managing computer memory. This involves providing ways to

### Chapters 9 & 10: Memory Management and Virtual Memory

Chapters 9 & 10: Memory Management and Virtual Memory Important concepts (for final, projects, papers) addressing: physical/absolute, logical/relative/virtual overlays swapping and paging memory protection

### Objectives and Functions Convenience. William Stallings Computer Organization and Architecture 7 th Edition. Efficiency

William Stallings Computer Organization and Architecture 7 th Edition Chapter 8 Operating System Support Objectives and Functions Convenience Making the computer easier to use Efficiency Allowing better

### Chapter 8: Main Memory. Operating System Concepts 9 th Edition

Chapter 8: Main Memory Silberschatz, Galvin and Gagne 2013 Chapter 8: Memory Management Background Swapping Contiguous Memory Allocation Segmentation Paging Structure of the Page Table Example: The Intel

### Chapter 5 The Memory System

Chapter 5 The Memory System 5.. The block diagram is essentially the same as in Figure 5., except that 6 rows (of four 5 8 chips) are needed. Address lines A 8 are connected to all chips. Address lines

### Chapter 9: Virtual Memory. Chapter 9: Virtual Memory. Objectives. Background. Virtual-address address Space

Chapter 9: Virtual Memory Chapter 9: Virtual Memory Background Demand Paging Copy-on-Write Page Replacement Allocation of Frames Thrashing Memory-Mapped Files Allocating Kernel Memory Other Considerations

### Principles of Operating Systems

Principles of Operating Systems Lecture 24-26 - File-System Interface and Implementation Ardalan Amiri Sani (ardalan@uci.edu) [lecture slides contains some content adapted from previous slides by Prof.

### Why Is This Important? Overview of Storage and Indexing. Components of a Disk. Data on External Storage. Accessing a Disk Page. Records on a Disk Page

Why Is This Important? Overview of Storage and Indexing Chapter 8 DB performance depends on time it takes to get the data from storage system and time to process Choosing the right index for faster access