Disks and I/O Hakan Uraz - File Organization 1

Transcription

1 Disks and I/O 2006 Hakan Uraz - File Organization 1

2 Disk Drive 2006 Hakan Uraz - File Organization 2

3 Tracks and Sectors on Disk Surface 2006 Hakan Uraz - File Organization 3

4 A Set of Cylinders on Disk Drive 2006 Hakan Uraz - File Organization 4

5 Capacities Track capacity = number of sectors per track X bytes per sector Cylinder capacity = number of tracks per cylinder X track capacity Drive capacity = number of cylinders X cylinder capacity 2006 Hakan Uraz - File Organization 5

6 Example We want to store a file with fixedlength data records on a 300-megabyte small disk with the following characteristics: Number of bytes per sector = 512 Number of sectors per track = 40 Number of tracks per cylinder = 11 Number of cylinders = 1331 How many cylinders does the file require if each data record requires 256 bytes? 2006 Hakan Uraz - File Organization 6

7 Answer Since each sector can hold two records, the file requires 20000/2 = sectors. One cylinder can hold 40 X 11 = 440 sectors. So the number of cylinders required is 10000/440 = 22.7 cylinders. It may be that a disk drive with 22.7 cylinders of available space does not have 22.7 physically contiguous cylinders available. The file may be spread of hundreds of cylinders Hakan Uraz - File Organization 7

8 Organizing data on a disk Organizing tracks by sector, or Organizing tracks by user-defined blocks Hakan Uraz - File Organization 8

9 Organizing Tracks by Sector The physical placement of sectors The simplest view is that sectors are adjacent fixed-sized segments of a track that happen to hold a file. It is an adequate way to view a file logically, but it may not be a good way to store sectors physically Hakan Uraz - File Organization 9

10 Organizing Tracks by Sector The physical placement of sectors To read a series of sectors, you often cannot read adjacent sectors. It takes the disk controller some time to process the received information before it is ready to accept more. Consequently, we would be able to read only one sector per revolution of the disk Hakan Uraz - File Organization 10

11 Organizing Tracks by Sector Interleaving Hence sectors are interleaved, leaving an interval of several physical sectors between logically adjacent sectors. In the following figure, interleaving factor is 5. It takes five revolutions to read 32 sectors. An improvement over 32 revolutions. High performance disks can offer 1:1 interleaving Hakan Uraz - File Organization 11

12 Interleaving 2006 Hakan Uraz - File Organization 12

13 Organizing Tracks by Sector Clusters The view maintained by Operating System s file manager. A cluster is a fixed number of contiguous sectors. It is not always physically contiguous; the degree of physical contiguity is determined by the interleaving factor. All sectors in a cluster can be accessed without requiring additional seeks Hakan Uraz - File Organization 13

14 Organizing Tracks by Sector Clusters To view a file as a series of clusters and still maintain the sectored view, the file manager ties logical sectors to the physical clusters that they belong to by using a file allocation table (FAT). The FAT contains a list of all the clusters in a file, ordered according to the logical order of the sectors they contain. With each cluster entry in the FAT is an entry giving the physical location of the cluster Hakan Uraz - File Organization 14

15 Organizing Tracks by Sector 2006 Hakan Uraz - File Organization 15

16 Organizing Tracks by Sector Extents A further attempt to emphasize physical contiguity of sectors in a file, hence minimizing seeking even more. It may be possible to make a file consist entirely of contiguous clusters. Then, we say that the file consists of one extent: All of its sectors, tracks, and (if it is large enough) cylinders form one contiguous whole. If there is not enough contiguous space available to contain an entire file, the file is divided into two or more noncontiguous parts. Each part is an extent Hakan Uraz - File Organization 16

17 Organizing Tracks by Sector 2006 Hakan Uraz - File Organization 17

18 Organizing Tracks by Sector Fragmentation If the size of a sector is 512 bytes and the size of all records in a file is 300 bytes, there is no convenient fit between records and sectors. Solutions: 1. Store only one record per sector. 2. Allow records to span sectors, so the beginning of a record might be found in one sector and the end of it in another Hakan Uraz - File Organization 18

19 Organizing Tracks by Sector 2006 Hakan Uraz - File Organization 19

20 Organizing Tracks by Sector Fragmentation The first option has the advantage that any record can be retrieved by retrieving just one sector, but it has the disadvantage that it might leave an unused space within each sector called internal fragmentation. The second option has the advamtage that it loses no space from internal fragmentation, but it has the disadvantage that some records may be retrieved only by accessing two sectors Hakan Uraz - File Organization 20

21 Organizing Tracks by Sector Fragmentation The use of clusters may also result in internal fragmentation. When the number of bytes in a file is not an exact multiple of the cluster size, there will be internal fragmentation in the last extent of the file. A disk holding smaller files or files that are accessed only randomly would normally be set up with small clusters Large files processed sequentially are usually given large cluster sizes for performance Hakan Uraz - File Organization 21

22 Organizing Tracks by Block Sometimes disk tracks are not divided into sectors, but into integral numbers of user-defined blocks whose size can vary. The amount of data transferred in a single I/O operation can vary depending on the needs of the software designer, not the hardware. Blocks can normally be either fixed or variable in length, depending on the requirements of the file designer. As with sectors, blocks are often referred to as physical records Hakan Uraz - File Organization 22

23 Organizing Tracks by Block 2006 Hakan Uraz - File Organization 23

24 Organizing Tracks by Block A block organization does not present the sectorspanning and fragmentation problems of sectors because blocks can vary in size to fit the logical organization of the data. A block is usually organized to hold an integral number of logical records. The term blocking factor is used to indicate the number of records that are to be stored in each block in a file. Blocks are superior to sectors when it is desirable to have the physical allocation of space for records correspond to their logical organization Hakan Uraz - File Organization 24

25 Organizing Tracks by Block Each block of data is usually accompanied by one or more subblocks containing extra information about the data block. Count subblock contains (among other things) the number of bytes in the data block. There may be a key subblock containing the key for the last record in the data block. When key subblocks are used, a track can be searched by the disk controller for a block or record identified by a given key Hakan Uraz - File Organization 25

26 Organizing Tracks by Block 2006 Hakan Uraz - File Organization 26

27 Nondata Overhead Some of the overhead consists of information that is stored on the disk during preformatting. On sector-addressable disks, preformatting involves storing, at the beginning of each sector, such information as sector address, track address, and condition (whether the sector is usable or defective). Preformatting also involves placing gaps and synchronization marks between fields of information to help the read/write mechanism distinguish between them. These overhead is of no concern to programmer Hakan Uraz - File Organization 27

28 Nondata Overhead On a block-organized disk, some overhead is invisible some is visible to programmer. Since subblocks and interblock gaps have to be provided with every block, there is more overhead with blocks than with sectors. Also, since the number and sizes of blocks can vary from one application to another, the relative amount of space taken up by overhead can vary when block addressing is used Hakan Uraz - File Organization 28

29 Nondata Overhead Suppose we have a block-addressable disk drive with bytes per track, and the amount of space taken up by subblocks and interblock gaps is equivalent to 300 bytes per block. We want to store a file containing 100-byte records on the disk. How many records can be stored per track if the blocking factor is 10, or if it is 60? 2006 Hakan Uraz - File Organization 29

30 Nondata Overhead If there are byte records per block, each block holds 1000 bytes of data and uses or 1300 bytes of track space. The number of blocks which can fit on a byte track can be expressed as floor(20000/1300) = 15 So 15 blocks or 150 records can be stored per track. Note that we take the floor of the result because a block cannot span two tracks Hakan Uraz - File Organization 30

31 Nondata Overhead If there are byte records per block, each block holds 6000 bytes of data and uses 6300 bytes of track space. The number of blocks per track is floor(20000/6300) = 3 So 3 blocks, or 180 records, can be stored per track. Clearly, the larger blocking factor can lead to more efficient use of storage Hakan Uraz - File Organization 31

32 Nondata Overhead Larger blocking factors do not always lead to more efficient storage utilization. Since we can put only an integral number of blocks on a track, we lose some space at the end of the track. Here we have the internal fragmentation problem again within a track. The greater the block size, the greater potential amount of internal track fragmentation Hakan Uraz - File Organization 32

33 Nondata Overhead The use of blocks, rather than sectors introduces flexibility, since it lets the programmer determine how data are to be organized physically. As a disadvantage, this is a requirement for programmer and/or operating system. Also, the flexibility introduced precludes the synchronization of I/O operations with the physical movement of the disk. This means strategies such as sector interleaving cannot be used to improve performance Hakan Uraz - File Organization 33

34 The Cost of a Disk Access Three distinct physical operations: Seek time Rotational delay (rotational latency) Transfer time Hakan Uraz - File Organization 34

35 The Cost of a Disk Access Seek Time The time required to move the access arm to the correct cylinder. Seeking is likely to be more costly in a multiuser environment. Average seek time is the time to traverse 1/3 of the total number of cylinders Hakan Uraz - File Organization 35

36 The Cost of a Disk Access Rotational Delay The time it takes for the disk to rotate so the sector we want is under the read/write head. On average, the rotational delay is half a revolution Hakan Uraz - File Organization 36

37 The Cost of a Disk Access 2006 Hakan Uraz - File Organization 37

38 The Cost of a Disk Access Transfer Time Once the data we want is under the read/write head, it can be transferred. Transfer time = number of bytes transferred number of bytes on a track X rotation time 2006 Hakan Uraz - File Organization 38

39 The Cost of a Disk Access Let s compare the time it takes to access a file in sequence with the time it takes to access all of the records in the file randomly 2006 Hakan Uraz - File Organization 39

40 Specifications of disk drive Minimum seek time 6 msec Average seek time 18 msec Rotational delay 8.3 msec Maximum transfer rate 16.7 msec/track, or 1229 bytes/msec Bytes per sector 512 Sectors per track 40 Tracks per cylinder 11 Tracks per surface 1331 Interleave factor 1 Cluster size 8 sectors Smallest extent size 5 clusters 2006 Hakan Uraz - File Organization 40

41 The Cost of a Disk Access Smallest extent is 5 clusters. Space is allocated in one-track units. Let s read a 2048 Kbyte file that is divided into byte records. A cluster holds 16 records. There are 500 clusters to hold the file in 100 extents occupying 100 tracks Hakan Uraz - File Organization 41

42 To read sector by sector in sequence Average seek 18 msec Rotational delay 8.3 msec Read one track 16.7 msec Total 43 msec (Assume 100 tracks are randomly dispersed) Total time = 100 X 43 msec = 4.3 seconds 2006 Hakan Uraz - File Organization 42

43 To read same records with random access For each record Average seek 18 msec Rotational delay 8.3 msec Read one cluster (1/5 X 16.7) 3.3 msec Total 29.6 msec Total time = 8000 X 29.6 msec = seconds Hakan Uraz - File Organization 43

44 Disk as Bottleneck To speed up system, multiprogramming is used. Another technique is disk striping which involves splitting the parts of a file on several different drives, then letting the separate drives deliver data simultaneously. A RAM disk is a large part of RAM configured to simulate the behavior of a mechanical disk in every respect except speed and volatility Hakan Uraz - File Organization 44

45 Disk as Bottleneck A disk cache is a large block of RAM configured to contain pages of data from a disk. When data is requested from secondary memory, the file manager first looks into the disk cache if it contains the data. Only if cache does not contain, disk is used. Caches improve performance because of the programs locality of references Hakan Uraz - File Organization 45

46 A Journey of a Byte What happens when a program writes a byte to a file on a disk? What happens between the program and the disk? Suppose we want to append a byte representing the character P stored in a character variable c to a file named in the variable TEXT. From the program s point of view, the journey is represented by the statement WRITE(TEXT, c, 1); But the journey is longer than that Hakan Uraz - File Organization 46

47 A Journey of a Byte The WRITE() statement results in a call to the operating system (OS). OS File manager deals with files and I/O. File manager may be thought of layers of procedures. Each layer calls the one below. At the lowest level, the byte is actually written to disk Hakan Uraz - File Organization 47

48 2006 Hakan Uraz - File Organization 48

49 A Journey of a Byte Instead of sending the sector immediately to the disk, the file manager usually waits to see if it can accumulate more bytes going to the same sector before transmitting. Also when the file is closed, the output buffers are flushed to disk Hakan Uraz - File Organization 49

50 A Journey of a Byte The Byte Leaves RAM: The I/O Processor and Disk Controller Because of bottlenecks created by the differences in speed and data-path widths, our byte might have to wait for an external data path to become available. Because of speed differences, the CPU can transmit to several external devices simultaneously Hakan Uraz - File Organization 50

51 The I/O Processor A Journey of a Byte I/O Processor disassembles and assembles groups of bytes for transmission to/from devices. It takes the instructions from the OS but once it begins processing I/O, it runs independently. So, I/O processes and internal computing overlap. It can take data directly from RAM without the involvement of the CPU (direct memory access DMA). The file manager might tell the I/O processor that there is data in the buffer, how much data there is, and where it is to go on the disk. This info might be in the form of a little program that the OS constructs and I/O processor executes Hakan Uraz - File Organization 51

52 A Journey of a Byte The Disk Controller The I/O processor asks the disk controller if the disk drive is available. If not, our byte will have to wait in its buffer. The disk drive is instructed to move its head to the track and sector where to store our byte and its companions (a mechanical movement). Now, the I/O processor (or the controller) can send out the bytes, one at a time, to the drive where they are written on the disk surface Hakan Uraz - File Organization 52

53 Buffer Management Involves working with large chunks of data in RAM, so the number of disk accesses can be reduced. Consider what happens if a program is performing both input and output on one character at a time, and only one I/O buffer is available. To avoid such problems there is the need for several system buffers Hakan Uraz - File Organization 53

54 Buffer Management Even if a program transmits data in only one direction, the use of a single system I/O buffer can slow it down considerably. (The operation of reading a sector from a disk is extremely slow compared to RAM speed). So, we use more than one buffer and have the I/O system filling the next sector or block of data while the CPU is processing the current one Hakan Uraz - File Organization 54

55 Multiple Buffering Suppose that a I/O bound program is only writing to disk. If two buffers are used and I/O-CPU overlapping is permitted, the CPU can be filling one buffer while the contents of the other are being transmitted to disk. When both tasks are finished, the roles of the buffers can be exchanged. This technique is called double buffering. The idea of swapping system buffers to allow processing and I/O to overlap need not be restricted to two buffers Hakan Uraz - File Organization 55

56 Multiple Buffering 2006 Hakan Uraz - File Organization 56

57 Buffer Pooling When a system buffer is needed, it is taken from a pool of available buffers and used. When the system receives a request to read a certain sector or block, it looks to see if one of its buffers already contains that sector or block. If no buffer contains it, then the system finds from its pool of buffers one that is not currently in use and loads the sector or block into it Hakan Uraz - File Organization 57

58 Buffer Pooling Sevaral schemes are used to decide which buffer to take from a buffer pool. One generally effective strategy is to take from the pool that buffer that is least recently used (LRU). When a buffer is accessed, it is put on a least-recentlyused queue, so it is allowed to retain its data until all other less-recently used buffers have been accessed. LRU strategy is based on the assumption that a block of data that has been used recently is more likely to be needed in the near future than one that has been used less recently Hakan Uraz - File Organization 58

59 Buffers Move Mode and Locate Mode When data must always be copied from a system buffer to a program buffer (or vice versa), the amount of time to perform the move can be substantial. This way of handling buffered data is called move mode. There are two ways to avoid move mode. Both techniques are examples of the approach called locate mode. These are: The file manager performing I/O directly between disk and the program s data area. The file manager using system buffers to handle I/O, but provide the program with the locations, through the use of pointer variables, of the system buffers Hakan Uraz - File Organization 59

60 I/O in UNIX The topmost I/O layer in UNIX deals with data primarily in logical terms. This layer consists of processes that impose certain logical views on files. Processes include shell routines, user programs that operate on files and library routines. Below this layer is the UNIX kernel, which incorporates all the rest of the layers. The kernel views all I/O as operating on a sequence of bytes, so once we pass control to the kernel all assumptions about the logical view of a file are gone Hakan Uraz - File Organization 60

61 UNIX-Kernel Let s illustrate the journey of a byte through the kernel. Considers the system call such as write(fd, &c, 1); The routines that let processes communicate directly with the kernel make up the system call interface Hakan Uraz - File Organization 61

62 UNIX-Kernel The kernel I/O system begins by connecting the file descriptor (fd) in your program to some file or device in the filesystem. It does this by proceeding through a series of four tables: A file descriptor table. An open file table, with information about open files A file allocation table, which is part of a structure called an index node. A table of index nodes, with one entry for each file in use Hakan Uraz - File Organization 62

63 UNIX-Kernel The file descriptor table is owned by the process (your program). The open file table and index node tables are owned by the kernel. The index node itself is part of the filesystem Hakan Uraz - File Organization 63

64 UNIX-Kernel Every process has its own descriptor table Hakan Uraz - File Organization 64

65 UNIX-Kernel It is possible for several different processes to refer to the same open file table entry, so one process could read part of a file, another process could read the next part, and so forth. On the other hand, if the same file is opened by two separate open() statements, two separate entries are made in the table, and the two processes operate on the file quite independently Hakan Uraz - File Organization 65

66 UNIX-Kernel The information in the open file table is transitory. The kernel needs more information about the file itself. This information is found in an index node referred to as an inode Hakan Uraz - File Organization 66

67 UNIX-Kernel An inode is more permanent than an open file table s entry (called file structure). A file structure exists only while a file is open for access, but an inode exists as long as its corresponding file exists. So, inode is kept on disk with the file (though not adjacent to the file). When a file is opened, a copy of its inode is loaded into RAM where it is added to the inode table for rapid access Hakan Uraz - File Organization 67

68 UNIX-Linking File Names to Files All references to files begin with a directory. A directory is a small file that contains, for each file, a file name together with a pointer to the file s inode on disk. This pointer is called a hard link. When a file is opened, this hard link is used to bring the inode into RAM and to set up the corresponding entry in the open file table. It is possible for several file names to point to the same inode. A field in the inode tells how many hard links there are to the inode Hakan Uraz - File Organization 68

69 UNIX-Linking File Names to Files Another kind of link, called a soft link (or symbolic link) links a file name to another file name, rather than to an actual file. Instead of being a pointer to an inode, a soft link is a pathname of some file. Since a symbolic link does not point to an actual file, it can refer to a directory or even a file in a different file system Hakan Uraz - File Organization 69

70 UNIX-Normal Files, Special Files, and Sockets Normal files are block I/O files. Special files represent a stream of characters (e.g. the first three file descriptors in the descriptor table) Sockets are abstractions that serve as endpoints for interprocess communication Hakan Uraz - File Organization 70

71 UNIX-Block I/O The UNIX view of a block device is similar to disk. It is a randomly addressable array of fixed blocks. No other organization (such as clusters) was imposed on the placement of files on disk Hakan Uraz - File Organization 71

72 UNIX-Device Drivers They perform the actual I/O between the I/O buffer and the device. For block I/O, the device driver s job is to take a block from a buffer, destined for one of these physical blocks, and see that it gets deposited in the proper physical place on the device Hakan Uraz - File Organization 72

73 UNIX-The Kernel and Filesystems A filesystem includes the directory structure, the directories, ordinary files, and the inodes that describe the files. All parts of a filesystem reside on disk, rather than in RAM. These parts are brought into RAM by the kernel as needed. One advantage of keeping the filesystem and I/O system distinct is that we can have separate filesystems that are organized differently, perhaps on different devices, but are accesible by the same kernel Hakan Uraz - File Organization 73