Wednesday, May 3, Several RAID "levels" have been defined. Some are more commercially viable than others.
|
|
- Megan Evans
- 5 years ago
- Views:
Transcription
1 Wednesday, May 3, 2017 Topics for today RAID: Level 0 Level 1 Level 3 Level 4 Level 5 Beyond RAID 5 File systems RAID revisited Several RAID "levels" have been defined. Some are more commercially viable than others. RAID Level 0 (see Fig. 9.26) In RAID 0 we have the idea of striping the data. If we have N discs then virtual sector k will be stored as sector k/n on disc number k%n. There is no redundancy if a drive fails there is data loss. However, we may be able to perform, in parallel, operations that would have to be sequential on a single big disc, for example reading virtual sector S and virtual sector S+1. RAID level 1 (see Fig. 9.27) In RAID 1 we have mirror discs. Every disc is duplicated. Now if a single drive crashes, there is no data loss because an exact copy of it is available. Possibly slightly slower write operations (because two discs have to be written for a write to be complete). Variation - synchronize discs 180 degrees apart (reads are faster because we can read from either disc but writes are slower because we have to write to both discs) RAID level 0+1 or 1+0 (see Fig. 9.28) A combination of levels 1 and 0 - data striping with mirroring. One option (1+0: striped mirrors) might be more expensive than the other (0+1: mirrored striped) but might have greater reliability. Comp 162 Notes Page 1 of 12 May 3, 2017
2 RAID level 2 (see Fig. 9.29) Bit interleaving with Error Correction (Hamming Codes). Bits of a byte are stored on, for example, 8 separate discs with 4 parity bits stored on 4 other drives. If a single drive fails we do not even need to know which is the bad one in order to be able to recreate the data. There is a lot of overhead and RAID 2 is just of theoretical interest. RAID level 3 (see Fig. 9.30) Bit-interleaving with simple parity. Write a bit of each byte onto a separate data drive, write the parity bit onto a parity drive. Requires 9 drives? Note that if we have a data item with parity check (for example, the number of 1 bits is even) we can discover the value of one bit by examining the others. Consider If you cover up one bit you can tell what it was by looking at the others. Thus, if a single drive fails in a RAID 3 system we can recover its data by looking at the corresponding bits on the other drives. In contrast to RAID 2, we need to know which is the bad disc (although this is normally not a problem.) A disadvantage of RAID 3 is that there is a large minimum-size transfer amount. If we can transfer no less than 4096 bytes (for example) at a time to a single disk, we can transfer no less than 4096*(N-1) data bytes to a RAID 3 system with N discs. RAID level 4 (see Fig. 9.31) Sector-level interleaving (as in RAID 0). In addition, each group of N data sectors generates a parity sector. For example, the parity might be the exclusive-or of the data Data sector X Data sector Y Data sector Z Parity(X,Y,Z) Reads are fast and reliable. If a sector becomes unreadable we can determine its contents by looking at the corresponding sectors on the other drives. Writes require that both the data sector and the corresponding parity sector are updated. This requires two reads (current data and current parity) and two writes (new data and new parity). The reads are needed so we can compute the new value of the parity sector. Comp 162 Notes Page 2 of 12 May 3, 2017
3 In a RAID 4 system all the parity sectors are on the same drive so it becomes a bottleneck in the system RAID level 5 (see Fig. 9.32) In RAID 5, the parity sectors are distributed cyclically over the set of drives. This permits greater parallelism in write operations than is possible with RAID 4. The RAID simulator ( ) We can use the program written by Martha Hogan to simulate 4-disc RAID 0, RAID 10 and RAID 5 systems. The program reports the Actual Read Failure Rate (ARFR) the Virtual Read Failure Rate (VRFR). RAID 0. There is no redundancy so VRFR will be the same as ARFR. (a) If we set Disc Failure Probabilities to (1.0, 0.0, 0.0, 0.0) we would expect ARFR = VRFR = 0.25 because one quarter of the read operations on average would map to the drive that fails 100% of the time. (b) If we set the probabilities to (0.0, 0.0, 0.0, 0.5) we would expect ARFR = VRFR = RAID 10 There are two pairs of discs. We can determine, by observation, which discs are paired. Setting probabilities (1.0, 1.0, 0.0, 0.0) VRFR = 0.0 so Disc 1 and Disc 2 are in different pairs Setting probabilities (1.0, 0.0, 1.0, 0.0) VRFR 0.5 so Disc 1 and Disc 3 are paired If probabilities are (1.0, 0.0, 0,0, 0,0) VRFR = 0.0 and ARFR = Why? Suppose we have 60 virtual reads. On average 30 will be to disc 1 and 30 will be to disc 2. The 30 on disc 1 will fail requiring 30 reads from the mirror disc (Disc 3). So a total of 90 reads, 30 of which fail so ARFR = 1/3. Comp 162 Notes Page 3 of 12 May 3, 2017
4 RAID 5 In a RAID 5 configuration, if a read fails, corresponding sectors are read from the other discs. If all those reads succeed then we have a successful virtual read. (a) If failure probabilities are (1.0, 0.0, 0.0, 0.0) then VRFR = 0.0 and ARFR = The ARFR (1/7) can be explained by the following diagram representing 1200 initial reads. The symbol denotes a successful read and X denotes a failure X reads are attempted (1200 original and 900 to correct error). Of these, 300 fail so the ARFR is 1/7. (b) If failure probabilities are (1.0, 1.0, 0.0, 0.0) then VRFR = 0.5 and ARFR = 0.4 as you can see from the following X 300 X X X (c) If failure probabilities are (0.5, 0.0, 0.0, 0.25) then VRFR = and ARFR = 0.16 as you can see from the following Comp 162 Notes Page 4 of 12 May 3, 2017
5 X X X X ARFR = ( )/( ) = 400/2500 = 0.16 VRFR = (50+50)/1600 = Beyond RAID 5 After a failure in a RAID 5 system, a disc must be rebuilt. If there is any read failure during this time then there is data loss. Example (due to Chen et al 1993), If we are rebuilding a 100GB disc array we need to perform 200 million sector reads. If the bit error rate is one in bits, the probability of an error-free rebuild is 99.2% Errors during rebuild may be more likely if the discs in the system are of the same type and the same age. RAID level 6 is the current industry standard. In RAID 6 (dual-distributed parity) there are two independent parity sectors for each group of N data sectors thus we need more space: N+2 discs for N discs worth of data compared with N+1 disc for RAID 5 (single distributed parity). Reads are still fast but writes are slower than RAID 5 Comp 162 Notes Page 5 of 12 May 3, 2017
6 because of the need to update two parity sectors. We can improve reliability because the system is not running in a non-redundancy mode during a rebuild. File systems (section 9.3) Users are not normally concerned with the operation of the disc(s) in a system. Their interface is usually through the file management component of the Operating System. Warford looks at three ways in which we might store files on discs. We look at these methods plus, briefly, the Unix system and a system without directories that is based on hashing. Directories/Folders Directories (folders) hold information about files - including other directories so we can have a hierarchical system. If we can have links across directories then the structure is a graph rather than a tree. Information stored about a file might include the following (you may be able to think of additional items). name generation number (each modification bumps this by one) type - or pointer to application that created it. owner times: created last modified last read last archived expiration(?) access controls: user(s)/permitted-actions size and location of file - see below Space Allocation Warford s three ways of allocating space to files are termed (a) contiguous, (b) linked and (c) indexed. (a) contiguous (Fig. 9.18) In this first scheme, a file occupies contiguous space on a drive. This has advantages in that both sequential access to the whole file and random access to a part of the file are fast. The directory entry is likely to be simple too, just indicating where the file starts and how big it is. However, over time, the free space on a disc is likely to be fragmented. If there are multiple free areas, which one does the OS use when a user program opens a new file for writing (without any indication of how big the file might get)? Suppose a user tries to make a copy of file A. The OS knows how much space is required and if there is a free area big enough we are OK. Similarly, if the total free space is less than the size of A there is no way we can make a copy. But what if there Comp 162 Notes Page 6 of 12 May 3, 2017
7 is enough free space but it is broken into chunks no single one of which is big enough? The operating system may have to move files around to create a big enough area. This defragmentation is very time consuming. (b) linked storage (Fig. 9.19) In Warford s second scheme, a file is stored as a linked list of allocation units with each unit except the last pointing to the next unit. The directory entry identifies the number of units and the location of the first. The operating system now has much more flexibility about placing units of the file on the disc. However, sequential access through the file may be slow depending on the scattering of the units. Random access is very slow because to get to an arbitrary point in the file we typically have to follow the pointers through the list. (c) indexed (Fig. 9.20) Warford's third scheme also allows allocation units of a file to be placed anywhere on the disc. It has multiple pointers in the directory entry that point directly to allocation units. This reduces the time required to access a random component of the file. However, when we design a directory entry, how many pointer fields do we include? If it is N, how do we store a file having more than N units? (d) Unix-like approach In the Unix system, a directory entry points to an inode (index node) which points to the file rather than the directory entry pointing directly to the file. Because an inode can be pointed to from multiple places, a particular file can appear in multiple directories (and have different names in those directories). The inode contains a count of the number of pointers to it and a file is only deleted when the count reaches zero. There are a fixed number of pointers in the inode but they are used in a creative way to permit very large files to be accessed. Here is an example of how they might be used (different Unix implementations differ in the details). Suppose there are 13 pointers; we might use those pointers as follows: Comp 162 Notes Page 7 of 12 May 3, 2017
8 Pointer(s) 1 through 10 Used for point directly to the first 10 allocation units of the file 11 points to a block of pointers. The pointers in the block point to the next M blocks in the file. For example, if there are 256 pointers in the block, they would point to allocation units 11 through points to a block of pointers each of which points to a block of pointers. This double indirection lets us point to a further 256*256 blocks 267 through points to a block of pointers each pointing to a double-indirect block. Now we can access (via this triple indirection) a further 256*256*256 blocks through This scheme enables very large files to be accessed. If allocation units are 4k bytes, we can have files in excess of 64Gb. While there is a time penalty for accessing all but the first few blocks, most of the files on a typical user account are small and can be accessed without indirection. The inode of a currently active file is held in main memory. (e) Directory-less file system Why might we want a directory-less file system? Sometimes corruption might occur in a directory/folder that makes a file unreadable even though the file itself is intact. The following is a possible scheme for storing files that does not use directories at all. However, some operations are slow; it is a scheme appropriate only for a specialized environment. Background: hashing Hashing is a storage and retrieval technique that uses a key-to-address function. We apply the function to the key of the item to be stored and the function returns the address where it should be stored. When we need to find an item we apply the function to the search key and it gives us the address where to look. We saw hashing when discussing symbol tables in assemblers. In the best circumstances no two items hash to the same address and storage and retrieval is very fast. However, in practice we need to make provision for an item hashing to an address that is Comp 162 Notes Page 8 of 12 May 3, 2017
9 already occupied. One way of dealing with overflows is to look at successive addresses until free space is found. The hash function is customized for the type of keys and range of addresses in the application. For example if we are hashing names into a table with 30 entries we might use address = ( rank (first letter) + rank (last letter)) % 30 Thus "Yeltsin" hashes to ( ) % 30 = 39%30 = 9. Hashing to disc Imagine a disc as an array of allocation units each of which has room for user data plus a key field. When we initialize the file system we put a reserved key e.g. %%EMPTY in each key field. To write block b of file F to the disc we construct a composite key out of the filename and the block number then hash this key to get a block number. For example if the file is mydata and we are writing block 7, the composite key might be mydata%%007 If the block with the address is is free we put the data there otherwise we apply an overflow method to find a free space. When we wish to read block b of file F we go through the same hashing process. To speed up operations, the OS keeps a table showing the actual address of blocks that have overflowed and are not at their hash address. Unreliable portions of the disc can be flagged with a particular marker key, e.g. %%BAD Depending on the overflow method used, we may need a marker for a deleted unit e.g. %%DEL Some operations on this structure are slow and some are fast Slow: Medium: Fast: getting a list of all the files on the disc together with their sizes deleting a file accessing block I of a file We can simulate a hierarchical directory by permitting file names of the form X/Y/Z. Reading Section 9.1 has a page or two on disc drives. Section 9.4 discusses error detecting and correcting codes. Sections 9.5 for RAID and 9.3 for file systems. Comp 162 Notes Page 9 of 12 May 3, 2017
10 Review Questions 1. We have seen that a RAID system may be able to recover from an unsuccessful read operation. Can it recover from an unsuccessful write operation? 2. On an N-disc RAID 4 system, exactly one disc has a non-zero failure rate. Fill in the following table with the read and write operations required Error-free read of virtual sector Read of virtual sector with errors Error-free write of virtual sector Reads Writes 3. If, in a RAID 5 system, one disc has a 10% failure rate and the other N-1 discs read without error then the VRFR is 0.0. How does the ARFR change as we increase N (a) stays the same (b) increases (c) decreases? 4. You are in charge of a 4-disc RAID 5 system. On arriving at work one day you notice that one disc has a 10% failure rate (one in ten reads fails). The other discs are fine. Your boss wants to know (a) the VRFR (b) the ARFR. What do you tell her? Later, returning from lunch you notice that a second disc is also failing but on this drive, one in five reads fails. Your boss wants to know (c) the updated VRFR and (d) the ARFR. What numbers do you give her? 5. Research the Unix touch command. What is the effect of touch X (a) If file X already exists (b) if file X does not exist. Comp 162 Notes Page 10 of 12 May 3, 2017
11 Review Answers 1. Only action is to retry the write. 2. Reads Writes Error-free read of virtual sector 1 0 Read of virtual sector with errors N 0 Error-free write of virtual sector (c) decreases. For example if N is 4 VRFR is 1/43 and if N is 6 VRFR is 1/ (a) 0.0 (b) 30/1290 = 1/ X Comp 162 Notes Page 11 of 12 May 3, 2017
12 (c) 12/1200 = 0.01 (d) ( )/( ) = 102/ X X X 54 6 X 5. (a) updates one or both of the access and modification timestamps of the file (held in the inode) (b) creates an empty file. Comp 162 Notes Page 12 of 12 May 3, 2017
File Directories Associated with any file management system and collection of files is a file directories The directory contains information about
1 File Management 2 File Directories Associated with any file management system and collection of files is a file directories The directory contains information about the files, including attributes, location
More informationChapter 11: File System Implementation. Objectives
Chapter 11: File System Implementation Objectives To describe the details of implementing local file systems and directory structures To describe the implementation of remote file systems To discuss block
More informationCSE380 - Operating Systems
CSE380 - Operating Systems Notes for Lecture 17-11/10/05 Matt Blaze, Micah Sherr (some examples by Insup Lee) Implementing File Systems We ve looked at the user view of file systems names, directory structure,
More informationCS 537 Fall 2017 Review Session
CS 537 Fall 2017 Review Session Deadlock Conditions for deadlock: Hold and wait No preemption Circular wait Mutual exclusion QUESTION: Fix code List_insert(struct list * head, struc node * node List_move(struct
More informationFile System Internals. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
File System Internals Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics File system implementation File descriptor table, File table
More informationFile Systems. File Systems. G53OPS: Operating Systems. File Systems. File Systems 11/27/2008. Why Use Files? Graham Kendall. Two Views of File System
Why Use s? Introduction Graham Kendall It allows data to be stored between processes It allows us to store large volumes of data Allows more than one process to access the data at the same time 27 Nov
More informationDefinition of RAID Levels
RAID The basic idea of RAID (Redundant Array of Independent Disks) is to combine multiple inexpensive disk drives into an array of disk drives to obtain performance, capacity and reliability that exceeds
More informationOperating Systems. Lecture File system implementation. Master of Computer Science PUF - Hồ Chí Minh 2016/2017
Operating Systems Lecture 7.2 - File system implementation Adrien Krähenbühl Master of Computer Science PUF - Hồ Chí Minh 2016/2017 Design FAT or indexed allocation? UFS, FFS & Ext2 Journaling with Ext3
More informationChapter 14: File-System Implementation
Chapter 14: File-System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery 14.1 Silberschatz, Galvin and Gagne 2013 Objectives To describe
More informationCOMP 3361: Operating Systems 1 Final Exam Winter 2009
COMP 3361: Operating Systems 1 Final Exam Winter 2009 Name: Instructions This is an open book exam. The exam is worth 100 points, and each question indicates how many points it is worth. Read the exam
More informationWednesday, November 4, 2009
Wednesday, November 4, 2009 Topics for today Storage management Main memory (1) Uniprogramming (2) Fixed-partition multiprogramming (3) Variable-partition multiprogramming (4) Paging (5) Virtual memory
More informationCOSC 6385 Computer Architecture. Storage Systems
COSC 6385 Computer Architecture Storage Systems Spring 2012 I/O problem Current processor performance: e.g. Pentium 4 3 GHz ~ 6GFLOPS Memory Bandwidth: 133 MHz * 4 * 64Bit ~ 4.26 GB/s Current network performance:
More informationI/O, Disks, and RAID Yi Shi Fall Xi an Jiaotong University
I/O, Disks, and RAID Yi Shi Fall 2017 Xi an Jiaotong University Goals for Today Disks How does a computer system permanently store data? RAID How to make storage both efficient and reliable? 2 What does
More informationFILE SYSTEM IMPLEMENTATION. Sunu Wibirama
FILE SYSTEM IMPLEMENTATION Sunu Wibirama File-System Structure Outline File-System Implementation Directory Implementation Allocation Methods Free-Space Management Discussion File-System Structure Outline
More informationFile System Implementation. Sunu Wibirama
File System Implementation Sunu Wibirama File-System Structure Outline File-System Implementation Directory Implementation Allocation Methods Free-Space Management Discussion File System Structure File
More informationFile System Internals. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
File System Internals Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics File system implementation File descriptor table, File table
More informationChapter 4 File Systems. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved
Chapter 4 File Systems File Systems The best way to store information: Store all information in virtual memory address space Use ordinary memory read/write to access information Not feasible: no enough
More informationCS 4284 Systems Capstone
CS 4284 Systems Capstone Disks & File Systems Godmar Back Filesystems Files vs Disks File Abstraction Byte oriented Names Access protection Consistency guarantees Disk Abstraction Block oriented Block
More informationFile Management. Chapter 12
File Management Chapter 12 Files Used for: input to a program Program output saved for long-term storage Terms Used with Files Field basic element of data contains a single value characterized by its length
More informationFile. File System Implementation. Operations. Permissions and Data Layout. Storing and Accessing File Data. Opening a File
File File System Implementation Operating Systems Hebrew University Spring 2007 Sequence of bytes, with no structure as far as the operating system is concerned. The only operations are to read and write
More informationCS370: System Architecture & Software [Fall 2014] Dept. Of Computer Science, Colorado State University
CS 370: SYSTEM ARCHITECTURE & SOFTWARE [MASS STORAGE] Frequently asked questions from the previous class survey Shrideep Pallickara Computer Science Colorado State University L29.1 L29.2 Topics covered
More informationFile systems CS 241. May 2, University of Illinois
File systems CS 241 May 2, 2014 University of Illinois 1 Announcements Finals approaching, know your times and conflicts Ours: Friday May 16, 8-11 am Inform us by Wed May 7 if you have to take a conflict
More informationChapter 6: File Systems
Chapter 6: File Systems File systems Files Directories & naming File system implementation Example file systems Chapter 6 2 Long-term information storage Must store large amounts of data Gigabytes -> terabytes
More informationPrinciples of Operating Systems
Principles of Operating Systems Lecture 24-26 - File-System Interface and Implementation Ardalan Amiri Sani (ardalan@uci.edu) [lecture slides contains some content adapted from previous slides by Prof.
More informationComputer Systems Laboratory Sungkyunkwan University
File System Internals Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics File system implementation File descriptor table, File table
More informationOperating System Concepts Ch. 11: File System Implementation
Operating System Concepts Ch. 11: File System Implementation Silberschatz, Galvin & Gagne Introduction When thinking about file system implementation in Operating Systems, it is important to realize the
More informationCS3600 SYSTEMS AND NETWORKS
CS3600 SYSTEMS AND NETWORKS NORTHEASTERN UNIVERSITY Lecture 11: File System Implementation Prof. Alan Mislove (amislove@ccs.neu.edu) File-System Structure File structure Logical storage unit Collection
More informationChapter 12: File System Implementation
Chapter 12: File System Implementation Virtual File Systems. Allocation Methods. Folder Implementation. Free-Space Management. Directory Block Placement. Recovery. Virtual File Systems An object-oriented
More informationFile. File System Implementation. File Metadata. File System Implementation. Direct Memory Access Cont. Hardware background: Direct Memory Access
File File System Implementation Operating Systems Hebrew University Spring 2009 Sequence of bytes, with no structure as far as the operating system is concerned. The only operations are to read and write
More informationIntroduction to OS. File Management. MOS Ch. 4. Mahmoud El-Gayyar. Mahmoud El-Gayyar / Introduction to OS 1
Introduction to OS File Management MOS Ch. 4 Mahmoud El-Gayyar elgayyar@ci.suez.edu.eg Mahmoud El-Gayyar / Introduction to OS 1 File Management Objectives Provide I/O support for a variety of storage device
More informationFile system internals Tanenbaum, Chapter 4. COMP3231 Operating Systems
File system internals Tanenbaum, Chapter 4 COMP3231 Operating Systems Architecture of the OS storage stack Application File system: Hides physical location of data on the disk Exposes: directory hierarchy,
More informationThe term "physical drive" refers to a single hard disk module. Figure 1. Physical Drive
HP NetRAID Tutorial RAID Overview HP NetRAID Series adapters let you link multiple hard disk drives together and write data across them as if they were one large drive. With the HP NetRAID Series adapter,
More informationDirectory Structure and File Allocation Methods
ISSN:0975-9646 Mandeep Kaur et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 7 (2), 2016, 577-582 Directory Structure and ile Allocation Methods Mandeep Kaur,
More informationFile System Internals. Jo, Heeseung
File System Internals Jo, Heeseung Today's Topics File system implementation File descriptor table, File table Virtual file system File system design issues Directory implementation: filename -> metadata
More informationCSE 153 Design of Operating Systems
CSE 153 Design of Operating Systems Winter 2018 Lecture 22: File system optimizations and advanced topics There s more to filesystems J Standard Performance improvement techniques Alternative important
More informationICS Principles of Operating Systems
ICS 143 - Principles of Operating Systems Lectures 17-20 - FileSystem Interface and Implementation Prof. Ardalan Amiri Sani Prof. Nalini Venkatasubramanian ardalan@ics.uci.edu nalini@ics.uci.edu Outline
More information1. Introduction. Traditionally, a high bandwidth file system comprises a supercomputer with disks connected
1. Introduction Traditionally, a high bandwidth file system comprises a supercomputer with disks connected by a high speed backplane bus such as SCSI [3][4] or Fibre Channel [2][67][71]. These systems
More informationCOMP 530: Operating Systems File Systems: Fundamentals
File Systems: Fundamentals Don Porter Portions courtesy Emmett Witchel 1 Files What is a file? A named collection of related information recorded on secondary storage (e.g., disks) File attributes Name,
More informationCurrent Topics in OS Research. So, what s hot?
Current Topics in OS Research COMP7840 OSDI Current OS Research 0 So, what s hot? Operating systems have been around for a long time in many forms for different types of devices It is normally general
More informationFilesystem. Disclaimer: some slides are adopted from book authors slides with permission 1
Filesystem Disclaimer: some slides are adopted from book authors slides with permission 1 Storage Subsystem in Linux OS Inode cache User Applications System call Interface Virtual File System (VFS) Filesystem
More informationCSC 553 Operating Systems
CSC 553 Operating Systems Lecture 12 - File Management Files Data collections created by users The File System is one of the most important parts of the OS to a user Desirable properties of files: Long-term
More informationToday: Coda, xfs! Brief overview of other file systems. Distributed File System Requirements!
Today: Coda, xfs! Case Study: Coda File System Brief overview of other file systems xfs Log structured file systems Lecture 21, page 1 Distributed File System Requirements! Transparency Access, location,
More informationFILE SYSTEMS. CS124 Operating Systems Winter , Lecture 23
FILE SYSTEMS CS124 Operating Systems Winter 2015-2016, Lecture 23 2 Persistent Storage All programs require some form of persistent storage that lasts beyond the lifetime of an individual process Most
More informationFile Systems. What do we need to know?
File Systems Chapter 4 1 What do we need to know? How are files viewed on different OS s? What is a file system from the programmer s viewpoint? You mostly know this, but we ll review the main points.
More informationFile Management. Ezio Bartocci.
File Management Ezio Bartocci ezio.bartocci@tuwien.ac.at Cyber-Physical Systems Group Institute for Computer Engineering Faculty of Informatics, TU Wien Motivation A process can only contain a limited
More informationChapter 11: Implementing File Systems. Operating System Concepts 8 th Edition,
Chapter 11: Implementing File Systems, Silberschatz, Galvin and Gagne 2009 Chapter 11: Implementing File Systems File-System Structure File-System Implementation Directory Implementation Allocation Methods
More informationExample Implementations of File Systems
Example Implementations of File Systems Last modified: 22.05.2017 1 Linux file systems ext2, ext3, ext4, proc, swap LVM Contents ZFS/OpenZFS NTFS - the main MS Windows file system 2 Linux File Systems
More informationFile System Implementation
File System Implementation Last modified: 16.05.2017 1 File-System Structure Virtual File System and FUSE Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance. Buffering
More informationFile Systems. ECE 650 Systems Programming & Engineering Duke University, Spring 2018
File Systems ECE 650 Systems Programming & Engineering Duke University, Spring 2018 File Systems Abstract the interaction with important I/O devices Secondary storage (e.g. hard disks, flash drives) i.e.
More informationWednesday, April 22, 2015
Wednesday, April 22, 2015 Topics for today Topics for Exam 3 Process management (Chapter 8) Loader Traps Interrupts, Time-sharing Storage management (Chapter 9) Main memory (1) Uniprogramming (2) Fixed-partition
More informationUNIX File System. UNIX File System. The UNIX file system has a hierarchical tree structure with the top in root.
UNIX File System UNIX File System The UNIX file system has a hierarchical tree structure with the top in root. Files are located with the aid of directories. Directories can contain both file and directory
More informationFile-System Structure. Allocation Methods. Free-Space Management. Directory Implementation. Efficiency and Performance. Recovery
CHAPTER 11: FILE-SYSTEM IMPLEMENTATION File-System Structure Allocation Methods Free-Space Management Directory Implementation Efficiency and Performance Recovery Operating System Concepts, Addison-Wesley
More informationWednesday, April 19, 2017
Wednesday, April 19, 2017 Topics for today Process management (Chapter 8) Loader Traps Interrupts, Time-sharing Storage management (Chapter 9) Main memory (1) Uniprogramming (2) Fixed-partition multiprogramming
More information6. Results. This section describes the performance that was achieved using the RAMA file system.
6. Results This section describes the performance that was achieved using the RAMA file system. The resulting numbers represent actual file data bytes transferred to/from server disks per second, excluding
More informationSegmentation with Paging. Review. Segmentation with Page (MULTICS) Segmentation with Page (MULTICS) Segmentation with Page (MULTICS)
Review Segmentation Segmentation Implementation Advantage of Segmentation Protection Sharing Segmentation with Paging Segmentation with Paging Segmentation with Paging Reason for the segmentation with
More informationIntroduction. Secondary Storage. File concept. File attributes
Introduction Secondary storage is the non-volatile repository for (both user and system) data and programs As (integral or separate) part of an operating system, the file system manages this information
More informationCS5460: Operating Systems Lecture 20: File System Reliability
CS5460: Operating Systems Lecture 20: File System Reliability File System Optimizations Modern Historic Technique Disk buffer cache Aggregated disk I/O Prefetching Disk head scheduling Disk interleaving
More informationFile system internals Tanenbaum, Chapter 4. COMP3231 Operating Systems
File system internals Tanenbaum, Chapter 4 COMP3231 Operating Systems Summary of the FS abstraction User's view Hierarchical structure Arbitrarily-sized files Symbolic file names Contiguous address space
More informationCOSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters
COSC 6374 Parallel I/O (I) I/O basics Fall 2010 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network card 1 Network card
More informationECE 650 Systems Programming & Engineering. Spring 2018
ECE 650 Systems Programming & Engineering Spring 2018 File Systems Tyler Bletsch Duke University Slides are adapted from Brian Rogers (Duke) File Systems Disks can do two things: read_block and write_block
More informationCSE380 - Operating Systems. Communicating with Devices
CSE380 - Operating Systems Notes for Lecture 15-11/4/04 Matt Blaze (some examples by Insup Lee) Communicating with Devices Modern architectures support convenient communication with devices memory mapped
More informationMonday, May 4, Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes
Monday, May 4, 2015 Topics for today Secondary memory Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes Storage management (Chapter
More informationStoring Data: Disks and Files
Storing Data: Disks and Files Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Data Access Disks and Files DBMS stores information on ( hard ) disks. This
More informationCSE 380 Computer Operating Systems
CSE 380 Computer Operating Systems Instructor: Insup Lee University of Pennsylvania Fall 2003 Lecture Note on Disk I/O 1 I/O Devices Storage devices Floppy, Magnetic disk, Magnetic tape, CD-ROM, DVD User
More informationFile Layout and Directories
COS 318: Operating Systems File Layout and Directories Jaswinder Pal Singh Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) Topics File system structure Disk
More informationChapter 5 Input/Output
Chapter 5 Input/Output 5.1 Principles of I/O hardware 5.2 Principles of I/O software 5.3 I/O software layers 5.4 Disks 5.5 Clocks 5.6 Character-oriented terminals 5.7 Graphical user interfaces 5.8 Network
More informationChapter 1 Disk Storage, Basic File Structures, and Hashing.
Chapter 1 Disk Storage, Basic File Structures, and Hashing. Adapted from the slides of Fundamentals of Database Systems (Elmasri et al., 2003) 1 Chapter Outline Disk Storage Devices Files of Records Operations
More informationToday: Secondary Storage! Typical Disk Parameters!
Today: Secondary Storage! To read or write a disk block: Seek: (latency) position head over a track/cylinder. The seek time depends on how fast the hardware moves the arm. Rotational delay: (latency) time
More informationSYSTEM UPGRADE, INC Making Good Computers Better. System Upgrade Teaches RAID
System Upgrade Teaches RAID In the growing computer industry we often find it difficult to keep track of the everyday changes in technology. At System Upgrade, Inc it is our goal and mission to provide
More informationToday s Papers. Array Reliability. RAID Basics (Two optional papers) EECS 262a Advanced Topics in Computer Systems Lecture 3
EECS 262a Advanced Topics in Computer Systems Lecture 3 Filesystems (Con t) September 10 th, 2012 John Kubiatowicz and Anthony D. Joseph Electrical Engineering and Computer Sciences University of California,
More informationINTERNAL REPRESENTATION OF FILES:
INTERNAL REPRESENTATION OF FILES: Every file on a UNIX system has a unique inode. The inode contains the information necessary for a process to access a file, such as file ownership, access rights, file
More informationCS 162 Operating Systems and Systems Programming Professor: Anthony D. Joseph Spring Lecture 18: Naming, Directories, and File Caching
CS 162 Operating Systems and Systems Programming Professor: Anthony D. Joseph Spring 2004 Lecture 18: Naming, Directories, and File Caching 18.0 Main Points How do users name files? What is a name? Lookup:
More informationPhysical Storage Media
Physical Storage Media These slides are a modified version of the slides of the book Database System Concepts, 5th Ed., McGraw-Hill, by Silberschatz, Korth and Sudarshan. Original slides are available
More informationChapter 11: Implementing File-Systems
Chapter 11: Implementing File-Systems Chapter 11 File-System Implementation 11.1 File-System Structure 11.2 File-System Implementation 11.3 Directory Implementation 11.4 Allocation Methods 11.5 Free-Space
More informationFrequently asked questions from the previous class survey
CS 370: OPERATING SYSTEMS [FILE SYSTEMS] Shrideep Pallickara Computer Science Colorado State University L28.1 Frequently asked questions from the previous class survey How are files recovered if the drive
More informationCS 162 Operating Systems and Systems Programming Professor: Anthony D. Joseph Spring Lecture 18: Naming, Directories, and File Caching
CS 162 Operating Systems and Systems Programming Professor: Anthony D. Joseph Spring 2002 Lecture 18: Naming, Directories, and File Caching 18.0 Main Points How do users name files? What is a name? Lookup:
More informationCS252 S05. Main memory management. Memory hardware. The scale of things. Memory hardware (cont.) Bottleneck
Main memory management CMSC 411 Computer Systems Architecture Lecture 16 Memory Hierarchy 3 (Main Memory & Memory) Questions: How big should main memory be? How to handle reads and writes? How to find
More informationRAID. Redundant Array of Inexpensive Disks. Industry tends to use Independent Disks
RAID Chapter 5 1 RAID Redundant Array of Inexpensive Disks Industry tends to use Independent Disks Idea: Use multiple disks to parallelise Disk I/O for better performance Use multiple redundant disks for
More informationSilberschatz, et al. Topics based on Chapter 13
Silberschatz, et al. Topics based on Chapter 13 Mass Storage Structure CPSC 410--Richard Furuta 3/23/00 1 Mass Storage Topics Secondary storage structure Disk Structure Disk Scheduling Disk Management
More informationChe-Wei Chang Department of Computer Science and Information Engineering, Chang Gung University
Che-Wei Chang chewei@mail.cgu.edu.tw Department of Computer Science and Information Engineering, Chang Gung University Chapter 10: File System Chapter 11: Implementing File-Systems Chapter 12: Mass-Storage
More informationWednesday, April 25, Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes
Wednesday, April 25, 2018 Topics for today Secondary memory Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes Storage management (Chapter
More informationDa-Wei Chang CSIE.NCKU. Professor Hao-Ren Ke, National Chiao Tung University Professor Hsung-Pin Chang, National Chung Hsing University
Chapter 11 Implementing File System Da-Wei Chang CSIE.NCKU Source: Professor Hao-Ren Ke, National Chiao Tung University Professor Hsung-Pin Chang, National Chung Hsing University Outline File-System Structure
More informationFile System CS170 Discussion Week 9. *Some slides taken from TextBook Author s Presentation
File System CS170 Discussion Week 9 *Some slides taken from TextBook Author s Presentation File-System Structure File structure Logical storage unit Collection of related information File system resides
More informationLecture 21: Reliable, High Performance Storage. CSC 469H1F Fall 2006 Angela Demke Brown
Lecture 21: Reliable, High Performance Storage CSC 469H1F Fall 2006 Angela Demke Brown 1 Review We ve looked at fault tolerance via server replication Continue operating with up to f failures Recovery
More informationI/O CANNOT BE IGNORED
LECTURE 13 I/O I/O CANNOT BE IGNORED Assume a program requires 100 seconds, 90 seconds for main memory, 10 seconds for I/O. Assume main memory access improves by ~10% per year and I/O remains the same.
More informationRAID SEMINAR REPORT /09/2004 Asha.P.M NO: 612 S7 ECE
RAID SEMINAR REPORT 2004 Submitted on: Submitted by: 24/09/2004 Asha.P.M NO: 612 S7 ECE CONTENTS 1. Introduction 1 2. The array and RAID controller concept 2 2.1. Mirroring 3 2.2. Parity 5 2.3. Error correcting
More informationmakes floppy bootable o next comes root directory file information ATTRIB command used to modify name
File Systems File system o Designed for storing and managing files on disk media o Build logical system on top of physical disk organization Tasks o Partition and format disks to store and retrieve information
More informationChapter 13 Disk Storage, Basic File Structures, and Hashing.
Chapter 13 Disk Storage, Basic File Structures, and Hashing. Copyright 2004 Pearson Education, Inc. Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files Ordered Files
More informationLecture S3: File system data layout, naming
Lecture S3: File system data layout, naming Review -- 1 min Intro to I/O Performance model: Log Disk physical characteristics/desired abstractions Physical reality Desired abstraction disks are slow fast
More informationChapter 11: Implementing File Systems
Chapter 11: Implementing File Systems Operating System Concepts 99h Edition DM510-14 Chapter 11: Implementing File Systems File-System Structure File-System Implementation Directory Implementation Allocation
More informationToday: Coda, xfs. Case Study: Coda File System. Brief overview of other file systems. xfs Log structured file systems HDFS Object Storage Systems
Today: Coda, xfs Case Study: Coda File System Brief overview of other file systems xfs Log structured file systems HDFS Object Storage Systems Lecture 20, page 1 Coda Overview DFS designed for mobile clients
More informationRicardo Rocha. Department of Computer Science Faculty of Sciences University of Porto
Ricardo Rocha Department of Computer Science Faculty of Sciences University of Porto Slides based on the book Operating System Concepts, 9th Edition, Abraham Silberschatz, Peter B. Galvin and Greg Gagne,
More informationPage 1. Recap: File System Goals" Recap: Linked Allocation"
Recap: File System Goals" CS162 Operating Systems and Systems Programming Lecture 14 File Systems (cont d), Key Value Storage Systems" Maximize sequential performance Efiicient random access to file Easy
More informationMain Points. File layout Directory layout
File Systems Main Points File layout Directory layout File System Design Constraints For small files: Small blocks for storage efficiency Files used together should be stored together For large files:
More information(a) Which of these two conditions (high or low) is considered more serious? Justify your answer.
CS140 Winter 2006 Final Exam Solutions (1) In class we talked about the link count in the inode of the Unix file system being incorrect after a crash. The reference count can either be either too high
More informationStoring Data: Disks and Files. Storing and Retrieving Data. Why Not Store Everything in Main Memory? Chapter 7
Storing : Disks and Files Chapter 7 base Management Systems, R. Ramakrishnan and J. Gehrke 1 Storing and Retrieving base Management Systems need to: Store large volumes of data Store data reliably (so
More informationChapter 11: Implementing File
Chapter 11: Implementing File Systems Chapter 11: Implementing File Systems File-System Structure File-System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency
More informationChapter 14 Mass-Storage Structure
Chapter 14 Mass-Storage Structure 1 Outline Disk Structure Disk Scheduling Disk Management Swap-Space Management RAID Structure Disk Attachment Stable-Storage Implementation Tertiary Storage Devices 2
More informationChapter 11. I/O Management and Disk Scheduling
Operating System Chapter 11. I/O Management and Disk Scheduling Lynn Choi School of Electrical Engineering Categories of I/O Devices I/O devices can be grouped into 3 categories Human readable devices
More informationStoring and Retrieving Data. Storing Data: Disks and Files. Solution 1: Techniques for making disks faster. Disks. Why Not Store Everything in Tapes?
Storing and Retrieving Storing : Disks and Files Chapter 9 base Management Systems need to: Store large volumes of data Store data reliably (so that data is not lost!) Retrieve data efficiently Alternatives
More information