Database Systems II. Secondary Storage
|
|
- Chastity Francis
- 5 years ago
- Views:
Transcription
1 Database Systems II Secondary Storage CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 29 The Memory Hierarchy Swapping, Main-memory DBMS s Tertiary Storage: Tape, Network Backup 3,200 MB/s 6,400 MB/s 12,800 MB/s (DDR2, dual channel, 800MHz) Virtual Memory Disk Disk-Cache (2 16MB) Main Memory CPU File System L1/L2-Cache (256KB 4MB) 300 MB/s (SATA-300) 16 GB/s (64bit@2GHz) CPU-to-Main-Memory: ~200 cycles latency CPU-to-L1-Cache: ~5 cycles initial latency, then burst mode CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 30
2 The Memory Hierarchy Cache Data and instructions in cache when needed by CPU. On-board (L1) cache on same chip as CPU, L2 cache on separate chip. Capacity ~ 1MB, access time a few nanoseconds. Main memory All active programs and data need to be in main memory. Capacity ~ 1 GB, access time nanoseconds. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 31 The Memory Hierarchy Secondary storage Secondary storage is used for permanent storage of large amounts of data, typically a magnetic disk. Capacity up to 1 TB, access time ~ 10 milliseconds. Tertiary storage To store data collections that do not fit onto secondary storage, e.g. magnetic tapes or optical disks. Capacity ~ 1 PB, access time seconds / minutes. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 32
3 The Memory Hierarchy Trade-off The larger the capacity of a storage device, the slower the access (and vice versa). A volatile storage device forgets its contents when power is switched off, a non-volatile device remembers its content. Secondary storage and tertiary storage is nonvolatile, all others are volatile. DBS needs non-volatile (secondary) storage devices to store data permanently. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 33 The Memory Hierarchy RAM (main memory) for subset of database used by current transactions. Disk to store current version of entire database (secondary storage). Tapes for archiving older versions of the database (tertiary storage). CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 34
4 The Memory Hierarchy Typically programs are executed in virtual memory of size equal to the address space of the processor. Virtual memory is managed by the operating system, which keeps the most relevant part in the main memory and the rest on disk. A DBS manages the data itself and does not rely on the virtual memory. However, main memory DBS do manage their data through virtual memory. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 35 Moore s Law Gordon Moore in 1965 observed that the density of integrated circuits (i.e., number of transistors per unit) increased at an exponential rate, thus roughly doubles every 18 months. Parameters that follow Moore s law: - number of instructions per second that can be exceuted for unit cost, - number of main memory bits that can be bought for unit cost, - number of bytes on a disk that can be bought for unit cost. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 36
5 Moore s Law Number of transistors on an integrated circuit CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 37 Moore s Law Disk capacity CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 38
6 Moore s Law But some other important hardware parameters do not follow Moore s law and grow much slower. Theses are, in particular, - speed of main memory access, and - speed of disk access. For example, disk latencies (seek times) have almost stagnated for past 5 years. Thus, moving data from one level of the memory hierarchy to the next becomes progressively larger. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 39 Disks Secondary storage device of choice. Data is stored and retrieved in units called disk blocks or pages. Main advantage over tapes: random access vs. sequential access. Unlike RAM, time to retrieve a disk page varies depending upon location on disk. Therefore, relative placement of pages on disk has major impact on DBMS performance! CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 40
7 Disks Disk consists of two main, moving parts: disk assembly and head assembly. Disk assembly stores information, head assembly reads and writes information. Arm assembly Disk head Arm movement Spindle Tracks Platters CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 41 Disks The platters rotate around central spindle. Upper and lower platter surfaces are covered with magnetic material, which is used to store bits. The arm assembly is moved in or out to position a head on a desired track. All tracks under heads at the same time make a cylinder (imaginary!). Only one head reads/writes at any one time. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 42
8 Disks Sector Track Top view of a platter surface Gap CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 43 Disks Block size is a multiple of sector size (which is fixed). Time to access (read/write) a disk block (disk latency) consists of three components: - seek time: moving arms to position disk head on track, - rotational delay (waiting for block to rotate under head), and - transfer time (actually moving data to/from disk surface). Seek time and rotational delay dominate. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 44
9 Disks Time Seek time 3 or 5x x 1 N Cylinders Traveled CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 45 Disks Average seek time S = N N i=1 j=1 j i SEEKTIME (i j) N(N-1) Typical average seek time = 5 ms CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 46
10 Disks Average rotational delay Head Here Block I Want Average rotational delay R = 1/2 revolution Typical R = 5 ms CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 47 Disks Transfer time Typical transfer rate: 100 MB/sec Typical block size: 16KB Transfer time: block size transfer rate Typical transfer time = 0.16 ms CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 48
11 Disks Typical average disk latency is 10 ms, maximum latency 20 ms. In 10 ms, a modern microprocessor can execute millions of instructions. Thus, the time for a block access by far dominates the time typically needed for processing the data in memory. The number of disk I/Os (block accesses) is a good approximation for the cost of a database operation. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 49 Accelerating Disk Access Organize data by cylinders to minimize the seek time and rotational delay. Next block concept: - blocks on same track, followed by - blocks on same cylinder, followed by - blocks on adjacent cylinder. Blocks in a file are placed sequentially on disk (by next ). Disk latency can approach the transfer rate. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 50
12 Accelerating Disk Access Example Assuming 10 ms average seek time, no rotational delay, 40 MB/s transfer rate. Read a single 4 KB Block Random I/O: 10 ms Sequential I/O: 10 ms Read 4 MB in 4 KB Blocks (amortized) Random I/O: 10 s Sequential I/O: 0.1 s Speedup factor of 100 CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 51 Accelerating Disk Access Block size selection Bigger blocks amortize I/O cost. Bigger blocks read in more useless stuff and takes longer to read. Good trade-off block size from 4KB to 16 KB. With decreasing memory costs, blocks are becoming bigger! CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 52
13 Accelerating Disk Access Using multiple disks Replace one disk (with one independent head) by many disks (with many independent heads). Striping a relation R: divide its blocks over n disks in a round robin fashion. Assuming that disk controller, bus and main memory can handle n times the transfer rate, striping a relation across n disks can lead to a speedup factor of up to n. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 53 Accelerating Disk Access Disk scheduling For I/O requests from different processes, let the disk controller choose the processing order. According to the elevator algorithm, the disk controller keeps sweeping from the innermost to the outermost cylinder, stopping at a cylinder for which there is an I/O request. Can reverse sweep direction as soon as there is no I/O request ahead in the current direction. Optimizes the throughput and average response time. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 54
14 Accelerating Disk Access Double buffering In some scenarios, we can predict the order in which blocks will be requested from disk by some process. Prefetching (double buffering) is the method of fetching the necessary blocks into the buffer in advance. Requires enough buffer space. Speedup factor up to n, where n is the number of blocks requested by a process. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 55 Accelerating Disk Access (1) Read B1 Buffer (2) Process Data in Buffer Single buffering (3) Read B2 Buffer (4) Process Data in Buffer... Execution time = n(p+r) where P = time to process one block R = time to read in one block n = # blocks read. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 56
15 Accelerating Disk Access Double buffering (1) Read B1,..., Bn Buffer (2) Process B1 in Buffer (3) Process B2 in Buffer... Execution time = R + np as opposed to n(p+r). remember that R >> P CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 57 Disk Failures In an intermittent failure, a read or write operation is unsuccessful, but succeeds with repeated tries. parity checks to detect intermittent failures Media decay is a permanent corruption of one or more bits which make the corresponding sector impossible to read / write. stable storage to recover from media decay A disk crash makes the entire disk permanently unreadable. RAID to recover from disk crashes CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 58
16 Disk Failures Checksums Add n parity bits every m data bits. The number of 1 s among a collection of bits and their parity bit is always even. The parity bit is the modulo-2 sum of its data bits. m=8, n=1 Block A: :1 Block B: :0 If Block A instead contains Block A : :1 error detected (odd # of 1 s) (even # of 1 s) (has odd # of 1 s) CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 59 Disk Failures Checksums But what if multiple bits are corrupted? E.g., if Block A instead contains Block A : :1 (has even # of 1 s) error cannot be detected Probability that a single parity bit cannot detect a corrupt block is ½. This is assuming that the probability of disk failures involving an odd / even number of bits is identical. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 60
17 Disk Failures Checksums More parity bits decrease the probability of an undetected failure. With n m independent parity bits, this probability is only 1/2 n. E.g., we can have eight parity bits, one for the first bit of every byte, the second one for the second bit of every byte... The chance for not detecting a disk failure is then only 1/256. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 61 Disk Failures Stable storage Sectors are paired, and information X is written both on sectors X l and X r. Assume that both copies are written with a sufficient number of parity bits so that bad sectors can be detected. If sector is bad (according to checksum), write to alternative sector. Alternate reading X l and X r until a good value is returned. Probability of X l and X r both failing is very low. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 62
18 Disk Failures Disk arrays So far, we cannot recover from disk crashes. To address this problem, use Redundant Arrays of Independent Disks (RAID), arrangements of several disks that gives abstraction of a single, large disk. Goals: Increase reliability (and performance). Redundant information allows reconstruction of data if a disk fails. Data striping improves the disk performance. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 63 Disk Failures Failure Models for Disks What is the expected time until disk crash? We assume uniform distribution of failures over time. Mean time to failure: time period by which 50% of a population of disks have failed (crashed). Typical mean time to failure is 10 years. In this case, 5% of disks crash in the first year, 5% crash in the second year,..., 5% crash in the tenth year,..., 5% crash in the twentieth year. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 64
19 Disk Failures Failure Models for Disks Given the mean time to failure (mtf) in years, we can derive the probability p of a particular disk failing in a given year. p = 1 / (2 * mtf) Ex.: mtf = 10, p = 1/20 = 5% Mean time to data loss: time period by which 50% of a population of disks have had a crash that resulted in data loss. The mean time to disk failure is not necessarily the same as the mean time to data loss. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 65 Disk Failures Failure Models for Disks Failure rate: percentage of disks of a population that have failed until a certain point of time. Survival rate: percentage of disks of a population that have not failed until a certain point of time. While it simplifies the analysis, the assumption of uniform distribution of failures is unrealistic. Disks tend to fail early (manufacturing defects that have not been detected) or late (wear-andtear). CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 66
20 Disk Failures Failure Models for Disks Survival rate (realistic) time CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 67 Disk Failures Mirroring The data disk is copied unto a second disk, the mirror disk. When one of the disk crashes, we replace it by a new disk and copy the other disk to the new one. Data loss can only occur if the second disk crashes while the first one is being replaced. This probability is negligible. Mirroring is referred to as RAID level 1. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 68
21 Disk Failures Parity blocks Mirroring doubles the number of disks needed. The parity block approach needs only one redundant disk for n (arbitray) data disks. In the redundant disk, the ith block stores parity checks for the ith blocks of all the n data disks. A B C P Parity block approach is called RAID level 4. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 69 Disk Failures Parity blocks Reading blocks is the same as without parity blocks. When writing a block on a data disk, we also need to update the corresponding block of the redundant disk. This can be done using four (three additional) disk I/O: read old value of data disk block, read corresponding block of redundant (parity) disk, write new data block, recompute and write new redundant block. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 70
22 Disk Failures Parity blocks If one of the disks crashes, we bring in a new disk. The content of this disk can be computed, bit by bit, using the remaining n disks. No difference between data disks and parity disk. Computation based on the definition of parity, i.e. total number of ones is even. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 71 Disk Failures n = 3 data disks Disk 1, block 1: Disk 2, block 1: Disk 3, block 1: and one parity disk Disk 4, block 1: Example Sum over each column is always an even number of 1 s Mod-2 sum can recover any missing single row CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 72
23 Disk Failures Suppose we have: Example Disk 1, block 1: Disk 2, block 1:???????? Disk 3, block 1: Disk 4, block 1: (parity) Use mod-2 sums for block 1 over disks 1,3,4 to recover block 1 of failed disk 2: Disk 2, block 1: CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 73 Disk Failures RAID level 5 In the RAID 4 scheme, the parity disk is the bottleneck. On average, n-times as many writes on the parity disk as on the data disks. However, the failure recovery method does not distinguish the types of the n + 1 disks. RAID level 5 does not use a fixed parity disk, but use block i of disk j as redundant if i MOD n+1 = j. A B C D CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 74
Data Storage and Query Answering. Data Storage and Disk Structure (2)
Data Storage and Query Answering Data Storage and Disk Structure (2) Review: The Memory Hierarchy Swapping, Main-memory DBMS s Tertiary Storage: Tape, Network Backup 3,200 MB/s (DDR-SDRAM @200MHz) 6,400
More informationPrinciples of Data Management. Lecture #2 (Storing Data: Disks and Files)
Principles of Data Management Lecture #2 (Storing Data: Disks and Files) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Topics v Today
More informationCS 554: Advanced Database System
CS 554: Advanced Database System Notes 02: Hardware Hector Garcia-Molina CS 245 Notes 2 1 Outline Hardware: Disks Access Times (disk) Optimizations (disk access time) Other Topics: Storage costs Using
More informationDatabase Systems. November 2, 2011 Lecture #7. topobo (mit)
Database Systems November 2, 2011 Lecture #7 1 topobo (mit) 1 Announcement Assignment #2 due today Assignment #3 out today & due on 11/16. Midterm exam in class next week. Cover Chapters 1, 2,
More informationStoring Data: Disks and Files
Storing Data: Disks and Files Chapter 9 CSE 4411: Database Management Systems 1 Disks and Files DBMS stores information on ( 'hard ') disks. This has major implications for DBMS design! READ: transfer
More informationDisks, Memories & Buffer Management
Disks, Memories & Buffer Management The two offices of memory are collection and distribution. - Samuel Johnson CS3223 - Storage 1 What does a DBMS Store? Relations Actual data Indexes Data structures
More informationMass-Storage Structure
CS 4410 Operating Systems Mass-Storage Structure Summer 2011 Cornell University 1 Today How is data saved in the hard disk? Magnetic disk Disk speed parameters Disk Scheduling RAID Structure 2 Secondary
More informationIntroduction Disks RAID Tertiary storage. Mass Storage. CMSC 420, York College. November 21, 2006
November 21, 2006 The memory hierarchy Red = Level Access time Capacity Features Registers nanoseconds 100s of bytes fixed Cache nanoseconds 1-2 MB fixed RAM nanoseconds MBs to GBs expandable Disk milliseconds
More informationStoring and Retrieving Data. Storing Data: Disks and Files. Solution 1: Techniques for making disks faster. Disks. Why Not Store Everything in Tapes?
Storing and Retrieving Storing : Disks and Files Chapter 9 base Management Systems need to: Store large volumes of data Store data reliably (so that data is not lost!) Retrieve data efficiently Alternatives
More informationStoring Data: Disks and Files
Storing Data: Disks and Files CS 186 Fall 2002, Lecture 15 (R&G Chapter 7) Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Stuff Rest of this week My office
More informationStoring Data: Disks and Files. Storing and Retrieving Data. Why Not Store Everything in Main Memory? Database Management Systems need to:
Storing : Disks and Files base Management System, R. Ramakrishnan and J. Gehrke 1 Storing and Retrieving base Management Systems need to: Store large volumes of data Store data reliably (so that data is
More informationStoring Data: Disks and Files
Storing Data: Disks and Files Chapter 9 Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke Disks
More informationStoring and Retrieving Data. Storing Data: Disks and Files. Solution 1: Techniques for making disks faster. Disks. Why Not Store Everything in Tapes?
Storing and Retrieving Storing : Disks and Files base Management Systems need to: Store large volumes of data Store data reliably (so that data is not lost!) Retrieve data efficiently Alternatives for
More informationCSCI-GA Database Systems Lecture 8: Physical Schema: Storage
CSCI-GA.2433-001 Database Systems Lecture 8: Physical Schema: Storage Mohamed Zahran (aka Z) mzahran@cs.nyu.edu http://www.mzahran.com View 1 View 2 View 3 Conceptual Schema Physical Schema 1. Create a
More informationStoring Data: Disks and Files. Storing and Retrieving Data. Why Not Store Everything in Main Memory? Chapter 7
Storing : Disks and Files Chapter 7 base Management Systems, R. Ramakrishnan and J. Gehrke 1 Storing and Retrieving base Management Systems need to: Store large volumes of data Store data reliably (so
More informationVirtual Memory. Reading. Sections 5.4, 5.5, 5.6, 5.8, 5.10 (2) Lecture notes from MKP and S. Yalamanchili
Virtual Memory Lecture notes from MKP and S. Yalamanchili Sections 5.4, 5.5, 5.6, 5.8, 5.10 Reading (2) 1 The Memory Hierarchy ALU registers Cache Memory Memory Memory Managed by the compiler Memory Managed
More informationCMSC 424 Database design Lecture 12 Storage. Mihai Pop
CMSC 424 Database design Lecture 12 Storage Mihai Pop Administrative Office hours tomorrow @ 10 Midterms are in solutions for part C will be posted later this week Project partners I have an odd number
More informationCISC 7310X. C11: Mass Storage. Hui Chen Department of Computer & Information Science CUNY Brooklyn College. 4/19/2018 CUNY Brooklyn College
CISC 7310X C11: Mass Storage Hui Chen Department of Computer & Information Science CUNY Brooklyn College 4/19/2018 CUNY Brooklyn College 1 Outline Review of memory hierarchy Mass storage devices Reliability
More informationL9: Storage Manager Physical Data Organization
L9: Storage Manager Physical Data Organization Disks and files Record and file organization Indexing Tree-based index: B+-tree Hash-based index c.f. Fig 1.3 in [RG] and Fig 2.3 in [EN] Functional Components
More informationStorage Devices for Database Systems
Storage Devices for Database Systems 5DV120 Database System Principles Umeå University Department of Computing Science Stephen J. Hegner hegner@cs.umu.se http://www.cs.umu.se/~hegner Storage Devices for
More informationData Storage and Disk Structure
Data Storage and Disk Structure A Simple Implementation of DBMS One file per table Students(name, id, dept) in a file Students A meta symbol # to separate attributes Smith#123#CS Johnson#522#EE Database
More informationUNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Computer Architecture ECE 568
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Computer Architecture ECE 568 Part 6 Input/Output Israel Koren ECE568/Koren Part.6. Motivation: Why Care About I/O? CPU Performance:
More informationDatabase Architecture 2 & Storage. Instructor: Matei Zaharia cs245.stanford.edu
Database Architecture 2 & Storage Instructor: Matei Zaharia cs245.stanford.edu Summary from Last Time System R mostly matched the architecture of a modern RDBMS» SQL» Many storage & access methods» Cost-based
More informationI/O CANNOT BE IGNORED
LECTURE 13 I/O I/O CANNOT BE IGNORED Assume a program requires 100 seconds, 90 seconds for main memory, 10 seconds for I/O. Assume main memory access improves by ~10% per year and I/O remains the same.
More informationStorage and File Structure. Classification of Physical Storage Media. Physical Storage Media. Physical Storage Media
Storage and File Structure Classification of Physical Storage Media Overview of Physical Storage Media Magnetic Disks RAID Tertiary Storage Storage Access File Organization Organization of Records in Files
More informationStoring Data: Disks and Files
Storing Data: Disks and Files Chapter 7 (2 nd edition) Chapter 9 (3 rd edition) Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Database Management Systems,
More informationStorage Systems. Storage Systems
Storage Systems Storage Systems We already know about four levels of storage: Registers Cache Memory Disk But we've been a little vague on how these devices are interconnected In this unit, we study Input/output
More informationDisks. Storage Technology. Vera Goebel Thomas Plagemann. Department of Informatics University of Oslo
Disks Vera Goebel Thomas Plagemann 2014 Department of Informatics University of Oslo Storage Technology [Source: http://www-03.ibm.com/ibm/history/exhibits/storage/storage_photo.html] 1 Filesystems & Disks
More informationToday: Secondary Storage! Typical Disk Parameters!
Today: Secondary Storage! To read or write a disk block: Seek: (latency) position head over a track/cylinder. The seek time depends on how fast the hardware moves the arm. Rotational delay: (latency) time
More informationChe-Wei Chang Department of Computer Science and Information Engineering, Chang Gung University
Che-Wei Chang chewei@mail.cgu.edu.tw Department of Computer Science and Information Engineering, Chang Gung University l Chapter 10: File System l Chapter 11: Implementing File-Systems l Chapter 12: Mass-Storage
More informationDisks and Files. Storage Structures Introduction Chapter 8 (3 rd edition) Why Not Store Everything in Main Memory?
Why Not Store Everything in Main Memory? Storage Structures Introduction Chapter 8 (3 rd edition) Sharma Chakravarthy UT Arlington sharma@cse.uta.edu base Management Systems: Sharma Chakravarthy Costs
More informationStoring Data: Disks and Files
Storing Data: Disks and Files Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Data Access Disks and Files DBMS stores information on ( hard ) disks. This
More informationStoring Data: Disks and Files
Storing Data: Disks and Files Module 2, Lecture 1 Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet Database Management Systems, R. Ramakrishnan 1 Disks and
More informationChapter 10: Mass-Storage Systems
Chapter 10: Mass-Storage Systems Silberschatz, Galvin and Gagne 2013 Chapter 10: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management Swap-Space
More informationCh 11: Storage and File Structure
Ch 11: Storage and File Structure Overview of Physical Storage Media Magnetic Disks RAID Tertiary Storage Storage Access File Organization Organization of Records in Files Data-Dictionary Dictionary Storage
More informationUNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Computer Architecture ECE 568
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Computer Architecture ECE 568 Part 6 Input/Output Israel Koren ECE568/Koren Part.6. CPU performance keeps increasing 26 72-core Xeon
More informationI/O, Disks, and RAID Yi Shi Fall Xi an Jiaotong University
I/O, Disks, and RAID Yi Shi Fall 2017 Xi an Jiaotong University Goals for Today Disks How does a computer system permanently store data? RAID How to make storage both efficient and reliable? 2 What does
More informationStorage. CS 3410 Computer System Organization & Programming
Storage CS 3410 Computer System Organization & Programming These slides are the product of many rounds of teaching CS 3410 by Deniz Altinbuke, Kevin Walsh, and Professors Weatherspoon, Bala, Bracy, and
More informationChapter 10: Mass-Storage Systems. Operating System Concepts 9 th Edition
Chapter 10: Mass-Storage Systems Silberschatz, Galvin and Gagne 2013 Chapter 10: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management Swap-Space
More informationComputer System Architecture
CSC 203 1.5 Computer System Architecture Department of Statistics and Computer Science University of Sri Jayewardenepura Secondary Memory 2 Technologies Magnetic storage Floppy, Zip disk, Hard drives,
More informationDisks & Files. Yanlei Diao UMass Amherst. Slides Courtesy of R. Ramakrishnan and J. Gehrke
Disks & Files Yanlei Diao UMass Amherst Slides Courtesy of R. Ramakrishnan and J. Gehrke DBMS Architecture Query Parser Query Rewriter Query Optimizer Query Executor Lock Manager for Concurrency Access
More informationSemiconductor Memory Types Microprocessor Design & Organisation HCA2102
Semiconductor Memory Types Microprocessor Design & Organisation HCA2102 Internal & External Memory Semiconductor Memory RAM Misnamed as all semiconductor memory is random access Read/Write Volatile Temporary
More informationChapter 12: Mass-Storage
Chapter 12: Mass-Storage Systems Chapter 12: Mass-Storage Systems Revised 2010. Tao Yang Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management Swap-Space Management
More informationV. Mass Storage Systems
TDIU25: Operating Systems V. Mass Storage Systems SGG9: chapter 12 o Mass storage: Hard disks, structure, scheduling, RAID Copyright Notice: The lecture notes are mainly based on modifications of the slides
More informationBBM371- Data Management. Lecture 2: Storage Devices
BBM371- Data Management Lecture 2: Storage Devices 18.10.2018 Memory Hierarchy cache Main memory disk Optical storage Tapes V NV Traveling the hierarchy: 1. speed ( higher=faster) 2. cost (lower=cheaper)
More informationCS3600 SYSTEMS AND NETWORKS
CS3600 SYSTEMS AND NETWORKS NORTHEASTERN UNIVERSITY Lecture 9: Mass Storage Structure Prof. Alan Mislove (amislove@ccs.neu.edu) Moving-head Disk Mechanism 2 Overview of Mass Storage Structure Magnetic
More informationA memory is what is left when something happens and does not completely unhappen.
7 STORING DATA: DISKS &FILES A memory is what is left when something happens and does not completely unhappen. Edward DeBono This chapter initiates a study of the internals of an RDBMS. In terms of the
More informationI/O CANNOT BE IGNORED
LECTURE 13 I/O I/O CANNOT BE IGNORED Assume a program requires 100 seconds, 90 seconds for main memory, 10 seconds for I/O. Assume main memory access improves by ~10% per year and I/O remains the same.
More informationSTORING DATA: DISK AND FILES
STORING DATA: DISK AND FILES CS 564- Spring 2018 ACKs: Dan Suciu, Jignesh Patel, AnHai Doan WHAT IS THIS LECTURE ABOUT? How does a DBMS store data? disk, SSD, main memory The Buffer manager controls how
More informationCPSC 421 Database Management Systems. Lecture 11: Storage and File Organization
CPSC 421 Database Management Systems Lecture 11: Storage and File Organization * Some material adapted from R. Ramakrishnan, L. Delcambre, and B. Ludaescher Today s Agenda Start on Database Internals:
More informationModule 13: Secondary-Storage Structure
Module 13: Secondary-Storage Structure Disk Structure Disk Scheduling Disk Management Swap-Space Management Disk Reliability Stable-Storage Implementation Operating System Concepts 13.1 Silberschatz and
More informationCOS 318: Operating Systems. Storage Devices. Vivek Pai Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Vivek Pai Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall11/cos318/ Today s Topics Magnetic disks Magnetic disk
More informationCOS 318: Operating Systems. Storage Devices. Kai Li Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Kai Li Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall11/cos318/ Today s Topics Magnetic disks Magnetic disk
More informationProfessor: Pete Keleher! Closures, candidate keys, canonical covers etc! Armstrong axioms!
Professor: Pete Keleher! keleher@cs.umd.edu! } Mechanisms and definitions to work with FDs! Closures, candidate keys, canonical covers etc! Armstrong axioms! } Decompositions! Loss-less decompositions,
More informationCS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Fall 2016 Lecture 35 Mass Storage Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 Questions For You Local/Global
More informationCOMP283-Lecture 3 Applied Database Management
COMP283-Lecture 3 Applied Database Management Introduction DB Design Continued Disk Sizing Disk Types & Controllers DB Capacity 1 COMP283-Lecture 3 DB Storage: Linear Growth Disk space requirements increases
More informationChapter 12: Mass-Storage Systems. Operating System Concepts 8 th Edition,
Chapter 12: Mass-Storage Systems, Silberschatz, Galvin and Gagne 2009 Chapter 12: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management
More informationPhysical Storage Media
Physical Storage Media These slides are a modified version of the slides of the book Database System Concepts, 5th Ed., McGraw-Hill, by Silberschatz, Korth and Sudarshan. Original slides are available
More informationu Covered: l Management of CPU & concurrency l Management of main memory & virtual memory u Currently --- Management of I/O devices
Where Are We? COS 318: Operating Systems Storage Devices Jaswinder Pal Singh Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) u Covered: l Management of CPU
More informationAdministrivia. CMSC 411 Computer Systems Architecture Lecture 19 Storage Systems, cont. Disks (cont.) Disks - review
Administrivia CMSC 411 Computer Systems Architecture Lecture 19 Storage Systems, cont. Homework #4 due Thursday answers posted soon after Exam #2 on Thursday, April 24 on memory hierarchy (Unit 4) and
More informationDisks and Files. Jim Gray s Storage Latency Analogy: How Far Away is the Data? Components of a Disk. Disks
Review Storing : Disks and Files Lecture 3 (R&G Chapter 9) Aren t bases Great? Relational model SQL Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet A few
More informationChapter 14: Mass-Storage Systems
Chapter 14: Mass-Storage Systems Disk Structure Disk Scheduling Disk Management Swap-Space Management RAID Structure Disk Attachment Stable-Storage Implementation Tertiary Storage Devices Operating System
More informationChapter 10: Mass-Storage Systems
COP 4610: Introduction to Operating Systems (Spring 2016) Chapter 10: Mass-Storage Systems Zhi Wang Florida State University Content Overview of Mass Storage Structure Disk Structure Disk Scheduling Disk
More informationContents. Memory System Overview Cache Memory. Internal Memory. Virtual Memory. Memory Hierarchy. Registers In CPU Internal or Main memory
Memory Hierarchy Contents Memory System Overview Cache Memory Internal Memory External Memory Virtual Memory Memory Hierarchy Registers In CPU Internal or Main memory Cache RAM External memory Backing
More informationCS 261 Fall Mike Lam, Professor. Memory
CS 261 Fall 2016 Mike Lam, Professor Memory Topics Memory hierarchy overview Storage technologies SRAM DRAM PROM / flash Disk storage Tape and network storage I/O architecture Storage trends Latency comparisons
More informationLecture 15 - Chapter 10 Storage and File Structure
CMSC 461, Database Management Systems Spring 2018 Lecture 15 - Chapter 10 Storage and File Structure These slides are based on Database System Concepts 6th edition book (whereas some quotes and figures
More informationUNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Computer Architecture ECE 568
UNIVERSITY OF MASSACHUSETTS Dept of Electrical & Computer Engineering Computer Architecture ECE 568 art 5 Input/Output Israel Koren ECE568/Koren art5 CU performance keeps increasing 26 72-core Xeon hi
More informationStorage and File Structure
Storage and File Structure 1 Roadmap of This Lecture Overview of Physical Storage Media Magnetic Disks RAID Tertiary Storage Storage Access File Organization Organization of Records in Files Data-Dictionary
More informationModule 13: Secondary-Storage
Module 13: Secondary-Storage Disk Structure Disk Scheduling Disk Management Swap-Space Management Disk Reliability Stable-Storage Implementation Tertiary Storage Devices Operating System Issues Performance
More informationCOS 318: Operating Systems. Storage Devices. Jaswinder Pal Singh Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Jaswinder Pal Singh Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall13/cos318/ Today s Topics Magnetic disks
More informationChapter 9: Peripheral Devices: Magnetic Disks
Chapter 9: Peripheral Devices: Magnetic Disks Basic Disk Operation Performance Parameters and History of Improvement Example disks RAID (Redundant Arrays of Inexpensive Disks) Improving Reliability Improving
More informationChapter 14: Mass-Storage Systems. Disk Structure
1 Chapter 14: Mass-Storage Systems Disk Structure Disk Scheduling Disk Management Swap-Space Management RAID Structure Disk Attachment Stable-Storage Implementation Tertiary Storage Devices Operating System
More informationDisks and RAID. CS 4410 Operating Systems. [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse]
Disks and RAID CS 4410 Operating Systems [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse] Storage Devices Magnetic disks Storage that rarely becomes corrupted Large capacity at low cost Block
More informationMASS-STORAGE STRUCTURE
UNIT IV MASS-STORAGE STRUCTURE Mass-Storage Systems ndescribe the physical structure of secondary and tertiary storage devices and the resulting effects on the uses of the devicesnexplain the performance
More informationChapter 10: Mass-Storage Systems. Operating System Concepts 9 th Edition
Chapter 10: Mass-Storage Systems Silberschatz, Galvin and Gagne 2013 Objectives To describe the physical structure of secondary storage devices and its effects on the uses of the devices To explain the
More informationCENG3420 Lecture 08: Memory Organization
CENG3420 Lecture 08: Memory Organization Bei Yu byu@cse.cuhk.edu.hk (Latest update: February 22, 2018) Spring 2018 1 / 48 Overview Introduction Random Access Memory (RAM) Interleaving Secondary Memory
More informationOutlines. Chapter 2 Storage Structure. Structure of a DBMS (with some simplification) Structure of a DBMS (with some simplification)
Outlines Chapter 2 Storage Structure Instructor: Churee Techawut 1) Structure of a DBMS 2) The memory hierarchy 3) Magnetic tapes 4) Magnetic disks 5) RAID 6) Disk space management 7) Buffer management
More informationCSE325 Principles of Operating Systems. Mass-Storage Systems. David P. Duggan. April 19, 2011
CSE325 Principles of Operating Systems Mass-Storage Systems David P. Duggan dduggan@sandia.gov April 19, 2011 Outline Storage Devices Disk Scheduling FCFS SSTF SCAN, C-SCAN LOOK, C-LOOK Redundant Arrays
More informationMagnetic Disks. Tore Brox-Larsen Including material developed by Pål Halvorsen, University of Oslo (primarily) Kai Li, Princeton University
Magnetic Disks Tore Brox-Larsen Including material developed by Pål Halvorsen, University of Oslo (primarily) Kai Li, Princeton University Magnetic Disks (Hard disk drive) Data are represented physically
More informationMass-Storage. ICS332 - Fall 2017 Operating Systems. Henri Casanova
Mass-Storage ICS332 - Fall 2017 Operating Systems Henri Casanova (henric@hawaii.edu) Magnetic Disks! Magnetic disks (a.k.a. hard drives ) are (still) the most common secondary storage devices today! They
More informationSilberschatz, et al. Topics based on Chapter 13
Silberschatz, et al. Topics based on Chapter 13 Mass Storage Structure CPSC 410--Richard Furuta 3/23/00 1 Mass Storage Topics Secondary storage structure Disk Structure Disk Scheduling Disk Management
More informationChapter 12: Mass-Storage
hapter 12: Mass-Storage Systems hapter 12: Mass-Storage Systems To explain the performance characteristics of mass-storage devices To evaluate disk scheduling algorithms To discuss operating-system services
More informationChapter 10: Storage and File Structure
Chapter 10: Storage and File Structure Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 10: Storage and File Structure Overview of Physical Storage Media Magnetic
More informationChapter 12: Mass-Storage
hapter 12: Mass-Storage Systems hapter 12: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management RAID Structure Objectives Moving-head Disk
More informationMass-Storage Structure
Operating Systems (Fall/Winter 2018) Mass-Storage Structure Yajin Zhou (http://yajin.org) Zhejiang University Acknowledgement: some pages are based on the slides from Zhi Wang(fsu). Review On-disk structure
More informationIntroduction to I/O and Disk Management
1 Secondary Storage Management Disks just like memory, only different Introduction to I/O and Disk Management Why have disks? Ø Memory is small. Disks are large. Short term storage for memory contents
More informationCSE 451: Operating Systems Spring Module 12 Secondary Storage
CSE 451: Operating Systems Spring 2017 Module 12 Secondary Storage John Zahorjan 1 Secondary storage Secondary storage typically: is anything that is outside of primary memory does not permit direct execution
More informationIntroduction to I/O and Disk Management
Introduction to I/O and Disk Management 1 Secondary Storage Management Disks just like memory, only different Why have disks? Ø Memory is small. Disks are large. Short term storage for memory contents
More informationStorage systems. Computer Systems Architecture CMSC 411 Unit 6 Storage Systems. (Hard) Disks. Disk and Tape Technologies. Disks (cont.
Computer Systems Architecture CMSC 4 Unit 6 Storage Systems Alan Sussman November 23, 2004 Storage systems We already know about four levels of storage: registers cache memory disk but we've been a little
More informationPhysical Representation of Files
Physical Representation of Files A disk drive consists of a disk pack containing one or more platters stacked like phonograph records. Information is stored on both sides of the platter. Each platter is
More informationLecture 18: Memory Systems. Spring 2018 Jason Tang
Lecture 18: Memory Systems Spring 2018 Jason Tang 1 Topics Memory hierarchy Memory operations Cache basics 2 Computer Organization Computer Processor Memory Devices Control Datapath Input Output So far,
More informationTape pictures. CSE 30341: Operating Systems Principles
Tape pictures 4/11/07 CSE 30341: Operating Systems Principles page 1 Tape Drives The basic operations for a tape drive differ from those of a disk drive. locate positions the tape to a specific logical
More informationDatabase Management Systems, 2nd edition, Raghu Ramakrishnan, Johannes Gehrke, McGraw-Hill
Lecture Handout Database Management System Lecture No. 34 Reading Material Database Management Systems, 2nd edition, Raghu Ramakrishnan, Johannes Gehrke, McGraw-Hill Modern Database Management, Fred McFadden,
More informationData Storage - I: Memory Hierarchies & Disks. Contains slides from: Naci Akkök, Pål Halvorsen, Hector Garcia-Molina, Ketil Lund, Vera Goebel
Data Storage - I: Memory Hierarchies & Disks Contains slides from: Naci Akkök, Pål Halvorsen, Hector Garcia-Molina, Ketil Lund, Vera Goebel Overview Implementing a DBS is easy!!?? Memory hierarchies caches
More informationCS 405G: Introduction to Database Systems. Storage
CS 405G: Introduction to Database Systems Storage It s all about disks! Outline That s why we always draw databases as And why the single most important metric in database processing is the number of disk
More informationCSE 451: Operating Systems Winter Secondary Storage. Steve Gribble. Secondary storage
CSE 451: Operating Systems Winter 2005 Secondary Storage Steve Gribble Secondary storage Secondary storage typically: is anything that is outside of primary memory does not permit direct execution of instructions
More informationReview 1-- Storing Data: Disks and Files
Review 1-- Storing Data: Disks and Files Chapter 9 [Sections 9.1-9.7: Ramakrishnan & Gehrke (Text)] AND (Chapter 11 [Sections 11.1, 11.3, 11.6, 11.7: Garcia-Molina et al. (R2)] OR Chapter 2 [Sections 2.1,
More informationOS and HW Tuning Considerations!
Administração e Optimização de Bases de Dados 2012/2013 Hardware and OS Tuning Bruno Martins DEI@Técnico e DMIR@INESC-ID OS and HW Tuning Considerations OS " Threads Thread Switching Priorities " Virtual
More informationPart IV I/O System. Chapter 12: Mass Storage Structure
Part IV I/O System Chapter 12: Mass Storage Structure Disk Structure Three elements: cylinder, track and sector/block. Three types of latency (i.e., delay) Positional or seek delay mechanical and slowest
More informationThe Memory Hierarchy & Cache
Removing The Ideal Memory Assumption: The Memory Hierarchy & Cache The impact of real memory on CPU Performance. Main memory basic properties: Memory Types: DRAM vs. SRAM The Motivation for The Memory
More information