Storage Technologies - 3
|
|
|
- Cynthia Flynn
- 5 years ago
- Views:
Transcription
1 Storage Technologies - 3 COMP Lecture 10 Antoniu Pop antoniu.pop@manchester.ac.uk 1 March 2019 Antoniu Pop Storage Technologies / 20
2 Learning Objectives - Storage 3 Understand characteristics of Solid-State Drives (SSD) Antoniu Pop Storage Technologies / 20
3 Learning Objectives - Storage 3 Understand characteristics of Solid-State Drives (SSD) Compare SSD and Hard Disks Antoniu Pop Storage Technologies / 20
4 Learning Objectives - Storage 3 Understand characteristics of Solid-State Drives (SSD) Compare SSD and Hard Disks Understand Logical Volume Management (LVM) Antoniu Pop Storage Technologies / 20
5 Learning Objectives - Storage 3 Understand characteristics of Solid-State Drives (SSD) Compare SSD and Hard Disks Understand Logical Volume Management (LVM) Understand Storage Area Networks (SAN) Antoniu Pop Storage Technologies / 20
6 Learning Objectives - Storage 3 Understand characteristics of Solid-State Drives (SSD) Compare SSD and Hard Disks Understand Logical Volume Management (LVM) Understand Storage Area Networks (SAN) Relate LVM and SAN to a modern File System implementation Antoniu Pop Storage Technologies / 20
7 Solid-State Drive/Disk (SSD) Flash Memory: Floating Gate Field Effect Transistor Source Control Gate Floating Gate Drain Antoniu Pop Storage Technologies / 20
8 Solid-State Drive/Disk (SSD) Flash Memory: Floating Gate Field Effect Transistor Control Gate Floating Gate Charge stored on the floating gate Source Drain Antoniu Pop Storage Technologies / 20
9 Solid-State Drive/Disk (SSD) Flash Memory: Floating Gate Field Effect Transistor Control Gate Floating Gate Charge stored on the floating gate No electrical connection Source Drain Antoniu Pop Storage Technologies / 20
10 Solid-State Drive/Disk (SSD) Flash Memory: Floating Gate Field Effect Transistor Control Gate Floating Gate Charge stored on the floating gate No electrical connection Source Drain Conceptually like a switch: on (0) / off (1) Antoniu Pop Storage Technologies / 20
11 Solid-State Drive/Disk (SSD) Flash Memory: Floating Gate Field Effect Transistor Control Gate Floating Gate Charge stored on the floating gate No electrical connection Source Drain Conceptually like a switch: on (0) / off (1) Possibly multi-level (4 states - 2 bits) Antoniu Pop Storage Technologies / 20
12 Flash Controller Issues: Data retention (10 year?) Antoniu Pop Storage Technologies / 20
13 Flash Controller Issues: Data retention (10 year?) Wear-out with write cycles (aka P/E cycles) Antoniu Pop Storage Technologies / 20
14 Flash Controller Issues: Data retention (10 year?) Wear-out with write cycles (aka P/E cycles) Performance degradation with wearout Antoniu Pop Storage Technologies / 20
15 Flash Controller Issues: Data retention (10 year?) Wear-out with write cycles (aka P/E cycles) Performance degradation with wearout Implement: Error Correcting Codes Antoniu Pop Storage Technologies / 20
16 Flash Controller Issues: Data retention (10 year?) Wear-out with write cycles (aka P/E cycles) Performance degradation with wearout Implement: Error Correcting Codes (Bad) block remapping Antoniu Pop Storage Technologies / 20
17 Flash Controller Issues: Data retention (10 year?) Wear-out with write cycles (aka P/E cycles) Performance degradation with wearout Implement: Error Correcting Codes (Bad) block remapping Wear-levelling: Antoniu Pop Storage Technologies / 20
18 Flash Controller Issues: Data retention (10 year?) Wear-out with write cycles (aka P/E cycles) Performance degradation with wearout Implement: Error Correcting Codes (Bad) block remapping Wear-levelling: remap Logical Block Addresses (LBA) to physical addresses Antoniu Pop Storage Technologies / 20
19 Flash Controller Issues: Data retention (10 year?) Wear-out with write cycles (aka P/E cycles) Performance degradation with wearout Implement: Error Correcting Codes (Bad) block remapping Wear-levelling: remap Logical Block Addresses (LBA) to physical addresses avoid wearing out specific blocks Antoniu Pop Storage Technologies / 20
20 Disk access example (recap) Rotation Host initiates read sends a list of blocks to read Block schedule requested may not be optimal and leads to extra revolutions HDD internal processor optimizes the schedule No direct mapping from block numbers to the sector/track/cylinder position (high-level interfaces like ATA / SCSI) Antoniu Pop Storage Technologies / 20
21 Hard Disk Performance (recap) Seek time Time for the head to reach the target track. Search time Time for the target sector to arrive under the head. Also called rotational latency. Transfer rate Amount of data that can be read / written per unit of time. Dependent on access patterns. Aka. sustained transfer rate in contrast to interface transfer rate Disk access time = seek time + search time + transfer time Note: all values are average as they depend on many factors. Antoniu Pop Storage Technologies / 20
22 Hard Disks are too slow (recap) Slow because of: High seek time Reduce the number of times the head must move Multiple platters = more tracks/sectors per cylinder High search time (aka. rotational latency) Increase the rotation speed (e.g., server disks up to RPM) Low sustained transfer rate Increase rotation speed (physical limitations) Increase the recording density (physical limitations) Apply cache and prefetch principles Stripe file system across multiple disks Antoniu Pop Storage Technologies / 20
23 SSD vs. HDD HDD SSD Hitachi 7k4000 Samsung SSD 840 Antoniu Pop Storage Technologies / 20
24 SSD vs. HDD HDD SSD Streaming Reads 205 MB/s 530 MB/s Hitachi 7k4000 Samsung SSD 840 Antoniu Pop Storage Technologies / 20
25 SSD vs. HDD HDD SSD Streaming Reads 205 MB/s 530 MB/s Streaming Writes 205 MB/s 240 MB/s Hitachi 7k4000 Samsung SSD 840 Antoniu Pop Storage Technologies / 20
26 SSD vs. HDD HDD SSD Streaming Reads 205 MB/s 530 MB/s Streaming Writes 205 MB/s 240 MB/s Random 4kB Read 15.5 ms 11 µs Hitachi 7k4000 Samsung SSD 840 Antoniu Pop Storage Technologies / 20
27 SSD vs. HDD HDD SSD Streaming Reads 205 MB/s 530 MB/s Streaming Writes 205 MB/s 240 MB/s Random 4kB Read 15.5 ms 11 µs Random 4kB Write 6.4 ms 23 µs Hitachi 7k4000 Samsung SSD 840 Antoniu Pop Storage Technologies / 20
28 SSD vs. HDD HDD SSD Streaming Reads 205 MB/s 530 MB/s Streaming Writes 205 MB/s 240 MB/s Random 4kB Read 15.5 ms 11 µs Random 4kB Write 6.4 ms 23 µs Power 4/6/8 W 0.3/4.2 W Hitachi 7k4000 Samsung SSD 840 Antoniu Pop Storage Technologies / 20
29 SSD vs. HDD HDD SSD Streaming Reads 205 MB/s 530 MB/s Streaming Writes 205 MB/s 240 MB/s Random 4kB Read 15.5 ms 11 µs Random 4kB Write 6.4 ms 23 µs Power 4/6/8 W 0.3/4.2 W Capacity Price 4 TB GB 125 Hitachi 7k4000 Samsung SSD 840 Antoniu Pop Storage Technologies / 20
30 SSD vs. HDD HDD SSD Streaming Reads 205 MB/s 530 MB/s Streaming Writes 205 MB/s 240 MB/s Random 4kB Read 15.5 ms 11 µs Random 4kB Write 6.4 ms 23 µs Power 4/6/8 W 0.3/4.2 W Capacity Price 4 TB GB 125 Price per GB 0.035/GB 0.5/GB Hitachi 7k4000 Samsung SSD 840 Antoniu Pop Storage Technologies / 20
31 Example: disk access time (1 - recap) How long would it take on average to read / write a 512 byte sector on this disk? Disk access time = seek time + search time + transfer time seek time: 8.5 ms search time: the disk must, on average, complete a half rotation sec 0.5 rotations 60 min 7200 RPM = = 4.16 ms 7200 RPM transfer time: 512 B B/sec = 2.89 µs access time = = ms Antoniu Pop Storage Technologies / 20
32 Example: disk access time (2 - recap) How long would it take on average to read / write 512 MB on this disk? (assuming sectors are contiguous ) Disk access time = seek time + search time + transfer time seek time: 8.5 ms search time: the disk must, on average, complete a half rotation sec 0.5 rotations 60 min 7200 RPM = = 4.16 ms 7200 RPM transfer time: B B/sec = 2.89 s access time = = 2.9 s Antoniu Pop Storage Technologies / 20
33 Storage Virtualization File System (classical) FS to HDD partition mapping FS does not span multiple drives File System OS Device Drivers Antoniu Pop Storage Technologies / 20
34 Storage Virtualization File System (classical) FS to HDD partition mapping FS does not span multiple drives File System RAID changes this E.g., Striping or Mirroring OS Device Drivers Antoniu Pop Storage Technologies / 20
35 Storage Virtualization File System (classical) FS to HDD partition mapping FS does not span multiple drives File System RAID changes this E.g., Striping or Mirroring OS Device Drivers Storage Virtualization: break the FS/HDD mapping Antoniu Pop Storage Technologies / 20
36 Logical Volume Management Virtual mapping between file system code and physical device Antoniu Pop Storage Technologies / 20
37 Logical Volume Management Virtual mapping between file system code and physical device Similar (but not identical!) to virtual memory addressing Antoniu Pop Storage Technologies / 20
38 Logical Volume Management Virtual mapping between file system code and physical device Similar (but not identical!) to virtual memory addressing FTSE: one more level of indirection Antoniu Pop Storage Technologies / 20
39 Logical Volume Management Virtual mapping between file system code and physical device Similar (but not identical!) to virtual memory addressing FTSE: one more level of indirection Volume Group : set of drives in a pool Antoniu Pop Storage Technologies / 20
40 Logical Volume Management Virtual mapping between file system code and physical device Similar (but not identical!) to virtual memory addressing FTSE: one more level of indirection Volume Group : set of drives in a pool Storage space in Volume Group divided into Physical Extents usually all same size Antoniu Pop Storage Technologies / 20
41 Logical Volume Management Virtual mapping between file system code and physical device Similar (but not identical!) to virtual memory addressing FTSE: one more level of indirection Volume Group : set of drives in a pool Storage space in Volume Group divided into Physical Extents usually all same size Logical Volume is a set of Physical Extents Antoniu Pop Storage Technologies / 20
42 Logical Volume Management Mirror/Stripe/RAID within the LVM layer File System LVM layer Device Drivers Antoniu Pop Storage Technologies / 20
43 Logical Volume Management Mirror/Stripe/RAID within the LVM layer Resize the File System add physical extents extend the partition File System LVM layer Device Drivers Antoniu Pop Storage Technologies / 20
44 Logical Volume Management Mirror/Stripe/RAID within the LVM layer Resize the File System add physical extents extend the partition File System LVM layer Device Drivers Snapshot a live filesystem Antoniu Pop Storage Technologies / 20
45 Example: the Linux File System Antoniu Pop Storage Technologies / 20
46 Example: the Linux File System / mostly read: want fast seeks, high read transfer rates Antoniu Pop Storage Technologies / 20
47 Example: the Linux File System / mostly read: want fast seeks, high read transfer rates swap read / write: want high bandwidth, data loss (?) Antoniu Pop Storage Technologies / 20
48 Example: the Linux File System / mostly read: want fast seeks, high read transfer rates swap read / write: want high bandwidth, data loss (?) /opt infrequent access Antoniu Pop Storage Technologies / 20
49 Example: the Linux File System / mostly read: want fast seeks, high read transfer rates swap read / write: want high bandwidth, data loss (?) /opt infrequent access /var huge, infrequent access Antoniu Pop Storage Technologies / 20
50 Example: the Linux File System / mostly read: want fast seeks, high read transfer rates swap read / write: want high bandwidth, data loss (?) /opt infrequent access /var huge, infrequent access Mirror / Antoniu Pop Storage Technologies / 20
51 Example: the Linux File System / mostly read: want fast seeks, high read transfer rates swap read / write: want high bandwidth, data loss (?) /opt infrequent access /var huge, infrequent access Mirror / Stripe swap Antoniu Pop Storage Technologies / 20
52 Example: the Linux File System / mostly read: want fast seeks, high read transfer rates swap read / write: want high bandwidth, data loss (?) /opt infrequent access /var huge, infrequent access Mirror / Stripe swap spare space to /opt and /var Antoniu Pop Storage Technologies / 20
53 LVM Example Efficient / flexible resource management Antoniu Pop Storage Technologies / 20
54 LVM Example Efficient / flexible resource management / is mirrored across two disks Antoniu Pop Storage Technologies / 20
55 LVM Example Efficient / flexible resource management / is mirrored across two disks swap is striped across two other disks Antoniu Pop Storage Technologies / 20
56 LVM Example Efficient / flexible resource management / is mirrored across two disks swap is striped across two other disks /opt uses space on one disk Antoniu Pop Storage Technologies / 20
57 LVM Example Efficient / flexible resource management / is mirrored across two disks swap is striped across two other disks /opt uses space on one disk /var takes the remaining space across other 3 disks Antoniu Pop Storage Technologies / 20
58 Storage Area Networks Implement LVM features in a separate storage controller Antoniu Pop Storage Technologies / 20
59 Storage Area Networks Implement LVM features in a separate storage controller Connect multiple servers to storage controller via SCSI, or FibreChannel, or Infiniband, or... Antoniu Pop Storage Technologies / 20
60 Storage Area Networks Implement LVM features in a separate storage controller Connect multiple servers to storage controller via SCSI, or FibreChannel, or Infiniband, or... SAN over Ethernet, aka Networked Attached Storage (NAS) Antoniu Pop Storage Technologies / 20
61 Storage Area Networks Implement LVM features in a separate storage controller Connect multiple servers to storage controller via SCSI, or FibreChannel, or Infiniband, or... SAN over Ethernet, aka Networked Attached Storage (NAS) Share disk resources across multiple servers Antoniu Pop Storage Technologies / 20
62 Storage Area Networks Implement LVM features in a separate storage controller Connect multiple servers to storage controller via SCSI, or FibreChannel, or Infiniband, or... SAN over Ethernet, aka Networked Attached Storage (NAS) Share disk resources across multiple servers Rapid migration of disk images Antoniu Pop Storage Technologies / 20
63 SAN Controller SAN Controller "Blade servers" Antoniu Pop Storage Technologies / 20
64 SAN Controller SAN Controller "Blade servers" Decouple compute servers from storage servers. Antoniu Pop Storage Technologies / 20
65 SAN Controller SAN Controller "Blade servers" Decouple compute servers from storage servers. Connect through network: Antoniu Pop Storage Technologies / 20
66 SAN Controller SAN Controller "Blade servers" Decouple compute servers from storage servers. Connect through network: Bandwidth? Antoniu Pop Storage Technologies / 20
67 SAN Controller SAN Controller "Blade servers" Decouple compute servers from storage servers. Connect through network: Bandwidth? Latency? Antoniu Pop Storage Technologies / 20
68 SAN Key Features Functionality Key element of System Virtualization Antoniu Pop Storage Technologies / 20
69 SAN Key Features Functionality Key element of System Virtualization Migrating virtual machines Antoniu Pop Storage Technologies / 20
70 SAN Key Features Functionality Key element of System Virtualization Migrating virtual machines De-duping share common subsets of file systems (think Virtual Machine images!) Antoniu Pop Storage Technologies / 20
71 SAN Key Features Functionality Key element of System Virtualization Migrating virtual machines De-duping share common subsets of file systems (think Virtual Machine images!) Management: Manage storage separately from server physical resources Maximize flexibility of storage provisioning Antoniu Pop Storage Technologies / 20
72 ZFS Volume Aware File System Marketing claims: Antoniu Pop Storage Technologies / 20
73 ZFS Volume Aware File System Marketing claims: Lost a file? Antoniu Pop Storage Technologies / 20
74 ZFS Volume Aware File System Marketing claims: Lost a file? Run out of space? Antoniu Pop Storage Technologies / 20
75 ZFS Volume Aware File System Marketing claims: Lost a file? Run out of space? Difficult disk upgrade? Antoniu Pop Storage Technologies / 20
76 ZFS Volume Aware File System Marketing claims: Lost a file? Run out of space? Difficult disk upgrade? Want to grow/shrink? Antoniu Pop Storage Technologies / 20
77 ZFS Volume Aware File System Marketing claims: Lost a file? Run out of space? Difficult disk upgrade? Want to grow/shrink? Data Corruption? Antoniu Pop Storage Technologies / 20
78 ZFS Solutions Lost a file? Antoniu Pop Storage Technologies / 20
79 ZFS Solutions Lost a file? Copy-on-Write (CoW) Antoniu Pop Storage Technologies / 20
80 ZFS Solutions Lost a file? Copy-on-Write (CoW) simple rollback/recovery Antoniu Pop Storage Technologies / 20
81 ZFS Solutions Lost a file? Copy-on-Write (CoW) simple rollback/recovery (indirect wear-leveling) Antoniu Pop Storage Technologies / 20
82 ZFS Solutions Lost a file? Copy-on-Write (CoW) simple rollback/recovery (indirect wear-leveling) Run out of space / difficult disk upgrade? Antoniu Pop Storage Technologies / 20
83 ZFS Solutions Lost a file? Copy-on-Write (CoW) simple rollback/recovery (indirect wear-leveling) Run out of space / difficult disk upgrade? Add new storage to live systems Self-checking, self-healing Antoniu Pop Storage Technologies / 20
84 ZFS Solutions Lost a file? Copy-on-Write (CoW) simple rollback/recovery (indirect wear-leveling) Run out of space / difficult disk upgrade? Add new storage to live systems Self-checking, self-healing Want to grow / shrink? Antoniu Pop Storage Technologies / 20
85 ZFS Solutions Lost a file? Copy-on-Write (CoW) simple rollback/recovery (indirect wear-leveling) Run out of space / difficult disk upgrade? Add new storage to live systems Self-checking, self-healing Want to grow / shrink? Data Corruption? end-to-end sumchecking Antoniu Pop Storage Technologies / 20
86 ZFS Solutions Lost a file? Copy-on-Write (CoW) simple rollback/recovery (indirect wear-leveling) Run out of space / difficult disk upgrade? Add new storage to live systems Self-checking, self-healing Want to grow / shrink? Data Corruption? end-to-end sumchecking ZFS combines File System and Logical Volume Management Antoniu Pop Storage Technologies / 20
CSCI-GA Database Systems Lecture 8: Physical Schema: Storage
CSCI-GA.2433-001 Database Systems Lecture 8: Physical Schema: Storage Mohamed Zahran (aka Z) mzahran@cs.nyu.edu http://www.mzahran.com View 1 View 2 View 3 Conceptual Schema Physical Schema 1. Create a
COMP283-Lecture 3 Applied Database Management
COMP283-Lecture 3 Applied Database Management Introduction DB Design Continued Disk Sizing Disk Types & Controllers DB Capacity 1 COMP283-Lecture 3 DB Storage: Linear Growth Disk space requirements increases
Chapter 10: Mass-Storage Systems. Operating System Concepts 9 th Edition
Chapter 10: Mass-Storage Systems Silberschatz, Galvin and Gagne 2013 Chapter 10: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management Swap-Space
Mass-Storage Structure
Operating Systems (Fall/Winter 2018) Mass-Storage Structure Yajin Zhou (http://yajin.org) Zhejiang University Acknowledgement: some pages are based on the slides from Zhi Wang(fsu). Review On-disk structure
Secondary storage. CS 537 Lecture 11 Secondary Storage. Disk trends. Another trip down memory lane
Secondary storage CS 537 Lecture 11 Secondary Storage Michael Swift Secondary storage typically: is anything that is outside of primary memory does not permit direct execution of instructions or data retrieval
Chapter 10: Mass-Storage Systems
Chapter 10: Mass-Storage Systems Silberschatz, Galvin and Gagne 2013 Chapter 10: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management Swap-Space
CS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Fall 2016 Lecture 35 Mass Storage Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 Questions For You Local/Global
Lecture 29. Friday, March 23 CS 470 Operating Systems - Lecture 29 1
Lecture 29 Reminder: Homework 7 is due on Monday at class time for Exam 2 review; no late work accepted. Reminder: Exam 2 is on Wednesday. Exam 2 review sheet is posted. Questions? Friday, March 23 CS
Chapter 12: Mass-Storage
Chapter 12: Mass-Storage Systems Chapter 12: Mass-Storage Systems Revised 2010. Tao Yang Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management Swap-Space Management
Disks and RAID. CS 4410 Operating Systems. [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse]
![Disks and RAID. CS 4410 Operating Systems. [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse]](/thumbs/93/111187825.jpg "Disks and RAID. CS 4410 Operating Systems. [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse]")
Disks and RAID CS 4410 Operating Systems [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse] Storage Devices Magnetic disks Storage that rarely becomes corrupted Large capacity at low cost Block
Virtual Memory. Reading. Sections 5.4, 5.5, 5.6, 5.8, 5.10 (2) Lecture notes from MKP and S. Yalamanchili
Virtual Memory Lecture notes from MKP and S. Yalamanchili Sections 5.4, 5.5, 5.6, 5.8, 5.10 Reading (2) 1 The Memory Hierarchy ALU registers Cache Memory Memory Memory Managed by the compiler Memory Managed
CS3600 SYSTEMS AND NETWORKS
CS3600 SYSTEMS AND NETWORKS NORTHEASTERN UNIVERSITY Lecture 9: Mass Storage Structure Prof. Alan Mislove (amislove@ccs.neu.edu) Moving-head Disk Mechanism 2 Overview of Mass Storage Structure Magnetic
Frequently asked questions from the previous class survey
CS 370: OPERATING SYSTEMS [MASS STORAGE] Shrideep Pallickara Computer Science Colorado State University L29.1 Frequently asked questions from the previous class survey How does NTFS compare with UFS? L29.2
Solid State Drives (SSDs) Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
Solid State Drives (SSDs) Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Memory Types FLASH High-density Low-cost High-speed Low-power High reliability
CISC 7310X. C11: Mass Storage. Hui Chen Department of Computer & Information Science CUNY Brooklyn College. 4/19/2018 CUNY Brooklyn College
CISC 7310X C11: Mass Storage Hui Chen Department of Computer & Information Science CUNY Brooklyn College 4/19/2018 CUNY Brooklyn College 1 Outline Review of memory hierarchy Mass storage devices Reliability
Chapter 10: Mass-Storage Systems
COP 4610: Introduction to Operating Systems (Spring 2016) Chapter 10: Mass-Storage Systems Zhi Wang Florida State University Content Overview of Mass Storage Structure Disk Structure Disk Scheduling Disk
COS 318: Operating Systems. Storage Devices. Vivek Pai Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Vivek Pai Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall11/cos318/ Today s Topics Magnetic disks Magnetic disk
Che-Wei Chang Department of Computer Science and Information Engineering, Chang Gung University
Che-Wei Chang chewei@mail.cgu.edu.tw Department of Computer Science and Information Engineering, Chang Gung University l Chapter 10: File System l Chapter 11: Implementing File-Systems l Chapter 12: Mass-Storage
Storage. CS 3410 Computer System Organization & Programming
Storage CS 3410 Computer System Organization & Programming These slides are the product of many rounds of teaching CS 3410 by Deniz Altinbuke, Kevin Walsh, and Professors Weatherspoon, Bala, Bracy, and
u Covered: l Management of CPU & concurrency l Management of main memory & virtual memory u Currently --- Management of I/O devices
Where Are We? COS 318: Operating Systems Storage Devices Jaswinder Pal Singh Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) u Covered: l Management of CPU
Chapter 6. Storage and Other I/O Topics
Chapter 6 Storage and Other I/O Topics Introduction I/O devices can be characterized by Behaviour: input, output, storage Partner: human or machine Data rate: bytes/sec, transfers/sec I/O bus connections
CSE 120. Operating Systems. March 27, 2014 Lecture 17. Mass Storage. Instructor: Neil Rhodes. Wednesday, March 26, 14
CSE 120 Operating Systems March 27, 2014 Lecture 17 Mass Storage Instructor: Neil Rhodes Paging and Translation Lookaside Buffer frame dirty? no yes CPU checks TLB PTE in TLB? Free page frame? no yes OS
Ricardo Rocha. Department of Computer Science Faculty of Sciences University of Porto
Ricardo Rocha Department of Computer Science Faculty of Sciences University of Porto Slides based on the book Operating System Concepts, 9th Edition, Abraham Silberschatz, Peter B. Galvin and Greg Gagne,
COS 318: Operating Systems. Storage Devices. Jaswinder Pal Singh Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Jaswinder Pal Singh Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall13/cos318/ Today s Topics Magnetic disks
CS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Spring 2018 Lecture 24 Mass Storage, HDFS/Hadoop Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 FAQ What 2
Computer Architecture 计算机体系结构. Lecture 6. Data Storage and I/O 第六讲 数据存储和输入输出. Chao Li, PhD. 李超博士
Computer Architecture 计算机体系结构 Lecture 6. Data Storage and I/O 第六讲 数据存储和输入输出 Chao Li, PhD. 李超博士 SJTU-SE346, Spring 2018 Review Memory hierarchy Cache and virtual memory Locality principle Miss cache, victim
Wednesday, April 25, Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes
Wednesday, April 25, 2018 Topics for today Secondary memory Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes Storage management (Chapter
Mass-Storage. ICS332 - Fall 2017 Operating Systems. Henri Casanova
Mass-Storage ICS332 - Fall 2017 Operating Systems Henri Casanova (henric@hawaii.edu) Magnetic Disks! Magnetic disks (a.k.a. hard drives ) are (still) the most common secondary storage devices today! They
Mass-Storage. ICS332 Operating Systems
Mass-Storage ICS332 Operating Systems Magnetic Disks Magnetic disks are (still) the most common secondary storage devices today They are messy Errors, bad blocks, missed seeks, moving parts And yet, the
Module 1: Basics and Background Lecture 4: Memory and Disk Accesses. The Lecture Contains: Memory organisation. Memory hierarchy. Disks.
The Lecture Contains: Memory organisation Example of memory hierarchy Memory hierarchy Disks Disk access Disk capacity Disk access time Typical disk parameters Access times file:///c /Documents%20and%20Settings/iitkrana1/My%20Documents/Google%20Talk%20Received%20Files/ist_data/lecture4/4_1.htm[6/14/2012
Monday, May 4, Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes
Monday, May 4, 2015 Topics for today Secondary memory Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes Storage management (Chapter
Lecture 16: Storage Devices
CS 422/522 Design & Implementation of Operating Systems Lecture 16: Storage Devices Zhong Shao Dept. of Computer Science Yale University Acknowledgement: some slides are taken from previous versions of
Advanced Database Systems
Lecture II Storage Layer Kyumars Sheykh Esmaili Course s Syllabus Core Topics Storage Layer Query Processing and Optimization Transaction Management and Recovery Advanced Topics Cloud Computing and Web
EI 338: Computer Systems Engineering (Operating Systems & Computer Architecture)
EI 338: Computer Systems Engineering (Operating Systems & Computer Architecture) Dept. of Computer Science & Engineering Chentao Wu wuct@cs.sjtu.edu.cn Download lectures ftp://public.sjtu.edu.cn User:
V. Mass Storage Systems
TDIU25: Operating Systems V. Mass Storage Systems SGG9: chapter 12 o Mass storage: Hard disks, structure, scheduling, RAID Copyright Notice: The lecture notes are mainly based on modifications of the slides
CSE 451: Operating Systems Spring Module 12 Secondary Storage. Steve Gribble
CSE 451: Operating Systems Spring 2009 Module 12 Secondary Storage Steve Gribble Secondary storage Secondary storage typically: is anything that is outside of primary memory does not permit direct execution
CS143: Disks and Files
CS143: Disks and Files 1 System Architecture CPU Word (1B 64B) ~ x GB/sec Main Memory System Bus Disk Controller... Block (512B 50KB) ~ x MB/sec Disk 2 Magnetic disk vs SSD Magnetic Disk Stores data on
Mass-Storage Systems. Mass-Storage Systems. Disk Attachment. Disk Attachment
TDIU11 Operating systems Mass-Storage Systems [SGG7/8/9] Chapter 12 Copyright Notice: The lecture notes are mainly based on Silberschatz s, Galvin s and Gagne s book ( Operating System Copyright Concepts,
Operating Systems. Operating Systems Professor Sina Meraji U of T
Operating Systems Operating Systems Professor Sina Meraji U of T How are file systems implemented? File system implementation Files and directories live on secondary storage Anything outside of primary
16:30 18:00, June 20 (Monday), 2011 # (even student IDs) # (odd student IDs) Scope
Final Exam 16:30 18:00, June 20 (Monday), 2011 #440102 (even student IDs) #440112 (odd student IDs) Scope Chap. 1 5 (except 3.7, 5.7) Chap. 6.1, 6.3, 6.4 Chap. 7.1 7.6 Closed-book exam 1 Storage Jin-Soo
COS 318: Operating Systems. Storage Devices. Kai Li Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Kai Li Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall11/cos318/ Today s Topics Magnetic disks Magnetic disk
Database Architecture 2 & Storage. Instructor: Matei Zaharia cs245.stanford.edu
Database Architecture 2 & Storage Instructor: Matei Zaharia cs245.stanford.edu Summary from Last Time System R mostly matched the architecture of a modern RDBMS» SQL» Many storage & access methods» Cost-based
CSE 451: Operating Systems Spring Module 12 Secondary Storage
CSE 451: Operating Systems Spring 2017 Module 12 Secondary Storage John Zahorjan 1 Secondary storage Secondary storage typically: is anything that is outside of primary memory does not permit direct execution
CS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Spring 2019 L22 File-system Implementation Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 FAQ When is contiguous
Ricardo Rocha. Department of Computer Science Faculty of Sciences University of Porto
Ricardo Rocha Department of Computer Science Faculty of Sciences University of Porto Slides based on the book Operating System Concepts, 9th Edition, Abraham Silberschatz, Peter B. Galvin and Greg Gagne,
CS370: Operating Systems [Spring 2017] Dept. Of Computer Science, Colorado State University
![CS370: Operating Systems [Spring 2017] Dept. Of Computer Science, Colorado State University](/thumbs/89/98522481.jpg "CS370: Operating Systems [Spring 2017] Dept. Of Computer Science, Colorado State University")
Frequently asked questions from the previous class survey CS 370: OPERATING SYSTEMS [MASS STORAGE] How does the OS caching optimize disk performance? How does file compression work? Does the disk change
Storage Profiles. Storage Profiles. Storage Profiles, page 12
, page 1 Disk Groups and Disk Group Configuration Policies, page 2 RAID Levels, page 6 Automatic Disk Selection, page 7 Supported LUN Modifications, page 8 Unsupported LUN Modifications, page 8 Disk Insertion
CSE 153 Design of Operating Systems
CSE 153 Design of Operating Systems Winter 2018 Lecture 20: File Systems (1) Disk drives OS Abstractions Applications Process File system Virtual memory Operating System CPU Hardware Disk RAM CSE 153 Lecture
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Computer Architecture ECE 568
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Computer Architecture ECE 568 Part 6 Input/Output Israel Koren ECE568/Koren Part.6. Motivation: Why Care About I/O? CPU Performance:
Operating Systems. Mass-Storage Structure Based on Ch of OS Concepts by SGG
Operating Systems Mass-Storage Structure Based on Ch. 10-11 of OS Concepts by SGG Mass Storage Devices Disks A r/w head flies above the spinning disk Each disk is divided into tracks and each track in
CSE 451: Operating Systems Winter Secondary Storage. Steve Gribble. Secondary storage
CSE 451: Operating Systems Winter 2005 Secondary Storage Steve Gribble Secondary storage Secondary storage typically: is anything that is outside of primary memory does not permit direct execution of instructions
Storage Systems. Storage Systems
Storage Systems Storage Systems We already know about four levels of storage: Registers Cache Memory Disk But we've been a little vague on how these devices are interconnected In this unit, we study Input/output
Hard Disk Drives. Nima Honarmand (Based on slides by Prof. Andrea Arpaci-Dusseau)
Hard Disk Drives Nima Honarmand (Based on slides by Prof. Andrea Arpaci-Dusseau) Storage Stack in the OS Application Virtual file system Concrete file system Generic block layer Driver Disk drive Build
IST346. Data Storage
IST346 Data Storage Data Storage Why Data Storage? Information is a the center of all organizations. Organizations need to store data. Lots of it. What Kinds of Data? Documents and Files (Reports, Proposals,
Introduction to I/O and Disk Management
1 Secondary Storage Management Disks just like memory, only different Introduction to I/O and Disk Management Why have disks? Ø Memory is small. Disks are large. Short term storage for memory contents
Introduction to I/O and Disk Management
Introduction to I/O and Disk Management 1 Secondary Storage Management Disks just like memory, only different Why have disks? Ø Memory is small. Disks are large. Short term storage for memory contents
Quiz for Chapter 6 Storage and Other I/O Topics 3.10
Date: 3.10 Not all questions are of equal difficulty. Please review the entire quiz first and then budget your time carefully. Name: Course: 1. [6 points] Give a concise answer to each of the following
Mass-Storage Systems
CS307 Mass-Storage Systems Fan Wu Department of Computer Science and Engineering Shanghai Jiao Tong University Spring 2018 The First Commercial Disk Drive 1956 IBM RAMDAC computer included the IBM Model
Main Points. File systems. Storage hardware characteristics. File system usage patterns. Useful abstractions on top of physical devices
Storage Systems Main Points File systems Useful abstractions on top of physical devices Storage hardware characteristics Disks and flash memory File system usage patterns File Systems Abstraction on top
CHAPTER 12: MASS-STORAGE SYSTEMS (A) By I-Chen Lin Textbook: Operating System Concepts 9th Ed.
CHAPTER 12: MASS-STORAGE SYSTEMS (A) By I-Chen Lin Textbook: Operating System Concepts 9th Ed. Chapter 12: Mass-Storage Systems Overview of Mass-Storage Structure Disk Structure Disk Attachment Disk Scheduling
CS24: INTRODUCTION TO COMPUTING SYSTEMS. Spring 2017 Lecture 13
CS24: INTRODUCTION TO COMPUTING SYSTEMS Spring 2017 Lecture 13 COMPUTER MEMORY So far, have viewed computer memory in a very simple way Two memory areas in our computer: The register file Small number
Chapter 12: Mass-Storage
hapter 12: Mass-Storage Systems hapter 12: Mass-Storage Systems To explain the performance characteristics of mass-storage devices To evaluate disk scheduling algorithms To discuss operating-system services
CS252 S05. CMSC 411 Computer Systems Architecture Lecture 18 Storage Systems 2. I/O performance measures. I/O performance measures
CMSC 411 Computer Systems Architecture Lecture 18 Storage Systems 2 I/O performance measures I/O performance measures diversity: which I/O devices can connect to the system? capacity: how many I/O devices
Chapter 12: Mass-Storage
hapter 12: Mass-Storage Systems hapter 12: Mass-Storage Systems Overview of Mass Storage Structure Disk Structure Disk Attachment Disk Scheduling Disk Management RAID Structure Objectives Moving-head Disk
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Computer Architecture ECE 568
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Computer Architecture ECE 568 Part 6 Input/Output Israel Koren ECE568/Koren Part.6. CPU performance keeps increasing 26 72-core Xeon
File. File System Implementation. File Metadata. File System Implementation. Direct Memory Access Cont. Hardware background: Direct Memory Access
File File System Implementation Operating Systems Hebrew University Spring 2009 Sequence of bytes, with no structure as far as the operating system is concerned. The only operations are to read and write
Hard Disk Drives (HDDs)
Hard Disk Drives (HDDs) Jinkyu Jeong (jinkyu@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu EEE3052: Introduction to Operating Systems, Fall 2017, Jinkyu Jeong (jinkyu@skku.edu)
CSE 451: Operating Systems Winter Lecture 12 Secondary Storage. Steve Gribble 323B Sieg Hall.
CSE 451: Operating Systems Winter 2001 Lecture 12 Steve Gribble gribble@cs.washington.edu 323B Sieg Hall Secondary storage typically: is anything that is outside of primary memory does not permit direct
Operating Systems. Lecture File system implementation. Master of Computer Science PUF - Hồ Chí Minh 2016/2017
Operating Systems Lecture 7.2 - File system implementation Adrien Krähenbühl Master of Computer Science PUF - Hồ Chí Minh 2016/2017 Design FAT or indexed allocation? UFS, FFS & Ext2 Journaling with Ext3
BBM371- Data Management. Lecture 2: Storage Devices
BBM371- Data Management Lecture 2: Storage Devices 18.10.2018 Memory Hierarchy cache Main memory disk Optical storage Tapes V NV Traveling the hierarchy: 1. speed ( higher=faster) 2. cost (lower=cheaper)
Today: Secondary Storage! Typical Disk Parameters!
Today: Secondary Storage! To read or write a disk block: Seek: (latency) position head over a track/cylinder. The seek time depends on how fast the hardware moves the arm. Rotational delay: (latency) time
Key Points. Rotational delay vs seek delay Disks are slow. Techniques for making disks faster. Flash and SSDs
IO 1 Today IO 2 Key Points CPU interface and interaction with IO IO devices The basic structure of the IO system (north bridge, south bridge, etc.) The key advantages of high speed serial lines. The benefits
Storage. Hwansoo Han
Storage Hwansoo Han I/O Devices I/O devices can be characterized by Behavior: input, out, storage Partner: human or machine Data rate: bytes/sec, transfers/sec I/O bus connections 2 I/O System Characteristics
Administrivia. CMSC 411 Computer Systems Architecture Lecture 19 Storage Systems, cont. Disks (cont.) Disks - review
Administrivia CMSC 411 Computer Systems Architecture Lecture 19 Storage Systems, cont. Homework #4 due Thursday answers posted soon after Exam #2 on Thursday, April 24 on memory hierarchy (Unit 4) and
SSD (Solid State Disk)
SSD (Solid State Disk) http://en.wikipedia.org/wiki/solid-state_drive SSD (Solid State Disk) drives Most SSD drives gives very good performance 4x ~ 100x No noise, low weight, power and heat generation
I/O CANNOT BE IGNORED
LECTURE 13 I/O I/O CANNOT BE IGNORED Assume a program requires 100 seconds, 90 seconds for main memory, 10 seconds for I/O. Assume main memory access improves by ~10% per year and I/O remains the same.
NAND Flash-based Storage. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
NAND Flash-based Storage Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics NAND flash memory Flash Translation Layer (FTL) OS implications
CS370: Operating Systems [Fall 2018] Dept. Of Computer Science, Colorado State University
![CS370: Operating Systems [Fall 2018] Dept. Of Computer Science, Colorado State University](/thumbs/91/106691121.jpg "CS370: Operating Systems [Fall 2018] Dept. Of Computer Science, Colorado State University")
CS 370: OPERATING SYSTEMS [DISK SCHEDULING ALGORITHMS] Shrideep Pallickara Computer Science Colorado State University L30.1 Frequently asked questions from the previous class survey ECCs: How does it impact
4th Slide Set Operating Systems
Prof. Dr. Christian Baun 4th Slide Set Operating Systems Frankfurt University of Applied Sciences SS2016 1/37 4th Slide Set Operating Systems Prof. Dr. Christian Baun Frankfurt University of Applied Sciences
Hard Disk Drives (HDDs) Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
Hard Disk Drives (HDDs) Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Virtualization Virtual CPUs Virtual memory Concurrency Threads Synchronization
NAND Flash-based Storage. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
NAND Flash-based Storage Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics NAND flash memory Flash Translation Layer (FTL) OS implications
Disk Scheduling COMPSCI 386
Disk Scheduling COMPSCI 386 Topics Disk Structure (9.1 9.2) Disk Scheduling (9.4) Allocation Methods (11.4) Free Space Management (11.5) Hard Disk Platter diameter ranges from 1.8 to 3.5 inches. Both sides
CS370: Operating Systems [Fall 2018] Dept. Of Computer Science, Colorado State University
![CS370: Operating Systems [Fall 2018] Dept. Of Computer Science, Colorado State University](/thumbs/91/106691152.jpg "CS370: Operating Systems [Fall 2018] Dept. Of Computer Science, Colorado State University")
Frequently asked questions from the previous class survey CS 370: OPERATING SYSTEMS [DISK SCHEDULING ALGORITHMS] Shrideep Pallickara Computer Science Colorado State University ECCs: How does it impact
Disks, Memories & Buffer Management
Disks, Memories & Buffer Management The two offices of memory are collection and distribution. - Samuel Johnson CS3223 - Storage 1 What does a DBMS Store? Relations Actual data Indexes Data structures
Specifying Storage Servers for IP security applications
Specifying Storage Servers for IP security applications The migration of security systems from analogue to digital IP based solutions has created a large demand for storage servers high performance PCs
Main Points. File systems. Storage hardware characteris7cs. File system usage Useful abstrac7ons on top of physical devices
Storage Systems Main Points File systems Useful abstrac7ons on top of physical devices Storage hardware characteris7cs Disks and flash memory File system usage pa@erns File Systems Abstrac7on on top of
Main Points. File systems. Storage hardware characteris7cs. File system usage Useful abstrac7ons on top of physical devices
Storage Systems Main Points File systems Useful abstrac7ons on top of physical devices Storage hardware characteris7cs Disks and flash memory File system usage pa@erns File System Abstrac7on File system
Choosing Hardware and Operating Systems for MySQL. Apr 15, 2009 O'Reilly MySQL Conference and Expo Santa Clara,CA by Peter Zaitsev, Percona Inc
Choosing Hardware and Operating Systems for MySQL Apr 15, 2009 O'Reilly MySQL Conference and Expo Santa Clara,CA by Peter Zaitsev, Percona Inc -2- We will speak about Choosing Hardware Choosing Operating
Principles of Data Management. Lecture #2 (Storing Data: Disks and Files)
Principles of Data Management Lecture #2 (Storing Data: Disks and Files) Instructor: Mike Carey mjcarey@ics.uci.edu Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Topics v Today
HPC File Systems and Storage. Irena Johnson University of Notre Dame Center for Research Computing
HPC File Systems and Storage Irena Johnson University of Notre Dame Center for Research Computing HPC (High Performance Computing) Aggregating computer power for higher performance than that of a typical
High Performance Computing Course Notes High Performance Storage
High Performance Computing Course Notes 2008-2009 2009 High Performance Storage Storage devices Primary storage: register (1 CPU cycle, a few ns) Cache (10-200 cycles, 0.02-0.5us) Main memory Local main
Disks. Storage Technology. Vera Goebel Thomas Plagemann. Department of Informatics University of Oslo
Disks Vera Goebel Thomas Plagemann 2014 Department of Informatics University of Oslo Storage Technology [Source: http://www-03.ibm.com/ibm/history/exhibits/storage/storage_photo.html] 1 Filesystems & Disks
NAND Flash-based Storage. Computer Systems Laboratory Sungkyunkwan University
NAND Flash-based Storage Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics NAND flash memory Flash Translation Layer (FTL) OS implications
SSD Architecture Considerations for a Spectrum of Enterprise Applications. Alan Fitzgerald, VP and CTO SMART Modular Technologies
SSD Architecture Considerations for a Spectrum of Enterprise Applications Alan Fitzgerald, VP and CTO SMART Modular Technologies Introduction Today s SSD delivers form-fit-function compatible solid-state
Appendix D: Storage Systems
Appendix D: Storage Systems Instructor: Josep Torrellas CS433 Copyright Josep Torrellas 1999, 2001, 2002, 2013 1 Storage Systems : Disks Used for long term storage of files temporarily store parts of pgm
Computer Science 61C Spring Friedland and Weaver. Input/Output
Input/Output 1 A Computer is Useless without I/O I/O handles persistent storage Disks, SSD memory, etc I/O handles user interfaces Keyboard/mouse/display I/O handles network 2 Basic I/O: Devices are Memory
NAND Flash-based Storage. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
NAND Flash-based Storage Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics NAND flash memory Flash Translation Layer (FTL) OS implications
UCS Invicta: A New Generation of Storage Performance. Mazen Abou Najm DC Consulting Systems Engineer
UCS Invicta: A New Generation of Storage Performance Mazen Abou Najm DC Consulting Systems Engineer HDDs Aren t Designed For High Performance Disk 101 Can t spin faster (200 IOPS/Drive) Can t seek faster
Chapter 6 - External Memory
Chapter 6 - External Memory Luis Tarrataca luis.tarrataca@gmail.com CEFET-RJ L. Tarrataca Chapter 6 - External Memory 1 / 66 Table of Contents I 1 Motivation 2 Magnetic Disks Write Mechanism Read Mechanism
I/O & Storage. Jin-Soo Kim ( Computer Systems Laboratory Sungkyunkwan University
I/O & Storage Jin-Soo Kim ( jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics I/O systems Device characteristics: block vs. character I/O systems