SMR in Linux Systems. Seagate's Contribution to Legacy File Systems. Adrian Palmer, Drive Development Engineering
|
|
- Daisy Cooper
- 6 years ago
- Views:
Transcription
1 SMR in Linux Systems Seagate's Contribution to Legacy File Systems Adrian Palmer, Drive Development Engineering
2 SEAGATE combines DIFFERENT TECHNOLOGIES in new ways to SOLVE customer data storage CHALLENGES 2
3 Shingled Magnetic Recording (SMR) Areal density growth curve lowest cost/gb 10 TB/in^2 HAMR with SMR BPMR with SMR HDMR with BPMR 1 PMR SMR SMR (Shingle Magnetic Recording) HAMR (Heat Assisted Magnetic Recording) BPMR (Bit Patterned Magnetic Recording) 0.1 PMR (Perpendicular Magnetic Recording) Data adapted from ASTC Technology Roadmap ( 3
4 Shingled Magnetic Recording (SMR) Forward-write only: Radial AND Rotational Image: Wood, R.; Williams, Mason; Kavcic, A.; Miles, Jim, "The Feasibility of Magnetic Recording at 10 Terabits Per Square Inch on Conventional Media," Magnetics, IEEE Transactions on, vol.45, no.2, pp.917,923, Feb
5 SMR Drive Types Drive Managed (DM) Mimics Traditional drives Backwards compatible Direct Replacement for conventional drives in conventional apps Host Managed (HM) Not backwards compatible Host required to manage data ordering for performance mitigation Extensions required in ATA and SCSI command sets Host Aware (HA) Combination of DM and HM. Backwards compatible / Able to use extensions in ATA and SCSI
6 Drive Managed Compatibility for Today Host Aware Performance for Tomorrow Drive Managed Host Managed Host Aware No change: regular SD drive Requires new device & FS Regular SD drive FS benefits from knowledge of media layout Host Aware: Capacity gains like Drive Managed Performance like Conventional 6
7 SMR Can we avoid it? Benefits Provides continued growth in Areal Density. Enables lower cost/gb disc drives Base of new technologies HAMR Support Readiness ZBC/ZAC specifications are nearing completion T10/T13 committees work actively progressing Availability Millions of DM drives shipped! Seagate s 8TB Archive HDD v2 drive is SMR DM in production, HA forthcoming ZBC: Zoned Block Commands ZAC: Zoned device ATA Commands
8 ZAC/ZBC Standards Inspired by SMR. Applicable to any media Separates media into bounded zones Write Pointer Zones Sequential write only for Host Managed (restrictive) Sequential write preferred for Host Aware (permissive) New common ATA/SCSI commands REPORT ZONES RESET WRITE POINTER OPEN/CLOSE/FINISH ZONE Requires communication with FS beyond simple Read/Write 8
9 SMR Friendly File System Requirements Forward-write only CoW requirement Zone aware (ZAC/ZBC) Boundaries New Commands REPORT_ZONES RESET_WRITE_POINTER OPEN/CLOSE/FINISH ZONE New algorithms Defragmentation Goals Optimally Directed Writes efficient streams Excellent Reads streaming File defragmentation Metadata handling Backwards compatibility Provide reference design for other file systems 9
10 SMRFS -EXT4 Default FS of many distros Build upon strengths Popular FS Stability Market Acceptance Compatible on-disk format Popular in commercial storage applications Minimal architecture changes; only rearrangement of existing data 10
11 Proposed Stages EXT4 SMRFFS Project Scope Steps v7 Enforce Host Managed v5 v4 Garbage Collection Kernel Integration Utilities v6 1. cmd line arguments v1 2. internal handling changes v2 3. kernel stack changes v3 v2 Internal changes v1 v2 Specify mkfs Options for HA v2 EXT4 SCSI Disk v2 Kernel Stack v3 4. IOCTL integrations v4 5. Algorithm enhancements v5 6. Utility updates v6 7. Host Managed Compliance v7 12
12 Proposed Stack Changes V3 - Kernel Stack Changes, V4 Kernel Integration VFS VFS EXT4 Examine/Enforce I/O Ordering O_Direct IOCTL Page Cache I/O Scheduler SCSI S-A-T ATA AHCI Disk 13
13 State of project We ve done Laid out design, made prototypes Discussion at LSF Consensus of key developers We ve got to do A lot of work in a short time And we need community help! Ask how to contribute and how to get sample drives at our booth! Contact us after Vault at adrian.palmer@seagate.com or timothy.r.feldman@seagate.com Seagate Confidential 14
14 ext4 ext4 flex_bg ext4 dm-cow zfs btrfs ext4 ext4 flex_bg ext4 dm-cow zfs btrfs ext4 ext4 flex_bg ext4 dm-cow zfs btrfs ext4 ext4 flex_bg ext4 dm-cow zfs btrfs Throughput - MB/s File System Parameters Influence Performance For Drive Managed SMR Drive Managed SMR - Performance by FS Parameters, for different Workloads drivea driveb drivec Mimic OS Load Mimic DVD Download 5meg files Deep Dir 1meg files Deep Dir Re-arranging file system parameters for CoW to enforce forward-writeonly improves performance of a DM-SMR-enabled system 15
15 Q&A Thank You! Attendees and Partners
October 30-31, 2014 Paris
SMR, the ZBC/ZAC Standards and the New libzbc Open Source Project Jorge Campello Director of Systems Architecture, HGST October 30-31, 2014 Paris Magnetic Recording System Technologies New recording system
More informationShingled Magnetic Recording (SMR) Panel: Data Management Techniques Examined Tom Coughlin Coughlin Associates
Shingled Magnetic Recording (SMR) Panel: Data Management Techniques Examined Tom Coughlin Coughlin Associates 2016 Data Storage Innovation Conference. Insert Your Company Name. All Rights Reserved. Introduction
More informationLinux SMR Support Status
Linux SMR Support Status Damien Le Moal Vault Linux Storage and Filesystems Conference - 2017 March 23rd, 2017 Outline Standards and Kernel Support Status Kernel Details - What was needed Block stack File
More informationScaling the Areal Density Mountain. Dave Anderson Seagate
Scaling the Areal Density Mountain Dave Anderson Seagate Technology Progression - ASTC Challenges to Higher Capacity Drives Rdr2 100 nm Rdr1 Thermal Stability Writer/Reader/HMS Scalability Fixed Form Factor
More informationLinux Kernel Support for Hybrid SMR Devices. Damien Le Moal, Director, System Software Group, Western Digital
Linux Kernel Support for Hybrid SMR Devices Damien Le Moal, Director, System Software Group, Western Digital Outline Hybrid SMR device host view - Changes from ZBC Kernel block I/O stack support - Background:
More informationAn SMR-aware Append-only File System Chi-Young Ku Stephen P. Morgan Futurewei Technologies, Inc. Huawei R&D USA
An SMR-aware Append-only File System Chi-Young Ku Stephen P. Morgan Futurewei Technologies, Inc. Huawei R&D USA SMR Technology (1) Future disk drives will be based on shingled magnetic recording. Conventional
More informationWhither Hard Disk Archives? Dave Anderson Seagate Technology 6/2016
Whither Hard Disk Archives? Dave Anderson Seagate Technology 6/2016 Topics as They Relate to Large Storage Archives Where Topology might go Basic HDD Topologies advantages & disadvantages Hyper converged
More informationHGST Shingled Magnetic Recording + HelioSeal Technology
SEPTEMBER 2017 HGST Shingled Magnetic Recording + HelioSeal Technology Achieving Unprecedented Storage Capacity through Innovation Introduction The recent convergence of three macro forces Web 2.0, Cloudbased
More informationHiSMRfs a High Performance File System for Shingled Storage Array
HiSMRfs a High Performance File System for Shingled Storage Array Abstract HiSMRfs, a general purpose file system with standard interface suitable for Shingled Magnetic Recording (SMR) drives has been
More informationShingled Magnetic Recording + HelioSeal Technology. Achieving Unprecedented Storage Capacity Through Innovation
WHITE PAPER JUNE 2018 Shingled Magnetic Recording + HelioSeal Technology Achieving Unprecedented Storage Capacity Through Innovation Introduction The recent convergence of three macro forces Web 2.0, Cloud-based
More informationGearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction
GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction Ting Yao 1,2, Jiguang Wan 1, Ping Huang 2, Yiwen Zhang 1, Zhiwen Liu 1 Changsheng Xie 1, and Xubin He 2 1 Huazhong University of
More informationSMORE: A Cold Data Object Store for SMR Drives
SMORE: A Cold Data Object Store for SMR Drives Peter Macko, Xiongzi Ge, John Haskins Jr.*, James Kelley, David Slik, Keith A. Smith, and Maxim G. Smith Advanced Technology Group NetApp, Inc. * Qualcomm
More informationUsing SMR Drives with Smart Storage Stack-Based HBA and RAID Solutions
White Paper Using SMR Drives with Smart Storage Stack-Based HBA and RAID Solutions October 2017 Contents 1 What Are SMR Drives and Why Are They Used?... 1 2 SMR Drives in HBA or RAID Configurations...
More informationFile system internals Tanenbaum, Chapter 4. COMP3231 Operating Systems
File system internals Tanenbaum, Chapter 4 COMP3231 Operating Systems Architecture of the OS storage stack Application File system: Hides physical location of data on the disk Exposes: directory hierarchy,
More informationStorage Systems for Shingled Disks
Storage Systems for Shingled Disks Garth Gibson Carnegie Mellon University and Panasas Inc Anand Suresh, Jainam Shah, Xu Zhang, Swapnil Patil, Greg Ganger Kryder s Law for Magnetic Disks Market expects
More informationInvest in New Technologies or Divest in Market Share
Invest in New Technologies or Divest in Market Share (Hard Disk Drive and Component Companies Face a Critical Decision to Grow or Die) Thomas Coughlin Coughlin Associates www.tomcoughlin.com Outline Slowing
More informationDeep Storage for Exponential Data. Nathan Thompson CEO, Spectra Logic
Deep Storage for Exponential Data Nathan Thompson CEO, Spectra Logic HISTORY Partnered with Fujifilm on a variety of projects HQ in Boulder, 35 years of business Customers in 54 countries Spectra builds
More informationRed Hat Enterprise 7 Beta File Systems
Red Hat Enterprise 7 Beta File Systems New Scale, Speed & Features Ric Wheeler Director Red Hat Kernel File & Storage Team Red Hat Storage Engineering Agenda Red Hat Enterprise Linux 7 Storage Features
More informationSkylight A Window on Shingled Disk Operation. Abutalib Aghayev, Peter Desnoyers Northeastern University
Skylight A Window on Shingled Disk Operation Abutalib Aghayev, Peter Desnoyers Northeastern University What is Shingled Magnetic Recording (SMR)? A new way of recording tracks on the disk platter. Evolutionary
More informationAdvanced Format in Legacy Infrastructures More Transparent than Disruptive
Advanced Format in Legacy Infrastructures More Transparent than Disruptive Sponsored by IDEMA Presented by Curtis E. Stevens Agenda AF History Enterprise AF Futures SMR & LBA Indirection Hybrids & SSDs
More informationKinetic Open Storage Platform: Enabling Break-through Economics in Scale-out Object Storage PRESENTATION TITLE GOES HERE Ali Fenn & James Hughes
Kinetic Open Storage Platform: Enabling Break-through Economics in Scale-out Object Storage PRESENTATION TITLE GOES HERE Ali Fenn & James Hughes Seagate Technology 2020: 7.3 Zettabytes 56% of total = in
More informationOverview and Current Topics in Solid State Storage
Overview and Current Topics in Solid State Storage Presenter name, company affiliation Presenter Rob name, Peglar company affiliation Xiotech Corporation SNIA Legal Notice The material contained in this
More informationOperating Systems. Operating Systems Professor Sina Meraji U of T
Operating Systems Operating Systems Professor Sina Meraji U of T How are file systems implemented? File system implementation Files and directories live on secondary storage Anything outside of primary
More informationOverview and Current Topics in Solid State Storage
Overview and Current Topics in Solid State Storage Presenter name, company affiliation Presenter Rob name, Peglar company affiliation Xiotech Corporation SNIA Legal Notice The material contained in this
More informationOperating Systems. File Systems. Thomas Ropars.
1 Operating Systems File Systems Thomas Ropars thomas.ropars@univ-grenoble-alpes.fr 2017 2 References The content of these lectures is inspired by: The lecture notes of Prof. David Mazières. Operating
More informationZEA, A Data Management Approach for SMR. Adam Manzanares
ZEA, A Data Management Approach for SMR Adam Manzanares Co-Authors Western Digital Research Cyril Guyot, Damien Le Moal, Zvonimir Bandic University of California, Santa Cruz Noah Watkins, Carlos Maltzahn
More informationMODERN FILESYSTEM PERFORMANCE IN LOCAL MULTI-DISK STORAGE SPACE CONFIGURATION
INFORMATION SYSTEMS IN MANAGEMENT Information Systems in Management (2014) Vol. 3 (4) 273 283 MODERN FILESYSTEM PERFORMANCE IN LOCAL MULTI-DISK STORAGE SPACE CONFIGURATION MATEUSZ SMOLIŃSKI Institute of
More informationLightNVM: The Linux Open-Channel SSD Subsystem Matias Bjørling (ITU, CNEX Labs), Javier González (CNEX Labs), Philippe Bonnet (ITU)
½ LightNVM: The Linux Open-Channel SSD Subsystem Matias Bjørling (ITU, CNEX Labs), Javier González (CNEX Labs), Philippe Bonnet (ITU) 0% Writes - Read Latency 4K Random Read Latency 4K Random Read Percentile
More informationNovel Address Mappings for Shingled Write Disks
Novel Address Mappings for Shingled Write Disks Weiping He and David H.C. Du Department of Computer Science, University of Minnesota, Twin Cities {weihe,du}@cs.umn.edu Band Band Band Abstract Shingled
More informationFile System Case Studies. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
File System Case Studies Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics The Original UNIX File System FFS Ext2 FAT 2 UNIX FS (1)
More informationDDN s Vision for the Future of Lustre LUG2015 Robert Triendl
DDN s Vision for the Future of Lustre LUG2015 Robert Triendl 3 Topics 1. The Changing Markets for Lustre 2. A Vision for Lustre that isn t Exascale 3. Building Lustre for the Future 4. Peak vs. Operational
More informationCSE 451: Operating Systems Spring Module 12 Secondary Storage
CSE 451: Operating Systems Spring 2017 Module 12 Secondary Storage John Zahorjan 1 Secondary storage Secondary storage typically: is anything that is outside of primary memory does not permit direct execution
More informationCOS 318: Operating Systems. Storage Devices. Jaswinder Pal Singh Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Jaswinder Pal Singh Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall13/cos318/ Today s Topics Magnetic disks
More informationFilesystems Lecture 10. Credit: some slides by John Kubiatowicz and Anthony D. Joseph
Filesystems Lecture 10 Credit: some slides by John Kubiatowicz and Anthony D. Joseph Today and some of next class Overview of file systems Papers on basic file systems A Fast File System for UNIX Marshall
More informationTopics. Lecture 8: Magnetic Disks
Lecture 8: Magnetic Disks SONGS ABOUT COMPUTER SCIENCE Topics Basic terms and operation Some history and trends Performance Disk arrays (RAIDs) SAVE THE CODE Written by Mikolaj Franaszczuk To the tune
More informationFile System Case Studies. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
File System Case Studies Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics The Original UNIX File System FFS Ext2 FAT 2 UNIX FS (1)
More informationThe Btrfs Filesystem. Chris Mason
The Btrfs Filesystem Chris Mason The Btrfs Filesystem Jointly developed by a number of companies Oracle, Redhat, Fujitsu, Intel, SUSE, many others All data and metadata is written via copy-on-write CRCs
More informationHard Disk Drives. Nima Honarmand (Based on slides by Prof. Andrea Arpaci-Dusseau)
Hard Disk Drives Nima Honarmand (Based on slides by Prof. Andrea Arpaci-Dusseau) Storage Stack in the OS Application Virtual file system Concrete file system Generic block layer Driver Disk drive Build
More informationFlash Storage with 24G SAS Leads the Way in Crunching Big Data
Flash Storage with 24G SAS Leads the Way in Crunching Big Data SCSI Trade Association August 8th, 2018 1 Today s Panel Dennis Martin Founder and President Demartek Mohamad El-Batal Sr. Director of Architecture,
More informationThe Datacentered Future Greg Huff CTO, LSI Corporation
The Datacentered Future Greg Huff CTO, LSI Corporation 1 Tremendous Growth in Connected Data Sources, Consumption Devices, and Services 2 Nearly limitless data depth and breadth needed Execution of millions
More informationBIg data era calls for Petabyte storage systems with
Performance Evaluation of Host Aware Shingled Magnetic Recording (HA-SMR) Drives Fenggang Wu, Ziqi Fan, Ming-Chang Yang, Baoquan Zhang, Xiongzi Ge and David H.C. Du 1 Abstract Shingled Magnetic Recording
More informationMaking Storage Smarter Jim Williams Martin K. Petersen
Making Storage Smarter Jim Williams Martin K. Petersen Agenda r Background r Examples r Current Work r Future 2 Definition r Storage is made smarter by exchanging information between the application and
More informationStorage Systems : Disks and SSDs. Manu Awasthi July 6 th 2018 Computer Architecture Summer School 2018
Storage Systems : Disks and SSDs Manu Awasthi July 6 th 2018 Computer Architecture Summer School 2018 Why study storage? Scalable High Performance Main Memory System Using Phase-Change Memory Technology,
More informationCOS 318: Operating Systems. Storage Devices. Vivek Pai Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Vivek Pai Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall11/cos318/ Today s Topics Magnetic disks Magnetic disk
More informationModular Drive Process System Design for Optimal Factory Efficiency September 9, 2010
Modular Drive Process System Design for Optimal Factory Efficiency September 9, 2010 Presenter: Pete Goglia Contributors: Mark McCrimmon, Kevin Richardson, Nick Granger-Brown Modular Drive Process System
More informationMarty Czekalski President, SCSI Trade Association - Emerging Interface and Architecture Program Manager, Seagate Technology
SAS: The PRESENTATION Fabric for TITLE Storage GOES HERE Solutions Marty Czekalski President, SCSI Trade Association - Emerging Interface and Architecture Program Manager, Seagate Technology Greg McSorley
More informationCSE 153 Design of Operating Systems
CSE 153 Design of Operating Systems Winter 2018 Lecture 20: File Systems (1) Disk drives OS Abstractions Applications Process File system Virtual memory Operating System CPU Hardware Disk RAM CSE 153 Lecture
More informationHigh-Performance and Large-Capacity Storage: A Winning Combination for Future Data Centers. Phil Brace August 12, 2015
High-Performance and Large-Capacity Storage: A Winning Combination for Future Data Centers Phil Brace August 12, 2015 Data is Changing Bigger Different $ Constrained Zettabytes 45 40 35 30 25 20 15 10
More informationPASIG Disk Trends. Oracle Storage Technology 101 Session. Philippe Deverchère EMEA Storage CTO. September 16, 2014
PASIG Disk Trends Oracle Storage Technology 101 Session Philippe Deverchère EMEA Storage CTO September 16, 2014 Copyright 2014 Oracle and/or its affiliates. All rights reserved. Storage Technologies Areal
More informationI/O & Storage. Jin-Soo Kim ( Computer Systems Laboratory Sungkyunkwan University
I/O & Storage Jin-Soo Kim ( jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics I/O systems Device characteristics: block vs. character I/O systems
More informationMonday, May 4, Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes
Monday, May 4, 2015 Topics for today Secondary memory Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes Storage management (Chapter
More informationExt4-zcj: An evolved journal optimized for Drive-Managed Shingled Magnetic Recording Disks
Ext4-zcj: An evolved journal optimized for Drive-Managed Shingled Magnetic Recording Disks Abutalib Aghayev 1, Theodore Ts o 2, Garth Gibson 1, and Peter Desnoyers 3 1 Carnegie Mellon University 2 Google
More informationOpen-Channel SSDs Offer the Flexibility Required by Hyperscale Infrastructure Matias Bjørling CNEX Labs
Open-Channel SSDs Offer the Flexibility Required by Hyperscale Infrastructure Matias Bjørling CNEX Labs 1 Public and Private Cloud Providers 2 Workloads and Applications Multi-Tenancy Databases Instance
More informationLinux File Systems: Challenges and Futures Ric Wheeler Red Hat
Linux File Systems: Challenges and Futures Ric Wheeler Red Hat Overview The Linux Kernel Process What Linux Does Well Today New Features in Linux File Systems Ongoing Challenges 2 What is Linux? A set
More informationAnnouncements. Persistence: Log-Structured FS (LFS)
Announcements P4 graded: In Learn@UW; email 537-help@cs if problems P5: Available - File systems Can work on both parts with project partner Watch videos; discussion section Part a : file system checker
More informationSeagate Point of View Cloud and Data Center Trend
DATA IS IN OUR DNA Seagate Point of View Cloud and Data Center Trend Raj Rajagopalan March 2018 1 Safe Harbor Statement This document contains forward-looking statements within the meaning of Section 27A
More informationNVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory
NVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory Dhananjoy Das, Sr. Systems Architect SanDisk Corp. 1 Agenda: Applications are KING! Storage landscape (Flash / NVM)
More informationHigh Performance Solid State Storage Under Linux
High Performance Solid State Storage Under Linux Eric Seppanen, Matthew T. O Keefe, David J. Lilja Electrical and Computer Engineering University of Minnesota April 20, 2010 Motivation SSDs breaking through
More informationSTORAGE SYSTEMS. Operating Systems 2015 Spring by Euiseong Seo
STORAGE SYSTEMS Operating Systems 2015 Spring by Euiseong Seo Today s Topics HDDs (Hard Disk Drives) Disk scheduling policies Linux I/O schedulers Secondary Storage Anything that is outside of primary
More informationCOS 318: Operating Systems. Storage Devices. Kai Li Computer Science Department Princeton University
COS 318: Operating Systems Storage Devices Kai Li Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall11/cos318/ Today s Topics Magnetic disks Magnetic disk
More informationQuickSpecs. What's New. Models. HP SATA Hard Drives. Overview
Overview HP SATA drives are designed for the reliability and larger capacities demanded by today's entry server and external storage environments. HP SATA Midline drives are designed with economical reliability
More informationLong-term Information Storage Must store large amounts of data Information stored must survive the termination of the process using it Multiple proces
File systems 1 Long-term Information Storage Must store large amounts of data Information stored must survive the termination of the process using it Multiple processes must be able to access the information
More informationFile System Internals. Jo, Heeseung
File System Internals Jo, Heeseung Today's Topics File system implementation File descriptor table, File table Virtual file system File system design issues Directory implementation: filename -> metadata
More informationSSD/Flash for Modern Databases. Peter Zaitsev, CEO, Percona November 1, 2014 Highload Moscow,Russia
SSD/Flash for Modern Databases Peter Zaitsev, CEO, Percona November 1, 2014 Highload++ 2014 Moscow,Russia Percona We love Open Source Software Percona Server Percona Xtrabackup Percona XtraDB Cluster Percona
More informationCSE 333 Lecture 9 - storage
CSE 333 Lecture 9 - storage Steve Gribble Department of Computer Science & Engineering University of Washington Administrivia Colin s away this week - Aryan will be covering his office hours (check the
More informationFile system internals Tanenbaum, Chapter 4. COMP3231 Operating Systems
File system internals Tanenbaum, Chapter 4 COMP3231 Operating Systems Summary of the FS abstraction User's view Hierarchical structure Arbitrarily-sized files Symbolic file names Contiguous address space
More informationGFS: The Google File System
GFS: The Google File System Brad Karp UCL Computer Science CS GZ03 / M030 24 th October 2014 Motivating Application: Google Crawl the whole web Store it all on one big disk Process users searches on one
More informationSolid State Drives (SSDs) Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
Solid State Drives (SSDs) Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Memory Types FLASH High-density Low-cost High-speed Low-power High reliability
More informationMass-Storage Structure
Operating Systems (Fall/Winter 2018) Mass-Storage Structure Yajin Zhou (http://yajin.org) Zhejiang University Acknowledgement: some pages are based on the slides from Zhi Wang(fsu). Review On-disk structure
More informationNAND Flash-based Storage. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
NAND Flash-based Storage Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics NAND flash memory Flash Translation Layer (FTL) OS implications
More informationPERSISTENCE: FSCK, JOURNALING. Shivaram Venkataraman CS 537, Spring 2019
PERSISTENCE: FSCK, JOURNALING Shivaram Venkataraman CS 537, Spring 2019 ADMINISTRIVIA Project 4b: Due today! Project 5: Out by tomorrow Discussion this week: Project 5 AGENDA / LEARNING OUTCOMES How does
More informationOpen Source for OSD. Dan Messinger
Open Source for OSD Dan Messinger The Goal To make OSD technology available to the public. (public == anybody outside the small group of developers working on OSD itself) Requires that OSD drivers be available
More informationSegmentation with Paging. Review. Segmentation with Page (MULTICS) Segmentation with Page (MULTICS) Segmentation with Page (MULTICS)
Review Segmentation Segmentation Implementation Advantage of Segmentation Protection Sharing Segmentation with Paging Segmentation with Paging Segmentation with Paging Reason for the segmentation with
More informationCrashMonkey: A Framework to Systematically Test File-System Crash Consistency. Ashlie Martinez Vijay Chidambaram University of Texas at Austin
CrashMonkey: A Framework to Systematically Test File-System Crash Consistency Ashlie Martinez Vijay Chidambaram University of Texas at Austin Crash Consistency File-system updates change multiple blocks
More informationI/O CANNOT BE IGNORED
LECTURE 13 I/O I/O CANNOT BE IGNORED Assume a program requires 100 seconds, 90 seconds for main memory, 10 seconds for I/O. Assume main memory access improves by ~10% per year and I/O remains the same.
More informationJanuary 28-29, 2014 San Jose
January 28-29, 2014 San Jose Flash for the Future Software Optimizations for Non Volatile Memory Nisha Talagala, Lead Architect, Fusion-io Gary Orenstein, Chief Marketing Officer, Fusion-io @garyorenstein
More informationChunling Wang, Dandan Wang, Yunpeng Chai, Chuanwen Wang and Diansen Sun Renmin University of China
Chunling Wang, Dandan Wang, Yunpeng Chai, Chuanwen Wang and Diansen Sun Renmin University of China Data volume is growing 44ZB in 2020! How to store? Flash arrays, DRAM-based storage: high costs, reliability,
More informationQuickSpecs. What's New. Models. HP SATA Hard Drives. Overview. HP 6G SATA SmartDrive Carriers
Overview HP SATA drives are designed for the reliability and larger capacities demanded by today's entry server and external storage environments. HP SATA Midline drives are designed with economical reliability
More informationAlternatives to Solaris Containers and ZFS for Linux on System z
Alternatives to Solaris Containers and ZFS for Linux on System z Cameron Seader (cs@suse.com) SUSE Tuesday, March 11, 2014 Session Number 14540 Agenda Quick Overview of Solaris Containers and ZFS Linux
More informationQuickSpecs. HPE SAS Hard Drives. Overview. What's New
HPE s Overview HPE s Serial Attached SCSI () provides a superior storage solution. With some storage requirements escalating and others becoming more complex, factors such as flexibility, performance,
More information<Insert Picture Here> End-to-end Data Integrity for NFS
End-to-end Data Integrity for NFS Chuck Lever Consulting Member of Technical Staff Today s Discussion What is end-to-end data integrity? T10 PI overview Adapting
More informationIntroduction to Open-Channel Solid State Drives and What s Next!
Introduction to Open-Channel Solid State Drives and What s Next! Matias Bjørling Director, Solid-State System Software September 25rd, 2018 Storage Developer Conference 2018, Santa Clara, CA Forward-Looking
More informationDisk Scheduling COMPSCI 386
Disk Scheduling COMPSCI 386 Topics Disk Structure (9.1 9.2) Disk Scheduling (9.4) Allocation Methods (11.4) Free Space Management (11.5) Hard Disk Platter diameter ranges from 1.8 to 3.5 inches. Both sides
More informationCS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Spring 2018 Lecture 22 File Systems Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 Disk Structure Disk can
More informationUNIT 2 Data Center Environment
UNIT 2 Data Center Environment This chapter provides an understanding of various logical components of hosts such as file systems, volume managers, and operating systems, and their role in the storage
More informationChapter 11: Implementing File Systems
Chapter 11: Implementing File Systems Operating System Concepts 99h Edition DM510-14 Chapter 11: Implementing File Systems File-System Structure File-System Implementation Directory Implementation Allocation
More informationFile. File System Implementation. File Metadata. File System Implementation. Direct Memory Access Cont. Hardware background: Direct Memory Access
File File System Implementation Operating Systems Hebrew University Spring 2009 Sequence of bytes, with no structure as far as the operating system is concerned. The only operations are to read and write
More informationu Covered: l Management of CPU & concurrency l Management of main memory & virtual memory u Currently --- Management of I/O devices
Where Are We? COS 318: Operating Systems Storage Devices Jaswinder Pal Singh Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) u Covered: l Management of CPU
More informationCold Storage: The Road to Enterprise Ilya Kuznetsov YADRO
Cold Storage: The Road to Enterprise Ilya Kuznetsov YADRO Agenda Technical challenge Custom product Growth of aspirations Enterprise requirements Making an enterprise cold storage product 2 Technical Challenge
More informationWednesday, April 25, Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes
Wednesday, April 25, 2018 Topics for today Secondary memory Discs RAID: Introduction Error detection and correction Error detection: Simple parity Error correction: Hamming Codes Storage management (Chapter
More informationFile System Case Studies. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University
File System Case Studies Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics The Original UNIX File System FFS Ext2 FAT 2 UNIX FS (1)
More informationLonghorn Large Sector Size Support. Anuraag Tiwari Program Manager Core File System
Longhorn Large Sector Size Support Anuraag Tiwari Program Manager Core File System anuraagt@microsoft.com Agenda Historical OS Support for Large Sector Size Drives A Brief Overview of the OS Disk I/O Components
More informationI/O CANNOT BE IGNORED
LECTURE 13 I/O I/O CANNOT BE IGNORED Assume a program requires 100 seconds, 90 seconds for main memory, 10 seconds for I/O. Assume main memory access improves by ~10% per year and I/O remains the same.
More informationStorage Technologies - 3
Storage Technologies - 3 COMP 25212 - Lecture 10 Antoniu Pop antoniu.pop@manchester.ac.uk 1 March 2019 Antoniu Pop Storage Technologies - 3 1 / 20 Learning Objectives - Storage 3 Understand characteristics
More informationMain Points. File layout Directory layout
File Systems Main Points File layout Directory layout File System Design Constraints For small files: Small blocks for storage efficiency Files used together should be stored together For large files:
More informationOpen-Channel SSDs Then. Now. And Beyond. Matias Bjørling, March 22, Copyright 2017 CNEX Labs
Open-Channel SSDs Then. Now. And Beyond. Matias Bjørling, March 22, 2017 What is an Open-Channel SSD? Then Now - Physical Page Addressing v1.2 - LightNVM Subsystem - Developing for an Open-Channel SSD
More informationCS370: Operating Systems [Spring 2017] Dept. Of Computer Science, Colorado State University
Frequently asked questions from the previous class survey CS 370: OPERATING SYSTEMS [FILE SYSTEMS] Shrideep Pallickara Computer Science Colorado State University If you have a file with scattered blocks,
More informationStorage Speed and Human Behavior. PRESENTATION TITLE GOES HERE Eric Herzog CMO and Senior VP of Business Development Violin Memory
Storage Speed and Human Behavior PRESENTATION TITLE GOES HERE Eric Herzog CMO and Senior VP of Business Development Violin Memory I Feel the Need for Speed Enterprises Software-as-a- Service Cloud Providers
More informationStorage Systems : Disks and SSDs. Manu Awasthi CASS 2018
Storage Systems : Disks and SSDs Manu Awasthi CASS 2018 Why study storage? Scalable High Performance Main Memory System Using Phase-Change Memory Technology, Qureshi et al, ISCA 2009 Trends Total amount
More informationIntroduction to I/O and Disk Management
1 Secondary Storage Management Disks just like memory, only different Introduction to I/O and Disk Management Why have disks? Ø Memory is small. Disks are large. Short term storage for memory contents
More information