Freewrite: Creating (Almost) Zero-Cost Writes to SSD

Size: px
Start display at page:

Download "Freewrite: Creating (Almost) Zero-Cost Writes to SSD"

Transcription

1 Freewrite: Creating (Almost) Zero-Cost Writes to SSD Chunyi Liu, Fan Ni, Xingbo Wu, and Song Jiang Xiao Zhang University of Texas at Arlington, USA Northwestern Polytechnical University, China

2 The Achilles' Heel in SSD: Writes SSD has high performance that is not sensitive to access sequentiality However, write operations are expensive and destructive. Write latency can be higher than reads by many times. Out-of-place writes may require mapping change, block erasure, and even page migrations. Expensive garbage collections can be triggered. Writes reduce its limited lifespan. The goal: Remove Unnecessary Writes to the SSD -2-

3 Applications Powerpoint Video Editing Database Key-value Store The Opportunity and Challenge UPDATE FILE (insert/delete/overwrite) READ FILE The OS Kernel Block-level Dedup WRITE UPDATED FILE SSD -3-

4 Applications Powerpoint Video Editing Database Key-value Store The Opportunity and Challenge UPDATE FILE (insert/delete/overwrite) READ FILE The OS Kernel Block-level Dedup SSD? WRITE UPDATED FILE -4-

5 Applications Powerpoint Video Editing Database Key-value Store The Opportunities and Challenge UPDATE FILE (insert/delete/overwrite) Block Boundaries READ FILE The OS Kernel Block-level Dedup SSD -5-

6 Applications Powerpoint Video Editing Database Key-value Store The Opportunities and Challenge UPDATE FILE (insert/delete/overwrite) Block Boundaries READ FILE The OS Kernel Block-level Dedup SSD Duplicated blocks not recognized due to mis-alignment! WRITE UPDATED FILE -6-

7 Freewrite: Retain Alignments to Maximize Duplicates The approach: allow small unused spaces (bubbles) in a file to keep un-updated blocks aligned. Update Ops Before Update After Update w/o Freewrite After Update w/ Freewrite Block # Insert Delete Un-updated blocks Mis-aligned! Insert -7- Unupdated blocks Overwrite Un-updated blocks

8 Freewrite: Retain Alignments to Maximize Duplicates The approach: allow small unused spaces (bubbles) in a file to keep un-updated blocks aligned. Update Ops Before Update After Update w/o Frewrite After Update w/ Freewrite Block # Insert Delete Un-update blocks Mis-aligned! Insert Aligned! -8- Unupdate blocks Overwrite Unupdate blocks Insert a bubble

9 Use Case I: Compaction in LevelDB LevelDB is a key-value store in which KV items are incrementally sorted in a multi-level structure. Level k+1 is much larger than Level k. Adjacent levels are constantly merge-sorted. Freewrite inserts bubbles into the new Level k+1 to reduce its writes. Level k Update Insert bubbles (fake KV items) to retain alignment evel k+1 Merge Sort -9-

10 Use Case II: Garbage Collection in FAWN FAWN is a key-value store always appending KV items in a log file. GC is periodically performed in the log file to remove invalidated items. Freewrite inserts bubbles into the new log file to reduce its writes. Old Log Garbage Collection New Log Insert bubbles (fake KV items) to retain alignment -10-

11 Experimental Setup Implemented at the application level (LevelDB and FAWN) Use Dmdedup as block-level dedup. Use Samsung 840 EVO 1GB SSD. Use put requests with 4% repeated keys. Threshold of bubble size is 12.5% of block size (512B), and total bubble space overhead is capped at 5%. Three key distributions: uniform, clustered, and mixed. -11-

12 Experiment Results (LevelDB) 22% improvement of Throughput 24% reduction of written data -12-

13 Experiment Results (FAWN) 4.5X improvement of Throughput 8.2X reduction of written data -13-

14 Summary Freewrite creates deduplication opportunities to leverage the readily available block-level deduplication system. Freewrite improves write performance and helps to extend SSD lifetime. The overhead of using bubbles is very limited. -14-

LSM-trie: An LSM-tree-based Ultra-Large Key-Value Store for Small Data

LSM-trie: An LSM-tree-based Ultra-Large Key-Value Store for Small Data LSM-trie: An LSM-tree-based Ultra-Large Key-Value Store for Small Data Xingbo Wu Yuehai Xu Song Jiang Zili Shao The Hong Kong Polytechnic University The Challenge on Today s Key-Value Store Trends on workloads

More information

Performance Benefits of Running RocksDB on Samsung NVMe SSDs

Performance Benefits of Running RocksDB on Samsung NVMe SSDs Performance Benefits of Running RocksDB on Samsung NVMe SSDs A Detailed Analysis 25 Samsung Semiconductor Inc. Executive Summary The industry has been experiencing an exponential data explosion over the

More information

The What, Why and How of the Pure Storage Enterprise Flash Array. Ethan L. Miller (and a cast of dozens at Pure Storage)

The What, Why and How of the Pure Storage Enterprise Flash Array. Ethan L. Miller (and a cast of dozens at Pure Storage) The What, Why and How of the Pure Storage Enterprise Flash Array Ethan L. Miller (and a cast of dozens at Pure Storage) Enterprise storage: $30B market built on disk Key players: EMC, NetApp, HP, etc.

More information

HashKV: Enabling Efficient Updates in KV Storage via Hashing

HashKV: Enabling Efficient Updates in KV Storage via Hashing HashKV: Enabling Efficient Updates in KV Storage via Hashing Helen H. W. Chan, Yongkun Li, Patrick P. C. Lee, Yinlong Xu The Chinese University of Hong Kong University of Science and Technology of China

More information

GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction

GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction Ting Yao 1,2, Jiguang Wan 1, Ping Huang 2, Yiwen Zhang 1, Zhiwen Liu 1 Changsheng Xie 1, and Xubin He 2 1 Huazhong University of

More information

SmartMD: A High Performance Deduplication Engine with Mixed Pages

SmartMD: A High Performance Deduplication Engine with Mixed Pages SmartMD: A High Performance Deduplication Engine with Mixed Pages Fan Guo 1, Yongkun Li 1, Yinlong Xu 1, Song Jiang 2, John C. S. Lui 3 1 University of Science and Technology of China 2 University of Texas,

More information

Strata: A Cross Media File System. Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, Thomas Anderson

Strata: A Cross Media File System. Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, Thomas Anderson A Cross Media File System Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, Thomas Anderson 1 Let s build a fast server NoSQL store, Database, File server, Mail server Requirements

More information

Application-Managed Flash

Application-Managed Flash Application-Managed Flash Sungjin Lee*, Ming Liu, Sangwoo Jun, Shuotao Xu, Jihong Kim and Arvind *Inha University Massachusetts Institute of Technology Seoul National University Operating System Support

More information

Maximizing Data Center and Enterprise Storage Efficiency

Maximizing Data Center and Enterprise Storage Efficiency Maximizing Data Center and Enterprise Storage Efficiency Enterprise and data center customers can leverage AutoStream to achieve higher application throughput and reduced latency, with negligible organizational

More information

SILT: A Memory-Efficient, High- Performance Key-Value Store

SILT: A Memory-Efficient, High- Performance Key-Value Store SILT: A Memory-Efficient, High- Performance Key-Value Store SOSP 11 Presented by Fan Ni March, 2016 SILT is Small Index Large Tables which is a memory efficient high performance key value store system

More information

PebblesDB: Building Key-Value Stores using Fragmented Log Structured Merge Trees

PebblesDB: Building Key-Value Stores using Fragmented Log Structured Merge Trees PebblesDB: Building Key-Value Stores using Fragmented Log Structured Merge Trees Pandian Raju 1, Rohan Kadekodi 1, Vijay Chidambaram 1,2, Ittai Abraham 2 1 The University of Texas at Austin 2 VMware Research

More information

S-FTL: An Efficient Address Translation for Flash Memory by Exploiting Spatial Locality

S-FTL: An Efficient Address Translation for Flash Memory by Exploiting Spatial Locality S-FTL: An Efficient Address Translation for Flash Memory by Exploiting Spatial Locality Song Jiang, Lei Zhang, Xinhao Yuan, Hao Hu, and Yu Chen Department of Electrical and Computer Engineering Wayne State

More information

Atlas: Baidu s Key-value Storage System for Cloud Data

Atlas: Baidu s Key-value Storage System for Cloud Data Atlas: Baidu s Key-value Storage System for Cloud Data Song Jiang Chunbo Lai Shiding Lin Liqiong Yang Guangyu Sun Jason Cong Wayne State University Zhenyu Hou Can Cui Peking University University of California

More information

ParaFS: A Log-Structured File System to Exploit the Internal Parallelism of Flash Devices

ParaFS: A Log-Structured File System to Exploit the Internal Parallelism of Flash Devices ParaFS: A Log-Structured File System to Exploit the Internal Parallelism of Devices Jiacheng Zhang, Jiwu Shu, Youyou Lu Tsinghua University 1 Outline Background and Motivation ParaFS Design Evaluation

More information

WOJ: Enabling Write-Once Full-data Journaling in SSDs by Using Weak-Hashing-based Deduplication

WOJ: Enabling Write-Once Full-data Journaling in SSDs by Using Weak-Hashing-based Deduplication WOJ: Enabling Write-Once Full-data Journaling in SSDs by Using Weak-Hashing-based Deduplication Fan Ni a,, Xingbo Wu b, Weijun Li c, Lei Wang d, Song Jiang a a University of Texas at Arlington, Arlington,

More information

AutoStream: Automatic Stream Management for Multi-stream SSDs in Big Data Era

AutoStream: Automatic Stream Management for Multi-stream SSDs in Big Data Era AutoStream: Automatic Stream Management for Multi-stream SSDs in Big Data Era Changho Choi, PhD Principal Engineer Memory Solutions Lab (San Jose, CA) Samsung Semiconductor, Inc. 1 Disclaimer This presentation

More information

AutoStream: Automatic Stream Management for Multi-stream SSDs

AutoStream: Automatic Stream Management for Multi-stream SSDs AutoStream: Automatic Stream Management for Multi-stream SSDs Jingpei Yang, PhD, Rajinikanth Pandurangan, Changho Choi, PhD, Vijay Balakrishnan Memory Solutions Lab Samsung Semiconductor Agenda SSD NAND

More information

Middleware and Flash Translation Layer Co-Design for the Performance Boost of Solid-State Drives

Middleware and Flash Translation Layer Co-Design for the Performance Boost of Solid-State Drives Middleware and Flash Translation Layer Co-Design for the Performance Boost of Solid-State Drives Chao Sun 1, Asuka Arakawa 1, Ayumi Soga 1, Chihiro Matsui 1 and Ken Takeuchi 1 1 Chuo University Santa Clara,

More information

Linux Kernel Abstractions for Open-Channel SSDs

Linux Kernel Abstractions for Open-Channel SSDs Linux Kernel Abstractions for Open-Channel SSDs Matias Bjørling Javier González, Jesper Madsen, and Philippe Bonnet 2015/03/01 1 Market Specific FTLs SSDs on the market with embedded FTLs targeted at specific

More information

The Google File System

The Google File System October 13, 2010 Based on: S. Ghemawat, H. Gobioff, and S.-T. Leung: The Google file system, in Proceedings ACM SOSP 2003, Lake George, NY, USA, October 2003. 1 Assumptions Interface Architecture Single

More information

A Light-weight Compaction Tree to Reduce I/O Amplification toward Efficient Key-Value Stores

A Light-weight Compaction Tree to Reduce I/O Amplification toward Efficient Key-Value Stores A Light-weight Compaction Tree to Reduce I/O Amplification toward Efficient Key-Value Stores T i n g Y a o 1, J i g u a n g W a n 1, P i n g H u a n g 2, X u b i n He 2, Q i n g x i n G u i 1, F e i W

More information

Storage Systems for Shingled Disks

Storage Systems for Shingled Disks Storage Systems for Shingled Disks Garth Gibson Carnegie Mellon University and Panasas Inc Anand Suresh, Jainam Shah, Xu Zhang, Swapnil Patil, Greg Ganger Kryder s Law for Magnetic Disks Market expects

More information

SLM-DB: Single-Level Key-Value Store with Persistent Memory

SLM-DB: Single-Level Key-Value Store with Persistent Memory SLM-DB: Single-Level Key-Value Store with Persistent Memory Olzhas Kaiyrakhmet and Songyi Lee, UNIST; Beomseok Nam, Sungkyunkwan University; Sam H. Noh and Young-ri Choi, UNIST https://www.usenix.org/conference/fast19/presentation/kaiyrakhmet

More information

Part II: Data Center Software Architecture: Topic 2: Key-value Data Management Systems. SkimpyStash: Key Value Store on Flash-based Storage

Part II: Data Center Software Architecture: Topic 2: Key-value Data Management Systems. SkimpyStash: Key Value Store on Flash-based Storage ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective Part II: Data Center Software Architecture: Topic 2: Key-value Data Management Systems SkimpyStash: Key Value

More information

arxiv: v1 [cs.db] 25 Nov 2018

arxiv: v1 [cs.db] 25 Nov 2018 Enabling Efficient Updates in KV Storage via Hashing: Design and Performance Evaluation Yongkun Li, Helen H. W. Chan, Patrick P. C. Lee, and Yinlong Xu University of Science and Technology of China The

More information

Cheap and Large CAMs for High Performance Data-Intensive Networked Systems- The Bufferhash KV Store

Cheap and Large CAMs for High Performance Data-Intensive Networked Systems- The Bufferhash KV Store Cheap and Large CAMs for High Performance Data-Intensive Networked Systems- The Bufferhash KV Store Presented by Akhila Nookala M.S EE Wayne State University ECE7650 Scalable and Secure Internet Services

More information

LSM-trie: An LSM-tree-based Ultra-Large Key-Value Store for Small Data

LSM-trie: An LSM-tree-based Ultra-Large Key-Value Store for Small Data LSM-trie: An LSM-tree-based Ultra-Large Key-Value Store for Small Data Xingbo Wu and Yuehai Xu, Wayne State University; Zili Shao, The Hong Kong Polytechnic University; Song Jiang, Wayne State University

More information

Gecko: Contention-Oblivious Disk Arrays for Cloud Storage

Gecko: Contention-Oblivious Disk Arrays for Cloud Storage Gecko: Contention-Oblivious Disk Arrays for Cloud Storage Ji-Yong Shin Cornell University In collaboration with Mahesh Balakrishnan (MSR SVC), Tudor Marian (Google), and Hakim Weatherspoon (Cornell) FAST

More information

A DEDUPLICATION-INSPIRED FAST DELTA COMPRESSION APPROACH W EN XIA, HONG JIANG, DA N FENG, LEI T I A N, M I N FU, YUKUN Z HOU

A DEDUPLICATION-INSPIRED FAST DELTA COMPRESSION APPROACH W EN XIA, HONG JIANG, DA N FENG, LEI T I A N, M I N FU, YUKUN Z HOU A DEDUPLICATION-INSPIRED FAST DELTA COMPRESSION APPROACH W EN XIA, HONG JIANG, DA N FENG, LEI T I A N, M I N FU, YUKUN Z HOU PRESENTED BY ROMAN SHOR Overview Technics of data reduction in storage systems:

More information

The Unwritten Contract of Solid State Drives

The Unwritten Contract of Solid State Drives The Unwritten Contract of Solid State Drives Jun He, Sudarsun Kannan, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau Department of Computer Sciences, University of Wisconsin - Madison Enterprise SSD

More information

Parallelizing Inline Data Reduction Operations for Primary Storage Systems

Parallelizing Inline Data Reduction Operations for Primary Storage Systems Parallelizing Inline Data Reduction Operations for Primary Storage Systems Jeonghyeon Ma ( ) and Chanik Park Department of Computer Science and Engineering, POSTECH, Pohang, South Korea {doitnow0415,cipark}@postech.ac.kr

More information

Accelerating Restore and Garbage Collection in Deduplication-based Backup Systems via Exploiting Historical Information

Accelerating Restore and Garbage Collection in Deduplication-based Backup Systems via Exploiting Historical Information Accelerating Restore and Garbage Collection in Deduplication-based Backup Systems via Exploiting Historical Information Min Fu, Dan Feng, Yu Hua, Xubin He, Zuoning Chen *, Wen Xia, Fangting Huang, Qing

More information

An Efficient Memory-Mapped Key-Value Store for Flash Storage

An Efficient Memory-Mapped Key-Value Store for Flash Storage An Efficient Memory-Mapped Key-Value Store for Flash Storage Anastasios Papagiannis, Giorgos Saloustros, Pilar González-Férez, and Angelos Bilas Institute of Computer Science (ICS) Foundation for Research

More information

BPCLC: An Efficient Write Buffer Management Scheme for Flash-Based Solid State Disks

BPCLC: An Efficient Write Buffer Management Scheme for Flash-Based Solid State Disks BPCLC: An Efficient Write Buffer Management Scheme for Flash-Based Solid State Disks Hui Zhao 1, Peiquan Jin *1, Puyuan Yang 1, Lihua Yue 1 1 School of Computer Science and Technology, University of Science

More information

Yiying Zhang, Leo Prasath Arulraj, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. University of Wisconsin - Madison

Yiying Zhang, Leo Prasath Arulraj, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. University of Wisconsin - Madison Yiying Zhang, Leo Prasath Arulraj, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau University of Wisconsin - Madison 1 Indirection Reference an object with a different name Flexible, simple, and

More information

SILT: A MEMORY-EFFICIENT, HIGH-PERFORMANCE KEY- VALUE STORE PRESENTED BY PRIYA SRIDHAR

SILT: A MEMORY-EFFICIENT, HIGH-PERFORMANCE KEY- VALUE STORE PRESENTED BY PRIYA SRIDHAR SILT: A MEMORY-EFFICIENT, HIGH-PERFORMANCE KEY- VALUE STORE PRESENTED BY PRIYA SRIDHAR AGENDA INTRODUCTION Why SILT? MOTIVATION SILT KV STORAGE SYSTEM LOW READ AMPLIFICATION CONTROLLABLE WRITE AMPLIFICATION

More information

I/O Devices & SSD. Dongkun Shin, SKKU

I/O Devices & SSD. Dongkun Shin, SKKU I/O Devices & SSD 1 System Architecture Hierarchical approach Memory bus CPU and memory Fastest I/O bus e.g., PCI Graphics and higherperformance I/O devices Peripheral bus SCSI, SATA, or USB Connect many

More information

Baoping Wang School of software, Nanyang Normal University, Nanyang , Henan, China

Baoping Wang School of software, Nanyang Normal University, Nanyang , Henan, China doi:10.21311/001.39.7.41 Implementation of Cache Schedule Strategy in Solid-state Disk Baoping Wang School of software, Nanyang Normal University, Nanyang 473061, Henan, China Chao Yin* School of Information

More information

Flashed-Optimized VPSA. Always Aligned with your Changing World

Flashed-Optimized VPSA. Always Aligned with your Changing World Flashed-Optimized VPSA Always Aligned with your Changing World Yair Hershko Co-founder, VP Engineering, Zadara Storage 3 Modern Data Storage for Modern Computing Innovating data services to meet modern

More information

WORT: Write Optimal Radix Tree for Persistent Memory Storage Systems

WORT: Write Optimal Radix Tree for Persistent Memory Storage Systems WORT: Write Optimal Radix Tree for Persistent Memory Storage Systems Se Kwon Lee K. Hyun Lim 1, Hyunsub Song, Beomseok Nam, Sam H. Noh UNIST 1 Hongik University Persistent Memory (PM) Persistent memory

More information

BCStore: Bandwidth-Efficient In-memory KV-Store with Batch Coding. Shenglong Li, Quanlu Zhang, Zhi Yang and Yafei Dai Peking University

BCStore: Bandwidth-Efficient In-memory KV-Store with Batch Coding. Shenglong Li, Quanlu Zhang, Zhi Yang and Yafei Dai Peking University BCStore: Bandwidth-Efficient In-memory KV-Store with Batch Coding Shenglong Li, Quanlu Zhang, Zhi Yang and Yafei Dai Peking University Outline Introduction and Motivation Our Design System and Implementation

More information

ChunkStash: Speeding Up Storage Deduplication using Flash Memory

ChunkStash: Speeding Up Storage Deduplication using Flash Memory ChunkStash: Speeding Up Storage Deduplication using Flash Memory Biplob Debnath +, Sudipta Sengupta *, Jin Li * * Microsoft Research, Redmond (USA) + Univ. of Minnesota, Twin Cities (USA) Deduplication

More information

Reducing Solid-State Storage Device Write Stress Through Opportunistic In-Place Delta Compression

Reducing Solid-State Storage Device Write Stress Through Opportunistic In-Place Delta Compression Reducing Solid-State Storage Device Write Stress Through Opportunistic In-Place Delta Compression Xuebin Zhang, Jiangpeng Li, Hao Wang, Kai Zhao and Tong Zhang xuebinzhang.rpi@gmail.com ECSE Department,

More information

p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer

p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer Wei Wang, Youyou Lu, and Jiwu Shu Department of Computer Science and Technology, Tsinghua University, Beijing, China Tsinghua National

More information

January 28-29, 2014 San Jose

January 28-29, 2014 San Jose January 28-29, 2014 San Jose Flash for the Future Software Optimizations for Non Volatile Memory Nisha Talagala, Lead Architect, Fusion-io Gary Orenstein, Chief Marketing Officer, Fusion-io @garyorenstein

More information

Purity: building fast, highly-available enterprise flash storage from commodity components

Purity: building fast, highly-available enterprise flash storage from commodity components Purity: building fast, highly-available enterprise flash storage from commodity components J. Colgrove, J. Davis, J. Hayes, E. Miller, C. Sandvig, R. Sears, A. Tamches, N. Vachharajani, and F. Wang 0 Gala

More information

Algorithms and Data Structures for Efficient Free Space Reclamation in WAFL

Algorithms and Data Structures for Efficient Free Space Reclamation in WAFL Algorithms and Data Structures for Efficient Free Space Reclamation in WAFL Ram Kesavan Technical Director, WAFL NetApp, Inc. SDC 2017 1 Outline Garbage collection in WAFL Usenix FAST 2017 ACM Transactions

More information

SSD WRITE AMPLIFICATION

SSD WRITE AMPLIFICATION SSD WRITE AMPLIFICATION Application Note Document #AN0035 Viking Rev. A Table of Contents 1 INTRODUCTION AND DEFINITION OF WRITE AMPLIFICATION 3 2 FACTORS AFFECTING WRITE AMPLIFICATION (WA) 3 3 ESTIMATION

More information

Partitioned Real-Time NAND Flash Storage. Katherine Missimer and Rich West

Partitioned Real-Time NAND Flash Storage. Katherine Missimer and Rich West Partitioned Real-Time NAND Flash Storage Katherine Missimer and Rich West Introduction Eric Risberg AP CircuitsToday 2 Introduction Eric Risberg AP CircuitsToday Analytics Vidhya 3 Chesky_W Mapping Ignorance

More information

Data Organization and Processing

Data Organization and Processing Data Organization and Processing Indexing Techniques for Solid State Drives (NDBI007) David Hoksza http://siret.ms.mff.cuni.cz/hoksza Outline SSD technology overview Motivation for standard algorithms

More information

Storage Systems : Disks and SSDs. Manu Awasthi CASS 2018

Storage Systems : Disks and SSDs. Manu Awasthi CASS 2018 Storage Systems : Disks and SSDs Manu Awasthi CASS 2018 Why study storage? Scalable High Performance Main Memory System Using Phase-Change Memory Technology, Qureshi et al, ISCA 2009 Trends Total amount

More information

Optimizing Software-Defined Storage for Flash Memory

Optimizing Software-Defined Storage for Flash Memory Optimizing Software-Defined Storage for Flash Memory Avraham Meir Chief Scientist, Elastifile Santa Clara, CA 1 Agenda What is SDS? SDS Media Selection Flash-Native SDS Non-Flash SDS File system benefits

More information

Don t stack your Log on my Log

Don t stack your Log on my Log Don t stack your Log on my Log Jingpei Yang, Ned Plasson, Greg Gillis, Nisha Talagala, Swaminathan Sundararaman Oct 5, 2014 c 1 Outline Introduction Log-stacking models Problems with stacking logs Solutions

More information

Bigtable. A Distributed Storage System for Structured Data. Presenter: Yunming Zhang Conglong Li. Saturday, September 21, 13

Bigtable. A Distributed Storage System for Structured Data. Presenter: Yunming Zhang Conglong Li. Saturday, September 21, 13 Bigtable A Distributed Storage System for Structured Data Presenter: Yunming Zhang Conglong Li References SOCC 2010 Key Note Slides Jeff Dean Google Introduction to Distributed Computing, Winter 2008 University

More information

SHRD: Improving Spatial Locality in Flash Storage Accesses by Sequentializing in Host and Randomizing in Device

SHRD: Improving Spatial Locality in Flash Storage Accesses by Sequentializing in Host and Randomizing in Device SHRD: Improving Spatial Locality in Flash Storage Accesses by Sequentializing in Host and Randomizing in Device Hyukjoong Kim 1, Dongkun Shin 1, Yun Ho Jeong 2 and Kyung Ho Kim 2 1 Samsung Electronics

More information

SSD Garbage Collection Detection and Management with Machine Learning Algorithm 1

SSD Garbage Collection Detection and Management with Machine Learning Algorithm 1 , pp.197-206 http//dx.doi.org/10.14257/ijca.2018.11.4.18 SSD Garbage Collection Detection and Management with Machine Learning Algorithm 1 Jung Kyu Park 1 and Jaeho Kim 2* 1 Department of Computer Software

More information

Scalable and Secure Internet Services and Architecture PRESENTATION REPORT Semester: Winter 2015 Course: ECE 7650

Scalable and Secure Internet Services and Architecture PRESENTATION REPORT Semester: Winter 2015 Course: ECE 7650 Scalable and Secure Internet Services and Architecture PRESENTATION REPORT Semester: Winter 2015 Course: ECE 7650 SUBMITTED BY: Yashwanth Boddu fq9316@wayne.edu (1) Our base design uses less than 1 byte

More information

Moneta: A High-performance Storage Array Architecture for Nextgeneration, Micro 2010

Moneta: A High-performance Storage Array Architecture for Nextgeneration, Micro 2010 Moneta: A High-performance Storage Array Architecture for Nextgeneration, Non-volatile Memories Micro 2010 NVM-based SSD NVMs are replacing spinning-disks Performance of disks has lagged NAND flash showed

More information

Solid State Drives (SSDs) Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University

Solid State Drives (SSDs) Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University Solid State Drives (SSDs) Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Memory Types FLASH High-density Low-cost High-speed Low-power High reliability

More information

vsan 6.6 Performance Improvements First Published On: Last Updated On:

vsan 6.6 Performance Improvements First Published On: Last Updated On: vsan 6.6 Performance Improvements First Published On: 07-24-2017 Last Updated On: 07-28-2017 1 Table of Contents 1. Overview 1.1.Executive Summary 1.2.Introduction 2. vsan Testing Configuration and Conditions

More information

Speeding Up Cloud/Server Applications Using Flash Memory

Speeding Up Cloud/Server Applications Using Flash Memory Speeding Up Cloud/Server Applications Using Flash Memory Sudipta Sengupta and Jin Li Microsoft Research, Redmond, WA, USA Contains work that is joint with Biplob Debnath (Univ. of Minnesota) Flash Memory

More information

LSbM-tree: Re-enabling Buffer Caching in Data Management for Mixed Reads and Writes

LSbM-tree: Re-enabling Buffer Caching in Data Management for Mixed Reads and Writes 27 IEEE 37th International Conference on Distributed Computing Systems LSbM-tree: Re-enabling Buffer Caching in Data Management for Mixed Reads and Writes Dejun Teng, Lei Guo, Rubao Lee, Feng Chen, Siyuan

More information

Design of Global Data Deduplication for A Scale-out Distributed Storage System

Design of Global Data Deduplication for A Scale-out Distributed Storage System 218 IEEE 38th International Conference on Distributed Computing Systems Design of Global Data Deduplication for A Scale-out Distributed Storage System Myoungwon Oh, Sejin Park, Jungyeon Yoon, Sangjae Kim,

More information

Mass-Storage Structure

Mass-Storage Structure Operating Systems (Fall/Winter 2018) Mass-Storage Structure Yajin Zhou (http://yajin.org) Zhejiang University Acknowledgement: some pages are based on the slides from Zhi Wang(fsu). Review On-disk structure

More information

Optimizing Flash-based Key-value Cache Systems

Optimizing Flash-based Key-value Cache Systems Optimizing Flash-based Key-value Cache Systems Zhaoyan Shen, Feng Chen, Yichen Jia, Zili Shao Department of Computing, Hong Kong Polytechnic University Computer Science & Engineering, Louisiana State University

More information

PowerVault MD3 SSD Cache Overview

PowerVault MD3 SSD Cache Overview PowerVault MD3 SSD Cache Overview A Dell Technical White Paper Dell Storage Engineering October 2015 A Dell Technical White Paper TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS

More information

HPDedup: A Hybrid Prioritized Data Deduplication Mechanism for Primary Storage in the Cloud

HPDedup: A Hybrid Prioritized Data Deduplication Mechanism for Primary Storage in the Cloud HPDedup: A Hybrid Prioritized Data Deduplication Mechanism for Primary Storage in the Cloud Huijun Wu 1,4, Chen Wang 2, Yinjin Fu 3, Sherif Sakr 1, Liming Zhu 1,2 and Kai Lu 4 The University of New South

More information

NVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory

NVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory NVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory Dhananjoy Das, Sr. Systems Architect SanDisk Corp. 1 Agenda: Applications are KING! Storage landscape (Flash / NVM)

More information

Example. program sort; var a : array[0..10] of integer; procedure readarray; : function partition (y, z :integer) :integer; var i, j,x, v :integer; :

Example. program sort; var a : array[0..10] of integer; procedure readarray; : function partition (y, z :integer) :integer; var i, j,x, v :integer; : Runtime Environment Relationship between names and data objects (of target machine) Allocation & de-allocation is managed by run time support package Each execution of a procedure is an activation of the

More information

SWAP: EFFECTIVE FINE-GRAIN MANAGEMENT

SWAP: EFFECTIVE FINE-GRAIN MANAGEMENT : EFFECTIVE FINE-GRAIN MANAGEMENT OF SHARED LAST-LEVEL CACHES WITH MINIMUM HARDWARE SUPPORT Xiaodong Wang, Shuang Chen, Jeff Setter, and José F. Martínez Computer Systems Lab Cornell University Page 1

More information

GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction

GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction Ting Yao, Huazhong University of Science and Technology and Temple University; Jiguang Wan, Huazhong University of Science and Technology;

More information

Page Mapping Scheme to Support Secure File Deletion for NANDbased Block Devices

Page Mapping Scheme to Support Secure File Deletion for NANDbased Block Devices Page Mapping Scheme to Support Secure File Deletion for NANDbased Block Devices Ilhoon Shin Seoul National University of Science & Technology ilhoon.shin@snut.ac.kr Abstract As the amount of digitized

More information

Flash Memory Based Storage System

Flash Memory Based Storage System Flash Memory Based Storage System References SmartSaver: Turning Flash Drive into a Disk Energy Saver for Mobile Computers, ISLPED 06 Energy-Aware Flash Memory Management in Virtual Memory System, islped

More information

The Logic of Physical Garbage Collection in Deduplicating Storage

The Logic of Physical Garbage Collection in Deduplicating Storage The Logic of Physical Garbage Collection in Deduplicating Storage Fred Douglis Abhinav Duggal Philip Shilane Tony Wong Dell EMC Shiqin Yan University of Chicago Fabiano Botelho Rubrik 1 Deduplication in

More information

Turning Object. Storage into Virtual Machine Storage. White Papers

Turning Object. Storage into Virtual Machine Storage. White Papers Turning Object Open vstorage is the World s fastest Distributed Block Store that spans across different Datacenter. It combines ultrahigh performance and low latency connections with a data integrity that

More information

Configuring EMC Isilon

Configuring EMC Isilon This chapter contains the following sections: System, page 1 Configuring SMB Shares, page 3 Creating an NFS Export, page 5 Configuring Quotas, page 6 Creating a Group for the Isilon Cluster, page 8 Creating

More information

SmartSaver: Turning Flash Drive into a Disk Energy Saver for Mobile Computers

SmartSaver: Turning Flash Drive into a Disk Energy Saver for Mobile Computers SmartSaver: Turning Flash Drive into a Disk Energy Saver for Mobile Computers Feng Chen 1 Song Jiang 2 Xiaodong Zhang 1 The Ohio State University, USA Wayne State University, USA Disks Cost High Energy

More information

SSDs vs HDDs for DBMS by Glen Berseth York University, Toronto

SSDs vs HDDs for DBMS by Glen Berseth York University, Toronto SSDs vs HDDs for DBMS by Glen Berseth York University, Toronto So slow So cheap So heavy So fast So expensive So efficient NAND based flash memory Retains memory without power It works by trapping a small

More information

Why Datrium DVX is Best for VDI

Why Datrium DVX is Best for VDI Why Datrium DVX is Best for VDI 385 Moffett Park Dr. Sunnyvale, CA 94089 844-478-8349 www.datrium.com Technical Report Introduction Managing a robust and growing virtual desktop infrastructure in current

More information

Understanding Data Locality in VMware vsan First Published On: Last Updated On:

Understanding Data Locality in VMware vsan First Published On: Last Updated On: Understanding Data Locality in VMware vsan First Published On: 07-20-2016 Last Updated On: 09-30-2016 1 Table of Contents 1. Understanding Data Locality in VMware vsan 1.1.Introduction 1.2.vSAN Design

More information

LevelDB-Raw: Eliminating File System Overhead for Optimizing Performance of LevelDB Engine

LevelDB-Raw: Eliminating File System Overhead for Optimizing Performance of LevelDB Engine 777 LevelDB-Raw: Eliminating File System Overhead for Optimizing Performance of LevelDB Engine Hak-Su Lim and Jin-Soo Kim *College of Info. & Comm. Engineering, Sungkyunkwan University, Korea {haksu.lim,

More information

ECE 598-MS: Advanced Memory and Storage Systems Lecture 7: Unified Address Translation with FlashMap

ECE 598-MS: Advanced Memory and Storage Systems Lecture 7: Unified Address Translation with FlashMap ECE 598-MS: Advanced Memory and Storage Systems Lecture 7: Unified Address Translation with Map Jian Huang Use As Non-Volatile Memory DRAM (nanoseconds) Application Memory Component SSD (microseconds)

More information

JULEA: A Flexible Storage Framework for HPC

JULEA: A Flexible Storage Framework for HPC JULEA: A Flexible Storage Framework for HPC Workshop on Performance and Scalability of Storage Systems Michael Kuhn Research Group Scientific Computing Department of Informatics Universität Hamburg 2017-06-22

More information

WiscKey: Separating Keys from Values in SSD-Conscious Storage

WiscKey: Separating Keys from Values in SSD-Conscious Storage WiscKey: Separating Keys from Values in SSD-Conscious Storage LANYUE LU, THANUMALAYAN SANKARANARAYANA PILLAI, HARIHARAN GOPALAKRISHNAN, ANDREA C. ARPACI-DUSSEAU, and REMZI H. ARPACI-DUSSEAU, University

More information

FlashKV: Accelerating KV Performance with Open-Channel SSDs

FlashKV: Accelerating KV Performance with Open-Channel SSDs FlashKV: Accelerating KV Performance with Open-Channel SSDs JIACHENG ZHANG, YOUYOU LU, JIWU SHU, and XIONGJUN QIN, Department of Computer Science and Technology, Tsinghua University As the cost-per-bit

More information

Virtual Desktop Infrastructure (VDI) Bassam Jbara

Virtual Desktop Infrastructure (VDI) Bassam Jbara Virtual Desktop Infrastructure (VDI) Bassam Jbara 1 VDI Historical Overview Desktop virtualization is a software technology that separates the desktop environment and associated application software from

More information

NAND Flash-based Storage. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University

NAND Flash-based Storage. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University NAND Flash-based Storage Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics NAND flash memory Flash Translation Layer (FTL) OS implications

More information

NetApp Data Compression, Deduplication, and Data Compaction

NetApp Data Compression, Deduplication, and Data Compaction Technical Report NetApp Data Compression, Deduplication, and Data Compaction Data ONTAP 8.3.1 and Later Karthik Viswanath, NetApp February 2018 TR-4476 Abstract This technical report focuses on implementing

More information

JOURNALING techniques have been widely used in modern

JOURNALING techniques have been widely used in modern IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, XXXX 2018 1 Optimizing File Systems with a Write-efficient Journaling Scheme on Non-volatile Memory Xiaoyi Zhang, Dan Feng, Member, IEEE, Yu Hua, Senior

More information

EMC Isilon. Cisco UCS Director Support for EMC Isilon

EMC Isilon. Cisco UCS Director Support for EMC Isilon Cisco UCS Director Support for, page 1 Adding an Account, page 2 Storage Pool Tiers, page 3 Storage Node Pools, page 4 SMB Shares, page 5 Creating an NFS Export, page 7 Quotas, page 9 Configuring a space

More information

CHAPTER 7 CONCLUSION AND FUTURE WORK

CHAPTER 7 CONCLUSION AND FUTURE WORK CHAPTER 7 CONCLUSION AND FUTURE WORK 7.1 Conclusion Data pre-processing is very important in data mining process. Certain data cleaning techniques usually are not applicable to all kinds of data. Deduplication

More information

Toward SLO Complying SSDs Through OPS Isolation

Toward SLO Complying SSDs Through OPS Isolation Toward SLO Complying SSDs Through OPS Isolation October 23, 2015 Hongik University UNIST (Ulsan National Institute of Science & Technology) Sam H. Noh 1 Outline Part 1: FAST 2015 Part 2: Beyond FAST 2

More information

Storage Architecture and Software Support for SLC/MLC Combined Flash Memory

Storage Architecture and Software Support for SLC/MLC Combined Flash Memory Storage Architecture and Software Support for SLC/MLC Combined Flash Memory Soojun Im and Dongkun Shin Sungkyunkwan University Suwon, Korea {lang33, dongkun}@skku.edu ABSTRACT We propose a novel flash

More information

Asynchronous Logging and Fast Recovery for a Large-Scale Distributed In-Memory Storage

Asynchronous Logging and Fast Recovery for a Large-Scale Distributed In-Memory Storage Asynchronous Logging and Fast Recovery for a Large-Scale Distributed In-Memory Storage Kevin Beineke, Florian Klein, Michael Schöttner Institut für Informatik, Heinrich-Heine-Universität Düsseldorf Outline

More information

Hardware-Supported Pointer Detection for common Garbage Collections

Hardware-Supported Pointer Detection for common Garbage Collections 2013 First International Symposium on Computing and Networking Hardware-Supported Pointer Detection for common Garbage Collections Kei IDEUE, Yuki SATOMI, Tomoaki TSUMURA and Hiroshi MATSUO Nagoya Institute

More information

Deduplication File System & Course Review

Deduplication File System & Course Review Deduplication File System & Course Review Kai Li 12/13/13 Topics u Deduplication File System u Review 12/13/13 2 Storage Tiers of A Tradi/onal Data Center $$$$ Mirrored storage $$$ Dedicated Fibre Clients

More information

Overprovisioning and the SanDisk X400 SSD

Overprovisioning and the SanDisk X400 SSD and the SanDisk X400 SSD Improving Performance and Endurance with Rev 1.2 October 2016 CSS Technical Marketing Western Digital Technologies, Inc. 951 SanDisk Dr. Milpitas, CA 95035 Phone (408) 801-1000

More information

SolidFire and Ceph Architectural Comparison

SolidFire and Ceph Architectural Comparison The All-Flash Array Built for the Next Generation Data Center SolidFire and Ceph Architectural Comparison July 2014 Overview When comparing the architecture for Ceph and SolidFire, it is clear that both

More information

[537] Flash. Tyler Harter

[537] Flash. Tyler Harter [537] Flash Tyler Harter Flash vs. Disk Disk Overview I/O requires: seek, rotate, transfer Inherently: - not parallel (only one head) - slow (mechanical) - poor random I/O (locality around disk head) Random

More information

White Paper: Understanding the Relationship Between SSD Endurance and Over-Provisioning. Solid State Drive

White Paper: Understanding the Relationship Between SSD Endurance and Over-Provisioning. Solid State Drive White Paper: Understanding the Relationship Between SSD Endurance and Over-Provisioning Solid State Drive 2 Understanding the Relationship Between Endurance and Over-Provisioning Each of the cells inside

More information