Disk Cache-Aware Task Scheduling

Size: px
Start display at page:

Download "Disk Cache-Aware Task Scheduling"

Transcription

1 Disk ache-aware Task Scheduling For Data-Intensive and Many-Task Workflow Masahiro Tanaka and Osamu Tatebe University of Tsukuba, JST/REST Japan Science and Technology Agency IEEE luster

2 Outline Introduction Workflow Scheduling for Data-Intensive and Many-Task omputing Disk ache-aware Task Scheduling Proposed Method (1) LIFO + HRF (2) Rank Equalization + HRF Evaluation (1) I/O-only workflow (2) Montage astronomy workflow Related Work onclusion IEEE luster

3 Background Many Task omputing (MT) Raicu et al. (MTAGS 2008) Task throughput for tasks Pwrake : Parallel Workflow extension to Rake Rake = Ruby version of make Target: Data-intensive and Many-task Workflows Gfarm distributed file system Scalable I/O performance Use local storage of compute nodes IEEE luster

4 Objective: I/O-aware Task Scheduling Issues: File Locality our previous work Disk cache (buffer/page cache) Read performance of Gfarm file (HDD, GbE) this work process MB/s disk cache Local file cache file disk Gfarm file cache file disk Remote 0 Remote Local IEEE luster

5 Architecture of Pwrake Master node Worker nodes Pwrake process Task Graph worker thread Task process process files files Queue worker thread SSH Gfarm worker thread enq deq worker thread process files process files IEEE luster

6 Task Queue of Pwrake Task NodeQueue TaskQueue enq Node 1 Node 2 deq worker thread Node 3 Other nodes IEEE luster

7 Method to define andidate Nodes 1. Based on the location of input file 47 % local access 2. MGP (Multi-onstraint Graph Partitioning) Our previous work (Grid 2012) use METIS library 88 % local access IEEE luster

8 Disk ache-aware Task Scheduling Later-saved file has high probability that the file is cached. The order of task execution relates to Disk ache hit rate. In the following slides, I show the behavior of FIFO and LIFO queues. IEEE luster

9 FIFO behavior Workflow DAG Last NodeQueue First B1 B2 B3 B4 B5 B6 IEEE luster

10 FIFO behavior Workflow DAG Last NodeQueue First B1 B2 B3 B4 B5 B6 IEEE luster

11 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 time B1 B2 B3 B4 B5 B6 IEEE luster

12 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 time B1 B2 B3 B4 B5 B6 Storage of compute node cache disk file2 file1 file2 file1 IEEE luster

13 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B2 B1 time B1 B2 B3 B4 B5 B6 cache file2 file1 disk file2 file1 IEEE luster

14 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B2 B1 time B1 B2 B3 B4 B5 B6 cache file2 file1 disk file2 file1 IEEE luster

15 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B2 B1 time B1 B2 B3 B4 B5 B6 cache file4 file3 file2 file1 disk file4 file3 file2 file1 IEEE luster

16 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B4 B3 B2 B1 time B1 B2 B3 B4 B5 B6 cache file4 file3 file2 file1 disk file4 file3 file2 file1 IEEE luster

17 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B4 B3 B2 B1 time B1 B2 B3 B4 B5 B6 cache file4 file3 file2 file1 disk file4 file3 file2 file1 IEEE luster

18 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B4 B3 B2 B1 time B1 B2 B3 B4 B5 B6 cache file6 file5 file4 file3 file2 file1 disk file6 file5 file4 file3 file2 file1 Evicted IEEE luster

19 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 B4 B3 B2 B1 time B1 B2 B3 B4 B5 B6 cache file6 file5 file4 file3 file2 file1 disk file6 file5 file4 file3 file2 file1 IEEE luster

20 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 B4 B3 time B1 B2 B3 B4 B5 B6 B1 B2 cache file6 file5 file4 file3 file2 file1 disk file6 file5 file4 file3 file2 file1 IEEE luster

21 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 B4 B3 time B1 B2 B3 B4 B5 B6 B1 B2 B2 B1 cache file6 file5 file4 file3 file2 file1 disk file6 file5 file4 file3 file2 file1 Evicted IEEE luster

22 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 time B3 B4 B5 B6 B1 B2 B2 B1 B4 B3 cache file2b file1b file6 file5 file4 file3 disk file2b file1b file6 file5 file4 file3 B3 B4 Evicted IEEE luster

23 FIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 time B5 B6 B1 B2 B4 B3 B6 B5 cache file4b file3b file2b file1b file6 file5 disk file4b file4b file2b file1b file6 file5 B3 B5 B4 B6 Evicted IEEE luster

24 FIFO behavior Workflow DAG ore allocation core1 core2 File output time B1 B2 B3 B4 B5 B6 File input B1 B2 B3 B4 Average difference between A i B i ~ # of A tasks # of cores tasks B5 B6 IEEE luster

25 LIFO behavior Workflow DAG Last NodeQueue First B1 B2 B3 B4 B5 B6 IEEE luster

26 LIFO behavior Workflow DAG Last NodeQueue First B1 B2 B3 B4 B5 B6 IEEE luster

27 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 time B1 B2 B3 B4 B5 B6 Storage of compute node cache disk file5 file6 file5 file6 IEEE luster

28 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 time B1 B2 B3 B4 B5 B6 cache file5 file6 disk file5 file6 IEEE luster

29 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 time B1 B2 B3 B4 B5 B6 cache disk B5 B6 file5 file6 file5 file6 IEEE luster

30 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 time B1 B2 B3 B4 B5 B6 B5 B6 cache file5b file6b file5 file6 disk file5b file6b file5 file6 IEEE luster

31 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 time B1 B2 B3 B4 cache file3 file4 file5b file6b file5 file6 disk file3 file4 file5b file6b file5 file6 Evicted IEEE luster

32 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B4 B3 B6 B5 time B1 B2 B3 B4 cache file3 file4 file5b file6b file5 file6 disk file3 file4 file5b file6b file5 file6 IEEE luster

33 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 time B1 B2 B3 B4 B4 B3 B3 B4 cache file3 file4 file5b file6b file5 file6 disk file3 file4 file5b file6b file5 file6 IEEE luster

34 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 time B1 B2 B4 B3 cache file1 file2 file3b file4b file3 file4 disk file1 file2 file3b file4b file3 file4 IEEE luster

35 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B2 B1 B6 B5 time B1 B2 B4 B3 cache file1 file2 file3b file4b file3 file4 disk file1 file2 file3b file4b file3 file4 IEEE luster

36 LIFO behavior Workflow DAG Last NodeQueue First ore allocation core1 core2 B6 B5 time B1 B2 B4 B3 B1 B2 cache file1 file2 file3b file4b file3 file4 disk file1 file2 file3b file4b file3 file4 B2 B1 IEEE luster

37 LIFO behavior Workflow DAG B1 B2 B3 B4 B5 B6 File output File input ore allocation core1 core2 B6 B5 time Average difference between A i B i ~ 1 task B4 B2 B3 B1 IEEE luster

38 Trailing Task Problem Armstrong et al. MTAGS 2010 Idle cores in final phase FIFO: max Task B LIFO: max Task A+B FIFO has advantage in trailing task problem Bn-6 Bn-4 Bn-2 Bn : FIFO Bn-5 Bn-3 Bn-1 idle core An-2 Bn-2 An Bn : LIFO An-1 Bn-1 idle core IEEE luster

39 FIFO for Trailing Task Problem? FIFO is better for trailing task problem than LIFO. FIFO s weak point 4 and D4 can be trailing tasks. and B4 should have higher priority than -3, B1-3. B1 B2 B3 E B4 4 D4 IEEE luster

40 Highest Rank First (HRF) Rank: Distance from the last task Same as upward rank in HEFT algorithm except uniform task cost Rank 2 Rank 1.. An B1 B2.. Bn priority high Highest Rank First (HRF) Priority to higher-rank tasks Rank 0 low (FIFO: downward rank ) Solution for Trailing Tasks Problem Bad for Disk ache IEEE luster

41 Proposed Methods (1) LIFO + HRF (2) Rank Equilization + HRF IEEE luster

42 Proposed method (1) LIFO+HRF Nc : # cores/node Nr : # tasks in the highest rank Algorithm: LIFO if Nr > Nc HRF if Nr Nc B1 B3 : B2 B4 LIFO An-2 Bn-2 An-1 An HRF Bn-1 Bn IEEE luster

43 Proposed method (2) Rank Equalization+HRF Purpose: B1 Overlap compute and I/O B2 B3 Bn-4 : An-2 Rank Equalization Task A: ompute intensive Task B: I/O intensive An-1 Bn-1 Bn-2 An Bn Bn-3 HRF IEEE luster

44 Rank Equalization Nr > Nc: Enq: Store tasks to RankQueue Deq: Select RankQueue with different frequency: enq NodeQueue RankQueue Rank1 Rank2 w[1] w[2] deq w[r] = 1/(average execution time) task invocation frequency Deq a task in LIFO policy. Nr Nc HRF Rank 1 Rank 2 t 1 Task B Task B Task B Task B Task A Task A Task A t 2 time # of tasks/sec 1/t 1 1/t 2 IEEE luster

45 Summary of Scheduling methods FIFO HRF LIFO Rank Eq. (w/ LIFO) Disk ache Trailing Task Rank Overlap LIFO+HRF Rank Eq.+HRF IEEE luster

46 Performance Evaluation IEEE luster

47 Evaluation Environment luster InTrigger Tohoku site PU Intel Xeon E GHz Main Memory 32 GiB # of cores / node 8 Max # of compute node 12 Network 1Gb Ethernet OS Debian Gfarm ver Ruby ver Pwrake ver IEEE luster

48 Evaluation-1: opyfile workload opyfile : I/O intensive workload Load an input file to main memory and write it to an output file. Workflow DAG Task A,B = opyfile program.. An B1 B2 B3.. Bn Scheduling FIFO, LIFO (no +HRF because of 1 core experiment) Input file Mem number n=100 one file size 3 GiB < 32 GiB total size 300 GiB > 32 GiB Used nodes 10 nodes 1 core/node IEEE luster

49 Elapsed Time of opyfile workflow 2500 Elapsed Time (sec) FIFO Estimated time = I/R + O/W 30% speedup measured estimated LIFO I, O = Input, Output filesize R, W = Read, Write bandwidth (depends on access target) IEEE luster

50 Evaluation-2: Montage Wowkflow Astronomy image processing DAG mprojectpp mdiff+mfitplane mbgmodel mbackground madd mshrink madd mjpeg Number of cores: 96 (12 nodes x 8 cores) Input file SDSS DR7 # of input files 421 Size of one input file 2.52 MB Size of total input files 1061 MB # of intermediary files 4720 Size of intermediary files 63.5 GB # of tasks 2707 Output file size of mprojectpp 12 (nnnnnnnnnn) 1.6 GB IEEE luster

51 Measurement of Strong Scaling (1-12 nodes, Logarithmic) 1node 8cores x1.9 speedup from FIFO 1/n cores 4nodes 32cores Next Slide IEEE luster

52 Measurement of Strong Scaling (4-12 nodes, Linear) 4nodes 32cores 12nodes 96cores 12 % speedup from LIFO (trailing task) 1/n cores IEEE luster

53 FIFO order Task Scheduling mprojectpp mdiff+mfitplane Sequential tasks IEEE luster

54 LIFO order Task Scheduling Trailing Tasks IEEE luster

55 LIFO+HRF No Trailing Tasks IEEE luster

56 LIFO+Rank Equalization IEEE luster

57 Result of 12-node experiment LIFO is the worst due to trailing task problem Other methods are comparable. Why FIFO is good? Data size per node < ache size Benefit of cache even in FIFO case. Why Rank Equalization is not better? In LIFO+HRF case, mdiff is always less than 50% core utilization. mdiff always overlaps with mprojectpp. IEEE luster

58 LIFO+HRF (non MGP) >50 %, No overlap IEEE luster

59 LIFO+Rank Equalization (non-mgp) mdiff overlaps mprojectpp elapsed time: s s (4.4 % speedup) IEEE luster

60 Related Work 1: Workflow System Swift + Falkon + data diffusion Swift (Wilde et al. 2011) Workflow language Falkon (Raicu et al. 2007) Task throughput (1500 tasks/sec) data diffusion (Raicu et al. 2008) Staging file management + Task scheduling GXP make (Taura et al. 2013) Workflow system based on GNU make Dispatches tasks invoked by GNU make IEEE luster

61 Related Work 2: Workflow Scheduling Shankar and DeWitt (HPD 2007) studied DAG-based data-aware workflow scheduling for the ondor system. focused on cached data on a local disk in order to avoid data movement. Armstrong et al. (MTAGS 2010) discussed trailing task problem. proposed tail-chopping approach. IEEE luster

62 onclusion Objective: Disk cache aware task scheduling for Data-intensive and Many-task workflow. Proposed methods: LIFO+HRF: Rank Equalization+HRF: Evaluation: opyfile workflow: LIFO: ~30% speedup Montage Workflow: LIFO: 1.9x speedup HRF: ~12% speedup Rank Equalization: ~4% speedup IEEE luster

63 Backup slides IEEE luster

64 ache FIFO and LIFO.. An core1 core2 core1 core2 B1 B2 B3.. Bn time : : B1 B2 An-2 An-1 B3 B4 An B1 A i B i difference (average) FIFO: n/2 tasks LIFO: 1 tasks LIFO is good for ache utilization B2 B3 : : Bn-3 Bn-2 Bn-1 Bn FIFO B5 B6 : : An-1 An Bn-1 Bn LIFO IEEE luster

65 Elapsed time and ore utilization (12 nodes) t elap = Elapsed time of workflow t cum = Summated time of all the tasks ore utilization = t cum /(t elap n cores ) Elapsed time (sec) ore utilization 100% 80% 60% 40% 20% 87.1% 84.0% 87.6% 87.8% 0 0% IEEE luster

Workflow System for Data-intensive Many-task Computing

Workflow System for Data-intensive Many-task Computing Workflow System for Data-intensive Many-task Computing Masahiro Tanaka and Osamu Tatebe University of Tsukuba, JST/CREST Japan Science and Technology Agency ISP2S2 Dec 4, 2014 1 Outline Background Pwrake

More information

System Software for Big Data and Post Petascale Computing

System Software for Big Data and Post Petascale Computing The Japanese Extreme Big Data Workshop February 26, 2014 System Software for Big Data and Post Petascale Computing Osamu Tatebe University of Tsukuba I/O performance requirement for exascale applications

More information

NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC

NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC Segregated storage and compute NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC Co-located storage and compute HDFS, GFS Data

More information

A Data Diffusion Approach to Large Scale Scientific Exploration

A Data Diffusion Approach to Large Scale Scientific Exploration A Data Diffusion Approach to Large Scale Scientific Exploration Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago Joint work with: Yong Zhao: Microsoft Ian Foster:

More information

NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC

NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC Segregated storage and compute NFS, GPFS, PVFS, Lustre Batch-scheduled systems: Clusters, Grids, and Supercomputers Programming paradigm: HPC, MTC, and HTC Co-located storage and compute HDFS, GFS Data

More information

Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago

Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago Running 1 Million Jobs in 10 Minutes via the Falkon Fast and Light-weight Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago In Collaboration with: Ian Foster,

More information

Managing and Executing Loosely-Coupled Large-Scale Applications on Clusters, Grids, and Supercomputers

Managing and Executing Loosely-Coupled Large-Scale Applications on Clusters, Grids, and Supercomputers Managing and Executing Loosely-Coupled Large-Scale Applications on Clusters, Grids, and Supercomputers Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago Collaborators:

More information

Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago

Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago Falkon, a Fast and Light-weight task execution framework for Clusters, Grids, and Supercomputers Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago In Collaboration

More information

Accelerating Large Scale Scientific Exploration through Data Diffusion

Accelerating Large Scale Scientific Exploration through Data Diffusion Accelerating Large Scale Scientific Exploration through Data Diffusion Ioan Raicu *, Yong Zhao *, Ian Foster #*+, Alex Szalay - {iraicu,yongzh }@cs.uchicago.edu, foster@mcs.anl.gov, szalay@jhu.edu * Department

More information

Synonymous with supercomputing Tightly-coupled applications Implemented using Message Passing Interface (MPI) Large of amounts of computing for short

Synonymous with supercomputing Tightly-coupled applications Implemented using Message Passing Interface (MPI) Large of amounts of computing for short Synonymous with supercomputing Tightly-coupled applications Implemented using Message Passing Interface (MPI) Large of amounts of computing for short periods of time Usually requires low latency interconnects

More information

MOHA: Many-Task Computing Framework on Hadoop

MOHA: Many-Task Computing Framework on Hadoop Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction

More information

Typically applied in clusters and grids Loosely-coupled applications with sequential jobs Large amounts of computing for long periods of times

Typically applied in clusters and grids Loosely-coupled applications with sequential jobs Large amounts of computing for long periods of times Typically applied in clusters and grids Loosely-coupled applications with sequential jobs Large amounts of computing for long periods of times Measured in operations per month or years 2 Bridge the gap

More information

Optimizing Local File Accesses for FUSE-Based Distributed Storage

Optimizing Local File Accesses for FUSE-Based Distributed Storage Optimizing Local File Accesses for FUSE-Based Distributed Storage Shun Ishiguro 1, Jun Murakami 1, Yoshihiro Oyama 1,3, Osamu Tatebe 2,3 1. The University of Electro-Communications, Japan 2. University

More information

Dynamic Fine Grain Scheduling of Pipeline Parallelism. Presented by: Ram Manohar Oruganti and Michael TeWinkle

Dynamic Fine Grain Scheduling of Pipeline Parallelism. Presented by: Ram Manohar Oruganti and Michael TeWinkle Dynamic Fine Grain Scheduling of Pipeline Parallelism Presented by: Ram Manohar Oruganti and Michael TeWinkle Overview Introduction Motivation Scheduling Approaches GRAMPS scheduling method Evaluation

More information

Storage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore

Storage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore Storage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago DSL Seminar November st, 006 Analysis

More information

davidklee.net gplus.to/kleegeek linked.com/a/davidaklee

davidklee.net gplus.to/kleegeek linked.com/a/davidaklee @kleegeek davidklee.net gplus.to/kleegeek linked.com/a/davidaklee Specialties / Focus Areas / Passions: Performance Tuning & Troubleshooting Virtualization Cloud Enablement Infrastructure Architecture

More information

Falkon: a Fast and Light-weight task execution framework

Falkon: a Fast and Light-weight task execution framework : a Fast and Light-weight task execution framework Ioan Raicu *, Yong Zhao *, Catalin Dumitrescu *, Ian Foster #*+, Mike Wilde #+ * Department of Computer Science, University of Chicago, IL, USA + Computation

More information

Flash-Conscious Cache Population for Enterprise Database Workloads

Flash-Conscious Cache Population for Enterprise Database Workloads IBM Research ADMS 214 1 st September 214 Flash-Conscious Cache Population for Enterprise Database Workloads Hyojun Kim, Ioannis Koltsidas, Nikolas Ioannou, Sangeetha Seshadri, Paul Muench, Clem Dickey,

More information

Virtual SQL Servers. Actual Performance. 2016

Virtual SQL Servers. Actual Performance. 2016 @kleegeek davidklee.net heraflux.com linkedin.com/in/davidaklee Specialties / Focus Areas / Passions: Performance Tuning & Troubleshooting Virtualization Cloud Enablement Infrastructure Architecture Health

More information

Ioan Raicu. Distributed Systems Laboratory Computer Science Department University of Chicago

Ioan Raicu. Distributed Systems Laboratory Computer Science Department University of Chicago The Quest for Scalable Support of Data Intensive Applications in Distributed Systems Ioan Raicu Distributed Systems Laboratory Computer Science Department University of Chicago In Collaboration with: Ian

More information

Enlightening the I/O Path: A Holistic Approach for Application Performance

Enlightening the I/O Path: A Holistic Approach for Application Performance Enlightening the I/O Path: A Holistic Approach for Application Performance Sangwook Kim 13, Hwanju Kim 2, Joonwon Lee 3, and Jinkyu Jeong 3 Apposha 1 Dell EMC 2 Sungkyunkwan University 3 Data-Intensive

More information

Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work

Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Introduction & Motivation Problem Statement Proposed Work Evaluation Conclusions Future Work Today (2014):

More information

PARDA: Proportional Allocation of Resources for Distributed Storage Access

PARDA: Proportional Allocation of Resources for Distributed Storage Access PARDA: Proportional Allocation of Resources for Distributed Storage Access Ajay Gulati, Irfan Ahmad, Carl Waldspurger Resource Management Team VMware Inc. USENIX FAST 09 Conference February 26, 2009 The

More information

MATE-EC2: A Middleware for Processing Data with Amazon Web Services

MATE-EC2: A Middleware for Processing Data with Amazon Web Services MATE-EC2: A Middleware for Processing Data with Amazon Web Services Tekin Bicer David Chiu* and Gagan Agrawal Department of Compute Science and Engineering Ohio State University * School of Engineering

More information

Lecture 13 Input/Output (I/O) Systems (chapter 13)

Lecture 13 Input/Output (I/O) Systems (chapter 13) Bilkent University Department of Computer Engineering CS342 Operating Systems Lecture 13 Input/Output (I/O) Systems (chapter 13) Dr. İbrahim Körpeoğlu http://www.cs.bilkent.edu.tr/~korpe 1 References The

More information

Data Diffusion: Dynamic Resource Provision and Data-Aware Scheduling for Data-Intensive Applications

Data Diffusion: Dynamic Resource Provision and Data-Aware Scheduling for Data-Intensive Applications Data Diffusion: Dynamic Resource Provision and Data-Aware Scheduling for Data-Intensive Applications Ioan Raicu, Yong Zhao 2, Ian Foster,3,4, Alex Szalay 5 iraicu@cs.uchicago.edu, yozha@microsoft.com,

More information

Device-Functionality Progression

Device-Functionality Progression Chapter 12: I/O Systems I/O Hardware I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Incredible variety of I/O devices Common concepts Port

More information

Chapter 12: I/O Systems. I/O Hardware

Chapter 12: I/O Systems. I/O Hardware Chapter 12: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations I/O Hardware Incredible variety of I/O devices Common concepts Port

More information

Cloud Programming. Programming Environment Oct 29, 2015 Osamu Tatebe

Cloud Programming. Programming Environment Oct 29, 2015 Osamu Tatebe Cloud Programming Programming Environment Oct 29, 2015 Osamu Tatebe Cloud Computing Only required amount of CPU and storage can be used anytime from anywhere via network Availability, throughput, reliability

More information

Concurrent programming: From theory to practice. Concurrent Algorithms 2015 Vasileios Trigonakis

Concurrent programming: From theory to practice. Concurrent Algorithms 2015 Vasileios Trigonakis oncurrent programming: From theory to practice oncurrent Algorithms 2015 Vasileios Trigonakis From theory to practice Theoretical (design) Practical (design) Practical (implementation) 2 From theory to

More information

Multi-core Programming Evolution

Multi-core Programming Evolution Multi-core Programming Evolution Based on slides from Intel Software ollege and Multi-ore Programming increasing performance through software multi-threading by Shameem Akhter and Jason Roberts, Evolution

More information

Module 12: I/O Systems

Module 12: I/O Systems Module 12: I/O Systems I/O hardwared Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Performance 12.1 I/O Hardware Incredible variety of I/O devices Common

More information

Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching

Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching Cascade Mapping: Optimizing Memory Efficiency for Flash-based Key-value Caching Kefei Wang and Feng Chen Louisiana State University SoCC '18 Carlsbad, CA Key-value Systems in Internet Services Key-value

More information

Advances of parallel computing. Kirill Bogachev May 2016

Advances of parallel computing. Kirill Bogachev May 2016 Advances of parallel computing Kirill Bogachev May 2016 Demands in Simulations Field development relies more and more on static and dynamic modeling of the reservoirs that has come a long way from being

More information

Energy and Thermal Aware Buffer Cache Replacement Algorithm

Energy and Thermal Aware Buffer Cache Replacement Algorithm MSST 2010 nergy and Thermal Aware uffer ache Replacement Algorithm ianhui Yue, Yifeng Zhu, Zhao ai, and Lin Lin lectrical and omputer ngineering University of Maine Research Summary Memory power consumption

More information

Storage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore Ioan Raicu, Ian Foster

Storage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore Ioan Raicu, Ian Foster Storage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore Ioan Raicu, Ian Foster. Overview Both the industry and academia have an increase demand for good policies and mechanisms to

More information

Chapter 13: I/O Systems

Chapter 13: I/O Systems Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance Objectives Explore the structure of an operating

More information

Chapter 13: I/O Systems

Chapter 13: I/O Systems Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance I/O Hardware Incredible variety of I/O devices Common

More information

The Quest for Scalable Support of Data-Intensive Workloads in Distributed Systems

The Quest for Scalable Support of Data-Intensive Workloads in Distributed Systems The Quest for Scalable Support of Data-Intensive Workloads in Distributed Systems Ioan Raicu, 1 Ian T. Foster, 1,2,3 Yong Zhao 4 Philip Little, 5 Christopher M. Moretti, 5 Amitabh Chaudhary, 5 Douglas

More information

Ref: Chap 12. Secondary Storage and I/O Systems. Applied Operating System Concepts 12.1

Ref: Chap 12. Secondary Storage and I/O Systems. Applied Operating System Concepts 12.1 Ref: Chap 12 Secondary Storage and I/O Systems Applied Operating System Concepts 12.1 Part 1 - Secondary Storage Secondary storage typically: is anything that is outside of primary memory does not permit

More information

Data Management in Parallel Scripting

Data Management in Parallel Scripting Data Management in Parallel Scripting Zhao Zhang 11/11/2012 Problem Statement Definition: MTC applications are those applications in which existing sequential or parallel programs are linked by files output

More information

A high-speed data processing technique for time-sequential satellite observation data

A high-speed data processing technique for time-sequential satellite observation data A high-speed data processing technique for time-sequential satellite observation data Ken T. Murata 1a), Hidenobu Watanabe 1, Kazunori Yamamoto 1, Eizen Kimura 2, Masahiro Tanaka 3,OsamuTatebe 3, Kentaro

More information

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

More information

Two hours - online. The exam will be taken on line. This paper version is made available as a backup

Two hours - online. The exam will be taken on line. This paper version is made available as a backup COMP 25212 Two hours - online The exam will be taken on line. This paper version is made available as a backup UNIVERSITY OF MANCHESTER SCHOOL OF COMPUTER SCIENCE System Architecture Date: Monday 21st

More information

SMORE: A Cold Data Object Store for SMR Drives

SMORE: A Cold Data Object Store for SMR Drives SMORE: A Cold Data Object Store for SMR Drives Peter Macko, Xiongzi Ge, John Haskins Jr.*, James Kelley, David Slik, Keith A. Smith, and Maxim G. Smith Advanced Technology Group NetApp, Inc. * Qualcomm

More information

Lecture 11: SMT and Caching Basics. Today: SMT, cache access basics (Sections 3.5, 5.1)

Lecture 11: SMT and Caching Basics. Today: SMT, cache access basics (Sections 3.5, 5.1) Lecture 11: SMT and Caching Basics Today: SMT, cache access basics (Sections 3.5, 5.1) 1 Thread-Level Parallelism Motivation: a single thread leaves a processor under-utilized for most of the time by doubling

More information

Chapter 13: I/O Systems

Chapter 13: I/O Systems Chapter 13: I/O Systems Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance 13.2 Silberschatz, Galvin

More information

Chapter 13: I/O Systems. Chapter 13: I/O Systems. Objectives. I/O Hardware. A Typical PC Bus Structure. Device I/O Port Locations on PCs (partial)

Chapter 13: I/O Systems. Chapter 13: I/O Systems. Objectives. I/O Hardware. A Typical PC Bus Structure. Device I/O Port Locations on PCs (partial) Chapter 13: I/O Systems Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance 13.2 Silberschatz, Galvin

More information

Data Throttling for Data-Intensive Workflows

Data Throttling for Data-Intensive Workflows Data Throttling for Data-Intensive Workflows Sang-Min Park and Marty Humphrey Dept. of Computer Science, University of Virginia This work was supported by the National Science Foundation under Grant No.

More information

Taming Non-blocking Caches to Improve Isolation in Multicore Real-Time Systems

Taming Non-blocking Caches to Improve Isolation in Multicore Real-Time Systems Taming Non-blocking Caches to Improve Isolation in Multicore Real-Time Systems Prathap Kumar Valsan, Heechul Yun, Farzad Farshchi University of Kansas 1 Why? High-Performance Multicores for Real-Time Systems

More information

Ioan Raicu. Everyone else. More information at: Background? What do you want to get out of this course?

Ioan Raicu. Everyone else. More information at: Background? What do you want to get out of this course? Ioan Raicu More information at: http://www.cs.iit.edu/~iraicu/ Everyone else Background? What do you want to get out of this course? 2 Data Intensive Computing is critical to advancing modern science Applies

More information

Oracle Database 12c: JMS Sharded Queues

Oracle Database 12c: JMS Sharded Queues Oracle Database 12c: JMS Sharded Queues For high performance, scalable Advanced Queuing ORACLE WHITE PAPER MARCH 2015 Table of Contents Introduction 2 Architecture 3 PERFORMANCE OF AQ-JMS QUEUES 4 PERFORMANCE

More information

All-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP

All-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP All-Flash High-Performance SAN/NAS Solutions for Virtualization & OLTP All-flash configurations are designed to deliver maximum IOPS and throughput numbers for mission critical workloads and applicati

More information

COL862 Programming Assignment-1

COL862 Programming Assignment-1 Submitted By: Rajesh Kedia (214CSZ8383) COL862 Programming Assignment-1 Objective: Understand the power and energy behavior of various benchmarks on different types of x86 based systems. We explore a laptop,

More information

AN ELASTIC MULTI-CORE ALLOCATION MECHANISM FOR DATABASE SYSTEMS

AN ELASTIC MULTI-CORE ALLOCATION MECHANISM FOR DATABASE SYSTEMS 1 AN ELASTIC MULTI-CORE ALLOCATION MECHANISM FOR DATABASE SYSTEMS SIMONE DOMINICO 1, JORGE A. MEIRA 2, MARCO A. Z. ALVES 1, EDUARDO C. DE ALMEIDA 1 FEDERAL UNIVERSITY OF PARANÁ, BRAZIL 1, UNIVERSITY OF

More information

W H I T E P A P E R. Comparison of Storage Protocol Performance in VMware vsphere 4

W H I T E P A P E R. Comparison of Storage Protocol Performance in VMware vsphere 4 W H I T E P A P E R Comparison of Storage Protocol Performance in VMware vsphere 4 Table of Contents Introduction................................................................... 3 Executive Summary............................................................

More information

Using Transparent Compression to Improve SSD-based I/O Caches

Using Transparent Compression to Improve SSD-based I/O Caches Using Transparent Compression to Improve SSD-based I/O Caches Thanos Makatos, Yannis Klonatos, Manolis Marazakis, Michail D. Flouris, and Angelos Bilas {mcatos,klonatos,maraz,flouris,bilas}@ics.forth.gr

More information

Platforms Design Challenges with many cores

Platforms Design Challenges with many cores latforms Design hallenges with many cores Raj Yavatkar, Intel Fellow Director, Systems Technology Lab orporate Technology Group 1 Environmental Trends: ell 2 *Other names and brands may be claimed as the

More information

Recovering Disk Storage Metrics from low level Trace events

Recovering Disk Storage Metrics from low level Trace events Recovering Disk Storage Metrics from low level Trace events Progress Report Meeting May 05, 2016 Houssem Daoud Michel Dagenais École Polytechnique de Montréal Laboratoire DORSAL Agenda Introduction and

More information

Software and Tools for HPE s The Machine Project

Software and Tools for HPE s The Machine Project Labs Software and Tools for HPE s The Machine Project Scalable Tools Workshop Aug/1 - Aug/4, 2016 Lake Tahoe Milind Chabbi Traditional Computing Paradigm CPU DRAM CPU DRAM CPU-centric computing 2 CPU-Centric

More information

Why Does Solid State Disk Lower CPI?

Why Does Solid State Disk Lower CPI? Why Does Solid State Disk Lower CPI? Blaine Gaither, Jay Veazey, Paul Cao Revision: June 23, 2010 " 2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change

More information

Chapter 5: CPU Scheduling

Chapter 5: CPU Scheduling Chapter 5: CPU Scheduling Basic Concepts Scheduling Criteria Scheduling Algorithms Thread Scheduling Multiple-Processor Scheduling Operating Systems Examples Algorithm Evaluation Chapter 5: CPU Scheduling

More information

Chapter 13: I/O Systems

Chapter 13: I/O Systems Chapter 13: I/O Systems DM510-14 Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations STREAMS Performance 13.2 Objectives

More information

Keywords: disk throughput, virtual machine, I/O scheduling, performance evaluation

Keywords: disk throughput, virtual machine, I/O scheduling, performance evaluation Simple and practical disk performance evaluation method in virtual machine environments Teruyuki Baba Atsuhiro Tanaka System Platforms Research Laboratories, NEC Corporation 1753, Shimonumabe, Nakahara-Ku,

More information

New HPE 3PAR StoreServ 8000 and series Optimized for Flash

New HPE 3PAR StoreServ 8000 and series Optimized for Flash New HPE 3PAR StoreServ 8000 and 20000 series Optimized for Flash AGENDA HPE 3PAR StoreServ architecture fundamentals HPE 3PAR Flash optimizations HPE 3PAR portfolio overview HPE 3PAR Flash example from

More information

Chapter 7. Multicores, Multiprocessors, and Clusters. Goal: connecting multiple computers to get higher performance

Chapter 7. Multicores, Multiprocessors, and Clusters. Goal: connecting multiple computers to get higher performance Chapter 7 Multicores, Multiprocessors, and Clusters Introduction Goal: connecting multiple computers to get higher performance Multiprocessors Scalability, availability, power efficiency Job-level (process-level)

More information

The Oracle Database Appliance I/O and Performance Architecture

The Oracle Database Appliance I/O and Performance Architecture Simple Reliable Affordable The Oracle Database Appliance I/O and Performance Architecture Tammy Bednar, Sr. Principal Product Manager, ODA 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

More information

Overcoming Data Locality: an In-Memory Runtime File System with Symmetrical Data Distribution

Overcoming Data Locality: an In-Memory Runtime File System with Symmetrical Data Distribution Overcoming Data Locality: an In-Memory Runtime File System with Symmetrical Data Distribution Alexandru Uta a,, Andreea Sandu a, Thilo Kielmann a a Dept. of Computer Science, VU University Amsterdam, The

More information

CSCE 626 Experimental Evaluation.

CSCE 626 Experimental Evaluation. CSCE 626 Experimental Evaluation http://parasol.tamu.edu Introduction This lecture discusses how to properly design an experimental setup, measure and analyze the performance of parallel algorithms you

More information

Chapter 13: I/O Systems

Chapter 13: I/O Systems Chapter 13: I/O Systems Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance 13.2 Silberschatz, Galvin

More information

Chapter 13: I/O Systems. Operating System Concepts 9 th Edition

Chapter 13: I/O Systems. Operating System Concepts 9 th Edition Chapter 13: I/O Systems Silberschatz, Galvin and Gagne 2013 Chapter 13: I/O Systems Overview I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations

More information

LRC: Dependency-Aware Cache Management for Data Analytics Clusters. Yinghao Yu, Wei Wang, Jun Zhang, and Khaled B. Letaief IEEE INFOCOM 2017

LRC: Dependency-Aware Cache Management for Data Analytics Clusters. Yinghao Yu, Wei Wang, Jun Zhang, and Khaled B. Letaief IEEE INFOCOM 2017 LRC: Dependency-Aware Cache Management for Data Analytics Clusters Yinghao Yu, Wei Wang, Jun Zhang, and Khaled B. Letaief IEEE INFOCOM 2017 Outline Cache Management for Data Analytics Clusters Inefficiency

More information

Energy Prediction for I/O Intensive Workflow Applications

Energy Prediction for I/O Intensive Workflow Applications Energy Prediction for I/O Intensive Workflow Applications ABSTRACT As workflow-based data-intensive applications have become increasingly popular, the lack of support tools to aid resource provisioning

More information

Performance COE 403. Computer Architecture Prof. Muhamed Mudawar. Computer Engineering Department King Fahd University of Petroleum and Minerals

Performance COE 403. Computer Architecture Prof. Muhamed Mudawar. Computer Engineering Department King Fahd University of Petroleum and Minerals Performance COE 403 Computer Architecture Prof. Muhamed Mudawar Computer Engineering Department King Fahd University of Petroleum and Minerals What is Performance? How do we measure the performance of

More information

Modification and Evaluation of Linux I/O Schedulers

Modification and Evaluation of Linux I/O Schedulers Modification and Evaluation of Linux I/O Schedulers 1 Asad Naweed, Joe Di Natale, and Sarah J Andrabi University of North Carolina at Chapel Hill Abstract In this paper we present three different Linux

More information

Chapter 12: I/O Systems

Chapter 12: I/O Systems Chapter 12: I/O Systems Chapter 12: I/O Systems I/O Hardware! Application I/O Interface! Kernel I/O Subsystem! Transforming I/O Requests to Hardware Operations! STREAMS! Performance! Silberschatz, Galvin

More information

Chapter 13: I/O Systems

Chapter 13: I/O Systems Chapter 13: I/O Systems Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations STREAMS Performance Silberschatz, Galvin and

More information

Chapter 12: I/O Systems. Operating System Concepts Essentials 8 th Edition

Chapter 12: I/O Systems. Operating System Concepts Essentials 8 th Edition Chapter 12: I/O Systems Silberschatz, Galvin and Gagne 2011 Chapter 12: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations STREAMS

More information

Jyotheswar Kuricheti

Jyotheswar Kuricheti Jyotheswar Kuricheti 1 Agenda: 1. Performance Tuning Overview 2. Identify Bottlenecks 3. Optimizing at different levels : Target Source Mapping Session System 2 3 Performance Tuning Overview: 4 What is

More information

White Paper. File System Throughput Performance on RedHawk Linux

White Paper. File System Throughput Performance on RedHawk Linux White Paper File System Throughput Performance on RedHawk Linux By: Nikhil Nanal Concurrent Computer Corporation August Introduction This paper reports the throughput performance of the,, and file systems

More information

Profiling Grid Data Transfer Protocols and Servers. George Kola, Tevfik Kosar and Miron Livny University of Wisconsin-Madison USA

Profiling Grid Data Transfer Protocols and Servers. George Kola, Tevfik Kosar and Miron Livny University of Wisconsin-Madison USA Profiling Grid Data Transfer Protocols and Servers George Kola, Tevfik Kosar and Miron Livny University of Wisconsin-Madison USA Motivation Scientific experiments are generating large amounts of data Education

More information

SRM-Buffer: An OS Buffer Management SRM-Buffer: An OS Buffer Management Technique toprevent Last Level Cache from Thrashing in Multicores

SRM-Buffer: An OS Buffer Management SRM-Buffer: An OS Buffer Management Technique toprevent Last Level Cache from Thrashing in Multicores SRM-Buffer: An OS Buffer Management SRM-Buffer: An OS Buffer Management Technique toprevent Last Level Cache from Thrashing in Multicores Xiaoning Ding The Ohio State University dingxn@cse.ohiostate.edu

More information

Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE

Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE Lustre2.5 Performance Evaluation: Performance Improvements with Large I/O Patches, Metadata Improvements, and Metadata Scaling with DNE Hitoshi Sato *1, Shuichi Ihara *2, Satoshi Matsuoka *1 *1 Tokyo Institute

More information

COSC 6374 Parallel Computation. Parallel Computer Architectures

COSC 6374 Parallel Computation. Parallel Computer Architectures OS 6374 Parallel omputation Parallel omputer Architectures Some slides on network topologies based on a similar presentation by Michael Resch, University of Stuttgart Edgar Gabriel Fall 2015 Flynn s Taxonomy

More information

Deployment Planning and Optimization for Big Data & Cloud Storage Systems

Deployment Planning and Optimization for Big Data & Cloud Storage Systems Deployment Planning and Optimization for Big Data & Cloud Storage Systems Bianny Bian Intel Corporation Outline System Planning Challenges Storage System Modeling w/ Intel CoFluent Studio Simulation Methodology

More information

Performance comparisons and trade-offs for various MySQL replication schemes

Performance comparisons and trade-offs for various MySQL replication schemes Performance comparisons and trade-offs for various MySQL replication schemes Darpan Dinker VP Engineering Brian O Krafka, Chief Architect Schooner Information Technology, Inc. http://www.schoonerinfotech.com/

More information

MULTIPROCESSORS AND THREAD-LEVEL. B649 Parallel Architectures and Programming

MULTIPROCESSORS AND THREAD-LEVEL. B649 Parallel Architectures and Programming MULTIPROCESSORS AND THREAD-LEVEL PARALLELISM B649 Parallel Architectures and Programming Motivation behind Multiprocessors Limitations of ILP (as already discussed) Growing interest in servers and server-performance

More information

Falcon: Scaling IO Performance in Multi-SSD Volumes. The George Washington University

Falcon: Scaling IO Performance in Multi-SSD Volumes. The George Washington University Falcon: Scaling IO Performance in Multi-SSD Volumes Pradeep Kumar H Howie Huang The George Washington University SSDs in Big Data Applications Recent trends advocate using many SSDs for higher throughput

More information

Indexing in RAMCloud. Ankita Kejriwal, Ashish Gupta, Arjun Gopalan, John Ousterhout. Stanford University

Indexing in RAMCloud. Ankita Kejriwal, Ashish Gupta, Arjun Gopalan, John Ousterhout. Stanford University Indexing in RAMCloud Ankita Kejriwal, Ashish Gupta, Arjun Gopalan, John Ousterhout Stanford University RAMCloud 1.0 Introduction Higher-level data models Without sacrificing latency and scalability Secondary

More information

MULTIPROCESSORS AND THREAD-LEVEL PARALLELISM. B649 Parallel Architectures and Programming

MULTIPROCESSORS AND THREAD-LEVEL PARALLELISM. B649 Parallel Architectures and Programming MULTIPROCESSORS AND THREAD-LEVEL PARALLELISM B649 Parallel Architectures and Programming Motivation behind Multiprocessors Limitations of ILP (as already discussed) Growing interest in servers and server-performance

More information

Adaptec MaxIQ SSD Cache Performance Solution for Web Server Environments Analysis

Adaptec MaxIQ SSD Cache Performance Solution for Web Server Environments Analysis Adaptec MaxIQ SSD Cache Performance Solution for Web Server Environments Analysis September 22, 2009 Page 1 of 7 Introduction Adaptec has requested an evaluation of the performance of the Adaptec MaxIQ

More information

Accelerate Applications Using EqualLogic Arrays with directcache

Accelerate Applications Using EqualLogic Arrays with directcache Accelerate Applications Using EqualLogic Arrays with directcache Abstract This paper demonstrates how combining Fusion iomemory products with directcache software in host servers significantly improves

More information

Caching and reliability

Caching and reliability Caching and reliability Block cache Vs. Latency ~10 ns 1~ ms Access unit Byte (word) Sector Capacity Gigabytes Terabytes Price Expensive Cheap Caching disk contents in RAM Hit ratio h : probability of

More information

Falkon: a Fast and Light-weight task execution framework

Falkon: a Fast and Light-weight task execution framework Falkon: a Fast and Light-weight task execution framework Ioan Raicu *, Yong Zhao *, Catalin Dumitrescu *, Ian Foster #*+, Mike Wilde #+ {iraicu,yongzh,catalind}@cs.uchicago.edu, {foster,wilde}@mcs.anl.gov

More information

by I.-C. Lin, Dept. CS, NCTU. Textbook: Operating System Concepts 8ed CHAPTER 13: I/O SYSTEMS

by I.-C. Lin, Dept. CS, NCTU. Textbook: Operating System Concepts 8ed CHAPTER 13: I/O SYSTEMS by I.-C. Lin, Dept. CS, NCTU. Textbook: Operating System Concepts 8ed CHAPTER 13: I/O SYSTEMS Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests

More information

EECS750: Advanced Operating Systems. 2/24/2014 Heechul Yun

EECS750: Advanced Operating Systems. 2/24/2014 Heechul Yun EECS750: Advanced Operating Systems 2/24/2014 Heechul Yun 1 Administrative Project Feedback of your proposal will be sent by Wednesday Midterm report due on Apr. 2 3 pages: include intro, related work,

More information

Implementing SQL Server 2016 with Microsoft Storage Spaces Direct on Dell EMC PowerEdge R730xd

Implementing SQL Server 2016 with Microsoft Storage Spaces Direct on Dell EMC PowerEdge R730xd Implementing SQL Server 2016 with Microsoft Storage Spaces Direct on Dell EMC PowerEdge R730xd Performance Study Dell EMC Engineering October 2017 A Dell EMC Performance Study Revisions Date October 2017

More information

Illinois Proposal Considerations Greg Bauer

Illinois Proposal Considerations Greg Bauer - 2016 Greg Bauer Support model Blue Waters provides traditional Partner Consulting as part of its User Services. Standard service requests for assistance with porting, debugging, allocation issues, and

More information

Staged Memory Scheduling

Staged Memory Scheduling Staged Memory Scheduling Rachata Ausavarungnirun, Kevin Chang, Lavanya Subramanian, Gabriel H. Loh*, Onur Mutlu Carnegie Mellon University, *AMD Research June 12 th 2012 Executive Summary Observation:

More information

A DEDUPLICATION-INSPIRED FAST DELTA COMPRESSION APPROACH W EN XIA, HONG JIANG, DA N FENG, LEI T I A N, M I N FU, YUKUN Z HOU

A DEDUPLICATION-INSPIRED FAST DELTA COMPRESSION APPROACH W EN XIA, HONG JIANG, DA N FENG, LEI T I A N, M I N FU, YUKUN Z HOU A DEDUPLICATION-INSPIRED FAST DELTA COMPRESSION APPROACH W EN XIA, HONG JIANG, DA N FENG, LEI T I A N, M I N FU, YUKUN Z HOU PRESENTED BY ROMAN SHOR Overview Technics of data reduction in storage systems:

More information