Automated load balancing in the ATLAS high-performance storage software
|
|
- Valentine Weaver
- 5 years ago
- Views:
Transcription
1
2 Automated load balancing in the ATLAS high-performance storage software Fabrice Le Go 1 Wainer Vandelli 1 On behalf of the ATLAS Collaboration 1 CERN May 25th, 2017
3 The ATLAS Experiment 3 / 20
4 ATLAS Trigger and Data Acquisition System 4 / 20
5 Data Logger System HBA1 Server 1 HBA2 Transient storage system to: Decouple online and oine operations Cope with disruption of permanent storage service or its connection Scale-out system, currently: 4 local-attached storage, 2 servers each 500 HDs, 430 TB, 8 GB/s Fully-redundant: no data loss caused in 2016 Server 2 HBA1 HBA2 Controller1 Controller2 Disks Expansion Disks Expansion Disks Expansion Disks Expansion HBA: Host Bus Adapter 5 / 20
6 Software Distributed in-house application (C++) Tasks: Receive selected events data Write data to disks Compute data checksum: le-by-le Adler32 Data-driven: events are distributed in classes called streams One le by stream New streams appear roughly every minute Stream distribution may vary rapidly Workload not uniform at all, cannot be fairly distributed Multi-threaded through task-oriented framework Typical Stream Bandwidth Distribution Another independent application sends the data to permanent storage 6 / 20
7 Threading Model Input threads IT1 IT2 ITN Events in streams Writing Manager creates File to thread bookkeeping S0 OT1 S1 OT2 S2 E0 E2 E7 E17 E3 E14 E10 E1... S0 S1 S0 S2 S0 S2 S0 S1 S1 S2 S2 S1 Writing Task event to 1 file Output threads File B OT1 assigned to OT2 File A File C OTM File A E0 E7 E3 E10... File B E0 E3 E2 E1... File C E2 E7 E17E The workload distribution is controlled by the le-to-thread assignment policy The application performance can be limited by one thread 7 / 20
8 Current Assignment Policy Round-robin: each new le is assigned to the next thread in a circular thread buer Simple implementation, very low overhead Deterministic behavior but events come with no specic order: non-deterministic assignment of les to thread The application's instantaneous performance is not predictable: The assignment of major streams to the same thread will degrade the application performance 8 / 20
9 Problem Modication in the operational conditions: higher throughput, dierent stream distribution Peak throughput 1.4 GB/s 3.2 GB/s S1: 80 % S1': 70 % Stream distribution S2: 6 % S2': 7 % S3: 3 % S3': 5 % Random assignment of major streams to the same thread will now degrade the application performance Synthetic test conrmed performance degradation: Conditions Writing rate Performance Loss No joint assignment 865 MB/s reference S1' and S2' together 797 MB/s - 8 % S1', S2' and S3' together 760 MB/s - 12 % 9 / 20
10 Weighted Assignment Policy A new workload distribution strategy was needed to restore performance and predictability Requirements: Data-driven: e.g. cannot assume any pattern in stream distribution Responsive: must cope with rapid evolution of stream distribution Low CPU and memory footprint Idea: Compute a load for thread: last-n-second sliding window of amount of processed data Assign a new le to the thread with the lowest load 10 / 20
11 Weighted Assignment Policy: Step 1 Threads load vs. time with assignments Zoom on the assignments: problematic in red Real-time load is ineective for close-enough assignments Reducing sliding window length: but cannot be too small, would be too sensitive to local uctuations (typical: 5 seconds) Another component needed to be added: Compute a load for the streams: same sliding-window amount of processed data by class of streams Add the stream load to the thread load upon assignment 11 / 20
12 Weighted Assignment Policy: Step 2 Threads load vs. time with assignments Zoom on the assignments Decisions are reected immediately: the likelihood of a thread to be selected again just after decision is inverse proportional to the load of the assigned stream 12 / 20
13 Testing Test in controlled environment with emulated data ow: Stream distribution and upstream event processing time emulated from 2016 monitoring data No wrong decision for + 40-hour runs Policy Writing rate Performance Gain Round-robin 865 MB/s reference Weighted 882 MB/s + 2 % Test on the actual ATLAS TDAQ infrastructure Used during ATLAS commissioning tests and cosmic data taking sessions 13 / 20
14 Conclusion The transient storage system of ATLAS TDAQ is a key component enabling for decoupling of online and oine operations Its workload is heavily unbalanced and cannot be fairly distributed In 2016 a new strategy was required to handle recent changes in operation conditions New workload distribution strategy: sensitive and self-adaptive to fast-evolving operation conditions and modications of the event selection process Validated in both test and production environments: proved to better use the parallel processing capabilities of modern CPUs for our workload This development will be part of the 2017 data-taking session 14 / 20
15
16 2015 Real-time Streams Writing Rate Figure: Instantaneous stream bandwidth for data collected on 28/10/2015. All streams are shown and each line represent a dierent stream. The highest line labeled "Global" is the sum of all streams representing the total bandwidth of selected events data. 16 / 20
17 2015 Stream Bandwidth Distribution Figure: Stream bandwidth distribution for data collected on 28/10/2015. Each bar represent the fraction of the total bandwidth for one stream over the considered period. 17 / 20
18 2015 Stream Bandwidth Distribution Figure: Bandwidth distribution between dierent streams for data collected on 28/10/2015. The four highest bandwidth streams are shown seperately and all other streams are summed together as "Other streams". 18 / 20
19 2016 Real-time Streams Writing Rate Figure: Instantaneous stream bandwidth for data collected on 24 and 25/10/2016. All streams are shown and each line represent a dierent stream. The highest line labeled "Global" is the sum of all streams representing the total bandwidth of selected events data. 19 / 20
20 2016 Stream Bandwidth Distribution Figure: Stream bandwidth distribution for data collected on 24 and 25/10/2016. Each bar represent the fraction of the total bandwidth for one stream over the considered period. 20 / 20
Modeling Resource Utilization of a Large Data Acquisition System
Modeling Resource Utilization of a Large Data Acquisition System Alejandro Santos CERN / Ruprecht-Karls-Universität Heidelberg On behalf of the ATLAS Collaboration 1 Outline Introduction ATLAS TDAQ Simulation
More informationModeling and Validating Time, Buffering, and Utilization of a Large-Scale, Real-Time Data Acquisition System
Modeling and Validating Time, Buffering, and Utilization of a Large-Scale, Real-Time Data Acquisition System Alejandro Santos, Pedro Javier García, Wainer Vandelli, Holger Fröning The 2017 International
More informationTHE ATLAS DATA ACQUISITION SYSTEM IN LHC RUN 2
THE ATLAS DATA ACQUISITION SYSTEM IN LHC RUN 2 M. E. Pozo Astigarraga, on behalf of the ATLAS Collaboration CERN, CH-1211 Geneva 23, Switzerland E-mail: eukeni.pozo@cern.ch The LHC has been providing proton-proton
More informationCommercial Real-time Operating Systems An Introduction. Swaminathan Sivasubramanian Dependable Computing & Networking Laboratory
Commercial Real-time Operating Systems An Introduction Swaminathan Sivasubramanian Dependable Computing & Networking Laboratory swamis@iastate.edu Outline Introduction RTOS Issues and functionalities LynxOS
More informationThe Oracle Database Appliance I/O and Performance Architecture
Simple Reliable Affordable The Oracle Database Appliance I/O and Performance Architecture Tammy Bednar, Sr. Principal Product Manager, ODA 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
More informationI/O Systems. Amir H. Payberah. Amirkabir University of Technology (Tehran Polytechnic)
I/O Systems Amir H. Payberah amir@sics.se Amirkabir University of Technology (Tehran Polytechnic) Amir H. Payberah (Tehran Polytechnic) I/O Systems 1393/9/15 1 / 57 Motivation Amir H. Payberah (Tehran
More informationThe Impact of SSD Selection on SQL Server Performance. Solution Brief. Understanding the differences in NVMe and SATA SSD throughput
Solution Brief The Impact of SSD Selection on SQL Server Performance Understanding the differences in NVMe and SATA SSD throughput 2018, Cloud Evolutions Data gathered by Cloud Evolutions. All product
More informationSummary of the LHC Computing Review
Summary of the LHC Computing Review http://lhc-computing-review-public.web.cern.ch John Harvey CERN/EP May 10 th, 2001 LHCb Collaboration Meeting The Scale Data taking rate : 50,100, 200 Hz (ALICE, ATLAS-CMS,
More informationOperating Systems 2010/2011
Operating Systems 2010/2011 Input/Output Systems part 2 (ch13, ch12) Shudong Chen 1 Recap Discuss the principles of I/O hardware and its complexity Explore the structure of an operating system s I/O subsystem
More informationAn Oracle White Paper April 2010
An Oracle White Paper April 2010 In October 2009, NEC Corporation ( NEC ) established development guidelines and a roadmap for IT platform products to realize a next-generation IT infrastructures suited
More informationThe ATLAS Data Flow System for LHC Run 2
The ATLAS Data Flow System for LHC Run 2 Andrei Kazarov on behalf of ATLAS Collaboration 1,2,a) 1 CERN, CH1211 Geneva 23, Switzerland 2 on leave from: Petersburg NPI Kurchatov NRC, Gatchina, Russian Federation
More informationL1 and Subsequent Triggers
April 8, 2003 L1 and Subsequent Triggers Abstract During the last year the scope of the L1 trigger has changed rather drastically compared to the TP. This note aims at summarising the changes, both in
More informationDatabase Services at CERN with Oracle 10g RAC and ASM on Commodity HW
Database Services at CERN with Oracle 10g RAC and ASM on Commodity HW UKOUG RAC SIG Meeting London, October 24 th, 2006 Luca Canali, CERN IT CH-1211 LCGenève 23 Outline Oracle at CERN Architecture of CERN
More informationExample: CPU-bound process that would run for 100 quanta continuously 1, 2, 4, 8, 16, 32, 64 (only 37 required for last run) Needs only 7 swaps
Interactive Scheduling Algorithms Continued o Priority Scheduling Introduction Round-robin assumes all processes are equal often not the case Assign a priority to each process, and always choose the process
More informationNetwork Design Considerations for Grid Computing
Network Design Considerations for Grid Computing Engineering Systems How Bandwidth, Latency, and Packet Size Impact Grid Job Performance by Erik Burrows, Engineering Systems Analyst, Principal, Broadcom
More informationSurFS Product Description
SurFS Product Description 1. ABSTRACT SurFS An innovative technology is evolving the distributed storage ecosystem. SurFS is designed for cloud storage with extreme performance at a price that is significantly
More informationGoogle File System. Arun Sundaram Operating Systems
Arun Sundaram Operating Systems 1 Assumptions GFS built with commodity hardware GFS stores a modest number of large files A few million files, each typically 100MB or larger (Multi-GB files are common)
More informationSilberschatz and Galvin Chapter 12
Silberschatz and Galvin Chapter 12 I/O Systems CPSC 410--Richard Furuta 3/19/99 1 Topic overview I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O requests to hardware operations
More informationDevice-Functionality Progression
Chapter 12: I/O Systems I/O Hardware I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Incredible variety of I/O devices Common concepts Port
More informationChapter 12: I/O Systems. I/O Hardware
Chapter 12: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations I/O Hardware Incredible variety of I/O devices Common concepts Port
More informationCPU Scheduling. Daniel Mosse. (Most slides are from Sherif Khattab and Silberschatz, Galvin and Gagne 2013)
CPU Scheduling Daniel Mosse (Most slides are from Sherif Khattab and Silberschatz, Galvin and Gagne 2013) Basic Concepts Maximum CPU utilization obtained with multiprogramming CPU I/O Burst Cycle Process
More informationCOMPARING COST MODELS - DETAILS
COMPARING COST MODELS - DETAILS SOFTLAYER TOTAL COST OF OWNERSHIP (TCO) CALCULATOR APPROACH The Detailed comparison tab in the TCO Calculator provides a tool with which to do a cost comparison between
More informationLecture 13 Input/Output (I/O) Systems (chapter 13)
Bilkent University Department of Computer Engineering CS342 Operating Systems Lecture 13 Input/Output (I/O) Systems (chapter 13) Dr. İbrahim Körpeoğlu http://www.cs.bilkent.edu.tr/~korpe 1 References The
More informationConference The Data Challenges of the LHC. Reda Tafirout, TRIUMF
Conference 2017 The Data Challenges of the LHC Reda Tafirout, TRIUMF Outline LHC Science goals, tools and data Worldwide LHC Computing Grid Collaboration & Scale Key challenges Networking ATLAS experiment
More informationRed Hat Gluster Storage performance. Manoj Pillai and Ben England Performance Engineering June 25, 2015
Red Hat Gluster Storage performance Manoj Pillai and Ben England Performance Engineering June 25, 2015 RDMA Erasure Coding NFS-Ganesha New or improved features (in last year) Snapshots SSD support Erasure
More informationOperating System: Chap13 I/O Systems. National Tsing-Hua University 2016, Fall Semester
Operating System: Chap13 I/O Systems National Tsing-Hua University 2016, Fall Semester Outline Overview I/O Hardware I/O Methods Kernel I/O Subsystem Performance Application Interface Operating System
More informationChapter 13: I/O Systems
Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance Objectives Explore the structure of an operating
More informationVirtual Security Server
Data Sheet VSS Virtual Security Server Security clients anytime, anywhere, any device CENTRALIZED CLIENT MANAGEMENT UP TO 50% LESS BANDWIDTH UP TO 80 VIDEO STREAMS MOBILE ACCESS INTEGRATED SECURITY SYSTEMS
More informationHP ProLiant BladeSystem Gen9 vs Gen8 and G7 Server Blades on Data Warehouse Workloads
HP ProLiant BladeSystem Gen9 vs Gen8 and G7 Server Blades on Data Warehouse Workloads Gen9 server blades give more performance per dollar for your investment. Executive Summary Information Technology (IT)
More information操作系统概念 13. I/O Systems
OPERATING SYSTEM CONCEPTS 操作系统概念 13. I/O Systems 东南大学计算机学院 Baili Zhang/ Southeast 1 Objectives 13. I/O Systems Explore the structure of an operating system s I/O subsystem Discuss the principles of I/O
More informationPeople, Process, Technology Transforming the Enterprise Desktop To An Enterprise Virtual Desktop
1 People, Process, Technology Transforming the Enterprise Desktop To An Enterprise Virtual Desktop 2 Elements of Abstraction in the Enterprise Hybrid Cloud 3 Management Elements of IT Infrastructure 4
More informationThe Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff and Shun Tak Leung Google* Shivesh Kumar Sharma fl4164@wayne.edu Fall 2015 004395771 Overview Google file system is a scalable distributed file system
More informationChapter 13: I/O Systems
Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance I/O Hardware Incredible variety of I/O devices Common
More informationInput/Output Systems
CSE325 Principles of Operating Systems Input/Output Systems David P. Duggan dduggan@sandia.gov April 2, 2013 Input/Output Devices Output Device Input Device Processor 4/2/13 CSE325 - I/O Systems 2 Why
More informationThe FTK to Level-2 Interface Card (FLIC)
The FTK to Level-2 Interface Card (FLIC) J. Anderson, B. Auerbach, R. Blair, G. Drake, A. Kreps, J. Love, J. Proudfoot, M. Oberling, R. Wang, J. Zhang November 5th, 2015 2015 IEEE Nuclear Science Symposium
More informationVblock Infrastructure Packages: Accelerating Deployment of the Private Cloud
Vblock Infrastructure Packages: Accelerating Deployment of the Private Cloud Roberto Missana - Channel Product Sales Specialist Data Center, Cisco 1 IT is undergoing a transformation Enterprise IT solutions
More informationJMR ELECTRONICS INC. WHITE PAPER
THE NEED FOR SPEED: USING PCI EXPRESS ATTACHED STORAGE FOREWORD The highest performance, expandable, directly attached storage can be achieved at low cost by moving the server or work station s PCI bus
More informationDistributed Video Systems Chapter 3 Storage Technologies
Distributed Video Systems Chapter 3 Storage Technologies Jack Yiu-bun Lee Department of Information Engineering The Chinese University of Hong Kong Contents 3.1 Introduction 3.2 Magnetic Disks 3.3 Video
More informationData Storage Institute. SANSIM: A PLATFORM FOR SIMULATION AND DESIGN OF A STORAGE AREA NETWORK Zhu Yaolong
Data Storage Institute SANSIM: A PLATFORM FOR SIMULATION AND DESIGN OF A STORAGE AREA NETWORK Zhu Yaolong e_mail:zhu_yaolong@dsi.a-star.edu.sg Outline Motivation Key Focuses Simulation Methodology SANSim
More informationI/O CANNOT BE IGNORED
LECTURE 13 I/O I/O CANNOT BE IGNORED Assume a program requires 100 seconds, 90 seconds for main memory, 10 seconds for I/O. Assume main memory access improves by ~10% per year and I/O remains the same.
More informationIntroduction Optimizing applications with SAO: IO characteristics Servers: Microsoft Exchange... 5 Databases: Oracle RAC...
HP StorageWorks P2000 G3 FC MSA Dual Controller Virtualization SAN Starter Kit Protecting Critical Applications with Server Application Optimization (SAO) Technical white paper Table of contents Introduction...
More informationNuttX Realtime Programming
NuttX RTOS NuttX Realtime Programming Gregory Nutt Overview Interrupts Cooperative Scheduling Tasks Work Queues Realtime Schedulers Real Time == == Deterministic Response Latency Stimulus Response Deadline
More informationLeveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands
Leveraging Software-Defined Storage to Meet Today and Tomorrow s Infrastructure Demands Unleash Your Data Center s Hidden Power September 16, 2014 Molly Rector CMO, EVP Product Management & WW Marketing
More informationHIGH PERFORMANCE STORAGE SOLUTION PRESENTATION All rights reserved RAIDIX
HIGH PERFORMANCE STORAGE SOLUTION PRESENTATION 2017 All rights reserved RAIDIX ABOUT COMPANY RAIDIX is a innovative solution provider and developer of high-performance storage systems. Patented erasure
More informationS K T e l e c o m : A S h a r e a b l e D A S P o o l u s i n g a L o w L a t e n c y N V M e A r r a y. Eric Chang / Program Manager / SK Telecom
S K T e l e c o m : A S h a r e a b l e D A S P o o l u s i n g a L o w L a t e n c y N V M e A r r a y Eric Chang / Program Manager / SK Telecom 2/23 Before We Begin SKT NV-Array (NVMe JBOF) has been
More informationComputer Science 4500 Operating Systems
Computer Science 4500 Operating Systems Module 6 Process Scheduling Methods Updated: September 25, 2014 2008 Stanley A. Wileman, Jr. Operating Systems Slide 1 1 In This Module Batch and interactive workloads
More informationOperating Systems, Fall
Input / Output & Real-time Scheduling Chapter 5.1 5.4, Chapter 7.5 1 I/O Software Device controllers Memory-mapped mapped I/O DMA & interrupts briefly I/O Content I/O software layers and drivers Disks
More informationGetafix: Workload-aware Distributed Interactive Analytics
Getafix: Workload-aware Distributed Interactive Analytics Presenter: Mainak Ghosh Collaborators: Le Xu, Xiaoyao Qian, Thomas Kao, Indranil Gupta, Himanshu Gupta Data Analytics 2 Picture borrowed from https://conferences.oreilly.com/strata/strata-ny-2016/public/schedule/detail/51640
More informationTPC-E testing of Microsoft SQL Server 2016 on Dell EMC PowerEdge R830 Server and Dell EMC SC9000 Storage
TPC-E testing of Microsoft SQL Server 2016 on Dell EMC PowerEdge R830 Server and Dell EMC SC9000 Storage Performance Study of Microsoft SQL Server 2016 Dell Engineering February 2017 Table of contents
More informationEMC Performance Optimization for VMware Enabled by EMC PowerPath/VE
EMC Performance Optimization for VMware Enabled by EMC PowerPath/VE Applied Technology Abstract This white paper is an overview of the tested features and performance enhancing technologies of EMC PowerPath
More informationCold Storage: The Road to Enterprise Ilya Kuznetsov YADRO
Cold Storage: The Road to Enterprise Ilya Kuznetsov YADRO Agenda Technical challenge Custom product Growth of aspirations Enterprise requirements Making an enterprise cold storage product 2 Technical Challenge
More informationThe ATLAS Data Acquisition System: from Run 1 to Run 2
Available online at www.sciencedirect.com Nuclear and Particle Physics Proceedings 273 275 (2016) 939 944 www.elsevier.com/locate/nppp The ATLAS Data Acquisition System: from Run 1 to Run 2 William Panduro
More informationDatabase Systems II. Secondary Storage
Database Systems II Secondary Storage CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 29 The Memory Hierarchy Swapping, Main-memory DBMS s Tertiary Storage: Tape, Network Backup 3,200 MB/s (DDR-SDRAM
More informationby I.-C. Lin, Dept. CS, NCTU. Textbook: Operating System Concepts 8ed CHAPTER 13: I/O SYSTEMS
by I.-C. Lin, Dept. CS, NCTU. Textbook: Operating System Concepts 8ed CHAPTER 13: I/O SYSTEMS Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests
More informationAccelerating Microsoft SQL Server 2016 Performance With Dell EMC PowerEdge R740
Accelerating Microsoft SQL Server 2016 Performance With Dell EMC PowerEdge R740 A performance study of 14 th generation Dell EMC PowerEdge servers for Microsoft SQL Server Dell EMC Engineering September
More informationToward Seamless Integration of RAID and Flash SSD
Toward Seamless Integration of RAID and Flash SSD Sang-Won Lee Sungkyunkwan Univ., Korea (Joint-Work with Sungup Moon, Bongki Moon, Narinet, and Indilinx) Santa Clara, CA 1 Table of Contents Introduction
More informationScientific data processing at global scale The LHC Computing Grid. fabio hernandez
Scientific data processing at global scale The LHC Computing Grid Chengdu (China), July 5th 2011 Who I am 2 Computing science background Working in the field of computing for high-energy physics since
More informationEvaluation Report: Improving SQL Server Database Performance with Dot Hill AssuredSAN 4824 Flash Upgrades
Evaluation Report: Improving SQL Server Database Performance with Dot Hill AssuredSAN 4824 Flash Upgrades Evaluation report prepared under contract with Dot Hill August 2015 Executive Summary Solid state
More informationChapter 13: I/O Systems
Chapter 13: I/O Systems Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance 13.2 Silberschatz, Galvin
More informationChapter 13: I/O Systems. Chapter 13: I/O Systems. Objectives. I/O Hardware. A Typical PC Bus Structure. Device I/O Port Locations on PCs (partial)
Chapter 13: I/O Systems Chapter 13: I/O Systems I/O Hardware Application I/O Interface Kernel I/O Subsystem Transforming I/O Requests to Hardware Operations Streams Performance 13.2 Silberschatz, Galvin
More informationLecture 23. Finish-up buses Storage
Lecture 23 Finish-up buses Storage 1 Example Bus Problems, cont. 2) Assume the following system: A CPU and memory share a 32-bit bus running at 100MHz. The memory needs 50ns to access a 64-bit value from
More informationHard Disk Drives. Nima Honarmand (Based on slides by Prof. Andrea Arpaci-Dusseau)
Hard Disk Drives Nima Honarmand (Based on slides by Prof. Andrea Arpaci-Dusseau) Storage Stack in the OS Application Virtual file system Concrete file system Generic block layer Driver Disk drive Build
More informationNexenta Technical Sales Professional (NTSP)
Global Leader in Software Defined Storage Nexenta Technical Sales Professional (NTSP) COURSE CONTENT Nexenta Technical Sales Professional (NTSP) Course USE CASE: MICROSOFT SHAREPOINT 2 Use Case Microsoft
More informationChapter 6: CPU Scheduling. Operating System Concepts 9 th Edition
Chapter 6: CPU Scheduling Silberschatz, Galvin and Gagne 2013 Chapter 6: CPU Scheduling Basic Concepts Scheduling Criteria Scheduling Algorithms Thread Scheduling Multiple-Processor Scheduling Real-Time
More informationT H. Runable. Request. Priority Inversion. Exit. Runable. Request. Reply. For T L. For T. Reply. Exit. Request. Runable. Exit. Runable. Reply.
Experience with Real-Time Mach for Writing Continuous Media Applications and Servers Tatsuo Nakajima Hiroshi Tezuka Japan Advanced Institute of Science and Technology Abstract This paper describes the
More informationDelivering High-Throughput SAN with Brocade Gen 6 Fibre Channel. Friedrich Hartmann SE Manager DACH & BeNeLux
Delivering High-Throughput SAN with Brocade Gen 6 Fibre Channel Friedrich Hartmann SE Manager DACH & BeNeLux 28.02.2018 Agenda Broadcom Acquisition Update Storage Trends Brocade Automation SAN Analytics
More informationExam Name: Midrange Storage Technical Support V2
Vendor: IBM Exam Code: 000-118 Exam Name: Midrange Storage Technical Support V2 Version: 12.39 QUESTION 1 A customer has an IBM System Storage DS5000 and needs to add more disk drives to the unit. There
More informationSoftNAS Cloud Performance Evaluation on Microsoft Azure
SoftNAS Cloud Performance Evaluation on Microsoft Azure November 30, 2016 Contents SoftNAS Cloud Overview... 3 Introduction... 3 Executive Summary... 4 Key Findings for Azure:... 5 Test Methodology...
More informationReliability Engineering Analysis of ATLAS Data Reprocessing Campaigns
Journal of Physics: Conference Series OPEN ACCESS Reliability Engineering Analysis of ATLAS Data Reprocessing Campaigns To cite this article: A Vaniachine et al 2014 J. Phys.: Conf. Ser. 513 032101 View
More informationEMC DATA DOMAIN OPERATING SYSTEM
EMC DATA DOMAIN OPERATING SYSTEM Powering EMC Protection Storage ESSENTIALS High-Speed, Scalable Deduplication Up to 31 TB/hr performance Reduces requirements for backup storage by 10 to 30x and archive
More informationFELI. : the detector readout upgrade of the ATLAS experiment. Soo Ryu. Argonne National Laboratory, (on behalf of the FELIX group)
LI : the detector readout upgrade of the ATLAS experiment Soo Ryu Argonne National Laboratory, sryu@anl.gov (on behalf of the LIX group) LIX group John Anderson, Soo Ryu, Jinlong Zhang Hucheng Chen, Kai
More informationStatus Update About COLO (COLO: COarse-grain LOck-stepping Virtual Machines for Non-stop Service)
Status Update About COLO (COLO: COarse-grain LOck-stepping Virtual Machines for Non-stop Service) eddie.dong@intel.com arei.gonglei@huawei.com yanghy@cn.fujitsu.com Agenda Background Introduction Of COLO
More informationVblock Architecture Accelerating Deployment of the Private Cloud
Vblock Architecture Accelerating Deployment of the Private Cloud René Raeber Technical Solutions Architect Datacenter rraeber@cisco.com 1 Vblock Frequently Asked Questions 2 What is a Vblock? It is a product
More informationEvolution of Cloud Computing in ATLAS
The Evolution of Cloud Computing in ATLAS Ryan Taylor on behalf of the ATLAS collaboration 1 Outline Cloud Usage and IaaS Resource Management Software Services to facilitate cloud use Sim@P1 Performance
More informationImproving Packet Processing Performance of a Memory- Bounded Application
Improving Packet Processing Performance of a Memory- Bounded Application Jörn Schumacher CERN / University of Paderborn, Germany jorn.schumacher@cern.ch On behalf of the ATLAS FELIX Developer Team LHCb
More informationI/O CANNOT BE IGNORED
LECTURE 13 I/O I/O CANNOT BE IGNORED Assume a program requires 100 seconds, 90 seconds for main memory, 10 seconds for I/O. Assume main memory access improves by ~10% per year and I/O remains the same.
More informationMoneta: A High-Performance Storage Architecture for Next-generation, Non-volatile Memories
Moneta: A High-Performance Storage Architecture for Next-generation, Non-volatile Memories Adrian M. Caulfield Arup De, Joel Coburn, Todor I. Mollov, Rajesh K. Gupta, Steven Swanson Non-Volatile Systems
More informationOptimizing Parallel Access to the BaBar Database System Using CORBA Servers
SLAC-PUB-9176 September 2001 Optimizing Parallel Access to the BaBar Database System Using CORBA Servers Jacek Becla 1, Igor Gaponenko 2 1 Stanford Linear Accelerator Center Stanford University, Stanford,
More informationECE 598 Advanced Operating Systems Lecture 22
ECE 598 Advanced Operating Systems Lecture 22 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 19 April 2016 Announcements Project update HW#9 posted, a bit late Midterm next Thursday
More informationAffordable and power efficient computing for high energy physics: CPU and FFT benchmarks of ARM processors
Affordable and power efficient computing for high energy physics: CPU and FFT benchmarks of ARM processors Mitchell A Cox, Robert Reed and Bruce Mellado School of Physics, University of the Witwatersrand.
More informationOptimizing Flash-based Key-value Cache Systems
Optimizing Flash-based Key-value Cache Systems Zhaoyan Shen, Feng Chen, Yichen Jia, Zili Shao Department of Computing, Hong Kong Polytechnic University Computer Science & Engineering, Louisiana State University
More informationEMC Virtual Infrastructure for Microsoft Exchange 2010 Enabled by EMC Symmetrix VMAX, VMware vsphere 4, and Replication Manager
EMC Virtual Infrastructure for Microsoft Exchange 2010 Enabled by EMC Symmetrix VMAX, VMware vsphere 4, and Replication Manager Reference Architecture Copyright 2010 EMC Corporation. All rights reserved.
More informationSome Joules Are More Precious Than Others: Managing Renewable Energy in the Datacenter
Some Joules Are More Precious Than Others: Managing Renewable Energy in the Datacenter Christopher Stewart The Ohio State University cstewart@cse.ohio-state.edu Kai Shen University of Rochester kshen@cs.rochester.edu
More informationTwo-Choice Randomized Dynamic I/O Scheduler for Object Storage Systems. Dong Dai, Yong Chen, Dries Kimpe, and Robert Ross
Two-Choice Randomized Dynamic I/O Scheduler for Object Storage Systems Dong Dai, Yong Chen, Dries Kimpe, and Robert Ross Parallel Object Storage Many HPC systems utilize object storage: PVFS, Lustre, PanFS,
More informationCopyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12
1 Copyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 Managing Oracle Database 12c with Oracle Enterprise Manager 12c Martin
More informationScaling to Petaflop. Ola Torudbakken Distinguished Engineer. Sun Microsystems, Inc
Scaling to Petaflop Ola Torudbakken Distinguished Engineer Sun Microsystems, Inc HPC Market growth is strong CAGR increased from 9.2% (2006) to 15.5% (2007) Market in 2007 doubled from 2003 (Source: IDC
More informationPerformance of ORBs on Switched Fabric Transports
Performance of ORBs on Switched Fabric Transports Victor Giddings Objective Interface Systems victor.giddings@ois.com 2001 Objective Interface Systems, Inc. Switched Fabrics High-speed interconnects High-bandwidth,
More informationCMPS 111 Spring 2003 Midterm Exam May 8, Name: ID:
CMPS 111 Spring 2003 Midterm Exam May 8, 2003 Name: ID: This is a closed note, closed book exam. There are 20 multiple choice questions and 5 short answer questions. Plan your time accordingly. Part I:
More informationCisco HyperFlex HX220c M4 Node
Data Sheet Cisco HyperFlex HX220c M4 Node A New Generation of Hyperconverged Systems To keep pace with the market, you need systems that support rapid, agile development processes. Cisco HyperFlex Systems
More informationThread Cluster Memory Scheduling: Exploiting Differences in Memory Access Behavior. Yoongu Kim Michael Papamichael Onur Mutlu Mor Harchol-Balter
Thread Cluster Memory Scheduling: Exploiting Differences in Memory Access Behavior Yoongu Kim Michael Papamichael Onur Mutlu Mor Harchol-Balter Motivation Memory is a shared resource Core Core Core Core
More informationImplementing Scheduling Algorithms. Real-Time and Embedded Systems (M) Lecture 9
Implementing Scheduling Algorithms Real-Time and Embedded Systems (M) Lecture 9 Lecture Outline Implementing real time systems Key concepts and constraints System architectures: Cyclic executive Microkernel
More informationFlash In the Data Center
Flash In the Data Center Enterprise-grade Morgan Littlewood: VP Marketing and BD Violin Memory, Inc. Email: littlewo@violin-memory.com Mobile: +1.650.714.7694 7/12/2009 1 Flash in the Data Center Nothing
More informationAnnouncements. Reading. Project #1 due in 1 week at 5:00 pm Scheduling Chapter 6 (6 th ed) or Chapter 5 (8 th ed) CMSC 412 S14 (lect 5)
Announcements Reading Project #1 due in 1 week at 5:00 pm Scheduling Chapter 6 (6 th ed) or Chapter 5 (8 th ed) 1 Relationship between Kernel mod and User Mode User Process Kernel System Calls User Process
More informationTwo hours - online. The exam will be taken on line. This paper version is made available as a backup
COMP 25212 Two hours - online The exam will be taken on line. This paper version is made available as a backup UNIVERSITY OF MANCHESTER SCHOOL OF COMPUTER SCIENCE System Architecture Date: Monday 21st
More informationVirtual Leverage: Server Consolidation in Open Source Environments. Margaret Lewis Commercial Software Strategist AMD
Virtual Leverage: Server Consolidation in Open Source Environments Margaret Lewis Commercial Software Strategist AMD What Is Virtualization? Abstraction of Hardware Components Virtual Memory Virtual Volume
More informationWebinar Series: Triangulate your Storage Architecture with SvSAN Caching. Luke Pruen Technical Services Director
Webinar Series: Triangulate your Storage Architecture with SvSAN Caching Luke Pruen Technical Services Director What can you expect from this webinar? To answer a simple question How can I create the perfect
More informationSoftNAS Cloud Performance Evaluation on AWS
SoftNAS Cloud Performance Evaluation on AWS October 25, 2016 Contents SoftNAS Cloud Overview... 3 Introduction... 3 Executive Summary... 4 Key Findings for AWS:... 5 Test Methodology... 6 Performance Summary
More informationB.H.GARDI COLLEGE OF ENGINEERING & TECHNOLOGY (MCA Dept.) Parallel Database Database Management System - 2
Introduction :- Today single CPU based architecture is not capable enough for the modern database that are required to handle more demanding and complex requirements of the users, for example, high performance,
More informationCourse Syllabus. Operating Systems
Course Syllabus. Introduction - History; Views; Concepts; Structure 2. Process Management - Processes; State + Resources; Threads; Unix implementation of Processes 3. Scheduling Paradigms; Unix; Modeling
More informationApplication of Virtualization Technologies & CernVM. Benedikt Hegner CERN
Application of Virtualization Technologies & CernVM Benedikt Hegner CERN Virtualization Use Cases Worker Node Virtualization Software Testing Training Platform Software Deployment }Covered today Server
More information