Jobs Resource Utilization as a Metric for Clusters Comparison and Optimization. Slurm User Group Meeting 9-10 October, 2012
|
|
- Erick Conley
- 6 years ago
- Views:
Transcription
1 Jobs Resource Utilization as a Metric for Clusters Comparison and Optimization Joseph Emeras Cristian Ruiz Jean-Marc Vincent Olivier Richard Slurm User Group Meeting 9-10 October, 2012
2 INRIA - MESCAL Jobs Resource Utilization 2 / 26 Tracing of jobs consumptions for about a year Processor, disk, memory, network, but unfortunately not energy 2 production clusters Analyze the processor resource consumption vs. the resource allocation Results were not exactly what we expected...
3 Outline INRIA - MESCAL Jobs Resource Utilization 3 / 26 1 Context CIMENT Mesocenter System Utilization Metric 2 Monitoring Tool 3 Analysis Data Processing Resource Utilization Criterion 4 Discussion
4 Outline INRIA - MESCAL Jobs Resource Utilization Context 4 / 26 1 Context CIMENT Mesocenter System Utilization Metric 2 Monitoring Tool 3 Analysis Data Processing Resource Utilization Criterion 4 Discussion
5 CIMENT Mesocenter Concepts Mesocenter, several universities and labs share computational resources. Lightweight grid software CiGri. Submission of multi-parametric jobs, bag of task, big amount of jobs. Based on the submission of Low priority jobs, preemptible called Best-eort jobs. Improve the cluster utilization, by using unused space (free resources). Production clusters. Various domains : Chemistry, Astronomy, Physics, etc. Underlying Batch Scheduler is OAR (it could have been Slurm...) INRIA - MESCAL Jobs Resource Utilization Context 5 / 26
6 CIGRI problematics INRIA - MESCAL Jobs Resource Utilization Context 6 / 26 Questions Best-Eort jobs seems a good idea, platform resources are highly allocated. But is it really a good thing to over-allocate? (bottlenecks?) Other optimizations possible? We need data from jobs consumptions. OAR, as SLURM provides an accounting plugin, gives means. We suspected that consumption means are not enough, we have very long jobs (days). Indeed, 30% of the jobs had a std dev over 20%, for CPU consumption. We need ner granularity.
7 System Utilization as a metric INRIA - MESCAL Jobs Resource Utilization Context 7 / 26 Metric commonly used to evaluate computational infrastructures utilization. Denition : ratio of the computing resources allocated to the jobs over the available resources in the system. Downsides : misses information about the computer sub-systems (disk, memory, processor). Jobs resource utilization.
8 INRIA - MESCAL Jobs Resource Utilization Context 8 / 26 5e+05 type Computational utilization System_utilization 4e+05 3e+05 Utilization 2e+05 1e+05 0e+00 Jun 11 Jul 11 Aug 11 Sep 11 Oct 11 Nov 11 Dec 11 Jan 12 Feb 12 Mar 12 Time Figure: System Utilization vs Computational Resource Utilization (for normal jobs from one of the CIMENT Clusters)
9 Outline INRIA - MESCAL Jobs Resource Utilization Monitoring Tool 9 / 26 1 Context CIMENT Mesocenter System Utilization Metric 2 Monitoring Tool 3 Analysis Data Processing Resource Utilization Criterion 4 Discussion
10 Jobs traces INRIA - MESCAL Jobs Resource Utilization Monitoring Tool 10 / 26 The idea is to have a trace of resources consumption per job. Dierent from other monitoring approaches such as Ganglia[2], Nagios[1], etc. Job centric monitor. Information about (CPU, memory, IO, Network) consumption. Figure: Job CPU consumption over time
11 Choices for Tracing Jobs Consumptions INRIA - MESCAL Jobs Resource Utilization Monitoring Tool 11 / 26 Characteristics Independent : No synchronization among the nodes. Use of mechanisms supported by most of the batch schedulers. Lightweight ( Sampling approach ) 0.35% speed down.
12 Choices for Tracing Jobs Consumptions INRIA - MESCAL Jobs Resource Utilization Monitoring Tool 11 / 26 Characteristics Independent : No synchronization among the nodes. Use of mechanisms supported by most of the batch schedulers. Lightweight ( Sampling approach ) 0.35% speed down. Lightweight approach 1 min frequency sampling, to avoid storage overhead and to not interfere with jobs execution. Capture not expensive using /proc directory. Data processing done o-line.
13 Clusters characteristics INRIA - MESCAL Jobs Resource Utilization Monitoring Tool 12 / 26 Foehn Gofree CPU Model Xeon X5550 Xeon L5640 Nodes CPU cores Total memory 480 GB 2016 GB Memory/node 24/48 GB 72 GB Total storage 7 TB 30 TB Network IB DDR IB QDR Total Gop/s Buy date
14 Trace summary INRIA - MESCAL Jobs Resource Utilization Monitoring Tool 13 / 26 Foehn Gofree Capture start Capture months 9 9 Number of jobs Log Size 2.5 Gb 3.0 Gb Besteort jobs Normal jobs
15 Outline INRIA - MESCAL Jobs Resource Utilization Analysis 14 / 26 1 Context CIMENT Mesocenter System Utilization Metric 2 Monitoring Tool 3 Analysis Data Processing Resource Utilization Criterion 4 Discussion
16 Data Processing Pre-analysis O-line data processing Traces Correlation Batch Scheduler and Resource Manager logs (SWF Format) INRIA - MESCAL Jobs Resource Utilization Analysis 15 / 26
17 Standard Workload Format INRIA - MESCAL Jobs Resource Utilization Analysis 16 / 26 SWF Format Headers comments Fields Version Job Number Computer Submit Time Installation Wait Time Acknowledge Run Time Information Number of Allocated Processors Conversion Average CPU Time Used MaxRecords Used Memory Preemption Requested Number of Processors UnixStartTime Requested Time TimeZone Requested Memory TimeZoneString Status StartTime User ID Endtime Group ID MaxNodes Executable Number MaxProcs Queue Number MaxRuntime Partition Number MaxMemory Preceding Job Number AllowOveruse Think Time form Proceeding Job MaxQueues Queues Queue MaxPartitions Partitions Partition Note
18 Data Processing INRIA - MESCAL Jobs Resource Utilization Analysis 17 / 26 Pre-analysis O-line data processing Traces Correlation Batch Scheduler and Resource Manager logs (SWF Format) Jobs resource consumption logs
19 Jobs Resource Consumption Logs INRIA - MESCAL Jobs Resource Utilization Analysis 18 / 26 Trace Fields Name Description Time Unix Time Stamp in seconds JOB ID Job id assigned by the batch scheduler PID PID of process that belongs to the job Node ID Provenance of the capture Measure List of measures of the resource consumptions Measure (simplied version) Name Description command Name of the binary executed memory_faults Number of memory faults and their type virtual_memory Virtual memory size pages Pages used by process and their type IO_r_w Num of bytes read/written from/to the storage layer core_usage Core utilization percentage net_r_w Network Read and Written Bytes......
20 Resource Utilization Criterion INRIA - MESCAL Jobs Resource Utilization Analysis 19 / 26 Criterion presentation Match Resource Consumption with Resource Allocation X axis : allocated resources over available Y axis : core consumption in function of X Green diagonal : Theoretical optimum, distances from the line = computing power loss Density graphs : darker point means denser distribution (alpha=0.1) Gofree Cluster Foehn Cluster
21 Gofree Cluster INRIA - MESCAL Jobs Resource Utilization Analysis 20 / 26 Gofree BestEort Jobs Full consumption of reserved cores BestEort jobs are core ecient. Gofree Normal Jobs Red stripe : 80% of the values. Linear regression gave a slope. (red stripe : stripe between the optimum and the mean of the distances from the optimum)
22 Foehn Cluster INRIA - MESCAL Jobs Resource Utilization Analysis 21 / 26 SA = alloc j run_time j. j jobs Foehn BestEort Jobs 2 patterns : one optimal, one 1/4. One user with particular needs for memory bandwidth. Foehn Normal Jobs 2 low-utilization clouds. Outliers (up to 110%) due to /proc roundings when consumption jitter.
23 Users Special Needs INRIA - MESCAL Jobs Resource Utilization Analysis 22 / 26 Needs not expressible by the user to the batch scheduler? IO/Memory Bandwidth Constraint accept the loss? modify batch scheduler? bandwidth as a resource?
24 Outline INRIA - MESCAL Jobs Resource Utilization Discussion 23 / 26 1 Context CIMENT Mesocenter System Utilization Metric 2 Monitoring Tool 3 Analysis Data Processing Resource Utilization Criterion 4 Discussion
25 Discussion INRIA - MESCAL Jobs Resource Utilization Discussion 24 / 26 System Instrumentation Interesting and at little cost (our implementation : 0.35% speed-down) Need to correlate with other metrics (memory size and bandwidth usage, IO) Results Processor consumption on 2 clusters of same grid can be very dierent Users behaviors impact Shows Batch Scheduler request constraint lacks Future Works Characterize jobs consumption patterns Learn from past, classify couple <User,Code> On-Line tagging of jobs at submission, useful to anticipate IO intensive jobs Ideas SWF : elds for Max Memory, particular user constraint (license, hardware, locality) Slurm : plugin for tracing the jobs by sampling
26 Data available in a git repository INRIA - MESCAL Jobs Resource Utilization Discussion 25 / 26
27 Thank you for your attention INRIA - MESCAL Jobs Resource Utilization Discussion 26 / 26 joseph.emeras@imag.fr
28 INRIA - MESCAL Jobs Resource Utilization Discussion 26 / 26 Emir Imamagic and Dobrisa Dobrenic. Grid infrastructure monitoring system based on nagios. In Proceedings of the 2007 workshop on Grid monitoring, GMW '07, pages 2328, New York, NY, USA, ACM. Matthew L. Massie, Brent N. Chun, and David E. Culler. The ganglia distributed monitoring system : Design, implementation and experience, 2004.
Monitoring ARC services with GangliARC
Journal of Physics: Conference Series Monitoring ARC services with GangliARC To cite this article: D Cameron and D Karpenko 2012 J. Phys.: Conf. Ser. 396 032018 View the article online for updates and
More informationData Transfers in the Grid: Workload Analysis of Globus GridFTP
Data Transfers in the Grid: Workload Analysis of Globus GridFTP Nicolas Kourtellis, Lydia Prieto, Gustavo Zarrate, Adriana Iamnitchi University of South Florida Dan Fraser Argonne National Laboratory Objective
More informationDiffusing Your Mobile Apps: Extending In-Network Function Virtualisation to Mobile Function Offloading
Diffusing Your Mobile Apps: Extending In-Network Function Virtualisation to Mobile Function Offloading Mario Almeida, Liang Wang*, Jeremy Blackburn, Konstantina Papagiannaki, Jon Crowcroft* Telefonica
More informationADAPTIVE AND DYNAMIC LOAD BALANCING METHODOLOGIES FOR DISTRIBUTED ENVIRONMENT
ADAPTIVE AND DYNAMIC LOAD BALANCING METHODOLOGIES FOR DISTRIBUTED ENVIRONMENT PhD Summary DOCTORATE OF PHILOSOPHY IN COMPUTER SCIENCE & ENGINEERING By Sandip Kumar Goyal (09-PhD-052) Under the Supervision
More informationCluster Computing. Resource and Job Management for HPC 16/08/2010 SC-CAMP. ( SC-CAMP) Cluster Computing 16/08/ / 50
Cluster Computing Resource and Job Management for HPC SC-CAMP 16/08/2010 ( SC-CAMP) Cluster Computing 16/08/2010 1 / 50 Summary 1 Introduction Cluster Computing 2 About Resource and Job Management Systems
More informationPRESENTATION TITLE GOES HERE
Performance Basics PRESENTATION TITLE GOES HERE Leah Schoeb, Member of SNIA Technical Council SNIA EmeraldTM Training SNIA Emerald Power Efficiency Measurement Specification, for use in EPA ENERGY STAR
More informationA Container On a Virtual Machine On an HPC? Presentation to HPC Advisory Council. Perth, July 31-Aug 01, 2017
A Container On a Virtual Machine On an HPC? Presentation to HPC Advisory Council Perth, July 31-Aug 01, 2017 http://levlafayette.com Necessary and Sufficient Definitions High Performance Computing: High
More informationAltix Usage and Application Programming
Center for Information Services and High Performance Computing (ZIH) Altix Usage and Application Programming Discussion And Important Information For Users Zellescher Weg 12 Willers-Bau A113 Tel. +49 351-463
More informationPerformance Measurement and Evaluation Tool for Large-scale Systems
Performance Measurement and Evaluation Tool for Large-scale Systems Hong Ong ORNL hongong@ornl.gov December 7 th, 2005 Acknowledgements This work is sponsored in parts by: The High performance Computing
More informationMOHA: Many-Task Computing Framework on Hadoop
Apache: Big Data North America 2017 @ Miami MOHA: Many-Task Computing Framework on Hadoop Soonwook Hwang Korea Institute of Science and Technology Information May 18, 2017 Table of Contents Introduction
More informationEindhoven University of Technology - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
Eindhoven University of Technology - FTP Site Statistics Property Value FTP Server ftp.tue.nl Description Eindhoven University of Technology Country Netherlands Scan Date 10/May/2014 Total Dirs 129 Total
More informationThe Power of Prediction: Cloud Bandwidth and Cost Reduction
The Power of Prediction: Cloud Bandwidth and Cost Reduction Eyal Zohar Israel Cidon Technion Osnat(Ossi) Mokryn Tel-Aviv College Traffic Redundancy Elimination (TRE) Traffic redundancy stems from downloading
More informationSparkBench: A Comprehensive Spark Benchmarking Suite Characterizing In-memory Data Analytics
SparkBench: A Comprehensive Spark Benchmarking Suite Characterizing In-memory Data Analytics Min LI,, Jian Tan, Yandong Wang, Li Zhang, Valentina Salapura, Alan Bivens IBM TJ Watson Research Center * A
More informationThe Fusion Distributed File System
Slide 1 / 44 The Fusion Distributed File System Dongfang Zhao February 2015 Slide 2 / 44 Outline Introduction FusionFS System Architecture Metadata Management Data Movement Implementation Details Unique
More informationLarge-scale cluster management at Google with Borg
Large-scale cluster management at Google with Borg Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune, John Wilkes Google Inc. Slides heavily derived from John Wilkes s presentation
More informationVxRail: Level Up with New Capabilities and Powers GLOBAL SPONSORS
VxRail: Level Up with New Capabilities and Powers GLOBAL SPONSORS VMware customers trust their infrastructure to vsan #1 Leading SDS Vendor >10,000 >100 83% vsan Customers Countries Deployed Critical Apps
More informationPredicting Web Service Levels During VM Live Migrations
Predicting Web Service Levels During VM Live Migrations 5th International DMTF Academic Alliance Workshop on Systems and Virtualization Management: Standards and the Cloud Helmut Hlavacs, Thomas Treutner
More informationUNITE Managing Metering: How to Plan for, Tune and Monitor Your Metered Platform. Session MCP3040/OS3040. Guy Bonney & Bob Morrow, MGS, Inc.
UNITE 2006 Managing Metering: How to Plan for, Tune and Monitor Your Metered Platform Session MCP3040/OS3040 Guy Bonney & Bob Morrow, MGS, Inc. Debi Ray, SightLine Systems Corporation MGS, Inc.! Software
More informationEECS 428 Final Project Report Distributed Real-Time Process Control Over TCP and the Internet Brian Robinson
EECS 428 Final Project Report Distributed Real-Time Process Control Over TCP and the Internet Brian Robinson 1.0 Introduction Distributed real-time process control, from a network communication view, involves
More informationIntroduction to Grid Computing
Milestone 2 Include the names of the papers You only have a page be selective about what you include Be specific; summarize the authors contributions, not just what the paper is about. You might be able
More informationData Intensive processing with irods and the middleware CiGri for the Whisper project Xavier Briand
and the middleware CiGri for the Whisper project Use Case of Data-Intensive processing with irods Collaboration between: IT part of Whisper: Sofware development, computation () Platform Ciment: IT infrastructure
More informationManaging CAE Simulation Workloads in Cluster Environments
Managing CAE Simulation Workloads in Cluster Environments Michael Humphrey V.P. Enterprise Computing Altair Engineering humphrey@altair.com June 2003 Copyright 2003 Altair Engineering, Inc. All rights
More informationImage Processing on the Cloud. Outline
Mars Science Laboratory! Image Processing on the Cloud Emily Law Cloud Computing Workshop ESIP 2012 Summer Meeting July 14 th, 2012 1/26/12! 1 Outline Cloud computing @ JPL SDS Lunar images Challenge Image
More informationMonthly SEO Report. Example Client 16 November 2012 Scott Lawson. Date. Prepared by
Date Monthly SEO Report Prepared by Example Client 16 November 212 Scott Lawson Contents Thanks for using TrackPal s automated SEO and Analytics reporting template. Below is a brief explanation of the
More informationTechnical University of Munich - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
Technical University of Munich - FTP Site Statistics Property Value FTP Server ftp.ldv.e-technik.tu-muenchen.de Description Technical University of Munich Country Germany Scan Date 23/May/2014 Total Dirs
More informationEDUCATION RESEARCH EXPERIENCE
PERSONAL Name: Mais Nijim Gender: Female Address: 901 walkway, apartment A1 Socorro, NM 87801 Email: mais@cs.nmt.edu Phone: (505)517-0150 (505)650-0400 RESEARCH INTEREST Computer Architecture Storage Systems
More informationPreemptive, Low Latency Datacenter Scheduling via Lightweight Virtualization
Preemptive, Low Latency Datacenter Scheduling via Lightweight Virtualization Wei Chen, Jia Rao*, and Xiaobo Zhou University of Colorado, Colorado Springs * University of Texas at Arlington Data Center
More informationUniversity of Rochester - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
Property Value FTP Server ftp.cs.rochester.edu Description University of Rochester Country United States Scan Date 02/Sep/2015 Total Dirs 204 Total Files 2,949 Total Data 20.85 GB Top 20 Directories Sorted
More informationNational Instruments Software Archive - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
National Instruments Software Archive - FTP Site Statistics Property Value FTP Server ftp.ni.com Description National Instruments Software Archive Country United States Scan Date 13/May/2015 Total Dirs
More informationServer Virtualization and Optimization at HSBC. John Gibson Chief Technical Specialist HSBC Bank plc
Server Virtualization and Optimization at HSBC John Gibson Chief Technical Specialist HSBC Bank plc Background Over 5,500 Windows servers in the last 6 years. Historically, Windows technology dictated
More informationOn the Performance of MapReduce: A Stochastic Approach
On the Performance of MapReduce: A Stochastic Approach Sarker Tanzir Ahmed and Dmitri Loguinov Internet Research Lab Department of Computer Science and Engineering Texas A&M University October 28, 2014
More informationdavidklee.net gplus.to/kleegeek linked.com/a/davidaklee
@kleegeek davidklee.net gplus.to/kleegeek linked.com/a/davidaklee Specialties / Focus Areas / Passions: Performance Tuning & Troubleshooting Virtualization Cloud Enablement Infrastructure Architecture
More informationSOFT 437. Software Performance Analysis. Ch 7&8:Software Measurement and Instrumentation
SOFT 437 Software Performance Analysis Ch 7&8: Why do we need data? Data is required to calculate: Software execution model System execution model We assumed that we have required data to calculate these
More informationUAntwerpen, 24 June 2016
Tier-1b Info Session UAntwerpen, 24 June 2016 VSC HPC environment Tier - 0 47 PF Tier -1 623 TF Tier -2 510 Tf 16,240 CPU cores 128/256 GB memory/node IB EDR interconnect Tier -3 HOPPER/TURING STEVIN THINKING/CEREBRO
More informationTask-based distributed processing for radio-interferometric imaging with CASA
H2020-Astronomy ESFRI and Research Infrastructure Cluster (Grant Agreement number: 653477). Task-based distributed processing for radio-interferometric imaging with CASA BOJAN NIKOLIC 2 nd ASTERICS-OBELICS
More informationHigh Performance Computing Cloud - a PaaS Perspective
a PaaS Perspective Supercomputer Education and Research Center Indian Institute of Science, Bangalore November 2, 2015 Overview Cloud computing is emerging as a latest compute technology Properties of
More informationRELIABILITY IN CLOUD COMPUTING SYSTEMS: SESSION 1
RELIABILITY IN CLOUD COMPUTING SYSTEMS: SESSION 1 Dr. Bahman Javadi School of Computing, Engineering and Mathematics Western Sydney University, Australia 1 TUTORIAL AGENDA Session 1: Reliability in Cloud
More informationAdvancing the Art of Internet Edge Outage Detection
Advancing the Art of Internet Edge Outage Detection ACM Internet Measurement Conference 2018 Philipp Richter MIT / Akamai Ramakrishna Padmanabhan University of Maryland Neil Spring University of Maryland
More informationUniversity of California - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
Property Value FTP Server ftp.icsi.berkeley.edu Description University of California Country United States Scan Date 15/Jun/2015 Total Dirs 591 Total Files 12,510 Total Data 10.83 GB Top 20 Directories
More informationDAS LRS Monthly Service Report
DAS LRS Monthly Service Report Customer Service Manager : Diploma Aggregation Service : Daniel Ward Project/document reference : DAS LRS 2010-12 Issue : 1.0 Issue date : 17 th January 2011 Reporting Period
More informationUndergraduate Admission File
Undergraduate Admission File June 13, 2007 Information Resources and Communications Office of the President University of California Overview Population The Undergraduate Admission File contains data on
More informationAPENet: LQCD clusters a la APE
Overview Hardware/Software Benchmarks Conclusions APENet: LQCD clusters a la APE Concept, Development and Use Roberto Ammendola Istituto Nazionale di Fisica Nucleare, Sezione Roma Tor Vergata Centro Ricerce
More informationGeant4 on Azure using Docker containers
http://www.geant4.org Geant4 on Azure using Docker containers Andrea Dotti (adotti@slac.stanford.edu) ; SD/EPP/Computing 1 Outlook Motivation/overview Docker + G4 Azure + G4 Conclusions 2 Motivation/overview
More informationAVM Networks - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
Property Value FTP Server ftp.avm.de Description AVM Networks Country Germany Scan Date 12/May/2014 Total Dirs 2,056 Total Files 2,698 Total Data 39.66 GB Top 20 Directories Sorted by Disk Space Name Dirs
More informationHigh-density Grid storage system optimization at ASGC. Shu-Ting Liao ASGC Operation team ISGC 2011
High-density Grid storage system optimization at ASGC Shu-Ting Liao ASGC Operation team ISGC 211 Outline Introduction to ASGC Grid storage system Storage status and issues in 21 Storage optimization Summary
More informationStatistical Methods in Trending. Ron Spivey RETIRED Associate Director Global Complaints Tending Alcon Laboratories
Statistical Methods in Trending Ron Spivey RETIRED Associate Director Global Complaints Tending Alcon Laboratories What s In It For You? Basic Statistics in Complaint Trending Basic Complaint Trending
More informationParallel Storage Systems for Large-Scale Machines
Parallel Storage Systems for Large-Scale Machines Doctoral Showcase Christos FILIPPIDIS (cfjs@outlook.com) Department of Informatics and Telecommunications, National and Kapodistrian University of Athens
More informationModeling and Tolerating Heterogeneous Failures in Large Parallel Systems
Modeling and Tolerating Heterogeneous Failures in Large Parallel Systems Eric Heien 1, Derrick Kondo 1, Ana Gainaru 2, Dan LaPine 2, Bill Kramer 2, Franck Cappello 1, 2 1 INRIA, France 2 UIUC, USA Context
More informationSection 1.2: What is a Function? y = 4x
Section 1.2: What is a Function? y = 4x y is the dependent variable because it depends on what x is. x is the independent variable because any value can be chosen to replace x. Domain: a set of values
More informationA More Realistic Way of Stressing the End-to-end I/O System
A More Realistic Way of Stressing the End-to-end I/O System Verónica G. Vergara Larrea Sarp Oral Dustin Leverman Hai Ah Nam Feiyi Wang James Simmons CUG 2015 April 29, 2015 Chicago, IL ORNL is managed
More informationCPU Scheduling. CSE 2431: Introduction to Operating Systems Reading: Chapter 6, [OSC] (except Sections )
CPU Scheduling CSE 2431: Introduction to Operating Systems Reading: Chapter 6, [OSC] (except Sections 6.7.2 6.8) 1 Contents Why Scheduling? Basic Concepts of Scheduling Scheduling Criteria A Basic Scheduling
More informationChapter 9 Memory Management
Contents 1. Introduction 2. Computer-System Structures 3. Operating-System Structures 4. Processes 5. Threads 6. CPU Scheduling 7. Process Synchronization 8. Deadlocks 9. Memory Management 10. Virtual
More informationThe Elasticity and Plasticity in Semi-Containerized Colocating Cloud Workload: a view from Alibaba Trace
The Elasticity and Plasticity in Semi-Containerized Colocating Cloud Workload: a view from Alibaba Trace Qixiao Liu* and Zhibin Yu Shenzhen Institute of Advanced Technology Chinese Academy of Science @SoCC
More informationLogging Mechanism. Cisco Logging Mechanism
Cisco, page 1 Cisco ISE System Logs, page 2 Configure Remote Syslog Collection Locations, page 7 Cisco ISE Message Codes, page 8 Cisco ISE Message Catalogs, page 8 Debug Logs, page 8 Endpoint Debug Log
More informationIntroduction to High Performance Parallel I/O
Introduction to High Performance Parallel I/O Richard Gerber Deputy Group Lead NERSC User Services August 30, 2013-1- Some slides from Katie Antypas I/O Needs Getting Bigger All the Time I/O needs growing
More informationMoab, TORQUE, and Gold in a Heterogeneous, Federated Computing System at the University of Michigan
Moab, TORQUE, and Gold in a Heterogeneous, Federated Computing System at the University of Michigan Andrew Caird Matthew Britt Brock Palen September 18, 2009 Who We Are College of Engineering centralized
More informationNew Concept for Article 36 Networking and Management of the List
New Concept for Article 36 Networking and Management of the List Kerstin Gross-Helmert, AFSCO 28 th Meeting of the Focal Point Network EFSA, MTG SEAT 00/M08-09 THE PRESENTATION Why a new concept? What
More informationPerformance Extrapolation for Load Testing Results of Mixture of Applications
Performance Extrapolation for Load Testing Results of Mixture of Applications Subhasri Duttagupta, Manoj Nambiar Tata Innovation Labs, Performance Engineering Research Center Tata Consulting Services Mumbai,
More information3. EXCEL FORMULAS & TABLES
Winter 2019 CS130 - Excel Formulas & Tables 1 3. EXCEL FORMULAS & TABLES Winter 2019 Winter 2019 CS130 - Excel Formulas & Tables 2 Cell References Absolute reference - refer to cells by their fixed position.
More informationXEmacs Project Archive - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
Property Value FTP Server ftp.ie.xemacs.org Description XEmacs Project Archive Country Ireland Scan Date 31/Oct/2014 Total Dirs 677 Total Files 4,166 Total Data 4.40 GB Top 20 Directories Sorted by Disk
More informationOutline. The demand The San Jose NAP. What s the Problem? Most things. Time. Part I AN OVERVIEW OF HARDWARE ISSUES FOR IP AND ATM.
Outline AN OVERVIEW OF HARDWARE ISSUES FOR IP AND ATM Name one thing you could achieve with ATM that you couldn t with IP! Nick McKeown Assistant Professor of Electrical Engineering and Computer Science
More informationTuning WebHound 4.0 and SAS 8.2 for Enterprise Windows Systems James R. Lebak, Unisys Corporation, Malvern, PA
Paper 272-27 Tuning WebHound 4.0 and SAS 8.2 for Enterprise Windows Systems James R. Lebak, Unisys Corporation, Malvern, PA ABSTRACT Windows is SAS largest and fastest growing platform. Windows 2000 Advanced
More informationChap 7, 8: Scheduling. Dongkun Shin, SKKU
Chap 7, 8: Scheduling 1 Introduction Multiprogramming Multiple processes in the system with one or more processors Increases processor utilization by organizing processes so that the processor always has
More informationBigDataBench-MT: Multi-tenancy version of BigDataBench
BigDataBench-MT: Multi-tenancy version of BigDataBench Gang Lu Beijing Academy of Frontier Science and Technology BigDataBench Tutorial, ASPLOS 2016 Atlanta, GA, USA n Software perspective Multi-tenancy
More informationsoftware.sci.utah.edu (Select Visitors)
software.sci.utah.edu (Select Visitors) Web Log Analysis Yearly Report 2002 Report Range: 02/01/2002 00:00:0-12/31/2002 23:59:59 www.webtrends.com Table of Contents Top Visitors...3 Top Visitors Over Time...5
More informationSeattle (NWMLS Areas: 140, 380, 385, 390, 700, 701, 705, 710) Summary
October, 2016 MTD MARKET UPDATE Data Current Through: October, 2016 (NWMLS Areas: 140, 380, 385, 390,, 701, 705, 710) Summary Active, Pending, & Months Supply of Inventory 4,500 4,000 3,500 4,197 4,128
More informationPowerVault MD3 SSD Cache Overview
PowerVault MD3 SSD Cache Overview A Dell Technical White Paper Dell Storage Engineering October 2015 A Dell Technical White Paper TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS
More informationMeasuring VDI Fitness and User Experience Technical White Paper
Measuring VDI Fitness and User Experience Technical White Paper 3600 Mansell Road Suite 200 Alpharetta, GA 30022 866.914.9665 main 678.397.0339 fax info@liquidwarelabs.com www.liquidwarelabs.com Table
More informationBuilding Simulation Modelers Are we big data ready? Jibonananda Sanyal and Joshua New Oak Ridge National Laboratory
Building Simulation Modelers Are we big data ready? Jibonananda Sanyal and Joshua New Oak Ridge National Laboratory Learning Objectives Appreciate emerging big-data needs in building sciences Identify
More informationPenetrating the Matrix Justin Z. Smith, William Gui Zupko II, U.S. Census Bureau, Suitland, MD
Penetrating the Matrix Justin Z. Smith, William Gui Zupko II, U.S. Census Bureau, Suitland, MD ABSTRACT While working on a time series modeling problem, we needed to find the row and column that corresponded
More informationThe Why and How of HPC-Cloud Hybrids with OpenStack
The Why and How of HPC-Cloud Hybrids with OpenStack OpenStack Australia Day Melbourne June, 2017 Lev Lafayette, HPC Support and Training Officer, University of Melbourne lev.lafayette@unimelb.edu.au 1.0
More informationChoosing Resources Wisely Plamen Krastev Office: 38 Oxford, Room 117 FAS Research Computing
Choosing Resources Wisely Plamen Krastev Office: 38 Oxford, Room 117 Email:plamenkrastev@fas.harvard.edu Objectives Inform you of available computational resources Help you choose appropriate computational
More informationUBS Data Center Efficiency Strategy
UBS Group Technology Public UBS Data Center Efficiency Strategy Steps in Efficiency, Automation leading to enabling the Cloud October 18, 2011 UBS one of the leading financial firms UBS draws on its 150-year
More informationIvane Javakhishvili Tbilisi State University High Energy Physics Institute HEPI TSU
Ivane Javakhishvili Tbilisi State University High Energy Physics Institute HEPI TSU Grid cluster at the Institute of High Energy Physics of TSU Authors: Arnold Shakhbatyan Prof. Zurab Modebadze Co-authors:
More informationNational Aeronautics and Space Admin. - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
National Aeronautics and Space Admin. - FTP Site Statistics Property Value FTP Server ftp.hq.nasa.gov Description National Aeronautics and Space Admin. Country United States Scan Date 26/Apr/2014 Total
More informationFall COMP3511 Review
Outline Fall 2015 - COMP3511 Review Monitor Deadlock and Banker Algorithm Paging and Segmentation Page Replacement Algorithms and Working-set Model File Allocation Disk Scheduling Review.2 Monitors Condition
More informationExcel Functions & Tables
Excel Functions & Tables SPRING 2016 Spring 2016 CS130 - EXCEL FUNCTIONS & TABLES 1 Review of Functions Quick Mathematics Review As it turns out, some of the most important mathematics for this course
More informationWide-Area Reliability Monitoring and Visualization Tools
OE Visualization and Controls Peer Review Wide-Area Reliability Monitoring and Visualization Tools Carlos Martinez CERTS - Electric Power Group 21 October 2008 Washington, D.C. Presentation Outline Research
More informationCharacterization and Modeling of Deleted Questions on Stack Overflow
Characterization and Modeling of Deleted Questions on Stack Overflow Denzil Correa, Ashish Sureka http://correa.in/ February 16, 2014 Denzil Correa, Ashish Sureka (http://correa.in/) ACM WWW-2014 February
More informationThe Oracle Database Appliance I/O and Performance Architecture
Simple Reliable Affordable The Oracle Database Appliance I/O and Performance Architecture Tammy Bednar, Sr. Principal Product Manager, ODA 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
More informationLatency on preemptible Real-Time Linux
Appendix 1 Latency on preemptible Real-Time Linux on DCP-SH7780 Contents 1.1 Getting Started.......................................... 1 1.2 Rtrtc Driver........................................... 2 1.3
More informationUsing BigSim to Estimate Application Performance
October 19, 2010 Using BigSim to Estimate Application Performance Ryan Mokos Parallel Programming Laboratory University of Illinois at Urbana-Champaign Outline Overview BigSim Emulator BigSim Simulator
More informationMilestone Solution Partner IT Infrastructure Components Certification Report
Milestone Solution Partner IT Infrastructure Components Certification Report Dell Storage PS6610, Dell EqualLogic PS6210, Dell EqualLogic FS7610 July 2015 Revisions Date July 2015 Description Initial release
More informationUniversity of Valencia - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
University of Valencia - FTP Site Statistics Property Value FTP Server ftp.uv.es Description University of Valencia Country Spain Scan Date 30/Apr/2014 Total Dirs 423 Total Files 2,010 Total Data 4.46
More informationLinux process scheduling. David Morgan
Linux process scheduling David Morgan General neediness categories realtime processes whenever they demand attention, need it immediately other processes interactive care about responsiveness demand no
More informationMULTITHERMAN: Out-of-band High-Resolution HPC Power and Performance Monitoring Support for Big-Data Analysis
MULTITHERMAN: Out-of-band High-Resolution HPC Power and Performance Monitoring Support for Big-Data Analysis EU H2020 FETHPC project ANTAREX (g.a. 671623) EU FP7 ERC Project MULTITHERMAN (g.a.291125) HPC
More informationWas ist dran an einer spezialisierten Data Warehousing platform?
Was ist dran an einer spezialisierten Data Warehousing platform? Hermann Bär Oracle USA Redwood Shores, CA Schlüsselworte Data warehousing, Exadata, specialized hardware proprietary hardware Introduction
More informationACTIVE MICROSOFT CERTIFICATIONS:
Last Activity Recorded : July 20, 2017 Microsoft Certification ID : 2665612 MARC GROTE Wittorfer Strasse 4 Bardowick, Lower Saxony 21357 DE marc.grote@it-consulting-grote.de ACTIVE MICROSOFT CERTIFICATIONS:
More informationSAS Scalable Performance Data Server 4.3
Scalability Solution for SAS Dynamic Cluster Tables A SAS White Paper Table of Contents Introduction...1 Cluster Tables... 1 Dynamic Cluster Table Loading Benefits... 2 Commands for Creating and Undoing
More informationGWDG Software Archive - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
GWDG Software Archive - FTP Site Statistics Property Value FTP Server ftp5.gwdg.de Description GWDG Software Archive Country Germany Scan Date 18/Jan/2016 Total Dirs 1,068,408 Total Files 30,248,505 Total
More informationDatura The new HPC-Plant at Albert Einstein Institute
Datura The new HPC-Plant at Albert Einstein Institute Nico Budewitz Max Planck Institue for Gravitational Physics, Germany Cluster Day, 2011 Outline 1 History HPC-Plants at AEI -2009 Peyote, Lagavulin,
More informationG Robert Grimm New York University
G22.3250-001 Receiver Livelock Robert Grimm New York University Altogether Now: The Three Questions What is the problem? What is new or different? What are the contributions and limitations? Motivation
More informationMilestone Solution Partner IT Infrastructure Components Certification Report
Milestone Solution Partner IT Infrastructure Components Certification Report Dell MD3860i Storage Array Multi-Server 1050 Camera Test Case 4-2-2016 Table of Contents Executive Summary:... 3 Abstract...
More informationOutline. March 5, 2012 CIRMMT - McGill University 2
Outline CLUMEQ, Calcul Quebec and Compute Canada Research Support Objectives and Focal Points CLUMEQ Site at McGill ETS Key Specifications and Status CLUMEQ HPC Support Staff at McGill Getting Started
More informationOpera Web Browser Archive - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
Property Value FTP Server ftp.opera.com Description Opera Web Browser Archive Country United States Scan Date 04/Nov/2015 Total Dirs 1,557 Total Files 2,211 Total Data 43.83 GB Top 20 Directories Sorted
More informationMonitoring and Analyzing Virtual Machines Resource Overcommitment Detection and Virtual Machine Classification
Monitoring and Analyzing Virtual Machines Resource Overcommitment Detection and Virtual Machine Classification Hani Nemati May 5, 2015 Polytechnique Montréal Laboratoire DORSAL Agenda Motivation Why detecting
More informationReal-time Scheduling of Skewed MapReduce Jobs in Heterogeneous Environments
Real-time Scheduling of Skewed MapReduce Jobs in Heterogeneous Environments Nikos Zacheilas, Vana Kalogeraki Department of Informatics Athens University of Economics and Business 1 Big Data era has arrived!
More informationUniversity of Stuttgart - FTP Site Statistics. Top 20 Directories Sorted by Disk Space
University of Stuttgart - FTP Site Statistics Property Value FTP Server ftp.informatik.uni-stuttgart.de Description University of Stuttgart Country Germany Scan Date 17/Nov/2015 Total Dirs 7,657 Total
More informationPerformance Pack. Benchmarking with PlanetPress Connect and PReS Connect
Performance Pack Benchmarking with PlanetPress Connect and PReS Connect Contents 2 Introduction 4 Benchmarking results 5 First scenario: Print production on demand 5 Throughput vs. Output Speed 6 Second
More informationOBTAINING AN ACCOUNT:
HPC Usage Policies The IIA High Performance Computing (HPC) System is managed by the Computer Management Committee. The User Policies here were developed by the Committee. The user policies below aim to
More information