Queuing and Scheduling on Compute Clusters
|
|
|
- Randolf Lester
- 6 years ago
- Views:
Transcription
1 Queuing and Scheduling on Compute Clusters Andrew Caird Queuing and Scheduling on Compute Clusters p.1/17
2 The reason for me being here Give some queuing background Introduce some queuing ideas Talk about what we do at the Center for Advanced Computing Queuing and Scheduling on Compute Clusters p.2/17
3 CAC The Center for Advanced Computing 3 full-time employees, 1 student Queuing and Scheduling on Compute Clusters p.3/17
4 CAC The Center for Advanced Computing 3 full-time employees, 1 student 4 clusters, 580 dual-processor nodes largest cluster is called nyx 256 nodes, 6 partitions: 5 private, 1 general 150 jobs scheduled per day for 150 people 50 jobs per day in the general partition for 100 people Queuing and Scheduling on Compute Clusters p.3/17
5 CAC The Center for Advanced Computing 3 full-time employees, 1 student 4 clusters, 580 dual-processor nodes largest cluster is called nyx 256 nodes, 6 partitions: 5 private, 1 general 150 jobs scheduled per day for 150 people 50 jobs per day in the general partition for 100 people Either we know what we re doing, or we re making an enormous disaster Queuing and Scheduling on Compute Clusters p.3/17
6 The Problem efficient use of resources On a 256 node cluster, a 10% increase in efficiency is 25 nodes, or about $50,000/year in hardware and staff costs guaranteeing the owners of private nodes access and the same efficiency Queuing and Scheduling on Compute Clusters p.4/17
7 The Resource Manager starts and stops jobs stage-in/-out of files used by end-users to submit jobs, check job status manages queues can manage resources usually comes with some sort of scheduler Queuing and Scheduling on Compute Clusters p.5/17
8 The Scheduler tells resource manager which jobs to start and stop maintains job properties (class, QoS, etc.) schedules jobs based on information from the resource manager and its own info can manage resources sets and manages reservations Queuing and Scheduling on Compute Clusters p.6/17
9 Queuing Products PBSPro from Altair ( We use this on Linux and MacOS Very stable, scales well Opaque scheduler, poor interface with other schedulers Queuing and Scheduling on Compute Clusters p.7/17
10 Queuing Products PBSPro from Altair ( We use this on Linux and MacOS Very stable, scales well Opaque scheduler, poor interface with other schedulers Torque (was OpenPBS) ( We used this on Linux Much more stable than in the past; lots of recent development Only a simple scheduler, great interface to Maui (same developers) Queuing and Scheduling on Compute Clusters p.7/17
11 Queuing Products Maui / Moab (only schedulers) ( Maui: free, Moab: commercial very transparent, very configurable Queuing and Scheduling on Compute Clusters p.8/17
12 Queuing Products Maui / Moab (only schedulers) ( Maui: free, Moab: commercial very transparent, very configurable Load Sharing Facility (LSF) from Platform Computing ( Sun Grid Engine (SGE) open-sourced from Sun (gridengine.sunsource.net) LoadLeveler from IBM ( clusters/software/ loadleveler.html) Queuing and Scheduling on Compute Clusters p.8/17
13 Our scheduling concepts Scheduling Concepts: Resource Manager dependent or Scheduler dependent queue permissions, node-to-queue mapping, etc. is controlled by one or the other Queuing and Scheduling on Compute Clusters p.9/17
14 Resource Manager Dependant Resource Manager Dependant (RMD) Queuing RM has node-to-queue mapping RM has collection-of-nodes information (partitions) scheduler gets its info from resource manager Advantages: more firm, less likely to be out of control, easier to understand Disadvantages: not as flexible as scheduler-controlled model Queuing and Scheduling on Compute Clusters p.10/17
15 Scheduler Dependant Scheduler Dependant (SD) Queuing Scheduler has node-to-queue mapping Scheduler has collections-of-nodes information (standing res) Advantages: more flexible; configuration all in one place (scheduler config); giving the scheduler more control can lead to more efficient use of resources Disadvantages: more complicated; less transparent to end users, can be confusing to them Queuing and Scheduling on Compute Clusters p.11/17
16 SD vs RMD for simple clusters, RMD is fine SD more appropriate for large, multi-user clusters Queuing and Scheduling on Compute Clusters p.12/17
17 Scheduling Concepts FIFO priority calculations (queue time, fairshare, class) Queuing and Scheduling on Compute Clusters p.13/17
18 Scheduling Concepts FIFO priority calculations (queue time, fairshare, class) reservations: standing, one-time, and job backfill Queuing and Scheduling on Compute Clusters p.13/17
19 Scheduling Concepts FIFO priority calculations (queue time, fairshare, class) reservations: standing, one-time, and job backfill resource usage limits, resource types maximum number of running and idle jobs Queuing and Scheduling on Compute Clusters p.13/17
20 Examples from the CAC: RMD We use resource manager dependent scheduling on our largest cluster, nyx and our Mac cluster, eliza. with a mix of standing and one-time reservations this is clearer for the users (they are sure they are getting their nodes nyx is the last Linux cluster to be upgraded; we ll move to scheduler dependent scheduling when we upgrade eliza may become part of aon (and aon s scheduling policies), or may get SD scheduling on its own, or may just live out its life as it stands today Queuing and Scheduling on Compute Clusters p.14/17
21 Examples from the CAC: SD We use scheduler dependent scheduling on our other two cluster, the Athlon+Myrinet cluster morpheus and the private faculty-owned cluster aon. the faculty asked for scheduler dependent scheduling in order to enforce group limits while also allowing for the most efficient use of their cluster. There are no partitions or reservations, just queue limits; this works best on a homogeneous cluster we use a different type of SD scheduling on morpheus; it is based on standing reservations where the private reservations can spill into the public pool of nodes, but not (yet) the other way. Queuing and Scheduling on Compute Clusters p.15/17
22 Things we don t or won t do preemption we will probably do this to allow use of the private nodes it will be a user-requested feature, for those punks who are feeling lucky Queuing and Scheduling on Compute Clusters p.16/17
23 Things we don t or won t do preemption we will probably do this to allow use of the private nodes it will be a user-requested feature, for those punks who are feeling lucky checkpoint/restart not practical yet in Linux or OS X that said, there was a surprising (to me) amount of talk about this at SuperComputing (see sc05.supercomp.org/schedule) Queuing and Scheduling on Compute Clusters p.16/17
24 Things we don t or won t do preemption we will probably do this to allow use of the private nodes it will be a user-requested feature, for those punks who are feeling lucky checkpoint/restart not practical yet in Linux or OS X that said, there was a surprising (to me) amount of talk about this at SuperComputing (see sc05.supercomp.org/schedule) suspend/restart memory/swap issues local storage issues Queuing and Scheduling on Compute Clusters p.16/17
25 The End Questions? Queuing and Scheduling on Compute Clusters p.17/17
Moab Workload Manager on Cray XT3
Moab Workload Manager on Cray XT3 presented by Don Maxwell (ORNL) Michael Jackson (Cluster Resources, Inc.) MOAB Workload Manager on Cray XT3 Why MOAB? Requirements Features Support/Futures 2 Why Moab?
Resource Managers, Schedulers, and Grid Computing
Resource Managers, Schedulers, and Grid Computing James E. Prewett October 8, 2008 Outline Resource Managers Practical: TORQUE Installation and Configuration Schedulers Practical: Maui Installation and
The University of Michigan Center for Advanced Computing
The University of Michigan Center for Advanced Computing Andy Caird acaird@umich.edu The University of MichiganCenter for Advanced Computing p.1/29 The CAC What is the Center for Advanced Computing? we
Cluster Computing. Resource and Job Management for HPC 16/08/2010 SC-CAMP. ( SC-CAMP) Cluster Computing 16/08/ / 50
Cluster Computing Resource and Job Management for HPC SC-CAMP 16/08/2010 ( SC-CAMP) Cluster Computing 16/08/2010 1 / 50 Summary 1 Introduction Cluster Computing 2 About Resource and Job Management Systems
Moab, TORQUE, and Gold in a Heterogeneous, Federated Computing System at the University of Michigan
Moab, TORQUE, and Gold in a Heterogeneous, Federated Computing System at the University of Michigan Andrew Caird Matthew Britt Brock Palen September 18, 2009 Who We Are College of Engineering centralized
High-Performance Computing at The University of Michigan College of Engineering
High-Performance Computing at The University of Michigan College of Engineering Andrew Caird acaird@umich.edu October 10, 2006 Who We Are College of Engineering centralized HPC support Been trying this
Introduction to Slurm
Introduction to Slurm Tim Wickberg SchedMD Slurm User Group Meeting 2017 Outline Roles of resource manager and job scheduler Slurm description and design goals Slurm architecture and plugins Slurm configuration
Slurm Overview. Brian Christiansen, Marshall Garey, Isaac Hartung SchedMD SC17. Copyright 2017 SchedMD LLC
Slurm Overview Brian Christiansen, Marshall Garey, Isaac Hartung SchedMD SC17 Outline Roles of a resource manager and job scheduler Slurm description and design goals Slurm architecture and plugins Slurm
An Advance Reservation-Based Computation Resource Manager for Global Scheduling
An Advance Reservation-Based Computation Resource Manager for Global Scheduling 1.National Institute of Advanced Industrial Science and Technology, 2 Suuri Giken Hidemoto Nakada 1, Atsuko Takefusa 1, Katsuhiko
SGE 6.0 configuration guide, version 1.1
SGE 6.0 configuration guide, version 1.1 Juha Jäykkä juolja@utu.fi Department of Physics Laboratory of Theoretical Physics University of Turku 18.03.2005 First, some notes This needs to be revised to include
PBS PROFESSIONAL VS. MICROSOFT HPC PACK
PBS PROFESSIONAL VS. MICROSOFT HPC PACK On the Microsoft Windows Platform PBS Professional offers many features which are not supported by Microsoft HPC Pack. SOME OF THE IMPORTANT ADVANTAGES OF PBS PROFESSIONAL
A Brief Introduction to The Center for Advanced Computing
A Brief Introduction to The Center for Advanced Computing May 1, 2006 Hardware 324 Opteron nodes, over 700 cores 105 Athlon nodes, 210 cores 64 Apple nodes, 128 cores Gigabit networking, Myrinet networking,
CS 326: Operating Systems. CPU Scheduling. Lecture 6
CS 326: Operating Systems CPU Scheduling Lecture 6 Today s Schedule Agenda? Context Switches and Interrupts Basic Scheduling Algorithms Scheduling with I/O Symmetric multiprocessing 2/7/18 CS 326: Operating
CENTER FOR HIGH PERFORMANCE COMPUTING. Overview of CHPC. Martin Čuma, PhD. Center for High Performance Computing
Overview of CHPC Martin Čuma, PhD Center for High Performance Computing m.cuma@utah.edu Spring 2014 Overview CHPC Services HPC Clusters Specialized computing resources Access and Security Batch (PBS and
Cluster Network Products
Cluster Network Products Cluster interconnects include, among others: Gigabit Ethernet Myrinet Quadrics InfiniBand 1 Interconnects in Top500 list 11/2009 2 Interconnects in Top500 list 11/2008 3 Cluster
Adding content to your Blackboard 9.1 class
Adding content to your Blackboard 9.1 class There are quite a few options listed when you click the Build Content button in your class, but you ll probably only use a couple of them most of the time. Note
Introduction to Operating System
Introduction to Operating System An operating system is a program which manages all the computer hardware. It provides the base for application program and acts as an intermediary between a user and the
OpenPBS Users Manual
How to Write a PBS Batch Script OpenPBS Users Manual PBS scripts are rather simple. An MPI example for user your-user-name: Example: MPI Code PBS -N a_name_for_my_parallel_job PBS -l nodes=7,walltime=1:00:00
A Brief Introduction to The Center for Advanced Computing
A Brief Introduction to The Center for Advanced Computing February 8, 2007 Hardware 376 Opteron nodes, over 890 cores Gigabit networking, Myrinet networking, Infiniband networking soon Hardware: nyx nyx
Computational Mini-Grid Research at Clemson University
Computational Mini-Grid Research at Clemson University Parallel Architecture Research Lab November 19, 2002 Project Description The concept of grid computing is becoming a more and more important one in
HPC and IT Issues Session Agenda. Deployment of Simulation (Trends and Issues Impacting IT) Mapping HPC to Performance (Scaling, Technology Advances)
HPC and IT Issues Session Agenda Deployment of Simulation (Trends and Issues Impacting IT) Discussion Mapping HPC to Performance (Scaling, Technology Advances) Discussion Optimizing IT for Remote Access
Chapter 9 Memory Management
Contents 1. Introduction 2. Computer-System Structures 3. Operating-System Structures 4. Processes 5. Threads 6. CPU Scheduling 7. Process Synchronization 8. Deadlocks 9. Memory Management 10. Virtual
A Brief Introduction to The Center for Advanced Computing
A Brief Introduction to The Center for Advanced Computing November 10, 2009 Outline 1 Resources Hardware Software 2 Mechanics: Access Transferring files and data to and from the clusters Logging into the
Reduces latency and buffer overhead. Messaging occurs at a speed close to the processors being directly connected. Less error detection
Switching Operational modes: Store-and-forward: Each switch receives an entire packet before it forwards it onto the next switch - useful in a general purpose network (I.e. a LAN). usually, there is a
Memory Allocation. Copyright : University of Illinois CS 241 Staff 1
Memory Allocation Copyright : University of Illinois CS 241 Staff 1 Allocation of Page Frames Scenario Several physical pages allocated to processes A, B, and C. Process B page faults. Which page should
Welcome to NUSSO (Northeastern University Space Scheduling Online)
Welcome to NUSSO (Northeastern University Space Scheduling Online) Presented by: Scheduling Operations 434 Curry Student Center 617-373-2632 cscreservations@neu.edu Overview What is NUSSO Why the change
Batch Systems. Running calculations on HPC resources
Batch Systems Running calculations on HPC resources Outline What is a batch system? How do I interact with the batch system Job submission scripts Interactive jobs Common batch systems Converting between
Batch Systems & Parallel Application Launchers Running your jobs on an HPC machine
Batch Systems & Parallel Application Launchers Running your jobs on an HPC machine Partners Funding Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike
MQ Jumping... Or, move to the front of the queue, pass go and collect 200
MQ Jumping.... Or, move to the front of the queue, pass go and collect 200 Martyn Ruks DEFCON 15 2007-08-03 One Year Ago Last year I talked about IBM Networking attacks and said I was going to continue
Memory - Paging. Copyright : University of Illinois CS 241 Staff 1
Memory - Paging Copyright : University of Illinois CS 241 Staff 1 Physical Frame Allocation How do we allocate physical memory across multiple processes? What if Process A needs to evict a page from Process
Resource Management at LLNL SLURM Version 1.2
UCRL PRES 230170 Resource Management at LLNL SLURM Version 1.2 April 2007 Morris Jette (jette1@llnl.gov) Danny Auble (auble1@llnl.gov) Chris Morrone (morrone2@llnl.gov) Lawrence Livermore National Laboratory
Lecture 9: MIMD Architectures
Lecture 9: MIMD Architectures Introduction and classification Symmetric multiprocessors NUMA architecture Clusters Zebo Peng, IDA, LiTH 1 Introduction A set of general purpose processors is connected together.
An Introduction to Cluster Computing Using Newton
An Introduction to Cluster Computing Using Newton Jason Harris and Dylan Storey March 25th, 2014 Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 1 / 26 Workshop design.
Introduction to Distributed HTC and overlay systems
Introduction to Distributed HTC and overlay systems Tuesday morning session Igor Sfiligoi University of California San Diego Logistical reminder It is OK to ask questions - During
ACCRE High Performance Compute Cluster
6 중 1 2010-05-16 오후 1:44 Enabling Researcher-Driven Innovation and Exploration Mission / Services Research Publications User Support Education / Outreach A - Z Index Our Mission History Governance Services
Day 9: Introduction to CHTC
Day 9: Introduction to CHTC Suggested reading: Condor 7.7 Manual: http://www.cs.wisc.edu/condor/manual/v7.7/ Chapter 1: Overview Chapter 2: Users Manual (at most, 2.1 2.7) 1 Turn In Homework 2 Homework
BOSCO Architecture. Derek Weitzel University of Nebraska Lincoln
BOSCO Architecture Derek Weitzel University of Nebraska Lincoln Goals We want an easy to use method for users to do computational research It should be easy to install, use, and maintain It should be simple
LECTURE 3:CPU SCHEDULING
LECTURE 3:CPU SCHEDULING 1 Outline Basic Concepts Scheduling Criteria Scheduling Algorithms Multiple-Processor Scheduling Real-Time CPU Scheduling Operating Systems Examples Algorithm Evaluation 2 Objectives
Multiprocessor Scheduling. Multiprocessor Scheduling
Multiprocessor Scheduling Will consider only shared memory multiprocessor or multi-core CPU Salient features: One or more caches: cache affinity is important Semaphores/locks typically implemented as spin-locks:
Multiprocessor Scheduling
Multiprocessor Scheduling Will consider only shared memory multiprocessor or multi-core CPU Salient features: One or more caches: cache affinity is important Semaphores/locks typically implemented as spin-locks:
Comp 310 Computer Systems and Organization
Comp 310 Computer Systems and Organization Lecture #9 Process Management (CPU Scheduling) 1 Prof. Joseph Vybihal Announcements Oct 16 Midterm exam (in class) In class review Oct 14 (½ class review) Ass#2
ECE 550D Fundamentals of Computer Systems and Engineering. Fall 2017
ECE 550D Fundamentals of Computer Systems and Engineering Fall 2017 The Operating System (OS) Prof. John Board Duke University Slides are derived from work by Profs. Tyler Bletsch and Andrew Hilton (Duke)
Multiprocessor Scheduling. Multiprocessor Scheduling
Multiprocessor Scheduling Will consider only shared memory multiprocessor or multi-core CPU Salient features: One or more caches: cache affinity is important Semaphores/locks typically implemented as spin-locks:
Lecture 1: January 23
CMPSCI 677 Distributed and Operating Systems Spring 2019 Lecture 1: January 23 Lecturer: Prashant Shenoy Scribe: Jonathan Westin (2019), Bin Wang (2018) 1.1 Introduction to the course The lecture started
X Grid Engine. Where X stands for Oracle Univa Open Son of more to come...?!?
X Grid Engine Where X stands for Oracle Univa Open Son of more to come...?!? Carsten Preuss on behalf of Scientific Computing High Performance Computing Scheduler candidates LSF too expensive PBS / Torque
Improving User Accounting and Isolation with Linux Kernel Features. Brian Bockelman Condor Week 2011
Improving User Accounting and Isolation with Linux Kernel Features Brian Bockelman Condor Week 2011 Case Study: MPD The MPICH2 library is a common implementation of the MPI interface, a popular parallel
What s new in HTCondor? What s coming? HTCondor Week 2018 Madison, WI -- May 22, 2018
What s new in HTCondor? What s coming? HTCondor Week 2018 Madison, WI -- May 22, 2018 Todd Tannenbaum Center for High Throughput Computing Department of Computer Sciences University of Wisconsin-Madison
Custom Fields in QuickBooks
Custom Fields in QuickBooks November 20, 2013 By Charlie Russell 41 Replies Every business has some sort of unique information that is important to its operation. While QuickBooks Desktop provides the
Queue systems. and how to use Torque/Maui. Piero Calucci. Scuola Internazionale Superiore di Studi Avanzati Trieste
Queue systems and how to use Torque/Maui Piero Calucci Scuola Internazionale Superiore di Studi Avanzati Trieste March 9th 2007 Advanced School in High Performance Computing Tools for e-science Outline
Batch Systems. Running your jobs on an HPC machine
Batch Systems Running your jobs on an HPC machine Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_us
Some popular Operating Systems include Linux Operating System, Windows Operating System, VMS, OS/400, AIX, z/os, etc.
An Operating System (OS) is an interface between computer user and computer hardware. An operating system is software which performs all the basic tasks like file management, memory management, process
Operating Systems Course Overview
Operating Systems Course Overview Instructor: Kartik Gopalan Class website: http://oscourse.github.io Interfaces in a Computer System (1) User-level processes (2) (3) Libraries Operating System (4) (5)
The ANLABM SP Scheduling System
The ANLABM SP Scheduling System David Lifka Argonne National Laboratory 2/1/95 A bstract Approximatelyfive years ago scientists discovered that modern LY.Y workstations connected with ethernet andfiber
Database Management Systems
Database Management Systems Distributed Databases Doug Shook What does it mean to be distributed? Multiple nodes connected by a network Data on the nodes is logically related The nodes do not need to be
and how to use TORQUE & Maui Piero Calucci
Queue and how to use & Maui Scuola Internazionale Superiore di Studi Avanzati Trieste November 2008 Advanced School in High Performance and Grid Computing Outline 1 We Are Trying to Solve 2 Using the Manager
Brigham Young University Fulton Supercomputing Lab. Ryan Cox
Brigham Young University Fulton Supercomputing Lab Ryan Cox SLURM User Group 2013 Fun Facts ~33,000 students ~70% of students speak a foreign language Several cities around BYU have gige at home #6 Top
Advanced cluster techniques with LoadLeveler
Advanced cluster techniques with LoadLeveler How to get your jobs to the top of the queue Ciaron Linstead 10th May 2012 Outline 1 Introduction 2 LoadLeveler recap 3 CPUs 4 Memory 5 Factors affecting job
Using the IAC Chimera Cluster
Using the IAC Chimera Cluster Ángel de Vicente (Tel.: x5387) SIE de Investigación y Enseñanza Chimera overview Beowulf type cluster Chimera: a monstrous creature made of the parts of multiple animals.
A NETWORK PRIMER. An introduction to some fundamental networking concepts and the benefits of using LANtastic.
A NETWORK PRIMER An introduction to some fundamental networking concepts and the benefits of using LANtastic. COPYRIGHT 1996 Artisoft, Inc. All Rights Reserved. This information file is copyrighted with
Dynamic Virtual Clusters in a Grid Site Manager
Dynamic Virtual Clusters in a Grid Site Manager Jeff Chase, David Irwin, Laura Grit, Justin Moore, Sara Sprenkle Department of Computer Science Duke University Dynamic Virtual Clusters Grid Services Grid
July 2014 IT Tips and TechSolutions News Vol. 4 No. 7 In This Months TechPoints
July 2014 IT Tips and TechSolutions News Vol. 4 No. 7 In This Months TechPoints 4 Great Tips for BYOD Security Continuity Metrics: RTO and RPO 5 iphone Messaging Tips 10 Important Virtualization Terms
A Guide to Condor. Joe Antognini. October 25, Condor is on Our Network What is an Our Network?
A Guide to Condor Joe Antognini October 25, 2013 1 Condor is on Our Network What is an Our Network? The computers in the OSU astronomy department are all networked together. In fact, they re networked
The Team. Using Sun Grid Engine and Globus to Schedule Jobs Across a Combination of Local and Remote Machines
The Team Using Sun Grid Engine and Globus to Schedule Jobs Across a Combination of Local and Remote Machines By Geoff Cawood and Paul Graham Edinburgh Parallel Computing Centre (EPCC) Telephone: +44 131
Multi-Level Feedback Queues
CS 326: Operating Systems Multi-Level Feedback Queues Lecture 8 Today s Schedule Building an Ideal Scheduler Priority-Based Scheduling Multi-Level Queues Multi-Level Feedback Queues Scheduling Domains
Unifying Heterogeneous Resources Moab Con Scott Jackson Engineering
Unifying Heterogeneous Resources Moab Con 2009 Scott Jackson Engineering Overview Introduction Heterogeneous Resources w/in the Cluster Disparate Clusters -- Multiple Resource Managers Disparate Clusters
ECE 574 Cluster Computing Lecture 4
ECE 574 Cluster Computing Lecture 4 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 31 January 2017 Announcements Don t forget about homework #3 I ran HPCG benchmark on Haswell-EP
Scheduling. CS 161: Lecture 4 2/9/17
Scheduling CS 161: Lecture 4 2/9/17 Where does the first process come from? The Linux Boot Process Machine turned on; BIOS runs BIOS: Basic Input/Output System Stored in flash memory on motherboard Determines
First evaluation of the Globus GRAM Service. Massimo Sgaravatto INFN Padova
First evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova massimo.sgaravatto@pd.infn.it Draft version release 1.0.5 20 June 2000 1 Introduction...... 3 2 Running jobs... 3 2.1 Usage examples.
Building Campus HTC Sharing Infrastructures. Derek Weitzel University of Nebraska Lincoln (Open Science Grid Hat)
Building Campus HTC Sharing Infrastructures Derek Weitzel University of Nebraska Lincoln (Open Science Grid Hat) HCC: Campus Grids Motivation We have 3 clusters in 2 cities. Our largest (4400 cores) is
Making Supercomputing More Available and Accessible Windows HPC Server 2008 R2 Beta 2 Microsoft High Performance Computing April, 2010
Making Supercomputing More Available and Accessible Windows HPC Server 2008 R2 Beta 2 Microsoft High Performance Computing April, 2010 Windows HPC Server 2008 R2 Windows HPC Server 2008 R2 makes supercomputing
Athena History. Modular Debathena. Debian Packages An example diversion. Other Athena customizations
Athena Project Athena started at MIT in 1983 grant from IBM and Digital Mission statement: By 1988, create a new educational computing environment at MIT built around high performance graphics workstations,
Grid Compute Resources and Job Management
Grid Compute Resources and Job Management How do we access the grid? Command line with tools that you'll use Specialised applications Ex: Write a program to process images that sends data to run on the
SUCCESS STORY THE POLYCLINIC THE POLYCLINIC SPEEDS UP ITS VDI ENVIRONMENT WITH NVIDIA GRID
THE POLYCLINIC SPEEDS UP ITS VDI ENVIRONMENT WITH NVIDIA GRID Updated VDI with NVIDIA GRID wins over staff and increases user density across the organization. was founded in 1917 by a handful of doctors.
M. Roehrig, Sandia National Laboratories. Philipp Wieder, Research Centre Jülich Nov 2002
Category: INFORMATIONAL Grid Scheduling Dictionary WG (SD-WG) M. Roehrig, Sandia National Laboratories Wolfgang Ziegler, Fraunhofer-Institute for Algorithms and Scientific Computing Philipp Wieder, Research
Linux Clusters for High- Performance Computing: An Introduction
Linux Clusters for High- Performance Computing: An Introduction Jim Phillips, Tim Skirvin Outline Why and why not clusters? Consider your Users Application Budget Environment Hardware System Software HPC
AASPI SOFTWARE PARALLELIZATION
AASPI SOFTWARE PARALLELIZATION Introduction Generation of multitrace and multispectral seismic attributes can be computationally intensive. For example, each input seismic trace may generate 50 or more
Chapter 1: Distributed Information Systems
Chapter 1: Distributed Information Systems Contents - Chapter 1 Design of an information system Layers and tiers Bottom up design Top down design Architecture of an information system One tier Two tier
14-Nov-11. Chapter 5: System Software: Operating Systems and Utility Programs. Learning Objectives. Learning Objectives. Overview
Learning Objectives Chapter 5: System Software: Operating Systems and 1. Understand the difference between system software and application software. 2. Explain the different functions of an operating system
EECS 750: Advanced Operating Systems. 01/29 /2014 Heechul Yun
EECS 750: Advanced Operating Systems 01/29 /2014 Heechul Yun 1 Administrative Next summary assignment Resource Containers: A New Facility for Resource Management in Server Systems, OSDI 99 due by 11:59
Processes. CS 475, Spring 2018 Concurrent & Distributed Systems
Processes CS 475, Spring 2018 Concurrent & Distributed Systems Review: Abstractions 2 Review: Concurrency & Parallelism 4 different things: T1 T2 T3 T4 Concurrency: (1 processor) Time T1 T2 T3 T4 T1 T1
Lecture 9: MIMD Architectures
Lecture 9: MIMD Architectures Introduction and classification Symmetric multiprocessors NUMA architecture Clusters Zebo Peng, IDA, LiTH 1 Introduction MIMD: a set of general purpose processors is connected
Lies, Damn Lies and Performance Metrics. PRESENTATION TITLE GOES HERE Barry Cooks Virtual Instruments
Lies, Damn Lies and Performance Metrics PRESENTATION TITLE GOES HERE Barry Cooks Virtual Instruments Goal for This Talk Take away a sense of how to make the move from: Improving your mean time to innocence
Scheduling, part 2. Don Porter CSE 506
Scheduling, part 2 Don Porter CSE 506 Logical Diagram Binary Memory Formats Allocators Threads Today s Lecture Switching System to CPU Calls RCU scheduling File System Networking Sync User Kernel Memory
(Refer Slide Time: 1:26)
Information Security-3 Prof. V Kamakoti Department of Computer science and Engineering Indian Institute of Technology Madras Basics of Unix and Network Administration Operating Systems Introduction Mod01,
Edinburgh (ECDF) Update
Edinburgh (ECDF) Update Wahid Bhimji On behalf of the ECDF Team HepSysMan,10 th June 2010 Edinburgh Setup Hardware upgrades Progress in last year Current Issues June-10 Hepsysman Wahid Bhimji - ECDF 1
Lecture 17: Threads and Scheduling. Thursday, 05 Nov 2009
CS211: Programming and Operating Systems Lecture 17: Threads and Scheduling Thursday, 05 Nov 2009 CS211 Lecture 17: Threads and Scheduling 1/22 Today 1 Introduction to threads Advantages of threads 2 User
HPC learning using Cloud infrastructure
HPC learning using Cloud infrastructure Florin MANAILA IT Architect florin.manaila@ro.ibm.com Cluj-Napoca 16 March, 2010 Agenda 1. Leveraging Cloud model 2. HPC on Cloud 3. Recent projects - FutureGRID
COPYRIGHTED MATERIAL PART I. LESSON 1: Introducing VBA. LESSON 2: Getting Started with Macros. LESSON 3: Introducing the Visual Basic Editor
PART I LESSON 1: Introducing VBA LESSON 2: Getting Started with Macros LESSON 3: Introducing the Visual Basic Editor LESSON 4: Working in the VBE COPYRIGHTED MATERIAL 1 Welcome to your first lesson in
Having said this, there are some workarounds you can use to integrate video into your course.
HOW TO SESSION : Options for creating & sharing video Overview Video is everywhere. We watch it on YouTube and Netflix. It s how we get our news, communicate with friends and family in other cities, and
Fully Optimize FULLY OPTIMIZE YOUR DBA RESOURCES
Fully Optimize FULLY OPTIMIZE YOUR DBA RESOURCES IMPROVE SERVER PERFORMANCE, UPTIME, AND AVAILABILITY WHILE LOWERING COSTS WE LL COVER THESE TOP WAYS TO OPTIMIZE YOUR RESOURCES: 1 Be Smart About Your Wait
2/26/2017. For instance, consider running Word Count across 20 splits
Based on the slides of prof. Pietro Michiardi Hadoop Internals https://github.com/michiard/disc-cloud-course/raw/master/hadoop/hadoop.pdf Job: execution of a MapReduce application across a data set Task:
Module 18: "TLP on Chip: HT/SMT and CMP" Lecture 39: "Simultaneous Multithreading and Chip-multiprocessing" TLP on Chip: HT/SMT and CMP SMT
TLP on Chip: HT/SMT and CMP SMT Multi-threading Problems of SMT CMP Why CMP? Moore s law Power consumption? Clustered arch. ABCs of CMP Shared cache design Hierarchical MP file:///e /parallel_com_arch/lecture39/39_1.htm[6/13/2012
Experiments with Job Scheduling in MetaCentrum
Experiments with Job Scheduling in MetaCentrum Dalibor Klusáček, Hana Rudová, and Miroslava Plachá Faculty of Informatics, Masaryk University Botanická 68a, 602 00 Brno Czech Republic {xklusac,hanka@fi.muni.cz
Managing Multi-user Windows (Citrix) from a System Manager's View
Managing Multi-user Windows (Citrix) from a System Manager's View Thomas Wirt Systems Manager Kittle s Home Furnishings Abstract This session will cover my experiences and advice on installing and managing
S i m p l i f y i n g A d m i n i s t r a t i o n a n d M a n a g e m e n t P r o c e s s e s i n t h e P o l i s h N a t i o n a l C l u s t e r
S i m p l i f y i n g A d m i n i s t r a t i o n a n d M a n a g e m e n t P r o c e s s e s i n t h e P o l i s h N a t i o n a l C l u s t e r Miroslaw Kupczyk, Norbert Meyer, Pawel Wolniewicz e-mail:
Experience the GRID Today with Oracle9i RAC
1 Experience the GRID Today with Oracle9i RAC Shig Hiura Pre-Sales Engineer Shig_Hiura@etagon.com 2 Agenda Introduction What is the Grid The Database Grid Oracle9i RAC Technology 10g vs. 9iR2 Comparison
CSC148H Week 3. Sadia Sharmin. May 24, /20
CSC148H Week 3 Sadia Sharmin May 24, 2017 1/20 Client vs. Developer I For the first couple of weeks, we have played the role of class designer I However, you are also often in the opposite role: when a
ROES SERVER SETUP. Presentation will be showing ROES Server. It will be on Mac and PC (Parallels) to show a second instance of ROES Server
ROES SERVER SETUP Presentation will be showing ROES Server. It will be on Mac and PC (Parallels) to show a second instance of ROES Server How many of you brought a computer with you? ROES- U.com - Go to
Cheese Cluster Training
Cheese Cluster Training The Biostatistics Computer Committee (BCC) Anjishnu Banerjee Dan Eastwood Chris Edwards Michael Martens Rodney Sparapani Sergey Tarima and The Research Computing Center (RCC) Matthew
University Integrates Voice and Collaboration Tools in One Interface
Customer Case Study University Integrates Voice and Collaboration Tools in One Interface Liberty University prepared for enrollment growth with Unified Communications Integration for Microsoft Office Communicator.