HPC at UZH: status and plans
|
|
- Jemimah Morris
- 6 years ago
- Views:
Transcription
1 HPC at UZH: status and plans Dec. 4, 2013
2 This presentation s purpose Meet the sysadmin team. Update on what s coming soon in Schroedinger s HW. Review old and new usage policies. Discussion (later on). UZH University of Zurich, GC3: Grid Computing Competence Center Dec. 4, 2013
3 We want your feedback! How do the policies impact your usage and productivity? What would you like to see introduced, and what removed? Would like changes introduced to Schroedinger to converge to the new HPC system? UZH University of Zurich, GC3: Grid Computing Competence Center Dec. 4, 2013
4 Meet the team
5 GC3 sysadmin team Tyanko Aleksiev Antonio Messina Riccardo Murri
6 Contact us or
7 Today s HW status
8 Status of current HW Schroedinger s HW has been in 24 7 use 4 years long. Pieces are now starting to fail: 39% of the total HW failures happened in 2013 every 2 weeks, 3 Lustre disks fail 1 storage blade on the Panasas system failed 16 compute nodes failures in the last two months
9 Status of current HW Schroedinger s HW has been in 24 7 use 4 years long. Pieces are now starting to fail: 39% of the total HW failures happened in 2013 every 2 weeks, 3 Lustre disks fail 1 storage blade on the Panasas system failed 16 compute nodes failures in the last two months Replacing storage is the most important task, as it holds data.
10 Review of current storage policies /home: 5GB quota, nightly back-ups on Tivoli /data: 50GB quota, no backups /lustre: no quota, no backups, originally meant as scratch space
11 Status of support contracts Support contract with Oracle expires mid-december. HW support on the compute nodes will not be renewed. Expect a little capacity degradation during next year (estimate around 5%)
12 New storage HW
13 New storage HW: Panasas New Panasas PAS8 already bought by Informatikdienste Arrives this week (likely) two storage shelves, total 76TB raw fully redundant configuration
14 New Panasas: deployment (December 2013) Try to switch to the new PAS8 during this month s scheduled maintenance window. (Dec. 18) 1. Migrate all data (estimated time: 7 10 days) The old Panasas still serves /home and /data directories for IDES We might need to throttle the migration bandwidth in order not to interfere with normal cluster operations 2. Do a final rsync during the downtime filesystem must be quiescent: cluster must be free from users and jobs might take longer than expected!
15 New storage HW: Lustre The Informatikdienste already bought a complete replacement for the Lustre filesystem. HW will be delivered in December, but it won t be ready for productive use until after Xmas
16 New Lustre: deployment (February 2014) 1. Burn-in and tune servers and new Lustre software until February 18, In the meanwhile, you have to copy your important data from /lustre into /home or /data 3. Switch-over to new Lustre filesystem during the scheduled maintenance in February. No Lustre files will be preserved across the switch!
17 New Lustre: deployment (February 2014) 1. Burn-in and tune servers and new Lustre software until February 18, In the meanwhile, you have to copy your important data from /lustre into /home or /data 3. Switch-over to new Lustre filesystem during the scheduled maintenance in February. No Lustre files will be preserved across the switch!
18 New Lustre: deployment plan rationale 1. Current Lustre s capacity is 230TB, over 70% full. Migrating all data will take ages! 2. Lustre was conceived as a scratch, but is in fact used as a data storage. Given the filesystem size, we cannot guarantee data safety (backups, redundancy). So we would like to enforce the policy that /lustre is for scratch (i.e., temporary) files only! 3. Lustre version bumps from to 2.4: possible co-existence problems
19 Review of future storage policies /home: 10GB quota, nightly back-ups on Tivoli /data: 100GB quota, no backups /lustre: no quota, no backups, scratch only files older than 60 days will be automatically deleted
20 Review of future storage policies /home: 10GB quota, nightly back-ups on Tivoli /data: 100GB quota, no backups /lustre: no quota, no backups, scratch only files older than 60 days will be automatically deleted
21 SW and policy changes
22 SW changes: Operating System No planned changes to the operating system. But we may need to update to SLES 11 SP2 because of the new Panasas. (So maybe you ll end up recompiling your applications anyway.)
23 modulefiles reboot (March 2014) Remove current module files and start with a new set: C/C++ compilers (GNU + Intel) FORTRAN compilers (GNU + Intel) OpenMPI FFTW MatLab what toolboxes do you use/need? R Need more supported software? Ask!
24 Standardize on OpenMPI (April 2014) OpenMPI 1.6 becomes the only MPI library supported by the Schroedinger admin team. Provided via module load openmpi ParastationMPI remains available, but the support contract with Par-Tec will be rescinded. Other MPI libraries will be removed (they have not been updated nor used since quite a while).
25 SW changes: batch system? (May 2014) Oracle is not developing GridEngine any more, nor supporting it. Switch to SLURM? already used at CSCS and in the zbox4 big changes in the usage and command-line organize a 1-day training session on the new batch system test-drive SLURM cluster available starting March 2014 what should the user-level documentation cover? Alternative: keep GridEngine keep submitting as usual known bugs will not be fixed
26 Timeline 2013 December New /home and /data 2014 February New /lustre March module files reboot April standardize on OpenMPI May new batch system?
27 Thank you! Any questions?
28 Timeline 2013 December New /home and /data 2014 February New /lustre March module files reboot April standardize on OpenMPI May new batch system?
29 Appendix
30 modulefiles usage Jun. Nov mpi/openmpi/gcc java/ intel/mkl 4556 intel/comp 4507 intel/comp/ mpi/openmpi-1.4.5/gcc mpi/openmpi-1.4.5/gcc R/ matlab/r2011a 1213 intel/comp/ intel/mkl/ mpi/parastationmpi/intel 889 fftw/3.2.2-double 422 binutils/ mpi/openmpi-1.6.2/gcc gcc/ mpi/openmpi/intel 181 mpi/parastationmpi/gcc 105 gcc/4.5.0-system 87 matlab/r2012a 40 gcc/ gcc/ R/ mpi/openmpi/gcc mpi/openmpi-1.4.3/gcc mpi/mvapich2 6 gcc/ mpi 2 matlab 2 gcc/ gcc 2 intel 1 gcc/ binutils 1 Back to Modulesfiles reboot
Filesystems on SSCK's HP XC6000
Filesystems on SSCK's HP XC6000 Computing Centre (SSCK) University of Karlsruhe Laifer@rz.uni-karlsruhe.de page 1 Overview» Overview of HP SFS at SSCK HP StorageWorks Scalable File Share (SFS) based on
More informationShared Object-Based Storage and the HPC Data Center
Shared Object-Based Storage and the HPC Data Center Jim Glidewell High Performance Computing BOEING is a trademark of Boeing Management Company. Computing Environment Cray X1 2 Chassis, 128 MSPs, 1TB memory
More informationChoosing Resources Wisely Plamen Krastev Office: 38 Oxford, Room 117 FAS Research Computing
Choosing Resources Wisely Plamen Krastev Office: 38 Oxford, Room 117 Email:plamenkrastev@fas.harvard.edu Objectives Inform you of available computational resources Help you choose appropriate computational
More informationHow to Use a Supercomputer - A Boot Camp
How to Use a Supercomputer - A Boot Camp Shelley Knuth Peter Ruprecht shelley.knuth@colorado.edu peter.ruprecht@colorado.edu www.rc.colorado.edu Outline Today we will discuss: Who Research Computing is
More informationGuillimin HPC Users Meeting February 11, McGill University / Calcul Québec / Compute Canada Montréal, QC Canada
Guillimin HPC Users Meeting February 11, 2016 guillimin@calculquebec.ca McGill University / Calcul Québec / Compute Canada Montréal, QC Canada Compute Canada News Scheduler Updates Software Updates Training
More informationBrutus. Above and beyond Hreidar and Gonzales
Brutus Above and beyond Hreidar and Gonzales Dr. Olivier Byrde Head of HPC Group, IT Services, ETH Zurich Teodoro Brasacchio HPC Group, IT Services, ETH Zurich 1 Outline High-performance computing at ETH
More informationACCRE High Performance Compute Cluster
6 중 1 2010-05-16 오후 1:44 Enabling Researcher-Driven Innovation and Exploration Mission / Services Research Publications User Support Education / Outreach A - Z Index Our Mission History Governance Services
More informationOBTAINING AN ACCOUNT:
HPC Usage Policies The IIA High Performance Computing (HPC) System is managed by the Computer Management Committee. The User Policies here were developed by the Committee. The user policies below aim to
More informationHPCF Cray Phase 2. User Test period. Cristian Simarro User Support. ECMWF April 18, 2016
HPCF Cray Phase 2 User Test period Cristian Simarro User Support advisory@ecmwf.int ECMWF April 18, 2016 Content Introduction Upgrade timeline Changes Hardware Software Steps for the testing on CCB Possible
More informationGenius Quick Start Guide
Genius Quick Start Guide Overview of the system Genius consists of a total of 116 nodes with 2 Skylake Xeon Gold 6140 processors. Each with 18 cores, at least 192GB of memory and 800 GB of local SSD disk.
More informationUpdating the HPC Bill Punch, Director HPCC Nov 17, 2017
Updating the HPC 2018 Bill Punch, Director HPCC Nov 17, 2017 Unique Opportunity The plan for HPC and the new data center is to stand up a new system in the DC, while maintaining the old system for awhile
More informationHP Storage and UMCG
HP Storage and Computing @ UMCG Pieter Neerincx Genomics Coordination Center UMCG SURF-DTL SIG Compute for life science reseh April 22 2015 Utrecht 1 Topics Expectation Management Shared lab / kitchen
More informationIntroduction to High Performance Computing and an Statistical Genetics Application on the Janus Supercomputer. Purpose
Introduction to High Performance Computing and an Statistical Genetics Application on the Janus Supercomputer Daniel Yorgov Department of Mathematical & Statistical Sciences, University of Colorado Denver
More informationKnights Landing production environment on MARCONI
Knights Landing production environment on MARCONI Alessandro Marani - a.marani@cineca.it March 20th, 2017 Agenda In this presentation, we will discuss - How we interact with KNL environment on MARCONI
More informationExercise Architecture of Parallel Computer Systems
Exercise Architecture of Parallel Computer Systems SoSe 18 L.079.05810 www.uni-paderborn.de/pc2 Architecture of Parallel Computer Systems SoSe 18 J.Simon 1 Overview Computer Systems Test Cluster (Arminius)
More informationLustre usages and experiences
Lustre usages and experiences at German Climate Computing Centre in Hamburg Carsten Beyer High Performance Computing Center Exclusively for the German Climate Research Limited Company, non-profit Staff:
More informationSPINOSO Vincenzo. Optimization of the job submission and data access in a LHC Tier2
EGI User Forum Vilnius, 11-14 April 2011 SPINOSO Vincenzo Optimization of the job submission and data access in a LHC Tier2 Overview User needs Administration issues INFN Bari farm design and deployment
More informationIntroduction to High-Performance Computing (HPC)
Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid
More informationOhio Supercomputer Center
Ohio Supercomputer Center Data Lifecycle Management No: Effective: OSC-1 May 1, 2009 Issued By: Kevin Wohlever Director of Supercomputer Operations Published By: Ohio Supercomputer Center Original Publication
More informationRunning Applications on The Sheffield University HPC Clusters
Running Applications on The Sheffield University HPC Clusters Deniz Savas dsavas.staff.sheffield.ac.uk June 2017 Topics 1. Software on an HPC system 2. Available Applications 3. Available Development Tools
More informationIntroduction to PICO Parallel & Production Enviroment
Introduction to PICO Parallel & Production Enviroment Mirko Cestari m.cestari@cineca.it Alessandro Marani a.marani@cineca.it Domenico Guida d.guida@cineca.it Nicola Spallanzani n.spallanzani@cineca.it
More informationIntroduction to High-Performance Computing (HPC)
Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid
More informationLinux HPC Software Stack
Linux HPC Software Stack Makia Minich Clustre Monkey, HPC Software Stack Lustre Group April 2008 1 1 Project Goals Develop integrated software stack for Linux-based HPC solutions based on Sun HPC hardware
More informationOur new HPC-Cluster An overview
Our new HPC-Cluster An overview Christian Hagen Universität Regensburg Regensburg, 15.05.2009 Outline 1 Layout 2 Hardware 3 Software 4 Getting an account 5 Compiling 6 Queueing system 7 Parallelization
More informationHabanero Operating Committee. January
Habanero Operating Committee January 25 2017 Habanero Overview 1. Execute Nodes 2. Head Nodes 3. Storage 4. Network Execute Nodes Type Quantity Standard 176 High Memory 32 GPU* 14 Total 222 Execute Nodes
More informationThe Last Bottleneck: How Parallel I/O can improve application performance
The Last Bottleneck: How Parallel I/O can improve application performance HPC ADVISORY COUNCIL STANFORD WORKSHOP; DECEMBER 6 TH 2011 REX TANAKIT DIRECTOR OF INDUSTRY SOLUTIONS AGENDA Panasas Overview Who
More informationAN INTRODUCTION TO CLUSTER COMPUTING
CLUSTERS AND YOU AN INTRODUCTION TO CLUSTER COMPUTING Engineering IT BrownBag Series 29 October, 2015 Gianni Pezzarossi Linux Systems Administrator Mark Smylie Hart Research Technology Facilitator WHAT
More informationDay 9: Introduction to CHTC
Day 9: Introduction to CHTC Suggested reading: Condor 7.7 Manual: http://www.cs.wisc.edu/condor/manual/v7.7/ Chapter 1: Overview Chapter 2: Users Manual (at most, 2.1 2.7) 1 Turn In Homework 2 Homework
More informationUAntwerpen, 24 June 2016
Tier-1b Info Session UAntwerpen, 24 June 2016 VSC HPC environment Tier - 0 47 PF Tier -1 623 TF Tier -2 510 Tf 16,240 CPU cores 128/256 GB memory/node IB EDR interconnect Tier -3 HOPPER/TURING STEVIN THINKING/CEREBRO
More informationFeedback on BeeGFS. A Parallel File System for High Performance Computing
Feedback on BeeGFS A Parallel File System for High Performance Computing Philippe Dos Santos et Georges Raseev FR 2764 Fédération de Recherche LUmière MATière December 13 2016 LOGO CNRS LOGO IO December
More informationCox Business Online Backup Administrator Guide. Version 2.0
Cox Business Online Backup Administrator Guide Version 2.0 2012 by Cox Communications. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means, electronic,
More informationFlux: The State of the Cluster
Flux: The State of the Cluster Andrew Caird acaird@umich.edu 7 November 2012 Questions Thank you all for coming. Questions? Andy Caird (acaird@umich.edu, hpc-support@umich.edu) Flux Since Last November
More informationINTRODUCTION TO THE CLUSTER
INTRODUCTION TO THE CLUSTER WHAT IS A CLUSTER? A computer cluster consists of a group of interconnected servers (nodes) that work together to form a single logical system. COMPUTE NODES GATEWAYS SCHEDULER
More informationExperiences with HP SFS / Lustre in HPC Production
Experiences with HP SFS / Lustre in HPC Production Computing Centre (SSCK) University of Karlsruhe Laifer@rz.uni-karlsruhe.de page 1 Outline» What is HP StorageWorks Scalable File Share (HP SFS)? A Lustre
More informationHPC Middle East. KFUPM HPC Workshop April Mohamed Mekias HPC Solutions Consultant. Agenda
KFUPM HPC Workshop April 29-30 2015 Mohamed Mekias HPC Solutions Consultant Agenda 1 Agenda-Day 1 HPC Overview What is a cluster? Shared v.s. Distributed Parallel v.s. Massively Parallel Interconnects
More informationHPC Workshop. Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing
HPC Workshop Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing NEEDED EQUIPMENT 1. Laptop with Secure Shell (ssh) for login A. Windows: download/install putty from https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
More informationCerebro Quick Start Guide
Cerebro Quick Start Guide Overview of the system Cerebro consists of a total of 64 Ivy Bridge processors E5-4650 v2 with 10 cores each, 14 TB of memory and 24 TB of local disk. Table 1 shows the hardware
More informationGetting started with the CEES Grid
Getting started with the CEES Grid October, 2013 CEES HPC Manager: Dennis Michael, dennis@stanford.edu, 723-2014, Mitchell Building room 415. Please see our web site at http://cees.stanford.edu. Account
More informationComputing with the Moore Cluster
Computing with the Moore Cluster Edward Walter An overview of data management and job processing in the Moore compute cluster. Overview Getting access to the cluster Data management Submitting jobs (MPI
More informationParallel Computing at DESY Zeuthen. Introduction to Parallel Computing at DESY Zeuthen and the new cluster machines
Parallel Computing at DESY Zeuthen. Introduction to Parallel Computing at DESY Zeuthen and the new cluster machines Götz Waschk Technical Seminar, Zeuthen April 27, 2010 > Introduction > Hardware Infiniband
More informationDatabase Services at CERN with Oracle 10g RAC and ASM on Commodity HW
Database Services at CERN with Oracle 10g RAC and ASM on Commodity HW UKOUG RAC SIG Meeting London, October 24 th, 2006 Luca Canali, CERN IT CH-1211 LCGenève 23 Outline Oracle at CERN Architecture of CERN
More informationMIGRATING TO THE SHARED COMPUTING CLUSTER (SCC) SCV Staff Boston University Scientific Computing and Visualization
MIGRATING TO THE SHARED COMPUTING CLUSTER (SCC) SCV Staff Boston University Scientific Computing and Visualization 2 Glenn Bresnahan Director, SCV MGHPCC Buy-in Program Kadin Tseng HPC Programmer/Consultant
More informationTriton file systems - an introduction. slide 1 of 28
Triton file systems - an introduction slide 1 of 28 File systems Motivation & basic concepts Storage locations Basic flow of IO Do's and Don'ts Exercises slide 2 of 28 File systems: Motivation Case #1:
More informationThe Hopper System: How the Largest* XE6 in the World Went From Requirements to Reality! Katie Antypas, Tina Butler, and Jonathan Carter
The Hopper System: How the Largest* XE6 in the World Went From Requirements to Reality! Katie Antypas, Tina Butler, and Jonathan Carter CUG 2011, May 25th, 2011 1 Requirements to Reality Develop RFP Select
More informationNUSGRID a computational grid at NUS
NUSGRID a computational grid at NUS Grace Foo (SVU/Academic Computing, Computer Centre) SVU is leading an initiative to set up a campus wide computational grid prototype at NUS. The initiative arose out
More informationIntroduction to the NCAR HPC Systems. 25 May 2018 Consulting Services Group Brian Vanderwende
Introduction to the NCAR HPC Systems 25 May 2018 Consulting Services Group Brian Vanderwende Topics to cover Overview of the NCAR cluster resources Basic tasks in the HPC environment Accessing pre-built
More informationIllinois Proposal Considerations Greg Bauer
- 2016 Greg Bauer Support model Blue Waters provides traditional Partner Consulting as part of its User Services. Standard service requests for assistance with porting, debugging, allocation issues, and
More informationUsing the IBM Opteron 1350 at OSC. October 19-20, 2010
Using the IBM Opteron 1350 at OSC October 19-20, 2010 Table of Contents Hardware Overview The Linux Operating System User Environment and Storage 2 Hardware Overview Hardware introduction Login node configuration
More informationOverview of High Performance Input/Output on LRZ HPC systems. Christoph Biardzki Richard Patra Reinhold Bader
Overview of High Performance Input/Output on LRZ HPC systems Christoph Biardzki Richard Patra Reinhold Bader Agenda Choosing the right file system Storage subsystems at LRZ Introduction to parallel file
More informationIntroduction to the SHARCNET Environment May-25 Pre-(summer)school webinar Speaker: Alex Razoumov University of Ontario Institute of Technology
Introduction to the SHARCNET Environment 2010-May-25 Pre-(summer)school webinar Speaker: Alex Razoumov University of Ontario Institute of Technology available hardware and software resources our web portal
More informationIntroduction to UBELIX
Science IT Support (ScITS) Michael Rolli, Nico Färber Informatikdienste Universität Bern 06.06.2017, Introduction to UBELIX Agenda > Introduction to UBELIX (Overview only) Other topics spread in > Introducing
More informationMy operating system is old but I don't care : I'm using NIX! B.Bzeznik BUX meeting, Vilnius 22/03/2016
My operating system is old but I don't care : I'm using NIX! B.Bzeznik BUX meeting, Vilnius 22/03/2016 CIMENT is the computing center of the University of Grenoble CIMENT computing platforms 132Tflops
More informationXSEDE New User Tutorial
April 2, 2014 XSEDE New User Tutorial Jay Alameda National Center for Supercomputing Applications XSEDE Training Survey Make sure you sign the sign in sheet! At the end of the module, I will ask you to
More informationIntroduction to Cheyenne. 12 January, 2017 Consulting Services Group Brian Vanderwende
Introduction to Cheyenne 12 January, 2017 Consulting Services Group Brian Vanderwende Topics we will cover Technical specs of the Cheyenne supercomputer and expanded GLADE file systems The Cheyenne computing
More informationGraham vs legacy systems
New User Seminar Graham vs legacy systems This webinar only covers topics pertaining to graham. For the introduction to our legacy systems (Orca etc.), please check the following recorded webinar: SHARCNet
More informationExperiences in Optimizing a $250K Cluster for High- Performance Computing Applications
Experiences in Optimizing a $250K Cluster for High- Performance Computing Applications Kevin Brandstatter Dan Gordon Jason DiBabbo Ben Walters Alex Ballmer Lauren Ribordy Ioan Raicu Illinois Institute
More informationGrid Code Planner EU Code Modifications GC0100/101/102/104
Grid Code Planner EU Code Modifications GC0100/101/102/104 Place your chosen image here. The four corners must just cover the arrow tips. For covers, the three pictures should be the same size and in a
More informationSherlock for IBIIS. William Law Stanford Research Computing
Sherlock for IBIIS William Law Stanford Research Computing Overview How we can help System overview Tech specs Signing on Batch submission Software environment Interactive jobs Next steps We are here to
More informationCMS Grid Computing at TAMU Performance, Monitoring and Current Status of the Brazos Cluster
CMS Grid Computing at TAMU Performance, Monitoring and Current Status of the Brazos Cluster Vaikunth Thukral Department of Physics and Astronomy Texas A&M University 1 Outline Grid Computing with CMS:
More informationOutline. March 5, 2012 CIRMMT - McGill University 2
Outline CLUMEQ, Calcul Quebec and Compute Canada Research Support Objectives and Focal Points CLUMEQ Site at McGill ETS Key Specifications and Status CLUMEQ HPC Support Staff at McGill Getting Started
More informationIntroduction to the Cluster
Follow us on Twitter for important news and updates: @ACCREVandy Introduction to the Cluster Advanced Computing Center for Research and Education http://www.accre.vanderbilt.edu The Cluster We will be
More informationShared Parallel Filesystems in Heterogeneous Linux Multi-Cluster Environments
LCI HPC Revolution 2005 26 April 2005 Shared Parallel Filesystems in Heterogeneous Linux Multi-Cluster Environments Matthew Woitaszek matthew.woitaszek@colorado.edu Collaborators Organizations National
More informationLustre at Scale The LLNL Way
Lustre at Scale The LLNL Way D. Marc Stearman Lustre Administration Lead Livermore uting - LLNL This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory
More informationExtraordinary HPC file system solutions at KIT
Extraordinary HPC file system solutions at KIT Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State Roland of Baden-Württemberg Laifer Lustre and tools for ldiskfs investigation
More informationIntroduction to HPC Using zcluster at GACRC
Introduction to HPC Using zcluster at GACRC On-class PBIO/BINF8350 Georgia Advanced Computing Resource Center University of Georgia Zhuofei Hou, HPC Trainer zhuofei@uga.edu Outline What is GACRC? What
More informationGMS/Analyzer 8.2 FAQ on Reporting Database Version 1 2:00 PM Jan 10, 2017
GMS/Analyzer 8.2 FAQ on Reporting Database Version 1 2:00 PM Jan 10, 2017 Contents What is Reporting Database Version 2?... 2 How does this look in MySonicWALL Download Center?... 2 What will happen to
More informationDeep Learning on SHARCNET:
Deep Learning on SHARCNET: Best Practices Fei Mao Outlines What does SHARCNET have? - Hardware/software resources now and future How to run a job? - A torch7 example How to train in parallel: - A Theano-based
More informationTechnology Insight Series
IBM ProtecTIER Deduplication for z/os John Webster March 04, 2010 Technology Insight Series Evaluator Group Copyright 2010 Evaluator Group, Inc. All rights reserved. Announcement Summary The many data
More informationXSEDE New User Training. Ritu Arora November 14, 2014
XSEDE New User Training Ritu Arora Email: rauta@tacc.utexas.edu November 14, 2014 1 Objectives Provide a brief overview of XSEDE Computational, Visualization and Storage Resources Extended Collaborative
More informationCompiling applications for the Cray XC
Compiling applications for the Cray XC Compiler Driver Wrappers (1) All applications that will run in parallel on the Cray XC should be compiled with the standard language wrappers. The compiler drivers
More informationHPC File Systems and Storage. Irena Johnson University of Notre Dame Center for Research Computing
HPC File Systems and Storage Irena Johnson University of Notre Dame Center for Research Computing HPC (High Performance Computing) Aggregating computer power for higher performance than that of a typical
More informationNCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017
NCAR Globally Accessible Data Environment (GLADE) Updated: 15 Feb 2017 Overview The Globally Accessible Data Environment (GLADE) provides centralized file storage for HPC computational, data-analysis,
More informationThe LWA1 User Computing Facility Ver. 1
The LWA1 User Computing Facility Ver. 1 J. Dowell November 25, 2012 Contents 1 Introduction 2 2 Available Hardware and Software 2 3 Networking 2 4 Storage 3 5 User Management and Access 3 6 Document History
More informationCluster Clonetroop: HowTo 2014
2014/02/25 16:53 1/13 Cluster Clonetroop: HowTo 2014 Cluster Clonetroop: HowTo 2014 This section contains information about how to access, compile and execute jobs on Clonetroop, Laboratori de Càlcul Numeric's
More informationComet Virtualization Code & Design Sprint
Comet Virtualization Code & Design Sprint SDSC September 23-24 Rick Wagner San Diego Supercomputer Center Meeting Goals Build personal connections between the IU and SDSC members of the Comet team working
More informationCouncil, 26 March Information Technology Report. Executive summary and recommendations. Introduction
Council, 26 March 2014 Information Technology Report Executive summary and recommendations Introduction This report sets out the main activities of the Information Technology Department since the last
More informationIFS migrates from IBM to Cray CPU, Comms and I/O
IFS migrates from IBM to Cray CPU, Comms and I/O Deborah Salmond & Peter Towers Research Department Computing Department Thanks to Sylvie Malardel, Philippe Marguinaud, Alan Geer & John Hague and many
More informationTECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 13 th CALL (T ier-0)
TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 13 th CALL (T ier-0) Contributing sites and the corresponding computer systems for this call are: BSC, Spain IBM System x idataplex CINECA, Italy Lenovo System
More informationOur Workshop Environment
Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2017 Our Environment This Week Your laptops or workstations: only used for portal access Bridges
More informationHigh Performance Computing (HPC) Using zcluster at GACRC
High Performance Computing (HPC) Using zcluster at GACRC On-class STAT8060 Georgia Advanced Computing Resource Center University of Georgia Zhuofei Hou, HPC Trainer zhuofei@uga.edu Outline What is GACRC?
More informationLUG 2012 From Lustre 2.1 to Lustre HSM IFERC (Rokkasho, Japan)
LUG 2012 From Lustre 2.1 to Lustre HSM Lustre @ IFERC (Rokkasho, Japan) Diego.Moreno@bull.net From Lustre-2.1 to Lustre-HSM - Outline About Bull HELIOS @ IFERC (Rokkasho, Japan) Lustre-HSM - Basis of Lustre-HSM
More informationThe Why and How of HPC-Cloud Hybrids with OpenStack
The Why and How of HPC-Cloud Hybrids with OpenStack OpenStack Australia Day Melbourne June, 2017 Lev Lafayette, HPC Support and Training Officer, University of Melbourne lev.lafayette@unimelb.edu.au 1.0
More informationDuke Compute Cluster Workshop. 3/28/2018 Tom Milledge rc.duke.edu
Duke Compute Cluster Workshop 3/28/2018 Tom Milledge rc.duke.edu rescomputing@duke.edu Outline of talk Overview of Research Computing resources Duke Compute Cluster overview Running interactive and batch
More informationThe Last Bottleneck: How Parallel I/O can attenuate Amdahl's Law
The Last Bottleneck: How Parallel I/O can attenuate Amdahl's Law ERESEARCH AUSTRALASIA, NOVEMBER 2011 REX TANAKIT DIRECTOR OF INDUSTRY SOLUTIONS AGENDA Parallel System Parallel processing goes mainstream
More informationTOSS - A RHEL-based Operating System for HPC Clusters
TOSS - A RHEL-based Operating System for HPC Clusters Supercomputing 2017 Red Hat Booth November 14, 2017 Ned Bass System Software Development Group Leader Livermore Computing Division LLNL-PRES-741473
More informationEarly X1 Experiences at Boeing. Jim Glidewell Information Technology Services Boeing Shared Services Group
Early X1 Experiences at Boeing Jim Glidewell Information Technology Services Boeing Shared Services Group Early X1 Experiences at Boeing HPC computing environment X1 configuration Hardware and OS Applications
More informationGuillimin HPC Users Meeting. Bart Oldeman
June 19, 2014 Bart Oldeman bart.oldeman@mcgill.ca McGill University / Calcul Québec / Compute Canada Montréal, QC Canada Outline Compute Canada News Upcoming Maintenance Downtime in August Storage System
More informationIntroduction to HPC Using zcluster at GACRC
Introduction to HPC Using zcluster at GACRC Georgia Advanced Computing Resource Center University of Georgia Zhuofei Hou, HPC Trainer zhuofei@uga.edu Outline What is GACRC? What is HPC Concept? What is
More informationChoosing Resources Wisely. What is Research Computing?
Choosing Resources Wisely Scott Yockel, PhD Harvard - Research Computing What is Research Computing? Faculty of Arts and Sciences (FAS) department that handles nonenterprise IT requests from researchers.
More informationServer Virtualization and Optimization at HSBC. John Gibson Chief Technical Specialist HSBC Bank plc
Server Virtualization and Optimization at HSBC John Gibson Chief Technical Specialist HSBC Bank plc Background Over 5,500 Windows servers in the last 6 years. Historically, Windows technology dictated
More informationSlurm basics. Summer Kickstart June slide 1 of 49
Slurm basics Summer Kickstart 2017 June 2017 slide 1 of 49 Triton layers Triton is a powerful but complex machine. You have to consider: Connecting (ssh) Data storage (filesystems and Lustre) Resource
More informationAn introduction to checkpointing. for scientific applications
damien.francois@uclouvain.be UCL/CISM - FNRS/CÉCI An introduction to checkpointing for scientific applications November 2013 CISM/CÉCI training session What is checkpointing? Without checkpointing: $./count
More informationChallenges in making Lustre systems reliable
Challenges in making Lustre systems reliable Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State Roland of Baden-Württemberg Laifer Challenges and in making Lustre systems reliable
More informationElastiCluster Automated provisioning of computational clusters in the cloud
ElastiCluster Automated provisioning of computational clusters in the cloud Riccardo Murri (with contributions from Antonio Messina, Nicolas Bär, Sergio Maffioletti, and Sigve
More informationPoS(EGICF12-EMITC2)004
: bridging the Grid and Cloud worlds Riccardo Murri GC3: Grid Computing Competence Center University of Zurich E-mail: riccardo.murri@gmail.com GC3: Grid Computing Competence Center University of Zurich
More informationNew User Seminar: Part 2 (best practices)
New User Seminar: Part 2 (best practices) General Interest Seminar January 2015 Hugh Merz merz@sharcnet.ca Session Outline Submitting Jobs Minimizing queue waits Investigating jobs Checkpointing Efficiency
More informationOur Workshop Environment
Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2018 Our Environment This Week Your laptops or workstations: only used for portal access Bridges
More informationOur Workshop Environment
Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2017 Our Environment This Week Your laptops or workstations: only used for portal access Bridges
More informationX Grid Engine. Where X stands for Oracle Univa Open Son of more to come...?!?
X Grid Engine Where X stands for Oracle Univa Open Son of more to come...?!? Carsten Preuss on behalf of Scientific Computing High Performance Computing Scheduler candidates LSF too expensive PBS / Torque
More informationG-WAN. Complete install process for Ubuntu (both for the 32 and the 64 OS versions).
G-WAN Complete install process for Ubuntu 11.10 (both for the 32 and the 64 OS versions). G-WAN (Global Web Area Network) is both a web server (for static web pages) and a web application server (for rich
More information