Running Galaxy in an HPC environment requirements, challenges and some solutions : the LIFEPORTAL
|
|
- Roland Holmes
- 6 years ago
- Views:
Transcription
1 Running Galaxy in an HPC environment requirements, challenges and some solutions : the LIFEPORTAL Nikolay Vazov University Center for Information Technologies University of Oslo
2 Major requirements Authentication and authorization How do we manage users and their access to the cluster resources Resource management How do we control and allocate number of CPUs, memory, CPU hours Friendly redeployment of Galaxy code How can we easily customize and re-deploy a Galaxy instance and keep the updates in the original code
3 Authentication Federated National Identity provider (FEIDE) Additional IdP for non-feide users but covering as many Galaxy instances as possible Integration of EduGAIN (a super body of partner federations) Integration of OpenID type domain specific IdPs (e.g. CLARIN) Drop the local authentication
4 Authorization For all users : based on project (account) affiliation Visitor-users : browse but can't execute Per application basis : some applications are open to everybody Reviewers : anonymous users with a limited active period and application access (used to reproduce the results in published articles)
5 Resource management Users grouped in projects (accounts) Projects receive allocations : CPU time Every job on the cluster follows the procedure: Reservation of resources Execution Charging Refunding (if requested resources are bigger than really used) Allocation manager - GOLD
6 GOLD Allocation Manager open source L(A)PP technology Linux Apache (optional, needed for GUI only) Postgresql (MySQL) Perl easy to install (compilation needed) terminal and GUI (optional) easy to plug into Galaxy gold commands are called as subprocesses in python excellent manual v at present, project stopped, the resumed, now 3.x available
7 User registration and job execution galaxy user registered in Galaxy user DB 2. job user launches a job to be executed on the cluster 1. (simultaneously) 3. check for available resources in GOLD 4. confirmation for available resources from GOLD, resources reserved GOLD allocation manager a. user registered in GOLD user DB b. user associated to an account in GOLD c. user receives an allocation within the account 7. job data sent to GOLD, account charged* 6. completed job info sent to Galaxy results displayed in history panel 5. job sent to cluster, all job parameters included all job parametres sent to the cluster Job running -> Job completed cluster
8 Resource management (screenshot)
9 What do we send and what do we receive? To the cluster >>> DRMAA.PY slurm script #!/bin/sh #SBATCH -A nn9108k #SBATCH -t 300:00:00 #SBATCH -D /work/projects/galaxy/data/database_galaxy_prod01/job_working_directory/000/629 #SBATCH -J #SBATCH -n 1 #SBATCH --mem-per-cpu=4000 #SBATCH --ntasks-per-node=8 #SBATCH --partition=normal source /etc/profile source /cluster/bin/jobsetup cd /work/projects/galaxy/data/database_galaxy_prod01/job_working_directory/000/629 module load blast+; blastn -query "/work/projects/galaxy/data/database_galaxy_prod01/files/000/dataset_922.dat" -db "/work/databases/bio/ncbipre/est_others" -task blastn -evalue out /work/projects/galaxy/data/database_galaxy_prod01/files/000/dataset_924.dat -outfmt 6 -num_threads 8 -dust yes -strand both -max_target_seqs 100 echo $? > /work/projects/galaxy/data/database_galaxy_prod01/job_working_directory/000/629/galaxy_629.ec From the cluster {'charge_duration': '451587', 'galaxy_job_id': '629', 'lifeportal_project': 'lp2', 'machine': 'c17-6', 'processes': '8', 'slurm_account': 'nn9108k', 'slurm_job_id': ' ', 'user': 'a.k.krabberod@ibv.uio.no'} Successfully charged job 629 for credits MESSAGE FROM GOLD galaxy.jobs.runners.drmaa DEBUG :27:26,034 Successfully charged job 629 for credits galaxy.jobs.runners.drmaa DEBUG :27:26,034 (629/ ) state change: job finished normally
10 Redeployment Keep the Galaxy code as clean as possible from customizations : AAI outside Galaxy code Minimal changes in the code Repositories (git) to organize versions with different features
Authentication in Galaxy : let's use what is out there - (National) Identity Providers
Authentication in Galaxy : let's use what is out there - (National) Identity Providers Nikolay Vazov University Center for Information Technologies University of Oslo Authentication challenges Galaxy on
More informationSetting up a Galaxy instance as a service
Setting up a Galaxy instance as a service (or... how can we make simple things more complicated) Nikolay Vazov University Center for Information Technologies (USIT) University of Oslo Norway Astonishment
More informationIntroduction to GACRC Teaching Cluster
Introduction to GACRC Teaching Cluster Georgia Advanced Computing Resource Center (GACRC) EITS/University of Georgia Zhuofei Hou zhuofei@uga.edu 1 Outline GACRC Overview Computing Resources Three Folders
More informationSlurm and Abel job scripts. Katerina Michalickova The Research Computing Services Group SUF/USIT October 23, 2012
Slurm and Abel job scripts Katerina Michalickova The Research Computing Services Group SUF/USIT October 23, 2012 Abel in numbers Nodes - 600+ Cores - 10000+ (1 node->2 processors->16 cores) Total memory
More informationSlurm and Abel job scripts. Katerina Michalickova The Research Computing Services Group SUF/USIT November 13, 2013
Slurm and Abel job scripts Katerina Michalickova The Research Computing Services Group SUF/USIT November 13, 2013 Abel in numbers Nodes - 600+ Cores - 10000+ (1 node->2 processors->16 cores) Total memory
More informationWorking with Shell Scripting. Daniel Balagué
Working with Shell Scripting Daniel Balagué Editing Text Files We offer many text editors in the HPC cluster. Command-Line Interface (CLI) editors: vi / vim nano (very intuitive and easy to use if you
More informationIntroduction to Abel/Colossus and the queuing system
Introduction to Abel/Colossus and the queuing system November 14, 2018 Sabry Razick Research Infrastructure Services Group, USIT Topics First 7 slides are about us and links The Research Computing Services
More informationExercises: Abel/Colossus and SLURM
Exercises: Abel/Colossus and SLURM November 08, 2016 Sabry Razick The Research Computing Services Group, USIT Topics Get access Running a simple job Job script Running a simple job -- qlogin Customize
More informationBright Cluster Manager
Bright Cluster Manager Using Slurm for Data Aware Scheduling in the Cloud Martijn de Vries CTO About Bright Computing Bright Computing 1. Develops and supports Bright Cluster Manager for HPC systems, server
More informationOverview of a virtual cluster
Overview of a virtual cluster using OpenNebula and SLURM Ismael Farfán Estrada ifarfane0900@ipn.mx October 27, 2011 Ismael Farfán Estrada ifarfane0900@ipn.mx () Overview of a virtual cluster October 27,
More informationIntroduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU
Introduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU What is Joker? NMSU s supercomputer. 238 core computer cluster. Intel E-5 Xeon CPUs and Nvidia K-40 GPUs. InfiniBand innerconnect.
More informationIntroduction to SLURM & SLURM batch scripts
Introduction to SLURM & SLURM batch scripts Anita Orendt Assistant Director Research Consulting & Faculty Engagement anita.orendt@utah.edu 6 February 2018 Overview of Talk Basic SLURM commands SLURM batch
More informationIntroduction to SLURM & SLURM batch scripts
Introduction to SLURM & SLURM batch scripts Anita Orendt Assistant Director Research Consulting & Faculty Engagement anita.orendt@utah.edu 16 Feb 2017 Overview of Talk Basic SLURM commands SLURM batch
More informationNotes for installing a local blast+ instance of NCBI BLAST F. J. Pineda 09/25/2017
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 Notes for installing a local blast+ instance of NCBI BLAST F. J. Pineda 09/25/2017
More informationUsing a Linux System 6
Canaan User Guide Connecting to the Cluster 1 SSH (Secure Shell) 1 Starting an ssh session from a Mac or Linux system 1 Starting an ssh session from a Windows PC 1 Once you're connected... 1 Ending an
More informationIntroduction to High-Performance Computing (HPC)
Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid
More informationSlurm basics. Summer Kickstart June slide 1 of 49
Slurm basics Summer Kickstart 2017 June 2017 slide 1 of 49 Triton layers Triton is a powerful but complex machine. You have to consider: Connecting (ssh) Data storage (filesystems and Lustre) Resource
More informationIntroduction to SLURM on the High Performance Cluster at the Center for Computational Research
Introduction to SLURM on the High Performance Cluster at the Center for Computational Research Cynthia Cornelius Center for Computational Research University at Buffalo, SUNY 701 Ellicott St Buffalo, NY
More informationBatch Systems & Parallel Application Launchers Running your jobs on an HPC machine
Batch Systems & Parallel Application Launchers Running your jobs on an HPC machine Partners Funding Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike
More informationIntroduction to SLURM & SLURM batch scripts
Introduction to SLURM & SLURM batch scripts Anita Orendt Assistant Director Research Consulting & Faculty Engagement anita.orendt@utah.edu 23 June 2016 Overview of Talk Basic SLURM commands SLURM batch
More informationHeterogeneous Job Support
Heterogeneous Job Support Tim Wickberg SchedMD SC17 Submitting Jobs Multiple independent job specifications identified in command line using : separator The job specifications are sent to slurmctld daemon
More informationHigh Performance Computing Cluster Basic course
High Performance Computing Cluster Basic course Jeremie Vandenplas, Gwen Dawes 30 October 2017 Outline Introduction to the Agrogenomics HPC Connecting with Secure Shell to the HPC Introduction to the Unix/Linux
More informationUsing Galaxy to provide a NGS Analysis Platform
11/15/11 Using Galaxy to provide a NGS Analysis Platform Friedrich Miescher Institute - part of the Novartis Research Foundation - affiliated institute of Basel University - member of Swiss Institute of
More informationHigh Performance Computing Cluster Advanced course
High Performance Computing Cluster Advanced course Jeremie Vandenplas, Gwen Dawes 9 November 2017 Outline Introduction to the Agrogenomics HPC Submitting and monitoring jobs on the HPC Parallel jobs on
More informationHelping Non-traditional HPC Users Using XSEDE Resources Efficiently
Helping Non-traditional HPC Users Using XSEDE Resources Efficiently PIs: Robert Sean Norman (U South Carolina) Atsuko Tanaka and Chao Fu (U Wisconsin) ECSS staff: Shiquan Su National Institute of Computational
More informationAn introduction to checkpointing. for scientific applications
damien.francois@uclouvain.be UCL/CISM - FNRS/CÉCI An introduction to checkpointing for scientific applications November 2013 CISM/CÉCI training session What is checkpointing? Without checkpointing: $./count
More informationBlog site (cont.) theme, 202 view creations, 205 Browser tools, 196 Buytaert, Dries, 185
Index A Administration, 157 backups and restore (see Backups and restore website) file system, 161 log files, 162 tasks, 157 updates and security patches, 165 user accounts, 166 Aggregator module, 218
More informationINTRODUCTION TO NEXTFLOW
INTRODUCTION TO NEXTFLOW Paolo Di Tommaso, CRG NETTAB workshop - Roma October 25th, 2016 @PaoloDiTommaso Research software engineer Comparative Bioinformatics, Notredame Lab Center for Genomic Regulation
More informationUsing Compute Canada. Masao Fujinaga Information Services and Technology University of Alberta
Using Compute Canada Masao Fujinaga Information Services and Technology University of Alberta Introduction to cedar batch system jobs are queued priority depends on allocation and past usage Cedar Nodes
More informationCluster Upgrade Procedure with Job Queue Migration.
Cluster Upgrade Procedure with Job Queue Migration. Zend Server 5.6 Overview Zend Server 5.6 introduces a new highly-reliable Job Queue architecture, based on a MySQL database storage backend. This document
More informationBash for SLURM. Author: Wesley Schaal Pharmaceutical Bioinformatics, Uppsala University
Bash for SLURM Author: Wesley Schaal Pharmaceutical Bioinformatics, Uppsala University wesley.schaal@farmbio.uu.se Lab session: Pavlin Mitev (pavlin.mitev@kemi.uu.se) it i slides at http://uppmax.uu.se/support/courses
More informationGraham vs legacy systems
New User Seminar Graham vs legacy systems This webinar only covers topics pertaining to graham. For the introduction to our legacy systems (Orca etc.), please check the following recorded webinar: SHARCNet
More informationApplications Software Example
Applications Software Example How to run an application on Cluster? Rooh Khurram Supercomputing Laboratory King Abdullah University of Science and Technology (KAUST), Saudi Arabia Cluster Training: Applications
More informationAn introduction to checkpointing. for scientifc applications
damien.francois@uclouvain.be UCL/CISM An introduction to checkpointing for scientifc applications November 2016 CISM/CÉCI training session What is checkpointing? Without checkpointing: $./count 1 2 3^C
More informationPorting SLURM to the Cray XT and XE. Neil Stringfellow and Gerrit Renker
Porting SLURM to the Cray XT and XE Neil Stringfellow and Gerrit Renker Background Cray XT/XE basics Cray XT systems are among the largest in the world 9 out of the top 30 machines on the top500 list June
More informationDARIAH-AAI. DASISH AAI Meeting. Nijmegen, March 9th,
DARIAH-AAI DASISH AAI Meeting Nijmegen, March 9th, 2014 www.dariah.eu What is DARIAH? DARIAH: Digital Research Infrastructure for the Arts and Humanities One of the few ESFRI research infrastructures for
More informationOpenMP Example. $ ssh # You might have to type yes if this is the first time you log on
OpenMP Example Day 1, afternoon session 1: We examine a serial and parallel implementation of a code solving an N-body problem with a star, a planet and many small particles near the planet s radius. The
More informationIntroduction to the NCAR HPC Systems. 25 May 2018 Consulting Services Group Brian Vanderwende
Introduction to the NCAR HPC Systems 25 May 2018 Consulting Services Group Brian Vanderwende Topics to cover Overview of the NCAR cluster resources Basic tasks in the HPC environment Accessing pre-built
More informationSelenium Testing Training
About Intellipaat Intellipaat is a fast-growing professional training provider that is offering training in over 150 most sought-after tools and technologies. We have a learner base of 600,000 in over
More informationpbsacct: A Workload Analysis System for PBS-Based HPC Systems
pbsacct: A Workload Analysis System for PBS-Based HPC Systems Troy Baer Senior HPC System Administrator National Institute for Computational Sciences University of Tennessee Doug Johnson Chief Systems
More informationResource Management at LLNL SLURM Version 1.2
UCRL PRES 230170 Resource Management at LLNL SLURM Version 1.2 April 2007 Morris Jette (jette1@llnl.gov) Danny Auble (auble1@llnl.gov) Chris Morrone (morrone2@llnl.gov) Lawrence Livermore National Laboratory
More informationIntroduction to High-Performance Computing (HPC)
Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid
More informationBatch Systems. Running your jobs on an HPC machine
Batch Systems Running your jobs on an HPC machine Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_us
More informationManual Backup Sql Server Express 2008 Schedule Database Sql Agent
Manual Backup Sql Server Express 2008 Schedule Database Sql Agent Automate the Backup of Your Microsoft SQL Server Express Databases Server Express database server, only to discover that the handy job
More informationIntroduction to GACRC Teaching Cluster
Introduction to GACRC Teaching Cluster Georgia Advanced Computing Resource Center (GACRC) EITS/University of Georgia Zhuofei Hou zhuofei@uga.edu 1 Outline GACRC Overview Computing Resources Three Folders
More informationUsing the Galaxy Local Bioinformatics Cloud at CARC
Using the Galaxy Local Bioinformatics Cloud at CARC Lijing Bu Sr. Research Scientist Bioinformatics Specialist Center for Evolutionary and Theoretical Immunology (CETI) Department of Biology, University
More informationIntroduction to High Performance Computing at Case Western Reserve University. KSL Data Center
Introduction to High Performance Computing at Case Western Reserve University Research Computing and CyberInfrastructure team KSL Data Center Presenters Emily Dragowsky Daniel Balagué Guardia Hadrian Djohari
More informationADAC Federated Testbed Creating a Blueprint for Portable Ecosystems
ADAC Federated Testbed Creating a Blueprint for Portable Ecosystems Sadaf Alam, Jeffrey Vetter, Mark Klein, Maxime Martinasso, ExCL team @ ORNL,... ADAC Workshop February 15, 2018 January, 2016 June, 2016
More informationA Brief Overview of CIPRES Job Management
A Brief Overview of CIPRES Job Management Mark A. Miller San Diego Supercomputer Center CIPRES is a classic browser-based gateway designed to submit jobs to multiple hosts. (REST services are now available
More informationScheduling By Trackable Resources
Scheduling By Trackable Resources Morris Jette and Dominik Bartkiewicz SchedMD Slurm User Group Meeting 2018 Thanks to NVIDIA for sponsoring this work Goals More flexible scheduling mechanism Especially
More informationg-eclipse A Framework for Accessing Grid Infrastructures Nicholas Loulloudes Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.
g-eclipse A Framework for Accessing Grid Infrastructures Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.cy) EGEE Training the Trainers May 6 th, 2009 Outline Grid Reality The Problem g-eclipse
More informationHow to run a job on a Cluster?
How to run a job on a Cluster? Cluster Training Workshop Dr Samuel Kortas Computational Scientist KAUST Supercomputing Laboratory Samuel.kortas@kaust.edu.sa 17 October 2017 Outline 1. Resources available
More informationBrigham Young University
Brigham Young University Fulton Supercomputing Lab Ryan Cox Slurm User Group September 16, 2015 Washington, D.C. Open Source Code I'll reference several codes we have open sourced http://github.com/byuhpc
More informationSubmitting and running jobs on PlaFRIM2 Redouane Bouchouirbat
Submitting and running jobs on PlaFRIM2 Redouane Bouchouirbat Summary 1. Submitting Jobs: Batch mode - Interactive mode 2. Partition 3. Jobs: Serial, Parallel 4. Using generic resources Gres : GPUs, MICs.
More informationSTARTING THE DDT DEBUGGER ON MIO, AUN, & MC2. (Mouse over to the left to see thumbnails of all of the slides)
STARTING THE DDT DEBUGGER ON MIO, AUN, & MC2 (Mouse over to the left to see thumbnails of all of the slides) ALLINEA DDT Allinea DDT is a powerful, easy-to-use graphical debugger capable of debugging a
More informationAn Introduction to Gauss. Paul D. Baines University of California, Davis November 20 th 2012
An Introduction to Gauss Paul D. Baines University of California, Davis November 20 th 2012 What is Gauss? * http://wiki.cse.ucdavis.edu/support:systems:gauss * 12 node compute cluster (2 x 16 cores per
More informationUsing and Developing with Azure. Joshua Drew
Using and Developing with Azure Joshua Drew Visual Studio Microsoft Azure X-Plat ASP.NET Visual Studio - Every App Our vision Every App Every Developer .NET and mobile development Desktop apps - WPF Universal
More informationHands-on CUDA exercises
Hands-on CUDA exercises CUDA Exercises We have provided skeletons and solutions for 6 hands-on CUDA exercises In each exercise (except for #5), you have to implement the missing portions of the code Finished
More informationUsing DC/OS for Continuous Delivery
Using DC/OS for Continuous Delivery DevPulseCon 2017 Elizabeth K. Joseph, @pleia2 Mesosphere 1 Elizabeth K. Joseph, Developer Advocate, Mesosphere 15+ years working in open source communities 10+ years
More informationIntroduction to the Cluster
Follow us on Twitter for important news and updates: @ACCREVandy Introduction to the Cluster Advanced Computing Center for Research and Education http://www.accre.vanderbilt.edu The Cluster We will be
More informationCNAG Advanced User Training
www.bsc.es CNAG Advanced User Training Aníbal Moreno, CNAG System Administrator Pablo Ródenas, BSC HPC Support Rubén Ramos Horta, CNAG HPC Support Barcelona,May the 5th Aim Understand CNAG s cluster design
More informationFederated access to e-infrastructures worldwide
Federated access to e-infrastructures worldwide Marco Fargetta, INFN Catania - Italy (marco.fargetta@ct.infn.it) DCIs developed in the last decade 2 Evolution Research organisations are moving to cloud
More informationTutorial 4 Data Persistence in Java
TCSS 360: Software Development Institute of Technology and Quality Assurance Techniques University of Washington Tacoma Winter 2017 http://faculty.washington.edu/wlloyd/courses/tcss360 Tutorial 4 Data
More informationProduct Data Sheet: Ignition 8 Industrial Application Platform. A Whole New View
Product Data Sheet: Ignition 8 Industrial Application Platform A Whole New View With the powerful tools, web technologies, and next-generation visualization system we re building in Ignition 8, you ll
More informationManaging PKI Deployments. WhiteRabbitSecurity
Managing PKI Deployments Overview Business requirements Automated CI/CD process overview Demo VM (with Vagrant) (directory layout, usage) Modifying the configuration Deployment in test environment Marking
More informationIntroduction to GALILEO
Introduction to GALILEO Parallel & production environment Mirko Cestari m.cestari@cineca.it Alessandro Marani a.marani@cineca.it Domenico Guida d.guida@cineca.it Maurizio Cremonesi m.cremonesi@cineca.it
More informationSchema Management with Flyway
Schema Management with Flyway Scott Mead scottm@openscg.com Sr. Architect OpenSCG & BigSQL PGConf US 2017 1 Strategies Single File Application Managed (ORM) Migration 2 Single File 3 Single File Easiest
More informationUsing the Computer Programming Environment
Information sheet EN064 Overview C2k has developed an environment to allow GCSE and A-Level students to undertake computer programming from within the C2k Managed Service. This environment will deliver
More informationIng. José A. Mejía Villar M.Sc. Computing Center of the Alfred Wegener Institute for Polar and Marine Research
Ing. José A. Mejía Villar M.Sc. jmejia@awi.de Computing Center of the Alfred Wegener Institute for Polar and Marine Research 29. November 2011 Contents 1. Fedora Commons Repository 2. Federico 3. Federico's
More informationCase Study. CMS for Management of Monetization Training Resources
Case Study CMS for Management of Monetization Training Resources Client Requirement The client is a digital marketing company providing efficient strategies for marketing and data monetization to their
More informationAndrej Filipčič
Singularity@SiGNET Andrej Filipčič SiGNET 4.5k cores, 3PB storage, 4.8.17 kernel on WNs and Gentoo host OS 2 ARC-CEs with 700TB cephfs ARC cache and 3 data delivery nodes for input/output file staging
More informationSlurm Birds of a Feather
Slurm Birds of a Feather Tim Wickberg SchedMD SC17 Outline Welcome Roadmap Review of 17.02 release (Februrary 2017) Overview of upcoming 17.11 (November 2017) release Roadmap for 18.08 and beyond Time
More informationAutomatic Dependency Management for Scientific Applications on Clusters. Ben Tovar*, Nicholas Hazekamp, Nathaniel Kremer-Herman, Douglas Thain
Automatic Dependency Management for Scientific Applications on Clusters Ben Tovar*, Nicholas Hazekamp, Nathaniel Kremer-Herman, Douglas Thain Where users are Scientist says: "This demo task runs on my
More informationTech Note 969 Port Configuration List for System Platform 2012 R2 Multi-Galaxy Environment
Tech Note 969 Port Configuration List for System Platform 2012 R2 Multi-Galaxy Environment All Tech Notes, Tech Alerts and KBCD documents and software are provided "as is" without warranty of any kind.
More informationiems Interactive Experiment Management System Final Report
iems Interactive Experiment Management System Final Report Pēteris Ņikiforovs Introduction Interactive Experiment Management System (Interactive EMS or iems) is an experiment management system with a graphical
More informationUL HPC Monitoring in practice: why, what, how, where to look
C. Parisot UL HPC Monitoring in practice: why, what, how, where to look 1 / 22 What is HPC? Best Practices Getting Fast & Efficient UL HPC Monitoring in practice: why, what, how, where to look Clément
More informationHPC Workshop. Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing
HPC Workshop Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing NEEDED EQUIPMENT 1. Laptop with Secure Shell (ssh) for login A. Windows: download/install putty from https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
More informationThe Why and How of HPC-Cloud Hybrids with OpenStack
The Why and How of HPC-Cloud Hybrids with OpenStack OpenStack Australia Day Melbourne June, 2017 Lev Lafayette, HPC Support and Training Officer, University of Melbourne lev.lafayette@unimelb.edu.au 1.0
More informationJune Workshop Series June 27th: All About SLURM University of Nebraska Lincoln Holland Computing Center. Carrie Brown, Adam Caprez
June Workshop Series June 27th: All About SLURM University of Nebraska Lincoln Holland Computing Center Carrie Brown, Adam Caprez Setup Instructions Please complete these steps before the lessons start
More informationCompilation and Parallel Start
Compiling MPI Programs Programming with MPI Compiling and running MPI programs Type to enter text Jan Thorbecke Delft University of Technology 2 Challenge the future Compiling and Starting MPI Jobs Compiling:
More informationSherlock for IBIIS. William Law Stanford Research Computing
Sherlock for IBIIS William Law Stanford Research Computing Overview How we can help System overview Tech specs Signing on Batch submission Software environment Interactive jobs Next steps We are here to
More informationCrystal Enterprise. Overview. Contents. Web Server Overview - Internet Information System (IIS)
Overview Contents This document provides an overview to web server technology particularly Microsoft s Internet Information Server (IIS) and its relationship with. Although this article has been written
More informationInternet2 Meeting September 2005
End User Agents: extending the "intelligence" to the edge in Distributed Systems Internet2 Meeting California Institute of Technology 1 OUTLINE (Monitoring Agents using a Large, Integrated s Architecture)
More informationReal Life Web Development. Joseph Paul Cohen
Real Life Web Development Joseph Paul Cohen joecohen@cs.umb.edu Index 201 - The code 404 - How to run it? 500 - Your code is broken? 200 - Someone broke into your server? 400 - How are people using your
More informationFusion Registry 9 SDMX Data and Metadata Management System
Registry 9 Data and Management System Registry 9 is a complete and fully integrated statistical data and metadata management system using. Whether you require a metadata repository supporting a highperformance
More informationSubmitting batch jobs
Submitting batch jobs SLURM on ECGATE Xavi Abellan Xavier.Abellan@ecmwf.int ECMWF February 20, 2017 Outline Interactive mode versus Batch mode Overview of the Slurm batch system on ecgate Batch basic concepts
More informationDatabase Replication in Tashkent. CSEP 545 Transaction Processing Sameh Elnikety
Database Replication in Tashkent CSEP 545 Transaction Processing Sameh Elnikety Replication for Performance Expensive Limited scalability DB Replication is Challenging Single database system Large, persistent
More informationMastering Linux. Paul S. Wang. CRC Press. Taylor & Francis Group. Taylor & Francis Croup an informa business. A CHAPMAN St HALL BOOK
Mastering Linux Paul S. Wang CRC Press Taylor & Francis Group Boca Raton London New York CRC Press is an Imprint of the Taylor & Francis Croup an informa business A CHAPMAN St HALL BOOK Contents Preface
More informationJenkins User Conference Israel. #jenkinsconf. CI / Liveperson. Gidi Samuels. July 16, #jenkinsconf
Jenkins User Conference Israel #jenkinsconf CI / CD @ Liveperson Gidi Samuels www.liveperson.com July 16, 2014 #jenkinsconf About Myself Lead the CI Team at Liveperson 3 years in Liveperson About Myself
More informationThe LGI Pilot job portal. EGI Technical Forum 20 September 2011 Jan Just Keijser Willem van Engen Mark Somers
The LGI Pilot job portal EGI Technical Forum 20 September 2011 Jan Just Keijser Willem van Engen Mark Somers Outline What? Why? How? Pro's and Cons What's next? Credits 2 What is LGI? LGI Project Server
More informationUsing CSC Environment Efficiently,
Using CSC Environment Efficiently, 13.2.2017 1 Exercises a) Log in to Taito either with your training or CSC user account, either from a terminal (with X11 forwarding) or using NX client b) Go to working
More informationSql 2008 Copy Table Structure And Database To
Sql 2008 Copy Table Structure And Database To Another Table Different you can create a table with same schema in another database first and copy the data like Browse other questions tagged sql-server sql-server-2008r2-express.
More informationOracle Enterprise Manager. 1 Introduction. System Monitoring Plug-in for Oracle Enterprise Manager Ops Center Guide 11g Release 1 (
Oracle Enterprise Manager System Monitoring Plug-in for Oracle Enterprise Manager Ops Center Guide 11g Release 1 (11.1.3.0.0) E18950-03 November 2011 This document provides describes the System Monitoring
More informationDOCS
HOME DOWNLOAD COMMUNITY DEVELOP NEWS DOCS Docker Images Docker Images for Avatica Docker is a popular piece of software that enables other software to run anywhere. In the context of Avatica, we can use
More informationManaging and Deploying GPU Accelerators. ADAC17 - Resource Management Stephane Thiell and Kilian Cavalotti Stanford Research Computing Center
Managing and Deploying GPU Accelerators ADAC17 - Resource Management Stephane Thiell and Kilian Cavalotti Stanford Research Computing Center OUTLINE GPU resources at the SRCC Slurm and GPUs Slurm and GPU
More informationLet's Play... Try to name the databases described on the following slides...
Database Software Let's Play... Try to name the databases described on the following slides... "World's most popular" Free relational database system (RDBMS) that... the "M" in "LAMP" and "XAMP" stacks
More informationRepository In A Box (RIB)
Repository In A Box (RIB) August 18, 2003 Yuanlei Zhang To Be Covered» Brief overview of RIB» The RIB patches» RIB v2.2 The last Perl version release» RIB v3.0 Java version release 1 RIB History & Overview»
More informationDevOps examples on NonStop Tools Overview. Cor Geboers, ATC Consultant
DevOps examples on NonStop Tools Overview Cor Geboers, ATC Consultant About me Cor Geboers Senior Consultant in NonStop ATC, based in Belgium 35+ years in IT development and support 25+ years NonStop experience
More informationEGI Check-in service. Secure and user-friendly federated authentication and authorisation
EGI Check-in service Secure and user-friendly federated authentication and authorisation EGI Check-in Secure and user-friendly federated authentication and authorisation Check-in provides a reliable and
More informationChapter 2. Operating-System Structures
Chapter 2 Operating-System Structures 2.1 Chapter 2: Operating-System Structures Operating System Services User Operating System Interface System Calls Types of System Calls System Programs Operating System
More information