Singularity: container formats
|
|
- April Ellen Cox
- 5 years ago
- Views:
Transcription
1
2 Singularity Easy to install and configure Easy to run/use: no daemons no root works with scheduling systems User outside container == user inside container Access to host resources Mount (parts of) filesystems of the host MPI
3 Singularity: container formats Singularity container format Unix directory Different compressed formats.tar,.tar.gz,.tar.bz2 cpio, cpio.gz Supported URIs: docker:// shub://
4 Singularity workflow User controlled system Shared computational resource Root/superuser Regular user Create a new container Bootstrap/install container Modify container Copy / share image Execute / run container Command line shell in container
5 Singularity workflow User controlled system Shared computational resource Root/superuser Regular user Create a new container Bootstrap/install container Modify container Copy / share image Today Execute / run container Command line shell in container
6 VirtualBox VM + Peregrine Ubuntu VM in VirtualBox username: tutorial password: singularity sudo privileges ssh pg-interactive.hpc.rug.nl username: f112511, f112512,, f password: regular user, no special privileges
7 Singularity command USAGE: singularity [global options...] <command> [command options...]... GLOBAL -d -h -q OPTIONS: --debug --help --quiet --version -v --verbose -x --sh-debug Print debugging information Display usage summary Only print errors Show application version Increase verbosity +1 Print shell wrapper debugging information GENERAL COMMANDS: help Show additional help for a command CONTAINER USAGE COMMANDS: exec Execute a command within container run Launch a runscript within container shell Run a Bourne shell within container test Execute any test code defined within container CONTAINER MANAGEMENT COMMANDS (requires root): bootstrap Bootstrap a new Singularity image from scratch copy Copy files from your host into the container create Create a new container image expand Grow the container image export Export the contents of a container via a tar pipe import Import/add container contents via a tar pipe mount Mount a Singularity container image For any additional help or support visit the Singularity website:
8 End-goal of this tutorial Submit a job that uses Singularity to run a Python script using pandas that takes an input file and processes it. The Python script is in /data/f1125xx/scripts/, while the input file is in /data/f1125xx/files/. Install Singularity in the VM Create an image, populate it with an OS and install pandas, matplotlib Copy the image to Peregrine and run it through a job
9 Exercise #1: Install Singularity On Peregrine we use the latest stable version Use latest stable version (2.2.1) from github.com Install Singularity: configure, make, make install./configure --prefix=/usr/local make sudo make install
10 How to create an empty image singularity create [options] <image> -s/--size Size of image (default 768 MiB) -F/--force Overwrite an image file if it exists
11 How to populate the container Reuse an existing image: singularity import Easy and quick Customize later Less reproducible
12 How to populate the container singularity import <container> <URI> http/https Pull image using curl docker Docker repository image shub Singularity Hub image file Local file
13 How to access the container [sudo] singularity shell [options] <container> -s/--shell Specify interactive shell -w/--writable Shell into write mode -c/--contains Do not carry over /home to the container -B/--bind <src:dest> Mount src in container at dest -H/--home Use a different home dir in your container --pwd <path> Container path of cwd
14 Exercise #2: Build container create: Create a Singularity image import: Build a container with Python from docker shell: Shell into the container to test it Verify which operating system is installed Check your username, user id and groups Check the disk space usage Do environment variables like $PATH get transferred from the hostto the container?
15 Exercise #2: Build container sudo singularity create image.img sudo singularity import image.img docker://python:3.5.2 singularity shell --shell /bin/bash image.img cat /etc/os-release id df -h env
16 How to expand an image singularity expand [options] <image> -s/--size Size of to expand BY (default 512 MiB) Expands the image <image> BY --size, NOT TO --size
17 How to execute a command singularity exec [options] <image> <command> [options] Same as for shell
18 Exercise #3: Modify and run expand: Expand the image size shell -w: Shell into the container (with write permission) apt, yum, etc: Upgrade software, install pandas and matplotlib run python3.5 script.py inputfile.csv in container
19 Exercise #3: Modify and run sudo singularity expand --size 1024 image.img sudo singularity shell --shell /bin/bash -w image.img apt update apt upgrade pip3 install pandas matplotlib singularity exec image.img python3.5 script.py inputfile.csv
20
21 Exercise #4: Copy to Peregrine scp: Copy image to Peregrine shell --contain: Shell into the container without /home What does it do? What happens if you store something in /home anyway? shell --bind: Map /data/f1125xx into container sudo singularity shell --shell /bin/bash -w image.img mkdir -p /data/f1125xx scp image.img singularity shell --contain --shell /bin/bash image.img pwd ls $HOME singularity shell --bind /data/f1125xx/:/data/f1125xx image.img ls /data/f1125xx
22 Exercise #5: Create and submit job exec: Create job script running <script.py> on <file.in> Submit job Check results nano jobscript.sh #SBATCH... singularity exec image.img <command> sbatch jobscript.sh less <file.out>
23 How to populate the container Start from scratch: singularity bootstrap Write a definition file Bit more complex and time-consuming Very reproducible: Easily rebuild your container Share the definition file with others
24 How to populate the container singularity bootstrap <image> <definition_file> -f/--force Reinstall the core of the OS in the container
25 Definition file Choose base OS or base container: yum: Redhat / CentOS / Scientific Linux debootstrap: Debian / Ubuntu arch: Arch Linux busybox: BusyBox docker: Docker image Define scriptable sections for: setup (run outside the container) post (run inside the container) runscript (run when executing the container) test (for testing the container)
26 Definition file: example BootStrap: debootstrap OSVersion: trusty MirrorURL: %post echo "Running post-installation steps inside the container" apt-get -y install python3.4 %runscript exec python3.4 %test python3.4 -V
27 How to run a container singularity run [options] <image.img> [args] or./image.img [args] [options] Same as for shell/exec [args] Arguments passed to the runscript executable, located inside the container at /singularity.
28 Exercise #6: Automate workflow bootstrap: Create a definition file for the workflow and bootstrap your container scp, run: Test on Peregrine nano pandas.def sudo singularity bootstrap image.img pandas.def singularity run --bind /data/f1125xx:/data/f1125xx image.img script.py inputfile.csv./image.img script.py inputfile.csv
29 Exercise #7 (optional): GPUs Compile a CUDA application on the login node, e.g.: Try to run the application in your container on a GPU node: Submit it to the gpu partition Request one gpu Use the GPU node that we reserved for this tutorial: #SBATCH --reservation=singularity Note that your application needs the Nvidia drivers in order to run on the GPU. On the GPU nodes these can be found in /usr/lib64/nvidia; you will need to make them available in your container as well. Hint: use a bind mount and set your LD_LIBRARY_PATH, or:
30 More information Singularity website: Singularity Hub: Peregrine wiki page: RIS Academy website:
31 Questions
Singularity: Containers for High-Performance Computing. Grigory Shamov Nov 21, 2017
Singularity: Containers for High-Performance Computing Grigory Shamov Nov 21, 2017 Outline Software and High Performance Computing: Installation/Maintenance of the HPC Software stack Why containers and
More informationCONTAINERIZING JOBS ON THE ACCRE CLUSTER WITH SINGULARITY
CONTAINERIZING JOBS ON THE ACCRE CLUSTER WITH SINGULARITY VIRTUAL MACHINE (VM) Uses so&ware to emulate an en/re computer, including both hardware and so&ware. Host Computer Virtual Machine Host Resources:
More informationDocker for HPC? Yes, Singularity! Josef Hrabal
Docker for HPC? Yes, Singularity! Josef Hrabal IT4Innovations josef.hrabal@vsb.cz support@it4i.cz Virtual Machine Hardware (CPU, Memory, NIC, HDD) Host OS (Windows, Linux, MacOS) Hypervisor (VirtualBox,
More informationIntroduction to Containers. Martin Čuma Center for High Performance Computing University of Utah
Introduction to Containers Martin Čuma Center for High Performance Computing University of Utah m.cuma@utah.edu Overview Why do we want to use containers? Containers basics Prepare your computer for containers
More informationPresented By: Gregory M. Kurtzer HPC Systems Architect Lawrence Berkeley National Laboratory CONTAINERS IN HPC WITH SINGULARITY
Presented By: Gregory M. Kurtzer HPC Systems Architect Lawrence Berkeley National Laboratory gmkurtzer@lbl.gov CONTAINERS IN HPC WITH SINGULARITY A QUICK REVIEW OF THE LANDSCAPE Many types of virtualization
More informationIntroduction to Containers. Martin Čuma Center for High Performance Computing University of Utah
Introduction to Containers Martin Čuma Center for High Performance Computing University of Utah m.cuma@utah.edu Overview Why do we want to use containers? Containers basics Prepare your computer for containers
More informationContainers. Pablo F. Ordóñez. October 18, 2018
Containers Pablo F. Ordóñez October 18, 2018 1 Welcome Song: Sola vaya Interpreter: La Sonora Ponceña 2 Goals Containers!= ( Moby-Dick ) Containers are part of the Linux Kernel Make your own container
More informationCENG 334 Computer Networks. Laboratory I Linux Tutorial
CENG 334 Computer Networks Laboratory I Linux Tutorial Contents 1. Logging In and Starting Session 2. Using Commands 1. Basic Commands 2. Working With Files and Directories 3. Permission Bits 3. Introduction
More informationIntroduction to the shell Part II
Introduction to the shell Part II Graham Markall http://www.doc.ic.ac.uk/~grm08 grm08@doc.ic.ac.uk Civil Engineering Tech Talks 16 th November, 1pm Last week Covered applications and Windows compatibility
More informationShifter at CSCS Docker Containers for HPC
Shifter at CSCS Docker Containers for HPC HPC Advisory Council Swiss Conference Alberto Madonna, Lucas Benedicic, Felipe A. Cruz, Kean Mariotti - CSCS April 9 th, 2018 Table of Contents 1. Introduction
More informationGPU Cluster Usage Tutorial
GPU Cluster Usage Tutorial How to make caffe and enjoy tensorflow on Torque 2016 11 12 Yunfeng Wang 1 PBS and Torque PBS: Portable Batch System, computer software that performs job scheduling versions
More informationContents. Note: pay attention to where you are. Note: Plaintext version. Note: pay attention to where you are... 1 Note: Plaintext version...
Contents Note: pay attention to where you are........................................... 1 Note: Plaintext version................................................... 1 Hello World of the Bash shell 2 Accessing
More informationSINGULARITY P1. Containers for Science, Reproducibility and Mobility
SINGULARITY P1 Containers for Science, Reproducibility and Mobility Presented By: Gregory M. Kurtzer HPC Systems Architect Lawrence Berkeley National Lab gmkurtzer@lbl.gov http://singularity.lbl.gov/ CONTAINERS
More informationCentre de Calcul de l Institut National de Physique Nucléaire et de Physique des Particules. Singularity overview. Vanessa HAMAR
Centre de Calcul de l Institut National de Physique Nucléaire et de Physique des Particules Singularity overview Vanessa HAMAR Disclaimer } The information in this presentation was compiled from different
More informationAndrej Filipčič
Singularity@SiGNET Andrej Filipčič SiGNET 4.5k cores, 3PB storage, 4.8.17 kernel on WNs and Gentoo host OS 2 ARC-CEs with 700TB cephfs ARC cache and 3 data delivery nodes for input/output file staging
More informationSingularity - Containers for Scientific Computing ZID Workshop. Innsbruck Nov 2018
Singularity - Containers for Scientific Computing ZID Workshop Michael Fink Universität Innsbruck Innsbruck Nov 2018 Overview Preliminaries Why containers Understanding Containers vs. Virtual Machines
More informationIntroduction: What is Unix?
Introduction Introduction: What is Unix? An operating system Developed at AT&T Bell Labs in the 1960 s Command Line Interpreter GUIs (Window systems) are now available Introduction: Unix vs. Linux Unix
More informationTravis Cardwell Technical Meeting
.. Introduction to Docker Travis Cardwell Tokyo Linux Users Group 2014-01-18 Technical Meeting Presentation Motivation OS-level virtualization is becoming accessible Docker makes it very easy to experiment
More informationFtp Command Line Commands Linux Example Windows Put
Ftp Command Line Commands Linux Example Windows Put Examples of typical uses of the command ftp. This lists the commands that you can use to show the directory contents, transfer files, and delete files.
More informationPlatform Migrator Technical Report TR
Platform Migrator Technical Report TR2018-990 Munir Contractor mmc691@nyu.edu Christophe Pradal christophe.pradal@inria.fr Dennis Shasha shasha@cs.nyu.edu May 12, 2018 CONTENTS: 1 Abstract 4 2 Platform
More informationLinux Essentials. Smith, Roderick W. Table of Contents ISBN-13: Introduction xvii. Chapter 1 Selecting an Operating System 1
Linux Essentials Smith, Roderick W. ISBN-13: 9781118106792 Table of Contents Introduction xvii Chapter 1 Selecting an Operating System 1 What Is an OS? 1 What Is a Kernel? 1 What Else Identifies an OS?
More informationGraham vs legacy systems
New User Seminar Graham vs legacy systems This webinar only covers topics pertaining to graham. For the introduction to our legacy systems (Orca etc.), please check the following recorded webinar: SHARCNet
More informationDocker task in HPC Pack
Docker task in HPC Pack We introduced docker task in HPC Pack 2016 Update1. To use this feature, set the environment variable CCP_DOCKER_IMAGE of a task so that it could be run in a docker container on
More informationSingularity. Containers for Science
Singularity Containers for Science Presented by: Gregory M. Kurtzer HPC Systems Architect and Open Source Developer Lawrence Berkeley National Laboratory and UC Berkeley Containers. In a
More informationLinux Essentials Objectives Topics:
Linux Essentials Linux Essentials is a professional development certificate program that covers basic knowledge for those working and studying Open Source and various distributions of Linux. Exam Objectives
More informationRecap From Last Time:
Recap From Last Time: BGGN 213 Working with UNIX Barry Grant http://thegrantlab.org/bggn213 Motivation: Why we use UNIX for bioinformatics. Modularity, Programmability, Infrastructure, Reliability and
More informationBGGN 213 Working with UNIX Barry Grant
BGGN 213 Working with UNIX Barry Grant http://thegrantlab.org/bggn213 Recap From Last Time: Motivation: Why we use UNIX for bioinformatics. Modularity, Programmability, Infrastructure, Reliability and
More informationIntroduction to Linux. Woo-Yeong Jeong Computer Systems Laboratory Sungkyunkwan University
Introduction to Linux Woo-Yeong Jeong (wooyeong@csl.skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu What is Linux? A Unix-like operating system of a computer What is an
More informationUnzip command in unix
Unzip command in unix Search 24-4-2015 Howto Extract Zip Files in a Linux and. You need to use the unzip command on a Linux or Unix like system. The nixcraft takes a lot of my time and. 16-4-2010 Howto:
More informationExercise 1: Basic Tools
Exercise 1: Basic Tools This exercise is created so everybody can learn the basic tools we will use during this course. It is really more like a tutorial than an exercise and, you are not required to submit
More informationIntroduction to Linux
Introduction to Linux Prof. Jin-Soo Kim( jinsookim@skku.edu) TA - Kisik Jeong (kisik@csl.skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu What is Linux? A Unix-like operating
More informationLING 408/508: Computational Techniques for Linguists. Lecture 5
LING 408/508: Computational Techniques for Linguists Lecture 5 Last Time Installing Ubuntu 18.04 LTS on top of VirtualBox Your Homework 2: did everyone succeed? Ubuntu VirtualBox Host OS: MacOS or Windows
More informationIntroduction to Linux
Introduction to Linux Prof. Jin-Soo Kim( jinsookim@skku.edu) TA - Dong-Yun Lee (dylee@csl.skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu What is Linux? A Unix-like operating
More informationShifter and Singularity on Blue Waters
Shifter and Singularity on Blue Waters Maxim Belkin June 7, 2018 A simplistic view of a scientific application DATA RESULTS My Application Received an allocation on Blue Waters! DATA RESULTS My Application
More informationThis tutorial will guide you how to setup and run your own minecraft server on a Linux CentOS 6 in no time.
This tutorial will guide you how to setup and run your own minecraft server on a Linux CentOS 6 in no time. Running your own server lets you play together with your friends and family with your own set
More informationIntroduction to remote command line Linux. Research Computing Team University of Birmingham
Introduction to remote command line Linux Research Computing Team University of Birmingham Linux/UNIX/BSD/OSX/what? v All different v UNIX is the oldest, mostly now commercial only in large environments
More informationEECS Software Tools. Lab 2 Tutorial: Introduction to UNIX/Linux. Tilemachos Pechlivanoglou
EECS 2031 - Software Tools Lab 2 Tutorial: Introduction to UNIX/Linux Tilemachos Pechlivanoglou (tipech@eecs.yorku.ca) Sep 22 & 25, 2017 Material marked with will be in your exams Sep 22 & 25, 2017 Introduction
More informationGNU/Linux 101. Casey McLaughlin. Research Computing Center Spring Workshop Series 2018
GNU/Linux 101 Casey McLaughlin Research Computing Center Spring Workshop Series 2018 rccworkshop IC;3df4mu bash-2.1~# man workshop Linux101 RCC Workshop L101 OBJECTIVES - Operating system concepts - Linux
More informationLINUX FUNDAMENTALS. Supported Distributions: Red Hat Enterprise Linux 6 SUSE Linux Enterprise 11 Ubuntu LTS. Recommended Class Length: 5 days
LINUX FUNDAMENTALS The course is a challenging course that focuses on the fundamental tools and concepts of Linux and Unix. Students gain proficiency using the command line. Beginners develop a solid foundation
More informationIntroduction to Linux
Introduction to Linux University of Bristol - Advance Computing Research Centre 1 / 47 Operating Systems Program running all the time Interfaces between other programs and hardware Provides abstractions
More informationHow To Start Mysql Using Linux Command Line Client In Ubuntu
How To Start Mysql Using Linux Command Line Client In Ubuntu Step One: Install MySQL Client On Debian, Ubuntu or Linux Mint: Before you start typing commands at the MySQL prompt, remember that each In
More informationCS197U: A Hands on Introduction to Unix
CS197U: A Hands on Introduction to Unix Lecture 4: My First Linux System Tian Guo University of Massachusetts Amherst CICS 1 Reminders Assignment 2 was due before class Assignment 3 will be posted soon
More informationFEniCS Containers Documentation
FEniCS Containers Documentation Release 1.0 FEniCS Project Jan 29, 2018 Contents 1 Quickstart 3 2 Introduction 5 2.1 What is Docker?............................................. 5 2.2 Installing Docker.............................................
More informationIntroduction in Unix. Linus Torvalds Ken Thompson & Dennis Ritchie
Introduction in Unix Linus Torvalds Ken Thompson & Dennis Ritchie My name: John Donners John.Donners@surfsara.nl Consultant at SURFsara And Cedric Nugteren Cedric.Nugteren@surfsara.nl Consultant at SURFsara
More informationHow To Start Mysql Use Linux Command Line Client In Ubuntu
How To Start Mysql Use Linux Command Line Client In Ubuntu Getting started with MySQL for web and server applications on Ubuntu 14.04 LTS (Trusty Tahr). get started with MySQL on an Ubuntu 14.04 LTS (Trusty
More informationDownloading and installing Db2 Developer Community Edition on Ubuntu Linux Roger E. Sanders Yujing Ke Published on October 24, 2018
Downloading and installing Db2 Developer Community Edition on Ubuntu Linux Roger E. Sanders Yujing Ke Published on October 24, 2018 This guide will help you download and install IBM Db2 software, Data
More informationLinux Training. for New Users of Cluster. Georgia Advanced Computing Resource Center University of Georgia Suchitra Pakala
Linux Training for New Users of Cluster Georgia Advanced Computing Resource Center University of Georgia Suchitra Pakala pakala@uga.edu 1 Overview GACRC Linux Operating System Shell, Filesystem, and Common
More informationAn introduction to Docker
An introduction to Docker Ing. Vincenzo Maffione Operating Systems Security Container technologies on Linux Several light virtualization technologies are available for Linux They build on cgroups, namespaces
More informationUoW HPC Quick Start. Information Technology Services University of Wollongong. ( Last updated on October 10, 2011)
UoW HPC Quick Start Information Technology Services University of Wollongong ( Last updated on October 10, 2011) 1 Contents 1 Logging into the HPC Cluster 3 1.1 From within the UoW campus.......................
More informationIntroduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU
Introduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU What is Joker? NMSU s supercomputer. 238 core computer cluster. Intel E-5 Xeon CPUs and Nvidia K-40 GPUs. InfiniBand innerconnect.
More informationQuick Start Guide. by Burak Himmetoglu. Supercomputing Consultant. Enterprise Technology Services & Center for Scientific Computing
Quick Start Guide by Burak Himmetoglu Supercomputing Consultant Enterprise Technology Services & Center for Scientific Computing E-mail: bhimmetoglu@ucsb.edu Contents User access, logging in Linux/Unix
More informationUnix / Linux Overview
Unix / Linux Overview Jonathan Brewer Network Startup Resource Center jon@nsrc.org These materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International license (http://creativecommons.org/licenses/by-nc/4.0/)
More informationShifter on Blue Waters
Shifter on Blue Waters Why Containers? Your Computer Another Computer (Supercomputer) Application Application software libraries System libraries software libraries System libraries Why Containers? Your
More informationLinux Systems Administration Getting Started with Linux
Linux Systems Administration Getting Started with Linux Network Startup Resource Center www.nsrc.org These materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International
More informationA Hands-On Tutorial: RNA Sequencing Using High-Performance Computing
A Hands-On Tutorial: RNA Sequencing Using Computing February 11th and 12th, 2016 1st session (Thursday) Preliminaries: Linux, HPC, command line interface Using HPC: modules, queuing system Presented by:
More informationMore Raspian. An editor Configuration files Shell scripts Shell variables System admin
More Raspian An editor Configuration files Shell scripts Shell variables System admin Nano, a simple editor Nano does not require the mouse. You must use your keyboard to move around the file and make
More informationIntroduction To Linux. Rob Thomas - ACRC
Introduction To Linux Rob Thomas - ACRC What Is Linux A free Operating System based on UNIX (TM) An operating system originating at Bell Labs. circa 1969 in the USA More of this later... Why Linux? Free
More informationTangeloHub Documentation
TangeloHub Documentation Release None Kitware, Inc. September 21, 2015 Contents 1 User s Guide 3 1.1 Managing Data.............................................. 3 1.2 Running an Analysis...........................................
More informationbwunicluster Tutorial Access, Data Transfer, Compiling, Modulefiles, Batch Jobs
bwunicluster Tutorial Access, Data Transfer, Compiling, Modulefiles, Batch Jobs Frauke Bösert, SCC, KIT 1 Material: Slides & Scripts https://indico.scc.kit.edu/indico/event/263/ @bwunicluster/forhlr I/ForHLR
More informationEE516: Embedded Software Project 1. Setting Up Environment for Projects
EE516: Embedded Software Project 1. Setting Up Environment for Projects By Dong Jae Shin 2015. 09. 01. Contents Introduction to Projects of EE516 Tasks Setting Up Environment Virtual Machine Environment
More informationCS155: Computer Security Spring Project #1
CS155: Computer Security Spring 2018 Project #1 Due: Part 1: Thursday, April 12-11:59pm, Parts 2 and 3: Thursday, April 19-11:59pm. The goal of this assignment is to gain hands-on experience finding vulnerabilities
More informationUbuntu LTS Install Guide
Ubuntu 16.04.5 LTS Install Guide Sirenia September 17, 2018 Contents 1 Content 2 2 Login to server 2 3 Ensure access to repositories 3 4 Install Docker 3 5 Install Docker Compose 4 6 Pull software 4 7
More informationOperating Systems Linux 1-2 Measurements Background material
Operating Systems Linux 1-2 Measurements Background material Introduction The Linux measurements were designed to allow you to have an impression about the administration of Linux severs along with providing
More informationPreparing Your Google Cloud VM for W4705
Preparing Your Google Cloud VM for W4705 August 27, 2017 1. Get a cloud.cs.columbia.edu account 1. Sign up for a cloud Columbia CS account using this link. Note that is is an entirely new account and is
More informationJackson State University Department of Computer Science CSC / Computer Security Fall 2013 Instructor: Dr. Natarajan Meghanathan
Jackson State University Department of Computer Science CSC 437-01/539-01 Computer Security Fall 2013 Instructor: Dr. Natarajan Meghanathan Lab Project # 2: Running Secure Shell (SSH) Server in a Virtual
More informationLinux Operating System Environment Computadors Grau en Ciència i Enginyeria de Dades Q2
Linux Operating System Environment Computadors Grau en Ciència i Enginyeria de Dades 2017-2018 Q2 Facultat d Informàtica de Barcelona This first lab session is focused on getting experience in working
More informationPrerequisites: Students should be comfortable with computers. No familiarity with Linux or other Unix operating systems is required.
GL-120: Linux Fundamentals Course Length: 4 days Course Description: The GL120 is a challenging course that focuses on the fundamental tools and concepts of Linux and Unix. Students gain proficiency using
More informationParallel Programming Pre-Assignment. Setting up the Software Environment
Parallel Programming Pre-Assignment Setting up the Software Environment Authors: B. Wilkinson and C. Ferner. Modification date: Aug 21, 2014 (Minor correction Aug 27, 2014.) Software The purpose of this
More informationAutomatic Dependency Management for Scientific Applications on Clusters. Ben Tovar*, Nicholas Hazekamp, Nathaniel Kremer-Herman, Douglas Thain
Automatic Dependency Management for Scientific Applications on Clusters Ben Tovar*, Nicholas Hazekamp, Nathaniel Kremer-Herman, Douglas Thain Where users are Scientist says: "This demo task runs on my
More informationdocker & HEP: containerization of applications for development, distribution and preservation
docker & HEP: containerization of applications for development, distribution and preservation Sébastien Binet LAL/IN2P3 2015-04-13 S. Binet (LAL) docker-hep 2015-04-13 1 / 16 Docker: what is it? http://www.docker.io/
More informationPREPARING TO USE CONTAINERS
PREPARING TO USE CONTAINERS DU-08786-001_v001 May 2018 Getting Started Guide TABLE OF CONTENTS Chapter Introduction To Docker And Containers...1 Chapter 2. Preparing Your DGX System For Use With nvidia-docker...
More informationPerl and R Scripting for Biologists
Perl and R Scripting for Biologists Lukas Mueller PLBR 4092 Course overview Linux basics (today) Linux advanced (Aure, next week) Why Linux? Free open source operating system based on UNIX specifications
More informationbwunicluster Tutorial Access, Data Transfer, Compiling, Modulefiles, Batch Jobs
bwunicluster Tutorial Access, Data Transfer, Compiling, Modulefiles, Batch Jobs Frauke Bösert, SCC, KIT 1 Material: Slides & Scripts https://indico.scc.kit.edu/indico/event/263/ @bwunicluster/forhlr I/ForHLR
More informationUSING NGC WITH GOOGLE CLOUD PLATFORM
USING NGC WITH GOOGLE CLOUD PLATFORM DU-08962-001 _v02 April 2018 Setup Guide TABLE OF CONTENTS Chapter 1. Introduction to... 1 Chapter 2. Deploying an NVIDIA GPU Cloud Image from the GCP Console...3 2.1.
More informationINTRODUCTION TO LINUX
INTRODUCTION TO LINUX REALLY SHORT HISTORY Before GNU/Linux there were DOS, MAC and UNIX. All systems were proprietary. The GNU project started in the early 80s by Richard Stallman Goal to make a free
More informationSupercomputing environment TMA4280 Introduction to Supercomputing
Supercomputing environment TMA4280 Introduction to Supercomputing NTNU, IMF February 21. 2018 1 Supercomputing environment Supercomputers use UNIX-type operating systems. Predominantly Linux. Using a shell
More informationUsing the computational resources at the GACRC
An introduction to zcluster Georgia Advanced Computing Resource Center (GACRC) University of Georgia Dr. Landau s PHYS4601/6601 course - Spring 2017 What is GACRC? Georgia Advanced Computing Resource Center
More informationSystems Programming and Computer Architecture ( ) Exercise Session 01 Data Lab
Systems Programming and Computer Architecture (252-0061-00) Exercise Session 01 Data Lab 1 Goal Get familiar with bit level representations, C and Linux Thursday, September 22, 2016 Systems Programming
More informationIT341 Introduction to System Administration. Project 4 - Backup Strategies with rsync and crontab
IT341 Introduction to System Administration Project 4 - Backup Strategies with rsync and crontab Backup is one of the most important things a system administrator does. It is important to decide what data
More information1Z Oracle Linux Fundamentals (Oracle Partner Network) Exam Summary Syllabus Questions
1Z0-409 Oracle Linux Fundamentals (Oracle Partner Network) Exam Summary Syllabus Questions Table of Contents Introduction to 1Z0-409 Exam on Oracle Linux Fundamentals (Oracle Partner Network)... 2 Oracle
More informationDockerfile Best Practices
Dockerfile Best Practices OpenRheinRuhr 2015 November 07th, 2015 1 Dockerfile Best Practices Outline About Dockerfile Best Practices Building Images This work is licensed under the Creative Commons Attribution-ShareAlike
More informationPrerequisites: Students must be proficient in general computing skills but not necessarily experienced with Linux or Unix. Supported Distributions:
This GL124 course is designed to follow an identical set of topics as the Red Hat RH124 course with the added benefit of very comprehensive lab exercises and detailed lecture material. The Red Hat Enterprise
More informationLinux Command Line Primer. By: Scott Marshall
Linux Command Line Primer By: Scott Marshall Draft: 10/21/2007 Table of Contents Topic Page(s) Preface 1 General Filesystem Background Information 2 General Filesystem Commands 2 Working with Files and
More informationHow to run NoMachine server inside Docker
How to run NoMachine server inside Docker Page 1 of 5 Given that Docker is installed on the host machine, to run NoMachine server inside Docker it's enough to build an image from the Dockerfile and launch
More informationSTARTING THE DDT DEBUGGER ON MIO, AUN, & MC2. (Mouse over to the left to see thumbnails of all of the slides)
STARTING THE DDT DEBUGGER ON MIO, AUN, & MC2 (Mouse over to the left to see thumbnails of all of the slides) ALLINEA DDT Allinea DDT is a powerful, easy-to-use graphical debugger capable of debugging a
More informationBright Cluster Manager: Using the NVIDIA NGC Deep Learning Containers
Bright Cluster Manager: Using the NVIDIA NGC Deep Learning Containers Technical White Paper Table of Contents Pre-requisites...1 Setup...2 Run PyTorch in Kubernetes...3 Run PyTorch in Singularity...4 Run
More informationIntroduction to UNIX. SURF Research Boot Camp April Jeroen Engelberts Consultant Supercomputing
Introduction to UNIX SURF Research Boot Camp April 2018 Jeroen Engelberts jeroen.engelberts@surfsara.nl Consultant Supercomputing Outline Introduction to UNIX What is UNIX? (Short) history of UNIX Cartesius
More informationBooting a Galaxy Instance
Booting a Galaxy Instance Create Security Groups First time Only Create Security Group for Galaxy Name the group galaxy Click Manage Rules for galaxy Click Add Rule Choose HTTPS and Click Add Repeat Security
More informationAbout the Tutorial. Audience. Prerequisites. Copyright & Disclaimer. Gerrit
Gerrit About the Tutorial Gerrit is a web-based code review tool, which is integrated with Git and built on top of Git version control system (helps developers to work together and maintain the history
More informationIntroduction To. Barry Grant
Introduction To Barry Grant bjgrant@umich.edu http://thegrantlab.org Working with Unix How do we actually use Unix? Inspecting text files less - visualize a text file: use arrow keys page down/page up
More informationPROJECT INFRASTRUCTURE AND BASH INTRODUCTION MARKUS PILMAN<
PROJECT INFRASTRUCTURE AND BASH INTRODUCTION MARKUS PILMAN< MPILMAN@INF.ETHZ.CH> ORGANIZATION Tutorials on Tuesdays - Sometimes, will be announced In General: no exercise sessions (unless you get an email
More informationGit Fusion Guide February 2016 Update
Git Fusion Guide 2016.1 February 2016 Update Git Fusion Guide 2016.1 February 2016 Update Copyright 1999-2016 Perforce Software. All rights reserved. Perforce software and documentation is available from
More informationdiskimage-builder: Building Linux Images for Cloud / Virtualization / Container
diskimage-builder: Building Linux Images for Cloud / Virtualization / Container Let s start with a little bit of history: Once upon a time... About the Author Andreas Florath andreas@florath.net Mathematician
More informationL.A.M.P. Stack Part I
L.A.M.P. Stack Part I By George Beatty and Matt Frantz This lab will cover the basic installation and some configuration of a LAMP stack on a Ubuntu virtual box. Students will download and install the
More informationswiftenv Documentation
swiftenv Documentation Release 1.3.0 Kyle Fuller Sep 27, 2017 Contents 1 The User Guide 3 1.1 Installation................................................ 3 1.2 Getting Started..............................................
More informationHPC Workshop. Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing
HPC Workshop Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing NEEDED EQUIPMENT 1. Laptop with Secure Shell (ssh) for login A. Windows: download/install putty from https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
More informationCS CS Tutorial 2 2 Winter 2018
CS CS 230 - Tutorial 2 2 Winter 2018 Sections 1. Unix Basics and connecting to CS environment 2. MIPS Introduction & CS230 Interface 3. Connecting Remotely If you haven t set up a CS environment password,
More informationHPC DOCUMENTATION. 3. Node Names and IP addresses:- Node details with respect to their individual IP addresses are given below:-
HPC DOCUMENTATION 1. Hardware Resource :- Our HPC consists of Blade chassis with 5 blade servers and one GPU rack server. a.total available cores for computing: - 96 cores. b.cores reserved and dedicated
More informationWindows Subsystem for Linux Guide Documentation
Windows Subsystem for Linux Guide Documentation Release Kenneth Reitz Aug 30, 2017 Contents: 1 Background on WSL (Windows Subsystem for Linux) 3 1.1 What is WSL?..............................................
More informationNew User Tutorial. OSU High Performance Computing Center
New User Tutorial OSU High Performance Computing Center TABLE OF CONTENTS Logging In... 3-5 Windows... 3-4 Linux... 4 Mac... 4-5 Changing Password... 5 Using Linux Commands... 6 File Systems... 7 File
More information