XC Series Shifter User Guide (CLE 6.0.UP02) S-2571
|
|
- Randolf Hunt
- 6 years ago
- Views:
Transcription
1 XC Series Shifter User Guide (CLE 6.0.UP02) S-2571
2 Contents Contents 1 About the XC Series Shifter User Guide Shifter System Introduction Download and Convert the Docker Image Submit Jobs to Use a UDI Run Shifter in Interactive Mode Without a Batch Reservation Test the Shifter Configuration for sitefs Mount
3 About the XC Series Shifter User Guide 1 About the XC Series Shifter User Guide The Cray XC Series Shifter User Guide includes information for using the Shifter system. XC Series Shifter User Guide S-2571 CLE 6.0 UP02 This version includes updated procedures to support Cray software release CLE 6.0 UP02, released November 3, 2016 and supports the following new features: The shifterimg command replaces the getdockerimage command used in CLE 5.2 UP04 The Docker daemon is replaced by the Shifter Image Gateway. Scope and Audience This publication is written for Cray customers using the Shifter system. Feedback Visit the Cray Publications Portal at and make comments online using the Contact Us button in the upper-right corner or pubs@cray.com. Your comments are important to us and we will respond within 24 hours. Command Prompt Conventions hostname in command prompts Hostnames in command prompts indicate where the command must be run. hostname# Run the command on this hostname. smw# cmc# boot# sdb# login# smw1# smw2# Run the command on the SMW. Run the command on the CMC. Run the command on the boot node. Run the command on the SDB node. Run the command on any login node. For a system configured with the SMW failover feature there are two SMWs one in an active role and the other in a passive role. The 3
4 About the XC Series Shifter User Guide SMW that is active at the start of a procedure is smw1. The SMW that is passive is smw2. smwactive# smwpassive# In some scenarios, the active SMW is smw1 at the start of a procedure then the procedure requires a failover to the other SMW. In this case, the documentation will continue to refer to the formerly active SMW as smw1, even though smw2 is now the active SMW. If further clarification is needed in a procedure, the active SMW will be called smwactive and the passive SMW will be called smwpassive. account name in command prompts The account that must run the command is also indicated in the prompt. smw# cmc# boot# sdb# login# hostname# The root or super-user account always has the # character at the end of the prompt. crayadm@smw> crayadm@login> user@hostname> Any non-root account is indicated with account@hostname. A user account that is neither root nor crayadm is referred to as user. command prompt inside chroot directory path in command prompt If the chroot command is used, the prompt changes to indicate that it is inside a chroot'd environment on the hostname. smw# chroot /path/to/chroot chroot-smw# Sometimes the current path can be so long that including it in the prompt does not add clarity to the command example. Most of the time, the command can be executed from any directory. When it matters which directory the command is invoked within, the cd command is used to change into the directory, and the directory is referenced with a period (.) to indicate the current directory. For example, here are actual prompts as they appear on the system: smw:~ # cd /etc smw:/etc# cd /var/tmp smw:/var/tmp# ls./file smw:/var/tmp# su - crayadm crayadm@smw:~> cd /usr/bin crayadm@smw:/usr/bin>./command And here are the same prompts as they would appear in examples in this publication: smw# cd /etc smw# cd /var/tmp smw# ls./file 4
5 About the XC Series Shifter User Guide smw# su - crayadm crayadm@smw> cd /usr/bin crayadm@smw>./command Typographic Conventions Monospace Monospaced Bold Oblique or Italics Indicates program code, reserved words, library functions, command-line prompts, screen output, file/path names, and other software constructs. Indicates commands that must be entered on a command line or in response to an interactive prompt. Indicates user-supplied values in commands or syntax definitions. Proportional Bold Indicates a GUI Window, GUI element, cascading menu (Ctrl Alt Delete), or key strokes (press Enter). \ (backslash) At the end of a command line, indicates the Linux shell line continuation character (lines joined by a backslash are parsed as a single line). Trademarks The following are trademarks of Cray Inc. and are registered in the United States and other countries: CRAY and design, SONEXION, URIKA, and YARCDATA. The following are trademarks of Cray Inc.: APPRENTICE2, CHAPEL, CLUSTER CONNECT, CRAYDOC, CRAYPAT, CRAYPORT, DATAWARP, ECOPHLEX, LIBSCI, NODEKARE. The following system family marks, and associated model number marks, are trademarks of Cray Inc.: CS, CX, XC, XE, XK, XMT, and XT. The registered trademark LINUX is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a worldwide basis. Other trademarks used in this document are the property of their respective owners. 5
6 Shifter System Introduction 2 Shifter System Introduction The Shifter system allows users to run commands in a user defined image (UDI). A UDI has many of the features of a Linux container. Some features (e.g., separate pid namespace) are missing because the UDI is intended to be used on a compute node where only one user's apps run at a time. A UDI is created from a Docker image. A user may download an image that has been exported by another site from the Docker Hub. This image can then be unpacked and then packaged into a UDI for the local user to use. Downloading and packaging is done with the shifterimg command. Once a user has a UDI to use, the user can submit a batch job to use the image. The batch job can request use of the image in one of two ways: 1. When the job is submitted by specifying an environment variable that indicates what UDI should be used, or 2. By submitting a job using the shifter command inside the batch job. Both of these methods are described in more detail in this User Guide. 6
7 Download and Convert the Docker Image 3 Download and Convert the Docker Image An image must be downloaded and converted before it can be used. This is done via the shifterimg command. Docker Image Creation Limitations and Considerations The importation of Docker images that are not on DockerHub or other reachable private registries requires either root access or inclusion in the Docker group. Only Docker images downloaded from Docker registries can be run via Shifter on Cray platforms. Cray currently does not support creation of Docker images on Cray platforms. Local Docker registries are supported by this release of Shifter, but they must be specifically included in the configuration of the Shifter image gateway. Contact a system administrator if you need access to another registry to be added. Docker registries that require authentication to access images are not supported by this release of Shifter. To delete an image, contact a system administrator. shifterimg The shifterimg command allows a user to list available images on the local system, look up a particular image, or pull (download and convert) a particular image. The command syntaxes are as follows: user@login> module load shifter user@login> shifterimg images This will display the contents of the Shifter image gateway. user@login> shifterimg lookup <image>:<tag> This will look up the specified image in the local image cache. The lookup command will not search the remote Docker hub. user@login> shifterimg -v lookup docker:<image>:<tag> This will show Shifter image meta data. user@login> shifterimg pull <image>:<tag> This will download and convert the specified image. The local image cache is searched first and then the Docker Hub (hub.docker.com). This form is used internally by the Shifter prologue and the shifter command. If the image hasn t already been downloaded, it is downloaded. If the image hasn t already been converted, it is converted for use. When the image has been downloaded and converted, its ID is printed. This ID is used internally to locate a converted image and to prepare it for use as a UDI. 7
8 Download and Convert the Docker Image shifterimg expire Command By default, only the root user is able to use the shifterimg expire command. shifterimg expire will mark an image for removal. Once no Shifter user is using that image, the Shifter Image Gateway will remove the UDI. shifterimg lookup <image>:<tag> 463ff6be4238c14f5b88898f17b47a9cf494f9a9be7b6170c3e852568d2b0432 shifterimg expire <image>:<tag> shifterimg lookup <image>:<tag> echo $? 1 8
9 Submit Jobs to Use a UDI 4 Submit Jobs to Use a UDI To initialize the UDI during the batch reservation, users must specify the requested UDI to qsub, salloc, or sbatch. Users may also specify the requested image to the shifter command in a batch job. If the shifter_slurm.so plugin has been configured, salloc and sbatch will accept the --image and --volume options directly. The UDI is specified on the qsub command line, as shown in the following examples (please note users must have the shifter module loaded before using any shifter command): Example 1: user@torque-mom> module load shifter user@torque-mom> qsub -I -v SHIFTER=docker:ubuntu:latest user@torque-mom> aprun -b shifter cat /etc/os-release Example 2: user@torque-mom> qsub -I -v SHIFTER_IMAGE=ubuntu:latest,SHIFTER_IMAGETYPE=docker user@torque-mom> aprun -b shifter cat /etc/os-release Example 3: user@torque-mom> qsub -I user@torque-mom> aprun -b shifter --image=docker:ubuntu:latest cat /etc/os-release The SHIFTER_VOLUME environment variable can also be used in place of the --volume option. The -b option to aprun is important. The -b option tells aprun not to transfer the Shifter binary from the mom node to the compute node. If no -b option is used, the transfer will remove the setuid bit required of Shifter and the command will not function. No -b option fails with the following: user@torque-mom> aprun -L 170 shifter --image ubuntu:latest cat /etc/os-release FAILED to find requested image. Application exit codes: 1 Application resources: utime ~0s, stime ~0s, Rss ~4480, inblocks ~0, outblocks ~0 The aprun command may be used multiple times and each run will get the same UDI. The aprun commands may be run concurrently or serially, one after another. A different UDI may also be run even if it was not specified at batch reservation time. Shifter will check if the UDI specified with --image is already present on the node. If it is not, Shifter will mount the UDI in a private namespace, as shown in the following example: user@torque-mom> aprun -b shifter --image docker:ubuntu:14.04 cat /etc/os-release NAME="Ubuntu" 9
10 Submit Jobs to Use a UDI VERSION=" LTS, Trusty Tahr" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu LTS" VERSION_ID="14.04" HOME_URL=" SUPPORT_URL=" BUG_REPORT_URL=" To see that Shifter is mounting the UDI on the node at batch reservation time, first specify a UDI at batch reservation time: user@torque-mom> qsub -I -v SHIFTER=docker:ubuntu:16.04 qsub: waiting for job 90.kachina to start qsub: job 90.kachina ready Initializing udiroot, please wait. Retrieving Docker Image... Next, run Shifter with a different UDI: user@torque-mom> aprun -b shifter --image docker:ubuntu:14.04 sleep 30 Application 628 resources: utime ~0s, stime ~1s, Rss ~4536, inblocks ~2178, outblocks ~0 Finally, check the loop devices mounted on the compute node using losetup: user@torque-mom> ssh root@nid00015 nid00015:~ # shifterimg lookup docker:ubuntu: b6e305389ce5151e289d96b1e0e1df3c86d1bd8d208c997ecbcdfd6e320a0 nid00015:~ # shifterimg lookup docker:ubuntu: bc df5e03e4327bb57c9aa9f1372f d7562a857e242b377d nid00015:~ # losetup -a /dev/loop0: [0027]: (/var/opt/cray/imps-distribution/squash/store/p0/.cache/ squashfs) /dev/loop1: [0027]: (/var/opt/cray/imps-distribution/squash/store/ global/.cache/squashfs) /dev/loop2: [0055]: (/home/users/alanm/shifter/kachina/udi/ 594b6e305389ce5151e289d96b1e0e1df3c86d1bd8d208c997ecbcdfd6e320a0.squashfs) /dev/loop3: [0055]: (/home/users/alanm/shifter/kachina/udi/ 9bc df5e03e4327bb57c9aa9f1372f d7562a857e242b377d.squashfs) 10
11 Run Shifter in Interactive Mode Without a Batch Reservation 5 Run Shifter in Interactive Mode Without a Batch Reservation Shifter can be run in interactive mode without a batch reservation. To do this, Shifter will execute the command from within the container environment. Run cat /etc/os-release inside the Ubuntu container, as shown in the following example: user@torque-mom> aprun -b shifter --image docker:ubuntu:16.04 cat /etc/os-release NAME="Ubuntu" VERSION="16.04 LTS (Xenial Xerus)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu LTS" VERSION_ID="16.04" HOME_URL=" SUPPORT_URL=" BUG_REPORT_URL=" UBUNTU_CODENAME=xenial Application resources: utime ~0s, stime ~0s, Rss ~4372, inblocks ~2248, outblocks ~0 11
12 Test the Shifter Configuration for sitefs Mount 6 Test the Shifter Configuration for sitefs Mount If /etc/shifter/udiroot.conf is configured with an appropriate sitefs option for users' home directories, users are able to access their home directory, as shown in the following example: user@torque-mom> cat /etc/shifter/udiroot.conf grep sitefs grep -v "#" sitefs=/home:/home user@torque-mom> echo "test sitefs mount" > shifter_home_file user@torque-mom> aprun -L 130 -b shifter --image docker:ubuntu:16.04 cat ~/ shifter_home_file test sitefs mount Application resources: utime ~0s, stime ~0s, Rss ~4372, inblocks ~2152, outblocks ~0 User bind mounts inside the container environment. The --volume option will let users bind mount files and directories outside the container, into the container: user@torque-mom> mkdir ~/user_volume user@torque-mom> touch ~/user_volume/user_file user@torque-mom> aprun -L 130 -b shifter --image docker:ubuntu: volume ~/ user_volume:/mnt ls /mnt user_file Application resources: utime ~0s, stime ~0s, Rss ~4372, inblocks ~2782, outblocks ~0 12
Shifter Configuration Guide 1.0
Shifter Configuration Guide 1.0 Contents Contents About Shifter Configuration Guide...3 Configure the Docker Daemon...4 Configure Shifter...7 Rebuild the Compute Node initramfs and cpio Files...10 Configure
More informationHands-On II: Ray Tracing (data parallelism) COMPUTE STORE ANALYZE
Hands-On II: Ray Tracing (data parallelism) Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking statements may include
More informationHow-to write a xtpmd_plugin for your Cray XC system Steven J. Martin
How-to write a xtpmd_plugin for your Cray XC system Steven J. Martin (stevem@cray.com) Cray XC Telemetry Plugin Introduction Enabling sites to get telemetry data off the Cray Plugin interface enables site
More informationXC System Management Usability BOF Joel Landsteiner & Harold Longley, Cray Inc. Copyright 2017 Cray Inc.
XC System Management Usability BOF Joel Landsteiner & Harold Longley, Cray Inc. 1 BOF Survey https://www.surveymonkey.com/r/kmg657s Aggregate Ideas at scale! Take a moment to fill out quick feedback form
More informationSonexion GridRAID Characteristics
Sonexion GridRAID Characteristics CUG 2014 Mark Swan, Cray Inc. 1 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking
More informationCray XC System Node Diagnosability. Jeffrey J. Schutkoske Platform Services Group (PSG)
Cray XC System Node Diagnosability Jeffrey J. Schutkoske Platform Services Group (PSG) jjs@cray.com Safe Harbor Statement This presentation may contain forward-looking statements that are based on our
More informationEvaluating Shifter for HPC Applications Don Bahls Cray Inc.
Evaluating Shifter for HPC Applications Don Bahls Cray Inc. Agenda Motivation Shifter User Defined Images (UDIs) provide a mechanism to access a wider array of software in the HPC environment without enduring
More informationShifter on Blue Waters
Shifter on Blue Waters Why Containers? Your Computer Another Computer (Supercomputer) Application Application software libraries System libraries software libraries System libraries Why Containers? Your
More informationProductive Programming in Chapel: A Computation-Driven Introduction Chapel Team, Cray Inc. SC16, Salt Lake City, UT November 13, 2016
Productive Programming in Chapel: A Computation-Driven Introduction Chapel Team, Cray Inc. SC16, Salt Lake City, UT November 13, 2016 Safe Harbor Statement This presentation may contain forward-looking
More informationArray, Domain, & Domain Map Improvements Chapel Team, Cray Inc. Chapel version 1.17 April 5, 2018
Array, Domain, & Domain Map Improvements Chapel Team, Cray Inc. Chapel version 1.17 April 5, 2018 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current
More informationQ & A, Project Status, and Wrap-up COMPUTE STORE ANALYZE
Q & A, Project Status, and Wrap-up Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking statements may include statements
More informationIntel Xeon PhiTM Knights Landing (KNL) System Software Clark Snyder, Peter Hill, John Sygulla
Intel Xeon PhiTM Knights Landing (KNL) System Software Clark Snyder, Peter Hill, John Sygulla Motivation The Intel Xeon Phi TM Knights Landing (KNL) has 20 different configurations 5 NUMA modes X 4 memory
More informationThe Use and I: Transitivity of Module Uses and its Impact
The Use and I: Transitivity of Module Uses and its Impact Lydia Duncan, Cray Inc. CHIUW 2016 May 27 th, 2016 Safe Harbor Statement This presentation may contain forward-looking statements that are based
More informationMemory Leaks Chapel Team, Cray Inc. Chapel version 1.14 October 6, 2016
Memory Leaks Chapel Team, Cray Inc. Chapel version 1.14 October 6, 2016 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward
More informationProject Caribou; Streaming metrics for Sonexion Craig Flaskerud
Project Caribou; Streaming metrics for Sonexion Craig Flaskerud Legal Disclaimer Information in this document is provided in connection with Cray Inc. products. No license, express or implied, to any intellectual
More informationLocality/Affinity Features COMPUTE STORE ANALYZE
Locality/Affinity Features Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking statements may include statements about
More informationPerformance Measurement and Analysis Tools Installation Guide S
Performance Measurement and Analysis Tools Installation Guide S-2474-63 Contents About Cray Performance Measurement and Analysis Tools...3 Install Performance Measurement and Analysis Tools on Cray Systems...4
More informationCompiler / Tools Chapel Team, Cray Inc. Chapel version 1.17 April 5, 2018
Compiler / Tools Chapel Team, Cray Inc. Chapel version 1.17 April 5, 2018 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward
More informationOpenFOAM Scaling on Cray Supercomputers Dr. Stephen Sachs GOFUN 2017
OpenFOAM Scaling on Cray Supercomputers Dr. Stephen Sachs GOFUN 2017 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking
More informationAn Exploration into Object Storage for Exascale Supercomputers. Raghu Chandrasekar
An Exploration into Object Storage for Exascale Supercomputers Raghu Chandrasekar Agenda Introduction Trends and Challenges Design and Implementation of SAROJA Preliminary evaluations Summary and Conclusion
More informationRedfish APIs on Next Generation Cray Hardware CUG 2018 Steven J. Martin, Cray Inc.
Redfish APIs on Next Generation Cray Hardware Steven J. Martin, Cray Inc. Modernizing Cray Systems Management Use of Redfish APIs on Next Generation Cray Hardware Steven Martin, David Rush, Kevin Hughes,
More informationGrab-Bag Topics / Demo COMPUTE STORE ANALYZE
Grab-Bag Topics / Demo Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking statements may include statements about
More informationReveal Heidi Poxon Sr. Principal Engineer Cray Programming Environment
Reveal Heidi Poxon Sr. Principal Engineer Cray Programming Environment Legal Disclaimer Information in this document is provided in connection with Cray Inc. products. No license, express or implied, to
More informationCompiler Improvements Chapel Team, Cray Inc. Chapel version 1.13 April 7, 2016
Compiler Improvements Chapel Team, Cray Inc. Chapel version 1.13 April 7, 2016 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations.
More informationShifter and Singularity on Blue Waters
Shifter and Singularity on Blue Waters Maxim Belkin June 7, 2018 A simplistic view of a scientific application DATA RESULTS My Application Received an allocation on Blue Waters! DATA RESULTS My Application
More informationLustre Lockahead: Early Experience and Performance using Optimized Locking. Michael Moore
Lustre Lockahead: Early Experience and Performance using Optimized Locking Michael Moore Agenda Purpose Investigate performance of a new Lustre and MPI-IO feature called Lustre Lockahead (LLA) Discuss
More informationAdding Lifetime Checking to Chapel Michael Ferguson, Cray Inc. CHIUW 2018 May 25, 2018
Adding Lifetime Checking to Chapel Michael Ferguson, Cray Inc. CHIUW 2018 May 25, 2018 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations.
More informationReveal. Dr. Stephen Sachs
Reveal Dr. Stephen Sachs Agenda Reveal Overview Loop work estimates Creating program library with CCE Using Reveal to add OpenMP 2 Cray Compiler Optimization Feedback OpenMP Assistance MCDRAM Allocation
More informationData-Centric Locality in Chapel
Data-Centric Locality in Chapel Ben Harshbarger Cray Inc. CHIUW 2015 1 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward
More informationTransferring User Defined Types in
Transferring User Defined Types in OpenACC James Beyer, Ph.D () 1 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking
More informationMPI for Cray XE/XK Systems & Recent Enhancements
MPI for Cray XE/XK Systems & Recent Enhancements Heidi Poxon Technical Lead Programming Environment Cray Inc. Legal Disclaimer Information in this document is provided in connection with Cray Inc. products.
More informationData Parallelism COMPUTE STORE ANALYZE
Data Parallelism Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations. Forward looking statements may include statements about our financial
More informationCaching Puts and Gets in a PGAS Language Runtime
Caching Puts and Gets in a PGAS Language Runtime Michael Ferguson Cray Inc. Daniel Buettner Laboratory for Telecommunication Sciences September 17, 2015 C O M P U T E S T O R E A N A L Y Z E Safe Harbor
More informationCray Performance Tools Enhancements for Next Generation Systems Heidi Poxon
Cray Performance Tools Enhancements for Next Generation Systems Heidi Poxon Agenda Cray Performance Tools Overview Recent Enhancements Support for Cray systems with KNL 2 Cray Performance Analysis Tools
More informationItron Riva Dev Mini Kit Getting Started Guide
Itron Riva Dev Mini Kit Getting Started Guide Table of Contents Introduction... 1 Host Computer... 2 Connect and Power-up ROOT... 2 Connect and Power-up NODE... 8 Summary and Next Steps... 14 Introduction
More informationLustre Networking at Cray. Chris Horn
Lustre Networking at Cray Chris Horn hornc@cray.com Agenda Lustre Networking at Cray LNet Basics Flat vs. Fine-Grained Routing Cost Effectiveness - Bandwidth Matching Connection Reliability Dealing with
More informationIntroduction to Cray Data Virtualization Service S
TM Introduction to Cray Data Virtualization Service S 0005 4002 2008-2011 Cray Inc. All Rights Reserved. This document or parts thereof may not be reproduced in any form unless permitted by contract or
More informationNew Tools and Tool Improvements Chapel Team, Cray Inc. Chapel version 1.16 October 5, 2017
New Tools and Tool Improvements Chapel Team, Cray Inc. Chapel version 1.16 October 5, 2017 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations.
More informationPorting the parallel Nek5000 application to GPU accelerators with OpenMP4.5. Alistair Hart (Cray UK Ltd.)
Porting the parallel Nek5000 application to GPU accelerators with OpenMP4.5 Alistair Hart (Cray UK Ltd.) Safe Harbor Statement This presentation may contain forward-looking statements that are based on
More informationReducing Cluster Compatibility Mode (CCM) Complexity
Reducing Cluster Compatibility Mode (CCM) Complexity Marlys Kohnke Cray Inc. St. Paul, MN USA kohnke@cray.com Abstract Cluster Compatibility Mode (CCM) provides a suitable environment for running out of
More informationSharpen Exercise: Using HPC resources and running parallel applications
Sharpen Exercise: Using HPC resources and running parallel applications Contents 1 Aims 2 2 Introduction 2 3 Instructions 3 3.1 Log into ARCHER frontend nodes and run commands.... 3 3.2 Download and extract
More informationInstallation, Configuration and Performance Tuning of Shifter V16 on Blue Waters
May 24, 2018 Installation, Configuration and Performance Tuning of Shifter V16 on Blue Waters HonWai Leong, Timothy Bouvet, Brett Bode, Jeremy Enos & David King Outline Background and Challenges Installation
More informationHybrid Warm Water Direct Cooling Solution Implementation in CS300-LC
Hybrid Warm Water Direct Cooling Solution Implementation in CS300-LC Roger Smith Mississippi State University Giridhar Chukkapalli Cray, Inc. C O M P U T E S T O R E A N A L Y Z E 1 Safe Harbor Statement
More informationData Management Platform (DMP) Administrator's Guide S 2327 B
R Data Management Platform (DMP) Administrator's Guide S 2327 B 2013 Cray Inc. All Rights Reserved. This document or parts thereof may not be reproduced in any form unless permitted by contract or by written
More informationCLE and How to. Jan Thorbecke
CLE and How to Start Your Application i Jan Thorbecke Scalable Software Architecture t 2 Scalable Software Architecture: Cray Linux Environment (CLE) Specialized ed Linux nodes Microkernel on Compute nodes,
More informationOpportunities for container environments on Cray XC30 with GPU devices
Opportunities for container environments on Cray XC30 with GPU devices Cray User Group 2016, London Sadaf Alam, Lucas Benedicic, T. Schulthess, Miguel Gila May 12, 2016 Agenda Motivation Container technologies,
More informationVectorization of Chapel Code Elliot Ronaghan, Cray Inc. June 13, 2015
Vectorization of Chapel Code Elliot Ronaghan, Cray Inc. CHIUW @PLDI June 13, 2015 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations.
More informationUsing Resource Utilization Reporting to Collect DVS Usage Statistics
Using Resource Utilization Reporting to Collect DVS Usage Statistics Tina Butler National Energy Research Scientific Computing Center Ernest Orlando Lawrence Berkeley National Laboratory Berkeley, CA USA
More informationDebugging on Blue Waters
Debugging on Blue Waters Debugging tools and techniques for Blue Waters are described here with example sessions, output, and pointers to small test codes. For tutorial purposes, this material will work
More informationChapel Hierarchical Locales
Chapel Hierarchical Locales Greg Titus, Chapel Team, Cray Inc. SC14 Emerging Technologies November 18 th, 2014 Safe Harbor Statement This presentation may contain forward-looking statements that are based
More informationXC Series Boot Troubleshooting Guide (CLE 6.0.UP01)
XC Series Boot Troubleshooting Guide (CLE 6.0.UP01) Contents Contents About the XC Series Boot Troubleshooting Guide...5 Introduction to Troubleshooting a Boot of an XC Series System...8 SMW and CLE Hardware
More informationDocker task in HPC Pack
Docker task in HPC Pack We introduced docker task in HPC Pack 2016 Update1. To use this feature, set the environment variable CCP_DOCKER_IMAGE of a task so that it could be run in a docker container on
More informationINSTALLING INSTALLING INSTALLING
Architect Version 6.0 Update 2 Release Notes April 2012 0898600-6.0-2 READ READ READ ME ME ME BEFORE BEFORE BEFORE INSTALLING INSTALLING INSTALLING THIS THIS THIS PRODUCT PRODUCT PRODUCT Disclaimer The
More informationShifter at CSCS Docker Containers for HPC
Shifter at CSCS Docker Containers for HPC HPC Advisory Council Swiss Conference Alberto Madonna, Lucas Benedicic, Felipe A. Cruz, Kean Mariotti - CSCS April 9 th, 2018 Table of Contents 1. Introduction
More informationParallels Virtuozzo Containers 4.6 for Windows
Parallels Parallels Virtuozzo Containers 4.6 for Windows Templates Management Guide Copyright 1999-2010 Parallels Holdings, Ltd. and its affiliates. All rights reserved. Parallels Holdings, Ltd. c/o Parallels
More informationEnhancing scalability of the gyrokinetic code GS2 by using MPI Shared Memory for FFTs
Enhancing scalability of the gyrokinetic code GS2 by using MPI Shared Memory for FFTs Lucian Anton 1, Ferdinand van Wyk 2,4, Edmund Highcock 2, Colin Roach 3, Joseph T. Parker 5 1 Cray UK, 2 University
More informationProductive Programming in Chapel:
Productive Programming in Chapel: A Computation-Driven Introduction Base Language with n-body Michael Ferguson and Lydia Duncan Cray Inc, SC15 November 15th, 2015 C O M P U T E S T O R E A N A L Y Z E
More informationInstalling Cray System Management Workstation (SMW) Software S
TM Installing Cray System Management Workstation (SMW) Software S 2480 6002 2007 2011 Cray Inc. All Rights Reserved. This document or parts thereof may not be reproduced in any form unless permitted by
More informationSharpen Exercise: Using HPC resources and running parallel applications
Sharpen Exercise: Using HPC resources and running parallel applications Andrew Turner, Dominic Sloan-Murphy, David Henty, Adrian Jackson Contents 1 Aims 2 2 Introduction 2 3 Instructions 3 3.1 Log into
More informationSingularity: container formats
Singularity Easy to install and configure Easy to run/use: no daemons no root works with scheduling systems User outside container == user inside container Access to host resources Mount (parts of) filesystems
More informationIntroduction to Linux
Introduction to Linux Mukesh Pund Principal Scientist, NISCAIR, New Delhi, India History In 1969, a team of developers developed a new operating system called Unix which was written using C Linus Torvalds,
More informationDocker A FRAMEWORK FOR DATA INTENSIVE COMPUTING
Docker A FRAMEWORK FOR DATA INTENSIVE COMPUTING Agenda Intro / Prep Environments Day 1: Docker Deep Dive Day 2: Kubernetes Deep Dive Day 3: Advanced Kubernetes: Concepts, Management, Middleware Day 4:
More informationContents. Note: pay attention to where you are. Note: Plaintext version. Note: pay attention to where you are... 1 Note: Plaintext version...
Contents Note: pay attention to where you are........................................... 1 Note: Plaintext version................................................... 1 Hello World of the Bash shell 2 Accessing
More informationSingularity: Containers for High-Performance Computing. Grigory Shamov Nov 21, 2017
Singularity: Containers for High-Performance Computing Grigory Shamov Nov 21, 2017 Outline Software and High Performance Computing: Installation/Maintenance of the HPC Software stack Why containers and
More informationCross platform enablement for the yocto project with containers. ELC 2017 Randy Witt Intel Open Source Technology Center
Cross platform enablement for the yocto project with containers ELC 2017 Randy Witt Intel Open Source Technology Center My personal problems Why d I even do this? THE multiple distro Problem Yocto Project
More informationParallels Plesk Control Panel. Plesk 8.4 for Linux/Unix Firewall Module Administrator's Guide. Revision 1.0
Parallels Plesk Control Panel Plesk 8.4 for Linux/Unix Firewall Module Administrator's Guide Revision 1.0 Copyright Notice ISBN: N/A Parallels 660 SW 39th Street Suite 205 Renton, Washington 98057 USA
More informationInfoblox Kubernetes1.0.0 IPAM Plugin
2h DEPLOYMENT GUIDE Infoblox Kubernetes1.0.0 IPAM Plugin NIOS version 8.X August 2018 2018 Infoblox Inc. All rights reserved. Infoblox Kubernetes 1.0.0 IPAM Deployment Guide August 2018 Page 1 of 18 Overview...
More informationChapel s New Adventures in Data Locality Brad Chamberlain Chapel Team, Cray Inc. August 2, 2017
Chapel s New Adventures in Data Locality Brad Chamberlain Chapel Team, Cray Inc. August 2, 2017 Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current
More informationExercise 1: Connecting to BW using ssh: NOTE: $ = command starts here, =means one space between words/characters.
Exercise 1: Connecting to BW using ssh: NOTE: $ = command starts here, =means one space between words/characters. Before you login to the Blue Waters system, make sure you have the following information
More informationCentre de Calcul de l Institut National de Physique Nucléaire et de Physique des Particules. Singularity overview. Vanessa HAMAR
Centre de Calcul de l Institut National de Physique Nucléaire et de Physique des Particules Singularity overview Vanessa HAMAR Disclaimer } The information in this presentation was compiled from different
More informationPython based Data Science on Cray Platforms Rob Vesse, Alex Heye, Mike Ringenburg - Cray Inc C O M P U T E S T O R E A N A L Y Z E
Python based Data Science on Cray Platforms Rob Vesse, Alex Heye, Mike Ringenburg - Cray Inc Overview Supported Technologies Cray PE Python Support Shifter Urika-XC Anaconda Python Spark Intel BigDL machine
More informationImage Sharpening. Practical Introduction to HPC Exercise. Instructions for Cirrus Tier-2 System
Image Sharpening Practical Introduction to HPC Exercise Instructions for Cirrus Tier-2 System 2 1. Aims The aim of this exercise is to get you used to logging into an HPC resource, using the command line
More informationLinux Kung Fu. Stephen James UBNetDef, Spring 2017
Linux Kung Fu Stephen James UBNetDef, Spring 2017 Introduction What is Linux? What is the difference between a client and a server? What is Linux? Linux generally refers to a group of Unix-like free and
More informationCENG 334 Computer Networks. Laboratory I Linux Tutorial
CENG 334 Computer Networks Laboratory I Linux Tutorial Contents 1. Logging In and Starting Session 2. Using Commands 1. Basic Commands 2. Working With Files and Directories 3. Permission Bits 3. Introduction
More informationLogging in to the CRAY
Logging in to the CRAY 1. Open Terminal Cray Hostname: cray2.colostate.edu Cray IP address: 129.82.103.183 On a Mac 2. type ssh username@cray2.colostate.edu where username is your account name 3. enter
More informationBatch environment PBS (Running applications on the Cray XC30) 1/18/2016
Batch environment PBS (Running applications on the Cray XC30) 1/18/2016 1 Running on compute nodes By default, users do not log in and run applications on the compute nodes directly. Instead they launch
More informationMethod of Procedure to Upgrade RMS OS to Red Hat Enterprise Linux 6.7
First Published: November 20, 2015 Contents Scope of MOP... 4 Release Components... 4 Pre Requisites... 4 Assumptions... 4 Process Information... 5 Upgrade Timing... 5 Requirements... 5 Pre Maintenance...
More informationEMC NetWorker Module for Microsoft for Windows Bare Metal Recovery Solution
EMC NetWorker Module for Microsoft for Windows Bare Metal Recovery Solution Release 3.0 SP1 User Guide P/N 302-000-098 REV 02 Copyright 2007-2014 EMC Corporation. All rights reserved. Published in the
More informationIntroduction. What is Linux? What is the difference between a client and a server?
Linux Kung Fu Introduction What is Linux? What is the difference between a client and a server? What is Linux? Linux generally refers to a group of Unix-like free and open-source operating system distributions
More informationLAB #7 Linux Tutorial
Gathering information: LAB #7 Linux Tutorial Find the password file on a Linux box Scenario You have access to a Linux computer. You must find the password file on the computer. Objective Get a listing
More informationINSTALLING INSTALLING INSTALLING
Architect Version 4.0 Release Notes July 2010 0898600-4.0 READ READ READ ME ME ME BEFORE BEFORE BEFORE INSTALLING INSTALLING INSTALLING THIS THIS THIS PRODUCT PRODUCT PRODUCT Disclaimer The information
More informationOverview LEARN. History of Linux Linux Architecture Linux File System Linux Access Linux Commands File Permission Editors Conclusion and Questions
Lanka Education and Research Network Linux Architecture, Linux File System, Linux Basic Commands 28 th November 2016 Dilum Samarasinhe () Overview History of Linux Linux Architecture Linux File System
More informationMore Raspian. An editor Configuration files Shell scripts Shell variables System admin
More Raspian An editor Configuration files Shell scripts Shell variables System admin Nano, a simple editor Nano does not require the mouse. You must use your keyboard to move around the file and make
More informationRunning Jobs on Blue Waters. Greg Bauer
Running Jobs on Blue Waters Greg Bauer Policies and Practices Placement Checkpointing Monitoring a job Getting a nodelist Viewing the torus 2 Resource and Job Scheduling Policies Runtime limits expected
More informationAdobe Marketing Cloud Using FTP and sftp with the Adobe Marketing Cloud
Adobe Marketing Using FTP and sftp with the Adobe Marketing Contents Using FTP and sftp with the Adobe Marketing...3 Setting Up FTP Accounts Hosted by Adobe...3 Classifications...3 Data Sources...4 Data
More informationCS 143A. Principles of Operating Systems. Instructor : Prof. Anton Burtsev
CS 143A Principles of Operating Systems Instructor : Prof. Anton Burtsev (aburtsev@uci.edu) Assistants : Junjie Shen junjies1@uci.edu Vikram Narayanan narayav1@uci.edu Biswadip Maity (Deep) Email : maityb@uci.edu
More informationNotes for Installing RedHawk Linux 7.0 with Red Hat Enterprise Linux 7.0. Installation Notes. March 22 nd, 2015
Notes for Installing RedHawk Linux 7.0 with Red Hat Enterprise Linux 7.0 Installation Notes March 22 nd, 2015 This page intentionally left blank 1. Introduction RedHawk Linux is supplied with CentOS Linux
More informationLab Authentication, Authorization, and Accounting
Objectives Given a scenario, select the appropriate authentication, authorization, or access control Install and configure security controls when performing account management, based on best practices
More informationLinux Essentials. Programming and Data Structures Lab M Tech CS First Year, First Semester
Linux Essentials Programming and Data Structures Lab M Tech CS First Year, First Semester Adapted from PDS Lab 2014 and 2015 Login, Logout, Password $ ssh mtc16xx@192.168.---.--- $ ssh X mtc16xx@192.168.---.---
More informationRed Hat Quay 2.9 Deploy Red Hat Quay - Basic
Red Hat Quay 2.9 Deploy Red Hat Quay - Basic Deploy Red Hat Quay Last Updated: 2018-09-14 Red Hat Quay 2.9 Deploy Red Hat Quay - Basic Deploy Red Hat Quay Legal Notice Copyright 2018 Red Hat, Inc. The
More informationINSTALLING INSTALLING INSTALLING
Architect Version 2.0 Release Notes July 2009 0898600-2.0 READ READ READ ME ME ME BEFORE BEFORE BEFORE INSTALLING INSTALLING INSTALLING THIS THIS THIS PRODUCT PRODUCT PRODUCT Disclaimer The information
More informationChapel: Productive Parallel Programming from the Pacific Northwest
Chapel: Productive Parallel Programming from the Pacific Northwest Brad Chamberlain, Cray Inc. / UW CS&E Pacific Northwest Prog. Languages and Software Eng. Meeting March 15 th, 2016 Safe Harbor Statement
More informationLinux Systems Administration Getting Started with Linux
Linux Systems Administration Getting Started with Linux Network Startup Resource Center www.nsrc.org These materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International
More informationTIBCO Kabira Adapter Factory for SNMP Installation. Software Release December 2017
TIBCO Kabira Adapter Factory for SNMP Installation Software Release 5.9.5 December 2017 Important Information SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCH EMBEDDED OR BUNDLED
More informationCS Fundamentals of Programming II Fall Very Basic UNIX
CS 215 - Fundamentals of Programming II Fall 2012 - Very Basic UNIX This handout very briefly describes how to use Unix and how to use the Linux server and client machines in the CS (Project) Lab (KC-265)
More informationIntegration Service. Admin Console User Guide. On-Premises
Kony MobileFabric TM Integration Service Admin Console User Guide On-Premises Release 7.3 Document Relevance and Accuracy This document is considered relevant to the Release stated on this title page and
More informationGNU/Linux 101. Casey McLaughlin. Research Computing Center Spring Workshop Series 2018
GNU/Linux 101 Casey McLaughlin Research Computing Center Spring Workshop Series 2018 rccworkshop IC;3df4mu bash-2.1~# man workshop Linux101 RCC Workshop L101 OBJECTIVES - Operating system concepts - Linux
More informationZenoss Resource Manager Upgrade Guide
Zenoss Resource Manager Upgrade Guide Release 5.3.2 Zenoss, Inc. www.zenoss.com Zenoss Resource Manager Upgrade Guide Copyright 2017 Zenoss, Inc. All rights reserved. Zenoss, Own IT, and the Zenoss logo
More informationSiemens PLM Software. HEEDS MDO Setting up a Windows-to- Linux Compute Resource.
Siemens PLM Software HEEDS MDO 2018.04 Setting up a Windows-to- Linux Compute Resource www.redcedartech.com. Contents Introduction 1 On Remote Machine B 2 Installing the SSH Server 2 Configuring the SSH
More informationINSTALLING INSTALLING INSTALLING
Architect Version 7.0 Release Notes May 2015 0898600-7.0 READ READ READ ME ME ME BEFORE BEFORE BEFORE INSTALLING INSTALLING INSTALLING THIS THIS THIS PRODUCT PRODUCT PRODUCT Disclaimer The information
More information