ICS-ACI System Basics Adam W. Lavely, Ph.D. Fall 2017 Slides available: goo.gl/ss9itf awl5173 ICS@PSU 1
Contents 1 Overview 2 HPC Overview 3 Getting Started on ACI 4 Moving On awl5173 ICS@PSU 2
Contents 1 Overview 2 HPC Overview 3 Getting Started on ACI 4 Moving On awl5173 ICS@PSU 3
Goals for Today Goal 1: Learn about HPC in general Goal 2: Learn about ICS-ACI Goal 3: Learn how to get more information Notes: Expect to be overwhelmed Experience is the best teacher Learn terminology and basic concepts Spend time playing/learning Replicate an analysis awl5173 ICS@PSU 4
Contents 1 Overview 2 HPC Overview 3 Getting Started on ACI 4 Moving On awl5173 ICS@PSU 5
High Performance Computing Compute that is large : Large numbers of processors Meteorology, nuclear physics, fluid dynamics Large numbers of runs Genomics, astronomy, structural design Large memory requirements Financial predictions, molecular analysis Large storage requirements Big data analysis, machine learning Large runtimes Optimization, Poor coding awl5173 ICS@PSU 6
High Performance Computing - 2 HPC does not improve every single computational process! Your Goal: Do research. Your Task: Find tools for the best/fastest research Options: Build/find a tool specific to your problem Find a multi-purpose (very capable) tool awl5173 ICS@PSU 7
HPC @ PSU: Clusters Internal Resources: ICS-ACI is a very capable tool ACI-B: Batch (high/std/basic memory) ACI-I: Interactive Being Rolled Out: Gateways, GPUs ACI Partitions (CyberLAMP) Hosted resources (LIGO) External Resources: ICS is more than ACI NSF XSEDE Blue Waters through GLCPC DoE INCITE awl5173 ICS@PSU 8
HPC @ PSU: ICS People ICS Engagement Team: Adam Lavely; PhD in Aerospace Engering Christopher Blanton; PhD in Chemistry Charles Pavloski; PhD in Meteorology ICS iask Staff & GAs: From 7 departments across 4 colleges ICS Staff & GAs: Help organizing large proposals Other (?) If nothing else, ICS can point you in the right direction. awl5173 ICS@PSU 9
Contents 1 Overview 2 HPC Overview 3 Getting Started on ACI 4 Moving On awl5173 ICS@PSU 10
Getting an Account Sign up for an account: https://accounts.aci.ics.psu.edu Non-Faculty Notes: List a faculty sponsor by their PSU ID Implicit approval granted after 2 business days Sponsor can grant explicit approval for quicker access Faculty Notes: Email iask@ics.psu.edu to grant explicit approval awl5173 ICS@PSU 11
System Overview ACI-I: Interactive System aci-i.aci.ics.psu.edu Process on the shared interactive nodes Available to anybody for use ACI-B: Batch System aci-b.aci.ics.psu.edu Log in to a head node and submit jobs to compute nodes Groups can purchase allocations or use open queue Both systems have a common file-system awl5173 ICS@PSU 12
ACI-I Log on: Either ssh or EoD Interactive Nodes: Placed on node based on load Run your processes there Named aci-ic-* Limits on shared resources: 4 processors 12 CPU hours per process 48 GB resident memory Use this for: Debugging Visualization Interaction awl5173 ICS@PSU 13
ACI-B Log on: You must ssh (no EoD) Log-in Nodes: Placed based on current load Used for submitting jobs, not running processes Named aci-lgn-* Compute Nodes: Jobs wait until requested resources are available Run on dedicated (not shared) resources You get what you ask for in your submission script Use this for: Production runs High memory/proc/time usage awl5173 ICS@PSU 14
File-systems Home: \storage\home\userid 10 GB, backed up Cannot be shared Store important things here Work: \gpfs\work\userid 128 GB, backed up Can be shared Scratch: \gpfs\scratch\userid 1 million file limit (no size limit), not backed up Files older than 30 days are purged Run things from here awl5173 ICS@PSU 15
File-systems - 2 Notes: Work and scratch should have links in your home directory Groups can purchase group storage Similar to Work space, but in 5 TB increments Groups can purchase archival storage Back-up data for long term Data is available on datamgr.aci.ics.psu.edu for transfers Don t leave data on scratch! awl5173 ICS@PSU 16
Connecting to Clusters Ways to Connect: goo.gl/wdkqdi ssh in terminal program (mac/linux) PuTTY or other emulator (any OS, Windows) Exceed ondemand (any OS, to ACI-I only) Connect with ssh: < userid > @ < host > SSH to ACI-B $ ssh awl5173@aci-b.aci.ics.psu.edu SSH to ACI-I $ ssh awl5173@aci-i.aci.ics.psu.edu Notes: a) Use your username; b) the password doesn t show * awl5173 ICS@PSU 17
Exceed ondemand Exceed ondemand is a GUI program for accessing ACI-I: goo.gl/z5jqf2 Connecting with EoD: Hostname: aci-i.aci.ics.psu.edu PSU User ID and password (with ICS account) Two-factor authentication Xconfig: Desktop_Mode_1280x1024.cfg Xstart: Gnome_Desktop.xs Notes: You can have one session open at a time You can come back to a session (quit, don t log out) You have the GUI interface, but still need to use the command line awl5173 ICS@PSU 18
EoD: Finding the Terminal Applications System Tools Terminal awl5173 ICS@PSU 19
Getting Started: Terminal Terminal: All functionality via the command line No difference between ssh and EoD terminal Things to know: Google is your best friend Command-line framework man banana awl5173 ICS@PSU 20
Command-line framework Command Prompt: Created by the environment with some information Command: The command to run Flags: The flags (options) to alter the typical command Output: What the computer responds with (possibly nothing) 2nd Command Prompt: Shows back up when the process has finished Find total and available allocation hours $ mam-list-funds -h awl5173 ICS@PSU 21
man Most commands come with a manual. View the manual for the du command $ man du Use q to exit, arrows to move up and down. Lots of info: skimming man pages is a good skill to develop. awl5173 ICS@PSU 22
banana Most commands give a list of options if an improper flag is used. Find total and available allocation hours $ mam-list-funds --banana Output (on ACI-B): Unknown option: banana Usage: mam-list-funds [[-f] *fund_id*] [-A -I] [-n *fund_name*] [-X, --extension *property*=*value*]... [-u *user_name*] [-g *group_name*] [-a *account_name*] [-o *organization_name*] [-c *class_name*] [-m *machine_name*] [--filter *filter_name*=*filter_value*]... [--filter-type ExactMatch Exclusive NonExclusive] [--full] [--show *attribute_name*,...] [--long] [--wide] [--format csv raw standard] [--hours] [--debug] [--site *site_name*] [--help] [--man] [--quiet] [--version] [--about] awl5173 ICS@PSU 23
Basic Commands List the contents of the current directory $ ls Print the name of the current directory $ pwd Change directory $ cd scratch Copy a file $ cp logfile logfile_13sept2017 awl5173 ICS@PSU 24
Helpful Command Line Hints Other commands you may find useful: history, mv, rm, mkdir, find, grep, awk, id, du, clear, env, ssh, more Special Characters: is your home directory. means here.. means up one directory * is the wildcard: * for all files or *.png for all png files is pipe (send the output to another command) > means write command output to a file ( ls > log.ls ) awl5173 ICS@PSU 25
Modules Modules are used to allow multiple versions of the same software on the system software stack. Load the module of the software you wish to use. Show currently available modules $ module avail Search for a module $ module spider vasp Load a module $ module load ansys/18.1 awl5173 ICS@PSU 26
Modules - 2 ICS uses module families: Modules built with a compiler module are only available with that compiler loaded Same for parallelization modules See modules in the GCC family $ module avail $ module load gcc/5.3.1 $ module avail Other module commands you may find useful: module list, module purge, module show <modulename> awl5173 ICS@PSU 27
Transferring Data Methods to transfer files to/from ACI: Command line: scp, rsync, sftp, etc Use: datamgr.aci.ics.psu.edu Local programs: WinSCP, Filezilla, etc Online methods: Box, dropbox Use EoD s Firefox for online interface No syncing capability Data-specific programs: Globus, Aspera Copy a file to ACI-B $ scp lfile awl5173@aci-b.aci.ics.psu.edu:~/work/. awl5173 ICS@PSU 28
Connecting with WinSCP/Filezilla: File protocol: SCP Host name: datamgr.aci.ics.psu.edu PSU user name and password Port 22 Be ready for 2FA Drag and drop between your computer and cluster awl5173 ICS@PSU 29
Submitting a Job on ACI-B Submission script: PBS directives Give requested resources to scheduler Starts with #PBS Only at the beginning Commands Same as command line operation Have access to PBS variables Load modules, go to correct location, run code Submitting a Job $ qsub subscript.pbs awl5173 ICS@PSU 30
Sample Matlab Submission Script #!/bin/bash #PBS -l nodes=1:ppn=1 #PBS -l walltime=5:00 #PBS -A open # Get started echo "Job started on hostname at date " # Load in matlab module purge module load matlab/r2016a # Go to the correct place cd $PBS_O_WORKDIR # Run the job itself matlab-bin -nodisplay -nosplash < runthis.m > log.matlabrun # Finish up echo "Job Ended at date " awl5173 ICS@PSU 31
Contents 1 Overview 2 HPC Overview 3 Getting Started on ACI 4 Moving On awl5173 ICS@PSU 32
Getting Help ICS Documents: ics.psu.edu Onboarding documentation Expanding support iask: https://iask.aci.ics.psu.edu Email: iask@ics.psu.edu Phone: 54275 Google: Linux commands & command line tutorials Other Batch systems (TACC, OSC) awl5173 ICS@PSU 33
ICS Training Series Seminar 2: October 20, 27 Submitting jobs Run the same matlab script using the GUI, in batch mode and as a job Compiling simple codes Parallel hello world Allocation usage Basic mam usage for users in allocations Intro to parallelization Distributed vs. shared memory Data moving Globus, rsync Seminars 1, 2 & 3: Jan/Feb 2018 awl5173 ICS@PSU 34
Seminar 2: Requirements A laptop with Exceed ondemand installed Seminar will be interactive An account on ACI Basic terminal skills Be comfortable with basic terminal commands Will be learning mam-*, qsub, qstat Ability to use a terminal text editor vi, emacs, other choice awl5173 ICS@PSU 35
Fin Thank you for your time! iask: https://iask.aci.ics.psu.edu Email: iask@ics.psu.edu Phone: 54275 awl5173 ICS@PSU 36