Using Compute Canada. Masao Fujinaga Information Services and Technology University of Alberta

Transcription

1 Using Compute Canada Masao Fujinaga Information Services and Technology University of Alberta

2 Introduction to cedar batch system jobs are queued priority depends on allocation and past usage

3 Cedar Nodes "Base" compute nodes: 576 nodes 128 GB of memory, 32 cores "Large" compute nodes: 128 nodes 256 GB of memory, 32 cores "Bigmem500" 24 nodes 0.5 TB (512 GB) of memory, 32 cores "Bigmem1500" nodes 24 nodes 1.5 TB of memory, 32 cores "GPU base" nodes: 114 nodes 128 GB of memory, 24 cores, 4 GPUs "GPU large" nodes. 32 nodes 256 GB of memory, 24 cores, 4 GPUs "Bigmem3000" nodes 4 nodes 3 TB of memory, 32 cores

4 Software for connecting to Compute Canada Windows MobaXterm Mac XQuartz

5 Install MobaXterm (Windows)

6 Start MobaXterm

7 Start local terminal

8 Connect with MobaXterm

9 Transfer files with MobaXterm

10 Connect with Terminal (mac or linux)

11 Transfer files mac/linux scp myfile FileZilla windows MobaXterm WinSCP

12 Connect to cedar Logging in ssh -Y Interactive use compiling testing, debugging use less than 1 hour and 2 cores longer interactive use: salloc

13 Untar workshop files home directory cp ~masao/workshopsept2017.tar.gz. tar -xf workshopsept2017.tar.gz cd workshopsept2017 current directory

14 Setting up the environment Modules module avail Organized according to compiler, MPI, cuda.

15

16 Finding modules Module spider module spider blast+ You will need to load all module(s) on any one of the lines below before the "blast+/2.6.0" module is available to load. nixpkgs/16.09 gcc/5.4.0 module load gcc blast+

17 Software Compute Canada website

18

19 Other software Install own python27-scipy-stack/2017a, python35-scipy-stack/2017a python27-mpi4py/2.0.0, python35-mpi4py/ r-bundle-bioconductor/3.4 (gcc) Request installation

20 Compiling compile icc -o hello hello.c ifort -o hello hello.f90 run./hello

21 Man pages man icc space bar to get a page /mysearch to search for mysearch q to quit man -k mytopic get pages related to mytopic

22 Editing graphical editors nedit, gedit non-graphical editors vi, emacs editing through Winscp

23 Batch job #!/bin/bash #SBATCH --reservation=masao_10 #SBATCH --account=def-myaccount #SBATCH --ntasks=1 #SBATCH --mem-per-cpu=2000mb #SBATCH --time=00:05:00 #SBATCH --mail-type=all #SBATCH

24 Job submission sbatch sbatch job-cedar.slm Submitted batch job squeue squeue -u masao JOBID USER ACCOUNT masao def-masao_cp NAME job-cedar.slm ST START_TIME PD N/A TIME_LEFT NODES CPUS GRES MIN_MEM NODELIST (REASON) 5: (null) 2000M (Priority) NAME job-cedar.slm ST START_TIME R T08:31 TIME_LEFT NODES CPUS GRES MIN_MEM NODELIST (REASON) 4: (null) 2000M cdr310 (None) running JOBID USER ACCOUNT masao def-masao_cp

25 Job output slurm-<jobnumber>.out Common errors out of walltime out of memory batch script in DOS format No end-of-line character at the end

26 pi program compile icc -o pi pi.c interactive run./pi input from terminal

27 pi job batch run create pi.in./pi < pi.in > pi.out no input file./pi << EOF-pi > pi.out EOF-pi

28 MPI compile mpicc -o hello-mpi hello-mpi.c mpif90 -o hello-mpi hello-mpi.f90 Run ulimit -l mpiexec -np 2./hello-mpi

29 Interactive batch job Submitting an interactive batch job salloc --time=6:0:0 --ntasks=2 --account=def-myaccount -reservation=masao_10 no batch script may have to wait a long time depending on how busy the system is

30 Benchmarking Timing reproducible, without other users time command walltime

31 Parallel benchmark Time the real time it takes to run the program using various number of processors For any parallel program, it is important to know the parallel efficiency of the program

32 benchmark exercise compile sample1.f capital oh zero mpif90 -o sample1 sample1.f90 -fixed -O0 submit script1.sbatch. This will run the program five times, changing -n to 1, 2, 4, 8, and 12 look at the real time ( grep real out.time ) plot using Replace t1,t2,t4,t8,t16 with your values plot time plot{{1,t1},{2,t2},{4,t4},{8,t8},{16,t16}} plot speedup plot{{1,1},{2,t1/t2},{4,t1/t4},{8,t1/t8},{16,t1/t16}}

33 benchmark results

34 Optimizing compiler optimization default: -O2 higher optimizations: -O3, -axcore-avx2 Exercise icc -o sample2 sample2.c -O1 time./sample2 time runs of sample2 after having compiled with -O0 -O1 -O2 -O3 -O3 -axcore-avx2 -O3 -axcore-avx2 -fp-model precise Note the checksum

35 Math Library Intel Math Kernel Library (MKL) -mkl=sequential

36 Filesystems home directory Project Default project Projects Links to other project directories scratch Use for running jobs quotas diskusage_report

37 Moving data in Compute Canada Globus web-based fault tolerant can transfer to your desktop

38 Upcoming ARC Training Sessions October 3 10:00 AM 11:00 AM PDT October 5 11:00 AM 1:00 PM PDT October :00-3:00 PM MDT (each day) November 1 11:00 AM 1:00 PM PDT November 21 11:00 AM 1:00 PM PST Data Visualization on Compute Canada's Supercomputers Intro to Parallel Programming for Shared Memory Machines Scheduling & Job Management: How to Get the Most from a Cluster Introduction to Classical Molecular Dynamics Simulations Exploring Containerization with Singularity

39 Getting Help use the search specify user ID which system job ID error messages script location