Introduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU

Transcription

1 Introduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU

2 What is Joker? NMSU s supercomputer. 238 core computer cluster. Intel E-5 Xeon CPUs and Nvidia K-40 GPUs. InfiniBand innerconnect. Linux (CentOS-7) operating system. Slurm workload manager. A head node and compute nodes.

3 Head Node -vs- Compute Node Logically, a Linux cluster looks something like User Network Head Node Compute Nodes

4 Head Node Login to cluster is usually login to head node. Launch point for jobs running on the cluster. Contains tools (e.g. scheduler) necessary to submit programs to the cluster. Basically the middle man between the cluster and the outside network. Not the place to run computational programs.

5 Compute Node Used for computational tasks. Generally not directly accessed by user. May be accessed if needed. Interactive and queueing system accessible with slurm.

6 Joker Account Do you have a Joker account? Why do you need an account? Where to register for an account?

7 Accessing joker. from within the NMSU domain using a terminal emulator. May be accessed by any one of the many SSH terminal programs (such as PuTTY ). Download/run PuTTY (Google putty ssh for download URL). Enter joker. in Hostname window. Click open.

8

9 Enter your joker account username and password when prompted. You will then be in the terminal window ready to go.

10 Accessing Joker from outside NMSU s domain with a terminal emulator. - Joker is not exposed to outside connections. -You must first connect using NMSU's VPN service. -The VPN client may be found at VPN.NMSU.EDU. -Install and run the client.

11 Enter your NMSU Username and Password. Once connected, you may then connect with a terminal emulator (PuTTY).

12 Upload and download files to and from Linux using sftp. -Download and install a sftp client (let s use WinSCP). -Google winscp, go to the site download, installation package. -After install, run the app. and enter the login information. Note: If outside the NMSU domain, you will need to connect with the VPN client before the sftp app.

13 WinSCP sftp Window -Local drive on left. -Linux on right. -Windows Explorer functionality (drag and drop, right click options, )

14 Module System Modules are not applications, they simply add the application s environmental variables to your environment. Modules are loaded and unloaded. Allow path definitions to shell environment. Default compilers, applications, and libraries may be set by Module commands.

15 Module System module avail module load <module name><module name2> module list module unload <module name1><module name2> module clear module purge

16 Module System Allows path definitions to shell environment. Default compilers, applications, and libraries may be set by Module commands. Modules are not applications, they simply add the application s environmental variables to your environment.

17 Installing R Worksheet A) Install and run R locally. B) Load and run R from module.

18 The Slurm Scheduler Job scheduler implemented on Joker. Basic commands - slurm cheat sheet sinfo squeue srun sbatch scancel Tutorial

19 Submit a job to Slurm File 1 is the program to run. File 2 is the Slurm script file. Submit the job with sbatch <File 2>.

20 Submit a Job to the Scheduler (slurm) Create 2 files using your favorite editor. File 1 #!/bin/bash echo Hello World! Save it. Change to an executable (chmod +x <File 1>) Run it to verify it works (./<File 1>).

21 Submitting to the scheduler Slurm (continued) File 2 #!/bin/bash #SBATCH -n 96 srun./<file 1> srun sleep 20 echo All done sleeping! Save it.

22 Submitting to the scheduler Slurm (continued) sinfo (get info about resources) squeue (have a look at the q) sbatch <File 2> (submit our job) squeue (see where our job is in the q)

23 More information about us and NMSU s High Performance Computer. hpc. cia.