High Performance Compu2ng Using Sapelo Cluster

Transcription

1 High Performance Compu2ng Using Sapelo Cluster Georgia Advanced Compu2ng Resource Center EITS/UGA Zhuofei Hou, Training Advisor 1

2 Outline GACRC What is High Performance Compu2ng (HPC) Sapelo Cluster Diagram How to Log on How to Transfer Files How to Use SoSware How to Submit a Job How to Find Support and Important Rules 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

3 GACRC Ø We are a high-performance compu2ng (HPC) center at UGA Ø We provide to the UGA research and educa2on community an advanced compu2ng environment: HPC compu2ng and networking infrastructure located at the Boyd Data Center Comprehensive collec2on of scien2fic, engineering and business applica2ons Consul2ng and training services h]p://gacrc.uga.edu/about/ 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

4 What is High Performance Compu2ng Task Mountain 10 years later 1 worker 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

5 What is High Performance Compu2ng JOB DONE 1 month later Task Mountain 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

6 What is High Performance Compu2ng Serial Job with 1 worker ü Only 1 instruc2on executed at any 2me point on a single processor ü 6 instruc2ons executed simultaneously Parallel Job with 6 workers on 6 processors ü Shared-memory threaded job or MPI job (Message Passing Interface) 6

7 What is a Cluster? When you work on cluster, 3 roles are working: Ø Ø Ø User: to submit a job Queueing System: to dispatch a job to run on cluster Cluster: to run a job Submit a job Dispatch a job 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

8 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

9 How to log on Mac/Linux 1. Open Terminal u2lity 2. Type command line: ssh 3. You will be prompted for your UGA MyID password (when you type in the password, the prompt blinks and does not move) 9

10 How to log on Windows 1 Download to install: h]p://eits.uga.edu/hardware_and_sosware/sosware/ You will be prompted for your UGA MyID password aser step 4 10

11 Once You Log on 1. Your working environment is Linux 2. You are on Login Node 3. You are in your home directory: /home/ugamyid/, for example, /home/zhuofei/ 11

12 How to Transfer Files Mac/Linux scp scp [Source] [Target] Transfer Node Local Computer è Sapelo; Type command in Terminal from your local computer scp file scp r folder zhuofei@xfer.gacrc.uga.edu:/lustre1/zhuofei/ Sapelo è Local Computer; Type command in Terminal from your local computer scp zhuofei@xfer.gacrc.uga.edu:/lustre1/zhuofei/file./ scp r zhuofei@xfer.gacrc.uga.edu:/lustre1/zhuofei/folder./ 12

13 How to Transfer Files - Windows /lustre1/zhuofei Use Transfer Node!

14 Next 1. Go to Work Node by typing command qlogin 2. Change to Work Drive by typing command cd /lustre1/ugamyid/ 3. Now you are in your working space and ready to Ø Check sosware informa2on Ø Create job submission script Ø Submit job 14

15 How to Use SoPware On Sapelo ~300 installed sosware for you to run Three module commands to: Ø Ø Ø Have a look on all sosware : module avail Choose a sosware to run : module load softwarename Have a look on loaded sosware : module list 15

16 How to Submit a Job You need to create a script to make a compute node to know: Ø Your working directory Ø The sosware you want to run Ø Any Linux commands you want the node to run Wiki Links: h]ps://wiki.gacrc.uga.edu/wiki/running_jobs_on_sapelo h]ps://wiki.gacrc.uga.edu/wiki/sosware h]ps://wiki.gacrc.uga.edu/wiki/command_list 16

17 A Common Example Running Samtools Samtools is a common bioinforma2cs u2lity h]ps://wiki.gacrc.uga.edu/wiki/samtools-sapelo Script name: sub.sh Working directory: /lustre1/zhuofei/ #PBS -S /bin/bash # Linux default shell (bash) #PBS -N testsamtools # define job name #PBS -q batch # use Sapelo batch queue #PBS -l nodes=1:ppn=1:amd # request 1 CPU from 1 AMD node #PBS -l mem=100gb # request 100GB RAM memory #PBS -l wall2me=48:00:00 # request 48 hours running 2me cd $PBS_O_WORKDIR # specify and enter (cd) a working directory module load samtools/1.3.1 # load samtools 2me samtools <command> [op2ons] # run samtools 17

18 A Common Example Submit Job Submit sub.sh to batch queue: Type command on command line from your working directory: $ qsub sub.sh submit a job defined in sub.sh Note: Once a job is submi]ed, your can logout (exit command) from the cluster and log back in later to check status of job and to retrieve the results. 18

19 Check Job Status Type command on command line: qstat zhuofei]$ qstat Job ID Name User Time Use S Queue pbs testjob1 zhuofei 900:58:0 C batch pbs testjob2 zhuofei 04:00:03 R batch pbs testjob3 zhuofei 0 Q batch R : Job is running C : Job is completed (canceled/crashed) and is no longer running; stays on list for 1 hour Q : Job is pending, wai2ng for resources to become available 19

20 Cancel a Job Type command on command line: qdel JobID [zhuofei@n78 zhuofei]$ qdel [zhuofei@n78 zhuofei]$ qstat Job ID Name User Time Use S Queue pbs testjob1 zhuofei 900:58:0 C batch pbs testjob2 zhuofei 04:00:03 R batch pbs testjob3 zhuofei 0 C batch Job is canceled and status is changed to C 20

21 Check Job Memory Usage Type command on command line: qstat -f JobID MPIs]$ qstat -f Job Id: pbs.scm Job_Name = testjob Job_Owner = zhuofei@uga-2f0f976.scm. resources_used.vmem = kb resources_used.walltime = 07:01:36 job_state = C. Resource_List.mem = 5gb Resource_List.walltime = 10:00:00 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

22 Sapelo Workflow 1. Linux/Mac user: ssh Windows user: 6. Linux/Mac user: scp file Windows user: Login Interac2ve 2. qlogin 3. cd /lustre1/username 7. nano./sub.sh 4. mkdir./workdir 5. cd./workdir lustre1 #PBS -S /bin/bash #PBS -q batch #PBS -N testblast #PBS -l nodes=1:ppn=1:amd #PBS -l mem=20gb #PBS -l wall2me=48:00:00 cd $PBS_O_WORKDIR module load ncbiblast+/ blastn [op2ons] > outpuwile 8. $ qsub sub.sh Note: -r op2on of scp command will recursively copy a directory 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

23 Sapelo Workflow 1. Log on to Sapelo Login node: ssh 2. From Sapelo Login node, transfer to Interac2ve node by issuing the command: qlogin 3. From Interac2ve node: Change directory to global scratch: cd /lustre1/username 4. Create a working subdirectory on global scratch: mkdir./workdir 5. Change directory to workdir: cd./workdir 6. Transfer data to workdir using scp or SSH Client File Transfer (with tar or gzip) 7. Make a Sapelo job submission script: nano./sub.sh 8. Submit job: qsub./sub.sh Useful PBS headers: #PBS l mem=200gb : use 256GB high-ram AMD nodes #PBS -l nodes=1:ppn=24:amd : request 24 cores for 24 threads, max 48! 2/12/17 GACRC SAPELO CLUSTER CLASS TRAINING

24 How to Find Support GACRC Support: h=p://help.gacrc.uga.edu/ Ø Job Troubleshoo2ng: Tell us details of your problem, please also include: ü ü ü ü UserID and JobID Working directory and script name Any command you used Brief of your problem Note: It s USER s responsibility to make sure the correctness of data being used by jobs! 24

25 Important Rules Ø No Sharing User Account! è Use your own user account Ø No Job Running Directly on Login node! è Submit job to batch queue Ø No Data Transferring into/out of Cluster Using Login node! è Use Transfer node 25

26 Useful Links GACRC Wiki: h]ps://wiki.gacrc.uga.edu/wiki/main_page GACRC SoSware: h]ps://wiki.gacrc.uga.edu/wiki/sosware GACRC Support: h]p://gacrc.uga.edu/help/ GACRC Training: h]ps://wiki.gacrc.uga.edu/wiki/training Thank You! Telephone Support EITS HELPDESK: Georgia Advanced Compu2ng Resource Center 4098C Stegeman Coliseum University of Georgia MONDAY THURSDAY: 8AM 10PM FRIDAY: 8AM 6PM SATURDAY SUNDAY: 1PM 7PM 26