Cloud Computing Research Cloud: NeCTAR Commercial Cloud: Amazon AWS, Microsoft Azure, etc. Seed money for exploration of new cloud technologies

Transcription

1 High Performance Computing (HPC) As a service: NCI Raijin Katana local HPC cluster Cloud Computing Research Cloud: NeCTAR Commercial Cloud: Amazon AWS, Microsoft Azure, etc. Seed money for exploration of new cloud technologies Research Data Help with Data Management Assistance with data moves, storage, planning ResTech@unsw.edu.au

2 Training 40+ courses run on campus, including: Introduction into Linux Getting started with HPC Parallel Programming Introduction to programming with Julia Programming courses for Python, R, Matlab, Excel Consulting Help with developing / optimizing code Help getting started on Raijin and Katana ResTech@unsw.edu.au

3 HPC Systems Raijin Katana Owner NCI (ANU) RTS (UNSW) Size Almost 4500 nodes Almost 200 nodes Access UNSW, Intersect and NCMAS. Project Storage 37 PB 1 PB Free for small users. Buy-in for groups. Node Interconnects Up to 100Gb/s Up to 10Gb/s Max Walltime 48 hours 200 hours Best for Large parallel jobs, complicated models. Bioscience and genomics.

4 NCI Resources User and project management inc. software at Mancini / MyNCI ( Resources (compute and storage) are provided at the project level. NCMAS Open: 3rd Sept, Close: 19th Oct, Announced: 11th Dec UNSW Scheme Request at any time but annual application in November, Announced: 19th Dec 2Gb home, 72Gb short per project (can be increased by NCI), /g/data and MDSS available on request to UNSW. Copyq is used for moving data around. Useful commands lquota How much storage are you using nci_account List queues including how much compute have you used nf_limits -P project -n ncpus -q queue Show memory, walltime, etc. limits

5 Your first steps Use the module avail command to see what is installed. Then module help and module load to get information and to load the module. Start with an interactive job. It is ok to request extra memory and CPU cores whilst you are figuring things out but you need to remember to trim the resources as you get more confident. Take note of what you are doing. You may want to open a text editor and copy your commands into it. That includes commands within your application. If you have a lot of similar jobs, Array Job on Katana or at NCI Have a look at the web site(s). NCI. It has everything! Instructions, software lists, queues, status, etc. - Katana

6 Job Script Options Raijin Katana Project Code #PBS -P a99 N/A Job Queue #PBS -q normal Automatic Memory #PBS -l mem=300gb #PBS -l vmem=300gb Nodes and CPU Cores #PBS -l ncpus=4 #PBS l nodes=1:ppn=3 Job Walltime #PBS l walltime=12:00:00 #PBS l walltime=12:00:00 Start in Current Dir. cd $PBS_O_WORKDIR cd $PBS_O_WORKDIR when job finishes #PBS -m ae #PBS -M user@example.com #PBS -m ae #PBS -M user@example.com

7 This is a Katana job script This is a sample job script. The same #PBS options can be used for interactive jobs.

8 Job Queues Katana automatic according to walltime 12 Hours Any node in the cluster 48 hours Shared nodes plus your nodes 100 hours Shared nodes plus your nodes 200 hours Your nodes NCI You need to specify the queue that you want to use. The easy way to list is to type nci_account. Visit All nodes within each queue are identical. No production runs in Express queue. Special queues for big memory, KNL, GPU, etc. Check the queue status.

9 Nodes, CPU cores and Memory Katana - nodes=1:ppn=1 1 CPU core on 1 compute node Katana - nodes=1:ppn=12 12 CPU cores on 1 compute node Katana - nodes=2:ppn=12 12 CPU cores on 2 separate compute nodes (Don t use unless you know what MPI is). If you do, speak to us. At NCI selecting number of nodes happens automatically via ncpus. DO NOT SPECIFY NODES. Make sure you know how many cores the nodes in the queue have. Read the module help to see if there is a default number of cores (often called CPUs). Set the number of cores to match your job request. Sometimes it is better to have 1 CPU core for overhead. There is a full list of compute nodes on the web sites. You need to leave some space for the operating system. (i.e. avoid 96, 128, 144, 256, etc.). Request 2Gb less so that the operating system has some capacity.

10 Once a Job has Run THE END OF JOB IS IMPORTANT! READ IT! How long did your job run for? If your job is less than 20 minutes combine calculations. If your job has multiple stages then make each stage a different job and chain them together. The easiest way is to have a qsub command at the end of the job but there are many ways of chaining jobs. Speak to us! How much memory did you use? Can you reduce the amount of memory that you request next time? How much CPU time did you use? If ppn=6 and cputime = 2 * walltime then don t bother with more than 1 CPU core. If 1 core and cputime is less than half walltime then consider local scratch. Global = big files. How do I figure out how my job is working (memory, CPU, I/O, etc.)? - NCI offers courses

11 NCI Helpful Links User account and project management Wiki page HelpDesk or/and Job History Raijin Live Status Software License Status

12 Contacts Data Management and ResData: Outreach Librarians HPC, cloud, specialist storage questions: Research Technology Services or OneDrive, Data Archive, Research Active storage questions: UNSW IT , Data Classification issues: Data Governance Research Integrity issues: Your faculty Research Integrity Advisor

13 Questions