AN OVERVIEW OF COMPUTING RESOURCES WITHIN MATHS AND UON

Transcription

1 AN OVERVIEW OF COMPUTING RESOURCES WITHIN MATHS AND UON 1

2 PURPOSE OF THIS TALK Give an overview of the provision of computing facilities within Maths and UoN (Theo). When does one realise that should take advantage of these facilities? (Richard) Version Control via Git/Github (Theo/Richard) Arrange for follow-up sessions 2

3 OVERVIEW OF IT FACILITIES Provision: The School provide students with a Windows or Linux PC on their allocated desk or a Windows or Linux laptop and shared desk space. Filestore: Windows users have 4Gb of networked filestore (Z: drive). Linux users have 4Gb home plus access to various types of scratch space. Mac users have a 4Gb samba share on request. All users have access to the local disk of their computer. More extensive information about storage (including Box) can be found here. 3

4 DESKTOP PC All desktop PCs are new, and have a reasonable computational specification capable of most light or medium duty in Matlab/R. There are a small number of PCs which have their processor and memory tweaked upwards, to provide extra resource for local processing where it is likely to be used. 4

5 MATHS SERVERS 5

6 MATHS SERVERS School s servers: For simple Linux compute jobs which require long runtimes or large amounts of memory, you can use the school's Linux servers. What are these? These run 64-bit Linux and share much of the same software and facilities as Linux desktop PCs in the school. How to connect? Connect to them using SSH with Linux username and password, and take care to carefully read the information displayed on screen after login. How to get an account? Ask your supervisor to send an to Dave Parkin or speak to him directly! 6

7 MATHS SERVERS 7

8 MATHS SERVERS Please choose your server carefully based on your expected processor and memory requirements. All hostnames are.maths.nottingham.ac.uk, for example rhea.maths.nottingham.ac.uk. Please don't hog resources and only run one CPU or memoryintensive process at once*. If you do need to run multiple jobs, please spread them across different servers and use the "nice" command to reduce the priority of those jobs. 8

9 MATHS SERVERS The Linux servers are for shared use and should be used with consideration for other users. Please look at Considerate use of shared resources on Maths Linux servers page on how to monitor and make use of the resources while not preventing others from doing so! There are tools available via which one can see how loaded a server is etc. Overall, please, use common sense! 9

10 MATHS SERVERS The Maths server are a great (local) resource and is pretty straightforward to use them. You do not need to be a Unix/Linux expert to use the servers. However, you do need to learn a few basic commands, but: there is plenty of material out there to help you doing that. ask people around! 10

11 HIGH PERFORMANCE COMPUTING 11

12 HIGH PERFORMANCE COMPUTING (HPC) HPC: The UoN offers an on-site HPC service that can be accessed by any academic/students student who has the need for compute resource substantially greater than a standard PC. This includes those who can run their applications on a PC, but find the calculations tie up their machine for an excessive amount of time, as well as those who need more processors and/or computer memory. In addition, the University of Nottingham is part of a consortium of Midlands universities supporting scientific and engineering research via the Midlands-Plus HPC facility. 12

13 WHAT IS (ACTUALLY) THE HPC? In brief, it is a supercomputer! not a big single computer lots of connected computers, ie. a compute cluster ; built of standard, high-spec, compute servers (called nodes ); nodes are fully independent computers. high performance storage, and internal networks; optimised software. 13

14 HPC - MINERVA 166 compute nodes, each with 2x 8-core processors, 2.6GHz 2656 compute cores available Mix of standard memory (32GB) and high-memory (128GB) 500GB (local) hard-disk 14

15 HOW TO ACCESS MINERVA? Any job that one wants to run on the HPC needs to be associated with a project ; therefore all HPC users need to be a member of at least one project. To be a member of a project, one needs to have a HPC account. How do you get an account? Applications for access to the HPC are done via a web-based BPM process ( The 'HPC Application' form can be used to request new HPC accounts and projects. All requests need to be associated to a permanent member of academic or research staff. 15

16 HOW TO ACCESS MINERVA? How do you get an account? Accounts and project applications for PhD students must be submitted by their supervisors. The supervisors can identify themselves as a group leader for this purpose. 16

17 GETTING AN ACCOUNT ON MINERVA Applying for a HPC account can be a very daunting experience: Name of user (if not the same as applicant) Name of PI/Group Leader (if not the same as applicant) Date they attended/will attend HPC training (or name of experienced user who will help them) Static IP address of PCs or workstation(s) to be used to connect to the HPC 17

18 GETTING AN ACCOUNT ON MINERVA Applying for a HPC project can be a very daunting experience: Description of the project; Agresso project codes associated with project (if any); Approximate resource requirements (number of cores, length of jobs); Expected outputs from the project (e.g. publications), if any. 18

19 GETTING AN ACCOUNT ON MINERVA Do not worry if you find it difficult to estimate your resource requirements! This will not affect your resource allocation, as all accounts get a 'standard' allocation, unless they apply for an exception. More details about Minerva can be found here. 19

20 (SOME) PRINCIPLES OF GOOD PROGRAMMING 20

21 FACTS Many of us spend a lot of time programming; i.e. writing code to implement ideas/algorithms/methods. The majority of us are primarily self-taught (ie never had any formal training). How do we avoid making mistakes? 21

22 GOOD PRINCIPLES OF PROGRAMMING Get some training! Adopt (some of the) principles of good programming. version control; write tests; get a code buddy; reproducibility; 22

23 ARE YOU AFRAID TO CHANGE YOUR CODE? 23

24 Is this familiar? VERSION CONTROL code_ver1_old.r code_ver1b_wrong.r code_ver1b_broken_working_march20.r code_ver1b_broken_working_march20_fortom.r During your project it is very likely to ask yourself or your supervisor may ask you: Which version of the code did the results come from? What broke the code? Can I revert back to the code that it was working last week? 24

25 VERSION CONTROL A version control system is mostly based around one concept, tracking changes that happen within directories or files. In most cases, you specify a directory or set of files that should have their changes tracked by version control. The set of files or directories that are under version control are more commonly called a repository. As you make changes, it will track each change behind the scenes. 25

26 WHICH VERSION CONTROL? There are different types/choices of version control which one should you choose? A matter of preference/taste. It depends what your supervisor is using. 26

27 GIT AND GITHUB Git is a very popular distributed version control system. the full repository resides locally; it contains the full history; you can commit changes as often as you want; back-ups are trivial and readily available! although things happen locally, you can link your local repository to a server (github). 27

28 REFERENCES Masterclass on Git (Alan Turing Institute): Unix Workbench (free book) Git for humans (presentation): 28