High Performance Computing Course Notes Course Administration

Transcription

1 High Performance Computing Course Notes Course Administration

2 Contacts details Dr. Ligang He Home page: Office hours: Room 205 Monday: 11:30-12:30 Wednesday: 11:30-12:30 2

3 Course Administration Course Format main lectures (30 hours): foundation classes/seminars week 1: 11am-12pm Thursday, CS101 Week 2-5: 11am-12pm Wednesday, CS101 Assessment: 15 CATs 70% examined, 30% coursework Coursework details announced in week 5 3

4 Agenda Fundamentals Programming modelling MPI Performance analysis and prediction Coursework Cluster computing High performance storage High performance I/O Grid Computing Revision class, Lab session and coursework marking 4

5 Learning Objectives By the end of the course, you should understand: Commonly used parallel programming models and HPC platforms The means by which to measure, analyse and predict the performance of HPC applications running on their supporting HPC platforms. The role of administration, scheduling, and data management in an HPC management system, with particular reference to cluster/grid computing 5

6 Materials The slides will be made available on-line after each lecture Relevant reference books, papers and on-line resources will be announced throughout the course. 6

7 Coursework Coursework will involve the development of a parallel application using the Message Passing Interface (MPI). It will involve performance analysis and modelling. Your coursework will be assessed by the demo of your program and performance model, and a written report about how you construct the performance model. Please attend the foundation classes/seminars! 7

8 High Performance Computing Course Notes HPC Fundamentals

9 Introduction What is High Performance Computing (HPC)? Difficult to define - it s a moving target. Later 1980s, a supercomputer performs 100m FLOPS Today, a normal desktop/laptop performs a few giga FLOPS Today, a supercomputer performs tens of Tera FLOPS (Top500) High performance: O(1000) more powerful than the latest desktops 9

10 Applications of HPC HPC is Driven by demand of computation-intensive applications from various areas Biology, neuroscience (e.g. simulation of brains) Finance (e.g. modelling the world economy) Military and Defence (e.g. modelling explosion of nuclear bombs) Engineering (e.g. simulations of a car crash) 10

11 An Example of Demands in Computing Capability Project: Blue Brain aim: construct a simulated brain Building blocks of a brain are neurocortical columns A column consists of about 60,000 neurons Human brain contains millions of such columns First stage: simulate a single column (each processor acting as one or two neurons) Then: simulate a small network of columns Ultimate goal: simulate the whole human brain IBM contributes Blue Gene/L supercomputer 11

12 Related Technologies HPC covers a wide range of technologies: Computer architecture CPU, memory, VLSI Networking Network topology, bandwidth, latency Compilers Identify inefficient implementations Make use of the characteristics of the computer architecture Choose suitable compiler for a certain architecture Algorithms (for parallel and distributed systems) How to program on parallel and distributed systems Middleware From Grid computing technology Application->middleware->operating system Resource discovery and sharing 12

13 History of High Performance Computing 1960s: Scalar processor Process one data item at a time 1970s: Vector processor Can process an array of data items at one go Architecture Overhead Difference between vector processor and scalar processor Later 1980s: Massively Parallel Processing (MPP) Up to thousands of processors, each with its own memory and OS Break down a problem Difference between MPP and vector processor Later 1990s: Cluster Not a new term itself, but renewed interests Connecting stand-alone computers with high-speed network Difference between cluster and MPP Later 1990s: Grid Tackle collaboration among geographically distributed organisations Draw an analogue from Power grid Difference between Grid and cluster 13

14 Parallel computing vs. distributed computing Parallel Computing Breaking the problem to be computed into parts that can be run simultaneously in different processors Example: an MPI program to perform matrix multiplication Solve tightly coupled problems Distributed Computing Parts of the work to be computed are computed in different places (Note: does not necessarily imply simultaneous processing) An example: C/S model Solve loosely-coupled problems (no much communication) 14

15 Architecture Types Shared memory (uniform memory access - SMP) Multiple CPUs, single memory, shared I/O All resources in a SMP machine are equally available to each CPU Processors share access to a common memory space. Implemented over a shared memory bus. Support for critical sections are required Local cache is critical: If not, bus contention (or network traffic) reduces the systems efficiency. For this reason, pure shared memory systems do not scale well. Cache introduces problems of coherency (ensuring that stale cache lines are invalidated when other processors alter shared memory). 15

16 Architecture Types Shared memory (Non-uniform memory access: NUMA) Multiple CPUs Each CPU has fast access to its local area of the memory, but slower access to other areas Scale well to a large number of processors Complicated memory access pattern and system bus Global address space. 16

17 Architecture Types Distributed Memory (MPP, cluster) Each processor has it s own local memory. When processors need to exchange (or share data), they must do this through an explicit communication Message passing (MPI language) Typically larger latencies between PEs (especially if they communicate via over-network interconnections). Scalability is good if the task to be computed can be divided properly. 17

18 Scalability Strong scaling: defined as how the execution time varies with the number of processors for a fixed total problem Weak scaling: defined as how the execution time varies with the number of processors for a fixed problem size per processor 18