High Performance Computing in C and C++

Transcription

1 High Performance Computing in C and C++ Rita Borgo Computer Science Department, Swansea University

2 WELCOME BACK

3 Course Administration

4 Contact Details Dr. Rita Borgo Home page: Office: Room 308, Faraday Tower Office Hours: Wednesday to Friday to Music Sound effects Speech

5 Course Administration Course Format: Main lectures: Monday to (Digital Technium PC Lab 002) Monday to (Glyndwr Rm F) Attendance is monitored, make sure you sign the attendance sheet.

6 Course Administration Course Format: Main lectures: Monday to (Digital Technium PC Lab 002) Monday to (Glyndwr Rm F) Attendance is monitored, make sure you sign the attendance sheet. Assessment: 100% Coursework: Assignment 0 (more on this later) 5% Assignment 1 (deadline 11/11/2013): 30% (100% programming) Assignment 2 (deadline 02/12/2013) 60% (100% programming) Lab. sessions run by HPCWales (MANDATORY): 5% Attendance & forms filling.

7 Assignments Assignment 0 (more on this later) 5% Assignment 1 (deadline 11/11/2012): 30% (100% programming) Two parts: Part 1: C programming (INDIVIDUAL Assignment) Part 2: MPI Programming (INDIVIDUAL Assignment) Assignment 2 (deadline 02/12/2013) 60% (100% programming)

8 Assignments Assignment 0 (more on this later) 5% Assignment 1 (deadline 11/11/2012): Assignment 2 (deadline 02/12/2013) 60% (100% programming) Two parts: Part 1: MPI programming Advanced (INDIVIDUAL Assignment) Part 2: Open Problem (GROUP Assignment)

9 Assignments Assignment 2 (deadline 02/12/2013) 60% (100% programming) Two parts: Part 1: MPI programming Advanced (INDIVIDUAL Assignment) Part 2: Open Problem (GROUP Assignment) Mandatory for distinction Groups MAXIMUM 3 people. You can choose your own partners.

10 Course Administration HPCWales Lab. Sessions: 4 Lab classes 12/11,16/11, 19/11, 23/11 5% of your Coursework: Complete all paperwork. Attend all Lab. Sessions, complete in-class exercises. If for any reason you are unable to attend the Lab. Sessions: you will be assigned a third piece of work.

11 Course Administration Coursework will be announced three weeks before the deadline; Platform used: Unix/Linux (no windows submission will be accepted) Prerequisites: Some experience of programming in a high level programming language such as: Java, Fortran, Pascal, Haskell etc. is required. Some experience with UNIX/Linux type of environment. C and C++ programming languages (MOSTLY C) will be surveyed, you will require basic programming knowledge such as: what an algorithm is, what a function is, what a variable is, what a type is etc.

12 Course Administration All exams and assignments must be completed individually, unless stated otherwise. Copying solutions is considered cheating. Submitted source code listings will be compared. Keep a copy of the listings to provide evidence of creative work. Unfair practice and plagiarism: University Definition and Procedure: gress/unfairpracticeprocedure/ School Definition and Procedure: refer to the Handbook

13 Course Administration Supporting Materials: Slides of each lecture will be made available online on Blackboard. Reading material or interesting resources will be made available online on Blackboard. Announcements will appear on Blackboard. CS-M78 Blackboard web page should re-direct you to CS- 358 Blackboard page.

14 Learning Objectives: What this module is about: Explore HPC platforms and programming models. Learn a C-based programming language and HPC library. Explore the means by which to measure, analyse and predict the performance of HPC applications. What this module is NOT about: Learning C and C++ in detail Learning to hack your serial code to improve its performance

15 Goal of the Module Introduction to HPC Practical experience for your research Finding the parallelism in your work Measuring speedup & efficiency and the factors that affect it Writing & debugging parallel code (MPI & OpenMP) Exposure to using production HPC systems at Swansea Effective techniques for inherently ( embarrassingly ) parallel codes Critical analysis of current & future HPC solutions (Cloud etc.)

16 Q&A Not trying to scare you off but to help you avoid any disappointment. Questions so far?

17 A bit about You Poll: what s your background? Familiar with Linux? Familiar with ssh? Comfortable with text editors (emacs, vi)?

18 Familiar with Linux? 1. True 2. False 100% 0% True False

19 Familiar with ssh? 1. True 2. False 100% 0% True False

20 Comfortable with text editors (emacs, vi)? 1. True 2. False 100% 0% True False

21 More about You Programming experience C, C++, Fortran, Others Scripting languages Practical experiences Ever written a program from scratch? Ever had to work with someone else s code? Which was harder? Why?

22 Experience w/ C, C++, Fortran, Others 1. True 2. False 100% 0% True False

23 Experience with Scripting Languages 1. True 2. False 100% 0% True False

24 Ever written a program from scratch? 1. True 2. False 100% 0% True False

25 Ever had to work with someone else s code? 1. True 2. False 0% 0% True False

26 Working with someone e... Writing a Program from... Enter Question Text 1. Writing a Program from scratch 2. Working with someone else code 0% 0%

27 Course Administration Books (worth consulting in the library) : The C Programming Language, (2ndEdition), Kernighan & Ritchie (worth consulting in the library) Parallel Programming with MPI, Peter Pacheco Online reference books: The C Book The C++ Language Tutorial Additional References: OpenMPI Home Page: OpenMPI FAQ: Relevant references, papers and online resources will be announced throughout the course.

28 Lecture 1 INTRODUCTION

29 High Performance Computing What is High Performance Computing (HPC) Difficult to define it s a moving target Late 1980s, supercomputers perform 100m FLOPs Today: A consumer desktop/laptop performs a few giga Flops A supercomputer performs tens of Tera Flops (on average) (IBM Sequoia : No.1 in Top 500 list, Peta Flops) High Performance: O(1000) more powerful than the latest desktops.

30 Applications of HPC HPC is Driven by demand of computation-intensive applications from various areas Biology, neuroscience (e.g. brain modelling) Finance (e.g. market prediction) Weather forecast (modelling and prediction) Military and Defence (e.g. simulations) Engineering (e.g. modelling and simulations) Anything else?

31 Example of Computational Demand Project: Blue Brain Aim: construct a simulated brain Building blocks of a brain are neurocortical columns A column consists of about 60,000 neurons Human brain contains millions of such columns First stage: simulate a single column (each processor acting as one or two neurons) Then: simulate a small network of columns Ultimate goal: simulate the whole human brain IBM contributes Blue Gene supercomputer

32 HPC Applied NASA Astrophysics and Goddard Space Flight Center (video)

33 HPC Applied

34 HPC Applied Two aspects to consider: Numerical Simulation: Provides the data Graphical Reconstruction: Provides the visual insight.

35 Architecture Details Damiana supercomputer Max Planck Institute for Gravitational Physics ( 7008Gb main memory 3504 Cores 462 nodes 3 vis machines, 26 storage server, 32 storage targets, 3 headnodes. Network: Infiniband DDR (20Gb/s), QDR (40Gb/s) 4-folded Lustre file system (500Tb).

36 Architecture Details Damiana supercomputer Max Planck Institute for Gravitational Physics. uters.aei.mpg.de /

37 Related Technologies HPC covers a wide range of technologies: Computer architecture CPU, memory, VLSI, multicore, GPU Networking Network topology, bandwidth, latency, communication protocols Compilers Identify inefficient implementations Make use of the characteristics of the computer architecture Choose suitable compiler for a certain architecture Algorithms (for parallel and distributed systems) How to program on parallel and distributed systems Middleware (Application->middleware->operating system) Grid Computing: Resource discovery and sharing

38 A BIT OF HISTORY

39 History of High Performance Computing 1960: Scalar processor Process one data item at a time (SISD) First HPC really Superscalar 1970s: Vector processor Can process an array of data items at one go (SIMD) Later 1980s: Massively Parallel Processor (MPP) Single computer with thousands of networked processors each with its own memory and OS Specialized interconnect networks

40 History of High Performance Computing Later 1990s: Cluster Not a new term itself, but renewed interests Connecting stand-alone computers Commodity hardware for networking Later 1990s: Grid Tackle collaboration among geographically distributed organisations Today: Cloud (fuzzy term) massively scalable ICT relatedcapabilities (e.g., networks, servers, storage, applications, services) are provided as a service to external customers using Internet technologies.

41 More about You Any HPC experience Your goals for this course: Take 5 minutes. Come up with at least one topic you would like to be explored in Lectures. Write it down on a sheet of paper.

42 Assignment 0 Available on Blackboard NOW. Due date: 02/10/2013 at 11am. Warm-up exercise. INDIVIDUAL Assignment.