High Performance Computing. Without a Degree in Computer Science

Transcription

1 High Performance Computing Without a Degree in Computer Science

2 Smalley s Top Ten 1. energy 2. water 3. food 4. environment 5. poverty 6. terrorism and war 7. disease 8. education 9. democracy 10. population

3 Number of Physical Scientists and Engineers Computation is an important research paradigm in physical science Today, many scientists spend (waste?) enormous amounts of time on structuring of computations A goal of computer science: help make these scientists more productive

4 A Bit of History In the beginning, there was machine language (or assembly language) 1957: Fortran (John Backus,et.al., IBM) made it possible for every scientist to develop applications Today: programming high end machines is once again the near-exclusive domain of experts.

5 Key: The Compiler Fortran Program Compiler Machine Code Takes care of the many details of making the machine perform well Goal: make the penalty for using the programming language as small as possible

6 Making Languages Usable It was our belief that if FORTRAN, during its first months, were to translate any reasonable scientific source program into an object program only half as fast as its hand-coded counterpart, then acceptance of our system would be in serious danger... I believe that had we failed to produce efficient programs, the widespread use of languages like FORTRAN would have been seriously delayed. John Backus

7 The Programming Problem Programming is hard, and getting harder with new platforms Professional programmers are (still) in short supply Programming systems that result in low performance will not be accepted

8 The Programming Problem: A Strategy Make it possible for end users to become programmers users integrate software components using problem-solving environments (PSEs) or scripting languages (e.g., Visual Basic, Matlab) professional programmers develop software components

9 The Programming Problem: An Obstacle Achieving High Performance: translate scripts and components to common intermediate language optimize the resulting program using whole-program compilation

10 Whole-Program Compilation Component Library Script Translator Global Optimizing Compiler Code Generator Problem: long compilation times, even for short scripts! Problem: expert knowledge on specialization lost

11 Telescoping Languages L1 Component Library Compiler Generator Could run for many hours Script Translator L1 Compiler understands library calls as primitives Code Generator Optimized Application

12 Telescoping Languages: Advantages Compile times can be reasonable High-level optimizations can be included User retains substantive control over language performance Generate a new language with userproduced libraries Reliability can be improved

13 Applications

14 Application: Matlab SP Signal processing users want simplicity, programming power, and performance Currently over 500,000 Matlab licenses Matlab gives them simplicity and power but not performance Codes prototyped in Matlab, then rewritten in low-level programming language

15 Matlab SP: Profitable Transformations Vectorization: conversion of loops to array expressions Optimization of array expressions, including array allocation and reshape Applying conventional expression optimizations to procedures

16 Procedure Strength Reduction for i = 1:N x = x + f(c1,c2,i,c3) end f 0 (c1, c2, c3) for i = 1:N x = x + f 1 (i) end

17 Strength Reduction Performance Before Strength Reduction 1.0 After Strength Reduction jmp1 ctss olbf Results courtesy of Arun Chauhan

18 Application: Matlab SP Role of Telescoping Languages: Critical signal processing code modules are reused many times Run these procedures through the language generator Produce Matlab SP, a high-level domainspecific environment

19 Component Integration System Component integration systems are viewed as important productivity tools Programs constructed from them are often slow because no context based code improvements can be applied Telescoping languages could be applied to construct component integration systems that yield high-performance applications

20 Component Integration: A Special Case Integration of different component libraries that Implement data structures (e.g., sparse matrices) Implement functions on data structures (e.g., linear algebra) Telescoping languages can handle this well

21 Parallelism in Matlab Add distributions to Matlab arrays Distributions can cross multiple processors A(1:100) A(101:200) A(201:300) A(301:400) Use distributions to guide parallelism Hide parallelism in component array operations

22 Library Generator (ARGen) Prof Dan Sorensen (Rice CAAM) maintains ARPACK, a large-scale eigenvalue solver He prototypes the algorithms in Matlab, then generates 8 variants in Fortran by hand: ({Double, Complex} x {Symmetric, Nonsymmetric} x {Dense, Sparse} Could this hand generation step be avoided?

23 ARGen Results 400 Matlab ARGen ARPACK Dense Symmetric Sparse Symmetric Results courtesy of Cheryl McCosh

24 A Statistical Analysis Language S: A high-level language for manipulating, analyzing, and displaying data, widely used for design of clinical studies in medicine S Optimization: All the Matlab optimizations Translation to C with folding of temporary arrays into usage points

25 S Optimization Results Speedup Geneshaving Gibbs Smpl Trial Design Results courtesy of Bradley Broom

26 Generator for Grid Computations The Grid: nets of interconnected supercomputers Several national science infrastructures under development Challenge: application development

27 National Distributed Problem Solving Database Supercomputer Supercomputer Database

28 Grid Programming Today Application development is possible Support for finding available computer cycles, accounting, job initiation, and communication between parts of a program running on different machines Applications are programmed by hand Requires special expertise

29 Grid Programming Challenges Finding parallelism Mapping applications to machines and network links with different capacities Adapting to changes in load

30 GrADSoft Architecture Software Components Real-time Performance Monitor P S E Program Integrator/ Compiler Application Configurable Object Program Scheduler/ Resource Negotiator Negotiation Grid Run- Time System Libraries Binder Program Preparation System Execution Environment GrADS Project (NSF NGS): Berman, Chien, Cooper, Dongarra, Foster, Gannon, Johnsson, Kennedy, Kesselman, Mellor-Crummey, Reed, Torczon, Wolski

31 Summary A goal of computer science research is to make professionals, especially scientists and engineers, more productive This goal is difficult to achieve because of the need for high-performance applications One solution is to develop technologies that directly translate prototyping languages to production code

32 Collaborators Bradley Broom Arun Chauhan Keith Cooper Jack Dongarra Rob Fowler Lennart Johnsson Chuck Koelbel Cheryl McCosh John Mellor-Crummey Linda Torczon

33 The End