High Performance Computing and Programming. Performance Analysis

Transcription

1 High Performance Computing and Programming Performance Analysis

2 What is performance? Performance is a total effectiveness of a program What are the measures of the performance? execution time FLOPS memory usage power consumption response time (time from request until result) throughput (number of operations per unit time) bandwidth (number of bits transmitted per unit time)

3 What is performance? Performance is a total effectiveness of a program What are the measures of the performance? execution time FLOPS memory usage power consumption response time (time from request until result) throughput (number of operations per unit time) bandwidth (number of bits transmitted per unit time)

4 What is performance analysis? Computers are systems of components that interact with each other. Complex and hard to model analytically Performance analysis is an experimental discipline of computer science involving: Measurement Interpretation Communication

5 Measuring time What is taking so much time? Which section of my program should be made better? How can you figure out why it is so slow? What are ways to speed it up?

6 CPU clock rate CPU clock speed or rate is the speed at which a microprocessor executes instructions number of cycles per second - Hz Today s processors 2-5GHz (~ thousand million cycles per second) Record (2013): IBM zec12 with 5.5 GH Amount of work different CPUs can do in one cycle varies

7 View of time by system and application System perspective 2 processes: A and B Perspective of the process A The system switches from process to process. Interval time time between interrupts. And it is operating in user or kernel mode. Useful computation are performed just in the user mode

8 Interval Counting OS measures runtimes using Interval Counting Maintains 2 counts per process User time - time executing instructions in current process System time - time executing instructions in kernel on behalf of current process Every X cycles, trigger a timer interrupt and increment a counter for executing process User time if running in user mode System time if running in kernel mode This is called a clock tick

9 Interval Counting OS measures runtimes using Interval Counting Maintains 2 counts per process User time - time executing instructions in current process System time - time executing instructions in kernel on behalf of current process Every X cycles, trigger a timer interrupt and increment a counter for executing process User time if running in user mode System time if running in kernel mode This is called a clock tick

10 Interval Counting Example 2 processes: A and B 10 ms Time segment is assigned to a process as part of either its user (u) or system (s) time A 110u + 40s B 70u + 30s

11 Interval Counting Example 2 processes: A and B 10 ms Time segment is assigned to a process as part of either its user (u) or system (s) time Actual times: A 110u + 40s B 70u + 30s A 120.0u s B 73.3u s Therefore this timing mechanism is only approximate

12 Accuracy of process timer 1 active process Process time unusable below 100 ms 11 active processes

13 Accuracy of process timer Conclusion: Coarseness of the timing method results in big error Interval counting is only useful for measuring relatively long computations Accuracy does not depend strongly on the system load

14 Cycle counter Most modern systems have built-in registers that are incremented every clock cycle Very fine grained No uniform, platform-independent interface Special assembly code instruction to access On (recent model) Intel machines: 64 bit unsigned number accessed with the RDTSC (read time stamp counter) instruction

15 Cycle counter Measuring time between two points in program Count the total number of cycles between this points Does not keep track of which process uses cycles Errors due to context switching, cache operation, and branch prediction But: errors will always cause overestimates of the true execution time => nothing will artificially speed up a program

16 Time on a Computer System Wall Clock = Elapsed Time includes everything from beginning to end (disk, OS activity, I/O ) CPU time time spent on the program (while it is actually running) user CPU time system CPU time Wall time Other processes System time for one core: CPU time User time

17 Time on a Computer System = user time = system time = some other user s time + + = real (wall clock) time Usually the word time refers to user time. cumulative user time

18 Multicore Effects Counters are incremented once per interval per process: Multithreaded programs running on p cores for t seconds can show pt seconds CPU time Use wall-time instead when timing parallel code

19 Timers Timer Usage Wallclock/ CPU Resolution (s) Lang Portability time shell/script both 10-2 / yes gettimeofday function wallclock 10-6 (*) C/C++ yes clock_gettim e function CPU time 10-9 C/C++ No (Unix) * hardware dependent minimum period of time that can be measured

20 Timers in code - clock() uses interval timer gives CPU time Example: #include <time.h> clock_t start = clock(); printf( doing stuff here\n ); clock_t end = clock(); printf( It took me %d ticks (%f seconds)\n,end-start, (double)(endstart)/clocks_per_sec);

21 Timers in code - gettimeofday() Unix gettimeofday() function Return elapsed time since reference time (Jan 1, 1970) Some implementations use interval counting, while others use cycle timers. System Resolution (ms) Latency (ms) Pentium II, Windows-NT Compaq Alpha Pentium III Linux Sun UltraSparc

22 Timers in code - clock_gettime() Support different clocks: CLOCK_REALTIME - system-wide clock that measures real (i.e., wall-clock) time affected by discontinuous jumps in the system time affected by the incremental adjustments performed by adjtime and NTP CLOCK_MONOTONIC - monotonic time since some unspecified starting point not affected by discontinuous jumps in the system time affected by the incremental adjustments performed by adjtime and NTP Strictly linearly increasing, and thus useful for calculating the difference in time between two samplings CLOCK_MONOTONIC_RAW - monotonic time since some unspecified starting point not affected by discontinuous jumps in the system time not affected by the incremental adjustments performed by adjtime and NTP

23 Guidelines for measuring execution time Use a high resolution timer, such as clock_gettime() Check the amount of time your program spends in the OS using the time command Do not trust timings unless they are at least 100 times longer than the CPU time resolution Measure several runs to gauge variability (3 or more) Pick the shortest time as a representative (or an average)

24 Profiling Profiling is a process of analizing the behavior of the program (CPU performance, memory, I/O) while it is running Gives a detailed description of execution times for each part of the program Allows to identify hot spots - big amount of executed instructions or place in the program where it uses a lot of CPU time bottlenecks - parts of the program which uses processor resourses inefficiently Profiling is done on working code!

25 Statistical Sampling and Code Instrumentation Sampling means polling the status of the program at regular time intervals Resulting profiles are statistical not exact Usually has minimal interference with program's runtime e.g. Unix time Instrumenting (tracing) means interposing profiling calls within your program's regular calls More exact, but also more resource-intensive Reproducible results e.g. valgrind

26 Measuring FLOPS Floating-point Operations Per Second FLOPS = # of floating point operations in the program Execution time Peak performance of the computer Frequency of the computer (Hz) Number of floating point operations per cycle MFLOPS = 10 6 FLOPS GFLOPS = 10 9 FLOPS TFLOPS = FLOPS PFLOPS = FLOPS Upper bound on performance FLOPS = CPU speed (Hz) * op per cycle * number of cores

27 Measuring FLOPS (1) Count the number of operations in your source code (or, preferably, examine *.s) (2) Time the operation (3) Calculate the rate How far from peak performance? 10% of the peak performance - room for optimization >50% of the peak performance - probably not going to get much more efficient (unless the basic algorithm is rewritten)

28 Scalar Product, example Sum = 0.0; for( i=0; i<n; i++ ) { sum += a[i]*b[i]; } ~N additions ~N multiplications ~2*N FLOP printf( Scalar product of a and b is %f\n,sum);

29 Debugging Debugging is the stage of development of the program during which one finds, isolates, and fix bugs (errors). In order to find the error you need to know: temporary values of variables how program was executed, which execution path was taken Two ways to debug program: using log files, i.e. using output function write state of the program in critical points using debuggers (includes user interface, can execute program step by step or till some condition is satisfied, write temporary values of variables and so on)

30 Debugging locating errors Ex.: GNU Debugger, GDB step through line by line stop on specified conditions show the values of variables contents of the call stack set breakpoints

31 How the debugger figures out where to find the functions and variables in the machine code? debugging information! (DI) Compile it with: gcc -g main.c - DI is stored in the object file - DI describes the data type of variables and functions and the correspondence between source line numbers and addresses in the executable code

32 objdump -h debug_aranges **0 CONTENTS, READONLY, DEBUGGING 27.debug_info c **0 CONTENTS, READONLY, DEBUGGING 28.debug_abbrev ae ee 2**0 CONTENTS, READONLY, DEBUGGING 29.debug_line c c 2**0 CONTENTS, READONLY, DEBUGGING 30.debug_str cf d8 2**0 CONTENTS, READONLY, DEBUGGING 31.debug_loc a7 2**0 CONTENTS, READONLY, DEBUGGING

33 objdump --dwarf=info <1><2d4>: Abbrev Number: 10 (DW_TAG_subprogram) <2d5> DW_AT_external : 1 <2d6> DW_AT_name 0x8a): main <2da> DW_AT_decl_file : 1 <2db> DW_AT_decl_line : 67 <2dc> DW_AT_type <2e0> DW_AT_low_pc <2e8> DW_AT_high_pc : (indirect string, offset: : <0x57> : 0x40093b : 0x400ac4

34 Valgrind Valgrind is a wrapper around a collection of tools for memory debugging, memory leak detection, and profiling: Memcheck - detects memory-management problems Cachegrind & Callgrind - cache profilers Massif - heap profiler...

35 Valgrind Memcheck Tool to check for memory errors and memory leaks Run the executable inside its own environment Memcheck can detect: Use of uninitialised memory Accesses memory it shouldn't areas not yet allocated areas that have been freed areas past the end of heap blocks Inaccessible areas of the stack Memory leaks - where pointers to malloc blocks are lost forever Mismatched use of malloc and free Overlapping of pointers in memcpy() and related functions...

36 Valgrind Memcheck memory leaks --leak-check=full give more information ==7623== LEAK SUMMARY: ==7623== definitely lost: 2,400 bytes in 3 blocks ==7623== indirectly lost: 120,000 bytes in 300 blocks ==7623== possibly lost: 0 bytes in 0 blocks ==7623== still reachable: 0 bytes in 0 blocks ==7623== suppressed: 0 bytes in 0 blocks ==7623== 40,800 (800 direct, 40,000 indirect) bytes in 1 blocks are definitely lost in loss record 6 of 6 ==7623== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheckamd64-linux.so) ==7623== by 0x400666: allocate_mem (matmul.c:18) ==7623== by 0x400A57: main (matmul.c:92)

37 Valgrind Memcheck Memcheck runs programs about 10-30x slower than normal Valgrind may not find all the errors especially those that don't happen every time. Valgrind doesn't perform bounds checking on static arrays (allocated on the stack)

38 Valgrind Cachegrind & Valgrind Callgrind Cache memory is a computer memory with very short access time used for storage of frequently or recently used instructions or data Cachegrind finds source location of cache misses Callgrind is Cachegrind + function call graph information Simulates CPU with 2-level cache: 1L and LL. Size of the caches can be specified, default is current machine s cache Large overhead: x times slower

39 Inclusive/Exclusive CPU time Inclusive CPU time is the time spent in a function and all its children Exclusive (self) CPU time is the time spent only executing code in the function itself

40 gprof gprof is a standard UNIX profiling utility for profiling executables and shared libraries (1) compile with and link with -pg (2) run your program > generates gmon.out (3) $ gprof <executable> gprof shows CPU time, not real time, disk delays due to input and output are not included instrumenting the code adds some book-keeping information collects samples from looking at the program counter times per second of run time (system dependend)

41 gprof gprof displays the following information: The parent of each procedure. An index number for each procedure. The percentage of CPU time taken by that procedure and all procedures it calls (the calling tree). A breakdown of time used by the procedure and its descendents. The number of times the procedure was called. The direct descendents of each procedure.

42 gprof output Flat profile: how many times each function got called, total times involved, sorted by time consumed Call-graph: each function, who called it, whom it called, and how many times

43 Profiler output What do do with profiler results? observe which methods are being called the most observe which methods are taking most time relative to others Profiler pitfalls: profiling slows down your code design your profiling tests to be very short it doesn't measure everything

44 Know when to stop Pareto Principle 20% of code takes 80% of execution time Procedure CPU time main() 13% procedure1() 17% procedure2() 20% procedure3() 50% A 20% increase in the performance of procedure3() results in a 10% performance increase overall. A 20% increase in the performance of main() results in only a 2.6% performance increase overall.

45 Optimization Flags to the compiler: -O0 reduce the cost of compilation and to make debugging produce the expected results [default] -O1 minimize code size with small speed optimizations -O2 maximize program speed, Intel default -O3 more aggressive optimizations than -O2, but not always better -Os optimize for size. Enables all -O2 optimizations that do not increase code size. Performs further optimizations designed to reduce code size. Note: on different compiler flags can include different kind of optimizations

46 Optimization optimization can affect debugging and profiling: Debugging & Profiling shows what is really in the code Optimization rearranging the code example: if you define a variable, but never use it

47 Measuring arithmetic intensity Arithmetic intensity: math operations per data movement flops/word High arithmetic intensity: program is computebound Reduce math operations or parallelize Low arithmetic intensity: program is bandwidthbound

48 Matrix-vector Product, example Matrix(n*k)-vector product: n*((k-1)+n) for( i=0; i<n; i++ ) for( j=0; j<k; j++) c[i] += a[i][j]*b[k];

49 Reporting computational experiments The goal is to communicate performance results To coworkers & yourself To an external audience The audience must be able to understand: What is being measured What you are showing them What it means

50 Reporting computational experiments Presentation of algorithms Complete description of the algorithm Specification of the domain of applicability of the algorithm If possible and relevant: Computational complexity in time and space If relevant: Convergence results If relevant: Rate of convergence, order of accuracy

51 Reporting computational experiments Traditional indicators are e.g. CPU time Number of iterations Operation count (flops) Memory footprint Numerical accuracy Scope, applicability Portability

52 Reporting computational experiments Presentation of implementation Programming language used Compiler name and options used Computer environment, system and OS (not a Linux machine with an AMD processor ) Input data Settings Provide a ready-to-run package!

53 Reporting computational experiments Presentation of experiments Objective of the experiment Description of problem generator/input data set (Why) Are the results general?

54 Reporting computational experiments Presentation of results, e.g. runtime: Description of how the timings were produced What parts of the code are included in the measurements? What is the variability of the timings? How is the variability handled?

55 Reporting computational experiments Before: make sure that you understand the task correctly plots must have a good quality (ex. use Matlab) think carefully what you exactly want to show all tables and figures must be numbered all tables and figures must have self-explanatory titles figures must have lables explain what tables and pictures means in the text give just relevant information give correctly references and links to the literature

56 Take-home message Use debuggers to find bugs in code Use profilers and timing utilities to: Understand what the computer is doing Expose performance bottlenecks Target your optimization effort Target your parallelization effort Do not optimize code before it is correct!