Intel Tools zur parallelen Programmierung Windows HPC RWTH Aachen 2007

Transcription

1 Intel Tools zur parallelen Programmierung Windows HPC RWTH Aachen 2007 Dr. Mario Deilmann Intel Compiler Group

2 Processor Evolution X86 New Quad-Core Intel Xeon 5300 for 2006 Dual-Core Intel Xeon processor 5100 series Intel Xeon processor Quad Core Dual Core Single Core 2

3 Parallel computing is omnipresent Over the next few years, all computers will be somehow parallel computers. Servers Laptops Cell phones What about software? Herb Sutter of Microsoft said in Dr. Dobbs Journal: The free lunch is over: Fundamental Turn towards Concurrency in software Software performance will no longer increase from one generation to the next as hardware improves unless it is parallel software 3

4 Intel Processor and Platform Evolution for the Next Decade 4

5 Independent Programs Application Parallel Programs Application Application Application Application Application Threads Thread Switch Process Switch Threads allows one application to utilize the power of multiple processors 5

6 How do we make use of the additional core s and CPU s on the software side?

7 What is Parallelism? Two or more processes or threads execute at the same time! Multiple processes communicate through an inter-process protocol Single process with multiple threads which communication through shared memory 7

8 Process model Process OS creates process for each program loaded Process Master thread Additional threads can be created within the process Threads share code and data Each thread has its separate Registers and Stack Stack Thread Stack Thread Stack Code segment Data segment 8

9 Parallel Programming Models Message Passing Create/fork multiple processes Node Typically one per node/core Explicit communication Send messages send(tid, tag, message) receive(tid, tag, message) P P P M M M Synchronization Block on messages Barriers 9 Interconnect

10 Parallel Programming Models Shared Memory Create one process with multiple threads Typically one per node/core Implicit communication P T T Using shared address space Loads and stores Synchronization Locks Atomic memory operators Barriers 10 T Bus Memory

11 Parallel programming approaches API / Library Threads - P-threads (POSIX), Win32* threading API Intel Threading Building Blocks (C++) MPI, PVM Programming language mechanisms Java*, C#, Erlang Programming language extension OpenMP (C, C++, Fortran, ) UPC (unified parallel C) Cilk (extension to GCC) 11

12 Multithreading introduces new class of problems Developing threaded or MPI applications is hard but new and advanced Intel architectures and software tool help to support these approaches. New class of problems is introduced due to the interaction between threads which are complicated and hard to find! Correctness problems (data races) Performance problems (contention) Runtime problems (crashes) Opened the Pandora s box 12

13 Performance versus effort MPI Code Performance Theoretical speedup limited by number of CPU s per cluster Cluster OpenMP Theoretical speedup limited by number of Core s per CPU Threads Theoretical speedup limited by Core OpenMP Serial optimization Development effort Code Restructuring 13

14 Intel Software Development Tools for Parallel Programming

15 How can we support parallel Development Analysis (seriell / parallel) VTune Performance Analyzer Intel Trace Analyzer Design (Introduce Parallelism or extend) Intel Performance libraries: IPP and MKL OpenMP* (Intel Compiler) Intel MPI Debug for Correctness (data races, locks) Intel Thread Checker Intel MPI Correctness Checker Tune for Performance (bottlenecks) Intel Thread Profiler VTune Performance Analyzer Intel Trace Collector 15

16 Intel Threading Tools

17 Intel VTune Analyzer 9.0 Identifies hard to find performance bottlenecks Features Tune process or thread parallel code "The Intel VTune Performance Analyzer took a multi-day task and turned it into a subday task." Randy Camp VP, Software R&D MUSICMATCH Inc. Low overhead sampling Graphical call graph View results on source or assembly What s New New tuning methodology Stall cycle accounting for Core 2 Duo and Core 2 Quad processors Windows: Microsoft Vista* support Linux: Connection to Intel compiler analysis & intuitive hotspot navigator Windows* Linux* Mac* IA32 Intel64 IA64 Multicore 17

18 VTune Performance Analyzer Helps you identify and characterize performance issues by: Collecting performance data from the system running your application Sampling: Event-based or Time-based Call Graph Counter Monitor Organizing and displaying the data in a variety of views From system-wide down to source code or processor instruction perspective GUI and CLI VTune Analyzer Driver Kit Rebuild VTune Analyzer Linux driver for non-standard kernels Red Hat, SuSE, Red Flag distributions supported 18

19 What Can You Profile with Vtune? Windows/Linux applications Stand-alone Win* DLLs Stand-alone COM+ DLLs Java applications.net* applications ASP.NET applications 19

20 Performance Analysis Technologies Identify Performance Bottlenecks (Sampling) Interrupt based sampling using CPU registers (PMU) Lower Overhead Examine flow of control through the app (Call Graph) Which functions took the longest Which functions were blocked the longest Calling sequence critical path Higher Overhead, more data 20

21 Performance Hotspots - Sampling Sample the CPU s execution context Periodically interrupts the processor Time-based: Triggered at a certain time intervall Event-based: Triggered by the occurrence of a certain events Collects the execution context Execution address in memory (CS:IP) Operating system process and thread ID Executable module loaded at that address If you have symbols for the module, post-processing can identify the function or method at the memory address. Line numbers from the symbol file can direct you to the relevant line of source code. Can measure performance sensitive CPU events Cache misses, branch mispredictions, 21

22 Sampling Module of Interest 22

23 Sampling over time 23

24 Sampling: Source View 24

25 Programmatic Flow of control Call Graph Instrumented technology Some performance degradation Binary is instrumented No special build needed Identifies function to function calling sequences Reports statistics for each called function Execution time Blocked time Calling sequences & frequency of occurrence 25

26 Call graph: Application workflow Filter view by self time The red lines show the critical path. The critical path is the most timeconsuming call path. It is based on self time. Bright orange nodes indicate functions with the highest self time. 26 Intel, VTune, and the Intel logo are trademarks or registered trademarks of Intel *Intel and the Intel logo are registered trademarks of Intel Corporation. Other brands and names are the property of their respective owners or its subsidiaries in the United States or other countries. Corporation

27 Graph Navigation Window Use the graph navigation window for an overview of the entire call graph. 27

28 Vtune Tuning Assistant For more detail, click hyperlink. 28 Intel and the Intel logo are trademarks or registered trademarks of Intel *Intel and the Intel logo are registered trademarks of Intel Corporation. Other brands and names are the property of their respective owners or its subsidiaries in the United States or other countries. Corporation

29 ICC optimization report VTune displays optimization reports generated with ICC 9.0 and later Allows simplify performance optimization work when ICC and VTune are used together and handles all optimization phases supported by ICC 29

30 Intel Thread Checker v3.1 Confidently pinpoint threading errors Features Detects challenging data races and deadlocks Pinpoints errors to the source code line Command line interface for Windows and Linux Works on standard debug builds without recompiling Batch scripts integration for regression test runs We couldn t have gotten the networking up and running as quickly and as efficiently without Thread Checker. Thread Checker is simply an awesome tool and we are not going to develop multi-threaded code without it. Doug Service, Dir. of Tech. Dev. Chris Stark, Software Engineer Ritual Entertainment What s New Faster analysis through performance optimizations Microsoft Vista* support Windows* Linux* Mac* IA32 Intel64 IA64 Multicore 30

31 Example: Not Quite Right #include #include <stdio.h> <stdio.h> const const long long NN == ; ; long long primes[n], primes[n], number_of_primes number_of_primes == 0; 0; main() main() {{ printf( printf( "Determining "Determining primes primes from from 1-%d 1-%d \n", \n", NN ); ); primes[ primes[ number_of_primes++ number_of_primes++ ]] == 2; 2; // // special special case case #pragma omp parallel for for for (( long long number number == 3; 3; number number <= <= N; N; number number += += 22 )) {{ long long factor factor == 3; 3; while while (( number number %% factor factor )) factor factor += += 2; 2; if if (( factor factor == == number number )) primes[ primes[ number_of_primes++ number_of_primes++ ]] == number; number; }} printf( printf( "Found "Found %d %d primes\n", primes\n", number_of_primes number_of_primes ); ); }} 31

32 Intel Thread Checker Key Benefits Detects challenging data races and deadlocks Pinpoints errors to the source code line Works on standard debug builds without recompiling Recommends modules to instrument by usage (minimize instrumentation overhead) Scriptable interface for test environment integration (enabling batch file runs) Supports 32 and 64-bit applications 32

33 Intel Thread Checker Intel Thread Checker Primes.exe Binary Instrumentation Primes.exe (Instrumented) Runtime Data Collector +DLLs (Instrumented) threadchecker.thr (results) Win32* threads, POSIX* threads, OpenMP* 33 *Intel and the Intel logo are registered trademarks of Intel Corporation. names are are the property of their owners *Third Other partybrands marksand and brands the property ofrespective their respective owners

34 Intel Thread Checker S T N OI E P PIN URC SO DE CO 34

35 Intel Thread Profiler v3.1 Pinpoints threading inefficiencies Features View application concurrency level to ensure full core utilization Identify where thread related overhead impacts performance Find out which created threads are active and which are inactive Included with VTune Analyzer for Windows* Intel Thread Profiler was very useful for analyzing bottlenecks in our threaded code. Thread Profiler quickly pinpointed problem areas and showed us the reasons for the slowdown, so we were able to restructure the code for better threaded performance. Martin Watt Software Architect Alias Support for Threading Building Blocks API What s New Easier to Use - Recall custom configuration settings Faster to Use - User selectable stack walking Microsoft Vista* support Windows* Linux* Mac* IA32 Intel64 IA64 Multicore 35

36 Speedup Performance Profile: Recap Threads Possible causes for this scalability profile: 1. Insufficient parallel work 2. Memory bandwidth limitations 3. Synchronization overhead 4. Load imbalance 36

37 Intel Thread Profiler Key Benefits Shows how much of your application is not optimally parallel and where Identifies where thread specific overhead impacts performance Highlights thread workload imbalances and thread activity Shows the number of cores utilized Pinpoints issues to the source code line 37

38 Intel Thread Profiler S INT IES O P PIN ICIENC F F E IN S INT IES O P C PIN ICIEN FF E N I 38

39 Intel Threading Building Blocks Some kind of STL for Parallel C++ Programming You specify task patterns instead of threads Library maps user-defined logical tasks onto physical threads, efficiently using cache and balancing load Full support for nested parallelism Targets threading for robust performance Designed to provide scalable, portable performance for computationally intense portions of shrink-wrapped applications. Compatible with other threading packages Designed to work well for CPU bound computation, not I/O bound or real-time. Library can be used in concert with other threading packages such as native threads and OpenMP. Emphasizes scalable, data parallel programming Solutions based on functional decomposition usually do not scale. 39

40 An Example using ParallelFor Independent iterations and fixed/known bounds const int N = ; void change_array(float array, int M) { for (int i = 0; i < M; i++){ array[i] *= 2; } } int main (){ float A[N]; initialize_array(a); change_array(a, N); return 0; } 40

41 An Example using ParallelFor Include and initialize the library Include Library Headers #include <tbb/taskschedulerinit.h> #include <tbb/blockedrange.h> #include <tbb/parallelfor.h> using namespace ThreadingBuildingBlocks; int main (){ TaskSchedulerInit init; float A[N]; initialize_array(a); parallel_change_array(a, N); Initialize scheduler return 0; } Use namespace blue = original code red = provided by TBB black = boilerplate for library 41

42 An Example using ParallelFor Use the ParallelFor pattern Define Task blue = original code red = provided by TBB black = boilerplate for library class ChangeArrayBody { float *array; public: ChangeArrayBody (float *a): array(a) {} void operator()( const BlockedRange <int>& r ) const{ for (int i=r.begin(); i!=r.end(); i++ ){ array[i] *= 2; } } }; void parallel_change_array(float *array, int M) { ParallelFor (BlockedRange <int>(0, M, IdealGrainSize), ChangeArrayBody(array)); } Use Pattern Establish grain size 42

43 Intel Threading Building Blocks overview Generic Parallel Algorithms parallel_for parallel_while parallel_reduce pipeline parallel_sort parallel_scan Concurrent Containers concurrent_hash_map concurrent_queue concurrent_vector Task scheduler Low-Level Synchronization Primitives atomic spin_mutex queuing_mutex spin_rw_mutex mutex Memory Allocation cache_aligned_allocator scalable_allocator Timing tick_count 43

44 Intel Threading Building Blocks Programming vs. OS threads Programming Intel TBB Parallel Work POSIX* threads void parallel_thread (void *arg) { int y1, y2; while (schedule_thread_work (y1, y2)) { for (int y = y1; y <= y2; y++) { for (int x=startx; x<=stopx; x++) { render_one_pixel (x, y); } } if (scene.displaymode == RT_DISPLAY_ENABLED) { pthread_mutex_lock (&MyMutex3); for (int y = y1; y <= y2; y++) { GraphicsDrawRow(startx-1, y-1, totalx, (unsigned char *) &global_buffer[(y-starty)*totalx*3]); } pthread_mutex_unlock (&MyMutex3); } } } #include "tbb/parallelfor.h" #include "tbb/blockedrange2d.h" class parallel_task { public: void operator() (const TBB::BlockedRange2D<int> &r) const { for (int y = r.rows().begin(); y!= r.rows().end(); ++y) { for (int x = r.cols().begin(); x!= r.cols().end(); x++) { render_one_pixel (x, y); } } if (scene.displaymode == RT_DISPLAY_ENABLED) { TBB::SpinMutex::scoped_lock lock (MyMutex2); for (int y = r.rows().begin(); y!= r.rows().end(); ++y) { GraphicsDrawRow(startx-1, y-1, totalx, (unsigned char *) &global_buffer[(y-starty)*totalx*3]); } } } parallel_task () {} }; ParallelFor (TBB::BlockedRange2D<int> (starty, stopy + 1, grain_size, startx, stopx + 1, grain_size), parallel_task ()); #include "tbb/parallelfor.h" #include "tbb/bl ockedrange2d.h" ParallelFor (TBB::BlockedRange2D<int> (starty, stopy + 1, grain_size, startx, stopx + 1, grain_size), parallel_task ()); const int MINPATCH = 150; const int DIVFAC TOR = 2; typedef struct work_queue_entry_s { patch pch; struct work_queue_en try_s *next; } work_queue_ entry_t; work_q ueue_en try_t *work_queue_ head = NULL; work_q ueue_en try_t *work_queue_ tail = NULL; void generate_work (patch* pchin) { int startx, stopx, starty, stopy; int xs,ys; startx=pchin- >startx; stopx= pchin->stopx; starty=pchin- >starty; Data Decomposition stopy= pchin->stopy; if(((stopx-startx) >= MINPATCH) ((stopy-starty) >= MINPATCH)) { int xpatchsize = (stopx-startx)/divfactor + 1; int ypatchsize = (stopy-starty)/divfactor + 1; for (ys=starty; ys<=stopy; ys+=ypatchsize) for (xs=startx; xs<=stopx; xs+=xpatchsize) { patch pch; pch.startx = xs; pch.starty = ys; pch.stopx = MIN(xs+xpatchsize-1,stopx); pch.stopy = MIN(ys+ypatchsize-1,stopy); generate_work (&pch); } } else { /* just trace this patch */ work_queue_en try_t *q = (work_queue_ entry_t *) malloc (sizeof (work_q ueue_ent ry_t)); q->pch.starty = starty; q->pch.stopy = stopy; q->pch.startx = startx; q->pch.stopx = stopx; q->next = NULL; if (work_queue_he ad == NULL) { work_q ueue_h ead = q; } else { work_q ueue_t ail->next = q; } work_queue_t ail = q; } } void generate_worklist (void) Intel TBB offers cleaner design and competitive performance { patch pch; pch.startx = startx; pch.stopx = stopx; pch.starty = starty; pch.stopy = stopy; generate_w ork (&pch); } bool schedule_thread _work (patch &pch) { pthread_mutex_lock (&MyMutex3); work_q ueue_ent ry_t *q = work_queue_head; if (q!= NULL) { pch = q->pch; work_queue_head = work_queue_ head->next; } pthread_mutex_unloc k (&MyMutex3 ); return (q!= NULL); } generate_worklist (); 44

45 Performance Libraries

46 Intel Math Kernel Library 8.1 (MKL) Multi-core ready Thread Safe Excellent scaling on multiprocessor systems Automatic runtime processor detection Support for C and Fortran interfaces Support for all Intel processors in one package Royalty-free distribution rights BLAS LAPACK ScaLAPACK Supports Intel MKL Sparse Solvers Fast Fourier Transforms Vector Math Windows* Linux* Mac OS* 64-Bit Multicore AMD* 46 "By adopting the Intel Intel MKL DGEMM libraries, our standard benchmarks timing improved between 43% and 71%, which is very impressive." Matt Dunbar Software Developer ABAQUS, Inc.

47 Intel Integrated Performance Primitives (IPP) Application Source Code Intel IPP Usage Code Samples Rapid Application Development Sample video/audio/speech codecs Image processing and JPEG Signal processing Data compression.net and Java* integration API calls "The Intel IPP [Intel Integrated Performance Primitives] is the fastest image processing library we've found, resulting in much greater interactivity and creative freedom for our users." Intel IPP Library C/C++ API Video coding Audio coding Speech coding Speech recognition Data compression Cryptography Matrix maths Cross-platform Compatibility & Code Re-Use Signal processing Image processing JPEG coding Computer vision Image colour conversion String processing Vector maths Static/Dynamic Link Intel IPP Processor-Optimized Binaries Intel Intel Intel Intel Intel Intel Intel Core Duo and Core Solo Processors Pentium D dual-core Processors Pentium M Processors Pentium 4 Processors Xeon Processors Itanium 2 Processors XScale Technology-based Processors Outstanding Performance Supports Windows* Linux* Mac* Intel IPP 64-Bit Multicore AMD* 47 Bruce Rady President RadTIME, Inc.

48 Intel Cluster tools Optimize MPI based applications

49 What Are the Biggest Bottlenecks Today in Creating Parallel Applications? Source: Developing Custom Parallel Computing Applications, Simon Management Group, September

50 Intel Cluster Toolkit Boost development and performance of cluster applications Universal MPI Library runs cluster applications on all networks Leading cluster development environment to efficiently create, analyze, optimize and deploy parallel applications Ready to support dual-core and multi-core cluster Intel Cluster Toolkit 3.0 Full-featured MPI tools environment Intel MPI Library 3.0 Intel Trace Analyzer and Collector 7.0 Intel Math Kernel Library Cluster Edition 9.0 Intel MPI Benchmarks

51 Summary Intel Software Development Products Lead the Way with support for the latest Operating Systems and Multi-core Processors Intel VTune Analyzer v9.0 Intel Core 2 processor event support and Hotspot navigator Intel Thread Profiler and Checker v3.1 Speed and usability improvements Intel Threading Building Blocks (Intel TBB) v1.1 Automatic grainsizes Intel Performance Libraries Speed improvements Intel Cluster Tools v3.0 Build and Optimize MPI based applications 51

52 Any Questions? 52