Revealing the performance aspects in your code

Transcription

1 Revealing the performance aspects in your code 1

2 Three corner stones of HPC The parallelism can be exploited at three levels: message passing, fork/join, SIMD Hyperthreading is not quite threading A popular strategy Nodes can have different is choose frequency, one Memory of is typically first non-uniform two and too let compiler do arch, the # of cores vectorization and SIMD width Latency and bandwidth are different between different nodes This view is functional, but performance agnostic Caches have many levels of hierarchy with own latencies and rules Different instructions may be available for different types (SP vs. DP) Alignment requirements can be strict and contradict with other constraints Software and Services Group 2

3 First Things First!

4 Get Base Line Performance (I/III) Single node: Xeon E5 and Xeon Phi separate Software and Services Group 4

5 Get Base Line Performance (II/III) Heterogeneous: Xeon E5 + Xeon Phi Software and Services Group 5

6 Get Base Line Performance (III/III) Heterogeneous Cluster: N*(Xeon E5 + Xeon Phi ) Software and Services Group 6

7 Base Line Results Analysis For 1 and 2 nodes we get a 2X Speedup for the heterogeneous version vs. Xeon E5. For higher node numbers Xeon E5 shows a super linear speedup while Xeon E5 + Xeon Phi saturates. Potential reasons may be: - message passing performance - changing load balance - sub optimal vectorization for Xeon Phi Software and Services Group 7

8 A few BKMs to Try!

9 BKM: Tune programs with affinity set Problem: results are not stable b/w runs Solution: use KMP_AFFINITY to bind threads Compact Scatter Balanced Software and Services Group 9

10 Check Scalability: Xeon & Xeon Phi Using timer functions is quick and easy! Xeon performance doesn t scale beyond 1 socket. Need to be investigated! 100T run on 60C KNC gives Best Performance. Why? 10

11 Loop Profiler - Identify Time Consuming Loops / Functions to Optimize Enables targeting parallelization/optimization efforts to most significant code areas ( hotspot identification ) Easy to use: Use compiler switches to add instrumentation to the application Compiler instruments entry and exits of all loops and functions icc -O1 -profile-functions -profile-loops=all -profile-loops-report=2 Running the application generates a report file with resulting counts Both a human-readable text file (a table) and an XML-file are generated Analyze data by looking at the raw text file, or use the GUI viewer shipped with compiler Report file contains information such as: Call count of routines Self-time of functions / loops Total-time of functions / loops Average, minimum, maximum iteration counter of loops!! 11

12 Loop Profile Data Viewer GUI 12

13 Original Parallel Version: Not vectorized Check Compiler VEC Reports! 10 static inline size_t postoidx(const size_t width, const Position& pos){ 11 return (pos.y * width) + pos.x; 12 } static void subtractpsf(const float* psf,...) { #pragma omp parallel for default(shared) 26 for (int y = starty; y <= stopy; ++y) { 27 for (int x = startx; x <= stopx; ++x) { 28 residual[postoidx(residualwidth, Position(x, y))] -= gain * abspeakval 29 * psf[postoidx(psfwidth, Position(x - diffx, y - diffy))]; 30 } 31 } 32 } Compiler can t identify the index pattern Compiler unable to vectorize the loop at line 27. Index compute complex! 13 13

14 Replacing OpenMP critical section: With a serial loop reduction loop Better Scalability! Original Code Optimized Code #pragma omp parallel { float threadabsmaxval = 0.0; int threadabsmaxpos = 0; Compute per thread Peak #pragma omp parallel { float threadabsmaxval = 0.0; size_t threadabsmaxpos = 0; Compute per thread Peak #pragma omp for schedule(static) for (int i = 0; i < size; ++i) { if (abs(image[i]) > (threadabsmaxval)) { threadabsmaxval = abs(image[i]); threadabsmaxpos = i; } #pragma omp critical if (threadabsmaxval > maxval) { maxval = threadabsmaxval; maxpos = threadabsmaxpos; } } maxval = image[maxpos]; Find global peak across all threads Performance Gain: 1.12X } #pragma omp for schedule(static) for (int i = 0; i < size; ++i) { if (abs(image[i]) > (threadabsmaxval)) { threadabsmaxval = abs(image[i]); threadabsmaxpos = i; } int t_num = omp_get_thread_num(); temp_peak[t_num] = threadabsmaxval; temp_pos[t_num] = threadabsmaxpos; for (int k = 0; k < num_t ; k++) { if ((temp_peak[k]) > (maxval)) { maxval = temp_peak[k]; maxpos = temp_pos[k]; } } maxval = image[maxpos]; Store per thread Peak Find global peak across all threads # KNC 48C 14

15 Profiling MPI + OpenMP Heterogeneous Execution

16 Profile using Intel Cluster Studio XE 10 MPI Xeon + 22 MPI Xeon Phi x 11 OpenMP Unbalanced Load 16

17 Profile using Intel Cluster Studio XE 12 MPI Xeon + 20 MPI Xeon Phi x 12 OpenMP Balanced Load 17

18 Profiling using Intel VTune Amplifier XE 18

19 Intel VTune Amplifier XE Performance Profiler Where is my application Spending Time? Wasting Time? Waiting Too Long? Focus tuning on functions taking time See call stacks See time on source See cache misses on your source See functions sorted by # of cache misses See locks by wait time Red/Green for CPU utilization during wait Windows & Linux Low overhead No special recompiles Advanced Profiling For Scalable Multicore Performance 19

20 Intel VTune Amplifier XE Tune Applications for Scalable Multicore Performance Fast, Accurate Performance Profiles Hotspot (Statistical call tree) Call counts (Statistical) Hardware-Event Sampling Thread Profiling Visualize thread interactions on timeline Balance workloads Easy set-up Pre-defined performance profiles Use a normal production build Find Answers Fast Filter extraneous data View results on the source / assembly Compatible Microsoft, GCC, Intel compilers C/C++, Fortran, Assembly,.NET, Java Latest Intel processors and compatible processors 1 Windows or Linux Visual Studio Integration (Windows) Standalone user i/f and command line 32 and 64-bit 1 IA32 and Intel 64 architectures. Many features work with compatible processors. Event based sampling requires a genuine Intel Processor. 20

21 Intel VTune Amplifier XE Get a quick snapshot 4 cores CPU Usage Thread Concurrency 21 21

22 Intel VTune Amplifier XE Identify hotspots Hottest Functions Hottest Call Stack Quickly identify what is important 22

23 Intel VTune Amplifier XE Identify threading inefficiency Coarse Grain Locks High Lock Contention Low Concurrency Load Imbalance 23

24 Intel VTune Amplifier XE Find Answers Fast Adjust Data Grouping (Partial list shown) Click [+] for Call Stack Double Click Function to View Source Filter by Timeline Selection (or by Grid selection) Filter by Module & Other Controls 24

25 Intel VTune Amplifier XE Timeline Visualizes Thread Behavior Locks & Waits Transitions Hotspot s CPU Time Lightweight Hotspots Hovers: Optional: Use API to mark frames and user tasks Optional: Add a mark during collection 25

26 Intel VTune Amplifier XE See Profile Data On Source / Asm Time on Source / Asm Quick Asm navigation: Select source to highlight Asm Quickly scroll to hot spots. Right click for instruction reference manual Intel VTune Amplifier XE Click jump to scroll Asm 26

27 High-level Features 27

28 Intel VTune Amplifier XE Feature Highlights Basic Hot Spot Analysis (Statistical Call Graph) Locates the time consuming regions of your application Provides associated call-stacks that let you know how you got to these time consuming regions Call-tree built using these call stacks Advanced Hotspot and architecture analysis Based on Hardware Event-based Sampling (EBS) Pre-defined tuning experiments Thread Profiling Visualize thread activity and lock transitions in the timeline Provides lock profiling capability Shows CPU/Core utilization and concurrency information GPU Compute Performance Analysis Collect GPU data for tuning OpenCL applications. Correlate GPU and CPU activities CPU Power Efficiency Analysis Wake-up rate and frequency measurement per Core 28

29 Intel VTune Amplifier XE Feature Highlights Attach to running processes Hotspot and Concurrency analysis modes can attach to running processes System wide data collection EBS modes allows system wide data collection and the tool provides the ability to filter this data GUI Standalone GUI available on Windows* and Linux Microsoft* Visual Studio integration Command Line Comprehensive support for regression analysis and remote collection Platform & application support Windows * and Linux (Android, Tizen, Yocto in the ISS) Microsoft *.NET/C# applications Java * and mixed applications Fortran applications 29

30 Intel VTune Amplifier XE Feature Highlights Event multiplexing Gather more information with each profiling run Timeline correlation of thread and event data Populates thread active time with event data collected for that thread Ability to filter regions on the timeline Advanced Source / Assembler View See event data graphed on the source / assembler View and analyze assembly as basic blocks Review the quality of vectorization in the assembly code display of your hot spot Provides pre-defined tuning experiments Predefined profiles for quick analysis configuration A user profile can be created on a basis of a predefined profile User API Rich set of user API for collection control, events highlighting, code instrumentation, and visualization enhancing. 30

31 Data Collectors and Analysis Types 31

32 Intel VTune Amplifier XE Analysis Types (based on technology) Software Collector Any x86 processor, any virtual, no driver Basic Hotspots Which functions use the most time? Concurrency Tune parallelism. Colors show number of cores used. Locks and Waits Tune the #1 cause of slow threaded performance waiting with idle cores. Hardware Collector Higher res., lower overhead, system wide Advanced Hotspots Which functions use the most time? Where to inline? Statistical call counts General Exploration Where is the biggest opportunity? Cache misses? Branch mispredictions? Advanced Analysis Dig deep to tune bandwidth, cache misses, access contention, etc. 32

33 Intel VTune Amplifier XE Pre-defined Analysis Types Advanced Hotspot analysis based on the underlying architecture User mode sampling, Threading, IO, Signaling API instrumentation 3 rd Generation Core Architecture (a.k.a SandyBridge) analysis types 4 th Generation Core Architecture (a.k.a Haswell) analysis types 33

34 GUI Layout 34

35 Creating a Project GUI Layout

36 Selecting type of data collection GUI Layout All available analysis types Different ways to start the analysis Helps creating new analysis types Copy the command line to clipboard 36

37 Profile a Running Application No need to stop and re-launch the app when profiling Two Techniques: Attach to Process: - Any type of analysis Profile System: - Advanced Hotspots & Custom EBS - Optional: Filter by process after collection 37

38 Summary View GUI Layout Clicking on the Summary tab shows a high level summary of the run Timing for the whole application run List of 5 Hotspot functions CPU Usage 38

39 Bottom-Up View GUI Layout Menu and Tool bars Analysis Type Current grouping Viewpoint currently being used Tabs within each result Grid area Stack Pane Filter area Timeline area 39

40 Top-Down View GUI Layout Clicking on the Top- Down Tree tab changes stack representation in the Grid Top-level function and it s tree Total Time (self + children s) Self Time 40

41 Caller/Callee View GUI Layout Select a function in the Bottom-Up and find the caller/callee List of functions sorted by CPU Time List of callers and their stacks List of callees and their stacks 41

42 Results Comparison 42

43 Intel VTune Amplifier XE Terminology Elapsed Time The total time your target application ran. Wall clock time at end of application Wall clock time at start of application CPU Time The amount of time a thread spends executing on a logical processor. For multiple threads, the CPU time of the threads is summed. Wait Time The amount of time that a given thread waited for some event to occur, such as: synchronization waits and I/O waits 43

44 Intel VTune Amplifier XE CPU Usage Thread1 Waiting Thread1 Thread2 Waiting Thread2 Thread3 Waiting Thread3 Thread running 1sec 1sec 1sec 1sec 1sec 1sec Thread waiting Elapsed Time: 6 seconds CPU Usage CPU Time: T1 (4s) + T2 (3s) + T3 (3s) = 10 seconds Wait Time: T1(2s) + T2(2s) + T3 (2s) = 6 seconds

45 CPU Usage Summary View: CPU Usage Histogram Only CPU Time measured Bottom-Up View: CPU Time Wait Time is not counted in Hotspots Function CPU Time By CPU Utilization My_Func() 10 s 45

46 Overhead and spin Threading library internals Thread1 Waiting lib Thread1 Thread2 Waiting lib Thread2 Thread3 Waiting user Thread3 code lib running or spin Spin wait Thread running 1sec 1sec 1sec 1sec 1sec 1sec Thread waiting Elapsed Time: 6 seconds CPU Usage CPU Time: T1 (4s) + T2 (3s) + T3 (3s) = 10 seconds Wait Time: T1(2s) + T2(2s) + T3 (2s) = 6 seconds Overhead and spin Time: T1(1s) + T2(1s) + T2(1s) = 3 s

47 CPU Usage Summary View: CPU Usage Histogram Overhead and Spin Time is not counted for CPU Usage Bottom-Up View: CPU Time Function CPU Time By CPU Utilization My_Func() 10 s 4 s Overhead and Spin Time 47

48 Hotspot analysis Displays hot functions in your application Shows most time consuming call sequences Statistical Call Graph Include timeline view of threads in your application Basic Hotspot Analysis Start the Analysis 48

49 Hotspot analysis Summary Note Elapsed Time and CPU Time 49

50 Hotspot analysis Summary (Continued) Note overall CPU Usage Note # of CPUs Available on the platform 50

51 Hotspots analysis Hotspot functions Adjust Data Grouping Hotspot Functions Change Viewpoint (Partial list shown) Function CPU time Click [+] for Call Stack Thread timeline Call stack Filter by Module & Other Controls Filter by Timeline Selection (or by Grid Selection) 51

52 Hotspots analysis Hotspot functions by CPU usage Double Click Function to View Source Coloring CPU Time by CPU Utilization Overhead and Spin Time Overhead and Spin on Timeline 52

53 Hotspots analysis Source View Source View Assembly View Self and Total Time on Source / Asm Right click for instruction reference manual Quick Asm navigation: Select source to highlight Asm Click jump to scroll Asm Quickly scroll to hot spots. Scroll Bar Heat Map is an overview of hot spots 53

54 Advanced Hotspot analysis Uses Intel s CPU hardware performance collectors Higher resolution of sampling (~1 /ms) System wide analysis (all processes running in a system) OS modules and drivers profiling (ring 0 level) OS context switches and threads synchronization issues Start the Analysis Advanced Hotspot Analysis Select level of data collected 54

55 Parallelism Methodology of performance profiling and tuning How to optimize the Hotspots? Maximize CPU utilization and minimize elapsed time Ensure CPU is busy all the time All Cores busy parallelism (high concurrency) Elapsed (Serial) Elapsed (N-threads) Elapsed (Serial) / N Serial T1 T2 T3 T4 4T optimal Potential Gain Gain Time Time Time 55

56 Intel VTune Amplifier XE Terminology Concurrency - Is a measurement of the number of active threads Thread1 Waiting Thread1 Thread2 Waiting Thread2 Concurrency Summary Thread3 Waiting Thread Thread running 1sec 1sec 1sec 1sec 1sec 1sec Thread waiting 56

57 Intel VTune Amplifier XE Parallelism/Concurrency Analysis For Parallelism / Concurrency analysis, Stack sampling is done just like in Hotspots analysis Wait functions are instrumented (e.g. WaitForSingleObject, EnterCriticalSection) Signal functions are instrumented (e.g. SetEvent, LeaveCriticalSection) I/O functions are instrumented (e.g. ReadFile, socket) Concurrency Analysis Start the Analysis 57

58 Concurrency Analysis Summary Concurrency Levels Adjustable Metrics 58

59 Concurrency Analysis Summary: Concurrency vs. CPU Usage Histogram Threads might be in active state, but not using CPU 11/26/

60 Concurrency View Concurrency Level Overhead Wait Thread is running Thread is waiting Thread Transitions Overhead 60

61 Concurrency Timeline Investigate reasons for transitions Select and Zoom Hover over a transition line 61

62 Source Code View by Concurrency Concurrency coloring for CPU Time against source lines 62

63 Waiting on locks Sync Object Sync Object Sync Object Signal Signal Signal Thread1 Waiting Thread1 Idle Thread2 Waiting Thread2 Idle Thread3 Waiting Thread3 Thread running 1sec 1sec 1sec 1sec 1sec 1sec Thread waiting Begin main thread Calculating Wait and Idle time End main thread 63

64 Intel VTune Amplifier XE Locks and Waits Analysis Identifies those threading items that are causing the most thread block time Synchronization locks Threading APIs I/O Start the Analysis Locks & Waits Analysis 64

65 Locks and Waits View Grouping by Sync Object Wait Objects CPU Utilization Waits # Spinning Stack for the wait object 65

66 Locks-and-Waits Source View Wait count Critical Section object Waiting time on the Critical Section 66

67 Intel VTune Amplifier XE User APIs User APIs Collection Control API Thread Naming API User-Defined Synchronization API Task API User Event API Frame API JIT Profiling API 67

68 Windows & Linux Versions Available Stand-alone GUI, Command line, Visual Studio Integration Microsoft Windows* OS Windows XP* 1, Windows 7*, Windows 8 Desktop* Windows Server* 2003, 2008 Microsoft Visual Studio* 2008, 2010 and 2012 Standalone GUI and command line IA32 and Intel 64 Linux* OS RHEL*, Fedora*, SUSE*, CentOS*, Ubuntu* Additional distributions may also work Standalone GUI and command line IA32 and Intel 64 Single user and floating licenses available 68

69 Intel VTune Amplifier XE Command Line Interface - Examples Display a list of available analysis types and preset configuration levels amplxe-cl collect-list Run Hot Spot analysis on target myapp and store result in default-named directory, such as r000hs amplxe-cl c hotspots -- myapp Run the Parallelism analysis, store the result in directory r001par amplxe-cl -c parallelism -result-dir r001par -- myapp 69

70 Intel VTune Amplifier XE Command Line Interface Gropof-like output 70

71

72 Legal Disclaimer & Optimization Notice INFORMATION IN THIS DOCUMENT IS PROVIDED AS IS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. Copyright, Intel Corporation. All rights reserved. Intel, the Intel logo, Xeon, Core, VTune, and Cilk are trademarks of Intel Corporation in the U.S. and other countries. Optimization Notice Intel s compilers may or may not optimize to the same degree for non-intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision #