Analyzing Performance and Power of Applications on GPUs with Dell 12G Platforms. Dr. Jeffrey Layton Enterprise Technologist HPC

Transcription

1 Analyzing Performance and Power of Applications on GPUs with Dell 12G Platforms Dr. Jeffrey Layton Enterprise Technologist HPC

2 Why GPUs? GPUs have very high peak compute capability! 6-9X CPU Challenges How feed enough data? Need to port applications! Example: Tesla M2090 GPU Cores 512 Memory 6 GB Memory BW GB/s Peak Performance Single Precision 1331 GFLOPs Double Precision 665 GFLOPs Tesla M2090 GPU 2

3 How can they be used? M610x Inside the server Limited Space, few can fit Limited Power, few can run Difficult to replace Outside the server Pros: Flexibility, Multiple GPUs GPUs can be shared Multiple Host Servers Cons: Oversubscription may limit performance Host GPU C410x 3

4 The Problem: Best Design Parameters are Unknown How many GPUs per server is ideal for my application? How much bandwidth do I need per GPU for a typical users? How does performance scale with increasing number of nodes? How does performance scale with increasing number of GPUs/node? What problem size is most suitable for GPU computing? What is the impact on power consumption and performance/watt? Etc. Etc. Even if you know some of your design parameters, They may change due to improved GPUs, CPUs, GPU drivers, Software/Algorithm redesign etc. 4

5 GPU Enabled Product Portfolio

6 Overview GPU enabled products throughput the portfolio Learn on a laptop Develop/Test on workstations Production on servers 6

7 Laptops Learn GPU programming Buisness laptop: E6520 Nvidia NVS 4200M XPS 15: 48 CUDA cores 512MB memory Nvidia Geforce GT540M 96 CUDA cores 2GB memory 7

8 Workstations Develop/Tune Applications Portable Workstation M display (1920 x 1080) Up to 16GB memory Intel quad-core i7 (Ivy Bridge coming soon) Up to 3 hard drives Quadro 3000M 240 CUDA cores 2GB GDDR5 memory Quadro 5010M 96 CUDA cores 4GB GDDR5 memory T7500: Tower Case Dual-socket Intel Westmere (X56-- processors) Up to 192GB memory (12 DIMM slots) Two PCIe Gen2 x16 slots Five internal SATA or SAS drives RAID cards available GigE on-board, optional 10GigE cards 8

9 Rackable Workstation Develop/Tune R5500: 2U rackable workstation Dual-socket Westmere 12 DIMM slots (up to 192GB) Up to 5 SATA or 6 SAS drives (2.5 ) Two PCIe Gen2 x16 slots Tesla C2070 is an example 9

10 Dell M610x blade Half-height blade (5U) 2S Westmere 12 DIMM slots (192GB memory) Mezz card for QDR IB, 10GigE One double-wide GPU per blade 10

11 Dell PowerEdge C410x Power & Flexibility Basically, Room and board for 16 GPUs Theoretical Max. of 16.5 TFLOPs Connects up to 8 hosts Connects up to 16 PCIe Gen-2 devices (GPGPUs) to hosts High density, 3U chassis. Flexibility to selecting number of GPGPUs Individually serviceable modules N W Power supplies (3+1) N+1 92mm Cooling fans (7+1) PCIe switches 8 PEX PEX

12 Dell PowerEdge C410x Sixteen (16) x16 Gen-2 Modules - PCIe Gen-2 x16 compliant - Independently serviceable LED and On/Off GPU card Power connector for GPGPU card Board-to-board connector for X16 Gen 2 PCIe signals and power Dell Research Computing 1

13 Learn more about Dell PEC C410x How can you dynamically allocate GPUs to host nodes using the Dell PEC C410x? Learn more at session S0309 (Thursday 10:30 Room K) Dynamically Allocating GPGPU to Host Nodes (servers) 13

14 Dell PEC C6100: 4 2S in one chassis Four 2-Socket Nodes in 2U Intel Westmere-EP Each Node: 12 DIMMs each 2 GigE (Intel) 1 Daughter Card (PCIe x8) QDR IB or 10GigE One PCIe x16 (half-length, half-height) Optional SAS controller (in-place of IB) Chassis Design: Hot Plug, Individual Nodes Up to 12 x 3.5 drives (3 per node) Up to 24 x 2.5 drives (6 per node) N+1 Power supplies (1100W or 1400W) NVIDIA HIC certified 14

15 Dell PEC C6145: Two AMD 4S in 2U Two 4-Socket Nodes in 2U 4S AMD Opteron 6200 series Each Node: 32 x DDR3 RDIMMs 2 x GbE Intel 1 x8 Gen II (custom mezzanine slot) QDR IB or 10GigE 3 x16 Gen II (low-profile, half height/half-length) Chassis Design: Hot Plug, Individual Nodes 24 x 2.5 or 12 x 3.5 HDD Redundant Power supplies (1100W or 1400W) Embedded x16 HIC and slots for additional HICs 15

16 Dell PEC C6220 Four 2S Sandy Bridge in 2U Four 2-Socket Nodes in 2U Intel Sandy Bridge-EP Each Node: 16 DIMMs each 2 GigE (Intel) 1 Daughter Card (PCIe Gen 3 x8) FDR IB or QDR IB or 10GigE One PCIe G3 x16 (half-length, half-height) Two node version has two PCIe G3 x16 slots Optional SAS controller (in-place of IB) Chassis Design: Hot Plug, Individual Nodes Up to 12 x 3.5 drives or Up to 24 x 2.5 drives N+1 Power supplies (1100W or 1400W) NVIDIA HIC certified 16

17 Dell Power R720 First Standard Server with internal GPUs Two-socket Intel Sandy Bridge-EP 24 DIMM slots (up to 768GB) Dell Select Network Adapters: 4x GigE 2x10GigE + 2xGigE Intel or Broadcom 7 PCIe Gen 3 slots: Up to two internal GPUS (passive) PCIe G3 x8 slot for network adapter (e.g. FDR IB) Up to 4 front-access, hot-swap, PCIe drives Up to 16 drives HIC certified to work with Dell C410x 17

18 Performance and Power Measurements

19 Engineering Performing benchmarks/tests of various GPU applications: Different host nodes C6100, C6145 are called Dell 11G C6220 and R720 are called Dell 12G Different number of GPUs Internal and external GPUs Measured performance AND power during tests Goal is to understand how applications scale: Number and type of GPUs Host node configuration Develop best practices for GPU configurations 19

20 Applications HPL NAMD XFDTD 3D Oil Reservoir Simulation ANSYS Mechanical 20

21 Thanks!!!!! Dr. Saeed Iqbal Shawn Gao Onur Celebioglu Mark Fernandez, Glen Otero Nvidia Mass Fatica, Stan Posey, Peter Lillian, Bob Cravella, Travis Wells, et al 21

22 Dell 11G

23 Normalized Performance (GFLOPS) Nromalized Power (W) Normalized GFLOPS/W Dell PowerEdge C C410x HPL Normalized Results % 64.91% 56.33% 39.93% 18.18% % 64.91% 56.33% 39.93% 18.18% % 64.91% 56.33% 39.93% 18.18% CPU Only CPU + 1 CPU + 2 CPU + 4 CPU + 8 CPU Only CPU + 1 CPU + 2 CPU + 4 CPU + 8 CPU Only CPU + 1 CPU + 2 CPU + 4 CPU + 8 One HIC from host node to C410x Recommended no more than 2 GPUs per HIC 23 1-node PE C6100, Dual X5650@2.67GHz, 48GB, 1333MHz Memory; C410x has Nvidia s

24 Normalized Performance (GFLOPS) Normalized GFLOPS/W Dell PowerEdge C C410x HPL Normalized Results % % 45.11% 19.74% 32.02% 12.03% 1 HIC 2 HIC % 45.11% 35.85% 46.36% 19.74% 32.02% 12.03% 20.11% 1 HIC 2 HIC CPU Only CPU + 1 CPU + 2 CPU + 4 M207 CPU + 8 CPU Only CPU + 1 CPU + 2 CPU + 4 M207 CPU + 8 One or Two HICs from host node to C410x Recommend no more than 2 GPUs per HIC (1 per HIC is better) 24 1-node PE C6145, Four 6132HE@2.2GHz, 128GB, 1333MHz Memory; C410x has Nvidia s

25 Normalized Performance (day/ns) Normalized Power (W) Normalized Perf (1/days/ns)/W Dell PowerEdge C C410x NAMD Normalized Results CPU Only CPU + 1 CPU + 2 CPU + 4 CPU + 8 CPU Only CPU + 1 CPU + 2 CPU + 4 CPU + 8 CPU Only CPU + 1 CPU + 2 CPU + 4 CPU + 8 STMV data set (1M atoms) Two GPUs is best, 4 GPUs is also good (Perf/W). 1 HIC is good 25 1-node PE C6100, Dual X5650@2.67GHz, 48GB, 1333MHz Memory; C410x has Nvidia s

26 Normalized Performance (1/time) Dell PowerEdge C C410x XFDTD Normalized Performance GPUs are best 2-4 GPUs per x16 HIC CPU Only CPU + 1 CPU + 2 CPU + 4 CPU + 6 CPU Node PE C6100, Dual X5670@2.93GHz, 48GB, 1333MHz Memory; C410x has 16 GPUs

27 Normalized Performance (time) Normalized Power (W) Normalized Perf/W (Time*W) Dell PowerEdge C C410x 3D Oil Reservoir Simulation (Elastic Model) Normalized Results GPUs 2 GPUs 4 GPUs CPU CPU CPU CPU 8.28M/2 GPUS 16.57M/2 GPUs 33.15M/4 GPUs 8 GPUs 66.34M/8 GPUs Problem Size/Number of GPUs GPUS 2 GPUS 4 GPUs CPU CPU CPU CPU 8.28M/2 GPUS 16.57M/2 GPUs 33.15M/4 GPUs 8 GPUs 66.34M/8 GPUs Problem Size/Number of GPUs GPUs CPU CPU CPU CPU 2 GPUs 2 GPUs 8.28M/2 GPUS 16.57M/2 GPUs 33.15M/4 GPUs 8 GPUs 66.34M/8 GPUs Problem Size/Number of GPUs Multiple data sets Smaller data sets: 2 GPUs. Larger data sets: 4-8 GPUs (per x16 HIC) 27 Node PE C6100, Dual X5670@2.93GHz, 48GB, 1333MHz Memory; C410x has 16 GPUs

28 Dell 12G

29 Normalized Performance (GFLOPS) Normalized Power (W) Normalized GFLOPS/W Dell PowerEdge R720 HPL-Normalized Results % 60.2% % 60.2% % 67.8% 60.2% % % R720 CPU only R M2090 R M2090 R720 CPU only R M2090 R M2090 R720 CPU only R M2090 Internal GPUs 2 GPUs seems like a good configuration (individual x16 slot) R M R720, Dual Intel E5-2660@2.2GHz, 64GB, 1333MHz Memory (8x 8GB); two internal M2090 GPUs

30 GFLOPs/Watts Compare GFLOPS/watt (Cap 225W) Comparisons ( C6100 & R720) C6100 ( 2.93GHz ) R720 ( 2.7GHz ) R720 ( 2.2GHz ) M2090 GPUs with 225W power capping

31 Normalized Performance (GFLOPS) Normalized Power (W) Normalized GFLOPS/W Dell PowerEdge C C410x HPL - Normalized Results % % 58.7% 34.4% % 58.7% % 67.2% 58.7% 34.4% % % CPU only CPU+1 M2090 CPU+2 M2090 CPU+4 M2090 CPU only CPU+1 M2090 CPU+2 M2090 CPU+4 M2090 CPU only CPU+1 M2090 CPU+2 M2090 CPU+4 M2090 External GPUs with single HIC Two GPUs seems to be sweet spot (two GPUs with 1 x16 HIC) 31

32 Normalized Performance (GFLOPS) Normalized Performance (GFLOPS) Comparison of Internal vs. External HPL - Normalized Performance Not quite apples-to-apples (PCIe G2 to G3) % % 34.4% % % % % R720 CPU only R M2090 R M2090 CPU only CPU+1 M2090 CPU+2 M2090 CPU+4 M

33 Normalized GFLOPS/W Normalized GFLOPS/W Comparison of Internal vs. External HPL - Normalized GFLOPS/W Not quite apples-to-apples (PCIe G2 to G3) % 67.8% 60.2% % 67.2% 58.7% 34.4% R720 CPU only R M2090 R M2090 CPU only CPU+1 M2090 CPU+2 M2090 CPU+4 M2090 Internal GPUs are more effective (perf/w) but external are still very good 33

34 Normalized Performance (Time) Normalized Perf/W - (1/time)/W Dell PowerEdge R720 ANSYS Mechanical - Normalized Results 1 core Core 1 Core + 1 M Core 1 Core + 1 M V14cg-1V14sp-1V14sp-2V14sp-3V14sp-4V14sp-5V14sp-6 Massive speedup with GPU (up to 3.6x) but not all cases Perf/W (efficiency) can be quite good (2 times better) 34

35 Normalized Performance (Time) Normalized Perf/W - (1/time)/W Dell PowerEdge R720 ANSYS Mechanical - Normalized Results 2 cores Cores 2 Cores + 1 M Cores 2 Cores + 1 M Less speedup than 1 core case Perf/W (efficiency) is still very good 35

36 Normalized Perforamnce (Time) Normalized Perf/W - (1/T)/W Dell PowerEdge R720 ANSYS Mechanical - Normalized Results 4 cores Cores 4 Cores + 1 M Cores 4 Cores + 1 M V14cg-1V14sp-1V14sp-2V14sp-3V14sp-4V14sp-5V14sp-6 V14cg-1V14sp-1V14sp-2V14sp-3V14sp-4V14sp-5V14sp-6 Less speedup than 1 and 2 core cases 36

37 Normalized Performance (Time) Normalized Perf/W - (1/T)/W Dell PowerEdge R720 ANSYS Mechanical - Normalized Results 8 cores Cores 8 Cores + 1 M Cores + 2 M Cores 8 Cores + 1 M Added 2 GPU tests (not much impact on performance) Efficiency is not good except for 2 cases (less than 1) 37

38 Normalized Performance (Speedup) Normalized Perf/W - (1/T)/W Dell PowerEdge R720 ANSYS Mechanical - Normalized Results 16 cores Cores 16 Cores + 1 M Cores + 2 M Cores 16 Cores + 1 M2090 Very little speed improvement Efficiency with GPUs is worse than CPUs only 38

39 Normalized Performance (1/T) Normalized Perf/W - (1/T)/W Dell PowerEdge R720 ANSYS Mechanical Trends Choose last 3 cases (best usage of GPUs) V14sp V14sp V14sp-5 V14sp-6 CPU Only V14sp-5 V14sp-6 CPU Only Core 2 Cores 4 Cores 8 Cores 16 Cores 1 Core 2 Cores 4 Cores 8 Cores 16 Cores 39

40 ANSYS Mechanical observations As the number of cores increased: The GPU speedup decreases Efficiency decreases (at 16 cores it s not good) Cross-over point is 8 or 16 cores (8 is a good rule of thumb) Recommended configuration: Small CPU core count (no more than 4) with 1 GPU With 2 GPUs in node, you can run 2 cases at the same time (uses 8 cores) Performance varies with case (solver) 40

41 Summary Lots of options for GPU configurations which one is best? How do you define best? Performance? Power efficiency? Both? Answers vary (depend upon application) You don t have to have a dedicated x16 slot for each GPU for good performance and good efficiency Many applications shown here illustrate this Dell C410x allows GPU Direct for up to 8 GPUs Other systems do not allow this 41

42 Thanks! Questions?