2008 International ANSYS Conference

Transcription

1 28 International ANSYS Conference Maximizing Performance for Large Scale Analysis on Multi-core Processor Systems Don Mize Technical Consultant Hewlett Packard 28 ANSYS, Inc. All rights reserved. 1 ANSYS, Inc. Proprietary

2 Abstract For many years, the advances in computer design have followed Moore s Law, which states that the number of transistors on a single chip increases at a fixed rate. Recently, all of the major developers of high-performance computers have adopted two architectural approaches to implement Moore s Law: Multicore processor chips and larger caches on the processor chip. A Multi-core processor contains more than one CPU (also known as core). To the operating system and the application software, each core functions as an independent CPU. Multi-core chips allow more cores to be put on smaller, cheaper machines. Many more people today, who have parallel applications, can run with higher parallelism. Also the dynamics of running parallel on these Multi-Core Multi- Processor machines is different than running on parallel processing machines in the past. This paper will look at the best ways to run ANSYS, CFX, and FLUENT on some of these machines. 28 ANSYS, Inc. All rights reserved. 2 ANSYS, Inc. Proprietary

3 Structures versus CFD what s important ANSYS Structures Floating point power! I/O! Memory bandwidth and latency! Scalable up to 16!! FLUENT and CFX Floating point power! Interconnect latency and bandwidth! Memory latency and bandwidth! Highly scalable! 28 ANSYS, Inc. All rights reserved. 3 ANSYS, Inc. Proprietary

4 Agenda Interconnects GIGE vs. Infiniband Process placement Processes per node. I/O Cost Conclusion 28 ANSYS, Inc. All rights reserved. 4 ANSYS, Inc. Proprietary

5 Interconnects solver ratings Fluent medium size benchmark (4 million cells) on dl16 3 GHz 16 GB of memory GIGE vs. IB (bigger is better) sedan_4m IB sedan_4m GIGE ANSYS, Inc. All rights reserved. 5 ANSYS, Inc. Proprietary

6 25 Interconnects Fluent Truck benchmark (14million cells) on dl16 3Ghz 16GB memory GIGE vs. IB (bigger is better) solver ratings truck_poly_14m IB truck_poly_14m GIGE ANSYS, Inc. All rights reserved. 6 ANSYS, Inc. Proprietary

7 Interconnects 25 Fluent 6.3 running a medium benchmark (4 M cells) on dl165 16GB memory IB vs GIGE solver ratings sedan_4m IB sedan_4m GIGE ANSYS, Inc. All rights reserved. 7 ANSYS, Inc. Proprietary

8 Interconnects 4 Fluent truck benchmark (14 million cells) on dl165 16GB memory IB vs. GIGE (bigger is better) solver ratings truck_poly_14m IB truck_poly_14m GIGE ANSYS, Inc. All rights reserved. 8 ANSYS, Inc. Proprietary

9 Process placement Socket /Package Socket /Package 1 Core Core 1 Core 2 Core 3 Core Core 1 Core 2 Core Shared Cache Shared Cache Shared Cache Shared Cache Frontside Bus 1 Frontside Bus 2 Memory Controller Hub Processor Number seen by Linux 28 ANSYS, Inc. All rights reserved. 9 ANSYS, Inc. Proprietary

10 Process placement (CFX) 6 Process Placement Xeon Quadcore 5 Relative Performance default, no binding min cache, max bandwidth min cache, min bandwidth max cache, min bandwidth max cache, max bandwidth Number of Processes 28 ANSYS, Inc. All rights reserved. 1 ANSYS, Inc. Proprietary

11 4 Processes per node Fluent 6.3 medium benchmark (4 million cells) on bl46 3.2GHz 16GB memory (bigger is better) solver ratings sedan_4m 8 procs/node sedan_4m 6 procs/node sedan_4m 4 procs/node ANSYS, Inc. All rights reserved. 11 ANSYS, Inc. Proprietary

12 Processes per node 7 Fluent 6.3 truck benchmarks (14 million cells) on bl46 3.2GHz 16 GB memory (bigger is better) solver ratings truck_poly_14m 8 procs/node truck_poly_14m 6 procs/node truck_poly_14m 4 procs/node ANSYS, Inc. All rights reserved. 12 ANSYS, Inc. Proprietary

13 Processes per node Fluent 6.3 medium benchmark (4 million cells) on dl GHz 16 GB memory (bigger is better) 3 solver ratings sedan_4m 8 procs/node sedan_4m 6 procs/node sedan_4m 4 procs/node ANSYS, Inc. All rights reserved. 13 ANSYS, Inc. Proprietary

14 Processes per node Fluent 6.3 truck benchmark (14 million cells) on dl GHz 16GB memory (bigger is better) 4 solver ratings truck_poly_14m 8 procs/node truck_poly_14m 6 procs/node truck_poly_14m 4 procs/node ANSYS, Inc. All rights reserved. 14 ANSYS, Inc. Proprietary

15 Processes per node 25 ANSYS 11 on dl 16 3 GHz (bigger is better) solver ratings bmd-5 one proc/node bmd-5 two procs/node bmd-5 four procs/node bmd-5 eight procs/node ANSYS, Inc. All rights reserved. 15 ANSYS, Inc. Proprietary

16 Processes per node 5 ANSYS 11 on dl 16 3 GHz (bigger is better) solver ratings bmd-7 one proc /node bmd-7 two procs /node bmd-7 four procs /node bmd-7 eight procs/node ANSYS, Inc. All rights reserved. 16 ANSYS, Inc. Proprietary

17 Processes per node 6 ANSYS 11 SMP benchmarks (summed) on dl16 dual core 3.4GHz vs. dl16 quad core 3.GHz Seconds dual core quad core Threads 28 ANSYS, Inc. All rights reserved. 17 ANSYS, Inc. Proprietary

18 File I/O 2 ANSYS 11 bm7(smp) seconds bm-7 2 disk stripe bm-7 4 disk stripe threads ANSYS, Inc. All rights reserved. 18 ANSYS, Inc. Proprietary

19 Cost Nodes blades, rackmount, or SMP servers Licenses cost is by number of task and Interconnects GIGE - lower cost and performance Infiniband higher performance and cost 28 ANSYS, Inc. All rights reserved. 19 ANSYS, Inc. Proprietary

20 Cost - power 45 4 fl5l2 power measurement on dl14 quad core fl5l2 watts ANSYS, Inc. All rights reserved. 2 ANSYS, Inc. Proprietary

21 Cost - power 4K 35K fl5l2 cumulative power usage for dl14 quad core in one day fl5l2 kilowatts 3K 25K 2K 15K 1K 5K ANSYS, Inc. All rights reserved. 21 ANSYS, Inc. Proprietary

22 Cost total considerations Total cost = cost of nodes + cost of interconnect + cost of power + cost of license + cost of time 28 ANSYS, Inc. All rights reserved. 22 ANSYS, Inc. Proprietary

23 Conclusion The best ways to run ANSYS, CFX, and Fluent optimally are to take in consideration. Performance gains from a certain configuration Cost of the configuration What you or those with the purse strings want to spend. 28 ANSYS, Inc. All rights reserved. 23 ANSYS, Inc. Proprietary