Automated Design Exploration and Optimization + HPC Best Practices

Transcription

1 Automated Design Exploration and Optimization + HPC Best Practices 1

2 Outline The Path to Robust Design ANSYS DesignXplorer Mesh Morphing and Optimizer RBF Morph Adjoint Solver HPC Best Practices 2

3 The Path to Robust Design Robust Design is an ANSYS Advantage Single Physics Solution Accuracy, robustness, speed Multiphysics Solution Integration Platform What if Study Parametric Platform Design Exploration DOE, Response Surfaces, Correlation, Sensitivity, Unified reporting, etc. Optimization Algorithms Published API Robust Design Six Sigma Analysis Probabilistic Algorithms Adjoint solver methods 3

4 What If? Interactively adjust the parameter values and Update Needed for "What If?" Parametric CAD Connections Pervasive Parameters Persistent Updates Managed State, Update Mechanisms Remote Solve Manager (RSM) Parametric Persistence is an ANSYS Advantage! 4

5 Design Exploration Design Exploration is an ANSYS Advantage 5

6 Optimization Optimal Candidates 6

7 Six Sigma Analysis Thermal Stress Pressure & Flow Velocity Exhaust manifold design Six Sigma Analysis Input Parameters Outlet Diameter of the manifold Thickness at inlet External Temperature Engine RPM Parametric Geometry Maximum Displacement should not exceed 1.5 mm Response Parameters Max Flow Temperature Max Deformation Max Von-Mises stress Deformation All samples reports max deformation below 1.5 mm Uncertainty of input parameters Response Surface showing the effect of engine speed and thickness at outlet on the maximum deformation 7

8 ANSYS DesignXplorer 9

9 ANSYS DesignXplorer DesignXplorer is everything under this Parameter bar Low cost & easy to use! It drives Workbench Improves the ROI! ANSYS Workbench Solvers DX 10

10 Design of Experiments With little more effort than for a single run, you can use DesignXplorer to create a DOE and run many variations. 11

11 Correlation Matrix Understand how your parameters are correlated/influenced by other parameters! 12

12 Sensitivity Understand which parameters your design is most sensitive to! 13

13 Response Surface Understand the sensitivities of the output parameters (results) wrt the input parameters. 3D Response 2D Slices Response 14

14 Goal-Driven Optimization Use an optimization algorithm or screening to understand tradeoffs or discover optimal design candidates! 15

15 Robustness Evaluation Input parameters have variation! Make sure your design is robust! Six Sigma, TQM Output parameters vary also! 16 Understand how your performance will vary with your design tolerances? Predict how many parts will likely fail? Understand which inputs require the greatest control?

16 Example 1: Slit Die Need uniform outflow Minimize pressure drop P2 P3 P1 Flow Uniformity BAD Bad GOOD Good Pressure Drop 17

17 Example 2: Combustor 3 parameters Diffuser Length Exit Height Outlet Minimize pressure loss Outlet Minimize mach number Outlet Inlet Dump Gap Sensitivity 18

18 Mesh Morphing and Optimizer 19

19 Fluent Morpher-Optimization Feature Allows users to optimize product design based on shape deformation to achieve design objective Based on free-form deformation tool coupled with various optimization methods 20

20 Mesh Morphing Applies a geometric design change directly to the mesh in the solver Uses a Bernstein polynomial-based morphing scheme Freeform mesh deformation defined on a matrix of control points leads to a smooth deformation Works on all mesh types (Tet/Prism, CutCell, HexaCore, Polyhedral) User prescribes the scale and direction of deformations to control points distributed evenly through the rectilinear region. 21

21 Process What if? Setup Case Run Setup Morph Morph Evaluate OR Regions Parameters Deformation Optimizer Optimizer Setup Case Run Setup Optimizer Optimize Auto 22 Choose best design Optimal Solution

22 Deformation Definition Define constraint(s) (if any) Select control points and prescribe the relative ranges of motion 23

23 Objective Function Objective Function: Equal flow rate Baseline Design Optimized Design 24

24 Optimizer Algorithms; Compass, Powell, Rosenbrock, Simplex, Torczon Auto Optimize! 25

25 Example: L-Shaped Duct Application: L-shaped duct Objective Function: Uniform flow at the outlet Significant Improvement in Flow Uniformity 26

26 RBF Morph 27

27 RBF Morph Features The user-friendly RBF Morph add-on module is fully integrated within Fluent (GUI, TUI & solving stage) and Workbench Mesh-independent RBF fit used for surface mesh morphing and volume mesh smoothing Parallel calculation allows to morph large size models (many millions of cells) in a short time Management of every kind of mesh element type (tetrahedral, hexahedral, polyhedral, etc.) Ability to convert morphed mesh surfaces back into CAD Multi fit makes the Fluent case truly parametric (only 1 mesh is stored) Precision: exact nodal movement and exact feature preservation. 28

28 How Does RBF-Morph work? A system of radial functions is used to fit a solution for the mesh movement/morphing, from a list of source points and their prescribed displacements Radial Basis Function interpolation is used to derive the displacement at any location in the space The RBF problem definition is mesh independent. 29

29 RBF-Morph is Integrated with Fluent 30

30 Example 1: Internal Flow Here, a pipe is projected onto a previously defined STL shape 31

31 Example 2: External Flow courtesy of Ignazio Maria Viola Ship sail rotation 32

32 Example 3: External Flow 33

33 Example 4: 50:50:50 Optimisation Courtesy of Volvo Cars Recently conducted conceptual study by ANSYS in conjunction with Volvo Cars 50 Million cell hybrid mesh of Volvo XC60 50 Design variants investigated using RBF-Morph Addon for ANSYS Fluent and Workbench Design Explorer 50 hours total clock time to complete full optimisation on HPC Cluster ~4% Reduction in total drag force 34

34 Example 5: Ship Hull Optimisation Courtesy of Leeds University Ship hull hydrodynamics optimisation study Block hexahedral mesh from ICEM CFD 50 Design variants investigated using RBF-Morph Addon for ANSYS Fluent and Workbench Design Explorer 7.9% Reduction in total drag force 35

35 Adjoint Solver 36

36 Adjoint Solution? An adjoint solver allows specific information about a fluid system to be computed that is very difficult to gather otherwise. The adjoint solution itself is a set of derivatives. They are not particularly useful in their raw form and must be post-processed appropriately. The derivative of an engineering quantity with respect to all of the inputs for the system can be computed in a single calculation. Example: Sensitivity of the drag on an airfoil to its shape. There are 4 main ways in which these derivatives can be used: 1. Qualitative guidance on what can influence the performance of a system strongly. 2. Quantitative guidance on the anticipated effect of specific design changes. 3. Guidance on important factors in solver numerics. 4. Gradient-based design optimization. 37

37 How to Use the Results - Qualitative GOAL: Identify features of a system design that are most influential in the performance of the system. EXAMPLE: Sensitivity of the Drag on a NACA 0012 airfoil to changes in the shape of the airfoil. The shape sensitivity field is extracted from the adjoint solution in a post-processing step. High sensitivity changes to shape have a big effect on drag Low sensitivity changes to shape have a small effect on drag 38

38 How to Use the Results - Quantitative GOAL: Identify specific system design changes that benefit the performance and quantify the improvement in performance that is anticipated. EXAMPLE: Design modifications to turning vanes in a 90 degree elbow to reduce the total pressure drop. The optimal adjustment that is made to the shape is defined by the shape sensitivity field (steepest descent algorithm). Effect of each change can be computed in advance based on linear extrapolation. Baseline Modified Original P = Pa Expected change computed using the adjoint and linear extrapolation = 10.0 Pa Make the change and recompute the solution. Actual change = 9.0 Pa 39

39 How to Use the Results - Solver Numerics GOAL: Identify aspects of the solver numerics and computational mesh that have a strong influence on quantities that are being computed that are of engineering interest. EXAMPLE: Use the adjoint solution to identify parts of the mesh where mesh adaption will benefit the computed drag by reducing the influence of discretization errors. Baseline Mesh Adapted Mesh Adapted Mesh Detail 40

40 How to Use the Results - Optimization GOAL: Perform a sequence of automated design modifications to improve a specific performance measure for a system EXAMPLE: Gradient-based optimization of the total pressure drop in a pipe. Flow solution is recomputed and the adjoint recomputed at each design iteration Initial design p tot [Pa] Final design 30% reduction in total pressure drop after 30 design iterations Iteration

41 Mesh Morphing Once a desired change to the geometry of the system has been selected, how is that change to be made? Mesh morphing provides a convenient and powerful means of changing the geometry and the computational mesh. Use Bernstein polynomial-based morphing scheme discussed earlier 42

42 Mesh Morphing & Adjoint Data Example: Sensitivity of lift to surface shape Flow Select portions of the geometry to be modified Adjoint to deformation operation Surface shape sensitivity becomes control point sensitivity (chain rule for differentiation) Benefit of this approach is two-fold Smooths the surface sensitivity field Provides a smooth interior and boundary mesh deformation 43

43 Mesh Morphing, Adjoint Data & Constraints The adjoint solution is determined based on the specific flow physics of the problem in hand. The effect of other practical engineering constraints must be reconciled with the adjoint data to decide on an allowable design change. Example: Some walls within the control volume may be constrained not to move. A minimal adjustment is made to the control-point sensitivity field so that deformation of the fixed walls is eliminated. Fixed wall Fixed wall Moveable walls 44

44 Current Functionality The adjoint solver is released with all Fluent 14 packages. Documentation is available Theory Usage Tutorial Case study Training is available Functionality is activated by Loading the adjoint solver add-on module A new menu item is added at the top level 45

45 Current Functionality Application Drivers Key initial application areas are: Low-speed external aerodynamics F1 (increase downforce) Production automobiles (decrease drag) Low-speed internal flows Total pressure drop (reduce losses) In Fluent 14.5 a mechanism for users to define a wide range of observables of interest will be provided. Forces Moments Pressure drop Swirl Ratios Products Variances Linear combinations Unary operations 46

46 Current Scope ANSYS-Fluent flow solver has very broad scope Adjoint is configured to compute solutions based on some assumptions Steady, incompressible, laminar flow. Steady, incompressible, turbulent flow with standard wall functions. First-order discretization in space. Frozen turbulence. The primary flow solution does NOT need to be run with these restrictions Strong evidence that these assumptions do not undermine the utility of the adjoint solution data for engineering purposes. Fully parallelized. Gradient algorithm for shape modification Mesh morphing using control points. Adjoint-based solution adaption 47

47 Example 1: Automotive Aerodynamics Surface map of the drag sensitivity to shape changes 48 Surface map of the drag sensitivity to shape changes Surface map of the drag sensitivity to shape changes

48 Example 2: Pressure Drop in a Duct Total Pressure Drop (Pa) Geometry Predicted Result Original Modified Aggressive adjustment results in a 17% reduction in loss in just one design iteration 49

49 HPC Best Practices 50

50 Guidelines : Know your hardware lifecycle Have a goal in mind for what you want to achieve Using Licensing productively Using ANSYS provided processes effectively 51

51 Hardware Considerations This section is meant to provide an overview of the different hardware components and how they can effect solution time. Hopefully this will give you some of the tools to understand why some of the benchmark numbers in better detail. ANSYS would always recommend that the best thing to do before buying a system is to look at the latest benchmarks. If you are not sure please ask. 52

52 Effect of Clock Speed Impact of CPU Clock on Application Performance Processor: Xeon X5600 Series Hyper Threading: OFF, TURBO: ON Active cores: 12/node; Memory speed: 1333 MHz (performance measure is improvement relative to CPU Clock 2.66 GHz) Higher is better Improvement due to Clock GHz 2.93 GHz 3.47 GHz Clock Ratio eddy_417k aircraft_2m turbo_500k sedan_4m truck_14m ANSYS/FLUENT Model 53

53 Effect of Memory Speed We can see here the effect of memory speed. This has implications on how you build your hardware. Some processors types have slower memory speeds by default. On other processors nonoptimally filling the memory channels can slow the memory speed. Impact of Memory Speed 130% 125% 120% 115% 110% 105% 100% 95% 90% 85% 80% Impact of DIMM speed on ANSYS/FLUENT Application Performance (Intel Xeon x5670, 2.93 GHz) Hyper Threading: OFF, TURBO: ON Active threads per node: 12 (performance measure improvement is relative to memory speed of 1066 MHz) eddy_417k turbo_500k aircraft_2m sedan_4m truck_14m ANSYS/FLUENT Model 1066 MHz 1333 MHz 54

54 Turbo Boost (Intel) / Turbo Core (AMD) Turbo Boost (Intel)/ Turbo Core(AMD) is a form of over-clocking that allows you to give more GHz to individual processors when others are idle. With the Intel s have seen variable performance with this ranging between 0-8% improvement depending on the numbers of cores in use. The graph below for CFX on a Intel X5550. This only sees a maximum of 2.5% improvement. 55

55 Hyper-Threading: ANSYS Fluent Hyper-Threading Technology makes a single physical processor appear as two logical processors. Evaluation of Hyperthreading on ANSYS/FLUENT Performance idataplex M3 (Intel Xeon x5670, 2.93 GHz) TURBO: ON (measurement is improvement relative ot Hyperthtreading OFF) This is not the same as physically having two logical processors and does not give double the speedup. In our tests we ve seen as high as a 20% increase in performance although you can see the actual performance can be quite variable from the graph opposite. It is worth noting that this has licensing implications as you would need to oversubscribe the physical cores and hence would need double the HPC Licenses. Higher is better Improvemet due to Hyperthreading HT OFF (12 threads on 12 physical cores) HT ON (24 threads on 12 physical cores) eddy_417k turbo_500k aircraft_2m sedan_4m truck_14m ANSYS/FLUENT Model 56

56 AMD vs. Intel Traditionally Intel take the power approach in general in their 2 socket systems (faster core but less of them per processor/socket). Traditionally AMD take the economies of scale approach (more cores per processor but individually slower clock speeds). Remember that this landscape changes because they are constantly in competition with each other. Please note that whilst we do have some numbers for the new Intel Sandy-bridge chips we do not have scaling numbers for the equivalent AMD 6200 series at the time of writing this presentation. 57

57 2 Socket vs. 4 Socket Systems Current 4 socket systems come up slower than their 2 socket counterparts (based on Intel Westmere vs. Xeon E7-8837). Clock speed slower Memory speed slower No additional memory bandwidth. Performance of ANSYS Fluent on two-socket and four-socket based systems Performance measure is Fluent Rating (higher values are better) 2-socket based Systems IBM HS22/HS22V Blade, 3550/3650 M3, Dx360 M3 (Xeon 5600 Series) 4-socket based Systems IBM HX5 Blade, X3850 (Xeon E series) Nodes Sockets Cores Fluent Fluent Nodes Sockets Cores Rating Rating

58 Effect of the Interconnect When going for multiple systems linked together the interconnect becomes an important factor. The interconnect is the fabric that connects the nodes. We can see from the graph opposite with FLUENT how quickly the performance of Gigabit Ethernet drops off. FLUENT Rating Higher is better ANSYS/FLUENT Performance idataplex M3 (Intel Xeon x5670, 12C 2.93 GHz) Network: Gigabit, 10-Gigabit, 4X QDR Infiniband (QLogic, Voltaire) Hyperthreading: OFF, TURBO: ON Models: truck_14m QLogic Voltaire 10-Gigabit Gigabit Number of Cores used by a single job 59

59 ANSYS Fluent Auto-Partitioning Time to Partition cavity 200M case, Cavity 768 Case cores over 768 cores Auto partitioning is now very quick Less than 10s to process 800M cells! Serial pre-partitioning step no longer required Time in seconds M 400M 600M 800M Time Time to Partition truck_111m 111M Truck Case 60 Time in seconds Time

60 ANSYS CFX Partitioning Optimize parallel partitioning in multi-core clusters (CFX) β Partitioner determines number of connections between partitions and optimizes part.-host assignments Re-use previous results to initialize calculations on large problem (CFX) β Large case interpolation for cases with >~100M nodes Clean up of coupled partitioning option for multi-domain cases (CFX) Eliminates isolated partition spots Dramatically reduced partitioning times for cases with fluid-solid interfaces and very large numbers of regions P2 P1 P4 P3 P1 P5 P6 P5 P3 P6 P7 P8 Partitioning step finds adjacency amongst partitions; partitions with max adjacency are grouped on same compute nodes P2 P4 P7 P8 Compute Node 1 Compute Node 2 61

61 ANSYS Fluent Parallel Scalability Consistently improved scalability across releases Intel Harpertown Sedan, 4M cells Xeon 2.80GHz (Nehalem EP) Intel Westmere ANSYS, 200 Inc. 300 May 4009,

62 ANSYS Fluent Parallel Scalability Consistently improved scalability across releases Truck, 111M cells Intel Harpertown SGI ICE 8400EX, Intel 6-core Intel Westmere hex-core 2.93 GHz ANSYS, 500 Inc May ,

63 ANSYS Fluent Parallel Scalability on Intel ANSYS Fluent 14 Relative Performance Higher is better Geomean core Xeon X core Xeon E Geomean 1 Sedan_4m Truck_14m 1.53 Leading Performance for fluid flow simulation The memory bandwidth of the Intel Xeon processor E product family allows excellent scalability and per core performance. Support for higher speed memory DIMMs, added on-core capacity for memory loads, as well as a larger cache size are key to extending performance and scalability. Higher memory bandwidth has a pronounced impact with fully coupled solver applications, which are the most memory intensive. Sedan_4m is shown as an example of fully coupled solver performance. Truck_14m is representative of segregated solver performance. The horizontal line at 1.63 represents the geomean speedup over 6 standard benchmarks. 6 core Xeon X core Xeon E Data Source: Approved/published results as of February 1, 2012 See backup for details 64 *Other names and brands may be claimed as the property of others

64 ANSYS CFX Parallel Scalability on Intel Intel Xeon 5650 Intel Xeon E Good scalability and more operations per clock make obtaining results on Intel Xeon E5 1.68x faster than on Intel Xeon 5600 platforms For end user it is about faster turnaround or solving larger tasks with the same resources along with lower TCO 65 0 AirliftReactor BigPipe CombBVM CombEDM Cylinder IndyCar Internal LeMansCar LES_001 Pump RadCity RadFurnace StageCompressor StaticMixer100MM StaticMixer100 StaticMixer200 StaticMixer 400k Turbine Wigley100 Source: Published/submitted/approved results as of March 6, Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. Configuration Details: Please reference speaker note. For more information go to *Other names and brands may be claimed as the property of others

65 Including Monitors 3072 cores Scalability with Monitors Scalability to higher core counts Simulations with monitors including plotting and printing Hex-core mesh, F1 car, 130 million cells monitor-enabled Example data for scaling with R14 monitors Monitor support optimizations maintain scalability expectations 66

66 Fluids I/O 67 FLUENT, CFX and AUTODYN use a singular file structure. This means there is one global set of files and every process writes to them. This methodology falls down at a large number of cores where the file I/O becomes a bottleneck. CFX deals with this by using inline compression (cdat) FLUENT has both inline compression (cdat) and at v12.x introduced support for a Parallel File (pdat). Parallel file system support in ANSYS FLUENT ~10x - 20x speedup for data write Eliminates scaling bottleneck for data intensive simulations on large clusters (e.g., transient flows) Serial I/O ANSYS FLUENT Parallel I/O

67 HPC Fluids Demonstration Case To Demonstrate 50:50:50 Method Volvo XC60 vehicle model Four shape parameters RBF Morph (Integrated within FLUENT) to define shape parameters Grid morphing in parallel ANSYS WorkBench (Frame Work to Automate Process) To drive shape parameters To create DOE To perform Goal Driven Optimization The 50:50:50 Method 50 design points in the design space 50 million cells used in CFD simulation of each design point 50 hours total elapsed time to simulate all the design points EXTENT ACCURACY SPEED One Click Entire design space is simulated and postprocessed completely automatically after the initial baseline case setup 68

68 HPC Fluids Demonstration Case STEP 1 Prepare Meshed Model for Baseline Vehicle Shape STEP 2 CFD Solver Setup, Define Shape Parameters STEP 3 Generate DOE using Input Shape Parameters STEP 4 Mesh Morpher Integrated within FLUENT Solver (FLUENT), Optimizer (DX) & Post Processor (CFD Post) Morph Integrated Vehicle within Shape ANSYS WorkBench Run CFD Simulation STEP 5 Collate Data, Perform Optimization 69

69 HPC Fluids Demonstration Case 768 Cores 384 Cores 288 Cores 240 Cores 144 Cores Task Time (Seconds) Time (Seconds) Time (Seconds) Time (Seconds) Time (Seconds) Read volume mesh of baseline case into the CFD solver and apply solver settings Baseline Case (i.e. Design Point 1) CFD Solution Writing CFD data file Each Subsequent Design Point Morph vehicle shape CFD Solution Writing CFD data file Total Run Time (Wall Clock) Needed for All 50 Design Points (Hours)

70 HPC Fluids Demonstration Case Compute Cluster Details 1. Intel s Endeavor Cluster 2. Intel Xeon X5670 (dual socket) 3. Clock speed 2.93 GHz 4. Six cores per socket (12 cores per node) GB 1333 MHz, SMT ON, Turbo ON 6. QDR Infiniband 7. RHEL Server Release

71 GPU Acceleration for CFD Radiation viewfactor calculation (ANSYS FLUENT 14 - beta) First capability for specialty physics view factors, ray tracing, reaction rates, etc. R&D focus on linear solvers, smoothers but potential limited by Amdahl s Law 72

72 Getting the right setup is balancing act.. 73

73 Factors to Consider HPC Licensing Cost Cost of Hardware Complexity of Deployment and Maintenance 74

74 HPC Licensing Cost ANSYS HPC is licensed in either the HPC Workgroup/Enterprise (or individually) or HPC Packs. Given that it is licensed per partition (which in most cases translated to a core) the best value for money is in getting the best scalability per core as possible. When running multiple cores make sure you are using them as effectively as the memory bandwidth allows. 75

75 Cost of Hardware ANSYS will, in general, recommend the best hardware for performance that gets you the best out of your licensing investment. However you may need to make trade-off's for your budget. 2 socket systems provide the best performance but more inherently more complexity (and hence cost) because of the need for high speed interconnects when in a cluster. Current 4 socket systems have less performance than their 2 socket counterparts but are also cheaper because of their lack of requirement for the high speed interconnects to get to higher numbers of nodes at the low end. 76

76 Complexity of Deployment and Maintenance A large cluster can have significant overheads in ease of deployment & on-going maintenance costs. A 4 socket system, whilst having less performance, may provide an easier deployment and maintenance route at the lower end and will be a better fit to what the average IT department is used to. Often users get too caught up on per core performance at the detriment of not getting any extra speedup at all. It is important to purchase something you feel you can internally support. Purchase 3 rd party support for high performance clusters if you do not feel you have the skills to support it internally. 77

77 Remember the Following... If you opt for unsupported infrastructure This does not mean that it will not work but you use them at your own risk. We may ask you to replicate it on a system that is supported before providing further support if you run into problems! We recommend: Buying Supported Operating systems and Hardware Using ANSYS Supported Practices Talking to us before buying! It is in all our interests that you get this right! 78

78 Information Available ANSYS Partner Solutions Reference configurations Performance data White papers Sales contact points Performance Data 79

79 Information Available ANSYS Platform Support Platform Support Policies Supported Platforms Supported Hardware Tested systems ANSYS Virtual Demo Room Click on HPC! 80

80 Information Available The Manual Sections on best practices and parallel processing for various solvers Installation walkthroughs for installing the products, parallel processing, licensing and RSM (remote solve manager) ANSYS Advantage Online Magazine 81

81 Information Available Customer Portal Knowledge Resources Installation and Systems FAQ s Customer Support Portal, or Phone 82 Global ANSYS network providing Comprehensive Support