Multidisciplinary System Design Optimization (MSDO)

Transcription

1 Multidisciplinary System Design Optimization (MSDO) Approximation Methods Karen Willcox Slides from: Theresa Robinson, Andrew March 1 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

2 Outline Introduction to approximation methods Data fit methods Polynomial response surfaces Kriging Model order reduction Reduced-basis methods Proper orthogonal decomposition Multifidelity methods Trust-region model management 2 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

3 Approximation Methods Replace the simulation with an approximation or surrogate Uses some data from the initial simulation Can be global or local Surrogate is much less computationally expensive to evaluate Not just optimization Uncertainty Quantification (e.g. Monte Carlo simulation methods) Visualization 3 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

4 Why Approximation Methods We have seen throughout the course the constant trade-off between computational cost and fidelity. high fidelity (e.g. CFD,FEM) can we do better? how to implement? intermediate fidelity (e.g. vortex lattice, beam theory) empirical models Fidelity Level trade studies Level of MSDO limited optimization/iteration can the results be believed? full MDO from Giesing, 1998 Approximation methods provide a way to get high-fidelity model information throughout the optimization without the computational expense. 4 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

5 Data Fit Methods Sample the simulation at some number of design points Use DOE methods, e.g. Latin hypercube, to select the points Fit a surrogate model using the sampled information Surrogate may be global (e.g., quadratic response surface) or local (e.g., Kriging interpolation) Surrogate may be updated adaptively by adding sample points based on surrogate performance (e.g., EGO) 5 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

6 Polynomial Response Surface Method Surrogate model is a local or global polynomial model Can be of any order Most often quadratic; higher order requires many samples Advantages: Simple to implement, visualize, and understand, easy to find the optimum of the response surface Disadvantages: May be too simple, doesn t capture multimodal functions well 6 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

7 Global Polynomial Response Surface Fit objective function with a polynomial e.g. quadratic approximation: J b x c x c x x 2 ( x) a 0 i i ii i ij i j i i i, j i Update model by including a new function evaluation then doing least squares fit to compute the new coefficients 7 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

8 Global Polynomial Response Surface J Estimation problem: J Xc 1 2 M J J J J=vector containing M responses T 1 1 x x x x x c c c c c=vector containing p coefficients p x 2 is the value of 1 x 1 for the 2 nd sample Least squares solution: X 1 x x x x M M M M T T c X X X J X=M p matrix Each row corresponds to one data sample; each column corresponds to an unnown coefficient 8 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

9 Kriging Adopted from the geostatistics literature Based on Gaussian process models Assumes that the output function values are correlated in design space, i.e. closer points are more highly correlated Can have multiple extrema Interpolating method Exact at sample points Gives estimate of mean squared error Can use to give error bounds Can use to choose new sample points 9 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

10 Kriging: Mathematical Bacground We want to mae a prediction of y at a point x Uncertain of value: model as a random variable, normally distributed with mean and variance 2 Consider two points x i and x j Expect values to be close if the distance between them is small Formalize this idea by setting: Corr[ Y ( x j ), Y ( x )] exp i n 1 i x ji x i p i 10 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

11 Kriging Basis Functions Corr[ Y ( x j ), Y ( x )] exp i n 1 i x ji x i p i Correlation Correlation Distance between points Distance between points Each p i and i is chosen to best fit the data 11 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

12 Kriging Mathematics Cont. Choose μ, p i, and θ i to maximize the lielihood of observing the data Detailed equations in Giunta and Watson (1998), derivation in Jones (2001) Kriging predictor is n * yˆ ( * p = j x ) μ + c i exp θ j x xi i= 1 j= 1 mean surface weighted sum of Gaussians, each centered at a sample point 12 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

13 Kriging Extensions Can combine polynomial RSM and Kriging Apply Kriging to difference between sample values and polynomial approximation Soft Kriging allows upper and lower bounds, prior CDFs Efficient Global Optimization (EGO) Uses Kriging to find expected improvement Samples the point with the largest expected improvement and adds it to the sample set 13 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

14 Efficient Global Optimization Jones 1998; based on probability theory Assumes: f ( x) T β x T β x + N : regression term N ( μ( x ), σ 2 ( x ) ) ( μ( x), σ 2 ( x) ) : error from regression model is normally distributed, with mean μ(x) and variance σ 2 (x) Estimate function values with a Kriging g model Predicts mean and variance Evaluate function at maximum expected improvement location(s) and update model Surrogate model is updated adaptively; th surrogate is m (x) =¹ (x)+ T x

15 f high Bayesian Model Calibration ( x) m ( x) f ( x) ( x) low Model the error between a high-fidelity and a low-fidelity function [Kennedy2000, 2001; Huang2006] If the low-fidelity function is good, converges faster Global calibration procedure 15

16 Comparison of Data Fit Methods 16 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

17 Reduced-Basis Methods Consider r feasible design vectors: x 1, x 2,..., x r We could consider the desired design to be a linear combination of these basis vectors: x* x x i r 1 i C scalar coefficient basis vector added for generality 17 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

18 Reduced-Basis Methods We can now optimize J(x) by finding the optimal values for the coefficients i. dimension n dimension r Do one full-order evaluation of resulting answer Approach is efficient if r << n Will give the true optimum only if x* lies in the span of {x i } Basis vectors could be previous designs solutions over a particular range (DoE) derived in some other way (e.g., proper orthogonal decomposition) 18 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

19 Reduced-Basis Example Example using a reduced-basis approach (van der Plaats Fig 7-2): airfoil design for a unique application. Many airfoil shapes with nown performance are available Design variables are (x,y) coordinates at chordwise locations (n~100) Use four basis airfoil shapes (low-speed airfoils) which contain the n geometry points Plus two basis shapes which allow trailing edge thicness to vary r=6 (r<<n) Optimize for high speed, maximum lift with a constraint on drag 19 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

20 Reduced-Basis Example From Vanderplaats Figs. 7-2 and 7-3, pg. 260 Airfoil basic shapes. NACA 2412 Basic shape 1 NACA 64, -412 Basic shape 2 NACA 65, -415 Basic shape 3 Basis functions NACA 65, A215 Basic shape 4 Basic shape 5 y - -x/c y - -0 y - -0 Basic shape 6 y - - -x/c result of optimization using reduced basis Minimum-drag airfoil. Image by MIT OpenCourseWare. 20 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

21 Proper Orthogonal Decomposition (aa Karhunen-Loève expansions, Principal Components Analysis, Empirical Orthogonal Eigenfunctions, ) Consider K snapshots (solutions at selected times or parameter values) Form the snapshot matrix Choose the n basis vectors to be left singular vectors of the snapshot matrix, with singular values This is the optimal projection in a least squares sense: 21 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

22 Multifidelity Methods Sometimes there is more than one model for the same system e.g. Navier Stoes and thin-airfoil theory for wing design, finite-element and beam theory for structural design Low-fidelity model may provide good information over a wide range, at much lower computational cost Would lie to find optimum of high-fidelity problem, but use low-fidelity model most of the time 22 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

23 23 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox A Hierarchy of Models Images of Figure 1b, 4, and 8 removed due to copyright restrictions. Figures from: Choi, S, Alonso, JJ, Kim, S., Kroo, IM. Two-level multi-fidelity design optimization studies for supersonic jets. 43th AIAA Aerospace Sciences Meeting & Exhibit. January Image of Low-fidelity EM and High fidelity EM models removed due to copyright restrictions.

24 Trust-Region Model Management A rigorous method for determining when to use high-fidelity function calls Solves a series of subproblems: Minimize Subject to gˆ x Jˆ ( x) ( x) x c 0 Several methods exist to handle the approximation of constraints. x c :center point of trust region at iteration :size of trust region at iteration 24 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

25 Trust-Region Model Management Size of trust region updated depending on how well surrogate predicts high-fidelity function value Merit function Γ [ J ( x ), g ( x ) ] Ratio of actual to predicted d improvement: ρ Γ( ) ( ) Γ x Γ x c * ( ) ˆ ( ) Γ xˆ Γ x * = c * 25 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

26 Trust-Region Model Management Trust region size update rules: 0 Reject step Accept step Accept step Accept step Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

27 Trust-Region Demonstration 27 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

28 Trust-Region Model Management Calls high-fidelity analysis once per iteration Calls surrogate analysis many times per iteration Provably convergent to local minimum of high fidelity function if surrogate is first-order accurate at center of trust region Extensions to the case of et al. (2008). in Robinson Derivative-free approaches in Conn et al. (2009) x xˆ 28 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

29 Corrections Include corrections in order to enforce consistency and gain provable convergence of trust-region approach Additive Correction: Jˆ ( x) J lo Multiplicative Correction: Jˆ ( x) J lo ( x) ( x) ( x) ( x) surrogate model low-fidelity model 29 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

30 Multifidelity Optimization Combines several elements: Trust regions Bayesian model calibration Adaptive sampling Surrogate models (e.g., interpolation models using Kriging) Estimation theory Active area of research 30 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

31 Combining Estimates of Multifidelity Models Use Kalman filtering approach to compute combine estimate Maximum lielihood estimate weights each model according to its variance (pay more attention to models in which we have more confidence) ¹ est (x) = ¹ med (x) ¾ 2 low(x) ¾ 2 low(x) + ¾ 2 med(x) + ¹ low(x) 1 ¾est 2 (x) = 1 ¾ 2 low (x) + 1 ¾ 2 (x): med ¾ 2 med(x) ¾ 2 low(x) + ¾ 2 med(x) Extends naturally to case with more than two models; much more efficient than nesting (March 2010) Image by MIT OpenCourseWare.

32 Lecture Summary A number of ways to create approximations, or surrogates Each has its own area of application, advantages, and disadvantages Data fit surrogates Polynomial response surfaces Kriging Model order reduction Reduced basis Proper orthogonal decomposition Multifidelity methods 32 Massachusetts Institute of Technology - Prof. de Wec and Prof. Willcox

33 References Alexandrov, N., Dennis, J.E., Lewis, R.M. and Torczon, V., A trust region framewor for managing the use of approximation models in optimization, NASA CR , ICASE Report No , October Barthelemy, J-F. M. and Hafta, R.T., Approximation concepts for optimum structural design a review, Structural Optimization, 5: , Conn, A.R., Scheinberg, K. and Vicente, L., Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points, SIAM Journal of Optimization, Vol. 20, No.1, pp , Gill, P.E., Murray,W. and Wright, M.H., Practical Optimization, Academic Press, Giunta, A.A. and Watson, L.T., A comparison of approximation modeling techniques: polynomial versus interpolating models, AIAA Paper , Jones, D.R., A taxonomy of global optimization methods based on response surfaces, Journal of Global Optimization, 21, , LeGresley, P.A. and Alonso, J.J., Airfoil design optimization using reduced order models based on proper orthogonal decomposition, AIAA Paper , March, A. and Willcox, K., A Provably Convergent Multifidelity Optimization Algorithm not Requiring High-Fidelity Derivatives, AIAA , presented at 3 rd MDO Specialist Conference, Orlando, FL, April 12-15, Robinson, T., Willcox, K., Eldred, M., and Haimes, R. Multifidelity Optimization for Variable- Complexity Design, AIAA Journal, Vol.46, No.11, pp , Vanderplaats, G.N., Numerical Optimization Techniques for Engineering Design, Vanderplaats 33 R&D, 1999.

34 MIT OpenCourseWare ESD.77 / Multidisciplinary System Design Optimization Spring 2010 For information about citing these materials or our Terms of Use, visit: