Modern Optimisation Techniques and Their Applications to Simulation driven Engineering Design Automation

Size: px

Start display at page:

Download "Modern Optimisation Techniques and Their Applications to Simulation driven Engineering Design Automation"

Lee Shelton
5 years ago
Views:

1 Modern Optimisation Techniques and Their Applications to Simulation driven Engineering Design Automation Bo Liu Department of Computing, Glyndwr University, UK

2 Outline Simulation driven design optimisation and challenges Surrogate model assisted evolutionary algorithms (SAEAs) The surrogate model aware search framework SAEAs based on multi fidelity simulation Handling design robustness Conclusions

3 Engineering Design Integrated circuit Design variables: W, L of each transistor, Cc Antenna For the goal of (such as): Aerospace Automobile Process engineering They are optimisation problems

4 Simulation Driven Design Optimisation No time to derive analytical formulas Too complex to derive analytical formulas Need a method which can provide optimised design solutions without deeply studying the engineering design problem Numerical simulation + simulation driven optimisation!

5 Design Optimisation Methods: Case Study CST Optimiser help file

6 Local and Global Optimisation (1) local optimisation global optimisation search space / range

7 Local and Global Optimisation (2) Local optimisation (SQP, NM simplex): Advantages: efficient if there is a good initial design Drawbacks: ad hoc (a good starting point) and less optimal Global optimisation (GA, DE, PSO): Advantages: general, highly optimal, robust Why not using global optimisation methods all the time? Because numerical simulations are often computationally expensive, the simulation driven optimisation process leads to prohibitive time!

8 Case Study Efficiency! Example: On chip antenna design optimisation (standard DE): EM simulation for a candidate design: 10 minutes by ADS Momentum Convergence: 800 generations Population size: 40 10min x 40 x 800 = 7 months!!! B. Liu, H. Aliakbarian, Z.Ma, G. Vandenbosch, G. Gielen, "An Efficient Method for Antenna Design Optimization Based on Evolutionary Computation and Machine Learning Techniques", IEEE Transactions on Antennas & Propagation, vol. 62, no. 1, pp. 7-18, 2014.

9 A Major Challenge of Simulation driven Design Optimisation Global optimisation algorithms often need several thousands of simulations to achieve highly optimised solutions for many engineering design optimisation problems with around 20 design variables. New optimisation techniques are highly needed, which is able to: Provide highly optimised designs (much better than manual design, comparable to using standard global optimisation methods) In a practical timeframe

10 Outline Simulation driven design optimisation and challenges Surrogate model assisted evolutionary algorithms (SAEAs) The surrogate model aware search framework SAEAs based on multi fidelity simulation Handling design robustness Conclusions

11 Evolutionary Computation (EC) Evolutionary computation is a computational intelligence method for optimisation EC is based on natural selection, survival of the fittest (objective function) Different global optimisation algorithms: GA, DE, PSO, IA, AC EA has strengths on black box (no derivatives) and multimodal (more than 1 local optima) problems

12 Differential Evolution Xˆ () t [ x, x,, x ] i 1, 2,, NP i,1 i,2 i, M Diversity vs. Optimality? u i, j ( t 1) v ( t 1), if ( rand( j) CR) or j randn( i), i, j x ( t), otherwise, j 1, 2,, M i, j

13 Solutions to the efficiency problem Efficiency enhancement Evolutionary algorithms Change the optimiser Speed up the simulation Use fewer simulations SAEA

14 Introduction to SAEA Surrogate model assisted evolutionary algorithm (SAEA): Using surrogate models to replace exact function evaluations Statistical learning Computationally cheap

15 Surrogate Modeling Surrogate modeling: Gaussian Process / Kriging (GP) Artificial neural network (ANN) Support vector machine (SVM) Radial basis function (RBF) Response surface method (RSM) Wrong convergence!!

16 Model Management Surrogate model predictions have error and have to be corrected by simulations Surrogate model and simulations are connected, because the surrogate model is constructed by the samples obtained by simulations How to cleverly use surrogate model prediction and simulation (related to surrogate model construction)? Who should be used for simulation?

Model Management (pioneer methods) Off line modelling based methods Only feasible for small scale problems Landscape with narrow peak? Generation based methods When to use exact evaluations?

17 Model Management (pioneer methods) Off line modelling based methods Only feasible for small scale problems Landscape with narrow peak? Generation based methods When to use exact evaluations? Efficiency of evaluating a whole population (how many evaluations are useful)? Iteration 1: EM Iteration 2: EM Iteration 3: EM Iteration 4:Prediction Iteration 5:Prediction Iteration 6:EM Iteration 7:EM Iteration 8:Prediction Iteration 9:Prediction Iteration 10:Prediction

18 Model Management (state of the art methods) Elitist candidates among search operators visited solutions Directly using top candidates by EA operators Combining EA with local search and (multiple) local surrogate models Prescreening methods utilising prediction uncertainty (GP model) Expected improvement Probability of improvement Lower confidence bound D. Jones, A taxonomy of global optimization methods based on response surfaces, Journal of global optimization, 2001, 21(4): D. Jones et al., Efficient global optimization of expensive black-box functions. Journal of Global optimization, 1998, 13(4): M. Emmerich et al., Single-and multiobjective evolutionary optimization assisted by gaussian random field metamodels, IEEE Transactions on Evolutionary Computation, 2006, 10(4): D. Lim et al., Generalizing surrogate-assisted evolutionary computation, IEEE Transactions on Evolutionary Computation, 2010, 14(3): Z. Zhou et al., Combining global and local surrogate models to accelerate evolutionary optimization, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2007, 37(1): M. Le et al. Evolution by adapting surrogates, Evolutionary computation, 2013, 21(2):

19 Ordinary GP modeling Given training data: Correlation function: Maximize likelihood function: Note: solve in closed form, estimate the hyper-parameters Best linear unbiased prediction and predictive distribution variants: simple/blind/

20 Prescreening With the uncertainty measurement, we can consider the quality of a candidate design in a global picture Even the predicted value is bad, promising solutions can still be discovered D. Jones, A Taxonomy of Global Optimization Methods Based on Response Surfaces, Journal of Global Optimization, pp

21 Prescreening Possible promising areas but with fewer training data points can be effectively explored. The guessed promising points may not be correct. Empirical experiments show that EI is not better than LCB. In medium scale (20 30d), in many cases / phases, prescreening prediction (evaluating the elitists)

22 Challenges Faced by SAEA Many practical engineering design problems have design variables (medium scale) and the landscapes of which are multimodal. A typical numerical simulation may need 20 minutes to several hours. Design engineers divide the optimisation time to a cup of tea, a night s time, a weekend, a week, two weeks and prohibitive. Existing SAEAs (often need several hundreds to thousands of simulations) still need substantial speed improvement to fit the industrial requirements.

23 Outline Simulation driven design optimisation and challenges Surrogate model assisted evolutionary algorithms (SAEAs) The surrogate model aware search framework SAEAs based on multi fidelity simulation Handling design robustness Conclusions

24 Goal and the Key Contradiction Goal: Make SAEAs much faster without sacrificing optimality Model quality Solution quality * x Efficiency Neval Good Solution Quality Good Model Quality More Samples (exponentially with d) Decrease Efficiency

25 Questions and Ideas (1) Why off line SAEA is not fit for medium scale problems? Do we need to model everywhere? Can we do greedy search? Most available SAEAs use the standard EA structure Standard EAs have excessive diversity Idea 1: only model the necessary regions! Question 1: to what extent we should explore the unknown parts?

26 Questions and Ideas (2) Good Solution Quality Good Model Quality Really more Samples? (exponentially with d) To describe a certain landscape, a certain amount of samples is necessary, no matter what the search is and how to model it. What decides the number of necessary samples? landscape complexity, number of design variables and? Decrease Efficiency

27 Summary of the Ideas We need to find a search method (only prescreening is not enough) which has enough (but not excessive) diversity leading to (global) optimum without exploring unnecessary regions. We may improve the locations of the samples to build surrogate models with better quality using the same number of samples, which is translated to efficiency. The samples are provided by a new search method.

28 Summary of the Ideas The search and modelling should work harmonically, reinforcing each other, rather than loosely connected How to make it easy to implement? How to make it insensitive to algorithm parameters? Surrogate model aware evolutionary search (SMAS) framework B. Liu, Q. Zhang, G. Gielen, "A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Black Box Optimization Problems", IEEE Transactions on Evolutionary Computation, vol. 18, no. 2, pp , 2014.

29 SMAS vs. Present SAEA Traditional SAEA SMAS

30 Properties of SMAS (1) Using the elitist candidates as the parent population emphasises exploitation Only at most one candidate is different from two consecutive parent populations. The training data points describing the current search region can thus be much denser. The exploration ability can be maintained by selecting appropriate EA operators and parameters.

31 Experimental Verifications (1) Problems: 20,30 dimensional Ellipsoid (F1 F3, opt:0), Rosenbrock (F4 F6, opt:0), Ackley(F7 F9, opt:0), Griewank(F10 F12, opt:0), 30 dimensional RS Rastrigin(F13, opt: 330), 30 dimensional RH composition function(f14, opt:10) evaluations, 20 runs. SMAS vs. GS SOMA [Lim IEEE TEVC 2010] Converge at last

32 Experimental Verifications (2) SMAS vs. SAGA GLS [Zhou IEEE TSMC 2007] SMAS vs. MAES [Emmerich IEEE TEVC 2006]

33 Automated Design of Complex Antennas (1) EAs have been widely used for antenna synthesis, but the long optimisation time largely limits their applications Example: Four-element antenna array (3.4GHz 3.8GHz, FR4 substrate)

Automated Design of Complex Antennas (2) Maximise realised gain (each sampling point at least 13dB) with S11 below 10dB Synthesis finished in only one night, with 71.

34 Automated Design of Complex Antennas (2) Maximise realised gain (each sampling point at least 13dB) with S11 below 10dB Synthesis finished in only one night, with 71.05dB (5 sampling points total) realised gain B. Liu, H. Aliakbarian, Z.Ma, G. Vandenbosch, G. Gielen, "An Efficient Method for Antenna Design Optimization Based on Evolutionary Computation and Machine Learning Techniques", IEEE Transactions on Antennas & Propagation, vol. 62, no. 1, pp. 7-18, 2014.

35 RF IC / mm wave IC Design Difficulties for manual design Passive component design is difficult Long simulation time: EM simulation (FEA), HB simulation Design experience intensive for the available step by step manual design method Difficulties for EA based automated design Accurate lumped models (computationally cheap) over a wide bandwidth for passive components are difficult to find at high frequencies Long simulation time: EM simulation, HB simulation Medium scale (15 40 variables) Complex and tight constraints

36 The GASPAD Method Focuses on 60GHz and above RF IC Three Main Goals of the Synthesiser Provide Highly Optimised Results General Enough to Any Circuit Configuration, Any Technology and Any Frequency Efficient Enough for Practical Use Update SMAS on constraint handling B. Liu, D. Zhao, G. Gielen, "GASPAD: A General and Efficient mm-wave Integrated Circuit Synthesis Method Based on Surrogate Model Assisted Evolutionary Algorithm", IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 33, no. 2, pp , 2014.

37 Updating SMAS for Constraint Handling New ranking methods considering constraint satisfaction Separate modelling objective and constraints New methods to model the focused search region Prescreening + predicted value

38 mm wave IC Design Automation (1) Synthesis of a 60GHz power amplifier in a 65nm CMOS technology (18 parameters) 15 min / simulation

39 mm wave IC Design Automation (2) Design parameters and their ranges

Synthesised Result Synthesised results: Power added efficiency (@P1dB): 9.85% 1dB compression point: 14.87dBm Power gain: 10.73dB K factor: 10.68 (stable) About 2 days synthesis time.

40 Synthesised Result Synthesised results: Power added efficiency 9.85% 1dB compression point: 14.87dBm Power gain: 10.73dB K factor: (stable) About 2 days synthesis time. Much better performance than manual design [He 2010 RFIC] Benchmark tests show 10x to 100x speed improvement compared to standard EA-based methods with the increasing of severity of constraints

41 MEMS Synthesis: Example Corrugated actuator: 9 parameters From several examples, AGDEMO obtains better solutions than ANSYS DesignXplorer, obtaining about 8-13 times speed enhancement when the number of parameters is around 10. Altair

42 Outline Simulation driven design optimisation and challenges Surrogate model assisted evolutionary algorithms (SAEAs) The surrogate model aware search framework SAEAs based on multi fidelity simulation Handling design robustness Conclusions

43 More Efficiency Improvements Efficiency improvement of SMAS based methods: The more complex the landscape, the more design variables (but within 50), the more spec constraints, the more efficiency improvement 5 20 times efficiency improvement than standard EAs for most microwave / RF design optimisation / synthesis problems 4 months > 1 week Another calculation: 30 minutes / simulation, 700 simulations, 2 weeks 1 hour / simulation, 700 simulations, 1 month Sometimes, we have to include the connector, housing, We need more speed improvement!

44 Solutions to the efficiency problem Efficiency enhancement Evolutionary algorithms Change the optimizer Speed up the simulation Use fewer simulations Multi fidelity simulation models SAEA

45 Why Multi fidelity Simulation Models? Opportunities Coarse mesh / reduced solver iterations / simplified mesh type numerical simulation models (e.g., FEA model) is almost universal Coarse models often have times speed improvement compared to fine model It is compatible to SAEA A natural idea: Use the coarse model to explore the design space, filtering out many non optimal regions, and use the fine model to exploit the near optimal regions. However, this is not straightforward.

46 Available Method 1 Co Kriging assisted multi fidelity optimisation Co kriging is a surrogate model constructed by both fine model data and coarse model data A GP model based on the coarse model data A GP model based on the residuals of the fine and coarse model data The co kriging model is used as the surrogate model and is compatible to SAEAs A. I. Forrester, A. Sóbester, and A. J. Keane, Multi-fidelity optimization via surrogate modelling, Mathematical, Physical and Engineering Sciences, vol. 463, no The Royal Society, 2007, pp

47 Available Method 1: Advantages and Drawbacks Advantages: Mathematical sound and reliable Drawbacks: Not scalable: when it goes to medium scale, the initial highquality co kriging model needs a tremendous number of coarse and fine model samples

48 Available Method 2 Use the fine model to fine tune the optimal designs obtained by the coarse model (popular for real world engineers) Advantages: Scalable and good compatibility Drawbacks: Ad hoc: Success depends on the fidelity of the coarse model due to the distorted landscape Sometimes, successful Sometimes, using low fidelity model does not translate to final efficiency improvement Sometimes, the optimisation simply fails. S. Koziel and S. Ogurtsov, Model management for cost-efficient surrogate-based optimisation of antennas using variablefidelity electromagnetic simulations, IET Microwaves, Antennas & Propagation, vol. 6, no. 15, pp , 2012.

49 This Method Is NOT Reliable Frequency shift (Missing/additional peaks) Spatial shift The optimal point of the coarse model may have a large distance with the optimal point of the fine model, wasting a lot of simulations coarse fine S. Koziel and S. Ogurtsov, Model management for cost-efficient surrogate-based optimisation of antennas using variable-fidelity electromagnetic simulations, IET Microwaves, Antennas & Propagation, vol. 6, no. 15, pp , The optimal point of the fine model may be not be accessible by fine tuning from the optimal point of the coarse model

50 Ideas Idea: granting a coarse model based optimisation based on SMAS (not waste much), mine the data to find the correct starting points.

51 Data Mining Process Goal: calibrate the distortion using as few fine simulations as possible.

52 Handling Distortion S11 of a microstrip antenna The focused distortion: cannot be obtained by local exploitation from the optimal points in terms of the coarse model, the landscape(s) of the coarse model maintains the general shape of that of the fine model

53 Real World Example (1) Example: Yagi-Uda antenna Rogers RT6010

54 Real World Example (2) 10GHz-11GHz Coarse model: 85,680 mesh cells, 2 minutes Fine model: 1,512,000 mesh cells, 40 minutes Specific challenge: gain of the fine model is smaller than the coarse model, and the optimal solutions of the coarse model are not feasible in terms of fine model evaluation.

55 Real World Example (3) Response: Yagi-Uda antenna SADEA-II vs. SADEA

56 Outline Simulation driven design optimisation and challenges Surrogate model assisted evolutionary algorithms (SAEAs) The surrogate model aware search framework SAEAs based on multi fidelity simulation Handling design robustness Conclusions

57 Robust design optimisation Two kinds of robust optimisation Robust design Example: mechanical engineering Requirement: the performance shows little degradation considering process variations High computing overhead per simulation (e.g., FEA) Yield optimisation, yield performance trade off Example: IC design (high value manufacturing) Requirement: accurate yield estimation High or low computing overhead per simulation Altair

58 Ordinal Optimisation Assisted DE The ORDE algorithm for analogue IC yield optimisation: Ordinal optimisation for intelligent computing budget allocation Advanced Monte Carlo Sampling method Integration with differential evolution B. Liu, F. Fernández, G. Gielen, "Efficient and Accurate Statistical Analog Yield Optimization and Variation-Aware Circuit Sizing Based on Computational Intelligence Techniques", IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 30, no. 6, pp , Altair

59 General Ideas Selection-based constraint handling OCBA (Optimal Computational Budget Allocation) Random-scale and trigonometric mutation seminar 59

60 Flow diagram seminar 60

61 Example Two stage fully differential folded cascode amplifier (TSMC 90nm CMOS technology): 21 parameters 25 minutes: yield=98.9% From several examples, ORDE obtains comparable solutions with Differential Evolution with primitive Monte Carlo simulation, obtaining about 10 times speed enhancement. Altair

62 Outline Simulation driven design optimisation Surrogate model assisted evolutionary algorithms (SAEAs) and challenges The surrogate model aware search framework Data mining techniques for multi fidelity SAEAs Conclusions

63 Conclusions Improving the efficiency of SAEA is a key issue for simulationdriven design optimisation / automation The SMAS framework significantly enhances the efficiency of SAEA for medium scale problems because the surrogate modelling and the search work together, reinforcing each other. The data mining assisted SMAS handles the distortion between simulation models of different fidelities, integrating the advantage of low fidelity simulation model in a reliable manner and is scalable. Future works include: (1) robust FEA simulation embedded design automation methods and various applications, (2) introducing computational intelligence techniques to FEA solvers.

64 Thank you Thank you!

Non-deterministic Search techniques. Emma Hart

Non-deterministic Search techniques Emma Hart Why do local search? Many real problems are too hard to solve with exact (deterministic) techniques Modern, non-deterministic techniques offer ways of getting