Global Optimization with MATLAB Products

Transcription

1 Global Optimization with MATLAB Products Account Manager 이장원차장 Application Engineer 엄준상 The MathWorks, Inc.

2 Agenda Introduction to Global Optimization Peaks Surve of Solvers with Eamples 8 MultiStart 6 Global Search Pattern Search 4 Simulated Annealing Genetic Algorithm / Multiobjective Genetic Algorithm Additional Resources Local minima Question & Answer Global minima

3 Optimization Finding answers to problems automaticall Modif Design Parameters NO Initial Design Parameters Model or Prototpe Objectives Achieved? YES Optimal Design OPTIMIZATION PROCESS Design process can be performed: Manuall Automaticall (trial-and-error or iterativel) (using optimization techniques) Optimization benefits include: Finding better (optimal) designs Faster design evaluations Useful for trade-off analsis (N dimensions) Non-intuitive designs ma be found Antenna Design Using Genetic Algorithm

4 c sin() + cos( ) Manifold Pressure Ratio sin() + cos( ) Eample Global Optimization Problems Wh does fmincon have a hard time finding the function minimum? 5 Starting at 5 Starting at 5 Starting at Wh didn t fminunc find the maimum volumetric efficienc? Peak VE Value = Starting at Starting at Starting at End.6 Wh didn t nonlinear regression find a good fit? c=b e -b 4 t +b e -b 5 t +b e -b 6 t Start Revolutions Per Minute, RPM t 4

5 Global Optimization Goal: Want to find the lowest/largest value of the nonlinear function that has man local minima/maima Problem: Traditional solvers often return one of the local minima (not the global) Solution: A solver that locates globall optimal solutions Rastrigin s Function 5

6 Global Optimization Toolbo For problems that contain multiple maima/minima or are non-smooth Faster/fewer function eval uations Larger problems (higher d imensions) Finds local minima/maim a Finds global minima/mai ma (most of the time) Better on non-smooth stochastic discontinuous undefined gradients Custom data tpes (in GA and SA solvers) Optimization T oolbo Global Optimization Toolbo 6

7 MULTISTART 7

8 What is MultiStart? Run a local solver from each set of start points Option to filter starting points based feasibilit Supports parallel computing 8

9 GLOBAL SEARCH

10 What is GlobalSearch? Multistart heuristic algorithm Calls fmincon from multiple start points to tr and find a global minimum Filters/removes non-promising start points

11 GlobalSearch Overview Schematic Problem Peaks function Three minima Green, z = -.65 Red, z= -.5 Blue, z =

12 GlobalSearch Overview Stage Run from specified

13 GlobalSearch Overview Stage Generate stage start points via Scatter Search

14 GlobalSearch Overview Stage Find stage start point with lowest penalt value

15 GlobalSearch Overview Stage Run from best stage point

16 GlobalSearch Overview Stage Generate stage start points using Scatter Search

17 GlobalSearch Overview Stage Analse each stage point in turn

18 GlobalSearch Overview Stage Don t run points that are in basins of eisting minimum

19 GlobalSearch Overview Stage Analse each stage point in turn

20 GlobalSearch Overview Stage Don t run points whose penalt value eceeds threshold 6 Current penalt threshold value :

21 GlobalSearch Overview Stage Analse each stage point in turn

22 GlobalSearch Overview Stage Run start point if it satisfies distance & merit criteria Current penalt threshold value :

23 GlobalSearch Overview Stage Epand basin of attraction if minimum alread found Current penalt threshold value : Basins can overlap

24 SIMULATED ANNEALING 8

25 What is Simulated Annealing? A probabilistic metaheuristic approach based upon the phsical process of annealing in metallurg. Controlled cooling of a metal allows atoms to realign from a random higher energ state to an ordered crstalline (globall) lower energ state B analog, simulated annealing replaces a current solution b randoml choosing a nearb solution A nearb solution is determined b the solution temperature 9

26 Simulated Annealing Overview Iteration Run from specified

27 Simulated Annealing Overview Iteration Randoml generate a new point according to probabilit distribution and current temperature Temperature =.9 - Possible New Points: Standard Normal N(,) * Temperature - - -

28 Simulated Annealing Overview Iteration If lower, accept the point, if higher, accept based upon acceptance probabilit Temperature = P e accept (.9)/ T

29 Simulated Annealing Overview Iteration Randoml generate a new point according to probabilit distribution and current temperature Temperature =

30 Simulated Annealing Overview Iteration Randoml generate a new point according to probabilit distribution and current temperature, accept new point if lower value Temperature =

31 Simulated Annealing Overview Iteration Lower temperature according to temperature schedule Temperature =

32 Simulated Annealing Overview Iteration Lower temperature according to temperature schedule and generate new point Temperature =

33 Simulated Annealing Overview Iteration N- After several iterations, the search radius becomes small and we narrow in on a local solution Temperature =

34 Simulated Annealing Overview Iteration N Reset temperature and start the process again (reanneling) Temperature =

35 Simulated Annealing Overview Iteration N Reset temperature and start the process again (reannealing) Temperature =.9 -. Paccept ( )) / T e - -. (

36 Simulated Annealing Overview Iteration N Reset temperature and start the process again (reannealing) Temperature =

37 Simulated Annealing Overview Iteration N+ Reduce temperature and continue Temperature =

38 Simulated Annealing Overview Iteration N+ Reduce temperature and continue Temperature =

39 Simulated Annealing Overview Iteration Reduce temperature and continue Temperature =

40 PATTERN SEARCH (DIRECT SEARCH) 47

41 What is a Pattern Search? An approach that uses a pattern of search directions around the eisting points, the mesh Polls the mesh for a better solution and moves to that point Epands/contracts the mesh around the current point when a solution is not found Does not rel on gradient information 48

42 Pattern Search Overview Iteration Run from specified

43 Pattern Search Overview Iteration Appl pattern vector, poll new points for improvement Mesh size = Pattern vectors = [,], [,], [-,], [-,-] First poll successful 4.6 Pnew mesh _ size* pattern _ vector.8.6 *[,].4 - Complete Poll (not default)

44 Pattern Search Overview Iteration Increase mesh size and repeat Mesh size = Pattern vectors = [,], [,], [-,], [-,-] Complete Poll 5

45 Pattern Search Overview Iteration Mesh epansion: increase mesh size and repeat Mesh size = 4 Pattern vectors = [,], [,], [-,], [-,-]

46 Pattern Search Overview Iteration 4 Refine mesh: decrease mesh size and repeat Mesh size = 4*.5 = Pattern vectors = [,], [,], [-,], [-,-]

47 Pattern Search Overview Iteration N Continue epansion/contraction until convergence

48 GENETIC ALGORITHM 57

49 What is a Genetic Algorithm? Genetic Algorithms use concepts from evolutionar biolog to find eact or approimate solutions to optimization problems Start with an initial generation of candidate solutions that are tested against the objective function Subsequent generations evolve from the st through selection, crossover and mutation The individual that best minimizes the given objective is returned as the ideal solution 58

50 How Evolution Works Binar Case Selection Retain the best performing bit strings from one generation to the net. Favor these for reproduction parent = [ ] parent = [ ] Crossover parent = [ ] parent = [ ] child = [ ] Mutation parent = [ ] child = [ ] 59

51 Genetic Algorithm Iteration Evaluate initial population

52 Genetic Algorithm Iteration Select a few good solutions for reproduction

53 Genetic Algorithm Iteration Generate new population and evaluate

54 Genetic Algorithm Iteration Select a few good solutions for reproduction

55 Genetic Algorithm Iteration Generate new population and evaluate

56 Genetic Algorithm Iteration Select a few good solutions for reproduction

57 Genetic Algorithm Iteration N Continue process until stopping criteria are met - Solution found

58 6 4 Comparison of Solver (Default) Performance FuncValue Min StartPt rf([,]) rf([,]) 6 FuncValue Min 4 StartPt rf([,]) 6 FuncValue Min StartPt

59 7

60 Additional Resources Upcoming Webinars Speeding Up Optimization with Parallel Computing (August ) On-demand Webinars Genetic Algorithm in Financial Applications Tips & Tricks: Getting Started with Optimization Introduction to Optimization 7

61 Contact Information North America Phone: Outside North America Contact our local MathWorks office or reseller: 7

62 Questions? 7

63 MATLAB Provides the Foundation for Optimization The leading environment for technical computing Customizable Numeric computation Data analsis and visualization The de facto industr-standard, high-level programming language for algorithm development Toolboes for statistics, optimization, smbolic math, signal and image processing, and other areas Foundation of the MathWorks product famil 74

64 Optimization Toolbo Solve standard and large-scale optimization problems Graphical user interface and command line functions for: Linear and nonlinear programming Quadratic programming Nonlinear least squares and nonlinear equations Multi-objective optimization Binar integer programming Additional Capabilities: Parallel computing support in selected solvers Customizable algorithm options Choose between standard and largescale algorithms Output diagnostics 75

65 Global Optimization Toolbo Solve multiple maima, multiple minima, and nonsmooth optimization problems Graphical user interface and command line functions for: Global Search solver Multistart solver Genetic algorithm solver Single objective Multiobjective with Pareto front Direct search solver Simulated annealing solver Useful for problems not easil addressed with Optimization Toolbo: Discontinuous Highl nonlinear Stochastic Discrete or custom data tpes Undefined derivatives Multiple maima/minima 76

66 Anatom of an Optimization Problem General Form Accepted b MATLAB Solvers Objective Function min Decision variables f ( ) Subject to Constraints (i.e. such that) Tpicall a linear or nonlinear function A b, c( ) Linear constraints inequalities equalities bounds A l eq b eq u, c eq ( ) Nonlinear constraints inequalities equalities 77

67 MATLAB Optimization Products and Eample Applications MATLAB Statistics Toolbo Curve Fitting Toolbo Optimization Toolbo Genetic Algorithm and Direct Search Toolbo Solving Equations Real roots finding (D): -5 Real roots finding (N-D): Nois, discontinuous root finding (N-D) f() = Root finding Sstems of equations Linear Sstems: F() = A-b = (i.e. A=b) >> = A \ b Nonlinear Sstems F(X) = Nois, discontinuous sstems: F(X) = Curve/Modeling Fitting Basic (linear) curve fitting.9 Advanced (nonlinear) curve fitting Model Fitting (least squares) Constrained curve fitting Nois, Discontinuous parameter estimation.8 Parameter Estimation curve fitting parameter estimation (model fitting) Trade-Off Studies Maimization Minimization Goal seeking Multiobjective Unconstrained nonlinear minimization Constrained nonlinear minimization Nois, Discontinuous, illdefined mimization 78