# An Evolutionary Algorithm for Minimizing Multimodal Functions

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 An Evolutionary Algorithm for Minimizing Multimodal Functions D.G. Sotiropoulos, V.P. Plagianakos and M.N. Vrahatis University of Patras, Department of Mamatics, Division of Computational Mamatics & Informatics, GR , Patras, Greece. dgs vpp Abstract A new evolutionary algorithm for global optimization of multimodal functions is presented. The algorithm is essentially a parallel direct search method which maintains a populations of individuals and utilizes an evolution operator to evolve m. This operator has two functions. Firstly, to exploit search space as much as possible, and secondly to form an improved population for next generation. This operator makes algorithm to rapidly converge to global optimum of various difficult multimodal test functions. 1 Introduction In this contribution we propose a new evolutionary algorithm for global minimization of nonlinear and non-differentiable continuous functions. More formally, we consider following general global optimization problem: f = min x D f(x), where f : D IR is a continuous function and compact set D IR n is an n-dimensional parallelepiped. Often, analytic derivatives of f are unavailable particularly in experimental settings where evaluation of function means measuring some physical or chemical quantity or performing an experiment. Inevitably, noise contaminates function values and as a consequence, finite difference approximation of gradient is difficult, if not impossible, procedure to estimate gradient fails, and methods that require derivatives are precluded. In such cases direct search approaches are methods of choice. If objective function is unimodal in search domain, one can choose among many good alternatives. Some of available minimization methods involve only function values, such as those of Hook and Jeeves [2] and Nelder and Mead [5]. However, if objective function is multimodal, number of available algorithms is reduced to very few. In this case, algorithms quoted above tend to stop at first minimum encountered, and cannot be used easily for finding global one. Search algorithms which enhance exploration of feasible region through use of a population, such as Genetic Algorithms (see, [4]), and probabilistic search techniques such as Simulated Annealing (see, [1, 7]), has been object of many publications. In analogy with behavior of natural organisms, random search algorithms have often called evolutionary methods. The generation of a new trial point corresponds to mutation while a step toward minimum can be viewed as selection. TECHNICAL REPORT No.TR

2 In present work we propose an evolutionary algorithm in which we enhance exploration of feasible region utilizing a specially designed evolution operator, called Simplex Operator, and based on idea of simplex search method. It starts with a specific number of N dimensional candidate solutions, as an initial population, and evolves m over time, in such a way that each new iteration of algorithm gives a better population. The remaining of paper is organized as follows: In Section 2 we present our Simplex Evolution algorithm (SE) for minimizing multimodal functions. The algorithm has been tested against Annelead Nelder and Mead simplex method and corresponding results are presented in Section 3. The final section contains concluding remarks and a short discussion for furr work. In appendix we list problems used in our tests from Levy et al. [3]. 2 The Simplex Evolution Simplex Evolution is based on collective learning process within a population of individuals, each of which represents a search point in space of potential solutions to a given problem. The population is chosen randomly and should try to cover entire search space D, and evolves towards better and better regions of search space by means of deterministic steps that use randomly selected individuals. The number of candidate solutions does not change during minimization process. At each iteration, called generation, evolution of population occurs through use of an operator called mutation. The key idea is to construct a mechanism, which at each generation takes each individual of population, and replace it with a better one for next generation. To this end, we have specially redesign traditional mutation operator in order to enhance process of exploitation of search space, in such a manner that typically avoids getting stuck in local minimum points. Our mutation operator, called Simplex Operator, is based on a simplex search, which is a geometrical figure consisting in N dimensions of N + 1 points and all interconnecting lines and faces. In general we are interested only in simplexes that are nondegenerate, i.e., that enclose a finite inner N dimensional volume. For each candidate solution, called BasePoint, of current generation, we choose N or candidates and we form a simplex. If we assume that BasePoint of a nondegenerate simplex is taken as origin, n N or points define vector directions that span N dimensional vector space. In sequence, for this simplex we perform two of three possible trial steps: a reflection, an expansion or a contraction step, in order to approach minimum by moving away from high values of objective function, without moving out of search domain. The current point is replaced in next generation by resulted mutated one. This last operation is called selection and produces an improved population for next generation. 2.1 The Simplex Operator The Simplex Operator is a specially designed evolution operator with deterministic steps that enhances exploitation of search space. This operator amounts to replacing a single individual, as a description of system state, by a simplex of N + 1 points. In practice, we are only interested in nondegenerate simplexes. The moves of simplex are same as described in [5, 6], namely reflection, expansion, and contraction. The least preferred point of simplex is reflected through or contracted towards center of opposite face of simplex. In line 3 we Actually, check if this simplex operator S performs is degenerate, only one regarding cycle of to function Simplex values method of as its shown vertices. in Specifically, following we algorithm check if which standard gives deviation basic steps of of function operator: at N + 1 vertices of TECHNICAL REPORT No.TR

3 SimplexOperator(BasePoint x 1 ) 1: Select points x i, where i = 2,..., N + 1; 2: Construct simplex S = x 1, x 2,..., x N+1 ; 3: if S is degenerate go to 1; 4: Rank x i based on function values; 5: Reflect(S) x r ; 6: if f(x r ) < f(x 1 ) n 7: Expand(S) x e ; 8: if f(x e ) < f(x r ) n 9: return expansion point x e ; 10: else 11: return reflection point x r ; 12: end if 13: else 14: Contract(S) x c ; 15: if f(x c ) < f(x r ) n 16: return contraction point x c ; 17: else 18: return best vertex of simplex S; 19: end if 20: end if current simplex is smaller than some prescribed small quantity ε, i.e. σ = { N+1 [f(x i ) f(x 0 )] 2 } 1/2 ε, (1) N + 1 where x 0 is centroid of simplex S. In implementation of algorithm we have used ε = There is no case of an infinite loop between lines 1 and 3, where simplex S is examined for degeneracy, since in SE algorithm (see Section 2.2), (1) has been enforced as a termination criterion for entire population. Therefore, we can form at least one non-degenerate simplex. Using (1) we avoid unnecessary function evaluations. To guarantee convergence in feasible region D, we have constrained operations reflection and/or expansion (lines 5 and 7), in a such a way that returned point is always in D. If returned point is outside of feasible region D, we subdivide prefixed values of reflection and/or expansion constants until new point is included in D. If simplex operator fails to improve base point x 1 during reflection, expansion or contraction operations n we return best vertex of simplex S. Simplex Operator has embedded (in lines 9, 11, 16, 18) selection step, as it returns always a better individual in order to replace BasePoint. This is consistent with idea of selection in evolution algorithms, since good individuals are reproduced more often than worse ones in next generation. In contrast with or evolution algorithm, SE deterministically choose individuals for next generation. 2.2 The SE Algorithm In this subsection we present our evolutionary algorithm (SE), which maintains a population of PopSize individuals and utilizes Simplex Operator to evolve m over time. The algorithm TECHNICAL REPORT No.TR

4 does not have any control parameters, as or global optimization methods. The user must supply only search region (a box) and population size Popsize. In our implementation we have chosen Popsize to be five times dimension of problem to be optimized. As population iterates through successive generations, individuals tend toward global optimum of objective function f. To generate a new population G(t + 1) on basis of previous one G(t) algorithm SE performs steps as shown in pseudo code. SE Algorithm 1: t = 0; 2: Generate initial population G(t); 3: Evaluate initial population G(t); 4: while termination criterion not reached do 5: for i = 1,..., P opsize do 6: BasePoint x i G(t); 7: x new i SimplexOperator(BasePoint); 8: Put x new i in next generation G(t + 1); 9: end for 10: t = t + 1; 11: end while Classical termination criteria for evolution algorithms are: a) maximum number of generations, and b) maximum number of function evaluations. Obviously, if global minimum value is known, n an efficient termination criterion can be defined; method is stopped when finds minimizer within a predetermined accuracy. This happens, for example, in case of non linear least squares. Except previous ones we can enforce an additional termination criterion, since SE is based on simplex method. This criterion checks at each generation if entire population has converged, examining Relation (1). If standard deviation of function values is smaller than ε, n this is a strong indication that furr search is impossible, since we cannot form simplexes big enough to exploit search space. Therefore, we can stop algorithm, or possibly, rank population and restart algorithm by reinitializing worst individuals. By doing this we enrich population and give algorithm a chance for furr exploitation of search space. 3 Tests and Results In this section we present some preliminary results of SE method. We decided to test effectiveness of SE method against Annealed Nelder and Mead method (ANM) [6], since it has few, easy tunable, control parameters, in contrast with or well-known pure annealing methods that have many control variables [1, 7]. ANM has been chosen also owing to its ability to accept downhill as well as uphill steps. The probability to accept uphill steps decreases with temperature of system, which in turn decrease with time. This mechanism enables ANM to escape local minima when temperature does not tend to zero. When annealing part is switched off, simplex method remains. We have not chosen a gradient method, since test functions are multimodal and such a method would immediately get trapped at first local minimum encountered. To evaluate performance of algorithms we have tested m in seven well-known multimodal test functions from Levy et. al. (see Appendix). The admissible domains of TECHNICAL REPORT No.TR

5 test functions were [ 10, 10] n, where n is dimension of problem. The test for ANM has been made starting from 100 regular simplexes randomly generated in region D. The SE method, since it maintains a population, has been initialized 100 times, each time with a different randomly chosen population. We retained two comparison criteria: success rate to find global minimum, and average number of function evaluations, during 100 independent runs. We consider that method has succeeded, if it has found global minimum with accuracy It must be noted that when a method fails to find global minimum its function evaluations are not counted. Table 1: Minimization of seven test functions. SE ANM N FE Succ. FE Succ. Levy No % % Levy No % 116 9% Levy No % % Levy No % % Levy No % % Levy No % 0% Levy No % 0% In Table 1 one can observe that SE is more costly compared to ANM, but excibited better success rate and number of function evaluations needed is quite predictable regarding to number of local minima. On or hand, ANM was unable to find global minimum in high-dimensional problems, such as Levy No. 11 and No. 12, where a multitude of local minima (10 8 and respectively) exist. Summing up, results show that SE method is a predictable and reliable method, in contrast with annealed version of simplex method, especially in high dimensions. 4 Conclusion Preliminary results indicate that this new evolutionary algorithm fulfills our promises, since it exhibits high performance and good convergence properties in consecutive independent trials. It can be applied to nondifferentiable, nonlinear and high-dimensional multimodal objective functions. Moreover, method is heuristic-free, i.e. does not uses any control parameters. In a recent work [8], we have successfully hybridize a genetic algorithm with an interval branch-and-bound one, for global optimization. In its present form, SE gives us one global minimum, without guarantee. Our aim is to combine SE with an interval global optimization algorithm in order to find all global minimizers with certainty. Future work will also include testing on bigger and more complex real-life optimization problems and comparison with or well-known global optimization methods, such as those found in [1, 7]. Last but not least, we will design and implement a parallel version of algorithm, since SE is a parallel direct search, and it can be efficiently executed on a parallel machine or a network of computers. TECHNICAL REPORT No.TR

6 References [1] A. Corana, M. Marchesi, C. Martini, and S. Ridella, Minimizing Multimodal Functions of Continuous Variables with Simulated Annealing Algorithm,ACM Trans. on Math. Software, 13, , [2] R. Hook, and T. Jeeves, Direct search solution of numerical and statistical problems, J. ACM, 7, , [3] A. Levy, A. Montalvo, S. Gomez, and A. Galderon, Topics in Global Optimization, Lecture Notes in Mamatics No. 909, Springer-Verlag, New York, [4] Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, Springer- Verlag, [5] Nelder J.A. and Mead R., A simplex method for function minimization. Computer J., 7, , [6] W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in FORTRAN, Second Edition, Cambridge University Press, [7] P. Siarry, G. Berthiau, F. Durbin, and J. Haussy, Enhanced Simulated Annealing for Globally Minimizing Functions of Many Continious Variables, ACM Trans. on Math. Software, 23, , [8] D. Sotiropoulos, E. Stavropoulos, and M. Vrahatis, A New Hybrid Genetic Algorithm for Global Optimization, Nonlinear Analysis, Theory, Methods & Applications, 30, , Appendix: List of test functions (1) Levy No. 3, (n = 2). f(x) = i cos[(i + 1)x 1 + i] j cos[(j + 1)x 2 + j], 10 x i 10 j=1 where 10 x i 10, i = 1, 2. There are about 760 local minima and 18 global minima. The global minimum is f = (2) Levy No. 5, (n = 2). f(x) = i cos[(i + 1)x 1 + i] j cos[(j + 1)x 2 + j] + (x ) 2 + (x ) 2, j=1 where 10 x i 10, i = 1, 2. There are about 760 local minima and one global minimum f = at x = ( , ). Levy No. 5 is identical to Levy No. 3 except for addition of a quadratic term. The large number of local optimizers makes it extremely difficult for any approximation method to find global minimizer. (3) Levy No. 8, (n = 3). n 1 f(x) = sin 2 (πy 1 ) + (y 1 1) 2 [ sin 2 (πy i+1 )] + (y n 1) 2 TECHNICAL REPORT No.TR

7 where y i = 1 + (x i 1)/4, (i = 1,..., n) and x i [ 10, 10], i = 1, 2, 3. There are about 125 local minima and one global minimum f = 0 at x = (1, 1, 1). Levy No. 8 is difficult due to combinations of different periods of sine function. (4) Levy No. 9, (n = 4). Same as problem (3) with n = 4 and x i [ 10, 10], i = 1,..., 4. There are about 625 local minima and one global minimum f = 0 at x = (1, 1, 1, 1). (5) Levy No. 10, (n = 5). Same as problem (3) with n = 5 and x i [ 10, 10], i = 1,..., 5. There are about 10 5 local minima and one global minimum f = 0 at x = (1,..., 1). (6) Levy No. 11, (n = 8). Same as problem (3) with n = 8 and x i [ 10, 10], i = 1,..., 8. There are about 10 8 local minima and one global minimum f = 0 at x = (1,..., 1). (7) Levy No. 12, (n = 10). Same as problem (3) with n = 10 and x i [ 10, 10], i = 1,..., 10. There are about local minima and one global minimum f = 0 at x = (1,..., 1). TECHNICAL REPORT No.TR

### SIMULATED ANNEALING TECHNIQUES AND OVERVIEW. Daniel Kitchener Young Scholars Program Florida State University Tallahassee, Florida, USA

SIMULATED ANNEALING TECHNIQUES AND OVERVIEW Daniel Kitchener Young Scholars Program Florida State University Tallahassee, Florida, USA 1. INTRODUCTION Simulated annealing is a global optimization algorithm

### Initializing the Particle Swarm Optimizer Using the Nonlinear Simplex Method

Initializing the Particle Swarm Optimizer Using the Nonlinear Simplex Method K.E. PARSOPOULOS, M.N. VRAHATIS Department of Mathematics University of Patras University of Patras Artificial Intelligence

### An evolutionary annealing-simplex algorithm for global optimisation of water resource systems

FIFTH INTERNATIONAL CONFERENCE ON HYDROINFORMATICS 1-5 July 2002, Cardiff, UK C05 - Evolutionary algorithms in hydroinformatics An evolutionary annealing-simplex algorithm for global optimisation of water

### Evolutionary operators in global optimization with dynamic search trajectories

Numerical Algorithms 34: 393 403, 2003. 2003 Kluwer Academic Publishers. Printed in the Netherlands. Evolutionary operators in global optimization with dynamic search trajectories E.C. Laskari, K.E. Parsopoulos

### Introduction to optimization methods and line search

Introduction to optimization methods and line search Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi How to find optimal solutions? Trial and error widely used in practice, not efficient and

### Evolutionary Computation Algorithms for Cryptanalysis: A Study

Evolutionary Computation Algorithms for Cryptanalysis: A Study Poonam Garg Information Technology and Management Dept. Institute of Management Technology Ghaziabad, India pgarg@imt.edu Abstract The cryptanalysis

### IMPROVING THE PARTICLE SWARM OPTIMIZER BY FUNCTION STRETCHING

Chapter 3 IMPROVING THE PARTICLE SWARM OPTIMIZER BY FUNCTION STRETCHING K.E. Parsopoulos Department of Mathematics, University of Patras, GR 26110, Patras, Greece. kostasp@math.upatras.gr V.P. Plagianakos

### Introduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7.

Chapter 7: Derivative-Free Optimization Introduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7.5) Jyh-Shing Roger Jang et al.,

### Introduction to unconstrained optimization - derivative-free methods

Introduction to unconstrained optimization - derivative-free methods Jussi Hakanen Post-doctoral researcher Office: AgC426.3 jussi.hakanen@jyu.fi Learning outcomes To understand the basic principles of

### 336 THE STATISTICAL SOFTWARE NEWSLETTER where z is one (randomly taken) pole of the simplex S, g the centroid of the remaining d poles of the simplex

THE STATISTICAL SOFTWARE NEWSLETTER 335 Simple Evolutionary Heuristics for Global Optimization Josef Tvrdk and Ivan Krivy University of Ostrava, Brafova 7, 701 03 Ostrava, Czech Republic Phone: +420.69.6160

### Lecture 4. Convexity Robust cost functions Optimizing non-convex functions. 3B1B Optimization Michaelmas 2017 A. Zisserman

Lecture 4 3B1B Optimization Michaelmas 2017 A. Zisserman Convexity Robust cost functions Optimizing non-convex functions grid search branch and bound simulated annealing evolutionary optimization The Optimization

### Fast oriented bounding box optimization on the rotation group SO(3, R)

Fast oriented bounding box optimization on the rotation group SO(3, R) Chia-Tche Chang 1, Bastien Gorissen 2,3 and Samuel Melchior 1,2 chia-tche.chang@uclouvain.be bastien.gorissen@cenaero.be samuel.melchior@uclouvain.be

### PARTICLE SWARM OPTIMIZATION FOR IMPRECISE PROBLEMS

PARTICLE SWARM OPTIMIZATION FOR IMPRECISE PROBLEMS K. E. Parsopoulos Department of Mathematics, University of Patras Artificial Intelligence Research Center (UPAIRC), University of Patras, GR-26110 Patras,

### Discrete Optimization. Lecture Notes 2

Discrete Optimization. Lecture Notes 2 Disjunctive Constraints Defining variables and formulating linear constraints can be straightforward or more sophisticated, depending on the problem structure. The

### Package optimization

Type Package Package optimization October 24, 2017 Title Flexible Optimization of Complex Loss Functions with State and Parameter Space Constraints Version 1.0-7 Date 2017-10-20 Author Kai Husmann and

### Introduction to Design Optimization: Search Methods

Introduction to Design Optimization: Search Methods 1-D Optimization The Search We don t know the curve. Given α, we can calculate f(α). By inspecting some points, we try to find the approximated shape

### Artificial Intelligence

Artificial Intelligence Local Search Vibhav Gogate The University of Texas at Dallas Some material courtesy of Luke Zettlemoyer, Dan Klein, Dan Weld, Alex Ihler, Stuart Russell, Mausam Systematic Search:

### Automatic PID tuning, a heuristic optimization approach

Congreso Anual 2 de la Asociación de México de Control Automático. Puerto Vallarta, Jalisco, México. Automatic PID tuning, a heuristic optimization approach Rodrigo Abrajan-Guerrero, Luis Alejandro Marquez-Martinez

### Global optimization using Lévy flights

Global optimization using Lévy flights Truyen Tran, Trung Thanh Nguyen, Hoang Linh Nguyen September 2004 arxiv:1407.5739v1 [cs.ne] 22 Jul 2014 Abstract This paper studies a class of enhanced diffusion

### Introduction. Optimization

Introduction to Optimization Amy Langville SAMSI Undergraduate Workshop N.C. State University SAMSI 6/1/05 GOAL: minimize f(x 1, x 2, x 3, x 4, x 5 ) = x 2 1.5x 2x 3 + x 4 /x 5 PRIZE: \$1 million # of independent

### Time Complexity Analysis of the Genetic Algorithm Clustering Method

Time Complexity Analysis of the Genetic Algorithm Clustering Method Z. M. NOPIAH, M. I. KHAIRIR, S. ABDULLAH, M. N. BAHARIN, and A. ARIFIN Department of Mechanical and Materials Engineering Universiti

### Algoritmi di Ottimizzazione: Parte A Aspetti generali e algoritmi classici

Identificazione e Controllo Intelligente Algoritmi di Ottimizzazione: Parte A Aspetti generali e algoritmi classici David Naso A.A. 2006-2007 Identificazione e Controllo Intelligente 1 Search and optimization

### Evolving Variable-Ordering Heuristics for Constrained Optimisation

Griffith Research Online https://research-repository.griffith.edu.au Evolving Variable-Ordering Heuristics for Constrained Optimisation Author Bain, Stuart, Thornton, John, Sattar, Abdul Published 2005

### Multidimensional Minimization

C. C. Kankelborg, 1999-11-18 (rev. 2009-Mar-24) I. INTRODUCTION Multidimensional Minimization This lecture is devoted to the task of minimization in N dimensions, which may be stated as follows: For a

### Binary Differential Evolution Strategies

Binary Differential Evolution Strategies A.P. Engelbrecht, Member, IEEE G. Pampará Abstract Differential evolution has shown to be a very powerful, yet simple, population-based optimization approach. The

### Multi Layer Perceptron trained by Quasi Newton learning rule

Multi Layer Perceptron trained by Quasi Newton learning rule Feed-forward neural networks provide a general framework for representing nonlinear functional mappings between a set of input variables and

### Using a Modified Genetic Algorithm to Find Feasible Regions of a Desirability Function

Using a Modified Genetic Algorithm to Find Feasible Regions of a Desirability Function WEN WAN 1 and JEFFREY B. BIRCH 2 1 Virginia Commonwealth University, Richmond, VA 23298-0032 2 Virginia Polytechnic

### VARIANCE REDUCTION TECHNIQUES IN MONTE CARLO SIMULATIONS K. Ming Leung

POLYTECHNIC UNIVERSITY Department of Computer and Information Science VARIANCE REDUCTION TECHNIQUES IN MONTE CARLO SIMULATIONS K. Ming Leung Abstract: Techniques for reducing the variance in Monte Carlo

### An Introduction to Evolutionary Algorithms

An Introduction to Evolutionary Algorithms Karthik Sindhya, PhD Postdoctoral Researcher Industrial Optimization Group Department of Mathematical Information Technology Karthik.sindhya@jyu.fi http://users.jyu.fi/~kasindhy/

### Combining Two Local Searches with Crossover: An Efficient Hybrid Algorithm for the Traveling Salesman Problem

Combining Two Local Searches with Crossover: An Efficient Hybrid Algorithm for the Traveling Salesman Problem Weichen Liu, Thomas Weise, Yuezhong Wu and Qi Qi University of Science and Technology of Chine

### A grid representation for Distributed Virtual Environments

A grid representation for Distributed Virtual Environments Pedro Morillo, Marcos Fernández, Nuria Pelechano Instituto de Robótica, Universidad de Valencia. Polígono Coma S/N. Aptdo.Correos 22085, CP: 46071

### Particle Swarm Optimization

Dario Schor, M.Sc., EIT schor@ieee.org Space Systems Department Magellan Aerospace Winnipeg Winnipeg, Manitoba 1 of 34 Optimization Techniques Motivation Optimization: Where, min x F(x), subject to g(x)

### Lecture 6 - Multivariate numerical optimization

Lecture 6 - Multivariate numerical optimization Björn Andersson (w/ Jianxin Wei) Department of Statistics, Uppsala University February 13, 2014 1 / 36 Table of Contents 1 Plotting functions of two variables

### 16.410/413 Principles of Autonomy and Decision Making

16.410/413 Principles of Autonomy and Decision Making Lecture 17: The Simplex Method Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology November 10, 2010 Frazzoli (MIT)

### A Nelder-Mead Tuner for Svm

A Nelder-Mead Tuner for Svm prepared by: Kester Smith approved by: reference: issue: 1 revision: 0 date: 2009-03-13 status: Draft Abstract Read at your own risk, as this is a working document and has not

### Sparse Matrices Reordering using Evolutionary Algorithms: A Seeded Approach

1 Sparse Matrices Reordering using Evolutionary Algorithms: A Seeded Approach David Greiner, Gustavo Montero, Gabriel Winter Institute of Intelligent Systems and Numerical Applications in Engineering (IUSIANI)

### A Random Number Based Method for Monte Carlo Integration

A Random Number Based Method for Monte Carlo Integration J Wang and G Harrell Department Math and CS, Valdosta State University, Valdosta, Georgia, USA Abstract - A new method is proposed for Monte Carlo

### Lecture: Iterative Search Methods

Lecture: Iterative Search Methods Overview Constructive Search is exponential. State-Space Search exhibits better performance on some problems. Research in understanding heuristic and iterative search

### CS 331: Artificial Intelligence Local Search 1. Tough real-world problems

CS 331: Artificial Intelligence Local Search 1 1 Tough real-world problems Suppose you had to solve VLSI layout problems (minimize distance between components, unused space, etc.) Or schedule airlines

### Simulated Annealing. Slides based on lecture by Van Larhoven

Simulated Annealing Slides based on lecture by Van Larhoven Iterative Improvement 1 General method to solve combinatorial optimization problems Principle: Start with initial configuration Repeatedly search

### Informed search algorithms. (Based on slides by Oren Etzioni, Stuart Russell)

Informed search algorithms (Based on slides by Oren Etzioni, Stuart Russell) The problem # Unique board configurations in search space 8-puzzle 9! = 362880 15-puzzle 16! = 20922789888000 10 13 24-puzzle

### A Modified Genetic Algorithm for Process Scheduling in Distributed System

A Modified Genetic Algorithm for Process Scheduling in Distributed System Vinay Harsora B.V.M. Engineering College Charatar Vidya Mandal Vallabh Vidyanagar, India Dr.Apurva Shah G.H.Patel College of Engineering

### Experience with the IMMa tyre test bench for the determination of tyre model parameters using genetic techniques

Vehicle System Dynamics Vol. 43, Supplement, 2005, 253 266 Experience with the IMMa tyre test bench for the determination of tyre model parameters using genetic techniques J. A. CABRERA*, A. ORTIZ, E.

### Introduction to Genetic Algorithms

Advanced Topics in Image Analysis and Machine Learning Introduction to Genetic Algorithms Week 3 Faculty of Information Science and Engineering Ritsumeikan University Today s class outline Genetic Algorithms

### NON-LINEAR INTERPOLATION OF MISSING IMAGE DATA USING MIN-MAX FUNCTIONS

NON-LINEAR INTERPOLATION OF MISSING IMAGE DATA USING MIN-MAX FUNCTIONS Steven Armstrong, And C. Kokaram, Peter J. W. Rayner Signal Processing and Communications Laboratory Cambridge University Engineering

### Short-Cut MCMC: An Alternative to Adaptation

Short-Cut MCMC: An Alternative to Adaptation Radford M. Neal Dept. of Statistics and Dept. of Computer Science University of Toronto http://www.cs.utoronto.ca/ radford/ Third Workshop on Monte Carlo Methods,

### Conflict-based Statistics

Conflict-based Statistics Tomáš Müller 1, Roman Barták 1 and Hana Rudová 2 1 Faculty of Mathematics and Physics, Charles University Malostranské nám. 2/25, Prague, Czech Republic {muller bartak}@ktiml.mff.cuni.cz

### A Direct Search Algorithm for Global Optimization

Article A Direct Search Algorithm for Global Optimization Enrique Baeyens, *, Alberto Herreros and José R. Perán 2 Instituto de las Tecnologías Avanzadas de la Producción (ITAP), Universidad de Valladolid,

### CS5401 FS2015 Exam 1 Key

CS5401 FS2015 Exam 1 Key This is a closed-book, closed-notes exam. The only items you are allowed to use are writing implements. Mark each sheet of paper you use with your name and the string cs5401fs2015

### Model Parameter Estimation

Model Parameter Estimation Shan He School for Computational Science University of Birmingham Module 06-23836: Computational Modelling with MATLAB Outline Outline of Topics Concepts about model parameter

### Constrained Functions of N Variables: Non-Gradient Based Methods

onstrained Functions of N Variables: Non-Gradient Based Methods Gerhard Venter Stellenbosch University Outline Outline onstrained Optimization Non-gradient based methods Genetic Algorithms (GA) Particle

### Tuning. Philipp Koehn presented by Gaurav Kumar. 28 September 2017

Tuning Philipp Koehn presented by Gaurav Kumar 28 September 2017 The Story so Far: Generative Models 1 The definition of translation probability follows a mathematical derivation argmax e p(e f) = argmax

### MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS

In: Journal of Applied Statistical Science Volume 18, Number 3, pp. 1 7 ISSN: 1067-5817 c 2011 Nova Science Publishers, Inc. MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS Füsun Akman

### PARAMETER OPTIMIZATION FOR AUTOMATED SIGNAL ANALYSIS FOR CONDITION MONITORING OF AIRCRAFT SYSTEMS. Mike Gerdes 1, Dieter Scholz 1

AST 2011 Workshop on Aviation System Technology PARAMETER OPTIMIZATION FOR AUTOMATED SIGNAL ANALYSIS FOR CONDITION MONITORING OF AIRCRAFT SYSTEMS Mike Gerdes 1, Dieter Scholz 1 1 Aero - Aircraft Design

### The Modified IWO Algorithm for Optimization of Numerical Functions

The Modified IWO Algorithm for Optimization of Numerical Functions Daniel Kostrzewa and Henryk Josiński Silesian University of Technology, Akademicka 16 PL-44-100 Gliwice, Poland {Daniel.Kostrzewa,Henryk.Josinski}@polsl.pl

### Algorithmic Game Theory and Applications. Lecture 6: The Simplex Algorithm

Algorithmic Game Theory and Applications Lecture 6: The Simplex Algorithm Kousha Etessami Recall our example 1 x + y

### 6. Dicretization methods 6.1 The purpose of discretization

6. Dicretization methods 6.1 The purpose of discretization Often data are given in the form of continuous values. If their number is huge, model building for such data can be difficult. Moreover, many

### A Novel Approach to Planar Mechanism Synthesis Using HEEDS

AB-2033 Rev. 04.10 A Novel Approach to Planar Mechanism Synthesis Using HEEDS John Oliva and Erik Goodman Michigan State University Introduction The problem of mechanism synthesis (or design) is deceptively

### Optimizing the TracePro Optimization Process

Optimizing the TracePro Optimization Process A TracePro Webinar December 17, 2014 Presenter Presenter Dave Jacobsen Sr. Application Engineer Lambda Research Corporation Moderator Mike Gauvin Vice President

### Constructive and destructive algorithms

Constructive and destructive algorithms Heuristic algorithms Giovanni Righini University of Milan Department of Computer Science (Crema) Constructive algorithms In combinatorial optimization problems every

### Simple shooting-projection method for numerical solution of two-point Boundary Value Problems

Simple shooting-projection method for numerical solution of two-point Boundary Value Problems Stefan M. Filipo Ivan D. Gospodino a Department of Programming and Computer System Application, University

### USING RANDOM LOCAL SEARCH HELPS IN AVOIDING LOCAL OPTIMUM IN DIFFERENTIAL EVOLUTION

Proceedings of the IASTED International Conference Artificial Intelligence and Applications (AIA 2014) February 17-19, 2014 Innsbruck, Austria USING RANDOM LOCAL SEARCH HELPS IN AVOIDING LOCAL OPTIMUM

### Mathematical Programming and Research Methods (Part II)

Mathematical Programming and Research Methods (Part II) 4. Convexity and Optimization Massimiliano Pontil (based on previous lecture by Andreas Argyriou) 1 Today s Plan Convex sets and functions Types

### Reducing Graphic Conflict In Scale Reduced Maps Using A Genetic Algorithm

Reducing Graphic Conflict In Scale Reduced Maps Using A Genetic Algorithm Dr. Ian D. Wilson School of Technology, University of Glamorgan, Pontypridd CF37 1DL, UK Dr. J. Mark Ware School of Computing,

### A Combined Meta-Heuristic with Hyper-Heuristic Approach to Single Machine Production Scheduling Problem

A Combined Meta-Heuristic with Hyper-Heuristic Approach to Single Machine Production Scheduling Problem C. E. Nugraheni, L. Abednego Abstract This paper is concerned with minimization of mean tardiness

### On a Class of Global Optimization Test Functions

On a Class of Global Optimization Test Functions Crina Grosan 1 and Ajith Abraham* 2 1 Department Of Computer Science Babes-Bolyai University, Cluj-Napoca, Romania Machine Intelligence Research Labs (MIR

### Offspring Generation Method using Delaunay Triangulation for Real-Coded Genetic Algorithms

Offspring Generation Method using Delaunay Triangulation for Real-Coded Genetic Algorithms Hisashi Shimosaka 1, Tomoyuki Hiroyasu 2, and Mitsunori Miki 2 1 Graduate School of Engineering, Doshisha University,

### A PARAMETRIC SIMPLEX METHOD FOR OPTIMIZING A LINEAR FUNCTION OVER THE EFFICIENT SET OF A BICRITERIA LINEAR PROBLEM. 1.

ACTA MATHEMATICA VIETNAMICA Volume 21, Number 1, 1996, pp. 59 67 59 A PARAMETRIC SIMPLEX METHOD FOR OPTIMIZING A LINEAR FUNCTION OVER THE EFFICIENT SET OF A BICRITERIA LINEAR PROBLEM NGUYEN DINH DAN AND

### Particle swarm algorithms for multi-local optimization A. Ismael F. Vaz 1, Edite M.G.P. Fernandes 1

I Congresso de Estatística e Investigação Operacional da Galiza e Norte de Portugal VII Congreso Galego de Estatística e Investigación de Operacións Guimarães 26, 27 e 28 de Outubro de 2005 Particle swarm

### Speeding up Evolutionary Algorithms Through Restricted Mutation Operators

Speeding up Evolutionary Algorithms Through Restricted Mutation Operators Benjamin Doerr 1, Nils Hebbinghaus 1 and Frank Neumann 2 1 Max-Planck-Institut für Informatik, 66123 Saarbrücken, Germany. 2 Institut

### Today. CS 188: Artificial Intelligence Fall Example: Boolean Satisfiability. Reminder: CSPs. Example: 3-SAT. CSPs: Queries.

CS 188: Artificial Intelligence Fall 2007 Lecture 5: CSPs II 9/11/2007 More CSPs Applications Tree Algorithms Cutset Conditioning Today Dan Klein UC Berkeley Many slides over the course adapted from either

### Dynamic local search algorithm for the clustering problem

UNIVERSITY OF JOENSUU DEPARTMENT OF COMPUTER SCIENCE Report Series A Dynamic local search algorithm for the clustering problem Ismo Kärkkäinen and Pasi Fränti Report A-2002-6 ACM I.4.2, I.5.3 UDK 519.712

### ^ Springer. Computational Intelligence. A Methodological Introduction. Rudolf Kruse Christian Borgelt. Matthias Steinbrecher Pascal Held

Rudolf Kruse Christian Borgelt Frank Klawonn Christian Moewes Matthias Steinbrecher Pascal Held Computational Intelligence A Methodological Introduction ^ Springer Contents 1 Introduction 1 1.1 Intelligent

### A New Exam Timetabling Algorithm

A New Exam Timetabling Algorithm K.J. Batenburg W.J. Palenstijn Leiden Institute of Advanced Computer Science (LIACS), Universiteit Leiden P.O. Box 9512, 2300 RA Leiden, The Netherlands {kbatenbu, wpalenst}@math.leidenuniv.nl

### The desirability Package

Max Kuhn max.kuhn@pfizer.com April 26, 27 Introduction The desirability package contains S3 classes for multivariate optimization using the desirability function approach of Harrington (965) using functional

### A Fair Extension of (Soft) Concurrent Constraint Languages

A Fair Extension of (Soft) Concurrent Constraint Languages Stefano Bistarelli 1,2,3 and Paola Campli 1 1 Department of Science, University G. d Annunzio of Chieti-Pescara, Italy [bista,campli]@sci.unich.it

### Scatter Search: Methodology and Applications

Scatter Search: Methodology and Applications Manuel Laguna University of Colorado Rafael Martí University of Valencia Based on Scatter Search: Methodology and Implementations in C Laguna, M. and R. Martí

### Computational Methods. Constrained Optimization

Computational Methods Constrained Optimization Manfred Huber 2010 1 Constrained Optimization Unconstrained Optimization finds a minimum of a function under the assumption that the parameters can take on

### 3 INTEGER LINEAR PROGRAMMING

3 INTEGER LINEAR PROGRAMMING PROBLEM DEFINITION Integer linear programming problem (ILP) of the decision variables x 1,..,x n : (ILP) subject to minimize c x j j n j= 1 a ij x j x j 0 x j integer n j=

### Using an outward selective pressure for improving the search quality of the MOEA/D algorithm

Comput Optim Appl (25) 6:57 67 DOI.7/s589-5-9733-9 Using an outward selective pressure for improving the search quality of the MOEA/D algorithm Krzysztof Michalak Received: 2 January 24 / Published online:

### Informed Search and Exploration

Informed Search and Exploration Chapter 4 (4.1-4.3) CS 2710 1 Introduction Ch.3 searches good building blocks for learning about search But vastly inefficient eg: Can we do better? Breadth Depth Uniform

### Non-Derivative Optimization: Mathematics or Heuristics?

Non-Derivative Optimization: Mathematics or Heuristics? Margaret H. Wright Computer Science Department Courant Institute of Mathematical Sciences New York University Kovalevskaya Colloquium Berlin Mathematical

### Intelligent Reduction of Tire Noise

Intelligent Reduction of Tire Noise Matthias Becker and Helena Szczerbicka University Hannover Welfengarten 3067 Hannover, Germany xmb@sim.uni-hannover.de Abstract. In this paper we report about deployment

### Chapter 8 SCATTER SEARCH 1. INTRODUCTION

Chapter 8 SCATTER SEARCH Manuel Laguna 1 and Rafael Martí 2 1 Leeds School of Business, University of Colorado at Boulder, laguna@colorado.edu; 2 Dpto. de Estadística e Investigación Operativa, Universidad

### Classification: Linear Discriminant Functions

Classification: Linear Discriminant Functions CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Discriminant functions Linear Discriminant functions

### A Hybrid Genetic Pattern Search Augmented Lagrangian Method for Constrained Global Optimization

A Hybrid Genetic Pattern Search Augmented Lagrangian Method for Constrained Global Optimization Lino Costa a, Isabel A.C.P. Espírito Santo a, Edite M.G.P. Fernandes b a Department of Production and Systems

### Reliable Nonlinear Parameter Estimation Using Interval Analysis: Error-in-Variable Approach

Reliable Nonlinear Parameter Estimation Using Interval Analysis: Error-in-Variable Approach Chao-Yang Gau and Mark A. Stadtherr Λ Department of Chemical Engineering University of Notre Dame Notre Dame,

### Simulated annealing/metropolis and genetic optimization

Simulated annealing/metropolis and genetic optimization Eugeniy E. Mikhailov The College of William & Mary Lecture 18 Eugeniy Mikhailov (W&M) Practical Computing Lecture 18 1 / 8 Nature s way to find a

### Ant Colony Optimization

Ant Colony Optimization CompSci 760 Patricia J Riddle 1 Natural Inspiration The name Ant Colony Optimization was chosen to reflect its original inspiration: the foraging behavior of some ant species. It

### Free-Form Shape Optimization using CAD Models

Free-Form Shape Optimization using CAD Models D. Baumgärtner 1, M. Breitenberger 1, K.-U. Bletzinger 1 1 Lehrstuhl für Statik, Technische Universität München (TUM), Arcisstraße 21, D-80333 München 1 Motivation

### Short Reminder of Nonlinear Programming

Short Reminder of Nonlinear Programming Kaisa Miettinen Dept. of Math. Inf. Tech. Email: kaisa.miettinen@jyu.fi Homepage: http://www.mit.jyu.fi/miettine Contents Background General overview briefly theory

### Random Search Report An objective look at random search performance for 4 problem sets

Random Search Report An objective look at random search performance for 4 problem sets Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA dwai3@gatech.edu Abstract: This report

### The Journal of MacroTrends in Technology and Innovation

MACROJOURNALS The Journal of MacroTrends in Technology and Innovation Automatic Knot Adjustment Using Dolphin Echolocation Algorithm for B-Spline Curve Approximation Hasan Ali AKYÜREK*, Erkan ÜLKER**,

### Analysis and optimization methods of graph based meta-models for data flow simulation

Rochester Institute of Technology RIT Scholar Works Theses Thesis/Dissertation Collections 8-1-2010 Analysis and optimization methods of graph based meta-models for data flow simulation Jeffrey Harrison

### Derivative-Free Optimization

Derivative-Free Optimization Chapter 7 from Jang Outline Simulated Annealing (SA) Downhill simplex search Random search Genetic algorithms (GA) 2 The Big Picture Model space Adaptive networks Neural networks

### Preprocessing of Stream Data using Attribute Selection based on Survival of the Fittest

Preprocessing of Stream Data using Attribute Selection based on Survival of the Fittest Bhakti V. Gavali 1, Prof. Vivekanand Reddy 2 1 Department of Computer Science and Engineering, Visvesvaraya Technological

### Vector Regression Machine. Rodrigo Fernandez. LIPN, Institut Galilee-Universite Paris 13. Avenue J.B. Clement Villetaneuse France.

Predicting Time Series with a Local Support Vector Regression Machine Rodrigo Fernandez LIPN, Institut Galilee-Universite Paris 13 Avenue J.B. Clement 9343 Villetaneuse France rf@lipn.univ-paris13.fr Abstract