Effects of Constant Optimization by Nonlinear Least Squares Minimization in Symbolic Regression


 Anissa Rice
 1 years ago
 Views:
Transcription
1 Effects of Constant Optimization by Nonlinear Least Squares Minimization in Symbolic Regression Michael Kommenda, Gabriel Kronberger, Stephan Winkler, Michael Affenzeller, and Stefan Wagner Contact: Michael Kommenda Heuristic and Evolutionary Algorithms Lab (HEAL) Softwarepark 11 A4232 Hagenberg Web:
2 Symbolic Regression Model a relationship between input variables x and target variable y without any predefined structure Minimization of ε using an evolutionary algorithm Model structure Used variables Constants / weights 2
3 Research Assumption The correct model structure is found during the algorithm execution, but not recognized due to misleading / wrong constants * X 0.6 * Y 5.0 * X * 2.0 * Y 0.3 3
4 Constants in Symbolic Regression Ephemeral Random Constants Randomly initialized constants Remain fixed during the algorithm run Evolutionary Constants Updated by mutation C new = C old + N 0, σ C new = C old N 1, σ 1.0 X 2.0 * 0.3 Finding correct constants combination of existing values mutation of constant symbol nodes undirected changes to values 4
5 Summary of Previous Research Faster genetic programming based on local gradient search of numeric leaf values (Topchy and Punch, GECCO 2001) Improving gene expression programming performance by using differential evolution (Zhang et al., ICMLA 2007) Evolution Strategies for Constants Optimization in Genetic Programming (Alonso, ICTAI 2009) Differential Evolution of Constants in Genetic Programming Improves Efficacy and Bloat (Mukherjee and Eppstein, GECCO 2012) 5
6 Linear Scaling Improving Symbolic Regression with Interval Arithmetic and Linear Scaling (Keijzer, EuroGP 2003) Use Pearson s R² as fitness function and perform linear scaling Removes necessity to find correct offset and scale Computationally efficient Outperforms the local gradient search 6
7 Constant Optimization Concept Treat all constants as parameters Local optimization step Multidimensional optimization LevenbergMarquardt Algorithm Least squares fitting of model parameters to empirical data m i=1 Minimize Q β = y i f x i, β 2 Uses gradient and Jacobian matrix information Implemented e.g. by ALGLIB 7
8 Gradient Calculation Transformation of symbolic expression tree Extract initial numerical values (starting point) Add scaling tree nodes + Automatic differentiation * β 6 Provided e.g. by AutoDiff Numerical gradient calculation in one pass  β 5 Faster compared to symbolic differentiation f = f, f,, β 1 β 2 f β n Update tree with optimized values βx 1 x + * 3.12 β 4 Optionally calculate new fitness 1.0 β β 3 8
9 Constant Optimization Improvement Improvement = Quality optimized Quality original Exemplary GP Run Average & median improvement stays constantly low Maximum improvement almost reaches the best quality found Crossover worsens good individuals The quality of few individuals can be dramatically increased 9
10 Problems Symbolic regression benchmarks Better GP Benchmarks: Community Survey Results and Proposals (White et al., GPEM 2013) Problem Function Training Test Nguyen7 f x = ln x ln (x 2 + 1) Keijzer6 f x, y, z = Vladislavleva4 f x 1,, x 5 = Pagie1 f x, y = 30xz x 10 y x i x y Poly10 f x 1,, x 10 = x 1 x 2 + x 3 x 4 + x 5 x 6 + x 1 x 7 x 9 + x 3 x 6 x Friedman2 f(x 1,, x 10 ) = 10 sin(πx 1 x 2 ) + 20 x x 4 + 5x 5 + N 0, Tower Real world data
11 Algorithm Configurations Genetic Programming with strict offspring selection Only child individuals with better quality compared to the fitter parent are accepted in the new generation Varying parameters Population size of 500, 1000, and 5000 for runs without constant optimization Probability for constant optimization 25%, 50%, and 100% (population size 500) All others parameters were not modified Maximum selection pressure of 100 was used as termination criterion Size constraints of tree length 50 and depth 12 Mutation rate of 25% Function set consists solely of arithmetic functions (except Nguyen7) 11
12 Results  Quality Success rate (test R² > 0.99) OSGP 500 OSGP 1000 OGSP 5000 CoOp 25% CoOp 50% CoOp 100% Nguyen7 Keijzer6 Vladislavleva4 Pagie1 Poly10 12
13 Results  Quality Noisy datasets Success rate not applicable R² of best training solution (μ ± σ) Configuration Friedman2 Tower Training Test Training Test OSGP ± ± ± ± OSGP ± ± ± ± OSGP ± ± ± ± CoOp 25% ± ± ± ± CoOp 50% ± ± ± ± CoOp 100% ± ± ± ±
14 Results LM Iterations Constant optimization probability of 50% Varying iterations for the LM algorithm (3x, 5x, 10x) success rate respectively test R² for noisy datasets Problem OGSP 5000 CoOp 50% 3x CoOp 50% 5x CoOp 50% 10x Nguyen Keijzer Vladislavleva Pagie Poly Friedman ± ± ± ± Tower ± ± ± ±
15 Results  Execution Effort Execution effort relative to OSGP OSGP 1000 OSGP 5000 CoOp 50% 3x CoOp 50% 5x CoOp 50% 10x Nguyen7 Keijzer6 Valdislavleva4 Paige1 Poly10 Friedmann2 Tower 16
16 Feature Selection Problems Artificial datasets 100 input variables Ɲ(0,1) Linear combination of 10/25 variables with weights U 0,10 noisy max R² = 0.90 Training 120 rows, Test 500 rows Population size 500 Constant optimization 50% 5x 1 Training Test Observation Constant optimization can lead to overfitting Selection of correct features is also an issue OSGP CoOp OSGP CoOp 10 Features 25 Features 17
17 Conclusion Constant optimization improves the success rate and quality of models Better results with smaller population size Especially useful for postprocessing of models Removes the effort of evolving correct constants Genetic programming can concentrate on the model structure and feature selection Readytouse implementation in HeuristicLab Configurable probability, iterations, random sampling All experiments available for download 19
18 Effects of Constant Optimization by Nonlinear Least Squares Minimization in Symbolic Regression Michael Kommenda, Gabriel Kronberger, Stephan Winkler, Michael Affenzeller, and Stefan Wagner Contact: Michael Kommenda Heuristic and Evolutionary Algorithms Lab (HEAL) Softwarepark 11 A4232 Hagenberg Web:
Effects of Constant Optimization by Nonlinear Least Squares Minimization in Symbolic Regression
Effects of Constant Optimization by Nonlinear Least Squares Minimization in Symbolic Regression Michael Kommenda, Gabriel Kronberger, Stephan Winkler, Michael Affenzeller, and Stefan Wagner Heuristic and
More informationarxiv: v1 [cs.ne] 19 Jun 2017
Unsure When to Stop? Ask Your Semantic Neighbors arxiv:16.6195v1 [cs.ne] 19 Jun 17 ABSTRACT Ivo Gonçalves NOVA IMS, Universidade Nova de Lisboa 312 Lisbon, Portugal igoncalves@novaims.unl.pt Carlos M.
More informationHybrid Adaptive Evolutionary Algorithm Hyper Heuristic
Hybrid Adaptive Evolutionary Algorithm Hyper Heuristic Jonatan Gómez Universidad Nacional de Colombia Abstract. This paper presents a hyper heuristic that is able to adapt two low level parameters (depth
More informationNeural Network Weight Selection Using Genetic Algorithms
Neural Network Weight Selection Using Genetic Algorithms David Montana presented by: Carl Fink, Hongyi Chen, Jack Cheng, Xinglong Li, Bruce Lin, Chongjie Zhang April 12, 2005 1 Neural Networks Neural networks
More informationTheoretical Concepts of Machine Learning
Theoretical Concepts of Machine Learning Part 2 Institute of Bioinformatics Johannes Kepler University, Linz, Austria Outline 1 Introduction 2 Generalization Error 3 Maximum Likelihood 4 Noise Models 5
More informationApplication of Genetic Algorithms to CFD. Cameron McCartney
Application of Genetic Algorithms to CFD Cameron McCartney Introduction define and describe genetic algorithms (GAs) and genetic programming (GP) propose possible applications of GA/GP to CFD Application
More informationIntroduction to Genetic Algorithms
Advanced Topics in Image Analysis and Machine Learning Introduction to Genetic Algorithms Week 3 Faculty of Information Science and Engineering Ritsumeikan University Today s class outline Genetic Algorithms
More informationA SteadyState Genetic Algorithm for Traveling Salesman Problem with Pickup and Delivery
A SteadyState Genetic Algorithm for Traveling Salesman Problem with Pickup and Delivery Monika Sharma 1, Deepak Sharma 2 1 Research Scholar Department of Computer Science and Engineering, NNSS SGI Samalkha,
More informationGenetic programming. Lecture Genetic Programming. LISP as a GP language. LISP structure. Sexpressions
Genetic programming Lecture Genetic Programming CIS 412 Artificial Intelligence Umass, Dartmouth One of the central problems in computer science is how to make computers solve problems without being explicitly
More informationBinary Differential Evolution Strategies
Binary Differential Evolution Strategies A.P. Engelbrecht, Member, IEEE G. Pampará Abstract Differential evolution has shown to be a very powerful, yet simple, populationbased optimization approach. The
More informationAn Evolutionary Algorithm for the Multiobjective Shortest Path Problem
An Evolutionary Algorithm for the Multiobjective Shortest Path Problem Fangguo He Huan Qi Qiong Fan Institute of Systems Engineering, Huazhong University of Science & Technology, Wuhan 430074, P. R. China
More informationOutline. Motivation. Introduction of GAs. Genetic Algorithm 9/7/2017. Motivation Genetic algorithms An illustrative example Hypothesis space search
Outline Genetic Algorithm Motivation Genetic algorithms An illustrative example Hypothesis space search Motivation Evolution is known to be a successful, robust method for adaptation within biological
More informationUsing Genetic Algorithms to Solve the Box Stacking Problem
Using Genetic Algorithms to Solve the Box Stacking Problem Jenniffer Estrada, Kris Lee, Ryan Edgar October 7th, 2010 Abstract The box stacking or strip stacking problem is exceedingly difficult to solve
More informationModel Parameter Estimation
Model Parameter Estimation Shan He School for Computational Science University of Birmingham Module 0623836: Computational Modelling with MATLAB Outline Outline of Topics Concepts about model parameter
More informationBenefits of PluginBased Heuristic Optimization Software Systems
Benefits of PluginBased Heuristic Optimization Software Systems Stefan Wagner 1, Stephan Winkler 2, Erik Pitzer 2, Gabriel Kronberger 2, Andreas Beham 2, Roland Braune 2, Michael Affenzeller 1 Upper Austrian
More informationAn Adaption of the Schema Theorem to Various Crossover and Mutation Operators for a Music Segmentation Problem
An Adaption of the Schema Theorem to Various Crossover and Mutation Operators for a Music Segmentation Problem Brigitte Rafael brigitte.rafael@ heuristiclab.com Michael Affenzeller michael.affenzeller@fhhagenberg.at
More informationIntroduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7.
Chapter 7: DerivativeFree Optimization Introduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7.5) JyhShing Roger Jang et al.,
More informationSparse Matrices Reordering using Evolutionary Algorithms: A Seeded Approach
1 Sparse Matrices Reordering using Evolutionary Algorithms: A Seeded Approach David Greiner, Gustavo Montero, Gabriel Winter Institute of Intelligent Systems and Numerical Applications in Engineering (IUSIANI)
More informationClustering. Mihaela van der Schaar. January 27, Department of Engineering Science University of Oxford
Department of Engineering Science University of Oxford January 27, 2017 Many datasets consist of multiple heterogeneous subsets. Cluster analysis: Given an unlabelled data, want algorithms that automatically
More informationPlanning and Search. Genetic algorithms. Genetic algorithms 1
Planning and Search Genetic algorithms Genetic algorithms 1 Outline Genetic algorithms Representing states (individuals, or chromosomes) Genetic operations (mutation, crossover) Example Genetic algorithms
More informationGENETIC ALGORITHM with HandsOn exercise
GENETIC ALGORITHM with HandsOn exercise Adopted From Lecture by Michael Negnevitsky, Electrical Engineering & Computer Science University of Tasmania 1 Objective To understand the processes ie. GAs Basic
More information4/22/2014. Genetic Algorithms. Diwakar Yagyasen Department of Computer Science BBDNITM. Introduction
4/22/24 s Diwakar Yagyasen Department of Computer Science BBDNITM Visit dylycknow.weebly.com for detail 2 The basic purpose of a genetic algorithm () is to mimic Nature s evolutionary approach The algorithm
More informationSegmentation of Noisy Binary Images Containing Circular and Elliptical Objects using Genetic Algorithms
Segmentation of Noisy Binary Images Containing Circular and Elliptical Objects using Genetic Algorithms B. D. Phulpagar Computer Engg. Dept. P. E. S. M. C. O. E., Pune, India. R. S. Bichkar Prof. ( Dept.
More informationOPTIMIZATION METHODS. For more information visit: or send an to:
OPTIMIZATION METHODS modefrontier is a registered product of ESTECO srl Copyright ESTECO srl 19992007 For more information visit: www.esteco.com or send an email to: modefrontier@esteco.com NEOS Optimization
More informationMidterm Examination CS 5402: Introduction to Artificial Intelligence
Midterm Examination CS 542: Introduction to Artificial Intelligence March 9, 217 LAST NAME: FIRST NAME: Problem Score Max Score 1 15 2 17 3 12 4 6 5 12 6 14 7 15 8 9 Total 1 1 of 1 Question 1. [15] State
More informationUninformed Search Methods. Informed Search Methods. Midterm Exam 3/13/18. Thursday, March 15, 7:30 9:30 p.m. room 125 Ag Hall
Midterm Exam Thursday, March 15, 7:30 9:30 p.m. room 125 Ag Hall Covers topics through Decision Trees and Random Forests (does not include constraint satisfaction) Closed book 8.5 x 11 sheet with notes
More informationEvolving VariableOrdering Heuristics for Constrained Optimisation
Griffith Research Online https://researchrepository.griffith.edu.au Evolving VariableOrdering Heuristics for Constrained Optimisation Author Bain, Stuart, Thornton, John, Sattar, Abdul Published 2005
More informationA Genetic Algorithm for Graph Matching using Graph Node Characteristics 1 2
Chapter 5 A Genetic Algorithm for Graph Matching using Graph Node Characteristics 1 2 Graph Matching has attracted the exploration of applying new computing paradigms because of the large number of applications
More informationComprehensive and Automatic Fitness Landscape Analysis using HeuristicLab
Comprehensive and Automatic Fitness Landscape Analysis using HeuristicLab Erik Pitzer, Michael Aenzeller Andreas Beham, and Stefan Wagner Heuristic and Evolutionary Algorithms Lab School of Informatics,
More informationMulti Expression Programming. Mihai Oltean
Multi Expression Programming Mihai Oltean Department of Computer Science, Faculty of Mathematics and Computer Science, BabeşBolyai University, Kogălniceanu 1, ClujNapoca, 3400, Romania. email: mihai.oltean@gmail.com
More informationUNBIASED ESTIMATION OF DESTINATION CHOICE MODELS WITH ATTRACTION CONSTRAINTS
UNBIASED ESTIMATION OF DESTINATION CHOICE MODELS WITH ATTRACTION CONSTRAINTS Vince Bernardin, PhD Steven Trevino John Gliebe, PhD APRIL 14, 2014 WHAT S WRONG WITH ESTIMATING DOUBLY CONSTRAINED DESTINATION
More informationInternal vs. External Parameters in Fitness Functions
Internal vs. External Parameters in Fitness Functions Pedro A. DiazGomez Computing & Technology Department Cameron University Lawton, Oklahoma 73505, USA pdiazgo@cameron.edu Dean F. Hougen School of
More informationGenetic Programming in the Wild:
Genetic Programming in the Wild: and orlovm, sipper@cs.bgu.ac.il Department of Computer Science BenGurion University, Israel GECCO 2009, July 8 12 Montréal, Québec, Canada 1 / 46 GP: Programs or Representations?
More informationLecture 4. Convexity Robust cost functions Optimizing nonconvex functions. 3B1B Optimization Michaelmas 2017 A. Zisserman
Lecture 4 3B1B Optimization Michaelmas 2017 A. Zisserman Convexity Robust cost functions Optimizing nonconvex functions grid search branch and bound simulated annealing evolutionary optimization The Optimization
More informationUsing an outward selective pressure for improving the search quality of the MOEA/D algorithm
Comput Optim Appl (25) 6:57 67 DOI.7/s589597339 Using an outward selective pressure for improving the search quality of the MOEA/D algorithm Krzysztof Michalak Received: 2 January 24 / Published online:
More informationJHPCSN: Volume 4, Number 1, 2012, pp. 17
JHPCSN: Volume 4, Number 1, 2012, pp. 17 QUERY OPTIMIZATION BY GENETIC ALGORITHM P. K. Butey 1, Shweta Meshram 2 & R. L. Sonolikar 3 1 Kamala Nehru Mahavidhyalay, Nagpur. 2 Prof. Priyadarshini Institute
More informationMATLAB Based Optimization Techniques and Parallel Computing
MATLAB Based Optimization Techniques and Parallel Computing Bratislava June 4, 2009 2009 The MathWorks, Inc. JörgM. Sautter Application Engineer The MathWorks Agenda Introduction Local and Smooth Optimization
More informationGenetic Algorithm for Job Shop Scheduling
Genetic Algorithm for Job Shop Scheduling Mr.P.P.Bhosale Department Of Computer Science and Engineering, SVERI s College Of Engineering Pandharpur, Solapur University Solapur Mr.Y.R.Kalshetty Department
More informationGenetic Algorithms. Kang Zheng Karl Schober
Genetic Algorithms Kang Zheng Karl Schober Genetic algorithm What is Genetic algorithm? A genetic algorithm (or GA) is a search technique used in computing to find true or approximate solutions to optimization
More informationStefan WAGNER *, Michael AFFENZELLER * HEURISTICLAB GRID A FLEXIBLE AND EXTENSIBLE ENVIRONMENT 1. INTRODUCTION
heuristic optimization, distributed computation, optimization frameworks Stefan WAGNER *, Michael AFFENZELLER * HEURISTICLAB GRID A FLEXIBLE AND EXTENSIBLE ENVIRONMENT FOR PARALLEL HEURISTIC OPTIMIZATION
More informationPreprocessing of Stream Data using Attribute Selection based on Survival of the Fittest
Preprocessing of Stream Data using Attribute Selection based on Survival of the Fittest Bhakti V. Gavali 1, Prof. Vivekanand Reddy 2 1 Department of Computer Science and Engineering, Visvesvaraya Technological
More informationA Genetic Algorithm Approach for Clustering
www.ijecs.in International Journal Of Engineering And Computer Science ISSN:23197242 Volume 3 Issue 6 June, 2014 Page No. 64426447 A Genetic Algorithm Approach for Clustering Mamta Mor 1, Poonam Gupta
More informationUniform Linear Transformation with Repair and Alternation in Genetic Programming
Preprint, to appear as: Spector, L., and T. Helmuth. 2014. Uniform Linear Transformation with Repair and Alternation in Genetic Programming. In Genetic Programming Theory and Practice XI, edited by R.
More informationReducing Graphic Conflict In Scale Reduced Maps Using A Genetic Algorithm
Reducing Graphic Conflict In Scale Reduced Maps Using A Genetic Algorithm Dr. Ian D. Wilson School of Technology, University of Glamorgan, Pontypridd CF37 1DL, UK Dr. J. Mark Ware School of Computing,
More informationCONCEPT FORMATION AND DECISION TREE INDUCTION USING THE GENETIC PROGRAMMING PARADIGM
1 CONCEPT FORMATION AND DECISION TREE INDUCTION USING THE GENETIC PROGRAMMING PARADIGM John R. Koza Computer Science Department Stanford University Stanford, California 94305 USA EMAIL: Koza@Sunburn.Stanford.Edu
More informationAn Introduction to Evolutionary Algorithms
An Introduction to Evolutionary Algorithms Karthik Sindhya, PhD Postdoctoral Researcher Industrial Optimization Group Department of Mathematical Information Technology Karthik.sindhya@jyu.fi http://users.jyu.fi/~kasindhy/
More informationA New Crossover Technique for Cartesian Genetic Programming
A New Crossover Technique for Cartesian Genetic Programming Genetic Programming Track Janet Clegg Intelligent Systems Group, Department of Electronics University of York, Heslington York,YODD,UK jc@ohm.york.ac.uk
More informationGA is the most popular population based heuristic algorithm since it was developed by Holland in 1975 [1]. This algorithm runs faster and requires les
Chaotic Crossover Operator on Genetic Algorithm Hüseyin Demirci Computer Engineering, Sakarya University, Sakarya, 54187, Turkey Ahmet Turan Özcerit Computer Engineering, Sakarya University, Sakarya, 54187,
More informationAnalytical model A structure and process for analyzing a dataset. For example, a decision tree is a model for the classification of a dataset.
Glossary of data mining terms: Accuracy Accuracy is an important factor in assessing the success of data mining. When applied to data, accuracy refers to the rate of correct values in the data. When applied
More informationDesign of an Optimal Nearest Neighbor Classifier Using an Intelligent Genetic Algorithm
Design of an Optimal Nearest Neighbor Classifier Using an Intelligent Genetic Algorithm ShinnYing Ho *, ChiaCheng Liu, Soundy Liu, and JunWen Jou Department of Information Engineering, Feng Chia University,
More informationGenetic Algorithm for Network Design ProblemAn Empirical Study of Crossover Operator with Generation and Population Variation
International Journal of Information Technology and Knowledge Management JulyDecember 2010, Volume 2, No. 2, pp. 605611 Genetic Algorithm for Network Design ProblemAn Empirical Study of Crossover Operator
More informationRESPONSE SURFACE METHODOLOGIES  METAMODELS
RESPONSE SURFACE METHODOLOGIES  METAMODELS Metamodels Metamodels (or surrogate models, response surface models  RSM), are analytic models that approximate the multivariate input/output behavior of complex
More informationA Genetic Algorithm Framework
Fast, good, cheap. Pick any two. The Project Triangle 3 A Genetic Algorithm Framework In this chapter, we develop a genetic algorithm based framework to address the problem of designing optimal networks
More informationAutomata Construct with Genetic Algorithm
Automata Construct with Genetic Algorithm Vít Fábera Department of Informatics and Telecommunication, Faculty of Transportation Sciences, Czech Technical University, Konviktská 2, Praha, Czech Republic,
More informationArtificial Intelligence Application (Genetic Algorithm)
Babylon University College of Information Technology Software Department Artificial Intelligence Application (Genetic Algorithm) By Dr. Asaad Sabah Hadi 20142015 EVOLUTIONARY ALGORITHM The main idea about
More informationIntroduction to Artificial Intelligence
Introduction to Artificial Intelligence COMP307 Evolutionary Computing 3: Genetic Programming for Regression and Classification Yi Mei yi.mei@ecs.vuw.ac.nz 1 Outline Statistical parameter regression Symbolic
More informationComputer Science, UCL, London
Genetically Improved CUDA C++ Software W. B. Langdon Computer Science, UCL, London 26.4.2014 Genetically Improved CUDA C++ Software W. B. Langdon Centre for Research on Evolution, Search and Testing Computer
More informationAn Evolutionary Algorithm for Minimizing Multimodal Functions
An Evolutionary Algorithm for Minimizing Multimodal Functions D.G. Sotiropoulos, V.P. Plagianakos and M.N. Vrahatis University of Patras, Department of Mamatics, Division of Computational Mamatics & Informatics,
More informationWhat is GOSET? GOSET stands for Genetic Optimization System Engineering Tool
Lecture 5: GOSET 1 What is GOSET? GOSET stands for Genetic Optimization System Engineering Tool GOSET is a MATLAB based genetic algorithm toolbox for solving optimization problems 2 GOSET Features Wide
More informationUsing a genetic algorithm for editing knearest neighbor classifiers
Using a genetic algorithm for editing knearest neighbor classifiers R. GilPita 1 and X. Yao 23 1 Teoría de la Señal y Comunicaciones, Universidad de Alcalá, Madrid (SPAIN) 2 Computer Sciences Department,
More informationUniversité Libre de Bruxelles
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Exposing a Bias Toward ShortLength Numbers in Grammatical Evolution Marco A.
More informationDifferential Evolution
Chapter 13 Differential Evolution Differential evolution (DE) is a stochastic, populationbased search strategy developed by Storn and Price [696, 813] in 1995. While DE shares similarities with other
More informationMultiobjective Optimization Using Adaptive Pareto Archived Evolution Strategy
Multiobjective Optimization Using Adaptive Pareto Archived Evolution Strategy Mihai Oltean BabeşBolyai University Department of Computer Science Kogalniceanu 1, ClujNapoca, 3400, Romania moltean@cs.ubbcluj.ro
More informationAn evolutionary annealingsimplex algorithm for global optimisation of water resource systems
FIFTH INTERNATIONAL CONFERENCE ON HYDROINFORMATICS 15 July 2002, Cardiff, UK C05  Evolutionary algorithms in hydroinformatics An evolutionary annealingsimplex algorithm for global optimisation of water
More informationHardware Neuronale Netzwerke  Lernen durch künstliche Evolution (?)
SKIP  May 2004 Hardware Neuronale Netzwerke  Lernen durch künstliche Evolution (?) S. G. Hohmann, Electronic Vision(s), Kirchhoff Institut für Physik, Universität Heidelberg Hardware Neuronale Netzwerke
More informationCoevolutionary Cartesian Genetic Programming in FPGA
Coevolutionary Cartesian Genetic Programming in FPGA Radek Hrbáček and Michaela Šikulová Brno University of Technology, Faculty of Information Technology, Božetěchova 2, 612 66 Brno, Czech Republic xhrbac01@stud.fit.vutbr.cz,
More informationNonparametric Methods Recap
Nonparametric Methods Recap Aarti Singh Machine Learning 10701/15781 Oct 4, 2010 Nonparametric Methods Kernel Density estimate (also Histogram) Weighted frequency Classification  KNN Classifier Majority
More informationA Genetic Approach for Solving Minimum Routing Cost Spanning Tree Problem
A Genetic Approach for Solving Minimum Routing Cost Spanning Tree Problem Quoc Phan Tan Abstract Minimum Routing Cost Spanning Tree (MRCT) is one of spanning tree optimization problems having several applications
More informationIntroduction to Design Optimization: Search Methods
Introduction to Design Optimization: Search Methods 1D Optimization The Search We don t know the curve. Given α, we can calculate f(α). By inspecting some points, we try to find the approximated shape
More informationEvolving Evolutionary Algorithms with Patterns
Evolving Evolutionary Algorithms with Patterns Mihai Oltean Department of Computer Science Faculty of Mathematics and Computer Science BabeşBolyai University, Kogălniceanu 1 ClujNapoca, 3400, Romania
More informationA Modified Genetic Algorithm for Process Scheduling in Distributed System
A Modified Genetic Algorithm for Process Scheduling in Distributed System Vinay Harsora B.V.M. Engineering College Charatar Vidya Mandal Vallabh Vidyanagar, India Dr.Apurva Shah G.H.Patel College of Engineering
More informationCHAPTER 4 GENETIC ALGORITHM
69 CHAPTER 4 GENETIC ALGORITHM 4.1 INTRODUCTION Genetic Algorithms (GAs) were first proposed by John Holland (Holland 1975) whose ideas were applied and expanded on by Goldberg (Goldberg 1989). GAs is
More informationEvolving Radial Basis Function Networks via GP for Estimating Fitness Values using Surrogate Models
Evolving Radial Basis Function Networks via GP for Estimating Fitness Values using Surrogate Models Ahmed Kattan and Edgar Galvan Abstract In realworld problems with candidate solutions that are very
More informationA Comparison of Several Linear Genetic Programming Techniques
A Comparison of Several Linear Genetic Programming Techniques Mihai Oltean Crina Groşan Department of Computer Science, Faculty of Mathematics and Computer Science, BabesBolyai University, Kogalniceanu
More informationInformed search algorithms. (Based on slides by Oren Etzioni, Stuart Russell)
Informed search algorithms (Based on slides by Oren Etzioni, Stuart Russell) The problem # Unique board configurations in search space 8puzzle 9! = 362880 15puzzle 16! = 20922789888000 10 13 24puzzle
More informationGenetic Algorithms with Mapreduce Runtimes
Genetic Algorithms with Mapreduce Runtimes Fei Teng 1, Doga Tuncay 2 Indiana University Bloomington School of Informatics and Computing Department CS PhD Candidate 1, Masters of CS Student 2 {feiteng,dtuncay}@indiana.edu
More informationEvolutionary Algorithms. CS Evolutionary Algorithms 1
Evolutionary Algorithms CS 478  Evolutionary Algorithms 1 Evolutionary Computation/Algorithms Genetic Algorithms l Simulate natural evolution of structures via selection and reproduction, based on performance
More informationCS5401 FS2015 Exam 1 Key
CS5401 FS2015 Exam 1 Key This is a closedbook, closednotes exam. The only items you are allowed to use are writing implements. Mark each sheet of paper you use with your name and the string cs5401fs2015
More informationA More Stable Approach To LISP Tree GP
A More Stable Approach To LISP Tree GP Joseph Doliner August 15, 2008 Abstract In this paper we begin by familiarising ourselves with the basic concepts of Evolutionary Computing and how it can be used
More informationConstructing an Optimisation Phase Using Grammatical Evolution. Brad Alexander and Michael Gratton
Constructing an Optimisation Phase Using Grammatical Evolution Brad Alexander and Michael Gratton Outline Problem Experimental Aim Ingredients Experimental Setup Experimental Results Conclusions/Future
More informationLINEAR AND NONLINEAR IMAGE RESTORATION METHODS FOR EDDY
LINEAR AND NONLINEAR IMAGE RESTORATION METHODS FOR EDDY CURRENT NONDESTRUCTIVE EVALUATION Bing Wang:12 John P. Basart,12 and John C. Moulderl lcenter for NDE 2Department of Electrical and Computer Engineering
More informationCorrespondence. Object Detection via Feature Synthesis Using MDLBased Genetic Programming
538 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART B: CYBERNETICS, VOL. 35, NO. 3, JUNE 2005 Correspondence Object Detection via Feature Synthesis Using MDLBased Genetic Programming Yingqiang
More informationUNIVERSITY OF DORTMUND
UNIVERSITY OF DORTMUND REIHE COMPUTATIONAL INTELLIGENCE COLLABORATIVE RESEARCH CENTER 531 Design and Management of Complex Technical Processes and Systems by means of Computational Intelligence Methods
More informationEvolutionary Computation Algorithms for Cryptanalysis: A Study
Evolutionary Computation Algorithms for Cryptanalysis: A Study Poonam Garg Information Technology and Management Dept. Institute of Management Technology Ghaziabad, India pgarg@imt.edu Abstract The cryptanalysis
More informationA Fitness Function to Find Feasible Sequences of Method Calls for Evolutionary Testing of ObjectOriented Programs
A Fitness Function to Find Feasible Sequences of Method Calls for Evolutionary Testing of ObjectOriented Programs Myoung Yee Kim and Yoonsik Cheon TR #757 November 7; revised January Keywords: fitness
More informationOffspring Generation Method using Delaunay Triangulation for RealCoded Genetic Algorithms
Offspring Generation Method using Delaunay Triangulation for RealCoded Genetic Algorithms Hisashi Shimosaka 1, Tomoyuki Hiroyasu 2, and Mitsunori Miki 2 1 Graduate School of Engineering, Doshisha University,
More informationEvolutionary Algorithms
Evolutionary Algorithms Per Kristian Lehre School of Computer Science University of Nottingham NATCOR 2016 Evolutionary Algorithms  Per Kristian Lehre 1/46 Optimisation Given a function f : X R, find
More informationWhat is machine learning?
Machine learning, pattern recognition and statistical data modelling Lecture 12. The last lecture Coryn BailerJones 1 What is machine learning? Data description and interpretation finding simpler relationship
More information11/17/2009 Comp 590/Comp Fall
Lecture 20: Clustering and Evolution Study Chapter 10.4 10.8 Problem Set #5 will be available tonight 11/17/2009 Comp 590/Comp 79090 Fall 2009 1 Clique Graphs A clique is a graph with every vertex connected
More informationIntroduction to optimization methods and line search
Introduction to optimization methods and line search Jussi Hakanen Postdoctoral researcher jussi.hakanen@jyu.fi How to find optimal solutions? Trial and error widely used in practice, not efficient and
More informationUsing a Modified Genetic Algorithm to Find Feasible Regions of a Desirability Function
Using a Modified Genetic Algorithm to Find Feasible Regions of a Desirability Function WEN WAN 1 and JEFFREY B. BIRCH 2 1 Virginia Commonwealth University, Richmond, VA 232980032 2 Virginia Polytechnic
More informationAutoconstructive Evolution for Structural Problems
Autoconstructive Evolution for Structural Problems ABSTRACT Kyle I. Harrington DEMO Lab Brandeis University Waltham, MA, USA kyleh@cs.brandeis.edu Jordan B. Pollack DEMO Lab Brandeis University Waltham,
More informationApplication of Genetic Algorithm Based Intuitionistic Fuzzy kmode for Clustering Categorical Data
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 17, No 4 Sofia 2017 Print ISSN: 13119702; Online ISSN: 13144081 DOI: 10.1515/cait20170044 Application of Genetic Algorithm
More informationImproved SCDAS using Crossover Controlling the Number of Crossed Genes for Manyobjective Optimization
Improved SCDAS using Crossover Controlling the Number of Crossed Genes for Manyobjective Optimization Hiroyuki Sato Faculty of Informatics and Engineering, The University of ElectroCommunications 5
More informationCombining Two Local Searches with Crossover: An Efficient Hybrid Algorithm for the Traveling Salesman Problem
Combining Two Local Searches with Crossover: An Efficient Hybrid Algorithm for the Traveling Salesman Problem Weichen Liu, Thomas Weise, Yuezhong Wu and Qi Qi University of Science and Technology of Chine
More informationMultiresponse Sparse Regression with Application to Multidimensional Scaling
Multiresponse Sparse Regression with Application to Multidimensional Scaling Timo Similä and Jarkko Tikka Helsinki University of Technology, Laboratory of Computer and Information Science P.O. Box 54,
More informationTesi di Laurea ALGORITMI GENETICI PER LA STATISTICA
UNIVERSITÀ DEGLI STUDI DI PADOVA Facoltà di Scienze Statistiche Corso di Laurea Triennale in Statistica e Gestione delle Imprese Tesi di Laurea ALGORITMI GENETICI PER LA STATISTICA Relatore: Ch.mo prof.
More informationHyperparameter optimization. CS6787 Lecture 6 Fall 2017
Hyperparameter optimization CS6787 Lecture 6 Fall 2017 Review We ve covered many methods Stochastic gradient descent Step size/learning rate, how long to run Minibatching Batch size Momentum Momentum
More informationGenetic Algorithm and Direct Search Toolbox
Genetic Algorithm and Direct Search Toolbox For Use with MATLAB User s Guide Version 1 How to Contact The MathWorks: www.mathworks.com comp.softsys.matlab support@mathworks.com suggest@mathworks.com bugs@mathworks.com
More informationCHAPTER VI BACK PROPAGATION ALGORITHM
6.1 Introduction CHAPTER VI BACK PROPAGATION ALGORITHM In the previous chapter, we analysed that multiple layer perceptrons are effectively applied to handle tricky problems if trained with a vastly accepted
More information