Outline of the module
|
|
- Wendy King
- 5 years ago
- Views:
Transcription
1 Evolutionary and Heuristic Optimisation (ITNPD8) Lecture 2: Heuristics and Metaheuristics Gabriela Ochoa Computing Science and Mathematics, School of Natural Sciences University of Stirling, Stirling, Scotland Outline of the module 1. Optimisation problems Optimisation & search Classic mathematical models Example problems (Knapsack, TSP, others) 2. Optimisation methods Heuristics and metaheuristcis Single point algorithms Population-based algorithms 3. Advanced topics Fitness landscape analysis Multi-objective optimisation Gabriela Ochoa, 2 1
2 Optimisation/search algorithms Guarantee finding optimal solution Useful when problems can be solved in Polynomial time, or for small instances Optimisation algorithms Do not Guarantee finding optimal solution For most interesting optimisation problems no polynomial methods are known Exact Approximate Special purpose General purpose Special purpose Meta and Hyper heuristics Generate bounds: dual ascent, Langrangean relax Branch and bound Cutting planes Approximation Greedy / Constructive Heuristics Single point Population based Approximation algorithms: An attempt to formalise heuristics (emerged from the field of theoretical computer science) Polynomial time heuristics that provide some sort of guarantee on the quality of the solution 3 Terminology and dates Heuristic: Greek word heuriskein, the art of discovering new strategies to solve problems Heuristics for solving optimization problems, G. Poyla (1945) A method for helping in solving of a problem, commonly informal rules of thumb, educated guesses, or simply common sense Prefix meta: Greek for upper level methodology Metaheuristics: term was introduced by Fred Glover (1986). Other terms: modern heuristics, heuristic optimisation, stochastic local search G. Poyla, How to Solve it. Princeton University Press, Princeton NJ, 1945 F. Glover, Future Paths for Integer Programming and Links to Artificial Intelligence, Computers & Ops. Res, Vol. 13, No.5, pp ,
3 What is a heuristic? An optimisation method that tries to exploit problem-specific knowledge, for which we have no guarantee to find the optimal solution Construction Search space: partial candidate solutions Search step: extension with one or more solution components Example in TSP: nearest neighbour Improvement Search space: complete candidate solutions Search step: modification of one or more solution components Example in TSP: 2-opt Video for TSP: Gabriela Ochoa, goc@stir.ac.uk 5 What is a metaheuristic? Extended variants of improvement heuristics General-purpose solvers, usually applicable to a large variety of problems Use two phases during search Intensification (exploitation): focus the applications of operators on high-quality solutions Diversification (exploration): systematically modifies existing solutions such as new areas of the search space are explored Gabriela Ochoa, goc@stir.ac.uk 6 3
4 Genealogy of metaheuristics The Simplex Algorithm (G. Dantzig, 1947) (J.Edmonds, 1971): Metaheuristics: From Design to Implementation By El-Ghazali Talbi (2009) Gabriela Ochoa, 7 Classification of Metaheuristcs Different ways of classifying metaheuristics Nature-inspired vs. non-nature in-spired Population-based vs. single point search Dynamic vs. static objective function One vs. various neighbourhood structures Memory usage vs. memory-less methods. Gabriela Ochoa, 8 4
5 Key components of metaheuristics Problem Representation Describes encoding of solutions Application of search operators Fitness Function Search/Variation Operators Initial Solution(s) Often same as the objective function Extensions might be necessary (e.g.. Infeasible solutions) Closely related to the representation Mutation, recombination, ruin-recreate Created randomly Seeding with higher quality or biased solutions Search Strategy Defines intensification/diversification mechanisms Many possibilities and alternatives! Gabriela Ochoa, The knapsack problem Formulation. Mathematical model maximise 4x 1 +2x 2 +x 3 +10x 4 +2x 5 subject to 12x 1 +2x 2 +x 3 +4x 4 +x 5 15 x 1,x 2,x 3,x 4,x 5 {0, 1} X i = 1 If we select item i 0 Otherwise Search space size = 2 n n = 100, A thief breaks into a store and wants to fill his knapsack with as much value in goods as possible before making his escape. Given the following list of items available, what should he take? Item A, weighting wa kg and valued at va tem B, weighting wb kg and valued at vb tem C, weighting wc kg and valued at vc Gabriela Ochoa, goc@stir.ac.uk 10 5
6 The knapsack problem Input Capacity K n items with weights w i and values v i Goal Output a set of items s such that the sum of weights of items in s is at most K and the sum of values of items in s is maximized Gabriela Ochoa, goc@stir.ac.uk 11 Exhaustive enumeration Try out all possible ways of packing/leaving out the items For each way, it is easy to calculate the total weight carried and the total value carried Consider the following knapsack problem instance: Where: The first line gives the number of items. The last line gives the capacity of the knapsack. The remaining lines give the index, value and weight of each item. 12 Question: How many possible solutions can you enumerate? 6
7 Knapsack, full enumeration Items Value Weight Feasible? Yes Yes Yes No Yes Yes No No Optimal!! 13 The notion of Neighbourhood Region of the search space that is near to some particular point in that space Define a distance function dist on the search space S Dist: S x S R N(x) = {y Є S: dist(x,y) ε } Examples: Euclidean distance, for search spaces defined over continuous variables Hamming distance, for search spaces defined over binary strings Hamming distance between two strings is the number of positions at which the corresponding symbols are different. A search space S, a potential solution x, and its neighbourhood N(x) x N(x) S 14 7
8 Defining neighbourhoods Binary representation 1-flip: Solutions generated by flipping a single bit in the given bit string If the string length is n, how many neighbours each solution has? Example: , 2-flip: Solutions generated by flipping two bits in the given bit string If the string length is n, how many neighbours each solution has? Example: k-flip: can be generalised for larger k, k < n Gabriela Ochoa, goc@stir.ac.uk 15 Defining neighbourhoods Permutation 2-swap: Solutions generated by swapping two cities from a given tour Every solution has n(n-1)/2 neighbours Examples: ,
9 The notions of Local Optimum and Global Optimum A solution is a Local Optimum, if no solution in its neighbourhood has a better evaluation. A solution is the Global Optimum, if it no solution in the whole search space has a better evaluation. Maximisation problem: local maximum and global maximum Minimisation problem: Local minimum and global minimum. Plurals: optima, maxima, minima Gabriela Ochoa, goc@stir.ac.uk 17 Hill climbing (Iterative Improvement) algorithm (first Improvement, random mutation) procedure first-hill-climbing begin s = random initial solution solution repeat evaluate solution (s) s = random neighbour of s if evaluation(s ) is better than evaluation(s) s = s until stopping-criterion satisfied return s end The stopping criterion can be a fixed number of iterations. Gabriela Ochoa, goc@stir.ac.uk 18 9
10 Hill climbing algorithm (Best Improvement) procedure best-hill-climbing begin s = random initial solution solution repeat evaluate solution (s) s = best solution in the Neighbourhood if evaluation(s ) is better than evaluation(s) s = s until s is a local optimum return s end Gabriela Ochoa, goc@stir.ac.uk 19 Constraint handling Not trivial to deal withcontraints Reject strategies: only feasible solutions are kept Penalising strategies: penalty functions Repairing strategies: repairin feasibles olutions Decoding strategies: only feasible solutions are generated Preserving strategies: specific representation and search operators which preserve the feasibility Gabriela Ochoa, goc@stir.ac.uk 20 10
11 Hill-climbing search Problem: depending on initial state, can get stuck in local maxima Gabriela Ochoa, 21 Hill-climbing search global optimum hill climb er hill climber (3,4,1,2),395 local optimum 11
12 The Hill-climbing algorithms is blind Like climbing a mountain in thick fog with amnesia? hill climb er?? hill climber? Hill-climbing methods Weaknesses Usually terminate at solutions that are local optima No information as to how much the discovered local optimum deviates from the global (or even other local optima) Obtained optimum depends on starting point Usually no upper bound on computation time Advantages Very easy to apply. Only needs A representation The evaluation function A neighbourhood Used in combination of other heuristics (memetic algorithms, iterated local search) Gabriela Ochoa, goc@stir.ac.uk 24 12
13 Implementation: binary representation 1-flip mutation: flipping a single bit in the given bit string For strings of length n, every solution has n neighbours Example: , Python implementation 25 Practical exercises Lab1: Random search to solve the simple knapsack problem. The random mutation operation. Optional: exhaustive enumeration Lab2: Implement variants of Hill-climbing (Best- Improvement, First-Improvement) to solve the simple knapsack problem. Implement a multi-start hill-climbing. Optional: Iterated Local Search Lab3: Implement a simple Genetic Algorithm to solve the simple knapsack problem. Optional: Hybridise with hillclimbing Extra Lab: Catch up with Check points. Which of the algorithms implemented is best for this problem and why? Gabriela Ochoa, goc@stir.ac.uk 26 13
14 What do we mean by random search? Attempting several solutions at random, and keeping the best found. procedure random-search begin s = random initial solution solution repeat evaluate solution (s) s = random solution if evaluation(s ) is better than evaluation(s) s = s until stopping-criterion satisfied return s end The stopping criterion can be a fixed number of iterations. Gabriela Ochoa, goc@stir.ac.uk 27 14
ARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 5: EVOLUTIONARY ALGORITHMS
ARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 5: EVOLUTIONARY ALGORITHMS Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Optimisation problems Optimisation & search Two Examples The knapsack problem
More informationNon-deterministic Search techniques. Emma Hart
Non-deterministic Search techniques Emma Hart Why do local search? Many real problems are too hard to solve with exact (deterministic) techniques Modern, non-deterministic techniques offer ways of getting
More informationHeuristic Optimisation
Heuristic Optimisation Part 2: Basic concepts Sándor Zoltán Németh http://web.mat.bham.ac.uk/s.z.nemeth s.nemeth@bham.ac.uk University of Birmingham S Z Németh (s.nemeth@bham.ac.uk) Heuristic Optimisation
More informationHeuristic Optimisation
Heuristic Optimisation Revision Lecture Sándor Zoltán Németh http://web.mat.bham.ac.uk/s.z.nemeth s.nemeth@bham.ac.uk University of Birmingham S Z Németh (s.nemeth@bham.ac.uk) Heuristic Optimisation University
More informationOutline. Search-based Approaches and Hyper-heuristics. Optimisation problems 24/06/2014. Optimisation problems are everywhere!
Outline Search-based Approaches and Hyper- Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ Computing Science and Mathematics, School of Natural Sciences University of Stirling, Stirling, Scotland 1. Optimisation
More informationPre-requisite Material for Course Heuristics and Approximation Algorithms
Pre-requisite Material for Course Heuristics and Approximation Algorithms This document contains an overview of the basic concepts that are needed in preparation to participate in the course. In addition,
More informationHeuristic Optimization Introduction and Simple Heuristics
Heuristic Optimization Introduction and Simple Heuristics José M PEÑA (jmpena@fi.upm.es) (Universidad Politécnica de Madrid) 1 Outline 1. What are optimization problems? 2. Exhaustive vs. Heuristic approaches
More informationHybridization EVOLUTIONARY COMPUTING. Reasons for Hybridization - 1. Naming. Reasons for Hybridization - 3. Reasons for Hybridization - 2
Hybridization EVOLUTIONARY COMPUTING Hybrid Evolutionary Algorithms hybridization of an EA with local search techniques (commonly called memetic algorithms) EA+LS=MA constructive heuristics exact methods
More informationTABU search and Iterated Local Search classical OR methods
TABU search and Iterated Local Search classical OR methods tks@imm.dtu.dk Informatics and Mathematical Modeling Technical University of Denmark 1 Outline TSP optimization problem Tabu Search (TS) (most
More informationOutline. TABU search and Iterated Local Search classical OR methods. Traveling Salesman Problem (TSP) 2-opt
TABU search and Iterated Local Search classical OR methods Outline TSP optimization problem Tabu Search (TS) (most important) Iterated Local Search (ILS) tks@imm.dtu.dk Informatics and Mathematical Modeling
More informationIntroduction to Optimization
Introduction to Optimization Approximation Algorithms and Heuristics November 21, 2016 École Centrale Paris, Châtenay-Malabry, France Dimo Brockhoff Inria Saclay Ile-de-France 2 Exercise: The Knapsack
More informationSLS Methods: An Overview
HEURSTC OPTMZATON SLS Methods: An Overview adapted from slides for SLS:FA, Chapter 2 Outline 1. Constructive Heuristics (Revisited) 2. terative mprovement (Revisited) 3. Simple SLS Methods 4. Hybrid SLS
More informationEscaping Local Optima: Genetic Algorithm
Artificial Intelligence Escaping Local Optima: Genetic Algorithm Dae-Won Kim School of Computer Science & Engineering Chung-Ang University We re trying to escape local optima To achieve this, we have learned
More informationIntroduction to Optimization Using Metaheuristics. Thomas J. K. Stidsen
Introduction to Optimization Using Metaheuristics Thomas J. K. Stidsen Outline General course information Motivation, modelling and solving Hill climbers Simulated Annealing 1 Large-Scale Optimization
More informationIntroduction to Optimization Using Metaheuristics. The Lecturer: Thomas Stidsen. Outline. Name: Thomas Stidsen: Nationality: Danish.
The Lecturer: Thomas Stidsen Name: Thomas Stidsen: tks@imm.dtu.dk Outline Nationality: Danish. General course information Languages: Danish and English. Motivation, modelling and solving Education: Ph.D.
More informationMVE165/MMG630, Applied Optimization Lecture 8 Integer linear programming algorithms. Ann-Brith Strömberg
MVE165/MMG630, Integer linear programming algorithms Ann-Brith Strömberg 2009 04 15 Methods for ILP: Overview (Ch. 14.1) Enumeration Implicit enumeration: Branch and bound Relaxations Decomposition methods:
More informationINF Biologically inspired computing Lecture 1: Marsland chapter 9.1, Optimization and Search Jim Tørresen
INF3490 - Biologically inspired computing Lecture 1: Marsland chapter 9.1, 9.4-9.6 2017 Optimization and Search Jim Tørresen Optimization and Search 2 Optimization and Search Methods (selection) 1. Exhaustive
More informationIntroduction to Optimization
Introduction to Optimization Approximation Algorithms and Heuristics November 6, 2015 École Centrale Paris, Châtenay-Malabry, France Dimo Brockhoff INRIA Lille Nord Europe 2 Exercise: The Knapsack Problem
More informationLocal Search and Optimization Chapter 4. Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld )
Local Search and Optimization Chapter 4 Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld ) 1 2 Outline Local search techniques and optimization Hill-climbing
More informationHeuristis for Combinatorial Optimization
Heuristis for Combinatorial Optimization Luigi De Giovanni Dipartimento di Matematica, Università di Padova Luigi De Giovanni Heuristic for Combinatorial Optimization 1 / 57 Exact and heuristic methods
More informationHeuristis for Combinatorial Optimization
Heuristis for Combinatorial Optimization Luigi De Giovanni Dipartimento di Matematica, Università di Padova Luigi De Giovanni Heuristic for Combinatorial Optimization 1 / 59 Exact and heuristic methods
More informationCS:4420 Artificial Intelligence
CS:4420 Artificial Intelligence Spring 2018 Beyond Classical Search Cesare Tinelli The University of Iowa Copyright 2004 18, Cesare Tinelli and Stuart Russell a a These notes were originally developed
More informationLocal Search and Optimization Chapter 4. Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld )
Local Search and Optimization Chapter 4 Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld ) 1 2 Outline Local search techniques and optimization Hill-climbing
More informationEvolutionary Computation for Combinatorial Optimization
Evolutionary Computation for Combinatorial Optimization Günther Raidl Vienna University of Technology, Vienna, Austria raidl@ads.tuwien.ac.at EvoNet Summer School 2003, Parma, Italy August 25, 2003 Evolutionary
More informationMETAHEURISTICS. Introduction. Introduction. Nature of metaheuristics. Local improvement procedure. Example: objective function
Introduction METAHEURISTICS Some problems are so complicated that are not possible to solve for an optimal solution. In these problems, it is still important to find a good feasible solution close to the
More informationMachine Learning for Software Engineering
Machine Learning for Software Engineering Introduction and Motivation Prof. Dr.-Ing. Norbert Siegmund Intelligent Software Systems 1 2 Organizational Stuff Lectures: Tuesday 11:00 12:30 in room SR015 Cover
More informationA Course on Meta-Heuristic Search Methods for Combinatorial Optimization Problems
A Course on Meta-Heuristic Search Methods for Combinatorial Optimization Problems AutOrI LAB, DIA, Roma Tre Email: mandal@dia.uniroma3.it January 16, 2014 Outline 1 An example Assignment-I Tips Variants
More informationMethods and Models for Combinatorial Optimization Heuristis for Combinatorial Optimization
Methods and Models for Combinatorial Optimization Heuristis for Combinatorial Optimization L. De Giovanni 1 Introduction Solution methods for Combinatorial Optimization Problems (COPs) fall into two classes:
More informationCS 331: Artificial Intelligence Local Search 1. Tough real-world problems
CS 331: Artificial Intelligence Local Search 1 1 Tough real-world problems Suppose you had to solve VLSI layout problems (minimize distance between components, unused space, etc.) Or schedule airlines
More informationIntroduction to Genetic Algorithms
Advanced Topics in Image Analysis and Machine Learning Introduction to Genetic Algorithms Week 3 Faculty of Information Science and Engineering Ritsumeikan University Today s class outline Genetic Algorithms
More informationFebruary 19, Integer programming. Outline. Problem formulation. Branch-andbound
Olga Galinina olga.galinina@tut.fi ELT-53656 Network Analysis and Dimensioning II Department of Electronics and Communications Engineering Tampere University of Technology, Tampere, Finland February 19,
More informationMVE165/MMG631 Linear and integer optimization with applications Lecture 9 Discrete optimization: theory and algorithms
MVE165/MMG631 Linear and integer optimization with applications Lecture 9 Discrete optimization: theory and algorithms Ann-Brith Strömberg 2018 04 24 Lecture 9 Linear and integer optimization with applications
More informationOptimization Techniques for Design Space Exploration
0-0-7 Optimization Techniques for Design Space Exploration Zebo Peng Embedded Systems Laboratory (ESLAB) Linköping University Outline Optimization problems in ERT system design Heuristic techniques Simulated
More informationHeuristic Optimisation
Heuristic Optimisation Part 3: Classification of algorithms. Exhaustive search Sándor Zoltán Németh http://web.mat.bham.ac.uk/s.z.nemeth s.nemeth@bham.ac.uk University of Birmingham S Z Németh (s.nemeth@bham.ac.uk)
More informationLocal Search. CS 486/686: Introduction to Artificial Intelligence Winter 2016
Local Search CS 486/686: Introduction to Artificial Intelligence Winter 2016 1 Overview Uninformed Search Very general: assumes no knowledge about the problem BFS, DFS, IDS Informed Search Heuristics A*
More informationFundamentals of Integer Programming
Fundamentals of Integer Programming Di Yuan Department of Information Technology, Uppsala University January 2018 Outline Definition of integer programming Formulating some classical problems with integer
More informationMetaheuristic Development Methodology. Fall 2009 Instructor: Dr. Masoud Yaghini
Metaheuristic Development Methodology Fall 2009 Instructor: Dr. Masoud Yaghini Phases and Steps Phases and Steps Phase 1: Understanding Problem Step 1: State the Problem Step 2: Review of Existing Solution
More informationTwo approaches. Local Search TSP. Examples of algorithms using local search. Local search heuristics - To do list
Unless P=NP, there is no polynomial time algorithm for SAT, MAXSAT, MIN NODE COVER, MAX INDEPENDENT SET, MAX CLIQUE, MIN SET COVER, TSP,. But we have to solve (instances of) these problems anyway what
More informationFuzzy Inspired Hybrid Genetic Approach to Optimize Travelling Salesman Problem
Fuzzy Inspired Hybrid Genetic Approach to Optimize Travelling Salesman Problem Bindu Student, JMIT Radaur binduaahuja@gmail.com Mrs. Pinki Tanwar Asstt. Prof, CSE, JMIT Radaur pinki.tanwar@gmail.com Abstract
More informationLecture: Iterative Search Methods
Lecture: Iterative Search Methods Overview Constructive Search is exponential. State-Space Search exhibits better performance on some problems. Research in understanding heuristic and iterative search
More informationLocal Search and Optimization Chapter 4. Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld )
Local Search and Optimization Chapter 4 Mausam (Based on slides of Padhraic Smyth, Stuart Russell, Rao Kambhampati, Raj Rao, Dan Weld ) 1 Outline Local search techniques and optimization Hill-climbing
More informationLocal Search. CS 486/686: Introduction to Artificial Intelligence
Local Search CS 486/686: Introduction to Artificial Intelligence 1 Overview Uninformed Search Very general: assumes no knowledge about the problem BFS, DFS, IDS Informed Search Heuristics A* search and
More informationIntroduction to Artificial Intelligence 2 nd semester 2016/2017. Chapter 4: Beyond Classical Search
Introduction to Artificial Intelligence 2 nd semester 2016/2017 Chapter 4: Beyond Classical Search Mohamed B. Abubaker Palestine Technical College Deir El-Balah 1 Outlines local search algorithms and optimization
More informationCMU-Q Lecture 8: Optimization I: Optimization for CSP Local Search. Teacher: Gianni A. Di Caro
CMU-Q 15-381 Lecture 8: Optimization I: Optimization for CSP Local Search Teacher: Gianni A. Di Caro LOCAL SEARCH FOR CSP Real-life CSPs can be very large and hard to solve Methods so far: construct a
More informationA Steady-State Genetic Algorithm for Traveling Salesman Problem with Pickup and Delivery
A Steady-State Genetic Algorithm for Traveling Salesman Problem with Pickup and Delivery Monika Sharma 1, Deepak Sharma 2 1 Research Scholar Department of Computer Science and Engineering, NNSS SGI Samalkha,
More informationSearch Algorithms for Regression Test Suite Minimisation
School of Physical Sciences and Engineering King s College London MSc in Advanced Software Engineering Search Algorithms for Regression Test Suite Minimisation By Benjamin Cook Supervised by Prof. Mark
More informationHeuristic Optimisation Lecture Notes
Heuristic Optimisation Lecture Notes Sándor Zoltán Németh School of Mathematics, University of Birmingham Watson Building, Room 34 Email: s.nemeth@bham.ac.uk Overview Module structure Topics Assessment
More informationData Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 8: Search and Optimization Methods Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Search & Optimization Search and Optimization method deals with
More informationAlgorithms & Complexity
Algorithms & Complexity Nicolas Stroppa - nstroppa@computing.dcu.ie CA313@Dublin City University. 2006-2007. November 21, 2006 Classification of Algorithms O(1): Run time is independent of the size of
More informationGRASP. Greedy Randomized Adaptive. Search Procedure
GRASP Greedy Randomized Adaptive Search Procedure Type of problems Combinatorial optimization problem: Finite ensemble E = {1,2,... n } Subset of feasible solutions F 2 Objective function f : 2 Minimisation
More informationConstruction of Minimum-Weight Spanners Mikkel Sigurd Martin Zachariasen
Construction of Minimum-Weight Spanners Mikkel Sigurd Martin Zachariasen University of Copenhagen Outline Motivation and Background Minimum-Weight Spanner Problem Greedy Spanner Algorithm Exact Algorithm:
More informationSPATIAL OPTIMIZATION METHODS
DELMELLE E. (2010). SPATIAL OPTIMIZATION METHODS. IN: B. WHARF (ED). ENCYCLOPEDIA OF HUMAN GEOGRAPHY: 2657-2659. SPATIAL OPTIMIZATION METHODS Spatial optimization is concerned with maximizing or minimizing
More informationGenetic Algorithms Variations and Implementation Issues
Genetic Algorithms Variations and Implementation Issues CS 431 Advanced Topics in AI Classic Genetic Algorithms GAs as proposed by Holland had the following properties: Randomly generated population Binary
More informationJob Shop Scheduling Problem (JSSP) Genetic Algorithms Critical Block and DG distance Neighbourhood Search
A JOB-SHOP SCHEDULING PROBLEM (JSSP) USING GENETIC ALGORITHM (GA) Mahanim Omar, Adam Baharum, Yahya Abu Hasan School of Mathematical Sciences, Universiti Sains Malaysia 11800 Penang, Malaysia Tel: (+)
More informationn Informally: n How to form solutions n How to traverse the search space n Systematic: guarantee completeness
Advanced Search Applications: Combinatorial Optimization Scheduling Algorithms: Stochastic Local Search and others Analyses: Phase transitions, structural analysis, statistical models Combinatorial Problems
More informationOutline. Best-first search. Greedy best-first search A* search Heuristics Local search algorithms
Outline Best-first search Greedy best-first search A* search Heuristics Local search algorithms Hill-climbing search Beam search Simulated annealing search Genetic algorithms Constraint Satisfaction Problems
More informationSLS Algorithms. 2.1 Iterative Improvement (revisited)
SLS Algorithms Stochastic local search (SLS) has become a widely accepted approach to solving hard combinatorial optimisation problems. An important characteristic of many recently developed SLS methods
More informationCHAPTER 2 CONVENTIONAL AND NON-CONVENTIONAL TECHNIQUES TO SOLVE ORPD PROBLEM
20 CHAPTER 2 CONVENTIONAL AND NON-CONVENTIONAL TECHNIQUES TO SOLVE ORPD PROBLEM 2.1 CLASSIFICATION OF CONVENTIONAL TECHNIQUES Classical optimization methods can be classified into two distinct groups:
More informationBeyond Classical Search: Local Search. CMPSCI 383 September 23, 2011
Beyond Classical Search: Local Search CMPSCI 383 September 23, 2011 1 Today s lecture Local Search Hill-climbing Simulated annealing Local beam search Genetic algorithms Genetic programming Local search
More informationARTIFICIAL INTELLIGENCE. Informed search
INFOB2KI 2017-2018 Utrecht University The Netherlands ARTIFICIAL INTELLIGENCE Informed search Lecturer: Silja Renooij These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html
More informationThe Heuristic (Dark) Side of MIP Solvers. Asja Derviskadic, EPFL Vit Prochazka, NHH Christoph Schaefer, EPFL
The Heuristic (Dark) Side of MIP Solvers Asja Derviskadic, EPFL Vit Prochazka, NHH Christoph Schaefer, EPFL 1 Table of content [Lodi], The Heuristic (Dark) Side of MIP Solvers, Hybrid Metaheuristics, 273-284,
More informationGenetic Algorithms and Genetic Programming Lecture 7
Genetic Algorithms and Genetic Programming Lecture 7 Gillian Hayes 13th October 2006 Lecture 7: The Building Block Hypothesis The Building Block Hypothesis Experimental evidence for the BBH The Royal Road
More informationLecture 6: The Building Block Hypothesis. Genetic Algorithms and Genetic Programming Lecture 6. The Schema Theorem Reminder
Lecture 6: The Building Block Hypothesis 1 Genetic Algorithms and Genetic Programming Lecture 6 Gillian Hayes 9th October 2007 The Building Block Hypothesis Experimental evidence for the BBH The Royal
More informationLecture 6: Genetic Algorithm. An Introduction to Meta-Heuristics, Produced by Qiangfu Zhao (Since 2012), All rights reserved
Lecture 6: Genetic Algorithm An Introduction to Meta-Heuristics, Produced by Qiangfu Zhao (Since 2012), All rights reserved Lec06/1 Search and optimization again Given a problem, the set of all possible
More informationOutline. Informed Search. Recall: Uninformed Search. An Idea. Heuristics Informed search techniques More on heuristics Iterative improvement
Outline Informed Search ECE457 Applied Artificial Intelligence Fall 2007 Lecture #3 Heuristics Informed search techniques More on heuristics Iterative improvement Russell & Norvig, chapter 4 Skip Genetic
More informationNote: In physical process (e.g., annealing of metals), perfect ground states are achieved by very slow lowering of temperature.
Simulated Annealing Key idea: Vary temperature parameter, i.e., probability of accepting worsening moves, in Probabilistic Iterative Improvement according to annealing schedule (aka cooling schedule).
More informationAlgorithmic problem-solving: Lecture 2. Algorithmic problem-solving: Tractable vs Intractable problems. Based on Part V of the course textbook.
Algorithmic problem-solving: Lecture 2 Algorithmic problem-solving: Tractable vs Intractable problems Based on Part V of the course textbook. Algorithmic techniques Question: Given a computational task,
More informationArtificial Intelligence
Artificial Intelligence Local Search Vibhav Gogate The University of Texas at Dallas Some material courtesy of Luke Zettlemoyer, Dan Klein, Dan Weld, Alex Ihler, Stuart Russell, Mausam Systematic Search:
More informationGeometric Semantic Genetic Programming ~ Theory & Practice ~
Geometric Semantic Genetic Programming ~ Theory & Practice ~ Alberto Moraglio University of Exeter 25 April 2017 Poznan, Poland 2 Contents Evolutionary Algorithms & Genetic Programming Geometric Genetic
More informationa local optimum is encountered in such a way that further improvement steps become possible.
Dynamic Local Search I Key Idea: Modify the evaluation function whenever a local optimum is encountered in such a way that further improvement steps become possible. I Associate penalty weights (penalties)
More informationArtificial Intelligence
Artificial Intelligence Informed Search and Exploration Chapter 4 (4.3 4.6) Searching: So Far We ve discussed how to build goal-based and utility-based agents that search to solve problems We ve also presented
More informationModule 1 Lecture Notes 2. Optimization Problem and Model Formulation
Optimization Methods: Introduction and Basic concepts 1 Module 1 Lecture Notes 2 Optimization Problem and Model Formulation Introduction In the previous lecture we studied the evolution of optimization
More informationSimple mechanisms for escaping from local optima:
The methods we have seen so far are iterative improvement methods, that is, they get stuck in local optima. Simple mechanisms for escaping from local optima: I Restart: re-initialise search whenever a
More informationACO and other (meta)heuristics for CO
ACO and other (meta)heuristics for CO 32 33 Outline Notes on combinatorial optimization and algorithmic complexity Construction and modification metaheuristics: two complementary ways of searching a solution
More informationPROBLEM SOLVING AND SEARCH IN ARTIFICIAL INTELLIGENCE
Artificial Intelligence, Computational Logic PROBLEM SOLVING AND SEARCH IN ARTIFICIAL INTELLIGENCE Lecture 2 Uninformed Search vs. Informed Search Sarah Gaggl Dresden, 28th April 2015 Agenda 1 Introduction
More informationIntroduction to Stochastic Optimization Methods (meta-heuristics) Modern optimization methods 1
Introduction to Stochastic Optimization Methods (meta-heuristics) Modern optimization methods 1 Efficiency of optimization methods Robust method Efficiency Specialized method Enumeration or MC kombinatorial
More informationAlgorithms for Integer Programming
Algorithms for Integer Programming Laura Galli November 9, 2016 Unlike linear programming problems, integer programming problems are very difficult to solve. In fact, no efficient general algorithm is
More informationGENETIC ALGORITHM with Hands-On exercise
GENETIC ALGORITHM with Hands-On exercise Adopted From Lecture by Michael Negnevitsky, Electrical Engineering & Computer Science University of Tasmania 1 Objective To understand the processes ie. GAs Basic
More information15.083J Integer Programming and Combinatorial Optimization Fall Enumerative Methods
5.8J Integer Programming and Combinatorial Optimization Fall 9 A knapsack problem Enumerative Methods Let s focus on maximization integer linear programs with only binary variables For example: a knapsack
More informationInformed search algorithms. (Based on slides by Oren Etzioni, Stuart Russell)
Informed search algorithms (Based on slides by Oren Etzioni, Stuart Russell) The problem # Unique board configurations in search space 8-puzzle 9! = 362880 15-puzzle 16! = 20922789888000 10 13 24-puzzle
More informationIntroduction to Computer Science and Programming for Astronomers
Introduction to Computer Science and Programming for Astronomers Lecture 9. István Szapudi Institute for Astronomy University of Hawaii March 21, 2018 Outline Reminder 1 Reminder 2 3 Reminder We have demonstrated
More informationARTIFICIAL INTELLIGENCE
BABEŞ-BOLYAI UNIVERSITY Faculty of Computer Science and Mathematics ARTIFICIAL INTELLIGENCE Solving search problems Informed search strategies Global and local 2 Topics A. Short introduction in Artificial
More informationExploration vs. Exploitation in Differential Evolution
Exploration vs. Exploitation in Differential Evolution Ângela A. R. Sá 1, Adriano O. Andrade 1, Alcimar B. Soares 1 and Slawomir J. Nasuto 2 Abstract. Differential Evolution (DE) is a tool for efficient
More informationParallel Computing in Combinatorial Optimization
Parallel Computing in Combinatorial Optimization Bernard Gendron Université de Montréal gendron@iro.umontreal.ca Course Outline Objective: provide an overview of the current research on the design of parallel
More informationMassively Parallel Approximation Algorithms for the Traveling Salesman Problem
Massively Parallel Approximation Algorithms for the Traveling Salesman Problem Vaibhav Gandhi May 14, 2015 Abstract This paper introduces the reader to massively parallel approximation algorithms which
More informationMultiobjective Optimisation. Why? Panorama. General Formulation. Decision Space and Objective Space. 1 of 7 02/03/15 09:49.
ITNPD8/CSCU9YO Multiobjective Optimisation An Overview Nadarajen Veerapen (nve@cs.stir.ac.uk) University of Stirling Why? Classic optimisation: 1 objective Example: Minimise cost Reality is often more
More informationAlgorithm Design (4) Metaheuristics
Algorithm Design (4) Metaheuristics Takashi Chikayama School of Engineering The University of Tokyo Formalization of Constraint Optimization Minimize (or maximize) the objective function f(x 0,, x n )
More informationAn Empirical Investigation of Meta-heuristic and Heuristic Algorithms for a 2D Packing Problem
European Journal of Operational Research 128/1, 34-57, 2000. An Empirical Investigation of Meta-heuristic and Heuristic Algorithms for a 2D Packing Problem E. Hopper and B. C. H. Turton School of Engineering,
More informationDynamically Configured λ-opt Heuristics for Bus Scheduling
Dynamically Configured λ-opt Heuristics for Bus Scheduling Prapa Rattadilok and Raymond S K Kwan School of Computing, University of Leeds, UK {prapa, rsk}@comp.leeds.ac.uk Bus scheduling is a complex combinatorial
More informationTravelling salesman problem using reduced algorithmic Branch and bound approach P. Ranjana Hindustan Institute of Technology and Science
Volume 118 No. 20 2018, 419-424 ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu ijpam.eu Travelling salesman problem using reduced algorithmic Branch and bound approach P. Ranjana Hindustan
More informationTheorem 2.9: nearest addition algorithm
There are severe limits on our ability to compute near-optimal tours It is NP-complete to decide whether a given undirected =(,)has a Hamiltonian cycle An approximation algorithm for the TSP can be used
More informationA Parallel Architecture for the Generalized Traveling Salesman Problem
A Parallel Architecture for the Generalized Traveling Salesman Problem Max Scharrenbroich AMSC 663 Project Proposal Advisor: Dr. Bruce L. Golden R. H. Smith School of Business 1 Background and Introduction
More informationInformed Search Algorithms. Chapter 4
Informed Search Algorithms Chapter 4 Outline Informed Search and Heuristic Functions For informed search, we use problem-specific knowledge to guide the search. Topics: Best-first search A search Heuristics
More informationGeneral Purpose Methods for Combinatorial Optimization
General Purpose Methods for Combinatorial Optimization 0/7/00 Maximum Contiguous Sum 3-4 9 6-3 8 97-93 -3 84 Σ = 87 Given:... N Z, at least one i > 0 ind i, j such that j k k = i is maximal 0/7/00 0/7/00
More informationAn efficient evolutionary algorithm for the orienteering problem
An efficient evolutionary algorithm for the orienteering problem Gorka Kobeaga a,d,, María Merino b,d, Jose A. Lozano a,c,e a BCAM - Basque Center for Applied Mathematics, Spain. b Dep. of Applied Mathematics
More informationLecture Plan. Best-first search Greedy search A* search Designing heuristics. Hill-climbing. 1 Informed search strategies. Informed strategies
Lecture Plan 1 Informed search strategies (KA AGH) 1 czerwca 2010 1 / 28 Blind vs. informed search strategies Blind search methods You already know them: BFS, DFS, UCS et al. They don t analyse the nodes
More informationTDDC17. Intuitions behind heuristic search. Recall Uniform-Cost Search. Best-First Search. f(n) =... + h(n) g(n) = cost of path from root node to n
Intuitions behind heuristic search The separation property of GRAPH-SEARCH TDDC17 Seminar III Search II Informed or Heuristic Search Beyond Classical Search Find a heuristic measure h(n) which estimates
More informationInteger Programming. Xi Chen. Department of Management Science and Engineering International Business School Beijing Foreign Studies University
Integer Programming Xi Chen Department of Management Science and Engineering International Business School Beijing Foreign Studies University Xi Chen (chenxi0109@bfsu.edu.cn) Integer Programming 1 / 42
More informationAn evolutionary annealing-simplex algorithm for global optimisation of water resource systems
FIFTH INTERNATIONAL CONFERENCE ON HYDROINFORMATICS 1-5 July 2002, Cardiff, UK C05 - Evolutionary algorithms in hydroinformatics An evolutionary annealing-simplex algorithm for global optimisation of water
More information