Particle Swarm Optimization

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Particle Swarm Optimization"

Transcription

1 Dario Schor, M.Sc., EIT Space Systems Department Magellan Aerospace Winnipeg Winnipeg, Manitoba 1 of 34

2 Optimization Techniques Motivation Optimization: Where, min x F(x), subject to g(x) = 0 g(x) represents constraints on the parameters. F(x) knowledge: With prior knowledge, we can select appropriate techniques to find a solution quickly and efficiently. E.g., Using gradient based techniques of unimodal functions. Without prior knowledge, the parameter space is too large to evaluate all options, so heuristics are needed to find good solutions efficiently. 2 of 34

3 Motivation (PSO) Evolutionary algorithm modeled after schools of fish and flocks of birds. Iterative algorithm where particles move through a parameter space looking for a good solution. At each step, the movement is based on a weighted sum between the personal knowledge of a particle and the collective knowledge of the swarm. 3 of 34

4 Verifying Algorithms Motivation Test Functions There are many different kinds of evolutionary algorithms like genetic algorithms, simulated annealing, ant colony optimization, and PSO. What s different in each one? There are many different implementations for a single algorithm; How do you compare the different implementations? How do you pick which implementation to use? A common approach is to use a set of benchmark functions (i.e., Sphere, Rastrigin, Griewank, and Rosenbrock). 4 of 34

5 A Study of PSO Trajectories for Real-Time Scheduling Outline Motivation Original algorithm Some variations Demonstration Simulation in 2D using NetLogo Experiments Benchmark functions Concluding Remarks 5 of 34

6 Original PSO Algorithm (1 of 2) Model behavior of schools of fish and birds [KeEb95] A set of S K particles are randomly distributed within the parameter space. The particles traverse through a K-dimensional space looking for an optimal solution. At each iterations, n, the particles update their position based on their personal and social knowledge. 6 of 34

7 Original PSO Algorithm (2 of 2) Movement of particles based on [KeEb01] S v,k S x,k ( n) = S v,k ( n 1) + S ϕ1 random(0,1) S p,k S x,k ( n 1) ( ) + S ϕ 2 random(0,1) S p,g S x,k ( n) = S x,k ( n 1) + S v,k ( n) ( n 1) ( ) Update velocity (1) Personal influence (2) Social influence (3) Update position (4) where, S x,k, S v,k, S p,k S ϕ1, S ϕ 2 S p,g Position, velocity, and best position Personal, and social weights Best position in neighbourhood 7 of 34

8 Original PSO Algorithm (2 of 2) The steps are updated independently for each of the K dimensions in the problem. The step is determined based on: 1. Update velocity This serves as an inertial force that encourages particles to continue moving in the direction they were already going. 2. Personal influence Memory of the particle from previous individual best solution found. 3. Social influence Memory and knowledge from the community/swarm. 4. Update position Increments the current position along one dimension. 8 of 34

9 Original PSO Algorithm Implementation Initialization: Randomly select starting position and velocities. Usually constrained to a range based on empirical testing. 9 of 34

10 Original PSO Algorithm Implementation Main loop: Result of velocity equation. Result of position equation. 10 of 34

11 PSO Variations Population Size Intuitively: More particles = greater exploration of the parameter space. More particles = more resources to execute algorithm. Recommended sizes in literature are [20, 100] Requires testing within the specified problem space to evaluate. Directly linked to the type of topology used. 11 of 34

12 PSO Variations Neighborhood Topologies Social knowledge derived from neighborhood topology used [ScKA10] Divided into global (gbest) and local (lbest) topologies gbest can converge faster, but may get stuck in a local minimum gbest lbest ring lbest star lbest torus lbest hierarchical 12 of 34

13 PSO Variations Limiting Step Size Required to prevent the velocity from growing too quickly. The compounded step size could cause particles to diverge from the rest of the swarm. Some options: 1. Limit velocities using hard-coded values Effective, but can lead to particles oscillating around the solution. Ideally, we want the step sizes to decrease as we approach the solution. Picked based on expected range for solutions. 13 of 34

14 PSO Variations Limiting Step Size Some options continued: 2. Use an inertial weight Like a temperature gradient in simulated annealing. Decreases a weight multiplier for the velocity from 0.9 to 0.4. Equation for the decrease is usually non-linear exponential decay. Although used in late 1990 s, this is seldom used today as it introduces yet another parameter to vary in the problem. Furthermore, it is more susceptible to getting stuck in local solutions. 14 of 34

15 PSO Variations Limiting Step Size Some options continued: 3. Using a constriction coefficient Derived from the eigenvalues such that they are real and nonnegative guaranteeing convergence. Commonly used in most PSO implementations. 15 of 34

16 PSO Variations Personal and Social Weights Initially picked at random in the range 0 to 1. Intuitively it is like any person: Sometimes you have your own plans, and sometimes you go along with your friends and do something else. A multiplier was added to help particles move faster towards the solution. 16 of 34

17 PSO Variations Stopping Criterion Different options: Solution found satisfies problem constrains. All particles converge on an region. Best solution does not change for a few iterations in a row. 17 of 34

18 Demonstration NetLogo Multi-agent programmable modeling environment. Developed by Dr. Uri Wilensky from Northwestern University. Software used in courses throughout North America such as the Santa Fe Institute. Free to download from: 18 of 34

19 Demonstration NetLogo Allows you to create: GUI to test parameters in a simulation. Quick and easy to edit. Follows the 1980 s Logo principles where a Turtle moves throughout the screen. 19 of 34

20 Test Functions Benchmark Functions In 1975, De Jong proposed a set of test functions for characterizing and comparing the performance of different algorithms and their implementations. The set of functions grown since then to incorporate other important features representative of real problems. E.g., Lots of local solutions, fast/slow gradients, deep peaks, etc. These functions: Have known optimal values for comparison with calculated solution; Are non-linear and often multimodal; Do not have constraints on the variables; and The nominal trajectories of the particles will differ as they approach the solution. 20 of 34

21 Benchmark Functions Sphere Function Simplest equation in the set. All points are monotonically decreasing towards the origin. Given by equation: Smooth and often used for comparing faster approaches. It is almost like testing a worst case sorted list with a sorting algorithm to see whether your algorithm is still smart enough to handle it. 21 of 34

22 Benchmark Functions Sphere Function 22 of 34

23 Benchmark Functions Sphere Trajectories Particles oscillate towards the solution [ScKi10] [ScKi11] Two distinct behaviors: Transient Steady state 23 of 34

24 Benchmark Functions Sphere Trajectories Select particle to study Fix particle initial conditions and run algorithm 200 times Nominal particle selected to minimize mean square error Position Position Position Position trajectory for S k =7 of 200 runs on G 1 (x) function Position 20 trajectory for S k =7 of 200 runs on G 1 (x) function Zoomed in Zoomed on typical in Iteration on and trajectories worst behaviour Zoomed in on typical and worst behaviour % CI Iteration Iteration Iteration Position Trajectories Mean 95% CI Typical (run 91) Worst (run 6) Traject Mean 95% C 95% Trajectories CI Typical Mean (run 91) Worst 95% (run CI6) 24 of 34

25 Benchmark Functions Rastrigin Function Given by equation: Shows envelope function that resembles the convex shape of the sphere to guide global search. Many local solutions through the cosine function. Usually implemented in a small range (-5.12, 5.12). 25 of 34

26 Benchmark Functions Rastrigin Function 26 of 34

27 Benchmark Functions Rastrigin Trajectories Particles oscillate towards the solution [ScKi10], [ScKi11]. Two distinct behaviors: Transient Steady state 27 of 34

28 Benchmark Functions Rastrigin Trajectories Select particle to study Fix particle initial conditions and run algorithm 200 times Nominal particle selected to minimize mean square error Position trajectory for Sk=7 of 200 runs on G1(x) function 1 Traject Mean 95% C Position Position trajectory for Sk=7 of 200 runs on G1(x) function PositionPosition of Zoomed initeration on trajectories Iteration Iteration Trajectories Mean 95% CI Trajectorie Mean 95% CI

29 Benchmark Functions More Benchmarks Rosenbrock Smooth curves with mix of steep and gentle gradients. The gentle gradient makes it hard for some implementations. E.g., Using an inertia weight to decrease the step size over time. From: 29 of 34

30 Benchmark Functions More Benchmarks Ackley Lots of local minima to get trapped without a significant global gradient to guide search. Once you enter the big peak, you find the answer very quickly. From: 30 of 34

31 Benchmark Functions More Benchmarks List arranged by type: Has equation, graph, nominal range, and a description of the functions. List of common functions: Has equation, graph, nominal range, description, and MATLAB code for each function. Hedar_files/TestGO_files/Page364.htm 31 of 34

32 Benchmark Functions Picking Benchmarks Selection of benchmarks depends on: Common functions used in the literature for the algorithm you are testing. Ultimately, you want to compare your implementation/variation to the state-of-the-art. Real-problem you are trying to solve (what is really representative) These are two separate objectives: 1 st points at Comp Sci analysis of algorithms, and 2 nd points to Engineering solution. It is a balancing act for selecting parameters. 32 of 34

33 Concluding Remarks Particle Swarm is a powerful algorithm for solving unconstrained optimization problems. Can be used for a variety of applications. Benchmark functions provide a standardized comparison between implementations and variations of the algorithm. 33 of 34

34 References [KeEb01] [KeEb95] James Kennedy and Russell Eberhart, Swarm Intelligence, Morgan Kaufmann, 2001, 512pp. {ISBN } James Kennedy and Russell Eberhart, Particle swarm optimization, in Proc. of the IEEE International Conference on Neural Networks, 1995, (Perth, WA, USA; November 27-December 1, 1995), vol. 4, pp , [Scho13] Dario Schor, A Study of Trajectories for Real-Time Scheduling, MSc. Thesis. University of Manitoba, [ScKA10] [ScKi10] [ScKi11] Dario Schor, Witold Kinsner, and John Anderson, A Study of Optimal Topologies in Swarm Intelligence, in Proc. of the IEEE Canadian Conference on Electrical and Computer Engineering, CCECE 2010, (Calgary, AB; May 2{5, 2010), pp. 1 8, Dario Schor and Witold Kinsner, A Study of for Cognitive Machines, in Proc. of the 9th IEEE Intermational Conference on Cognitive Informatics, ICCI 2010, (Beijin, China; July 7 9, 2010), pp , Dario Schor and Witold Kinsner, Time and Frequency Analysis of Particle Swarm Trajectories for Cognitive Machines," International Journal of Cognitive Informatics and Natural Intelligence, vol. 5, no. 1, pp , of 34

PARTICLE SWARM OPTIMIZATION (PSO)

PARTICLE SWARM OPTIMIZATION (PSO) PARTICLE SWARM OPTIMIZATION (PSO) J. Kennedy and R. Eberhart, Particle Swarm Optimization. Proceedings of the Fourth IEEE Int. Conference on Neural Networks, 1995. A population based optimization technique

More information

Optimized Algorithm for Particle Swarm Optimization

Optimized Algorithm for Particle Swarm Optimization Optimized Algorithm for Particle Swarm Optimization Fuzhang Zhao Abstract Particle swarm optimization (PSO) is becoming one of the most important swarm intelligent paradigms for solving global optimization

More information

Particle Swarm Optimization

Particle Swarm Optimization Particle Swarm Optimization Gonçalo Pereira INESC-ID and Instituto Superior Técnico Porto Salvo, Portugal gpereira@gaips.inesc-id.pt April 15, 2011 1 What is it? Particle Swarm Optimization is an algorithm

More information

A NEW APPROACH TO SOLVE ECONOMIC LOAD DISPATCH USING PARTICLE SWARM OPTIMIZATION

A NEW APPROACH TO SOLVE ECONOMIC LOAD DISPATCH USING PARTICLE SWARM OPTIMIZATION A NEW APPROACH TO SOLVE ECONOMIC LOAD DISPATCH USING PARTICLE SWARM OPTIMIZATION Manjeet Singh 1, Divesh Thareja 2 1 Department of Electrical and Electronics Engineering, Assistant Professor, HCTM Technical

More information

GENETIC ALGORITHM VERSUS PARTICLE SWARM OPTIMIZATION IN N-QUEEN PROBLEM

GENETIC ALGORITHM VERSUS PARTICLE SWARM OPTIMIZATION IN N-QUEEN PROBLEM Journal of Al-Nahrain University Vol.10(2), December, 2007, pp.172-177 Science GENETIC ALGORITHM VERSUS PARTICLE SWARM OPTIMIZATION IN N-QUEEN PROBLEM * Azhar W. Hammad, ** Dr. Ban N. Thannoon Al-Nahrain

More information

Automatic differentiation based for particle swarm optimization steepest descent direction

Automatic differentiation based for particle swarm optimization steepest descent direction International Journal of Advances in Intelligent Informatics ISSN: 2442-6571 Vol 1, No 2, July 2015, pp. 90-97 90 Automatic differentiation based for particle swarm optimization steepest descent direction

More information

Modified Particle Swarm Optimization with Novel Modulated Inertia for Velocity Update

Modified Particle Swarm Optimization with Novel Modulated Inertia for Velocity Update Modified Particle Swarm Optimization with Novel Modulated Inertia for Velocity Update Abdul Hadi Hamdan #1, Fazida Hanim Hashim #2, Abdullah Zawawi Mohamed *3, W. M. Diyana W. Zaki #4, Aini Hussain #5

More information

SwarmOps for Matlab. Numeric & Heuristic Optimization Source-Code Library for Matlab The Manual Revision 1.0

SwarmOps for Matlab. Numeric & Heuristic Optimization Source-Code Library for Matlab The Manual Revision 1.0 Numeric & Heuristic Optimization Source-Code Library for Matlab The Manual Revision 1.0 By Magnus Erik Hvass Pedersen November 2010 Copyright 2009-2010, all rights reserved by the author. Please see page

More information

Three-Dimensional Off-Line Path Planning for Unmanned Aerial Vehicle Using Modified Particle Swarm Optimization

Three-Dimensional Off-Line Path Planning for Unmanned Aerial Vehicle Using Modified Particle Swarm Optimization Three-Dimensional Off-Line Path Planning for Unmanned Aerial Vehicle Using Modified Particle Swarm Optimization Lana Dalawr Jalal Abstract This paper addresses the problem of offline path planning for

More information

Particle Swarm Optimizer for Finding Robust Optima. J.K. Vis

Particle Swarm Optimizer for Finding Robust Optima. J.K. Vis Particle Swarm Optimizer for Finding Robust Optima J.K. Vis jvis@liacs.nl June 23, 2009 ABSTRACT Many real-world processes are subject to uncertainties and noise. Robust Optimization methods seek to obtain

More information

Particle Swarm Optimization For N-Queens Problem

Particle Swarm Optimization For N-Queens Problem Journal of Advanced Computer Science and Technology, 1 (2) (2012) 57-63 Science Publishing Corporation www.sciencepubco.com/index.php/jacst Particle Swarm Optimization For N-Queens Problem Aftab Ahmed,

More information

Assessing Particle Swarm Optimizers Using Network Science Metrics

Assessing Particle Swarm Optimizers Using Network Science Metrics Assessing Particle Swarm Optimizers Using Network Science Metrics Marcos A. C. Oliveira-Júnior, Carmelo J. A. Bastos-Filho and Ronaldo Menezes Abstract Particle Swarm Optimizers (PSOs) have been widely

More information

Benchmark Functions for the CEC 2008 Special Session and Competition on Large Scale Global Optimization

Benchmark Functions for the CEC 2008 Special Session and Competition on Large Scale Global Optimization Benchmark Functions for the CEC 2008 Special Session and Competition on Large Scale Global Optimization K. Tang 1, X. Yao 1, 2, P. N. Suganthan 3, C. MacNish 4, Y. P. Chen 5, C. M. Chen 5, Z. Yang 1 1

More information

DS-PSO: Particle Swarm Optimization with Dynamic and Static Topologies

DS-PSO: Particle Swarm Optimization with Dynamic and Static Topologies Bowdoin College Bowdoin Digital Commons Honors Projects Student Scholarship and Creative Work 5-2017 DS-PSO: Particle Swarm Optimization with Dynamic and Static Topologies Dominick Sanchez mingosanch@gmail.com

More information

Feeding the Fish Weight Update Strategies for the Fish School Search Algorithm. Andreas Janecek

Feeding the Fish Weight Update Strategies for the Fish School Search Algorithm. Andreas Janecek Feeding the Fish Weight for the Fish School Search Algorithm Andreas Janecek andreas.janecek@univie.ac.at International Conference on Swarm Intelligence (ICSI) Chongqing, China - Jun 14, 2011 Outline Basic

More information

Binary Differential Evolution Strategies

Binary Differential Evolution Strategies Binary Differential Evolution Strategies A.P. Engelbrecht, Member, IEEE G. Pampará Abstract Differential evolution has shown to be a very powerful, yet simple, population-based optimization approach. The

More information

Particle Swarm Optimization applied to Pattern Recognition

Particle Swarm Optimization applied to Pattern Recognition Particle Swarm Optimization applied to Pattern Recognition by Abel Mengistu Advisor: Dr. Raheel Ahmad CS Senior Research 2011 Manchester College May, 2011-1 - Table of Contents Introduction... - 3 - Objectives...

More information

Defining a Standard for Particle Swarm Optimization

Defining a Standard for Particle Swarm Optimization Defining a Standard for Particle Swarm Optimization Daniel Bratton Department of Computing Goldsmiths College University of London London, UK Email: dbratton@gmail.com James Kennedy US Bureau of Labor

More information

What Makes A Successful Society?

What Makes A Successful Society? What Makes A Successful Society? Experiments With Population Topologies in Particle Swarms Rui Mendes and José Neves Departamento de Informática Universidade do Minho Portugal Abstract. Previous studies

More information

CHAPTER 6 ORTHOGONAL PARTICLE SWARM OPTIMIZATION

CHAPTER 6 ORTHOGONAL PARTICLE SWARM OPTIMIZATION 131 CHAPTER 6 ORTHOGONAL PARTICLE SWARM OPTIMIZATION 6.1 INTRODUCTION The Orthogonal arrays are helpful in guiding the heuristic algorithms to obtain a good solution when applied to NP-hard problems. This

More information

Model Parameter Estimation

Model Parameter Estimation Model Parameter Estimation Shan He School for Computational Science University of Birmingham Module 06-23836: Computational Modelling with MATLAB Outline Outline of Topics Concepts about model parameter

More information

Particle Swarm Optimization in Scilab ver 0.1-7

Particle Swarm Optimization in Scilab ver 0.1-7 Particle Swarm Optimization in Scilab ver 0.1-7 S. SALMON, Research engineer and PhD. student at M3M - UTBM Abstract This document introduces the Particle Swarm Optimization (PSO) in Scilab. The PSO is

More information

SwarmOps for C# Numeric & Heuristic Optimization Source-Code Library for C# The Manual Revision 3.0

SwarmOps for C# Numeric & Heuristic Optimization Source-Code Library for C# The Manual Revision 3.0 Numeric & Heuristic Optimization Source-Code Library for C# The Manual Revision 3.0 By Magnus Erik Hvass Pedersen January 2011 Copyright 2009-2011, all rights reserved by the author. Please see page 4

More information

Cell-to-switch assignment in. cellular networks. barebones particle swarm optimization

Cell-to-switch assignment in. cellular networks. barebones particle swarm optimization Cell-to-switch assignment in cellular networks using barebones particle swarm optimization Sotirios K. Goudos a), Konstantinos B. Baltzis, Christos Bachtsevanidis, and John N. Sahalos RadioCommunications

More information

Surrogate-assisted Self-accelerated Particle Swarm Optimization

Surrogate-assisted Self-accelerated Particle Swarm Optimization Surrogate-assisted Self-accelerated Particle Swarm Optimization Kambiz Haji Hajikolaei 1, Amir Safari, G. Gary Wang ±, Hirpa G. Lemu, ± School of Mechatronic Systems Engineering, Simon Fraser University,

More information

Objective Flow-Shop Scheduling Using PSO Algorithm

Objective Flow-Shop Scheduling Using PSO Algorithm Objective Flow-Shop Scheduling Using PSO Algorithm collage of science\computer department Abstract Swarm intelligence is the study of collective behavior in decentralized and selforganized systems. Particle

More information

Algorithm for Classification

Algorithm for Classification Comparison of Hybrid PSO-SA Algorithm and Genetic Algorithm for Classification S. G. Sanjeevi 1* A. Naga Nikhila 2 Thaseem Khan 3 G. Sumathi 4 1. Associate Professor, Dept. of Comp. Science & Engg., National

More information

Overlapping Swarm Intelligence for Training Artificial Neural Networks

Overlapping Swarm Intelligence for Training Artificial Neural Networks Overlapping Swarm Intelligence for Training Artificial Neural Networks Karthik Ganesan Pillai Department of Computer Science Montana State University EPS 357, PO Box 173880 Bozeman, MT 59717-3880 k.ganesanpillai@cs.montana.edu

More information

Particle Swarm Optimization for ILP Model Based Scheduling

Particle Swarm Optimization for ILP Model Based Scheduling Particle Swarm Optimization for ILP Model Based Scheduling Shilpa KC, C LakshmiNarayana Abstract This paper focus on the optimal solution to the time constraint scheduling problem with the Integer Linear

More information

International Conference on Modeling and SimulationCoimbatore, August 2007

International Conference on Modeling and SimulationCoimbatore, August 2007 International Conference on Modeling and SimulationCoimbatore, 27-29 August 2007 OPTIMIZATION OF FLOWSHOP SCHEDULING WITH FUZZY DUE DATES USING A HYBRID EVOLUTIONARY ALGORITHM M.S.N.Kiran Kumara, B.B.Biswalb,

More information

Initializing the Particle Swarm Optimizer Using the Nonlinear Simplex Method

Initializing the Particle Swarm Optimizer Using the Nonlinear Simplex Method Initializing the Particle Swarm Optimizer Using the Nonlinear Simplex Method K.E. PARSOPOULOS, M.N. VRAHATIS Department of Mathematics University of Patras University of Patras Artificial Intelligence

More information

The Modified IWO Algorithm for Optimization of Numerical Functions

The Modified IWO Algorithm for Optimization of Numerical Functions The Modified IWO Algorithm for Optimization of Numerical Functions Daniel Kostrzewa and Henryk Josiński Silesian University of Technology, Akademicka 16 PL-44-100 Gliwice, Poland {Daniel.Kostrzewa,Henryk.Josinski}@polsl.pl

More information

Clustering of datasets using PSO-K-Means and PCA-K-means

Clustering of datasets using PSO-K-Means and PCA-K-means Clustering of datasets using PSO-K-Means and PCA-K-means Anusuya Venkatesan Manonmaniam Sundaranar University Tirunelveli- 60501, India anusuya_s@yahoo.com Latha Parthiban Computer Science Engineering

More information

Training RBF Neural Network by hybrid Adaptive Modified Particle Swarm Optimization Tabu Search algorithm for Function Approximation

Training RBF Neural Network by hybrid Adaptive Modified Particle Swarm Optimization Tabu Search algorithm for Function Approximation International Journal of Scientific & Engineering Research Volume 3, Issue 8, August-2012 1 Training RBF Neural Network by hybrid Adaptive Modified Particle Swarm Optimization Tabu Search algorithm for

More information

GRID SCHEDULING USING ENHANCED PSO ALGORITHM

GRID SCHEDULING USING ENHANCED PSO ALGORITHM GRID SCHEDULING USING ENHANCED PSO ALGORITHM Mr. P.Mathiyalagan 1 U.R.Dhepthie 2 Dr. S.N.Sivanandam 3 1 Lecturer 2 Post Graduate Student 3 Professor and Head Department of Computer Science and Engineering

More information

INTEGRATION OF INVENTORY CONTROL AND SCHEDULING USING BINARY PARTICLE SWARM OPTIMIZATION ALGORITHM

INTEGRATION OF INVENTORY CONTROL AND SCHEDULING USING BINARY PARTICLE SWARM OPTIMIZATION ALGORITHM INTEGRATION OF INVENTORY CONTROL AND SCHEDULING USING BINARY PARTICLE SWARM OPTIMIZATION ALGORITHM Manash Dey Assistant Professor, Mechanical Engineering Department, JIMS EMTC Greater Noida (India) ABSTRACT

More information

Particle swarm algorithms for multi-local optimization A. Ismael F. Vaz 1, Edite M.G.P. Fernandes 1

Particle swarm algorithms for multi-local optimization A. Ismael F. Vaz 1, Edite M.G.P. Fernandes 1 I Congresso de Estatística e Investigação Operacional da Galiza e Norte de Portugal VII Congreso Galego de Estatística e Investigación de Operacións Guimarães 26, 27 e 28 de Outubro de 2005 Particle swarm

More information

Module 1 Lecture Notes 2. Optimization Problem and Model Formulation

Module 1 Lecture Notes 2. Optimization Problem and Model Formulation Optimization Methods: Introduction and Basic concepts 1 Module 1 Lecture Notes 2 Optimization Problem and Model Formulation Introduction In the previous lecture we studied the evolution of optimization

More information

A Particle Swarm Optimization Algorithm with Path Relinking for the Location Routing Problem

A Particle Swarm Optimization Algorithm with Path Relinking for the Location Routing Problem J Math Model lgor (2008) 7:59 78 DOI 10.1007/s10852-007-9073-6 Particle Swarm Optimization lgorithm with Path Relinking for the Location Routing Problem Yannis Marinakis Magdalene Marinaki Received: 5

More information

Grid Scheduling using PSO with Naive Crossover

Grid Scheduling using PSO with Naive Crossover Grid Scheduling using PSO with Naive Crossover Vikas Singh ABV- Indian Institute of Information Technology and Management, GwaliorMorena Link Road, Gwalior, India Deepak Singh Raipur Institute of Technology

More information

Step Size Optimization of LMS Algorithm Using Particle Swarm Optimization Algorithm in System Identification

Step Size Optimization of LMS Algorithm Using Particle Swarm Optimization Algorithm in System Identification IJCSNS International Journal of Computer Science and Network Security, VOL.13 No.6, June 2013 125 Step Size Optimization of LMS Algorithm Using Particle Swarm Optimization Algorithm in System Identification

More information

Derating NichePSO. Clive Naicker

Derating NichePSO. Clive Naicker Derating NichePSO by Clive Naicker Submitted in partial fulfillment of the requirements for the degree Magister Scientiae (Computer Science) in the Faculty of Engineering, Built Environment and Information

More information

PARTICLES SWARM OPTIMIZATION FOR THE CRYPTANALYSIS OF TRANSPOSITION CIPHER

PARTICLES SWARM OPTIMIZATION FOR THE CRYPTANALYSIS OF TRANSPOSITION CIPHER Journal of Al-Nahrain University Vol13 (4), December, 2010, pp211-215 Science PARTICLES SWARM OPTIMIZATION FOR THE CRYPTANALYSIS OF TRANSPOSITION CIPHER Sarab M Hameed * and Dalal N Hmood ** * Computer

More information

Comparing Classification Performances between Neural Networks and Particle Swarm Optimization for Traffic Sign Recognition

Comparing Classification Performances between Neural Networks and Particle Swarm Optimization for Traffic Sign Recognition Comparing Classification Performances between Neural Networks and Particle Swarm Optimization for Traffic Sign Recognition THONGCHAI SURINWARANGKOON, SUPOT NITSUWAT, ELVIN J. MOORE Department of Information

More information

Paper ID: 607 ABSTRACT 1. INTRODUCTION. Particle Field Optimization: A New Paradigm for Swarm Intelligence

Paper ID: 607 ABSTRACT 1. INTRODUCTION. Particle Field Optimization: A New Paradigm for Swarm Intelligence Particle Field Optimization: A New Paradigm for Swarm Intelligence Paper ID: 607 ABSTRACT Particle Swarm Optimization (PSO) has been a popular meta-heuristic for black-box optimization for almost two decades.

More information

1 Lab + Hwk 5: Particle Swarm Optimization

1 Lab + Hwk 5: Particle Swarm Optimization 1 Lab + Hwk 5: Particle Swarm Optimization This laboratory requires the following equipment: C programming tools (gcc, make). Webots simulation software. Webots User Guide Webots Reference Manual. The

More information

A Novel Face Recognition Algorithm for Distinguishing FaceswithVariousAngles

A Novel Face Recognition Algorithm for Distinguishing FaceswithVariousAngles International Journal of Automation and Computing 05(), April 008, 193-197 DOI: 10.1007/s11633-008-0193-x A Novel Face Recognition Algorithm for Distinguishing FaceswithVariousAngles Yong-Zhong Lu School

More information

APPLICATION OF BPSO IN FLEXIBLE MANUFACTURING SYSTEM SCHEDULING

APPLICATION OF BPSO IN FLEXIBLE MANUFACTURING SYSTEM SCHEDULING International Journal of Mechanical Engineering and Technology (IJMET) Volume 8, Issue 5, May 2017, pp. 186 195, Article ID: IJMET_08_05_020 Available online at http://www.ia aeme.com/ijmet/issues.asp?jtype=ijmet&vtyp

More information

Feature Selection Algorithm with Discretization and PSO Search Methods for Continuous Attributes

Feature Selection Algorithm with Discretization and PSO Search Methods for Continuous Attributes Feature Selection Algorithm with Discretization and PSO Search Methods for Continuous Attributes Madhu.G 1, Rajinikanth.T.V 2, Govardhan.A 3 1 Dept of Information Technology, VNRVJIET, Hyderabad-90, INDIA,

More information

A Novel Particle Swarm Optimization-based Algorithm for the Optimal Centralized Wireless Access Network

A Novel Particle Swarm Optimization-based Algorithm for the Optimal Centralized Wireless Access Network wwwijcsiorg 721 A ovel Particle Swarm Optimization-based Algorithm for the Optimal Centralized Wireless Access etwork Dac-huong Le 1, and Gia-hu guyen 2 1 Faculty of Information Technology, Haiphong University

More information

Individual Parameter Selection Strategy for Particle Swarm Optimization

Individual Parameter Selection Strategy for Particle Swarm Optimization Individual Parameter Selection Strategy for Xingjuan Cai, Zhihua Cui, Jianchao Zeng and Ying Tan Division of System Simulation and Computer Application, Taiyuan University of Science and Technology P.R.China

More information

Cost Functions in Machine Learning

Cost Functions in Machine Learning Cost Functions in Machine Learning Kevin Swingler Motivation Given some data that reflects measurements from the environment We want to build a model that reflects certain statistics about that data Something

More information

PARTICLE Swarm Optimization (PSO), an algorithm by

PARTICLE Swarm Optimization (PSO), an algorithm by , March 12-14, 2014, Hong Kong Cluster-based Particle Swarm Algorithm for Solving the Mastermind Problem Dan Partynski Abstract In this paper we present a metaheuristic algorithm that is inspired by Particle

More information

Enhancing the performance of Optimized Link State Routing Protocol using HPSO and Tabu Search Algorithm

Enhancing the performance of Optimized Link State Routing Protocol using HPSO and Tabu Search Algorithm [3] Enhancing the performance of Optimized Link State Routing Protocol using HPSO and Tabu Search Algorithm S. Meenakshi Sundaram [1] Dr. S. Palani [2] Dr. A. Ramesh Babu [3] [1] Professor and Head, Department

More information

A Comparison of Several Heuristic Algorithms for Solving High Dimensional Optimization Problems

A Comparison of Several Heuristic Algorithms for Solving High Dimensional Optimization Problems A Comparison of Several Heuristic Algorithms for Solving High imensional Optimization Problems Preliminary Communication Emmanuel Karlo Nyarko J. J. Strossmayer University of Osijek, Faculty of Electrical

More information

Convolutional Code Optimization for Various Constraint Lengths using PSO

Convolutional Code Optimization for Various Constraint Lengths using PSO International Journal of Electronics and Communication Engineering. ISSN 0974-2166 Volume 5, Number 2 (2012), pp. 151-157 International Research Publication House http://www.irphouse.com Convolutional

More information

Generation of Ultra Side lobe levels in Circular Array Antennas using Evolutionary Algorithms

Generation of Ultra Side lobe levels in Circular Array Antennas using Evolutionary Algorithms Generation of Ultra Side lobe levels in Circular Array Antennas using Evolutionary Algorithms D. Prabhakar Associate Professor, Dept of ECE DVR & Dr. HS MIC College of Technology Kanchikacherla, AP, India.

More information

3 Nonlinear Regression

3 Nonlinear Regression CSC 4 / CSC D / CSC C 3 Sometimes linear models are not sufficient to capture the real-world phenomena, and thus nonlinear models are necessary. In regression, all such models will have the same basic

More information

IMPROVING THE PARTICLE SWARM OPTIMIZER BY FUNCTION STRETCHING

IMPROVING THE PARTICLE SWARM OPTIMIZER BY FUNCTION STRETCHING Chapter 3 IMPROVING THE PARTICLE SWARM OPTIMIZER BY FUNCTION STRETCHING K.E. Parsopoulos Department of Mathematics, University of Patras, GR 26110, Patras, Greece. kostasp@math.upatras.gr V.P. Plagianakos

More information

Université Libre de Bruxelles

Université Libre de Bruxelles Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Incremental Social Learning in Particle Swarms Marco A. Montes de Oca, Thomas

More information

Hybridization of particle swarm optimization with quadratic approximation

Hybridization of particle swarm optimization with quadratic approximation OPSEARCH 46(1):3 24 3 THEORY AND METHODOLOGY Hybridization of particle swarm optimization with quadratic approximation Kusum Deep, Jagdish Chand Bansal Department of Mathematics Indian Institute of Technology

More information

Witold Pedrycz. University of Alberta Edmonton, Alberta, Canada

Witold Pedrycz. University of Alberta Edmonton, Alberta, Canada 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC) Banff Center, Banff, Canada, October 5-8, 2017 Analysis of Optimization Algorithms in Automated Test Pattern Generation for Sequential

More information

Introduction to unconstrained optimization - derivative-free methods

Introduction to unconstrained optimization - derivative-free methods Introduction to unconstrained optimization - derivative-free methods Jussi Hakanen Post-doctoral researcher Office: AgC426.3 jussi.hakanen@jyu.fi Learning outcomes To understand the basic principles of

More information

A New Keystream Generator Based on Swarm Intelligence

A New Keystream Generator Based on Swarm Intelligence A New Keystream Generator Based on Swarm Intelligence Ismail K. Ali / ismailkhlil747@yahoo.com Abdulelah I. Jarullah /Abdul567@yahoo.com Receiving Date: 2011/7/24 - Accept Date: 2011/9/13 Abstract Advances

More information

Simulated Tornado Optimization

Simulated Tornado Optimization Simulated Tornado Optimization S. Hossein Hosseini, Tohid Nouri, Afshin Ebrahimi, and S. Ali Hosseini ICT Research Center, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran

More information

Job Scheduling on Computational Grids Using Fuzzy Particle Swarm Algorithm

Job Scheduling on Computational Grids Using Fuzzy Particle Swarm Algorithm Job Scheduling on Computational Grids Using Fuzzy Particle Swarm Algorithm Ajith Abraham 1,3, Hongbo Liu 2, and Weishi Zhang 3 1 School of Computer Science and Engineering, Chung-Ang University, Seoul,

More information

Simplex of Nelder & Mead Algorithm

Simplex of Nelder & Mead Algorithm Simplex of N & M Simplex of Nelder & Mead Algorithm AKA the Amoeba algorithm In the class of direct search methods Unconstrained (although constraints can be added as part of error function) nonlinear

More information

GA is the most popular population based heuristic algorithm since it was developed by Holland in 1975 [1]. This algorithm runs faster and requires les

GA is the most popular population based heuristic algorithm since it was developed by Holland in 1975 [1]. This algorithm runs faster and requires les Chaotic Crossover Operator on Genetic Algorithm Hüseyin Demirci Computer Engineering, Sakarya University, Sakarya, 54187, Turkey Ahmet Turan Özcerit Computer Engineering, Sakarya University, Sakarya, 54187,

More information

CT79 SOFT COMPUTING ALCCS-FEB 2014

CT79 SOFT COMPUTING ALCCS-FEB 2014 Q.1 a. Define Union, Intersection and complement operations of Fuzzy sets. For fuzzy sets A and B Figure Fuzzy sets A & B The union of two fuzzy sets A and B is a fuzzy set C, written as C=AUB or C=A OR

More information

An Evolutionary Algorithm for Minimizing Multimodal Functions

An Evolutionary Algorithm for Minimizing Multimodal Functions An Evolutionary Algorithm for Minimizing Multimodal Functions D.G. Sotiropoulos, V.P. Plagianakos and M.N. Vrahatis University of Patras, Department of Mamatics, Division of Computational Mamatics & Informatics,

More information

Discussion of Various Techniques for Solving Economic Load Dispatch

Discussion of Various Techniques for Solving Economic Load Dispatch International Journal of Enhanced Research in Science, Technology & Engineering ISSN: 2319-7463, Vol. 4 Issue 7, July-2015 Discussion of Various Techniques for Solving Economic Load Dispatch Veerpal Kaur

More information

MLPSO: MULTI-LEADER PARTICLE SWARM OPTIMIZATION FOR MULTI-OBJECTIVE OPTIMIZATION PROBLEMS

MLPSO: MULTI-LEADER PARTICLE SWARM OPTIMIZATION FOR MULTI-OBJECTIVE OPTIMIZATION PROBLEMS MLPSO: MULTI-LEADER PARTICLE SWARM OPTIMIZATION FOR MULTI-OBJECTIVE OPTIMIZATION PROBLEMS Zuwairie Ibrahim 1, Kian Sheng Lim 2, Salinda Buyamin 2, Siti Nurzulaikha Satiman 1, Mohd Helmi Suib 1, Badaruddin

More information

Hybrid of Ant Colony Optimization and Gravitational Emulation Based Load Balancing Strategy in Cloud Computing

Hybrid of Ant Colony Optimization and Gravitational Emulation Based Load Balancing Strategy in Cloud Computing Hybrid of Ant Colony Optimization and Gravitational Emulation Based Load Balancing Strategy in Cloud Computing Jyoti Yadav 1, Dr. Sanjay Tyagi 2 1M.Tech. Scholar, Department of Computer Science & Applications,

More information

Biologically-Inspired Optimal Video Streaming over Wireless LAN

Biologically-Inspired Optimal Video Streaming over Wireless LAN Biologically-Inspired Optimal Video Streaming over Wireless LAN Yakubu S. Baguda, Norsheila Fisal, Rozeha A. Rashid, Sharifah K. Yusof, Sharifah H. Syed, and Dahiru S. Shuaibu UTM-MIMOS Centre of Excellence,

More information

Lecture: Iterative Search Methods

Lecture: Iterative Search Methods Lecture: Iterative Search Methods Overview Constructive Search is exponential. State-Space Search exhibits better performance on some problems. Research in understanding heuristic and iterative search

More information

CURRENT RESEARCH ON EXPLORATORY LANDSCAPE ANALYSIS

CURRENT RESEARCH ON EXPLORATORY LANDSCAPE ANALYSIS CURRENT RESEARCH ON EXPLORATORY LANDSCAPE ANALYSIS HEIKE TRAUTMANN. MIKE PREUSS. 1 EXPLORATORY LANDSCAPE ANALYSIS effective and sophisticated approach to characterize properties of optimization problems

More information

Side Lobe Reduction of Phased Array Antenna using Genetic Algorithm and Particle Swarm Optimization

Side Lobe Reduction of Phased Array Antenna using Genetic Algorithm and Particle Swarm Optimization 211 Side Lobe Reduction of Phased Array Antenna using Genetic Algorithm and Particle Swarm Optimization Pampa Nandi* and Jibendu Sekhar Roy School of Electronics Engineering, KIIT University, Bhubaneswar-751024,

More information

CS 331: Artificial Intelligence Local Search 1. Tough real-world problems

CS 331: Artificial Intelligence Local Search 1. Tough real-world problems CS 331: Artificial Intelligence Local Search 1 1 Tough real-world problems Suppose you had to solve VLSI layout problems (minimize distance between components, unused space, etc.) Or schedule airlines

More information

A PARTICLE SWARM OPTIMIZATION FOR THE VEHICLE ROUTING PROBLEM

A PARTICLE SWARM OPTIMIZATION FOR THE VEHICLE ROUTING PROBLEM University of Rhode Island DigitalCommons@URI Open Access Dissertations 2014 A PARTICLE SWARM OPTIMIZATION FOR THE VEHICLE ROUTING PROBLEM Choosak Pornsing University of Rhode Island, choosak@su.ac.th

More information

THE DEVELOPMENT OF THE POTENTIAL AND ACADMIC PROGRAMMES OF WROCLAW UNIVERISTY OF TECHNOLOGY METAHEURISTICS

THE DEVELOPMENT OF THE POTENTIAL AND ACADMIC PROGRAMMES OF WROCLAW UNIVERISTY OF TECHNOLOGY METAHEURISTICS METAHEURISTICS 1. Objectives The goals of the laboratory workshop are as follows: to learn basic properties of evolutionary computation techniques and other metaheuristics for solving various global optimization

More information

Optimised Blind Video Watermarking Technique Using SD-BPSO and DWT-SVD

Optimised Blind Video Watermarking Technique Using SD-BPSO and DWT-SVD Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 4, Issue. 6, June 2015, pg.131

More information

Recent Developments in Model-based Derivative-free Optimization

Recent Developments in Model-based Derivative-free Optimization Recent Developments in Model-based Derivative-free Optimization Seppo Pulkkinen April 23, 2010 Introduction Problem definition The problem we are considering is a nonlinear optimization problem with constraints:

More information

Artificial Intelligence

Artificial Intelligence Artificial Intelligence Information Systems and Machine Learning Lab (ISMLL) Tomáš Horváth 10 rd November, 2010 Informed Search and Exploration Example (again) Informed strategy we use a problem-specific

More information

An Introduction to Evolutionary Algorithms

An Introduction to Evolutionary Algorithms An Introduction to Evolutionary Algorithms Karthik Sindhya, PhD Postdoctoral Researcher Industrial Optimization Group Department of Mathematical Information Technology Karthik.sindhya@jyu.fi http://users.jyu.fi/~kasindhy/

More information

Informed search algorithms. (Based on slides by Oren Etzioni, Stuart Russell)

Informed search algorithms. (Based on slides by Oren Etzioni, Stuart Russell) Informed search algorithms (Based on slides by Oren Etzioni, Stuart Russell) The problem # Unique board configurations in search space 8-puzzle 9! = 362880 15-puzzle 16! = 20922789888000 10 13 24-puzzle

More information

A Parallel Particle Swarm Optimizer

A Parallel Particle Swarm Optimizer A Parallel Particle Swarm Optimizer J.F. Schutte (1), B.J. Fregly (1), R.T. Haftka (1), A. D. George (2) (1) Dept. of Mechanical & Aerospace Engineering University of Florida Gainesville, FL (2) Dept.

More information

ECONOMIC LOAD DISPATCH USING PARTICLE SWARM OPTIMIZATION

ECONOMIC LOAD DISPATCH USING PARTICLE SWARM OPTIMIZATION ECONOMIC LOAD DISPATCH USING PARTICLE SWARM OPTIMIZATION Shubham Tiwari 1, Ankit Kumar 2, G.S Chaurasia 3, G.S Sirohi 4 1 Department of Electrical&Electronics Engineering Ajay Kumar Garg Engineering College,

More information

Fast Hybrid PSO and Tabu Search Approach for Optimization of a Fuzzy Controller

Fast Hybrid PSO and Tabu Search Approach for Optimization of a Fuzzy Controller IJCSI International Journal of Computer Science Issues, Vol. 8, Issue 5, No, September ISSN (Online): 694-84 www.ijcsi.org 5 Fast Hybrid PSO and Tabu Search Approach for Optimization of a Fuzzy Controller

More information

APPLICATION OF PATTERN SEARCH METHOD TO POWER SYSTEM ECONOMIC LOAD DISPATCH

APPLICATION OF PATTERN SEARCH METHOD TO POWER SYSTEM ECONOMIC LOAD DISPATCH APPLICATION OF PATTERN SEARCH METHOD TO POWER SYSTEM ECONOMIC LOAD DISPATCH J S Alsumait, J K Sykulski A K Alothman University of Southampton Electronics and Computer Sience School Electrical Power Engineering

More information

Chaos Genetic Algorithm Instead Genetic Algorithm

Chaos Genetic Algorithm Instead Genetic Algorithm The International Arab Journal of Information Technology, Vol. 12, No. 2, March 215 163 Chaos Genetic Algorithm Instead Genetic Algorithm Mohammad Javidi and Roghiyeh Hosseinpourfard Faculty of Mathematics

More information

A DISCRETE PARTICLE SWARM OPTIMIZATION ALGORITHM FOR JOB-SHOP SCHEDULING PROBLEM TO MAXIMIZING PRODUCTION. Received March 2013; revised September 2013

A DISCRETE PARTICLE SWARM OPTIMIZATION ALGORITHM FOR JOB-SHOP SCHEDULING PROBLEM TO MAXIMIZING PRODUCTION. Received March 2013; revised September 2013 International Journal of Innovative Computing, Information and Control ICIC International c 2014 ISSN 1349-4198 Volume 10, Number 2, April 2014 pp. 729 740 A DISCRETE PARTICLE SWARM OPTIMIZATION ALGORITHM

More information

Comparison between Different Meta-Heuristic Algorithms for Path Planning in Robotics

Comparison between Different Meta-Heuristic Algorithms for Path Planning in Robotics Comparison between Different Meta-Heuristic Algorithms for Path Planning in Robotics Yogita Gigras Nikita Jora Anuradha Dhull ABSTRACT Path planning has been a part of research from a decade and has been

More information

EVOLUTIONARY DISTANCES INFERRING PHYLOGENIES

EVOLUTIONARY DISTANCES INFERRING PHYLOGENIES EVOLUTIONARY DISTANCES INFERRING PHYLOGENIES Luca Bortolussi 1 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste luca@dmi.units.it Trieste, 28 th November 2007 OUTLINE 1 INFERRING

More information

Global optimization using Lévy flights

Global optimization using Lévy flights Global optimization using Lévy flights Truyen Tran, Trung Thanh Nguyen, Hoang Linh Nguyen September 2004 arxiv:1407.5739v1 [cs.ne] 22 Jul 2014 Abstract This paper studies a class of enhanced diffusion

More information

TASK SCHEDULING USING HAMMING PARTICLE SWARM OPTIMIZATION IN DISTRIBUTED SYSTEMS

TASK SCHEDULING USING HAMMING PARTICLE SWARM OPTIMIZATION IN DISTRIBUTED SYSTEMS Computing and Informatics, Vol. 36, 2017, 950 970, doi: 10.4149/cai 2017 4 950 TASK SCHEDULING USING HAMMING PARTICLE SWARM OPTIMIZATION IN DISTRIBUTED SYSTEMS Subramaniam Sarathambekai, Kandaswamy Umamaheswari

More information

Lecture 4. Convexity Robust cost functions Optimizing non-convex functions. 3B1B Optimization Michaelmas 2017 A. Zisserman

Lecture 4. Convexity Robust cost functions Optimizing non-convex functions. 3B1B Optimization Michaelmas 2017 A. Zisserman Lecture 4 3B1B Optimization Michaelmas 2017 A. Zisserman Convexity Robust cost functions Optimizing non-convex functions grid search branch and bound simulated annealing evolutionary optimization The Optimization

More information

Finite element model selection using Particle Swarm Optimization

Finite element model selection using Particle Swarm Optimization Finite element model selection using Particle Swarm Optimization Linda Mthembu 1, Tshilidzi Marwala, Michael I. Friswell 3, Sondipon Adhikari 4 1 Visiting Researcher, Department of Electronic and Computer

More information

A NEW APPROACH TO THE SOLUTION OF ECONOMIC DISPATCH USING PARTICLE SWARM OPTIMIZATION WITH SIMULATED ANNEALING

A NEW APPROACH TO THE SOLUTION OF ECONOMIC DISPATCH USING PARTICLE SWARM OPTIMIZATION WITH SIMULATED ANNEALING A NEW APPROACH TO THE SOLUTION OF ECONOMIC DISPATCH USING PARTICLE SWARM OPTIMIZATION WITH SIMULATED ANNEALING V.Karthikeyan 1 S.Senthilkumar 2 and V.J.Vijayalakshmi 3 1Department of Electronics and communication

More information

III. PV PRIORITY CONTROLLER

III. PV PRIORITY CONTROLLER Proceedings of the 27 IEEE Swarm Intelligence Symposium (SIS 27) A Fuzzy-PSO Based Controller for a Grid Independent Photovoltaic System Richard Welch, Student Member, IEEE, and Ganesh K. Venayagamoorthy,

More information

Chlor-alkali industry production scheduling algorithm research based on adaptive weight PSO

Chlor-alkali industry production scheduling algorithm research based on adaptive weight PSO Available online www.jocpr.com Journal of Chemical and Pharmaceutical Research, 2014, 6(5):34-41 Research Article ISSN : 0975-7384 CODEN(USA) : JCPRC5 Chlor-alkali industry production scheduling algorithm

More information