Hardware Neuronale Netzwerke - Lernen durch künstliche Evolution (?)

Size: px
Start display at page:

Download "Hardware Neuronale Netzwerke - Lernen durch künstliche Evolution (?)"

Transcription

1 SKIP - May 2004 Hardware Neuronale Netzwerke - Lernen durch künstliche Evolution (?) S. G. Hohmann, Electronic Vision(s), Kirchhoff Institut für Physik, Universität Heidelberg Hardware Neuronale Netzwerke Evolutionäre Algorithmen und ihre Grenzen Inkrementelle Trainingsstrategien Experimente und Resultate

2 Inspiration : Biological Neural Networks e.g. the human brain parallel distributed fault tolerant adaptive

3 Artificial Neural Networks (ANNs) a 1 neuron N net i = ω a j= 1 ji j a 2 ω 2i ω 1i a i a = F i ( neti ) ω 3i a 3 synaptic connection

4 Artificial Neural Networks (ANNs) input neurons hidden neurons output neurons necessary : find synaptic weights for desired task training : optimization of weights common : supervised learning goal : generalization

5 Artificial Neural Networks (ANNs) shortcut connections 2 hidden layers feedback connections no hidden layer

6 Hardware Realization: HAGEN Chip Photo

7 Hardware Realization: HAGEN Chip Photo

8 Hardware Realization: Network Model HAGEN chip : contains 4 network blocks network block : implements a fully connected one layer network binary inputs binary outputs 8192 analog synapses fully connected threshold activation function : O = Θ( I ), O, I {0,1} i ω ij j (McCulloch-Pitts) j network block weight values configured with 10 bit (+ sign bit) Digital to Analog Converters on chip

9 Hardware Realization: Network Operation clocked network operation (50 MHz) feedback connections recurrent network or multi-layer architecture inter block connections networks distributed among blocks binary interface interconnect multiple chips for even larger networks

10 Hardware Realization: Training Setup neural network chip HAGEN custom-made PCI board RAM programmable logic (FPGA) digital controller low-level parts of training algorithm communication via PCI bus host computer user interaction training algorithm (C++)

11 Training the HAGEN chip consider binary input/output neurons device variations common gradient-based training chip-in-the-loop algorithms well proven evolutionary algorithms: require no knowledge of neuron characteristics can exploit the speed of the hardware

12 Evolutionary Algorithms (EAs) concept process multiple networks : population of individuals simulate natural selection (based on fitness) produce offspring by recombination (crossover) + introduce variations in form of mutation C. Darwin common : use genetic coding for individuals

13 Evolutionary Algorithms (EAs) critical aspects fitness function performance measure problem-dependent optimizing fitness optimizing networks? selection scheme simulate natural selection balance selection and diversity genetic coding & operators suited for problem structure promote recombination of advantageous features

14 Genetic Coding for HAGEN Networks keeping the architecture fixed fix network cycles enable/disable feedback connections set unused synapses to 0 optimize weights of used synapses of needed input and output neurons of involved network blocks

15 Genetic Coding for HAGEN Networks number [-1023,1023] codes one weight gene linear string of genes codes used synapses of one output neuron chromosome set of chromosomes codes used synapses of each used neuron genome

16 Genetic Operators used operators two-point crossover two random cut points intermediary genes swapped between individuals new cut points for each chromosome pair + = random uniform mutation affects single genes fixed probability replaces old gene with new random value

17 Evolutionary Coprocessor hardware acceleration of evolution implemented in configurable logic (FPGA) hosts population (dedicated RAM) performs all genetic manipulations fast processing of genetic material extensive instruction set * programmed from outside (software) flexibility of genetic operators fitness & selection in software * under development

18 Selection and Reproduction Scheme example: tournament selection processing a population of Φ individuals evaluate each network investigate its fitness fill new population : tournament selection with tournament size τ old population random best new population τ = 3 form mating pairs (random) & apply crossover with probability χ mutate each gene with probability µ

19 Example : Classification Problems common classification problem : allocate each of the N p instances to one of N c classes by evaluating N i input parameters per instance examples * : predict cellular localization sites of E.coli and yeast proteins classify iris plants input values normalized [0,1]: code as binary n-bit integer values N i E.coli Yeast 7 8 N p Iris N c 8 3 * data sets obtained from the UCI KDD online archive

20 Simple Evolutionary Training Approach one output neuron for each class when applying input activate correct neuron & deactivate all others class 1 class 3 class 2 intuitive fitness function : score 1 point for each correct active neuron correct inactive neuron after each applied input good fitness function?

21 Simple Evolutionary Training Approach experiment (e.g. E.coli ): evolved networks always stay totally inactive! correct output contains more 0s than 1s! correct activation score >> correct inactivation score

22 Simple Evolutionary Training Approach experiment (e.g. E.coli ): evolved networks always stay totally inactive! correct output contains more 0s than 1s! correct activation score >> correct inactivation score new experiment (again E.coli ): networks always predict most common class! classes not equally distributed (most common : 41%)! need adequate weighting of classes e.g. all classes equally?

23 Simple Evolutionary Training Approach experiment (e.g. E.coli ): evolved networks always stay totally inactive! correct output contains more 0s than 1s! correct activation score >> correct inactivation score new experiment (again E.coli ): networks always predict most common class! classes not equally distributed (most common : 41%)! need adequate weighting of classes e.g. all classes equally? etc

24 Discussing The Evolutionary Approach contradiction? real-life applications complex networks = complex search space simple genetic operators, simple fitness measure, simple selection scheme: suited for hardware implementation (EvoCop) fast processing exploit speed of hardware successful training?

25 Discussing The Evolutionary Approach contradiction? real-life applications complex networks = complex search space simple genetic operators, simple fitness measure, simple selection scheme: suited for hardware implementation (EvoCop) fast processing exploit speed of hardware successful training?

26 Discussing The Evolutionary Approach contradiction? real-life applications complex networks = complex search space simple genetic operators, simple fitness measure, simple selection scheme: suited for hardware implementation (EvoCop) fast processing exploit speed of hardware successful training? one approach : divide-and-conquer

27 Incremental Training : Stage I one problem with N classes = N problems with one class + rest train N nets : one for each problem task : activate neurons if input belongs to respective class, remain silent otherwise train nets independently (parallel or sequentially)

28 Incremental Training : Stage I one problem with N classes = N problems with one class + rest train N nets : one for each problem task : activate neurons if input belongs to respective class, remain silent otherwise train nets independently (parallel or sequentially)

29 Incremental Training : Stage I one problem with N classes = N problems with one class + rest train N nets : one for each problem task : activate neurons if input belongs to respective class, remain silent otherwise train nets independently (parallel or sequentially)

30 Incremental Training : Stage I one problem with N classes = N problems with one class + rest train N nets : one for each problem task : activate neurons if input belongs to respective class, remain silent otherwise train nets independently (parallel or sequentially)

31 Incremental Training : Stage I Cont. all nets can use the same inputs parts of one big net response of the network : class with the highest number of activated output neurons

32 Incremental Training : Stage II add and train connections between the subnetworks again train each subnetwork one by one the task remains the same

33 Incremental Training : Stage II add and train connections between the subnetworks again train each subnetwork one by one the task remains the same

34 Incremental Training : Stage II add and train connections between the subnetworks again train each subnetwork one by one the task remains the same

35 Incremental Training : Stage II add and train connections between the subnetworks again train each subnetwork one by one the task remains the same

36 Incremental Training : Stage II add and train connections between the subnetworks again train each subnetwork one by one the task remains the same

37 Advantages of Incremental Training in each training phase smaller networks = smaller search space simple two-class problem can use simple fitness function can use simple genetic operators suited for hardware implementation (EvoCop) better performance of evolutionary algorithm

38 Experiments : Cross-Validation setup 10-fold cross-validation test : randomly divide data set into 10 subsets nine for training, one for test of generalization repeat 10 x with different validation subset for each subset : take best net of N (e.g. 10) average over the 10 configurations : classification accuracy A in % generalization accuracy G in % repeat experiment multiple (5) times (for estimation of variance)

39 Results : E.coli, Yeast and Iris train nets on HAGEN using the EvoCop 6 hidden neurons and 4 outputs per subnetwork population size 10, 2000 generations per phase tournament size 2, mutation 3%, crossover 90% E.coli Yeast Iris classification accuracy A [%] / / /- 0.1 generalization accuracy G [%] / / /- 1.3 training all at once [%] 64.7** 34.2** /- 2.5 typical G in literature [%] 86* 60* 96 results much better than if trained all at once results comparable to other systems * Horton, Nakai, Better prediction of protein cellular localization sites with the k nearest neighbors classifier, 1997 ** Results obtained by pure software algorithm without EvoCop. Appears in : Hohmann, Schemmel, Schuermann, Meier Predicting Protein Cellular Localization Sites with a Hardware Analog Neural Network, 2003

40 Iris Problem : A Trained Network upper left corner of upper left block inputs intermediate outputs 2 network cycles layer 2 layer 1

41 Iris Problem : A Trained Network upper left corner of upper left block inputs intermediate outputs 2 network cycles phase 1 : subnetwork 1

42 Iris Problem : A Trained Network upper left corner of upper left block inputs intermediate outputs 2 network cycles phase 2 : subnetwork 2

43 Iris Problem : A Trained Network upper left corner of upper left block inputs intermediate outputs 2 network cycles phase 3 : subnetwork 3

44 Iris Problem : A Trained Network upper left corner of upper left block inputs intermediate outputs 2 network cycles phase 4 : subnetwork 1

45 Iris Problem : A Trained Network upper left corner of upper left block inputs intermediate outputs 2 network cycles phase 5 : subnetwork 2

46 Iris Problem : A Trained Network upper left corner of upper left block inputs intermediate outputs 2 network cycles phase 6 : subnetwork 3

47 Iris Problem : A Trained Network upper left corner of upper left block inputs intermediate outputs 2 network cycles network complete

48 Finish : The Iris Iris Virginica The End Thanks for your attention

Neural Network Weight Selection Using Genetic Algorithms

Neural Network Weight Selection Using Genetic Algorithms Neural Network Weight Selection Using Genetic Algorithms David Montana presented by: Carl Fink, Hongyi Chen, Jack Cheng, Xinglong Li, Bruce Lin, Chongjie Zhang April 12, 2005 1 Neural Networks Neural networks

More information

Evolutionary Algorithms. CS Evolutionary Algorithms 1

Evolutionary Algorithms. CS Evolutionary Algorithms 1 Evolutionary Algorithms CS 478 - Evolutionary Algorithms 1 Evolutionary Computation/Algorithms Genetic Algorithms l Simulate natural evolution of structures via selection and reproduction, based on performance

More information

ARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 5: EVOLUTIONARY ALGORITHMS

ARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 5: EVOLUTIONARY ALGORITHMS ARTIFICIAL INTELLIGENCE (CSCU9YE ) LECTURE 5: EVOLUTIONARY ALGORITHMS Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ OUTLINE Optimisation problems Optimisation & search Two Examples The knapsack problem

More information

Preprocessing of Stream Data using Attribute Selection based on Survival of the Fittest

Preprocessing of Stream Data using Attribute Selection based on Survival of the Fittest Preprocessing of Stream Data using Attribute Selection based on Survival of the Fittest Bhakti V. Gavali 1, Prof. Vivekanand Reddy 2 1 Department of Computer Science and Engineering, Visvesvaraya Technological

More information

Lecture 6: Genetic Algorithm. An Introduction to Meta-Heuristics, Produced by Qiangfu Zhao (Since 2012), All rights reserved

Lecture 6: Genetic Algorithm. An Introduction to Meta-Heuristics, Produced by Qiangfu Zhao (Since 2012), All rights reserved Lecture 6: Genetic Algorithm An Introduction to Meta-Heuristics, Produced by Qiangfu Zhao (Since 2012), All rights reserved Lec06/1 Search and optimization again Given a problem, the set of all possible

More information

Genetic Algorithms Variations and Implementation Issues

Genetic Algorithms Variations and Implementation Issues Genetic Algorithms Variations and Implementation Issues CS 431 Advanced Topics in AI Classic Genetic Algorithms GAs as proposed by Holland had the following properties: Randomly generated population Binary

More information

Information Fusion Dr. B. K. Panigrahi

Information Fusion Dr. B. K. Panigrahi Information Fusion By Dr. B. K. Panigrahi Asst. Professor Department of Electrical Engineering IIT Delhi, New Delhi-110016 01/12/2007 1 Introduction Classification OUTLINE K-fold cross Validation Feature

More information

Outline. Motivation. Introduction of GAs. Genetic Algorithm 9/7/2017. Motivation Genetic algorithms An illustrative example Hypothesis space search

Outline. Motivation. Introduction of GAs. Genetic Algorithm 9/7/2017. Motivation Genetic algorithms An illustrative example Hypothesis space search Outline Genetic Algorithm Motivation Genetic algorithms An illustrative example Hypothesis space search Motivation Evolution is known to be a successful, robust method for adaptation within biological

More information

Chapter 5 Components for Evolution of Modular Artificial Neural Networks

Chapter 5 Components for Evolution of Modular Artificial Neural Networks Chapter 5 Components for Evolution of Modular Artificial Neural Networks 5.1 Introduction In this chapter, the methods and components used for modular evolution of Artificial Neural Networks (ANNs) are

More information

Speeding up Hardware Evolution: A Coprocessor for Evolutionary Algorithms

Speeding up Hardware Evolution: A Coprocessor for Evolutionary Algorithms Speeding up Hardware Evolution: A Coprocessor for Evolutionary Algorithms Tillmann Schmitz, Steffen Hohmann, Karlheinz Meier, Johannes Schemmel, and Felix Schürmann University of Heidelberg, Kirchhoff

More information

Biology in Computation: Evolving Intelligent Controllers

Biology in Computation: Evolving Intelligent Controllers Biology in Computation: Evolving Intelligent Controllers Combining Genetic Algorithms with Neural Networks: Implementation & Application Dimitrios N. Terzopoulos terzopod@math.auth.gr 18/1/2017 Contents

More information

Improving Performance of Multi-objective Genetic for Function Approximation through island specialisation

Improving Performance of Multi-objective Genetic for Function Approximation through island specialisation Improving Performance of Multi-objective Genetic for Function Approximation through island specialisation A. Guillén 1, I. Rojas 1, J. González 1, H. Pomares 1, L.J. Herrera 1, and B. Paechter 2 1) Department

More information

Introduction to Evolutionary Computation

Introduction to Evolutionary Computation Introduction to Evolutionary Computation The Brought to you by (insert your name) The EvoNet Training Committee Some of the Slides for this lecture were taken from the Found at: www.cs.uh.edu/~ceick/ai/ec.ppt

More information

Review: Final Exam CPSC Artificial Intelligence Michael M. Richter

Review: Final Exam CPSC Artificial Intelligence Michael M. Richter Review: Final Exam Model for a Learning Step Learner initially Environm ent Teacher Compare s pe c ia l Information Control Correct Learning criteria Feedback changed Learner after Learning Learning by

More information

Supervised Learning in Neural Networks (Part 2)

Supervised Learning in Neural Networks (Part 2) Supervised Learning in Neural Networks (Part 2) Multilayer neural networks (back-propagation training algorithm) The input signals are propagated in a forward direction on a layer-bylayer basis. Learning

More information

Suppose you have a problem You don t know how to solve it What can you do? Can you use a computer to somehow find a solution for you?

Suppose you have a problem You don t know how to solve it What can you do? Can you use a computer to somehow find a solution for you? Gurjit Randhawa Suppose you have a problem You don t know how to solve it What can you do? Can you use a computer to somehow find a solution for you? This would be nice! Can it be done? A blind generate

More information

Neuro-fuzzy, GA-Fuzzy, Neural-Fuzzy-GA: A Data Mining Technique for Optimization

Neuro-fuzzy, GA-Fuzzy, Neural-Fuzzy-GA: A Data Mining Technique for Optimization International Journal of Computer Science and Software Engineering Volume 3, Number 1 (2017), pp. 1-9 International Research Publication House http://www.irphouse.com Neuro-fuzzy, GA-Fuzzy, Neural-Fuzzy-GA:

More information

Kyrre Glette INF3490 Evolvable Hardware Cartesian Genetic Programming

Kyrre Glette INF3490 Evolvable Hardware Cartesian Genetic Programming Kyrre Glette kyrrehg@ifi INF3490 Evolvable Hardware Cartesian Genetic Programming Overview Introduction to Evolvable Hardware (EHW) Cartesian Genetic Programming Applications of EHW 3 Evolvable Hardware

More information

1. Introduction. 2. Motivation and Problem Definition. Volume 8 Issue 2, February Susmita Mohapatra

1. Introduction. 2. Motivation and Problem Definition. Volume 8 Issue 2, February Susmita Mohapatra Pattern Recall Analysis of the Hopfield Neural Network with a Genetic Algorithm Susmita Mohapatra Department of Computer Science, Utkal University, India Abstract: This paper is focused on the implementation

More information

Genetic Algorithms. Kang Zheng Karl Schober

Genetic Algorithms. Kang Zheng Karl Schober Genetic Algorithms Kang Zheng Karl Schober Genetic algorithm What is Genetic algorithm? A genetic algorithm (or GA) is a search technique used in computing to find true or approximate solutions to optimization

More information

Evolving SQL Queries for Data Mining

Evolving SQL Queries for Data Mining Evolving SQL Queries for Data Mining Majid Salim and Xin Yao School of Computer Science, The University of Birmingham Edgbaston, Birmingham B15 2TT, UK {msc30mms,x.yao}@cs.bham.ac.uk Abstract. This paper

More information

The k-means Algorithm and Genetic Algorithm

The k-means Algorithm and Genetic Algorithm The k-means Algorithm and Genetic Algorithm k-means algorithm Genetic algorithm Rough set approach Fuzzy set approaches Chapter 8 2 The K-Means Algorithm The K-Means algorithm is a simple yet effective

More information

CS5401 FS2015 Exam 1 Key

CS5401 FS2015 Exam 1 Key CS5401 FS2015 Exam 1 Key This is a closed-book, closed-notes exam. The only items you are allowed to use are writing implements. Mark each sheet of paper you use with your name and the string cs5401fs2015

More information

11/14/2010 Intelligent Systems and Soft Computing 1

11/14/2010 Intelligent Systems and Soft Computing 1 Lecture 7 Artificial neural networks: Supervised learning Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in

More information

Chapter 9: Genetic Algorithms

Chapter 9: Genetic Algorithms Computational Intelligence: Second Edition Contents Compact Overview First proposed by Fraser in 1957 Later by Bremermann in 1962 and Reed et al in 1967 Popularized by Holland in 1975 Genetic algorithms

More information

CHAPTER 6 HYBRID AI BASED IMAGE CLASSIFICATION TECHNIQUES

CHAPTER 6 HYBRID AI BASED IMAGE CLASSIFICATION TECHNIQUES CHAPTER 6 HYBRID AI BASED IMAGE CLASSIFICATION TECHNIQUES 6.1 INTRODUCTION The exploration of applications of ANN for image classification has yielded satisfactory results. But, the scope for improving

More information

Genetic Programming for Multiclass Object Classification

Genetic Programming for Multiclass Object Classification Genetic Programming for Multiclass Object Classification by William Richmond Smart A thesis submitted to the Victoria University of Wellington in fulfilment of the requirements for the degree of Master

More information

Data Mining. Neural Networks

Data Mining. Neural Networks Data Mining Neural Networks Goals for this Unit Basic understanding of Neural Networks and how they work Ability to use Neural Networks to solve real problems Understand when neural networks may be most

More information

Predicting Diabetes using Neural Networks and Randomized Optimization

Predicting Diabetes using Neural Networks and Randomized Optimization Predicting Diabetes using Neural Networks and Randomized Optimization Kunal Sharma GTID: ksharma74 CS 4641 Machine Learning Abstract This paper analysis the following randomized optimization techniques

More information

Applied Cloning Techniques for a Genetic Algorithm Used in Evolvable Hardware Design

Applied Cloning Techniques for a Genetic Algorithm Used in Evolvable Hardware Design Applied Cloning Techniques for a Genetic Algorithm Used in Evolvable Hardware Design Viet C. Trinh vtrinh@isl.ucf.edu Gregory A. Holifield greg.holifield@us.army.mil School of Electrical Engineering and

More information

A GENETIC ALGORITHM FOR CLUSTERING ON VERY LARGE DATA SETS

A GENETIC ALGORITHM FOR CLUSTERING ON VERY LARGE DATA SETS A GENETIC ALGORITHM FOR CLUSTERING ON VERY LARGE DATA SETS Jim Gasvoda and Qin Ding Department of Computer Science, Pennsylvania State University at Harrisburg, Middletown, PA 17057, USA {jmg289, qding}@psu.edu

More information

Monika Maharishi Dayanand University Rohtak

Monika Maharishi Dayanand University Rohtak Performance enhancement for Text Data Mining using k means clustering based genetic optimization (KMGO) Monika Maharishi Dayanand University Rohtak ABSTRACT For discovering hidden patterns and structures

More information

GENETIC ALGORITHM with Hands-On exercise

GENETIC ALGORITHM with Hands-On exercise GENETIC ALGORITHM with Hands-On exercise Adopted From Lecture by Michael Negnevitsky, Electrical Engineering & Computer Science University of Tasmania 1 Objective To understand the processes ie. GAs Basic

More information

DETERMINING MAXIMUM/MINIMUM VALUES FOR TWO- DIMENTIONAL MATHMATICLE FUNCTIONS USING RANDOM CREOSSOVER TECHNIQUES

DETERMINING MAXIMUM/MINIMUM VALUES FOR TWO- DIMENTIONAL MATHMATICLE FUNCTIONS USING RANDOM CREOSSOVER TECHNIQUES DETERMINING MAXIMUM/MINIMUM VALUES FOR TWO- DIMENTIONAL MATHMATICLE FUNCTIONS USING RANDOM CREOSSOVER TECHNIQUES SHIHADEH ALQRAINY. Department of Software Engineering, Albalqa Applied University. E-mail:

More information

Genetic Algorithms for Vision and Pattern Recognition

Genetic Algorithms for Vision and Pattern Recognition Genetic Algorithms for Vision and Pattern Recognition Faiz Ul Wahab 11/8/2014 1 Objective To solve for optimization of computer vision problems using genetic algorithms 11/8/2014 2 Timeline Problem: Computer

More information

Probability Control Functions Settings in Continual Evolution Algorithm

Probability Control Functions Settings in Continual Evolution Algorithm Probability Control Functions Settings in Continual Evolution Algorithm Zdeněk Buk, Miroslav Šnorek Dept. of Computer Science and Engineering, Karlovo nám. 3, 2 35 Praha 2, Czech Republic bukz@fel.cvut.cz,

More information

Neural Optimization of Evolutionary Algorithm Parameters. Hiral Patel

Neural Optimization of Evolutionary Algorithm Parameters. Hiral Patel Neural Optimization of Evolutionary Algorithm Parameters Hiral Patel December 6, 23 Abstract This paper presents a novel idea of using an unsupervised neural network to optimize the on-line parameters

More information

Introduction to Optimization

Introduction to Optimization Introduction to Optimization Approximation Algorithms and Heuristics November 6, 2015 École Centrale Paris, Châtenay-Malabry, France Dimo Brockhoff INRIA Lille Nord Europe 2 Exercise: The Knapsack Problem

More information

Introduction to Optimization

Introduction to Optimization Introduction to Optimization Approximation Algorithms and Heuristics November 21, 2016 École Centrale Paris, Châtenay-Malabry, France Dimo Brockhoff Inria Saclay Ile-de-France 2 Exercise: The Knapsack

More information

Computer Systems Colloquium (EE380) Wednesday, 4:15-5:30PM 5:30PM in Gates B01

Computer Systems Colloquium (EE380) Wednesday, 4:15-5:30PM 5:30PM in Gates B01 Adapting Systems by Evolving Hardware Computer Systems Colloquium (EE380) Wednesday, 4:15-5:30PM 5:30PM in Gates B01 Jim Torresen Group Department of Informatics University of Oslo, Norway E-mail: jimtoer@ifi.uio.no

More information

Genetic Algorithms and the Evolution of Neural Networks for Language Processing

Genetic Algorithms and the Evolution of Neural Networks for Language Processing Genetic Algorithms and the Evolution of Neural Networks for Language Processing Jaime J. Dávila Hampshire College, School of Cognitive Science Amherst, MA 01002 jdavila@hampshire.edu Abstract One approach

More information

Genetic Programming. and its use for learning Concepts in Description Logics

Genetic Programming. and its use for learning Concepts in Description Logics Concepts in Description Artificial Intelligence Institute Computer Science Department Dresden Technical University May 29, 2006 Outline Outline: brief introduction to explanation of the workings of a algorithm

More information

Improving interpretability in approximative fuzzy models via multi-objective evolutionary algorithms.

Improving interpretability in approximative fuzzy models via multi-objective evolutionary algorithms. Improving interpretability in approximative fuzzy models via multi-objective evolutionary algorithms. Gómez-Skarmeta, A.F. University of Murcia skarmeta@dif.um.es Jiménez, F. University of Murcia fernan@dif.um.es

More information

Using a genetic algorithm for editing k-nearest neighbor classifiers

Using a genetic algorithm for editing k-nearest neighbor classifiers Using a genetic algorithm for editing k-nearest neighbor classifiers R. Gil-Pita 1 and X. Yao 23 1 Teoría de la Señal y Comunicaciones, Universidad de Alcalá, Madrid (SPAIN) 2 Computer Sciences Department,

More information

Genetic Algorithm for Finding Shortest Path in a Network

Genetic Algorithm for Finding Shortest Path in a Network Intern. J. Fuzzy Mathematical Archive Vol. 2, 2013, 43-48 ISSN: 2320 3242 (P), 2320 3250 (online) Published on 26 August 2013 www.researchmathsci.org International Journal of Genetic Algorithm for Finding

More information

A Parallel Evolutionary Algorithm for Discovery of Decision Rules

A Parallel Evolutionary Algorithm for Discovery of Decision Rules A Parallel Evolutionary Algorithm for Discovery of Decision Rules Wojciech Kwedlo Faculty of Computer Science Technical University of Bia lystok Wiejska 45a, 15-351 Bia lystok, Poland wkwedlo@ii.pb.bialystok.pl

More information

Mutations for Permutations

Mutations for Permutations Mutations for Permutations Insert mutation: Pick two allele values at random Move the second to follow the first, shifting the rest along to accommodate Note: this preserves most of the order and adjacency

More information

GENETIC ALGORITHM VERSUS PARTICLE SWARM OPTIMIZATION IN N-QUEEN PROBLEM

GENETIC ALGORITHM VERSUS PARTICLE SWARM OPTIMIZATION IN N-QUEEN PROBLEM Journal of Al-Nahrain University Vol.10(2), December, 2007, pp.172-177 Science GENETIC ALGORITHM VERSUS PARTICLE SWARM OPTIMIZATION IN N-QUEEN PROBLEM * Azhar W. Hammad, ** Dr. Ban N. Thannoon Al-Nahrain

More information

CHAPTER 5 ENERGY MANAGEMENT USING FUZZY GENETIC APPROACH IN WSN

CHAPTER 5 ENERGY MANAGEMENT USING FUZZY GENETIC APPROACH IN WSN 97 CHAPTER 5 ENERGY MANAGEMENT USING FUZZY GENETIC APPROACH IN WSN 5.1 INTRODUCTION Fuzzy systems have been applied to the area of routing in ad hoc networks, aiming to obtain more adaptive and flexible

More information

Random Search Report An objective look at random search performance for 4 problem sets

Random Search Report An objective look at random search performance for 4 problem sets Random Search Report An objective look at random search performance for 4 problem sets Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA dwai3@gatech.edu Abstract: This report

More information

Multi-label classification using rule-based classifier systems

Multi-label classification using rule-based classifier systems Multi-label classification using rule-based classifier systems Shabnam Nazmi (PhD candidate) Department of electrical and computer engineering North Carolina A&T state university Advisor: Dr. A. Homaifar

More information

Artificial Intelligence Application (Genetic Algorithm)

Artificial Intelligence Application (Genetic Algorithm) Babylon University College of Information Technology Software Department Artificial Intelligence Application (Genetic Algorithm) By Dr. Asaad Sabah Hadi 2014-2015 EVOLUTIONARY ALGORITHM The main idea about

More information

MODELLING DOCUMENT CATEGORIES BY EVOLUTIONARY LEARNING OF TEXT CENTROIDS

MODELLING DOCUMENT CATEGORIES BY EVOLUTIONARY LEARNING OF TEXT CENTROIDS MODELLING DOCUMENT CATEGORIES BY EVOLUTIONARY LEARNING OF TEXT CENTROIDS J.I. Serrano M.D. Del Castillo Instituto de Automática Industrial CSIC. Ctra. Campo Real km.0 200. La Poveda. Arganda del Rey. 28500

More information

Neural Networks CMSC475/675

Neural Networks CMSC475/675 Introduction to Neural Networks CMSC475/675 Chapter 1 Introduction Why ANN Introduction Some tasks can be done easily (effortlessly) by humans but are hard by conventional paradigms on Von Neumann machine

More information

CHAPTER 4 FEATURE SELECTION USING GENETIC ALGORITHM

CHAPTER 4 FEATURE SELECTION USING GENETIC ALGORITHM CHAPTER 4 FEATURE SELECTION USING GENETIC ALGORITHM In this research work, Genetic Algorithm method is used for feature selection. The following section explains how Genetic Algorithm is used for feature

More information

A genetic algorithms approach to optimization parameter space of Geant-V prototype

A genetic algorithms approach to optimization parameter space of Geant-V prototype A genetic algorithms approach to optimization parameter space of Geant-V prototype Oksana Shadura CERN, PH-SFT & National Technical Univ. of Ukraine Kyiv Polytechnic Institute Geant-V parameter space [1/2]

More information

Evolutionary Computation. Chao Lan

Evolutionary Computation. Chao Lan Evolutionary Computation Chao Lan Outline Introduction Genetic Algorithm Evolutionary Strategy Genetic Programming Introduction Evolutionary strategy can jointly optimize multiple variables. - e.g., max

More information

Introduction to Genetic Algorithms. Based on Chapter 10 of Marsland Chapter 9 of Mitchell

Introduction to Genetic Algorithms. Based on Chapter 10 of Marsland Chapter 9 of Mitchell Introduction to Genetic Algorithms Based on Chapter 10 of Marsland Chapter 9 of Mitchell Genetic Algorithms - History Pioneered by John Holland in the 1970s Became popular in the late 1980s Based on ideas

More information

Heuristic Optimisation

Heuristic Optimisation Heuristic Optimisation Part 10: Genetic Algorithm Basics Sándor Zoltán Németh http://web.mat.bham.ac.uk/s.z.nemeth s.nemeth@bham.ac.uk University of Birmingham S Z Németh (s.nemeth@bham.ac.uk) Heuristic

More information

International Journal of Scientific & Engineering Research, Volume 8, Issue 1, January ISSN

International Journal of Scientific & Engineering Research, Volume 8, Issue 1, January ISSN International Journal of Scientific & Engineering Research, Volume 8, Issue 1, January-2017 550 Using Neuro Fuzzy and Genetic Algorithm for Image Denoising Shaymaa Rashid Saleh Raidah S. Khaudeyer Abstract

More information

Advanced Search Genetic algorithm

Advanced Search Genetic algorithm Advanced Search Genetic algorithm Yingyu Liang yliang@cs.wisc.edu Computer Sciences Department University of Wisconsin, Madison [Based on slides from Jerry Zhu, Andrew Moore http://www.cs.cmu.edu/~awm/tutorials

More information

Argha Roy* Dept. of CSE Netaji Subhash Engg. College West Bengal, India.

Argha Roy* Dept. of CSE Netaji Subhash Engg. College West Bengal, India. Volume 3, Issue 3, March 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Training Artificial

More information

METAHEURISTICS Genetic Algorithm

METAHEURISTICS Genetic Algorithm METAHEURISTICS Genetic Algorithm Jacques A. Ferland Department of Informatique and Recherche Opérationnelle Université de Montréal ferland@iro.umontreal.ca Genetic Algorithm (GA) Population based algorithm

More information

Escaping Local Optima: Genetic Algorithm

Escaping Local Optima: Genetic Algorithm Artificial Intelligence Escaping Local Optima: Genetic Algorithm Dae-Won Kim School of Computer Science & Engineering Chung-Ang University We re trying to escape local optima To achieve this, we have learned

More information

Computational Intelligence

Computational Intelligence Computational Intelligence Module 6 Evolutionary Computation Ajith Abraham Ph.D. Q What is the most powerful problem solver in the Universe? ΑThe (human) brain that created the wheel, New York, wars and

More information

Genetic Fourier Descriptor for the Detection of Rotational Symmetry

Genetic Fourier Descriptor for the Detection of Rotational Symmetry 1 Genetic Fourier Descriptor for the Detection of Rotational Symmetry Raymond K. K. Yip Department of Information and Applied Technology, Hong Kong Institute of Education 10 Lo Ping Road, Tai Po, New Territories,

More information

Introduction to Genetic Algorithms

Introduction to Genetic Algorithms Advanced Topics in Image Analysis and Machine Learning Introduction to Genetic Algorithms Week 3 Faculty of Information Science and Engineering Ritsumeikan University Today s class outline Genetic Algorithms

More information

Machine Learning in Biology

Machine Learning in Biology Università degli studi di Padova Machine Learning in Biology Luca Silvestrin (Dottorando, XXIII ciclo) Supervised learning Contents Class-conditional probability density Linear and quadratic discriminant

More information

Evolution Evolves with Autoconstruction

Evolution Evolves with Autoconstruction Evolution Evolves with Autoconstruction 6th Workshop on Evolutionary Computation for the Automated Design of Algorithms Genetic and Evolutionary Computation Conference (GECCO) Denver, Colorado, USA, July,

More information

Evolutionary Optimization of Neural Networks for Face Detection

Evolutionary Optimization of Neural Networks for Face Detection Evolutionary Optimization of Neural Networks for Face Detection Stefan Wiegand Christian Igel Uwe Handmann Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany Viisage Technology

More information

Evolutionary origins of modularity

Evolutionary origins of modularity Evolutionary origins of modularity Jeff Clune, Jean-Baptiste Mouret and Hod Lipson Proceedings of the Royal Society B 2013 Presented by Raghav Partha Evolvability Evolvability capacity to rapidly adapt

More information

COMPUTER SIMULATION OF COMPLEX SYSTEMS USING AUTOMATA NETWORKS K. Ming Leung

COMPUTER SIMULATION OF COMPLEX SYSTEMS USING AUTOMATA NETWORKS K. Ming Leung POLYTECHNIC UNIVERSITY Department of Computer and Information Science COMPUTER SIMULATION OF COMPLEX SYSTEMS USING AUTOMATA NETWORKS K. Ming Leung Abstract: Computer simulation of the dynamics of complex

More information

Optimization of Association Rule Mining through Genetic Algorithm

Optimization of Association Rule Mining through Genetic Algorithm Optimization of Association Rule Mining through Genetic Algorithm RUPALI HALDULAKAR School of Information Technology, Rajiv Gandhi Proudyogiki Vishwavidyalaya Bhopal, Madhya Pradesh India Prof. JITENDRA

More information

Multilayer Feed-forward networks

Multilayer Feed-forward networks Multi Feed-forward networks 1. Computational models of McCulloch and Pitts proposed a binary threshold unit as a computational model for artificial neuron. This first type of neuron has been generalized

More information

A Genetic Algorithm for Graph Matching using Graph Node Characteristics 1 2

A Genetic Algorithm for Graph Matching using Graph Node Characteristics 1 2 Chapter 5 A Genetic Algorithm for Graph Matching using Graph Node Characteristics 1 2 Graph Matching has attracted the exploration of applying new computing paradigms because of the large number of applications

More information

Preliminary Background Tabu Search Genetic Algorithm

Preliminary Background Tabu Search Genetic Algorithm Preliminary Background Tabu Search Genetic Algorithm Faculty of Information Technology University of Science Vietnam National University of Ho Chi Minh City March 2010 Problem used to illustrate General

More information

Combinational Circuit Design Using Genetic Algorithms

Combinational Circuit Design Using Genetic Algorithms Combinational Circuit Design Using Genetic Algorithms Nithyananthan K Bannari Amman institute of technology M.E.Embedded systems, Anna University E-mail:nithyananthan.babu@gmail.com Abstract - In the paper

More information

Adaptive Crossover in Genetic Algorithms Using Statistics Mechanism

Adaptive Crossover in Genetic Algorithms Using Statistics Mechanism in Artificial Life VIII, Standish, Abbass, Bedau (eds)(mit Press) 2002. pp 182 185 1 Adaptive Crossover in Genetic Algorithms Using Statistics Mechanism Shengxiang Yang Department of Mathematics and Computer

More information

COMPUTATIONAL INTELLIGENCE

COMPUTATIONAL INTELLIGENCE COMPUTATIONAL INTELLIGENCE Fundamentals Adrian Horzyk Preface Before we can proceed to discuss specific complex methods we have to introduce basic concepts, principles, and models of computational intelligence

More information

CHAPTER 5. CHE BASED SoPC FOR EVOLVABLE HARDWARE

CHAPTER 5. CHE BASED SoPC FOR EVOLVABLE HARDWARE 90 CHAPTER 5 CHE BASED SoPC FOR EVOLVABLE HARDWARE A hardware architecture that implements the GA for EHW is presented in this chapter. This SoPC (System on Programmable Chip) architecture is also designed

More information

A Steady-State Genetic Algorithm for Traveling Salesman Problem with Pickup and Delivery

A Steady-State Genetic Algorithm for Traveling Salesman Problem with Pickup and Delivery A Steady-State Genetic Algorithm for Traveling Salesman Problem with Pickup and Delivery Monika Sharma 1, Deepak Sharma 2 1 Research Scholar Department of Computer Science and Engineering, NNSS SGI Samalkha,

More information

A Memetic Genetic Program for Knowledge Discovery

A Memetic Genetic Program for Knowledge Discovery A Memetic Genetic Program for Knowledge Discovery by Gert Nel Submitted in partial fulfilment of the requirements for the degree Master of Science in the Faculty of Engineering, Built Environment and Information

More information

The Genetic Algorithm for finding the maxima of single-variable functions

The Genetic Algorithm for finding the maxima of single-variable functions Research Inventy: International Journal Of Engineering And Science Vol.4, Issue 3(March 2014), PP 46-54 Issn (e): 2278-4721, Issn (p):2319-6483, www.researchinventy.com The Genetic Algorithm for finding

More information

Genetic Algorithm Based Template Optimization for a Vision System: Obstacle Detection

Genetic Algorithm Based Template Optimization for a Vision System: Obstacle Detection ISTET'09 Umair Ali Khan, Alireza Fasih, Kyandoghere Kyamakya, Jean Chamberlain Chedjou Transportation Informatics Group, Alpen Adria University, Klagenfurt, Austria. Genetic Algorithm Based Template Optimization

More information

Comparison Study of Multiple Traveling Salesmen Problem using Genetic Algorithm

Comparison Study of Multiple Traveling Salesmen Problem using Genetic Algorithm IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-661, p- ISSN: 2278-8727Volume 13, Issue 3 (Jul. - Aug. 213), PP 17-22 Comparison Study of Multiple Traveling Salesmen Problem using Genetic

More information

Research on time optimal trajectory planning of 7-DOF manipulator based on genetic algorithm

Research on time optimal trajectory planning of 7-DOF manipulator based on genetic algorithm Acta Technica 61, No. 4A/2016, 189 200 c 2017 Institute of Thermomechanics CAS, v.v.i. Research on time optimal trajectory planning of 7-DOF manipulator based on genetic algorithm Jianrong Bu 1, Junyan

More information

Introduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7.

Introduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7. Chapter 7: Derivative-Free Optimization Introduction (7.1) Genetic Algorithms (GA) (7.2) Simulated Annealing (SA) (7.3) Random Search (7.4) Downhill Simplex Search (DSS) (7.5) Jyh-Shing Roger Jang et al.,

More information

The Binary Genetic Algorithm. Universidad de los Andes-CODENSA

The Binary Genetic Algorithm. Universidad de los Andes-CODENSA The Binary Genetic Algorithm Universidad de los Andes-CODENSA 1. Genetic Algorithms: Natural Selection on a Computer Figure 1 shows the analogy between biological i l evolution and a binary GA. Both start

More information

METAHEURISTIC. Jacques A. Ferland Department of Informatique and Recherche Opérationnelle Université de Montréal.

METAHEURISTIC. Jacques A. Ferland Department of Informatique and Recherche Opérationnelle Université de Montréal. METAHEURISTIC Jacques A. Ferland Department of Informatique and Recherche Opérationnelle Université de Montréal ferland@iro.umontreal.ca March 2015 Overview Heuristic Constructive Techniques: Generate

More information

Toward Self-adaptive Embedded Systems: Multi-objective Hardware Evolution

Toward Self-adaptive Embedded Systems: Multi-objective Hardware Evolution Toward Self-adaptive Embedded Systems: Multi-objective Hardware Evolution Paul Kaufmann and Marco Platzner University of Paderborn Abstract. Evolutionary hardware design reveals the potential to provide

More information

Opening the Black Box Data Driven Visualizaion of Neural N

Opening the Black Box Data Driven Visualizaion of Neural N Opening the Black Box Data Driven Visualizaion of Neural Networks September 20, 2006 Aritificial Neural Networks Limitations of ANNs Use of Visualization (ANNs) mimic the processes found in biological

More information

Genetic Placement: Genie Algorithm Way Sern Shong ECE556 Final Project Fall 2004

Genetic Placement: Genie Algorithm Way Sern Shong ECE556 Final Project Fall 2004 Genetic Placement: Genie Algorithm Way Sern Shong ECE556 Final Project Fall 2004 Introduction Overview One of the principle problems in VLSI chip design is the layout problem. The layout problem is complex

More information

Machine Evolution. Machine Evolution. Let s look at. Machine Evolution. Machine Evolution. Machine Evolution. Machine Evolution

Machine Evolution. Machine Evolution. Let s look at. Machine Evolution. Machine Evolution. Machine Evolution. Machine Evolution Let s look at As you will see later in this course, neural networks can learn, that is, adapt to given constraints. For example, NNs can approximate a given function. In biology, such learning corresponds

More information

Search Algorithms for Regression Test Suite Minimisation

Search Algorithms for Regression Test Suite Minimisation School of Physical Sciences and Engineering King s College London MSc in Advanced Software Engineering Search Algorithms for Regression Test Suite Minimisation By Benjamin Cook Supervised by Prof. Mark

More information

A FINGER PRINT RECOGNISER USING FUZZY EVOLUTIONARY PROGRAMMING

A FINGER PRINT RECOGNISER USING FUZZY EVOLUTIONARY PROGRAMMING A FINGER PRINT RECOGNISER USING FUZZY EVOLUTIONARY PROGRAMMING Author1: Author2: K.Raghu Ram K.Krishna Chaitanya 4 th E.C.E 4 th E.C.E raghuram.kolipaka@gmail.com chaitu_kolluri@yahoo.com Newton s Institute

More information

A Web-Based Evolutionary Algorithm Demonstration using the Traveling Salesman Problem

A Web-Based Evolutionary Algorithm Demonstration using the Traveling Salesman Problem A Web-Based Evolutionary Algorithm Demonstration using the Traveling Salesman Problem Richard E. Mowe Department of Statistics St. Cloud State University mowe@stcloudstate.edu Bryant A. Julstrom Department

More information

Evolutionary Computation Part 2

Evolutionary Computation Part 2 Evolutionary Computation Part 2 CS454, Autumn 2017 Shin Yoo (with some slides borrowed from Seongmin Lee @ COINSE) Crossover Operators Offsprings inherit genes from their parents, but not in identical

More information

Particle swarm optimization of artificial neural networks for autonomous robots. Mahmood Rahmani

Particle swarm optimization of artificial neural networks for autonomous robots. Mahmood Rahmani Thesis for the degree of Master of Science in Complex Adaptive Systems Particle swarm optimization of artificial neural networks for autonomous robots Mahmood Rahmani Department of Applied Physics Chalmers

More information

MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS

MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS In: Journal of Applied Statistical Science Volume 18, Number 3, pp. 1 7 ISSN: 1067-5817 c 2011 Nova Science Publishers, Inc. MAXIMUM LIKELIHOOD ESTIMATION USING ACCELERATED GENETIC ALGORITHMS Füsun Akman

More information

Design Space Exploration Using Parameterized Cores

Design Space Exploration Using Parameterized Cores RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS UNIVERSITY OF WINDSOR Design Space Exploration Using Parameterized Cores Ian D. L. Anderson M.A.Sc. Candidate March 31, 2006 Supervisor: Dr. M. Khalid 1 OUTLINE

More information