Yuki Osada Andrew Cannon


 Reginald Scott
 1 years ago
 Views:
Transcription
1 Yuki Osada Andrew Cannon 1
2 Humans are an intelligent species One feature is the ability to learn The ability to learn comes down to the brain The brain learns from experience Research shows that the brain stores information as patterns This information is stored in neurons 2
3 Neurons do not regenerate suggesting that these cells are what provide us with the abilities to: remember, think, and apply previous experiences Humans generally have between 80 and 120 million neurons Each neuron typically connect with 1000 to other neurons The human brain is a huge network of neurons  a neural network 3
4 The power of the human mind comes from the sheer number of these neurons, and its connections The individual neurons act as a function of their incoming signals Although neurons themselves are complicated, they don't exhibit complex behaviour on their own This is the key feature that makes it a viable computational intelligence approach 4
5 Artificial Neural Networks are a computational model inspired by the neural structure of the human brain, a biological neural network They attempt to replicate only the basic elements of this complicated, versatile, and powerful organism It consists of an interconnected group of artificial neurons It learns by changing its structure based on information that flows through the network They are used to model complex relationships between inputs and outputs, or to find patterns in data 5
6 Neurons are the fundamental processing elements of a neural network Jarosz, Q (2009), "Neuron Handtunedsvg" Retrieved 10 September, 2012, from Wikipedia, Neuron 6
7 A biological neuron basically: 1 receives inputs from other sources (dendrites), 2 merges them in some way (soma), 3 performs an operation on the result (axon), then 4 outputs the result  possibly to other neurons (axon terminals) Artificial neurons follow this basic approach 7
8 The basic structure of an artificial neuron consists of: 1 input connections (dendrites) with weights, 2 a summation function or input function (soma), 3 a transfer function or activation function (axon), and 4 output connections (axon terminals) It has no learning process as such 8
9 The function: 1 Input values enter the neuron via the connections 2 The inputs are multiplied with the weighting factor of their respective connection There is often a separate bias connection, which can act as a threshold for the neuron to produce some useful output 3 The modified inputs are fed into a summation function Usually just sums the products 4 The result from the summation function is sent to a transfer function Usually a step function, or a sigmoid function 5 The neuron outputs the result of the transfer function into other neurons, or to an outside connection 9
10 How can the neurons be clustered together? The structure used in these networks is a layering approach These layers are connected to each other in a linear fashion It's possible that a neuron may have an output connection to itself How these layers may be connected is generally problemdependent 10
11 Single layer neurons are the simplest networks Multiple input sources will be fed into the set of neurons, which produce the outputs to the neural network These are called perceptrons These perceptrons can only represent linearlyseparable functions We can make the system represent more complex functions by adding more layers 11
12 Multilayered neural networks are more powerful than singlelayered neural networks The cost is that these hidden layers increase the complexity and training time of these networks Networks with a single hidden layer can approximate any continuous function with arbitrary accuracy Networks with two hidden layers can represent discontinuous functions 12
13 JokerXtreme (2011), "Artificial_neural_networksvg" Retrieved 10 September, 2012, from Wikipedia, Artificial neural network 13
14 There are two main types of multilayered neural networks: 1 Feedforward A simple acyclic structure: Information always moves in one direction; it never goes backwards Stateless encoding; no information is accumulated 14
15 There are two main types of multilayered neural networks: 2 Recurrent A structure with cyclic feedback loops: Information may be sent to any layer; it can process arbitrary sequences of input, and produce more complex results Stateful encoding; introduces shortterm memory into the system, and allows dynamic temporal behaviour 15
16 Artificial neural networks are used to model complex systems that were not understood by the programmer We usually don't know how to construct a perfect neural network for a problem We must train them to produce better results We can only train aspects of a neural network 16
17 Training is the adjusting of parameters with the aim to minimise a measure of error, the cost function What parameters in the artificial neural network do we want to adjust? The weighting factors The link weights influence the function represented by the neural network In the case when we have no idea for the link weights, these might be randomly generated at initialisation 17
18 There are 2 main approaches to training: Supervised: The user provides sample input and output data The network adjusts its weights to match the expected results Unsupervised Only input data is supplied, and the neural network must find patterns on its own 18
19 G McNeil and D Anderson, Artificial Neural Networks Technology, The Data & Analysis Center for Software Technical Report, 1992 Leslie S Smith, "An Introduction to Neural Networks", Centre for Cognitive and Computational Neuroscience, Department of Computing and Mathematics, University of Stirling, 2008 Retrieved 10 September, 2012, Jarosz, Q (2009), "Neuron Handtunedsvg" Retrieved 10 September, 2012, from Wikipedia, Neuron JokerXtreme (2011), "Artificial_neural_networksvg" Retrieved 10 September, 2012, from Wikipedia, Artificial neural network 19
20 Language processing Character recognition Pattern recognition Signal processing Prediction 20
21 Supervised learning Perceptron Feedforward, backpropagation Unsupervised learning Self organising maps 21
22 Simplest type of neural network Introduced by Rosenblatt (1958) 1 2 Inputs n Output Adapted from Haykin, SS 2009 (p48) 22
23 Simplest type of neural network Introduced by Rosenblatt (1958) 1 2 Inputs n Output Adapted from Haykin, SS 2009 (p48) 23
24 Simplest type of neural network Introduced by Rosenblatt (1958) 1 2 Inputs n Output Adapted from Haykin, SS 2009 (p48) 24
25 Input is a real vector i = (i 1,i 2,,i n ) Calculate a weighted scalar s from inputs s = Σ j w j i j + b Calculate output r = sgn(s) 25
26 Categorises input vectors as being in one of two categories A single perceptron can be trained to separate inputs into two linearly separable categories Category 1 Category 2 26
27 Need a training set of input/output pairs Initialise weights and bias (randomly or to zero) Calculate output Adjust the weights and bias in proportion to the difference between actual and expected values 27
28 Repeat until termination criteria is reached Rosenblatt (1962) showed that the weights and bias will converge to fixed values after a finite number of iterations (if the categories are linearly separable) 28
29 We want to classify points in R 2 into those points for which y x+1 and those for which y<x+1 y y=x+1 x 29
30 Initialise bias/weight vector to (0,0,0) Input is the point (1,1) (below the line) expressed as (1,1,1) s = 0x1+0x1+0x1 = 0 Actual output is sgn(0) = +1 Expected output is 1 (below the line) 30
31 Error (expectedactual) is 2 Constant learning rate of 025 So new weight vector is (0,0,0) 31
32 Error (expectedactual) is 2 Constant learning rate of 025 So new weight vector is (0,0,0)
33 Error (expectedactual) is 2 Constant learning rate of 025 So new weight vector is (0,0,0) + 025(2) 33
34 Error (expectedactual) is 2 Constant learning rate of 025 So new weight vector is (0,0,0) + 025(2)(1,1,1) 34
35 Error (expectedactual) is 2 Constant learning rate of 025 So new weight vector is (0,0,0) + 025(2)(1,1,1) = (05,05,05) 35
36 New bias/weight vector is (05,05,05) Input is the point (0,2) (above the line) expressed as (1,0,2) s = 05x1+05x0+05x2 = 05 Actual output is sgn(05) = +1 Expected output is +1 (above the line) no change to weight 36
37 Eventually, this will converge to the correct answer of (a,a,a) for some a>0 Generally, we won t know the correct answer! 37
38 Feedforward network has no connections looping backwards Backpropagation algorithm allows for learning 38
39 Operates similarly to perceptron learning Input Output 39
40 Inputs are fed forward through the network Input Output 40
41 Inputs are fed forward through the network Input Output 41
42 Inputs are fed forward through the network Input Output 42
43 Inputs are fed forward through the network Input Output 43
44 Inputs are fed forward through the network Input Output Compare to expected 44
45 Errors are propagated back Input Output 45
46 Errors are propagated back Input Output 46
47 Errors are propagated back Input Output 47
48 Errors are propagated back Input Output 48
49 Adjust weights based on errors Input Output 49
50 Weights might be updated after each pass or after multiple passes 50
51 Need a comprehensive training set Network cannot be too large for the training set No guarantees the network will learn Network design and learning strategies impact the speed and effectiveness of learning 51
52 More powerful (if you can make it work) No external notion of correct/incorrect output the network uses internal rules to adjust its output in response to inputs 52
53 One or more inputs connected to a set of outputs 53
54 Output neurons form a lattice in (usually) two dimensional space 54
55 Measurable distance between output neurons d 55
56 Based on the network weights, for each input, each output neuron is excited to a different degree 56
57 Select best matching unit (BMU) 57
58 Identify a neighbourhood around BMU 58
59 Based on their levels of excitation, adjust the weights of each output neuron in this neighbourhood to more closely match the input Hope that output neurons diverge into stable (and distinct) categories allowing the input data to be classified 59
60 Adapted from: AIJunkie nd, Kohonen's Self Organizing Feature Maps, Available from: < [11 September 2012] 60
61 AIJunkie nd, Kohonen's Self Organizing Feature Maps, Available from: < [11 September 2012] Bose, NK & Liang, P 1996, Neural network fundamentals with graphs, algorithms, and applications, McGrawHill, New York (Chapters 4,5 and 9) Fausett, LV 1994, Fundamentals of neural networks : architectures, algorithms, and applications, PrenticeHall, Englewood Cliffs, NJ (Chapter 6) Haykin, SS 2009, Neural networks and learning machines, 3 rd edn, Prentice Hall, New York (Chapters 1, 4 and 9) Kartalopoulos, SV 1996, Understanding neural networks and fuzzy logic  basic concepts and applications, IEEE Press, New York (Sections 32, 35 and 314) McNeil, G & Anderson, D 1992, 'Artificial Neural Networks Technology', The Data & Analysis Center for Software Technical Report 61
62 62
Lecture #11: The Perceptron
Lecture #11: The Perceptron Mat Kallada STAT2450  Introduction to Data Mining Outline for Today Welcome back! Assignment 3 The Perceptron Learning Method Perceptron Learning Rule Assignment 3 Will be
More informationData Mining. Neural Networks
Data Mining Neural Networks Goals for this Unit Basic understanding of Neural Networks and how they work Ability to use Neural Networks to solve real problems Understand when neural networks may be most
More information6. NEURAL NETWORK BASED PATH PLANNING ALGORITHM 6.1 INTRODUCTION
6 NEURAL NETWORK BASED PATH PLANNING ALGORITHM 61 INTRODUCTION In previous chapters path planning algorithms such as trigonometry based path planning algorithm and direction based path planning algorithm
More information11/14/2010 Intelligent Systems and Soft Computing 1
Lecture 7 Artificial neural networks: Supervised learning Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in
More informationNeural Networks. CE725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feedforward neural networks Single layer
More informationArtificial neural networks are the paradigm of connectionist systems (connectionism vs. symbolism)
Artificial Neural Networks Analogy to biological neural systems, the most robust learning systems we know. Attempt to: Understand natural biological systems through computational modeling. Model intelligent
More informationData Mining. Kohonen Networks. Data Mining Course: Sharif University of Technology 1
Data Mining Kohonen Networks Data Mining Course: Sharif University of Technology 1 SelfOrganizing Maps Kohonen Networks developed in 198 by Tuevo Kohonen Initially applied to image and sound analysis
More informationClimate Precipitation Prediction by Neural Network
Journal of Mathematics and System Science 5 (205) 20723 doi: 0.7265/259529/205.05.005 D DAVID PUBLISHING Juliana Aparecida Anochi, Haroldo Fraga de Campos Velho 2. Applied Computing Graduate Program,
More informationCharacter Recognition Using Convolutional Neural Networks
Character Recognition Using Convolutional Neural Networks David Bouchain Seminar Statistical Learning Theory University of Ulm, Germany Institute for Neural Information Processing Winter 2006/2007 Abstract
More informationCHAPTER 7 MASS LOSS PREDICTION USING ARTIFICIAL NEURAL NETWORK (ANN)
128 CHAPTER 7 MASS LOSS PREDICTION USING ARTIFICIAL NEURAL NETWORK (ANN) Various mathematical techniques like regression analysis and software tools have helped to develop a model using equation, which
More informationFor Monday. Read chapter 18, sections Homework:
For Monday Read chapter 18, sections 1012 The material in section 8 and 9 is interesting, but we won t take time to cover it this semester Homework: Chapter 18, exercise 25 ab Program 4 Model Neuron
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationNatural Language Processing CS 6320 Lecture 6 Neural Language Models. Instructor: Sanda Harabagiu
Natural Language Processing CS 6320 Lecture 6 Neural Language Models Instructor: Sanda Harabagiu In this lecture We shall cover: Deep Neural Models for Natural Language Processing Introduce Feed Forward
More informationDesign and Performance Analysis of and Gate using Synaptic Inputs for Neural Network Application
IJIRST International Journal for Innovative Research in Science & Technology Volume 1 Issue 12 May 2015 ISSN (online): 23496010 Design and Performance Analysis of and Gate using Synaptic Inputs for Neural
More informationCS 4510/9010 Applied Machine Learning. Neural Nets. Paula Matuszek Fall copyright Paula Matuszek 2016
CS 4510/9010 Applied Machine Learning 1 Neural Nets Paula Matuszek Fall 2016 Neural Nets, the very short version 2 A neural net consists of layers of nodes, or neurons, each of which has an activation
More informationSupervised Learning in Neural Networks (Part 2)
Supervised Learning in Neural Networks (Part 2) Multilayer neural networks (backpropagation training algorithm) The input signals are propagated in a forward direction on a layerbylayer basis. Learning
More informationIn this assignment, we investigated the use of neural networks for supervised classification
Paul Couchman Fabien Imbault Ronan Tigreat Gorka Urchegui Tellechea Classification assignment (group 6) Image processing MSc Embedded Systems March 2003 Classification includes a broad range of decisiontheoric
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou Sun yzsun@ccs.neu.edu November 19, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining
More informationCOMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS
COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS Toomas Kirt Supervisor: Leo Võhandu Tallinn Technical University Toomas.Kirt@mail.ee Abstract: Key words: For the visualisation
More informationLearning. Learning agents Inductive learning. Neural Networks. Different Learning Scenarios Evaluation
Learning Learning agents Inductive learning Different Learning Scenarios Evaluation Slides based on Slides by Russell/Norvig, Ronald Williams, and Torsten Reil Material from Russell & Norvig, chapters
More informationLECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS
LECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS Neural Networks Classifier Introduction INPUT: classification data, i.e. it contains an classification (class) attribute. WE also say that the class
More informationGraph Neural Network. learning algorithm and applications. Shujia Zhang
Graph Neural Network learning algorithm and applications Shujia Zhang What is Deep Learning? Enable computers to mimic human behaviour Artificial Intelligence Machine Learning Subset of ML algorithms using
More informationAssignment # 5. Farrukh Jabeen Due Date: November 2, Neural Networks: Backpropation
Farrukh Jabeen Due Date: November 2, 2009. Neural Networks: Backpropation Assignment # 5 The "Backpropagation" method is one of the most popular methods of "learning" by a neural network. Read the class
More informationArtificial Neuron Modelling Based on Wave Shape
Artificial Neuron Modelling Based on Wave Shape Kieran Greer, Distributed Computing Systems, Belfast, UK. http://distributedcomputingsystems.co.uk Version 1.2 Abstract This paper describes a new model
More information11/14/2010 Intelligent Systems and Soft Computing 1
Lecture 8 Artificial neural networks: Unsupervised learning Introduction Hebbian learning Generalised Hebbian learning algorithm Competitive learning Selforganising computational map: Kohonen network
More informationWHAT TYPE OF NEURAL NETWORK IS IDEAL FOR PREDICTIONS OF SOLAR FLARES?
WHAT TYPE OF NEURAL NETWORK IS IDEAL FOR PREDICTIONS OF SOLAR FLARES? Initially considered for this model was a feed forward neural network. Essentially, this means connections between units do not form
More information2. Neural network basics
2. Neural network basics Next commonalities among different neural networks are discussed in order to get started and show which structural parts or concepts appear in almost all networks. It is presented
More informationNeural Networks CMSC475/675
Introduction to Neural Networks CMSC475/675 Chapter 1 Introduction Why ANN Introduction Some tasks can be done easily (effortlessly) by humans but are hard by conventional paradigms on Von Neumann machine
More informationDr. Qadri Hamarsheh Supervised Learning in Neural Networks (Part 1) learning algorithm Δwkj wkj Theoretically practically
Supervised Learning in Neural Networks (Part 1) A prescribed set of welldefined rules for the solution of a learning problem is called a learning algorithm. Variety of learning algorithms are existing,
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationMultilayer Feedforward networks
Multi Feedforward networks 1. Computational models of McCulloch and Pitts proposed a binary threshold unit as a computational model for artificial neuron. This first type of neuron has been generalized
More informationNeural Networks. By Laurence Squires
Neural Networks By Laurence Squires Machine learning What is it? Type of A.I. (possibly the ultimate A.I.?!?!?!) Algorithms that learn how to classify data The algorithms slowly change their own variables
More informationReview on Methods of Selecting Number of Hidden Nodes in Artificial Neural Network
Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 11, November 2014,
More informationDEVELOPMENT OF NEURAL NETWORK TRAINING METHODOLOGY FOR MODELING NONLINEAR SYSTEMS WITH APPLICATION TO THE PREDICTION OF THE REFRACTIVE INDEX
DEVELOPMENT OF NEURAL NETWORK TRAINING METHODOLOGY FOR MODELING NONLINEAR SYSTEMS WITH APPLICATION TO THE PREDICTION OF THE REFRACTIVE INDEX THESIS CHONDRODIMA EVANGELIA Supervisor: Dr. Alex Alexandridis,
More informationArtificial Neural Networks Lecture Notes Part 5. Stephen Lucci, PhD. Part 5
Artificial Neural Networks Lecture Notes Part 5 About this file: If you have trouble reading the contents of this file, or in case of transcription errors, email gi0062@bcmail.brooklyn.cuny.edu Acknowledgments:
More informationMachine Learning Classifiers and Boosting
Machine Learning Classifiers and Boosting Reading Ch 18.618.12, 20.120.3.2 Outline Different types of learning problems Different types of learning algorithms Supervised learning Decision trees Naïve
More informationData Compression. The Encoder and PCA
Data Compression The Encoder and PCA Neural network techniques have been shown useful in the area of data compression. In general, data compression can be lossless compression or lossy compression. In
More informationOpening the Black Box Data Driven Visualizaion of Neural N
Opening the Black Box Data Driven Visualizaion of Neural Networks September 20, 2006 Aritificial Neural Networks Limitations of ANNs Use of Visualization (ANNs) mimic the processes found in biological
More informationCMPT 882 Week 3 Summary
CMPT 882 Week 3 Summary! Artificial Neural Networks (ANNs) are networks of interconnected simple units that are based on a greatly simplified model of the brain. ANNs are useful learning tools by being
More informationNeural Network Weight Selection Using Genetic Algorithms
Neural Network Weight Selection Using Genetic Algorithms David Montana presented by: Carl Fink, Hongyi Chen, Jack Cheng, Xinglong Li, Bruce Lin, Chongjie Zhang April 12, 2005 1 Neural Networks Neural networks
More informationARTIFICIAL NEURAL NETWORK CIRCUIT FOR SPECTRAL PATTERN RECOGNITION
ARTIFICIAL NEURAL NETWORK CIRCUIT FOR SPECTRAL PATTERN RECOGNITION A Thesis by FARAH RASHEED Submitted to the Office of Graduate and Professional Studies of Texas A&M University in partial fulfillment
More informationHybrid PSOSA algorithm for training a Neural Network for Classification
Hybrid PSOSA algorithm for training a Neural Network for Classification Sriram G. Sanjeevi 1, A. Naga Nikhila 2,Thaseem Khan 3 and G. Sumathi 4 1 Associate Professor, Dept. of CSE, National Institute
More informationSupervised Learning with Neural Networks. We now look at how an agent might learn to solve a general problem by seeing examples.
Supervised Learning with Neural Networks We now look at how an agent might learn to solve a general problem by seeing examples. Aims: to present an outline of supervised learning as part of AI; to introduce
More informationOptimization Methods for Machine Learning (OMML)
Optimization Methods for Machine Learning (OMML) 2nd lecture Prof. L. Palagi References: 1. Bishop Pattern Recognition and Machine Learning, Springer, 2006 (Chap 1) 2. V. Cherlassky, F. Mulier  Learning
More informationMachine Learning (CSE 446): Unsupervised Learning
Machine Learning (CSE 446): Unsupervised Learning Sham M Kakade c 2018 University of Washington cse446staff@cs.washington.edu 1 / 19 Announcements HW2 posted. Due Feb 1. It is long. Start this week! Today:
More informationInstructor: Jessica Wu Harvey Mudd College
The Perceptron Instructor: Jessica Wu Harvey Mudd College The instructor gratefully acknowledges Andrew Ng (Stanford), Eric Eaton (UPenn), David Kauchak (Pomona), and the many others who made their course
More informationNeural Network Learning. Today s Lecture. Continuation of Neural Networks. Artificial Neural Networks. Lecture 24: Learning 3. Victor R.
Lecture 24: Learning 3 Victor R. Lesser CMPSCI 683 Fall 2010 Today s Lecture Continuation of Neural Networks Artificial Neural Networks Compose of nodes/units connected by links Each link has a numeric
More informationSupport Vector Machines
Support Vector Machines RBFnetworks Support Vector Machines Good Decision Boundary Optimization Problem Soft margin Hyperplane Nonlinear Decision Boundary KernelTrick Approximation Accurancy Overtraining
More informationWeek 3: Perceptron and Multilayer Perceptron
Week 3: Perceptron and Multilayer Perceptron Phong Le, Willem Zuidema November 12, 2013 Last week we studied two famous biological neuron models, FitzhughNagumo model and Izhikevich model. This week,
More informationImage Compression: An Artificial Neural Network Approach
Image Compression: An Artificial Neural Network Approach Anjana B 1, Mrs Shreeja R 2 1 Department of Computer Science and Engineering, Calicut University, Kuttippuram 2 Department of Computer Science and
More informationAssignment 2. Classification and Regression using Linear Networks, Multilayer Perceptron Networks, and Radial Basis Functions
ENEE 739Q: STATISTICAL AND NEURAL PATTERN RECOGNITION Spring 2002 Assignment 2 Classification and Regression using Linear Networks, Multilayer Perceptron Networks, and Radial Basis Functions Aravind Sundaresan
More informationPerceptrons and Backpropagation. Fabio Zachert Cognitive Modelling WiSe 2014/15
Perceptrons and Backpropagation Fabio Zachert Cognitive Modelling WiSe 2014/15 Content History Mathematical View of Perceptrons Network Structures Gradient Descent Backpropagation (SingleLayer, MultilayerNetworks)
More informationInstantaneously trained neural networks with complex inputs
Louisiana State University LSU Digital Commons LSU Master's Theses Graduate School 2003 Instantaneously trained neural networks with complex inputs Pritam Rajagopal Louisiana State University and Agricultural
More informationThe use of Neural Networks in Text Categorization. Abhishek Sethi
The use of Neural Networks in Text Categorization Abhishek Sethi What is a Neural Network An attempt at creating a learning system that mimics the brain Brain consists of billions of neurons, interconnected,
More informationCOMPUTATIONAL INTELLIGENCE
COMPUTATIONAL INTELLIGENCE Fundamentals Adrian Horzyk Preface Before we can proceed to discuss specific complex methods we have to introduce basic concepts, principles, and models of computational intelligence
More informationCOMP 551 Applied Machine Learning Lecture 14: Neural Networks
COMP 551 Applied Machine Learning Lecture 14: Neural Networks Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise noted, all material posted for this course
More informationIntroduction to Neural Networks
Introduction to Neural Networks What are connectionist neural networks? Connectionism refers to a computer modeling approach to computation that is loosely based upon the architecture of the brain Many
More informationArtificial Neural Networks. Introduction to Computational Neuroscience Ardi Tampuu
Artificial Neural Networks Introduction to Computational Neuroscience Ardi Tampuu 7.0.206 Artificial neural network NB! Inspired by biology, not based on biology! Applications Automatic speech recognition
More informationA Dendrogram. Bioinformatics (Lec 17)
A Dendrogram 3/15/05 1 Hierarchical Clustering [Johnson, SC, 1967] Given n points in R d, compute the distance between every pair of points While (not done) Pick closest pair of points s i and s j and
More informationAn Intelligent Technique for Image Compression
An Intelligent Technique for Image Compression Athira Mayadevi Somanathan 1, V. Kalaichelvi 2 1 Dept. Of Electronics and Communications Engineering, BITS Pilani, Dubai, U.A.E. 2 Dept. Of Electronics and
More informationIntroduction AL Neuronale Netzwerke. VL Algorithmisches Lernen, Teil 2b. Norman Hendrich
Introduction AL 64360 Neuronale Netzwerke VL Algorithmisches Lernen, Teil 2b Norman Hendrich University of Hamburg, Dept. of Informatics VogtKöllnStr. 30, D22527 Hamburg hendrich@informatik.unihamburg.de
More informationClassification and Regression using Linear Networks, Multilayer Perceptrons and Radial Basis Functions
ENEE 739Q SPRING 2002 COURSE ASSIGNMENT 2 REPORT 1 Classification and Regression using Linear Networks, Multilayer Perceptrons and Radial Basis Functions Vikas Chandrakant Raykar Abstract The aim of the
More information4.12 Generalization. In backpropagation learning, as many training examples as possible are typically used.
1 4.12 Generalization In backpropagation learning, as many training examples as possible are typically used. It is hoped that the network so designed generalizes well. A network generalizes well when
More informationClassification Lecture Notes cse352. Neural Networks. Professor Anita Wasilewska
Classification Lecture Notes cse352 Neural Networks Professor Anita Wasilewska Neural Networks Classification Introduction INPUT: classification data, i.e. it contains an classification (class) attribute
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More informationUse of Artificial Neural Networks to Investigate the Surface Roughness in CNC Milling Machine
Use of Artificial Neural Networks to Investigate the Surface Roughness in CNC Milling Machine M. Vijay Kumar Reddy 1 1 Department of Mechanical Engineering, Annamacharya Institute of Technology and Sciences,
More informationObject Detection Lecture Introduction to deep learning (CNN) Idar Dyrdal
Object Detection Lecture 10.3  Introduction to deep learning (CNN) Idar Dyrdal Deep Learning Labels Computational models composed of multiple processing layers (nonlinear transformations) Used to learn
More informationNeural Nets. General Model Building
Neural Nets To give you an idea of how new this material is, let s do a little history lesson. The origins of neural nets are typically dated back to the early 1940 s and work by two physiologists, McCulloch
More informationAlex Waibel
Alex Waibel 815.11.2011 1 16.11.2011 Organisation Literatur: Introduction to The Theory of Neural Computation Hertz, Krogh, Palmer, Santa Fe Institute Neural Network Architectures An Introduction, Judith
More informationFunction approximation using RBF network. 10 basis functions and 25 data points.
1 Function approximation using RBF network F (x j ) = m 1 w i ϕ( x j t i ) i=1 j = 1... N, m 1 = 10, N = 25 10 basis functions and 25 data points. Basis function centers are plotted with circles and data
More informationSupport Vector Machines
Support Vector Machines RBFnetworks Support Vector Machines Good Decision Boundary Optimization Problem Soft margin Hyperplane Nonlinear Decision Boundary KernelTrick Approximation Accurancy Overtraining
More informationSimulation of Back Propagation Neural Network for Iris Flower Classification
American Journal of Engineering Research (AJER) eissn: 23200847 pissn : 23200936 Volume6, Issue1, pp200205 www.ajer.org Research Paper Open Access Simulation of Back Propagation Neural Network
More informationCSC 578 Neural Networks and Deep Learning
CSC 578 Neural Networks and Deep Learning Fall 2018/19 7. Recurrent Neural Networks (Some figures adapted from NNDL book) 1 Recurrent Neural Networks 1. Recurrent Neural Networks (RNNs) 2. RNN Training
More informationCPSC 340: Machine Learning and Data Mining. Deep Learning Fall 2018
CPSC 340: Machine Learning and Data Mining Deep Learning Fall 2018 Last Time: MultiDimensional Scaling Multidimensional scaling (MDS): Nonparametric visualization: directly optimize the z i locations.
More informationNeural Networks. Neural Network. Neural Network. Neural Network 2/21/2008. Andrew Kusiak. Intelligent Systems Laboratory Seamans Center
Neural Networks Neural Network Input Andrew Kusiak Intelligent t Systems Laboratory 2139 Seamans Center Iowa City, IA 522421527 andrewkusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak Tel. 319335
More informationNeural Networks. Singlelayer neural network. CSE 446: Machine Learning Emily Fox University of Washington March 10, /10/2017
3/0/207 Neural Networks Emily Fox University of Washington March 0, 207 Slides adapted from Ali Farhadi (via Carlos Guestrin and Luke Zettlemoyer) Singlelayer neural network 3/0/207 Perceptron as a neural
More informationSEMANTIC COMPUTING. Lecture 8: Introduction to Deep Learning. TU Dresden, 7 December Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 8: Introduction to Deep Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 7 December 2018 Overview Introduction Deep Learning General Neural Networks
More informationReview: Final Exam CPSC Artificial Intelligence Michael M. Richter
Review: Final Exam Model for a Learning Step Learner initially Environm ent Teacher Compare s pe c ia l Information Control Correct Learning criteria Feedback changed Learner after Learning Learning by
More informationBack propagation Algorithm:
Network Neural: A neural network is a class of computing system. They are created from very simple processing nodes formed into a network. They are inspired by the way that biological systems such as the
More informationINTRODUCTION TO DEEP LEARNING
INTRODUCTION TO DEEP LEARNING CONTENTS Introduction to deep learning Contents 1. Examples 2. Machine learning 3. Neural networks 4. Deep learning 5. Convolutional neural networks 6. Conclusion 7. Additional
More informationTechnical University of Munich. Exercise 7: Neural Network Basics
Technical University of Munich Chair for Bioinformatics and Computational Biology Protein Prediction I for Computer Scientists SoSe 2018 Prof. B. Rost M. Bernhofer, M. Heinzinger, D. Nechaev, L. Richter
More informationRough Set Approach to Unsupervised Neural Network based Pattern Classifier
Rough Set Approach to Unsupervised Neural based Pattern Classifier Ashwin Kothari, Member IAENG, Avinash Keskar, Shreesha Srinath, and Rakesh Chalsani Abstract Early Convergence, input feature space with
More informationIMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS
IMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS BOGDAN M.WILAMOWSKI University of Wyoming RICHARD C. JAEGER Auburn University ABSTRACT: It is shown that by introducing special
More informationSelfOrganizing Maps for cyclic and unbounded graphs
SelfOrganizing Maps for cyclic and unbounded graphs M. Hagenbuchner 1, A. Sperduti 2, A.C. Tsoi 3 1 University of Wollongong, Wollongong, Australia. 2 University of Padova, Padova, Italy. 3 Hong Kong
More informationPattern Classification Algorithms for Face Recognition
Chapter 7 Pattern Classification Algorithms for Face Recognition 7.1 Introduction The best pattern recognizers in most instances are human beings. Yet we do not completely understand how the brain recognize
More informationTraffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers
Traffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers A. Salhi, B. Minaoui, M. Fakir, H. Chakib, H. Grimech Faculty of science and Technology Sultan Moulay Slimane
More informationNeural Networks (Overview) Prof. Richard Zanibbi
Neural Networks (Overview) Prof. Richard Zanibbi Inspired by Biology Introduction But as used in pattern recognition research, have little relation with real neural systems (studied in neurology and neuroscience)
More informationIMPLEMENTATION OF FPGABASED ARTIFICIAL NEURAL NETWORK (ANN) FOR FULL ADDER. Research Scholar, IIT Kharagpur.
Journal of Analysis and Computation (JAC) (An International Peer Reviewed Journal), www.ijaconline.com, ISSN 09732861 Volume XI, Issue I, Jan December 2018 IMPLEMENTATION OF FPGABASED ARTIFICIAL NEURAL
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira JanuaryMarch 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationEfficient Object Extraction Using Fuzzy Cardinality Based Thresholding and Hopfield Network
Efficient Object Extraction Using Fuzzy Cardinality Based Thresholding and Hopfield Network S. Bhattacharyya U. Maulik S. Bandyopadhyay Dept. of Information Technology Dept. of Comp. Sc. and Tech. Machine
More informationturning data into dollars
turning data into dollars Tom s Ten Data Tips November 2008 Neural Networks Neural Networks (NNs) are sometimes considered the epitome of data mining algorithms. Loosely modeled after the human brain (hence
More informationArtificial Neural Networks
The Perceptron Rodrigo Fernandes de Mello Invited Professor at Télécom ParisTech Associate Professor at Universidade de São Paulo, ICMC, Brazil http://www.icmc.usp.br/~mello mello@icmc.usp.br Conceptually
More informationarxiv: v1 [physics.dataan] 27 Sep 2007
Classification of Interest Rate Curves Using SelfOrganising Maps arxiv:0709.4401v1 [physics.dataan] 27 Sep 2007 M.Kanevski a,, M.Maignan b, V.Timonin a,1, A.Pozdnoukhov a,1 a Institute of Geomatics and
More informationMachine Learning in Biology
Università degli studi di Padova Machine Learning in Biology Luca Silvestrin (Dottorando, XXIII ciclo) Supervised learning Contents Classconditional probability density Linear and quadratic discriminant
More informationLecture 20: Neural Networks for NLP. Zubin Pahuja
Lecture 20: Neural Networks for NLP Zubin Pahuja zpahuja2@illinois.edu courses.engr.illinois.edu/cs447 CS447: Natural Language Processing 1 Today s Lecture Feedforward neural networks as classifiers simple
More informationLecture 2 Notes. Outline. Neural Networks. The Big Idea. Architecture. Instructors: Parth Shah, Riju Pahwa
Instructors: Parth Shah, Riju Pahwa Lecture 2 Notes Outline 1. Neural Networks The Big Idea Architecture SGD and Backpropagation 2. Convolutional Neural Networks Intuition Architecture 3. Recurrent Neural
More informationA novel firing rule for training Kohonen selforganising
A novel firing rule for training Kohonen selforganising maps D. T. Pham & A. B. Chan Manufacturing Engineering Centre, School of Engineering, University of Wales Cardiff, P.O. Box 688, Queen's Buildings,
More informationNeural Networks: What can a network represent. Deep Learning, Fall 2018
Neural Networks: What can a network represent Deep Learning, Fall 2018 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning 2 Recap : NNets and the brain
More informationA modified and fast Perceptron learning rule and its use for Tag Recommendations in Social Bookmarking Systems
A modified and fast Perceptron learning rule and its use for Tag Recommendations in Social Bookmarking Systems Anestis Gkanogiannis and Theodore Kalamboukis Department of Informatics Athens University
More informationDEEP LEARNING REVIEW. Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature Presented by Divya Chitimalla
DEEP LEARNING REVIEW Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature 2015 Presented by Divya Chitimalla What is deep learning Deep learning allows computational models that are composed of multiple
More information