Introduction to Neural Networks
|
|
- Philip Ford
- 6 years ago
- Views:
Transcription
1 Introduction to Neural Networks
2 What are connectionist neural networks? Connectionism refers to a computer modeling approach to computation that is loosely based upon the architecture of the brain Many different models, but all include: Multiple, individual nodes or units that operate at the same time (in parallel) A network that connects the nodes together Information is stored in a distributed fashion among the links that connect the nodes Learning can occur with gradual changes in connection strength
3 History of Neural Networks () Attempts to mimic the human brain date back to work in the 93s, 94s, & 95s by Alan Turing, Warren McCullough, Walter Pitts, Donald Hebb and James von Neumann 943 McCulloch-Pitts: neuron as computing element 948 Wiener: cybernetics 949 Hebb: learning rule 957 Rosenblatt at Cornell developed Perceptron, a hardware neural net for character recognition 959 Widrow and Hoff at Stanford developed Adaline for adaptive control of noise on telephone lines 96 Widrow-Hoff: least mean square algorithm 3
4 History of Neural Networks () Recession 969 Minsky-Papert: limitations of perceptron model Linear Separability in Perceptrons 4
5 History of Neural Networks (3) Revival, mathematically tied together many of the ideas from previous research 98 Hopfield: recurrent network model 98 Kohonen: self-organizing maps 986 Rumelhart et. al.: backpropagation universial approximation Since then, growth has exploded. Over 8% of Fortune 5 have neural net R&D programs Thousands of research papers Commercial software applications 5
6 Application with Neural Network Forecasting/Market Prediction: finance and banking Manufacturing: quality control, fault diagnosis Medicine: analysis of electrocardiogram data, RNA & DNA sequencing, drug development without animal testing Pattern/Image recognition: handwriting recognition, airport bomb detection Optimization: without Simplex Control: process, robotics 6
7 Comparison of Brains and Traditional Computers billion neurons, 3 trillion synapses Element size: -6 m Energy use: 5W Processing speed: Hz Parallel, Distributed Fault Tolerant Learns: Yes Intelligent/Conscious: Usually billion bytes RAM but trillions of bytes on disk Element size: -9 m Energy watt: 3~9W (CPU) Processing speed: 9 Hz Serial, Centralized Generally not Fault Tolerant Learns: Some Intelligent/Conscious: Generally No 7
8 Biological Inspiration Idea : To make the computer more robust, intelligent, and learn, Let s model our computer software (and/or hardware) after the brain My brain: It's my second favorite organ. - Woody Allen, from the movie Sleeper 8
9 Neurons in the Brain Although heterogeneous, at a low level the brain is composed of neurons A neuron receives input from other neurons (generally thousands) from its synapses Inputs are approximately summed When the input exceeds a threshold the neuron sends an electrical spike that travels from the body, down the axon, to the next neuron(s) 9
10 Biological Neuron 3 major functional units Dendrites Cell body Axon Synapse x x xn Amount of signal passing through a neuron depends on: Intensity of signal from feeding neurons Their synaptic strengths Threshold of the receiving neuron Hebb rule (plays key part in learning) A synapse which repeatedly triggers the activation of a postsynaptic neuron will grow in strength, others will gradually weaken Learn by adjusting magnitudes of synapses strengths w g(ξ) w w n y ξ
11 Learning in the Brain Brains learn Altering strength between neurons Creating/deleting connections Hebb s Postulate (Hebbian Learning) When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased Long Term Potentiation (LTP) Cellular basis for learning and memory LTP is the long-lasting strengthening of the connection between two nerve cells in response to stimulation Discovered in many regions of the cortex
12 Artificial Neurons (basic computational entities of an ANN) Analogy between artificial and biological concepts (connection weights represent synapses) y In 958 Rosenblatt introduced mechanics (perceptron) Input to output: y=g( i w i x i ) Only when sum exceeds the threshold limit will neuron fire g( ) w.x Weights can enhance or inhibit Collective behaviour of neurons is what s interesting for intelligent data processing w w w 3 x x x 3
13 3 Model of a Neuron
14 Activation Function f(a) + A - Step Function f(a) + A - Sigmoid Function 4
15 Perceptrons Can be trained on a set of examples using a special learning rule (process) Weights are changed in proportion to the difference (error) between target output and perceptron output for each example Minimize summed square error function: E = / p i (o i (p) -t i (p) ) with respect to the weights Error is function of all the weights and forms an irregular multidimensional complex hyperplane with many peaks, saddle points and minima Error minimized by finding set of weights that correspond to global minimum Done with gradient descent method weights incrementally updated in proportion to δe/δw ij Updating reads: w ij (t + ) = w ij (t) w ij Aim is to produce a true mapping for all patterns w ij x j o i g(ξ) threshold ξ 5
16 6 Perceptron Structure
17 Learning for Perceptron. Initialize w ij with random values. Repeat until w ij (t + ) w ij (t): Pick pattern p from training set Feed input to network and calculate the output Update the weights according to w ij (t + ) = w ij (t) - w ij where w ij = -η δe/δw ij. When no change (within some accuracy) occurs, the weights are frozen and network is ready to use on data it has never seen 7
18 Example AND OR x x t x x t Perceptron learns these rules easily (ie, sets appropriate weights and threshold) w=(w,w,w ) = (-.5,.,.) and (-.5,.,.) where w corresponds to the threshold term 8
19 Problem & Solution Perceptrons can only perform accurately with linearly separable classes linear hyperplane can place one class of objects on one side of plane and other class on other x ANN research put on hold for yrs x Solution: additional (hidden) layers of neurons, MLP architecture Able to solve non-linear classification problems x x 9
20 Multilayer Perceptrons (MLPs) Learning procedure is extension of simple perceptron algorithm o i Response function: o i =g( i w ij g( k w jk x k )) which is non-linear so network able to perform non-linear mappings h j w ij Theory tells us that a neural network with at least hidden layer can represent any function w jk Vast number of ANN types exist x k
21 MLP Structure
22 Geometric Interpretation of Perceptron Learning
23 Backpropagation ANNs Most widely used type of network Feedforward Supervised (learns mapping from one data space to another using examples) Error propagated backwards Versatile. Used for data modelling, classification, forecasting, data and image compression and pattern recognition. 3
24 BP Learning Algorithm Like Perceptron, uses gradient descent to minimize error (generalized to case with hidden layers) Each iteration constitutes two sweeps To minimize Error we need δe/δw ij but also need δe/δw jk (which we get using the chain rule) Training of MLP using BP can be thought of as a walk in weight space along an energy surface, trying to find global minimum and avoiding local minima Unlike for Perceptron, there is no guarantee that global minimum will be reached, but most cases energy landscape is smooth 4
25 5 Backpropagation Net Structure
26 BP Learning Algorithm. Initialize w ij and w jk with random values. Repeat until w ij and w jk have converged or the desired performance level is reached: Pick pattern p from training set Present input and calculate the output Update weights according to: w ij (t + ) = w ij (t) w ij w jk (t + ) = w jk (t) w jk where w = -η δe/δw. ( etc for extra hidden layers) 6
27 Training Generalization: network s performance on a set of test patterns it has never seen before (lower than on training set) Training set used to let ANN capture features in data or mapping Error (eg SSE) Testing Initial large drop in error is due to learning, but subsequent slow reduction is due to:. Network memorization (too many training cycles used). Overfitting (too many hidden nodes) Training Optimum network (network learns individual training examples and loses generalization ability) No. of hidden nodes or training cycles 7
28 Other Popular ANNs Some problems can be solved using variety of ANN types, some only via specific. (problem logistics) Hopfield networks: optimization Presented with incomplete/noisy pattern, network responds by retrieving an internally stored pattern it most closely resembles Kohonen networks: (self-organizing) Trained in an unsupervised manner to form clusters in the data. Used for pattern classification and data compression 8
29 Summary of ANN Learning Artificial Neural Networks Feedforward Recurrent Unsupervised Supervised Unsupervised Supervised Kohonen, Hebbian MLP, RBF ART Elman, Jordan, Hopfield 9
30 홉필드망 : 구조와작동식 제약조건 w ij = w ji w ii = i I i, O i {,} 작동식 NET O j j wijoi + = I j + = O j if NET j if NET j if NET j > T = T j j < T j 구조 3
31 홉필드망 : 특성과목적 특성 피드백이있는 recurrent네트워크 동적네트워크 목적 입력에가장가까운패턴출력 응용분야 연상기억 (Associative memory) 최적화 (Optimization) 3
32 3 예 () 문제 : 두패턴벡터를저장 학습 : 연결강도를에의해구함 t t x x ),, (,, ),,, ( = = t t x x x x w + = = w
33 33 예 () 연상실험. 학습데이터의복구능력 = = 4 x w
34 34 예 (3) 불완전한데이터의복구능력 = = 4 4 x w limiting hard + - f h (x) x
35 실제예와문제점 연상시킬패턴들의유사도가적어야함 네트워크의용량 : 노드수의약 5% 예 : 개패턴의경우 7개이상의노드가필요 5개이상의연결필요 35
36 Boltzman Machine 시뮬레이티드어닐링 At temperature T, output value is determined Stochastically by Boltzman distribution With carefully designed Annealing schedule 볼쯔만분포 P( E i ) = α e E β/ T i 특성 시뮬레이티드어닐링등에의해통계학적으로작동하는신경망 전역최적화가가능한네트워크 36
37 37 에너지곡선
38 Self-Organizing Map Self-organizing map (SOM) Unsupervised learning Preserves the topology of data Widely used in data visualization or topology-preserving mapping Selection of winner: Weight update x mc = min{ x mi } mi( t + ) = mi( t) + α( t) nci( t) { x( t) mi( t)} i 38
39 39 SOM Structure
40 SASOM. Start with a basic SOM (4X4 map). Train the current network with the Kohonen s algorithm 3. Calibrate the network using known I/O patterns to determine Node should be replaced with a submap of several nodes (X map) Node should be deleted 4. Unless every node represents a unique class, go to step 4
41 Learning Procedure Input data Initialize map as 4X4 Train with Kohonen s algorithm Structure adaptation Find nodes whose hit_ratio is less than 95.% Split the nodes to X submap Train the split nodes with LVQ algorithm Remove nodes not participated in learning Stop condition satisfied? Yes Map generated No 4
42 Kohonen s Learning Initialization 4X4 rectangular map using Kohonen s learning algorithm Learning Winner node x m c = min{ i x mi } Kohonen s learning algorithm mi( t + ) = mi( t) + α( t) nci( t) { x( t) mi( t)} n ci (t) α(t) Neighborhood function Learning rate 4
43 Dynamic Node-Splitting Determining whether a node is to be split or not Hit ratio hit _ ratio i = max j P( c j n i ) where i =,, L, M and j =,, L, N Nodes less than 95.% of hit ratio are split,,,, 43
44 Initial Weight of Split Nodes C P N c S : Child node : Parent node C = ( P ) S + : Weights of neighbors : Total number of nodes that participate in weight initialization N c P P C C P P 4 P P P C C 3 P 3 P 3 44
45 LVQ Learning for Modified Map mi( t + ) = mi( t) + α( t) nci( t) hci( t) { x( t) mi( t)} { h ( t) = ci ) h ci ( t) = ), x (t and m i (t) belong to the same class, x (t and (t) belong to different classes m i Neighborhood function is used to preserve the topological order 45
46 Homework #. Information Geometry 에근거한 MLP 학습원리설명및학습성능향상을위한방법론을조사하시오.. MLP 를실제문제해결에사용하기위한 Tips 를네트워크의구조, 학습알고리즘, 학습데이터전처리로나누어조사하시오. 46
Neural Networks CMSC475/675
Introduction to Neural Networks CMSC475/675 Chapter 1 Introduction Why ANN Introduction Some tasks can be done easily (effortlessly) by humans but are hard by conventional paradigms on Von Neumann machine
More informationNeural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer
More informationData Mining. Neural Networks
Data Mining Neural Networks Goals for this Unit Basic understanding of Neural Networks and how they work Ability to use Neural Networks to solve real problems Understand when neural networks may be most
More informationCOMPUTATIONAL INTELLIGENCE
COMPUTATIONAL INTELLIGENCE Fundamentals Adrian Horzyk Preface Before we can proceed to discuss specific complex methods we have to introduce basic concepts, principles, and models of computational intelligence
More informationArtificial neural networks are the paradigm of connectionist systems (connectionism vs. symbolism)
Artificial Neural Networks Analogy to biological neural systems, the most robust learning systems we know. Attempt to: Understand natural biological systems through computational modeling. Model intelligent
More information11/14/2010 Intelligent Systems and Soft Computing 1
Lecture 8 Artificial neural networks: Unsupervised learning Introduction Hebbian learning Generalised Hebbian learning algorithm Competitive learning Self-organising computational map: Kohonen network
More informationLearning. Learning agents Inductive learning. Neural Networks. Different Learning Scenarios Evaluation
Learning Learning agents Inductive learning Different Learning Scenarios Evaluation Slides based on Slides by Russell/Norvig, Ronald Williams, and Torsten Reil Material from Russell & Norvig, chapters
More informationDr. Qadri Hamarsheh Supervised Learning in Neural Networks (Part 1) learning algorithm Δwkj wkj Theoretically practically
Supervised Learning in Neural Networks (Part 1) A prescribed set of well-defined rules for the solution of a learning problem is called a learning algorithm. Variety of learning algorithms are existing,
More informationPerceptrons and Backpropagation. Fabio Zachert Cognitive Modelling WiSe 2014/15
Perceptrons and Backpropagation Fabio Zachert Cognitive Modelling WiSe 2014/15 Content History Mathematical View of Perceptrons Network Structures Gradient Descent Backpropagation (Single-Layer-, Multilayer-Networks)
More informationFor Monday. Read chapter 18, sections Homework:
For Monday Read chapter 18, sections 10-12 The material in section 8 and 9 is interesting, but we won t take time to cover it this semester Homework: Chapter 18, exercise 25 a-b Program 4 Model Neuron
More informationNeural Networks. Neural Network. Neural Network. Neural Network 2/21/2008. Andrew Kusiak. Intelligent Systems Laboratory Seamans Center
Neural Networks Neural Network Input Andrew Kusiak Intelligent t Systems Laboratory 2139 Seamans Center Iowa City, IA 52242-1527 andrew-kusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak Tel. 319-335
More informationSupervised Learning in Neural Networks (Part 2)
Supervised Learning in Neural Networks (Part 2) Multilayer neural networks (back-propagation training algorithm) The input signals are propagated in a forward direction on a layer-bylayer basis. Learning
More information11/14/2010 Intelligent Systems and Soft Computing 1
Lecture 7 Artificial neural networks: Supervised learning Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in
More informationCS 4510/9010 Applied Machine Learning. Neural Nets. Paula Matuszek Fall copyright Paula Matuszek 2016
CS 4510/9010 Applied Machine Learning 1 Neural Nets Paula Matuszek Fall 2016 Neural Nets, the very short version 2 A neural net consists of layers of nodes, or neurons, each of which has an activation
More informationAssignment # 5. Farrukh Jabeen Due Date: November 2, Neural Networks: Backpropation
Farrukh Jabeen Due Date: November 2, 2009. Neural Networks: Backpropation Assignment # 5 The "Backpropagation" method is one of the most popular methods of "learning" by a neural network. Read the class
More information6. NEURAL NETWORK BASED PATH PLANNING ALGORITHM 6.1 INTRODUCTION
6 NEURAL NETWORK BASED PATH PLANNING ALGORITHM 61 INTRODUCTION In previous chapters path planning algorithms such as trigonometry based path planning algorithm and direction based path planning algorithm
More informationNeural Network Learning. Today s Lecture. Continuation of Neural Networks. Artificial Neural Networks. Lecture 24: Learning 3. Victor R.
Lecture 24: Learning 3 Victor R. Lesser CMPSCI 683 Fall 2010 Today s Lecture Continuation of Neural Networks Artificial Neural Networks Compose of nodes/units connected by links Each link has a numeric
More informationArtificial Neural Networks
The Perceptron Rodrigo Fernandes de Mello Invited Professor at Télécom ParisTech Associate Professor at Universidade de São Paulo, ICMC, Brazil http://www.icmc.usp.br/~mello mello@icmc.usp.br Conceptually
More informationCMPT 882 Week 3 Summary
CMPT 882 Week 3 Summary! Artificial Neural Networks (ANNs) are networks of interconnected simple units that are based on a greatly simplified model of the brain. ANNs are useful learning tools by being
More informationPattern Classification Algorithms for Face Recognition
Chapter 7 Pattern Classification Algorithms for Face Recognition 7.1 Introduction The best pattern recognizers in most instances are human beings. Yet we do not completely understand how the brain recognize
More informationArtificial Neural Networks. Introduction to Computational Neuroscience Ardi Tampuu
Artificial Neural Networks Introduction to Computational Neuroscience Ardi Tampuu 7.0.206 Artificial neural network NB! Inspired by biology, not based on biology! Applications Automatic speech recognition
More informationMultilayer Feed-forward networks
Multi Feed-forward networks 1. Computational models of McCulloch and Pitts proposed a binary threshold unit as a computational model for artificial neuron. This first type of neuron has been generalized
More information4.12 Generalization. In back-propagation learning, as many training examples as possible are typically used.
1 4.12 Generalization In back-propagation learning, as many training examples as possible are typically used. It is hoped that the network so designed generalizes well. A network generalizes well when
More informationNeural Networks. Robot Image Credit: Viktoriya Sukhanova 123RF.com
Neural Networks These slides were assembled by Eric Eaton, with grateful acknowledgement of the many others who made their course materials freely available online. Feel free to reuse or adapt these slides
More informationCharacter Recognition Using Convolutional Neural Networks
Character Recognition Using Convolutional Neural Networks David Bouchain Seminar Statistical Learning Theory University of Ulm, Germany Institute for Neural Information Processing Winter 2006/2007 Abstract
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou Sun yzsun@ccs.neu.edu November 19, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining
More informationNeural Networks (Overview) Prof. Richard Zanibbi
Neural Networks (Overview) Prof. Richard Zanibbi Inspired by Biology Introduction But as used in pattern recognition research, have little relation with real neural systems (studied in neurology and neuroscience)
More informationInstructor: Jessica Wu Harvey Mudd College
The Perceptron Instructor: Jessica Wu Harvey Mudd College The instructor gratefully acknowledges Andrew Ng (Stanford), Eric Eaton (UPenn), David Kauchak (Pomona), and the many others who made their course
More informationReview: Final Exam CPSC Artificial Intelligence Michael M. Richter
Review: Final Exam Model for a Learning Step Learner initially Environm ent Teacher Compare s pe c ia l Information Control Correct Learning criteria Feedback changed Learner after Learning Learning by
More informationClimate Precipitation Prediction by Neural Network
Journal of Mathematics and System Science 5 (205) 207-23 doi: 0.7265/259-529/205.05.005 D DAVID PUBLISHING Juliana Aparecida Anochi, Haroldo Fraga de Campos Velho 2. Applied Computing Graduate Program,
More informationSupervised Learning (contd) Linear Separation. Mausam (based on slides by UW-AI faculty)
Supervised Learning (contd) Linear Separation Mausam (based on slides by UW-AI faculty) Images as Vectors Binary handwritten characters Treat an image as a highdimensional vector (e.g., by reading pixel
More informationLECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS
LECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS Neural Networks Classifier Introduction INPUT: classification data, i.e. it contains an classification (class) attribute. WE also say that the class
More informationYuki Osada Andrew Cannon
Yuki Osada Andrew Cannon 1 Humans are an intelligent species One feature is the ability to learn The ability to learn comes down to the brain The brain learns from experience Research shows that the brain
More informationOpening the Black Box Data Driven Visualizaion of Neural N
Opening the Black Box Data Driven Visualizaion of Neural Networks September 20, 2006 Aritificial Neural Networks Limitations of ANNs Use of Visualization (ANNs) mimic the processes found in biological
More informationNeural Network and Deep Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina
Neural Network and Deep Learning Early history of deep learning Deep learning dates back to 1940s: known as cybernetics in the 1940s-60s, connectionism in the 1980s-90s, and under the current name starting
More informationFunction approximation using RBF network. 10 basis functions and 25 data points.
1 Function approximation using RBF network F (x j ) = m 1 w i ϕ( x j t i ) i=1 j = 1... N, m 1 = 10, N = 25 10 basis functions and 25 data points. Basis function centers are plotted with circles and data
More informationLogical Rhythm - Class 3. August 27, 2018
Logical Rhythm - Class 3 August 27, 2018 In this Class Neural Networks (Intro To Deep Learning) Decision Trees Ensemble Methods(Random Forest) Hyperparameter Optimisation and Bias Variance Tradeoff Biological
More informationMulti Layer Perceptron with Back Propagation. User Manual
Multi Layer Perceptron with Back Propagation User Manual DAME-MAN-NA-0011 Issue: 1.3 Date: September 03, 2013 Author: S. Cavuoti, M. Brescia Doc. : MLPBP_UserManual_DAME-MAN-NA-0011-Rel1.3 1 INDEX 1 Introduction...
More information6. Backpropagation training 6.1 Background
6. Backpropagation training 6.1 Background To understand well how a feedforward neural network is built and it functions, we consider its basic first steps. We return to its history for a while. In 1949
More informationImage Compression: An Artificial Neural Network Approach
Image Compression: An Artificial Neural Network Approach Anjana B 1, Mrs Shreeja R 2 1 Department of Computer Science and Engineering, Calicut University, Kuttippuram 2 Department of Computer Science and
More informationUnsupervised Learning
Unsupervised Learning Learning without a teacher No targets for the outputs Networks which discover patterns, correlations, etc. in the input data This is a self organisation Self organising networks An
More informationDEEP LEARNING REVIEW. Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature Presented by Divya Chitimalla
DEEP LEARNING REVIEW Yann LeCun, Yoshua Bengio & Geoffrey Hinton Nature 2015 -Presented by Divya Chitimalla What is deep learning Deep learning allows computational models that are composed of multiple
More informationAlex Waibel
Alex Waibel 815.11.2011 1 16.11.2011 Organisation Literatur: Introduction to The Theory of Neural Computation Hertz, Krogh, Palmer, Santa Fe Institute Neural Network Architectures An Introduction, Judith
More informationNotes on Multilayer, Feedforward Neural Networks
Notes on Multilayer, Feedforward Neural Networks CS425/528: Machine Learning Fall 2012 Prepared by: Lynne E. Parker [Material in these notes was gleaned from various sources, including E. Alpaydin s book
More informationReview on Methods of Selecting Number of Hidden Nodes in Artificial Neural Network
Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 11, November 2014,
More informationWeek 3: Perceptron and Multi-layer Perceptron
Week 3: Perceptron and Multi-layer Perceptron Phong Le, Willem Zuidema November 12, 2013 Last week we studied two famous biological neuron models, Fitzhugh-Nagumo model and Izhikevich model. This week,
More informationArtificial Neural Networks Unsupervised learning: SOM
Artificial Neural Networks Unsupervised learning: SOM 01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011 01101011 01110101 01110000 01101001 01101110 01100001
More informationA SourceForge.net Project: tmans, an Agentbased Neural Network Simulator, Repast, and SourceForge CVS
A SourceForge.net Project: tmans, an Agentbased Neural Network Simulator, Repast, and SourceForge CVS John Korecki Computer Science & Engineering REU University of Notre Dame Fall 04 - Spring 05 September
More informationMachine Learning 13. week
Machine Learning 13. week Deep Learning Convolutional Neural Network Recurrent Neural Network 1 Why Deep Learning is so Popular? 1. Increase in the amount of data Thanks to the Internet, huge amount of
More informationNEURAL NETWORKS. Typeset by FoilTEX 1
NEURAL NETWORKS Typeset by FoilTEX 1 Basic Concepts The McCulloch-Pitts model Hebb s rule Neural network: double dynamics. Pattern Formation and Pattern Recognition Neural network as an input-output device
More informationChapter 5 Neural Network Concepts and Paradigms
Chapter 5 Neural Network Concepts and Paradigms Chapter 5 Outline History Key elements and terminology Topologies Adaptation methods Recall dynamics Comparisons of neural and non-neural methods History
More informationNeural Network Approach for Automatic Landuse Classification of Satellite Images: One-Against-Rest and Multi-Class Classifiers
Neural Network Approach for Automatic Landuse Classification of Satellite Images: One-Against-Rest and Multi-Class Classifiers Anil Kumar Goswami DTRL, DRDO Delhi, India Heena Joshi Banasthali Vidhyapith
More informationArtificial Neural Networks Lecture Notes Part 5. Stephen Lucci, PhD. Part 5
Artificial Neural Networks Lecture Notes Part 5 About this file: If you have trouble reading the contents of this file, or in case of transcription errors, email gi0062@bcmail.brooklyn.cuny.edu Acknowledgments:
More informationLecture 20: Neural Networks for NLP. Zubin Pahuja
Lecture 20: Neural Networks for NLP Zubin Pahuja zpahuja2@illinois.edu courses.engr.illinois.edu/cs447 CS447: Natural Language Processing 1 Today s Lecture Feed-forward neural networks as classifiers simple
More informationBiologically inspired object categorization in cluttered scenes
Rochester Institute of Technology RIT Scholar Works Theses Thesis/Dissertation Collections 2008 Biologically inspired object categorization in cluttered scenes Theparit Peerasathien Follow this and additional
More informationArtificial Neural Networks MLP, RBF & GMDH
Artificial Neural Networks MLP, RBF & GMDH Jan Drchal drchajan@fel.cvut.cz Computational Intelligence Group Department of Computer Science and Engineering Faculty of Electrical Engineering Czech Technical
More informationNeural Networks (pp )
Notation: Means pencil-and-paper QUIZ Means coding QUIZ Neural Networks (pp. 106-121) The first artificial neural network (ANN) was the (single-layer) perceptron, a simplified model of a biological neuron.
More informationNeuro-Fuzzy Computing
CSE53 Neuro-Fuzzy Computing Tutorial/Assignment 3: Unsupervised Learning About this tutorial The objective of this tutorial is to study unsupervised learning, in particular: (Generalized) Hebbian learning.
More informationSeismic regionalization based on an artificial neural network
Seismic regionalization based on an artificial neural network *Jaime García-Pérez 1) and René Riaño 2) 1), 2) Instituto de Ingeniería, UNAM, CU, Coyoacán, México D.F., 014510, Mexico 1) jgap@pumas.ii.unam.mx
More informationCHAPTER 7 MASS LOSS PREDICTION USING ARTIFICIAL NEURAL NETWORK (ANN)
128 CHAPTER 7 MASS LOSS PREDICTION USING ARTIFICIAL NEURAL NETWORK (ANN) Various mathematical techniques like regression analysis and software tools have helped to develop a model using equation, which
More information464 Index. Associative memory paradigms
Index ABC analysis 5 Adaline 27 Adaptive resonance theory (ART) 56-9, 300, 309 architecture of 58 ART3 59 ARTMAP 63 bottom up weights 59 components of 57 fuzzy ART 63 gain control 58 learning algorithm
More informationIntroduction AL Neuronale Netzwerke. VL Algorithmisches Lernen, Teil 2b. Norman Hendrich
Introduction AL 64-360 Neuronale Netzwerke VL Algorithmisches Lernen, Teil 2b Norman Hendrich University of Hamburg, Dept. of Informatics Vogt-Kölln-Str. 30, D-22527 Hamburg hendrich@informatik.uni-hamburg.de
More informationMachine Learning in Biology
Università degli studi di Padova Machine Learning in Biology Luca Silvestrin (Dottorando, XXIII ciclo) Supervised learning Contents Class-conditional probability density Linear and quadratic discriminant
More informationClassification Lecture Notes cse352. Neural Networks. Professor Anita Wasilewska
Classification Lecture Notes cse352 Neural Networks Professor Anita Wasilewska Neural Networks Classification Introduction INPUT: classification data, i.e. it contains an classification (class) attribute
More informationMachine Learning Classifiers and Boosting
Machine Learning Classifiers and Boosting Reading Ch 18.6-18.12, 20.1-20.3.2 Outline Different types of learning problems Different types of learning algorithms Supervised learning Decision trees Naïve
More informationFuture Image Prediction using Artificial Neural Networks
Future Image Prediction using Artificial Neural Networks Abhishek Kar (Y8021) Dept. of Computer Science and Engineering, IIT Kanpur Abstract In this work we present an Artificial Neural Network approach
More informationNeural Nets. General Model Building
Neural Nets To give you an idea of how new this material is, let s do a little history lesson. The origins of neural nets are typically dated back to the early 1940 s and work by two physiologists, McCulloch
More informationII. ARTIFICIAL NEURAL NETWORK
Applications of Artificial Neural Networks in Power Systems: A Review Harsh Sareen 1, Palak Grover 2 1, 2 HMR Institute of Technology and Management Hamidpur New Delhi, India Abstract: A standout amongst
More informationFigure (5) Kohonen Self-Organized Map
2- KOHONEN SELF-ORGANIZING MAPS (SOM) - The self-organizing neural networks assume a topological structure among the cluster units. - There are m cluster units, arranged in a one- or two-dimensional array;
More informationNeural Nets for Adaptive Filter and Adaptive Pattern Recognition
Adaptive Pattern btyoung@gmail.com CSCE 636 10 February 2010 Outline Adaptive Combiners and Filters Minimal Disturbance and the Algorithm Madaline Rule II () Published 1988 in IEEE Journals Bernard Widrow
More informationSimulation of Zhang Suen Algorithm using Feed- Forward Neural Networks
Simulation of Zhang Suen Algorithm using Feed- Forward Neural Networks Ritika Luthra Research Scholar Chandigarh University Gulshan Goyal Associate Professor Chandigarh University ABSTRACT Image Skeletonization
More informationCP365 Artificial Intelligence
CP365 Artificial Intelligence Tech News! Apple news conference tomorrow? Tech News! Apple news conference tomorrow? Google cancels Project Ara modular phone Weather-Based Stock Market Predictions? Dataset
More informationA Dendrogram. Bioinformatics (Lec 17)
A Dendrogram 3/15/05 1 Hierarchical Clustering [Johnson, SC, 1967] Given n points in R d, compute the distance between every pair of points While (not done) Pick closest pair of points s i and s j and
More informationNeural network based Numerical digits Recognization using NNT in Matlab
Neural network based Numerical digits Recognization using NNT in Matlab ABSTRACT Amritpal kaur 1, Madhavi Arora 2 M.tech- ECE 1, Assistant Professor 2 Global institute of engineering and technology, Amritsar
More informationDeep (1) Matthieu Cord LIP6 / UPMC Paris 6
Deep (1) Matthieu Cord LIP6 / UPMC Paris 6 Syllabus 1. Whole traditional (old) visual recognition pipeline 2. Introduction to Neural Nets 3. Deep Nets for image classification To do : Voir la leçon inaugurale
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationChapter 7: Competitive learning, clustering, and self-organizing maps
Chapter 7: Competitive learning, clustering, and self-organizing maps António R. C. Paiva EEL 6814 Spring 2008 Outline Competitive learning Clustering Self-Organizing Maps What is competition in neural
More informationSlide07 Haykin Chapter 9: Self-Organizing Maps
Slide07 Haykin Chapter 9: Self-Organizing Maps CPSC 636-600 Instructor: Yoonsuck Choe Spring 2012 Introduction Self-organizing maps (SOM) is based on competitive learning, where output neurons compete
More informationCOMPUTATIONAL INTELLIGENCE
COMPUTATIONAL INTELLIGENCE Radial Basis Function Networks Adrian Horzyk Preface Radial Basis Function Networks (RBFN) are a kind of artificial neural networks that use radial basis functions (RBF) as activation
More informationA *69>H>N6 #DJGC6A DG C<>C::G>C<,8>:C8:H /DA 'D 2:6G, ()-"&"3 -"(' ( +-" " " % '.+ % ' -0(+$,
The structure is a very important aspect in neural network design, it is not only impossible to determine an optimal structure for a given problem, it is even impossible to prove that a given structure
More informationReservoir Computing for Neural Networks
Reservoir Computing for Neural Networks Felix Grezes CUNY Graduate Center fgrezes@gc.cuny.edu September 4, 2014 Felix Grezes (CUNY) Reservoir Computing September 4, 2014 1 / 33 Introduction The artificial
More informationRIMT IET, Mandi Gobindgarh Abstract - In this paper, analysis the speed of sending message in Healthcare standard 7 with the use of back
Global Journal of Computer Science and Technology Neural & Artificial Intelligence Volume 13 Issue 3 Version 1.0 Year 2013 Type: Double Blind Peer Reviewed International Research Journal Publisher: Global
More informationNeural Network Neurons
Neural Networks Neural Network Neurons 1 Receives n inputs (plus a bias term) Multiplies each input by its weight Applies activation function to the sum of results Outputs result Activation Functions Given
More informationCHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK IN GAIT RECOGNITION
75 CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK IN GAIT RECOGNITION 6.1 INTRODUCTION Counter propagation network (CPN) was developed by Robert Hecht-Nielsen as a means to combine an unsupervised Kohonen
More informationAssignment 2. Classification and Regression using Linear Networks, Multilayer Perceptron Networks, and Radial Basis Functions
ENEE 739Q: STATISTICAL AND NEURAL PATTERN RECOGNITION Spring 2002 Assignment 2 Classification and Regression using Linear Networks, Multilayer Perceptron Networks, and Radial Basis Functions Aravind Sundaresan
More informationEnsemble methods in machine learning. Example. Neural networks. Neural networks
Ensemble methods in machine learning Bootstrap aggregating (bagging) train an ensemble of models based on randomly resampled versions of the training set, then take a majority vote Example What if you
More informationA novel firing rule for training Kohonen selforganising
A novel firing rule for training Kohonen selforganising maps D. T. Pham & A. B. Chan Manufacturing Engineering Centre, School of Engineering, University of Wales Cardiff, P.O. Box 688, Queen's Buildings,
More informationRough Set Approach to Unsupervised Neural Network based Pattern Classifier
Rough Set Approach to Unsupervised Neural based Pattern Classifier Ashwin Kothari, Member IAENG, Avinash Keskar, Shreesha Srinath, and Rakesh Chalsani Abstract Early Convergence, input feature space with
More informationLinear Separability. Linear Separability. Capabilities of Threshold Neurons. Capabilities of Threshold Neurons. Capabilities of Threshold Neurons
Linear Separability Input space in the two-dimensional case (n = ): - - - - - - w =, w =, = - - - - - - w = -, w =, = - - - - - - w = -, w =, = Linear Separability So by varying the weights and the threshold,
More informationAn Algorithm For Training Multilayer Perceptron (MLP) For Image Reconstruction Using Neural Network Without Overfitting.
An Algorithm For Training Multilayer Perceptron (MLP) For Image Reconstruction Using Neural Network Without Overfitting. Mohammad Mahmudul Alam Mia, Shovasis Kumar Biswas, Monalisa Chowdhury Urmi, Abubakar
More informationNeuron Selectivity as a Biologically Plausible Alternative to Backpropagation
Neuron Selectivity as a Biologically Plausible Alternative to Backpropagation C.J. Norsigian Department of Bioengineering cnorsigi@eng.ucsd.edu Vishwajith Ramesh Department of Bioengineering vramesh@eng.ucsd.edu
More informationKhmer Character Recognition using Artificial Neural Network
Khmer Character Recognition using Artificial Neural Network Hann Meng * and Daniel Morariu * Faculty of Engineering, Lucian Blaga University of Sibiu, Sibiu, Romania E-mail: meng.hann@rupp.edu.kh Tel:
More informationIMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS
IMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS BOGDAN M.WILAMOWSKI University of Wyoming RICHARD C. JAEGER Auburn University ABSTRACT: It is shown that by introducing special
More informationDeep Learning. Practical introduction with Keras JORDI TORRES 27/05/2018. Chapter 3 JORDI TORRES
Deep Learning Practical introduction with Keras Chapter 3 27/05/2018 Neuron A neural network is formed by neurons connected to each other; in turn, each connection of one neural network is associated
More informationUnsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
More informationARTIFICIAL NEURAL NETWORK CIRCUIT FOR SPECTRAL PATTERN RECOGNITION
ARTIFICIAL NEURAL NETWORK CIRCUIT FOR SPECTRAL PATTERN RECOGNITION A Thesis by FARAH RASHEED Submitted to the Office of Graduate and Professional Studies of Texas A&M University in partial fulfillment
More informationNatural Language Processing CS 6320 Lecture 6 Neural Language Models. Instructor: Sanda Harabagiu
Natural Language Processing CS 6320 Lecture 6 Neural Language Models Instructor: Sanda Harabagiu In this lecture We shall cover: Deep Neural Models for Natural Language Processing Introduce Feed Forward
More informationCryptography Algorithms using Artificial Neural Network Arijit Ghosh 1 Department of Computer Science St. Xavier s College (Autonomous) Kolkata India
Volume 2, Issue 11, November 2014 International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case Study Available online at: www.ijarcsms.com
More informationNeural networks for variable star classification
Neural networks for variable star classification Vasily Belokurov, IoA, Cambridge Supervised classification Multi-Layer Perceptron (MLP) Neural Networks for Pattern Recognition by C. Bishop Unsupervised
More informationMachine Learning : Clustering, Self-Organizing Maps
Machine Learning Clustering, Self-Organizing Maps 12/12/2013 Machine Learning : Clustering, Self-Organizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The
More information