Data Mining. Kohonen Networks. Data Mining Course: Sharif University of Technology 1

Size: px
Start display at page:

Download "Data Mining. Kohonen Networks. Data Mining Course: Sharif University of Technology 1"

Transcription

1 Data Mining Kohonen Networks Data Mining Course: Sharif University of Technology 1

2 Self-Organizing Maps Kohonen Networks developed in 198 by Tuevo Kohonen Initially applied to image and sound analysis Represent Self-Organizing Map (SOM) Special class of Neural Networks SOM Convert high-dimensional input to low-dimensional, discrete map Applicable to cluster analysis Structure output nodes to clusters of nodes Nodes in close proximity more similar, compared to those farther apart Data Mining Course: Sharif University of Technology

3 Self-Organizing Maps (cont (cont d) Based on Competitive Learning,, where output nodes compete to become winning node (neuron) Nodes become selectively tuned to input patterns during the competitive learning process (Haykin) Example SOM architecture shown with two inputs, Age and Income Output Layer Connections with Weights Input Layer Age Income Data Mining Course: Sharif University of Technology 3

4 Self-Organizing Maps (cont.) Example: colors Data Mining Course: Sharif University of Technology 4

5 Start: End: Data Mining Course: Sharif University of Technology 5

6 Self-Organizing Maps (cont (cont d) Input nodes pass variable values downstream SOMs are Feedforward and Completely Connected Feedforward networks do not allow looping Each node in given layer, completely connected to every node in next layer Every connection between two nodes has weight Weight values initialized randomly 0 1 Adjusting weights key feature of learning process Attribute values are normalized or standardized SOMs do not have hidden layer Data passed directly from input layer to output layer Data Mining Course: Sharif University of Technology 6

7 Self-Organizing Maps (cont (cont d) SOM Process Example Input record has Age = 0.69 and Income = 0.88 Attribute values for Age and Income enter through respective input nodes Values passed to all output nodes These values, together with connection weights, determine value of Scoring Function for each output node Output node with best score designated Winning Node for record Data Mining Course: Sharif University of Technology 7

8 Self-Organizing Maps (cont (cont d) SOMs exhibit three characteristics Competition Output nodes compete with one another for best score Euclidean Distance function commonly used Winning node produces smallest distance between inputs and connection weights Cooperation Winning node becomes center of neighborhood Output nodes in neighborhood share excitement or reward Data Mining Course: Sharif University of Technology 8

9 Self-Organizing Maps (cont (cont d) Emulates behavior of biological neurons, which are sensitive to output of neighbors Nodes in output layer not directly connected However, share common features because of neighborhood behavior Adaptation Neighborhood nodes participate in adaptation (learning) Weights adjusted to improve score function For subsequent iterations, increases likelihood of winning records with similar values Data Mining Course: Sharif University of Technology 9

10 Kohonen Networks Kohonen Networks are SOMs exhibiting Kohonen Learning Nodes in neighborhood of winning node adjust their weights Adjustment is linear combination of input vector and current weight vector: w x ij, NEW n w j = x n1 = w = w, x 1 j n, w η,0 < η < 1 ij, CURRENT,..., x j nm,..., w + η( x mj ni w ij, CURRENT ), where m field values for nth record current set of m weights, for particular output node learning rate j Learning rate should be decreasing function of training epochs (Kohonen) Linearly or geometrically decreasing learning rate sufficient Data Mining Course: Sharif University of Technology 10

11 Kohonen Networks (cont (cont d) Kohonen Networks Algorithm (Fausett) START ALGORITHM: Initialize Assign random values to weights Initial learning rate and neighborhood size values assigned LOOP: For each input vector x, do: Competition For each output node j, calculate scoring function D(wj,, xn) x Euclidean Distance = D( w ) = Find winning node J, that minimizes D(wj,, xn) x j, x n x Data Mining Course: Sharif University of Technology 11 i ( w ij ni )

12 Kohonen Networks (cont (cont d) Cooperation Identify output nodes j, within neighborhood of J defined by neighborhood size R Adaptation Adjust weights of all neighborhood nodes j: w ij, NEW = wij, CURRENT + η( xni wij, CURRENT ) Adjust learning rate and neighborhood size (decreasing), as needed Nodes not attracting sufficient number of hits may be pruned Stop when termination criteria met END ALGORITHM: Data Mining Course: Sharif University of Technology 1

13 Example of a Kohonen Network Study Use simple x Kohonen Network Neighborhood Size = 0, Learning Rate = 0.5 Input data consists of four records, with attributes Age and Income (values normalized) Records with attribute values: 1 x11 = 0.8 x1 = 0.8 Older person with high income x1 = 0.8 x = 0.1 Older person with low income 3 x31 = 0. x3 = 0.9 Younger person with high income 4 x41 = 0.1 x4 = 0.1 Younger person with low income Initial network weights (randomly assigned): w11 = 0.9 w1 = 0.8 w1 = 0.9 w = 0. w13 = 0.1 w3 = 0.8 w14 = 0.1 w4 = 0. Data Mining Course: Sharif University of Technology 13

14 Example of a Kohonen Network Study (cont d) Study Figure shows network topology for example Input layer = nodes, and output layer = 4 nodes Node 1 Node w11 w1 w13 Output Layer Node 3 w3 Node 4 w1 w4 w14 w Input Layer Age Income Data Mining Course: Sharif University of Technology 14 14

15 Example of a Kohonen Network Study (cont d) Study First Record x1 x = (0.8, 0.8) Competition Phase Compute Euclidean Distance between input and weight vectors Node1: D( w, x 1 1 ) = ( ) + ( ) = 0.10 Node : D( w, x 1 ) = ( ) + (0. 0.8) = 0.61 Node3: D( w 3, x 1 ) = ( ) + ( ) = 0.70 Node 4 : D( w 4, x 1 ) = ( ) + (0. 0.8) = 0.9 The winning node is Node 1 (minimizes distance = 0.10) Note, node 1 weights most similar to input record values Node 1 may exhibit affinity (cluster) for records of older persons with high income Data Mining Course: Sharif University of Technology 15

16 Example of a Kohonen Network Study (cont d) Study First Record x1 x = (0.8, 0.8) Cooperation Phase Neighborhood Size R = 0 Therefore, nonexistent excitement of neighboring nodes Only winning node receives weight adjustment Adaptation Phase Weights for Node 1 adjusted, where j = 1 (Node 1), n = 1 (First record), and learning rate = 0.5: Age : Income : = w 11, NEW + 0.5( x = ( ) = 0.85 = w 1, NEW = w = w 11, CURRENT 1, CURRENT + 0.5( x = ( ) = w w 11, CURRENT ) 1, CURRENT ) Data Mining Course: Sharif University of Technology 16

17 Example of a Kohonen Network Study (cont d) Study Note direction of weight adjustments Weights move toward input field values Initial weight w11w 11 = 0.9, adjusted in direction of x11x 11 = 0.8 With learning rate = 0.5, w11w 11 moved half the distance from 0.9 to 0.8 Therefore, w11w 11 updated to 0.85 Node 1 becomes more proficient at capturing records of older, higher income persons Data Mining Course: Sharif University of Technology 17

18 Example of a Kohonen Network Study (cont d) Study Second Record x x = (0.8, 0.1) Competition Compute Euclidean Distance between input and weight vectors Node1: Node : Node3: Node 4: D( w, x 1 D( w D( w 3 D( w 4, x, x, x ) = ) = ) = ) = ( ) ( ) ( ) ( ) + ( ) + (0. 0.1) + ( ) + (0. 0.1) = 0.78 = 0.14 = 0.99 = 0.71 Node is the winning node with distance = 0.14 Node weights (0.9, 0.) most similar to input record values (0.8, 0.1) Records of older persons and low income may cluster to Node Data Mining Course: Sharif University of Technology 18

19 Example of a Kohonen Network Study (cont d) Study Adaptation Weights for Node adjusted, where j = (Node ), n = (Second record), and learning rate = 0.5: Age : Income: = w 1, NEW + 0.5( x = ( ) = 0.85 = w, NEW = w = w 1, CURRENT, CURRENT + 0.5( x = (0.1 0.) = 0.15 Again, weights move towards input field values Initial w1w = 0.9, adjusted to 0.85 (direction of x1x = 0.8) Initial ww = 0., adjusted to 0.15 (direction of xx = 0.1) Node develops affinity for records of older, lower income persons 1 w w 1, CURRENT Data Mining Course: Sharif University of Technology 19 ), CURRENT )

20 Example of a Kohonen Network Study (cont d) Study Similarly, records x3 x = (0., 0.9) and x4 x = (0.1, 0.1) are processed by network Summary Four output nodes represent distinct clusters Simple example demonstrates network algorithm Shows basic competition and Kohonen learning Cluster Associated With Description 1 Node 1 Older person with high income Node Older person with low income 3 Node 3 Younger person with high income 4 Node 4 Younger person with low income Data Mining Course: Sharif University of Technology 0

Chapter 7: Competitive learning, clustering, and self-organizing maps

Chapter 7: Competitive learning, clustering, and self-organizing maps Chapter 7: Competitive learning, clustering, and self-organizing maps António R. C. Paiva EEL 6814 Spring 2008 Outline Competitive learning Clustering Self-Organizing Maps What is competition in neural

More information

Figure (5) Kohonen Self-Organized Map

Figure (5) Kohonen Self-Organized Map 2- KOHONEN SELF-ORGANIZING MAPS (SOM) - The self-organizing neural networks assume a topological structure among the cluster units. - There are m cluster units, arranged in a one- or two-dimensional array;

More information

11/14/2010 Intelligent Systems and Soft Computing 1

11/14/2010 Intelligent Systems and Soft Computing 1 Lecture 8 Artificial neural networks: Unsupervised learning Introduction Hebbian learning Generalised Hebbian learning algorithm Competitive learning Self-organising computational map: Kohonen network

More information

Function approximation using RBF network. 10 basis functions and 25 data points.

Function approximation using RBF network. 10 basis functions and 25 data points. 1 Function approximation using RBF network F (x j ) = m 1 w i ϕ( x j t i ) i=1 j = 1... N, m 1 = 10, N = 25 10 basis functions and 25 data points. Basis function centers are plotted with circles and data

More information

Slide07 Haykin Chapter 9: Self-Organizing Maps

Slide07 Haykin Chapter 9: Self-Organizing Maps Slide07 Haykin Chapter 9: Self-Organizing Maps CPSC 636-600 Instructor: Yoonsuck Choe Spring 2012 Introduction Self-organizing maps (SOM) is based on competitive learning, where output neurons compete

More information

Yuki Osada Andrew Cannon

Yuki Osada Andrew Cannon Yuki Osada Andrew Cannon 1 Humans are an intelligent species One feature is the ability to learn The ability to learn comes down to the brain The brain learns from experience Research shows that the brain

More information

Review: Final Exam CPSC Artificial Intelligence Michael M. Richter

Review: Final Exam CPSC Artificial Intelligence Michael M. Richter Review: Final Exam Model for a Learning Step Learner initially Environm ent Teacher Compare s pe c ia l Information Control Correct Learning criteria Feedback changed Learner after Learning Learning by

More information

Unsupervised Learning

Unsupervised Learning Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised

More information

Self-Organizing Maps for cyclic and unbounded graphs

Self-Organizing Maps for cyclic and unbounded graphs Self-Organizing Maps for cyclic and unbounded graphs M. Hagenbuchner 1, A. Sperduti 2, A.C. Tsoi 3 1- University of Wollongong, Wollongong, Australia. 2- University of Padova, Padova, Italy. 3- Hong Kong

More information

Neural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani

Neural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer

More information

Giri Narasimhan. CAP 5510: Introduction to Bioinformatics. ECS 254; Phone: x3748

Giri Narasimhan. CAP 5510: Introduction to Bioinformatics. ECS 254; Phone: x3748 CAP 5510: Introduction to Bioinformatics Giri Narasimhan ECS 254; Phone: x3748 giri@cis.fiu.edu www.cis.fiu.edu/~giri/teach/bioinfs07.html 3/3/08 CAP5510 1 Gene g Probe 1 Probe 2 Probe N 3/3/08 CAP5510

More information

Preprocessing DWML, /33

Preprocessing DWML, /33 Preprocessing DWML, 2007 1/33 Preprocessing Before you can start on the actual data mining, the data may require some preprocessing: Attributes may be redundant. Values may be missing. The data contains

More information

PATTERN RECOGNITION USING NEURAL NETWORKS

PATTERN RECOGNITION USING NEURAL NETWORKS PATTERN RECOGNITION USING NEURAL NETWORKS Santaji Ghorpade 1, Jayshree Ghorpade 2 and Shamla Mantri 3 1 Department of Information Technology Engineering, Pune University, India santaji_11jan@yahoo.co.in,

More information

COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS

COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS Toomas Kirt Supervisor: Leo Võhandu Tallinn Technical University Toomas.Kirt@mail.ee Abstract: Key words: For the visualisation

More information

What is a receptive field? Why a sensory neuron has such particular RF How a RF was developed?

What is a receptive field? Why a sensory neuron has such particular RF How a RF was developed? What is a receptive field? Why a sensory neuron has such particular RF How a RF was developed? x 1 x 2 x 3 y f w 1 w 2 w 3 T x y = f (wx i i T ) i y x 1 x 2 x 3 = = E (y y) (y f( wx T)) 2 2 o o i i i

More information

Data Mining Chapter 9: Descriptive Modeling Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University

Data Mining Chapter 9: Descriptive Modeling Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Data Mining Chapter 9: Descriptive Modeling Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Descriptive model A descriptive model presents the main features of the data

More information

Cluster analysis of 3D seismic data for oil and gas exploration

Cluster analysis of 3D seismic data for oil and gas exploration Data Mining VII: Data, Text and Web Mining and their Business Applications 63 Cluster analysis of 3D seismic data for oil and gas exploration D. R. S. Moraes, R. P. Espíndola, A. G. Evsukoff & N. F. F.

More information

Centroid Neural Network based clustering technique using competetive learning

Centroid Neural Network based clustering technique using competetive learning Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics San Antonio, TX, USA - October 2009 Centroid Neural Network based clustering technique using competetive learning

More information

A Dendrogram. Bioinformatics (Lec 17)

A Dendrogram. Bioinformatics (Lec 17) A Dendrogram 3/15/05 1 Hierarchical Clustering [Johnson, SC, 1967] Given n points in R d, compute the distance between every pair of points While (not done) Pick closest pair of points s i and s j and

More information

A Population Based Convergence Criterion for Self-Organizing Maps

A Population Based Convergence Criterion for Self-Organizing Maps A Population Based Convergence Criterion for Self-Organizing Maps Lutz Hamel and Benjamin Ott Department of Computer Science and Statistics, University of Rhode Island, Kingston, RI 02881, USA. Email:

More information

Data Warehousing and Machine Learning

Data Warehousing and Machine Learning Data Warehousing and Machine Learning Preprocessing Thomas D. Nielsen Aalborg University Department of Computer Science Spring 2008 DWML Spring 2008 1 / 35 Preprocessing Before you can start on the actual

More information

SOMSN: An Effective Self Organizing Map for Clustering of Social Networks

SOMSN: An Effective Self Organizing Map for Clustering of Social Networks SOMSN: An Effective Self Organizing Map for Clustering of Social Networks Fatemeh Ghaemmaghami Research Scholar, CSE and IT Dept. Shiraz University, Shiraz, Iran Reza Manouchehri Sarhadi Research Scholar,

More information

Two-step Modified SOM for Parallel Calculation

Two-step Modified SOM for Parallel Calculation Two-step Modified SOM for Parallel Calculation Two-step Modified SOM for Parallel Calculation Petr Gajdoš and Pavel Moravec Petr Gajdoš and Pavel Moravec Department of Computer Science, FEECS, VŠB Technical

More information

CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK IN GAIT RECOGNITION

CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK IN GAIT RECOGNITION 75 CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK IN GAIT RECOGNITION 6.1 INTRODUCTION Counter propagation network (CPN) was developed by Robert Hecht-Nielsen as a means to combine an unsupervised Kohonen

More information

5.6 Self-organizing maps (SOM) [Book, Sect. 10.3]

5.6 Self-organizing maps (SOM) [Book, Sect. 10.3] Ch.5 Classification and Clustering 5.6 Self-organizing maps (SOM) [Book, Sect. 10.3] The self-organizing map (SOM) method, introduced by Kohonen (1982, 2001), approximates a dataset in multidimensional

More information

Unsupervised learning

Unsupervised learning Unsupervised learning Enrique Muñoz Ballester Dipartimento di Informatica via Bramante 65, 26013 Crema (CR), Italy enrique.munoz@unimi.it Enrique Muñoz Ballester 2017 1 Download slides data and scripts:

More information

GTSOM: GAME THEORETIC SELF- ORGANIZING MAPS

GTSOM: GAME THEORETIC SELF- ORGANIZING MAPS Chapter 8 GTSOM: GAME THEORETIC SELF- ORGANIZING MAPS Joseph Herbert, JingTao Yao Department of Computer Science University of Regina, Saskatchewan, Canada, S4S 0A2 [herbertj,jtyao]@cs.uregina.ca Abstract

More information

Supervised vs.unsupervised Learning

Supervised vs.unsupervised Learning Supervised vs.unsupervised Learning In supervised learning we train algorithms with predefined concepts and functions based on labeled data D = { ( x, y ) x X, y {yes,no}. In unsupervised learning we are

More information

Mineral Exploation Using Neural Netowrks

Mineral Exploation Using Neural Netowrks ABSTRACT I S S N 2277-3061 Mineral Exploation Using Neural Netowrks Aysar A. Abdulrahman University of Sulaimani, Computer Science, Kurdistan Region of Iraq aysser.abdulrahman@univsul.edu.iq Establishing

More information

Multi-Clustering Centers Approach to Enhancing the Performance of SOM Clustering Ability

Multi-Clustering Centers Approach to Enhancing the Performance of SOM Clustering Ability JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 25, 1087-1102 (2009) Multi-Clustering Centers Approach to Enhancing the Performance of SOM Clustering Ability CHING-HWANG WANG AND CHIH-HAN KAO * Department

More information

A Class of Instantaneously Trained Neural Networks

A Class of Instantaneously Trained Neural Networks A Class of Instantaneously Trained Neural Networks Subhash Kak Department of Electrical & Computer Engineering, Louisiana State University, Baton Rouge, LA 70803-5901 May 7, 2002 Abstract This paper presents

More information

USING OF THE K NEAREST NEIGHBOURS ALGORITHM (k-nns) IN THE DATA CLASSIFICATION

USING OF THE K NEAREST NEIGHBOURS ALGORITHM (k-nns) IN THE DATA CLASSIFICATION USING OF THE K NEAREST NEIGHBOURS ALGORITHM (k-nns) IN THE DATA CLASSIFICATION Gîlcă Natalia, Roșia de Amaradia Technological High School, Gorj, ROMANIA Gîlcă Gheorghe, Constantin Brîncuși University from

More information

The k-means Algorithm and Genetic Algorithm

The k-means Algorithm and Genetic Algorithm The k-means Algorithm and Genetic Algorithm k-means algorithm Genetic algorithm Rough set approach Fuzzy set approaches Chapter 8 2 The K-Means Algorithm The K-Means algorithm is a simple yet effective

More information

Neuro-Fuzzy Comp. Ch. 8 May 12, 2005

Neuro-Fuzzy Comp. Ch. 8 May 12, 2005 Neuro-Fuzzy Comp. Ch. 8 May, 8 Self-Organizing Feature Maps Self-Organizing Feature Maps (SOFM or SOM) also known as Kohonen maps or topographic maps were first introduced by von der Malsburg (97) and

More information

Data Mining. Moustafa ElBadry. A thesis submitted in fulfillment of the requirements for the degree of Bachelor of Arts in Mathematics

Data Mining. Moustafa ElBadry. A thesis submitted in fulfillment of the requirements for the degree of Bachelor of Arts in Mathematics Data Mining Moustafa ElBadry A thesis submitted in fulfillment of the requirements for the degree of Bachelor of Arts in Mathematics Department of Mathematics and Computer Science Whitman College 2016

More information

Artificial neural networks are the paradigm of connectionist systems (connectionism vs. symbolism)

Artificial neural networks are the paradigm of connectionist systems (connectionism vs. symbolism) Artificial Neural Networks Analogy to biological neural systems, the most robust learning systems we know. Attempt to: Understand natural biological systems through computational modeling. Model intelligent

More information

Non-linear dimension reduction

Non-linear dimension reduction Sta306b May 23, 2011 Dimension Reduction: 1 Non-linear dimension reduction ISOMAP: Tenenbaum, de Silva & Langford (2000) Local linear embedding: Roweis & Saul (2000) Local MDS: Chen (2006) all three methods

More information

Nonlinear dimensionality reduction of large datasets for data exploration

Nonlinear dimensionality reduction of large datasets for data exploration Data Mining VII: Data, Text and Web Mining and their Business Applications 3 Nonlinear dimensionality reduction of large datasets for data exploration V. Tomenko & V. Popov Wessex Institute of Technology,

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Learning without a teacher No targets for the outputs Networks which discover patterns, correlations, etc. in the input data This is a self organisation Self organising networks An

More information

Building Adaptive Basis Functions with a Continuous Self-Organizing Map

Building Adaptive Basis Functions with a Continuous Self-Organizing Map Neural Processing Letters 11: 59 78, 2000. 2000 Kluwer Academic Publishers. Printed in the Netherlands. 59 Building Adaptive Basis Functions with a Continuous Self-Organizing Map MARCOS M. CAMPOS and GAIL

More information

Growing Neural Gas A Parallel Approach

Growing Neural Gas A Parallel Approach Growing Neural Gas A Parallel Approach Lukáš Vojáček 1 and JiříDvorský 2 1 IT4Innovations Centre of Excellence Ostrava, Czech Republic lukas.vojacek@vsb.cz 2 Department of Computer Science, VŠB Technical

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Unsupervised learning Until now, we have assumed our training samples are labeled by their category membership. Methods that use labeled samples are said to be supervised. However,

More information

Methods for Intelligent Systems

Methods for Intelligent Systems Methods for Intelligent Systems Lecture Notes on Clustering (II) Davide Eynard eynard@elet.polimi.it Department of Electronics and Information Politecnico di Milano Davide Eynard - Lecture Notes on Clustering

More information

A Topography-Preserving Latent Variable Model with Learning Metrics

A Topography-Preserving Latent Variable Model with Learning Metrics A Topography-Preserving Latent Variable Model with Learning Metrics Samuel Kaski and Janne Sinkkonen Helsinki University of Technology Neural Networks Research Centre P.O. Box 5400, FIN-02015 HUT, Finland

More information

CS570: Introduction to Data Mining

CS570: Introduction to Data Mining CS570: Introduction to Data Mining Classification Advanced Reading: Chapter 8 & 9 Han, Chapters 4 & 5 Tan Anca Doloc-Mihu, Ph.D. Slides courtesy of Li Xiong, Ph.D., 2011 Han, Kamber & Pei. Data Mining.

More information

Data Compression. The Encoder and PCA

Data Compression. The Encoder and PCA Data Compression The Encoder and PCA Neural network techniques have been shown useful in the area of data compression. In general, data compression can be lossless compression or lossy compression. In

More information

Data Mining Classification: Alternative Techniques. Lecture Notes for Chapter 4. Instance-Based Learning. Introduction to Data Mining, 2 nd Edition

Data Mining Classification: Alternative Techniques. Lecture Notes for Chapter 4. Instance-Based Learning. Introduction to Data Mining, 2 nd Edition Data Mining Classification: Alternative Techniques Lecture Notes for Chapter 4 Instance-Based Learning Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar Instance Based Classifiers

More information

Neural Network Weight Selection Using Genetic Algorithms

Neural Network Weight Selection Using Genetic Algorithms Neural Network Weight Selection Using Genetic Algorithms David Montana presented by: Carl Fink, Hongyi Chen, Jack Cheng, Xinglong Li, Bruce Lin, Chongjie Zhang April 12, 2005 1 Neural Networks Neural networks

More information

DESIGN OF KOHONEN SELF-ORGANIZING MAP WITH REDUCED STRUCTURE

DESIGN OF KOHONEN SELF-ORGANIZING MAP WITH REDUCED STRUCTURE DESIGN OF KOHONEN SELF-ORGANIZING MAP WITH REDUCED STRUCTURE S. Kajan, M. Lajtman Institute of Control and Industrial Informatics, Faculty of Electrical Engineering and Information Technology, Slovak University

More information

Extract an Essential Skeleton of a Character as a Graph from a Character Image

Extract an Essential Skeleton of a Character as a Graph from a Character Image Extract an Essential Skeleton of a Character as a Graph from a Character Image Kazuhisa Fujita University of Electro-Communications 1-5-1 Chofugaoka, Chofu, Tokyo, 182-8585 Japan k-z@nerve.pc.uec.ac.jp

More information

Exploratory Data Analysis using Self-Organizing Maps. Madhumanti Ray

Exploratory Data Analysis using Self-Organizing Maps. Madhumanti Ray Exploratory Data Analysis using Self-Organizing Maps Madhumanti Ray Content Introduction Data Analysis methods Self-Organizing Maps Conclusion Visualization of high-dimensional data items Exploratory data

More information

Images Reconstruction using an iterative SOM based algorithm.

Images Reconstruction using an iterative SOM based algorithm. Images Reconstruction using an iterative SOM based algorithm. M.Jouini 1, S.Thiria 2 and M.Crépon 3 * 1- LOCEAN, MMSA team, CNAM University, Paris, France 2- LOCEAN, MMSA team, UVSQ University Paris, France

More information

Chapter 6: Cluster Analysis

Chapter 6: Cluster Analysis Chapter 6: Cluster Analysis The major goal of cluster analysis is to separate individual observations, or items, into groups, or clusters, on the basis of the values for the q variables measured on each

More information

Machine Learning : Clustering, Self-Organizing Maps

Machine Learning : Clustering, Self-Organizing Maps Machine Learning Clustering, Self-Organizing Maps 12/12/2013 Machine Learning : Clustering, Self-Organizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The

More information

Data Mining. Neural Networks

Data Mining. Neural Networks Data Mining Neural Networks Goals for this Unit Basic understanding of Neural Networks and how they work Ability to use Neural Networks to solve real problems Understand when neural networks may be most

More information

Associative Cellular Learning Automata and its Applications

Associative Cellular Learning Automata and its Applications Associative Cellular Learning Automata and its Applications Meysam Ahangaran and Nasrin Taghizadeh and Hamid Beigy Department of Computer Engineering, Sharif University of Technology, Tehran, Iran ahangaran@iust.ac.ir,

More information

Controlling the spread of dynamic self-organising maps

Controlling the spread of dynamic self-organising maps Neural Comput & Applic (2004) 13: 168 174 DOI 10.1007/s00521-004-0419-y ORIGINAL ARTICLE L. D. Alahakoon Controlling the spread of dynamic self-organising maps Received: 7 April 2004 / Accepted: 20 April

More information

Simulation of WSN in NetSim Clustering using Self-Organizing Map Neural Network

Simulation of WSN in NetSim Clustering using Self-Organizing Map Neural Network Simulation of WSN in NetSim Clustering using Self-Organizing Map Neural Network Software Recommended: NetSim Standard v11.0, Visual Studio 2015/2017, MATLAB 2016a Project Download Link: https://github.com/netsim-tetcos/wsn_som_optimization_v11.0/archive/master.zip

More information

Graph projection techniques for Self-Organizing Maps

Graph projection techniques for Self-Organizing Maps Graph projection techniques for Self-Organizing Maps Georg Pölzlbauer 1, Andreas Rauber 1, Michael Dittenbach 2 1- Vienna University of Technology - Department of Software Technology Favoritenstr. 9 11

More information

Artificial Neural Networks Unsupervised learning: SOM

Artificial Neural Networks Unsupervised learning: SOM Artificial Neural Networks Unsupervised learning: SOM 01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011 01101011 01110101 01110000 01101001 01101110 01100001

More information

Data visualization, feature reduction and cluster analysis

Data visualization, feature reduction and cluster analysis Data visualization, feature reduction and cluster analysis Ricco RAKOTOMALALA Université Lumière Lyon 2 Tutoriels Tanagra - http://tutoriels-data-mining.blogspot.fr/ 1 Outline 1. SOM Kohonen map Kohonen

More information

CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS

CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of

More information

Self-Organizing Map. presentation by Andreas Töscher. 19. May 2008

Self-Organizing Map. presentation by Andreas Töscher. 19. May 2008 19. May 2008 1 Introduction 2 3 4 5 6 (SOM) aka Kohonen Network introduced by Teuvo Kohonen implements a discrete nonlinear mapping unsupervised learning Structure of a SOM Learning Rule Introduction

More information

Effect of Grouping in Vector Recognition System Based on SOM

Effect of Grouping in Vector Recognition System Based on SOM Effect of Grouping in Vector Recognition System Based on SOM Masayoshi Ohta Graduate School of Science and Engineering Kansai University Osaka, Japan Email: k287636@kansai-u.ac.jp Yuto Kurosaki Department

More information

Advanced visualization techniques for Self-Organizing Maps with graph-based methods

Advanced visualization techniques for Self-Organizing Maps with graph-based methods Advanced visualization techniques for Self-Organizing Maps with graph-based methods Georg Pölzlbauer 1, Andreas Rauber 1, and Michael Dittenbach 2 1 Department of Software Technology Vienna University

More information

Clustering with Reinforcement Learning

Clustering with Reinforcement Learning Clustering with Reinforcement Learning Wesam Barbakh and Colin Fyfe, The University of Paisley, Scotland. email:wesam.barbakh,colin.fyfe@paisley.ac.uk Abstract We show how a previously derived method of

More information

Classification Lecture Notes cse352. Neural Networks. Professor Anita Wasilewska

Classification Lecture Notes cse352. Neural Networks. Professor Anita Wasilewska Classification Lecture Notes cse352 Neural Networks Professor Anita Wasilewska Neural Networks Classification Introduction INPUT: classification data, i.e. it contains an classification (class) attribute

More information

The Hitchhiker s Guide to TensorFlow:

The Hitchhiker s Guide to TensorFlow: The Hitchhiker s Guide to TensorFlow: Beyond Recurrent Neural Networks (sort of) Keith Davis @keithdavisiii iamthevastidledhitchhiker.github.io Topics Kohonen/Self-Organizing Maps LSTMs in TensorFlow GRU

More information

Affine Arithmetic Self Organizing Map

Affine Arithmetic Self Organizing Map Affine Arithmetic Self Organizing Map Tony Bazzi Department of Electrical and Systems Engineering Oakland University Rochester, MI 48309, USA Email: tbazzi [AT] oakland.edu Jasser Jasser Department of

More information

A novel firing rule for training Kohonen selforganising

A novel firing rule for training Kohonen selforganising A novel firing rule for training Kohonen selforganising maps D. T. Pham & A. B. Chan Manufacturing Engineering Centre, School of Engineering, University of Wales Cardiff, P.O. Box 688, Queen's Buildings,

More information

NEURAL NETWORK-BASED SEGMENTATION OF TEXTURES USING GABOR FEATURES

NEURAL NETWORK-BASED SEGMENTATION OF TEXTURES USING GABOR FEATURES NEURAL NETWORK-BASED SEGMENTATION OF TEXTURES USING GABOR FEATURES A. G. Ramakrishnan, S. Kumar Raja, and H. V. Raghu Ram Dept. of Electrical Engg., Indian Institute of Science Bangalore - 560 012, India

More information

Simulation of WSN in NetSim Clustering using Self-Organizing Map Neural Network

Simulation of WSN in NetSim Clustering using Self-Organizing Map Neural Network Simulation of WSN in NetSim Clustering using Self-Organizing Map Neural Network Software Recommended: NetSim Standard v11.1 (32/64bit), Visual Studio 2015/2017, MATLAB (32/64 bit) Project Download Link:

More information

Neural networks for variable star classification

Neural networks for variable star classification Neural networks for variable star classification Vasily Belokurov, IoA, Cambridge Supervised classification Multi-Layer Perceptron (MLP) Neural Networks for Pattern Recognition by C. Bishop Unsupervised

More information

CHAPTER 3 TUMOR DETECTION BASED ON NEURO-FUZZY TECHNIQUE

CHAPTER 3 TUMOR DETECTION BASED ON NEURO-FUZZY TECHNIQUE 32 CHAPTER 3 TUMOR DETECTION BASED ON NEURO-FUZZY TECHNIQUE 3.1 INTRODUCTION In this chapter we present the real time implementation of an artificial neural network based on fuzzy segmentation process

More information

Dr. Qadri Hamarsheh figure, plotsomnd(net) figure, plotsomplanes(net) figure, plotsomhits(net,x) figure, plotsompos(net,x) Example 2: iris_dataset:

Dr. Qadri Hamarsheh figure, plotsomnd(net) figure, plotsomplanes(net) figure, plotsomhits(net,x) figure, plotsompos(net,x) Example 2: iris_dataset: Self-organizing map using matlab Create a Self-Organizing Map Neural Network: selforgmap Syntax: selforgmap (dimensions, coversteps, initneighbor, topologyfcn, distancefcn) takes these arguments: dimensions

More information

Seismic regionalization based on an artificial neural network

Seismic regionalization based on an artificial neural network Seismic regionalization based on an artificial neural network *Jaime García-Pérez 1) and René Riaño 2) 1), 2) Instituto de Ingeniería, UNAM, CU, Coyoacán, México D.F., 014510, Mexico 1) jgap@pumas.ii.unam.mx

More information

Data Mining. Part 2. Data Understanding and Preparation. 2.4 Data Transformation. Spring Instructor: Dr. Masoud Yaghini. Data Transformation

Data Mining. Part 2. Data Understanding and Preparation. 2.4 Data Transformation. Spring Instructor: Dr. Masoud Yaghini. Data Transformation Data Mining Part 2. Data Understanding and Preparation 2.4 Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Normalization Attribute Construction Aggregation Attribute Subset Selection Discretization

More information

Discriminate Analysis

Discriminate Analysis Discriminate Analysis Outline Introduction Linear Discriminant Analysis Examples 1 Introduction What is Discriminant Analysis? Statistical technique to classify objects into mutually exclusive and exhaustive

More information

Supervised Learning in Neural Networks (Part 2)

Supervised Learning in Neural Networks (Part 2) Supervised Learning in Neural Networks (Part 2) Multilayer neural networks (back-propagation training algorithm) The input signals are propagated in a forward direction on a layer-bylayer basis. Learning

More information

Neural Networks. Neural Network. Neural Network. Neural Network 2/21/2008. Andrew Kusiak. Intelligent Systems Laboratory Seamans Center

Neural Networks. Neural Network. Neural Network. Neural Network 2/21/2008. Andrew Kusiak. Intelligent Systems Laboratory Seamans Center Neural Networks Neural Network Input Andrew Kusiak Intelligent t Systems Laboratory 2139 Seamans Center Iowa City, IA 52242-1527 andrew-kusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak Tel. 319-335

More information

LECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS

LECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS LECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS Neural Networks Classifier Introduction INPUT: classification data, i.e. it contains an classification (class) attribute. WE also say that the class

More information

CS6220: DATA MINING TECHNIQUES

CS6220: DATA MINING TECHNIQUES CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou Sun yzsun@ccs.neu.edu November 19, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining

More information

Performance Analysis of Data Mining Classification Techniques

Performance Analysis of Data Mining Classification Techniques Performance Analysis of Data Mining Classification Techniques Tejas Mehta 1, Dr. Dhaval Kathiriya 2 Ph.D. Student, School of Computer Science, Dr. Babasaheb Ambedkar Open University, Gujarat, India 1 Principal

More information

The Traveling Salesman

The Traveling Salesman Neural Network Approach To Solving The Traveling Salesman Problem The Traveling Salesman The shortest route for a salesman to visit every city, without stopping at the same city twice. 1 Random Methods

More information

Time Series Prediction as a Problem of Missing Values: Application to ESTSP2007 and NN3 Competition Benchmarks

Time Series Prediction as a Problem of Missing Values: Application to ESTSP2007 and NN3 Competition Benchmarks Series Prediction as a Problem of Missing Values: Application to ESTSP7 and NN3 Competition Benchmarks Antti Sorjamaa and Amaury Lendasse Abstract In this paper, time series prediction is considered as

More information

Unsupervised Recursive Sequence Processing

Unsupervised Recursive Sequence Processing Unsupervised Recursive Sequence Processing Marc Strickert, Barbara Hammer Dept. of Math./Comp. Science, University of Osnabrück, Germany e-mail: {marc,hammer}@informatik.uni-osnabrueck.de Abstract. We

More information

Chapter 2 Basic Structure of High-Dimensional Spaces

Chapter 2 Basic Structure of High-Dimensional Spaces Chapter 2 Basic Structure of High-Dimensional Spaces Data is naturally represented geometrically by associating each record with a point in the space spanned by the attributes. This idea, although simple,

More information

CS325 Artificial Intelligence Ch. 20 Unsupervised Machine Learning

CS325 Artificial Intelligence Ch. 20 Unsupervised Machine Learning CS325 Artificial Intelligence Cengiz Spring 2013 Unsupervised Learning Missing teacher No labels, y Just input data, x What can you learn with it? Unsupervised Learning Missing teacher No labels, y Just

More information

Machine Learning Reliability Techniques for Composite Materials in Structural Applications.

Machine Learning Reliability Techniques for Composite Materials in Structural Applications. Machine Learning Reliability Techniques for Composite Materials in Structural Applications. Roberto d Ippolito, Keiichi Ito, Silvia Poles, Arnaud Froidmont Noesis Solutions Optimus by Noesis Solutions

More information

CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK FOR IMAGE RESTORATION

CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK FOR IMAGE RESTORATION 135 CHAPTER 6 COUNTER PROPAGATION NEURAL NETWORK FOR IMAGE RESTORATION 6.1 INTRODUCTION Neural networks have high fault tolerance and potential for adaptive training. A Full Counter Propagation Neural

More information

Processing Missing Values with Self-Organized Maps

Processing Missing Values with Self-Organized Maps Processing Missing Values with Self-Organized Maps David Sommer, Tobias Grimm, Martin Golz University of Applied Sciences Schmalkalden Department of Computer Science D-98574 Schmalkalden, Germany Phone:

More information

Rough Set Approach to Unsupervised Neural Network based Pattern Classifier

Rough Set Approach to Unsupervised Neural Network based Pattern Classifier Rough Set Approach to Unsupervised Neural based Pattern Classifier Ashwin Kothari, Member IAENG, Avinash Keskar, Shreesha Srinath, and Rakesh Chalsani Abstract Early Convergence, input feature space with

More information

Cluster Analysis using Spherical SOM

Cluster Analysis using Spherical SOM Cluster Analysis using Spherical SOM H. Tokutaka 1, P.K. Kihato 2, K. Fujimura 2 and M. Ohkita 2 1) SOM Japan Co-LTD, 2) Electrical and Electronic Department, Tottori University Email: {tokutaka@somj.com,

More information

IMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS

IMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS IMPLEMENTATION OF RBF TYPE NETWORKS BY SIGMOIDAL FEEDFORWARD NEURAL NETWORKS BOGDAN M.WILAMOWSKI University of Wyoming RICHARD C. JAEGER Auburn University ABSTRACT: It is shown that by introducing special

More information

Using Self-Organizing Maps for Sentiment Analysis. Keywords Sentiment Analysis, Self-Organizing Map, Machine Learning, Text Mining.

Using Self-Organizing Maps for Sentiment Analysis. Keywords Sentiment Analysis, Self-Organizing Map, Machine Learning, Text Mining. Using Self-Organizing Maps for Sentiment Analysis Anuj Sharma Indian Institute of Management Indore 453331, INDIA Email: f09anujs@iimidr.ac.in Shubhamoy Dey Indian Institute of Management Indore 453331,

More information

CMPT 882 Week 3 Summary

CMPT 882 Week 3 Summary CMPT 882 Week 3 Summary! Artificial Neural Networks (ANNs) are networks of interconnected simple units that are based on a greatly simplified model of the brain. ANNs are useful learning tools by being

More information

IMAGE CLASSIFICATION USING COMPETITIVE NEURAL NETWORKS

IMAGE CLASSIFICATION USING COMPETITIVE NEURAL NETWORKS IMAGE CLASSIFICATION USING COMPETITIVE NEURAL NETWORKS V. Musoko, M. Kolı nova, A. Procha zka Institute of Chemical Technology, Department of Computing and Control Engineering Abstract The contribution

More information

arxiv: v1 [physics.data-an] 27 Sep 2007

arxiv: v1 [physics.data-an] 27 Sep 2007 Classification of Interest Rate Curves Using Self-Organising Maps arxiv:0709.4401v1 [physics.data-an] 27 Sep 2007 M.Kanevski a,, M.Maignan b, V.Timonin a,1, A.Pozdnoukhov a,1 a Institute of Geomatics and

More information

Data Clustering. Chapter 9

Data Clustering. Chapter 9 Chapter 9 Data Clustering When we are confronted with a novel data set it is customary to first do some exploratory analysis. A clustering of the data is an algorithm that can help us summarize how the

More information

Neuro-Fuzzy Computing

Neuro-Fuzzy Computing CSE53 Neuro-Fuzzy Computing Tutorial/Assignment 3: Unsupervised Learning About this tutorial The objective of this tutorial is to study unsupervised learning, in particular: (Generalized) Hebbian learning.

More information