Self-Organizing Map. presentation by Andreas Töscher. 19. May 2008

Size: px

Start display at page:

Download "Self-Organizing Map. presentation by Andreas Töscher. 19. May 2008"

Cody Erick Price
6 years ago
Views:

1 19. May 2008

2 1 Introduction

3 (SOM) aka Kohonen Network introduced by Teuvo Kohonen implements a discrete nonlinear mapping unsupervised learning

5 Structure of a SOM

6 Learning Rule Introduction w i [t] is the weight vector of neuron i at the time t x[t] is the input vector at time t η[t] is the learning rate at time t φ(i, k) is the neighbourhood function, k is the winner neuron w i [t + 1] w i [t] + η[t]φ(i, k)(x[t] w i [t]) (1)

7 Neighborhood Function φ(i, k) is aware of the mapping simplest choice: φ(i, k) = 1 if dist(i, k) < τ φ(i, k) = 0 else gaussian: ) dist(i, k)2 φ(i, k) = exp ( σ 2

8 General Tips for the Training rescale the input (e.g. µ = 0, σ 2 = 1) decrease the learning Rate η during the training start with a large neighbourhood for φ(i, k) and reduce it during the training avoid using too many neurons, in the extreme case the SOM ends up in a simple one to one mapping

9 Introduction

10 2D to 1D map Introduction input: randomly drawn points from a triangular 2D region 1D Self-Organizing map Peano Curve

11 2D to 2D map Introduction input: randomly drawn points from a quadratic region 100, 1000, 5000, epochs

12 2D Mapping with a Knot the nice unfolding is not guaranteed a possible solution is to start training with a bunch of random initializations

13 Remarks: Introduction in general 2D to 2D mappings are of no interest examples of interesting maps: a sphere to a plane a plane in a 3D space to a 2D space in the above examples the inner dimension of the data was known if the target is too low dimensional, the SOM folds like a Peano Curve

15 Robot Arm Movement input space: angle of the joints of the robot arm target: x and y position of the robot arm assumption: the robot arm is constructed in that way, that a given position (x, y) is only reachable with one exact one set of angels (α, β) modified training: no winner evaluation, the winner is the neuron closest to the given (x, y) position

16 to move the robot arm from A to B, you only have to interpolate between the weights of the neurons

17 Obstacle Avoidance works only if the obstacle was there during the training phase

18 Netflix Introduction users movies 100 Mio ratings user based Collaborative Filtering (aka KNN) v N(u,i) r ui = c vur vi v N(u,i) c vu (2) Problems 2 arbitrary users have few common ratings, which is very bad for the correlation coefficient the user/user correlation matrix could not be kept in main memory

19 r ui = v N(u,i) c vur vi v N(u,i) c vu (3) N(u, i) are the best K user neighbours c vu is a correlation between users, based on the mapping

20 Introduction Kohonen Maps learn a mapping (usually from a high to a low dimensional space) the mapping can be used easily in both directions the learning rule is simple, and can be implemented very efficiently the basics of Kohonen Maps are over 30 years old and have been used for very interesting projects

21 Thanks for your attention References R. Rojas, Neural Networks, Springer 1996 Scholarpedia, http: // Wikipedia,

Artificial Neural Networks Unsupervised learning: SOM

Artificial Neural Networks Unsupervised learning: SOM 01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011 01101011 01110101 01110000 01101001 01101110 01100001