Machne Learnng 9. week Mappng Concept Radal Bass Functons (RBF) RBF Networks 1 Mappng It s probably the best scenaro for the classfcaton of two dataset s to separate them lnearly. As you see n the below fgure, two dfferent dataset are classfed lnearly. But ths scenaro s not always possble n real lfe datasets. 1
Mappng All of the proposed machne learnng methods have been developed usng lnear classfcaton. So they can not fnd soluton for the non lnear dataset. Instead of solvng datasets n ther own space, data s moved to a new space where they can be separated lnearly. Ths process s called Mappng. 3 Mappng Generally a functon s used to convert x nto a lnear seperatable status. Φ: x Φ(x) 4
Mappng 5 Mappng wth RBF Functon RBF (Radal Bass Functon) s gven equaton below. Where (c j ) s the center ponts whch represents data, (x ) s the data ponts. Functon s the exponantal affect of the dstance between (c j ) and (x ) ( x) exp c x j r 6 3
RBF Networks Neurons n hdden layer are calculated as n below equaton, where x s the data pont from nput wth no weght, c j s the prototype hdden n the neuron. Smply t s a Eucldan dstance. d c j x After that ths dstance s evaluated n Radal Bass Functon. 7 RBF Networks Wdely used radal bass functon equaton and graphcal relaton of dstance-functon output s shown below. ( x) exp c x j r r value represents the dameter of prototype. d c j x 8 4
RBF Networks Archtecture 9 Estmaton n RBF Networks y represents target class, and t can be calculated wth followng equaton where x represents nput. y j 1 K w ( x ) j K value shows number of neuron n hdden layer. 10 5
Structure of RBF Networks In below fgures you can see neurons of hdden layer and output layer. x 1 f 1 w 1 x x n f 1 c-x d f w b f K w K +1 y Hdden layer neuron Output layer neuron 11 Features of RBF Networks RBF networks are feed forward networks wth supervsed learnng. Generally a unque hdden layer s used n whch each neuron contents a RBF functon. Although tranng of RBF networks looks smlar wth back-propogaton, they are traned faster than MLP. Wth advances of radal bass fuctons, they are less affected from unstable nput problems. 1 6
Tranng n RBF Networks There are four parameters n RBF networks whch s unknown and need to be learned: Number of neurons n hdden layer Coordnate of each prototype neuron Prototype dameter of each neuron Output Weghts 13 Tranng n RBF Networks In most of the proposed methods, t s outlned that number of neurons n hdden layer varable needs to be researched by tryng. Snce output weghts are components of a lnear equaton, soluton s easy and t depends on neuron outputs. So, most mportant pont n tranng depends on to know poston and dameter values of prototypes. 14 7
Tranng n RBF Networks Prototype poston nformaton n neurons can be found by unsupervsed learnng methods (lke K-Means) Tranng can be faster by runnng supervsed and unsupervsed tranngs paralelly. Accuracy of unsupervsed learnng classfers are always lower than fully supervsed learnng methods. 15 Tranng n RBF Networks Another proposal s to use the randomly selected samples as a prototype and not update poston nformaton. In ths case, dameter (varants) data should be known and selected samples should represent every regon n data space very well. 16 8
Tranng n RBF Networks In order to fnd Prototype dameter (wdth of each RBF unt, also known as spread) K-nearest neghbour algorthm s used. Root-mean squared dstance between current cluster center and K nearest neghbours s calculated. And ths s the value chosen for the unt wdth(r). So f current cluster center s cj, the r value s; k (cj c) rj = =1 k 17 Tranng n RBF Networks When we study structure of RBF networks, we can fnd some smlartes wth MLP network model. By means of lnear structured smlarty n output layer, t mght be benefted from back propagaton algorthm durng tranng of RBF networks. 1 E E e j w w j 18 9
Tranng n RBF Networks Output weghts are updated wth back propagaton tranng and update quanttes of functon exts n hdden layer neurons can be predcted. Ths can be used ndrectly n update of prototype dameter values and protoype coordnates. c E c r E r 19 Classfcaton wth RBF Networks 0 10
Homework Please compare success of the results usng lnear regresson for followng mappng functons by usng two dmensonal non-lnear artfcal dataset. Sgmod Hperbolc Tangent RBF 3th degree polynomal 1 11