KOHONEN'S SELF ORGANIZING NETWORKS WITH "CONSCIENCE"

Transcription

1 Kohonen's Self Organzng Maps and ther use n Interpretaton, Dr. M. Turhan (Tury) Taner, Rock Sold Images Page: 1 KOHONEN'S SELF ORGANIZING NETWORKS WITH "CONSCIENCE" By: Dr. M. Turhan (Tury) Taner, Rock Sold Images November 1997 INTRODUCTION: The effectveness of the nterpretve use "Sesmc Attrbutes" depends on the dscrmnaton ablty of the chosen set of attrbutes. As n many mathematcal problems, some attrbutes may be necessary; a few may be suffcent. Ths could be determned by expermentng wth logcal combnatons of varous mxtures of attrbutes. In cases where well logs or lthologcal columns are avalable, Feed Forward fully connected Neural Networks may be traned n a supervsed manner. In cases where no well nformaton s avalable, we wll have to use some unsupervsed methodology to cluster the data and check to see f t wll have any relaton to our experence based nterpretaton. In our case the data s the computed set of attrbutes. The user selects a set of attrbutes to be used n clusterng or n classfcaton. There are several unsupervsed clusterng methods to do the job. One of the smplest s to cluster the data sequentally. Assume that we are gven a set of data traces, where each sample s represented by a vector of attrbutes. Ths can be vewed as data represented n an N dmensonal space, where each attrbute s one of the axes. In the begnnng we compute a smple mean of all of the attrbutes; then we collect all of the data samples wthn some dstance to ths mean value. As we are checkng all of the samples, we can dentfy the data samples wthn the allowed dstance as members of the frst cluster and take them away from the further computaton. After all of the data sets are computed we have formed the frst cluster and dentfcaton. Whle ths s gong on we can compute the average of the rejected samples, whch wll represent the mean of the second set. We contnue clusterng n ths manner untl no sgnfcant data samples are left. The problem wth ths procedure, although t may be fast, s that all clusters are arbtrary and no reasonable relatonshp s mantaned between the clusters. Ths feature, as we wll dscuss later, represents an mportant aspect of Kohonen's method where clusters are formed n such a way that they relate to one another n an organzed manner. For example, clusterng for the sand, sandy-shale, shaly-sand and shale cluster may occupy adjacent locatons on the organzed map. Ths s analogous to the human brans "assocatve memory". Another, more classcal approach s the development of statstcs of the data set by formng the covarance matrx and computng the prncpal components. Ths wll gve us Egen values and Egen vectors from the most to the least mportant. The Egen vector correspondng to the most sgnfcant Egen value wll represent the attrbute combnaton whch s most abundant n the data set. The next Egen value wll represent the next most abundant and so on. Once these are determned, we can classfy the data set by computng the dstance between the data set and the Egen vectors and selectng the one wth the mnmum Eucldean dstance. Ths method wll form clusters based on populaton and may not have any relatonshp to adjacent Egen values and vectors. Furthermore, we wll have to solve a very large matrx, composed of correlatons between all pars of the data set. KOHONEN'S ANALOGY: Kohonen's self-organzng maps (SOM) are smple analogs of the human bran s way of organzng nformaton n a logcal manner. Research has shown that the cerebral cortex of the human bran s dvded nto functonal subdvsons. The cortex contans bllons of neurons wth many bllons of connectons (synapses) between them. The subdvson of the cortex s an orderly manner. The most mportant ones are motor cortex, sensory cortex, vsual cortex and audtory cortex. For example, the audtory cortex s also subdvded nto many cells, each functonng as cogntve parts for some audtory sgnal. It s theorzed that some of these are traned n a supervsed manner, and some are developed n an unsupervsed self-organzng manner. Topologcally adjacent areas perform somewhat related cogntve functons. Kohonen's method emulates the unsupervsed learnng n an elegant, extremely smple manner. Durng the self-organzng procedure the topologcally close relatonshp of the organzed nformaton s mantaned. Intally, a large area s treated n a smlar fashon. Later n the teraton ths zone shrnks. A number of other computatonal procedures were ntroduced after the orgnal Kohonen algorthm. Snce "wnner take all" logc

2 Kohonen's Self Organzng Maps and ther use n Interpretaton, Dr. M. Turhan (Tury) Taner, Rock Sold Images Page: may result n one neuron domnatng the tranng, some modfcaton s necessary. I wll descrbe the orgnal method wth only one smple modfcaton, called the " conscence. Kohonen's method conssts of one layer of neurons and uses the method of compettve learnng wth "wnner take all ". The modfcaton looks at the wnnng rate of each neuron, and f t sees any neuron domnatng, t uses the "conscence" algorthm and lets other neurons have a chance n the learnng procedure. METHOD: As n the case of covarance studes, nput to the self-organzng maps wll have to be normalzed. That s each set of attrbutes has to be scaled so that ther RMS value wll be equal to unty. Ths wll prevent any attrbute arbtrarly domnatng the clusterng. Kohonen's self-organzng maps consst of one layer of neurons organzed n one, two and mult-dmensonal arrays. Each neuron has as many nput connectons as there are number of attrbutes to be used n the classfcaton, as shown on fgure 1. The tranng procedure conssts of fndng the neuron wth weghts closest to the nput data vector and declarng that neuron as the wnnng neuron. Then the weghts of all of the neurons n the vcnty of the wnnng neuron are adjusted, by an amount nversely proportonal to the dstance. The radus of the accepted vcnty s reduced as the teraton numbers ncrease. The tranng process s termnated f RMS errors of all of the nput are reduced to an acceptable level or a prescrbed number of teratons are reached. There are two methods to determne the smlarty. In the frst method, each nput s weghted by a neuron s correspondng weght vector and the results are summed. Ths represents the net nput of the partcular neuron. Let k represent the k'th neuron and N attrbutes are used, then the net nput wll be (n terms of a vector scalar product); net k = N = 1 [ x( ). w(, k)] (1) The vector scalar product (1) wll gve the projecton of one vector on to the other. If the unt vectors are defned n the x() and w() drectons, ther dot product yelds the cosne of the angle between x and w vectors. Ths s one of the reasons we normalze each set of attrbutes to have the zero mean and ther RMS ampltude be unty. A net nput of 1 would represent two collnear vectors; they pont n the same drecton and ther parameters are smlar. A value of 0 wll mean that two vectors are perpendcular to each other, thus they are not smlar. The second method to measure the smlarty of two vectors s to compute the Eucldean dstance between two vectors, as; net k = N = 1 [ x( ) w(, k)] () In ths case a net result of zero wll mean that the two vectors are dentcal. A value of close to twce to ther normalzed ampltudes wll mean that they are n opposte drectons, thus they are not smlar. These are repeated for all of the neurons and the neuron wth largest output of dot product or the mnmum Eucldean dstance s chosen as the wnner. In order to mantan smlarty of topologcally close neurons, weghts of all neurons wthn a selected radus are adjusted. The wnnng neuron s adjusted by the most amount, whch wll brng the weghts of the wnnng neuron closer to the nput data values. All other neurons are adjusted by lesser amounts, nversely proportonal to ther dstance from the wnnng neuron. As the teraton progresses and RMS error reduces, the radus of correcton s also gradually reduced. Ths wll eventually become one neuron dstance; thus no other neuron s adjusted other than the wnnng neuron. In the case where one neuron s contnually the wnnng neuron, then ts computed dstance s also modfed by some amount to allow other neurons wn. Ths process s called the "conscence". At the concluson of tranng, each neuron's weght represent reference data, whch could later be used for classfcaton. If the nput s sesmc attrbutes, then the weghts represent the reference attrbutes (.e. the mean) of each classfcaton (.e. cluster). The classfcaton process at the concluson of tranng s then, the computaton of each nput data sets dstance to each of the neurons and classfy the nput as belongng to the class represented by the wnnng neuron.

3 Kohonen's Self Organzng Maps and ther use n Interpretaton, Dr. M. Turhan (Tury) Taner, Rock Sold Images Page: 3 Fgure 1. Connecton of nput data to a Neural Node TOPOLOGICAL ORGANIZATION: Neurons can be organzed n any topologcal manner. Fgure shows a one-dmensonal organzaton. Ths wll allow us clusterng wth a one-dmensonal topologcal relatonshp. That s, each adjacent neuron wll have smaller dfferences than the neurons two spaces away. The dfference wll gradually ncrease wth ncreasng dstance. Each neuron s connected to the nput data wth ther own weghts. In the case of the SOM these weghts wll be equvalent to the normalzed attrbutes representng the centrod coordnates of each cluster. Ths feature of the SOM s very dfferent than the feed forward fully connected Artfcal Neural Networks, where the weghts do not relate drectly to the nput data. Fgure Input to One Dmensonal Neural Topology

4 Kohonen's Self Organzng Maps and ther use n Interpretaton, Dr. M. Turhan (Tury) Taner, Rock Sold Images Page: 4 The same characterstcs wll preval n two-dmensonal mappng as shown on fgure 3. Ths tme we have four adjacent neurons to each neuron, east, west, north and south drecton. The dfferences wll be nversely proportonal to the topologcal dstance between the neurons. We can thnk of three dmensonal mappng the same way. In practce, one or two-dmensonal mappng s usually used. I wll dscuss the case of two-dmensonal mappng n sesmc nterpretaton. The mappng confguraton wll affect only the defnton of the neghborhood functon; thus t s very easy to handle the varous dmensonalty of the maps. USES OF SELF-ORGANIZING MAPS: SOM's can be used n sesmc nterpretaton n two prncpal ways, one as a cluster generator and the other as a classfer. Input data conssts of -D or 3-D trace segments or ther attrbutes. Each data sample and correspondng attrbutes representng a pont n the N dmensonal space s used to form a set of weghts for each Neural node accordng to the network topology. These weghts are called the assocated memory, because they represent reference attrbutes of each cluster and they have some organzed assocaton to ther neghbors. Ths process takes a large number of teratons, of the order of tens of thousands. Measurng the RMS error reducton after each teraton can montor convergence. Once the convergence reaches a satsfactory mnmum, then the weghts are saved for the classfcaton. Snce each weght vector represents a reference attrbute of a partcular cluster, we perform classfcaton by computng the Eucldean dstance between each data sample and the weght vector of each node, and select the node wth mnmum dstance. The SOM s an unsupervsed learnng algorthm; t does not gve us drect nformaton on the actual physcal or lthologcal classfcaton of the data samples. It merely clusters and classfes the data set based on the set of attrbutes used. We wll need to attach lthologcal meanng after the classfcaton s completed by the use of nearby well nformaton or based on our experence n the area. If there s a lthologcal column avalable n the general vcnty of the sesmc data, then we can experment wth dfferent sets of attrbutes to see whch set gves us smlar classfcaton boundares as the lthologcal column. Fgure 3. Input to -Dmensonal Neural Topology

5 Kohonen's Self Organzng Maps and ther use n Interpretaton, Dr. M. Turhan (Tury) Taner, Rock Sold Images Page: 5 COMPUTATIONAL PROCEDURE: I wll outlne the computatonal procedure step by step ncludng some dscusson assocated wth each step. 1. Data Intalzaton: Once the composton of the attrbutes s selected, each attrbute set wll be normalzed (scaled ) to RMS=1. Ths wll prevent one attrbute from overpowerng n clusterng and classfcaton. If, however, we consder some attrbutes more dagnostc than the others, we can upward adjust ther RMS ampltudes. Ths wll make ths partcular attrbute's dstance be more promnent than the rest, hence mnmzaton wll be affected by t.. Intalzaton of Neural Weghts: Each neuron wll have one weght for each attrbute. To begn, we assgn small random numbers to each of the weghts, as n the case of other Neural Network computatons. In the begnnng we wll also ntalze the neghborhood radus. Ths radus wll be large at the begnnng to cover a larger number of adjacent nodes. As the teraton proceeds, ths radus may be systematcally reduced to a unt dstance. We wll also ntalze a set of counters that wll ndcate the number of consecutve tmes a Neural node s declared as the wnner. In the "conscence" algorthm, f a node wns excessvely, t wll feel bad and allows others to wn. Ths s accomplshed by artfcally reducng the computed weghts, causng other nodes to wn. 3. Selecton of Input data: Select nput data at random from the gven set of sesmc data traces and ther attrbutes. Ths wll prevent smlar neghborng attrbutes from over nfluencng the weght-updatng scheme. All of the data samples are nput at each teraton cycle. 4. Computaton of Eucldean Dstance or Vector Dot Product and determnng the wnnng Node: The Eucldean dstance or Vector dot product s computed between the nput data set (vector) and each of the Neural Nodes ( as shown on equatons 1 and ). The Node wth the mnmum Eucldean dstance or maxmum vector dot product s declared the wnner. Ths determnes the node wth the most smlar set of weghts to the nput data attrbutes. Before the wnnng node s determned, the number of consecutve wns s checked. If the node had exceeded the lmt, then the results are reduced proportonally to allow another node to wn. The losng node counters are reset to zero, and the wnnng node counter s ncremented by one. In some nstances one node may contnually wn, thus overpowerng all the other nodes. To prevent ths DeSeno (1988) has ntroduced an algorthm called the "conscence". Ths algorthm keeps track of the wnnng neurons and f they exceed some level of number of wnnng, ther Eucldean dstances are ncreased or the dot products are decreased to allow the other neurons a chance to wn. Let the output of wnnng 'th element gven by; y =1 f w X < wj X for all j values, except, j y = 0 otherwse. (3) A bas s developed for each neuron based on the number of tmes t has won the competton shown on eq. (3) Let p represent the fracton of tme a neuron wns, then we can update the bas factor by;. p new old old = p + β [ y p ], (4) where 0 < β <<1. We are usng = β.. We compute now, the conscence factor; b = C 1/ N p ), (5) (

6 Kohonen's Self Organzng Maps and ther use n Interpretaton, Dr. M. Turhan (Tury) Taner, Rock Sold Images Page: 6 where C s the bas factor, and N s the number of neurons. Some of the examples on varous book use C=10. We have to experment wth ths value. Once the conscence factor s determned, new ( conscence modfed ) Eucldean dstances wll be computed as; z =1 f w X b < wj X b j for all j values, except, j z = 0 otherwse. (6) Ths wll determne the new wnner. We wll contnue the process by makng the weght vector of the wnnng neuron closer the nput data, as descrbed below. 5. Updatng the weghts of wnnng node and ts neghbors: The wnnng nodes weghts wll be modfed by ; w( n 1) ( k, = wn ( k, + ( n)[ x( wn ( k, ] + η (7) where n s the number of the teraton, k s the wnnng node and j s the j'th attrbute. η(n) s the modfcaton weght as a decreasng functon of the number of teratons. Ths s also called the learnng rate. All of the nodes wthn the assocatve radus wll also be modfed, nversely proportonal to the dstance on the map. If we assgn one, two or more dmensonalty to each node, we can compute the relatve dstance from the wnnng node to each other node and can easly compute the modfcatons. Let the radus d(m,k,n) be represented as a decreasng functon of the teraton number. 6. Update neghborng nodes accordng to ther dstances to the wnnng node; w ( ( n 1) m, = wn ( m, + d( m, k, n). ( n)[ x( wn ( m, ] + η (8) where d(m,k,n) s a functon of teraton number n, and the dstance between wnnng node k and ts neghbor m. All other nodes outsde the assocatve radus wll have no correcton. 7. Update assocatve correcton radus; In the begnnng the ntal radus may be kept long enough to cover most of the neural nodes. Ths s reduced as the teraton number ncreases. d(m,k,n) can be made any functon nversely proportonal to the teraton number n, and the dstance between the wnnng node k and the node to be adjusted m. 8. Check convergence rate to contnue or to go to fnsh teraton: We compute RMS error (square root of the sum of error squares, whch s the square of the dstance between the nput and the wnnng neural nodes weghts dvded by the number of nputs) and check f t s below some prescrbed level. If t s, then we have developed the desred weghts, whch can be used for classfcaton. If not, then we go back for another set of teraton (back to step 3). 9. Qualty control: One of the man characterstcs of Kohonen's mappng s the determnaton neurons wth topologcally assocatve weght sets. Therefore, a good qualty check s to see f the Eucldean dstance between neurons computed from ther weghts do satsfy the orgnal topologcal desgn. That s, computed dstances between neural nodes should be proportonal to the topologcal dstances of the desred map. In other words, fnal map coordnates should resemble shape of the user specfed map. If ths condton s not satsfed, some addtonal teraton may be needed. 10. Save computed weghts as classfers or as reference attrbutes: The computed weghts of the neural nodes are the memory functons, whch are used for classfcaton. They should have smaller dfferences between adjacent neurons. In our case they represent the Attrbute values of

7 Kohonen's Self Organzng Maps and ther use n Interpretaton, Dr. M. Turhan (Tury) Taner, Rock Sold Images Page: 7 each classfcaton. (For actual values we have to remove the scalng we appled to the attrbutes durng normalzng.) 11. Use computed reference attrbutes to classfy nput data set: In the classfcaton stage for each nput, we compute the Eucldean dstance to each neural node and classfy the nput as a member of the class represented by the wnnng node. Ths s very smlar to the actual teraton, except we no longer adjust the weghts. 1. Valdty check of the results: The "conscence" condton tres to develop a unform or equally balanced clusterng. Therefore we should expect that each classfcaton should have about the same number of members and smlar varance. Durng the classfcaton process, the number of members of each class and ther statstcs are accumulated, whch could then be used for a check of valdty. These results, as well as other qualty control statstcs, are output to the user for proper control of the process.. CONCLUSIONS: Kohonen's Self-Organzng Map method can be used n several dfferent ways for lthologc nterpretaton. One of the most obvous ways s to use a typcal porton of a -D or 3-D data set to tran the network; then use the reference weghts as classfers for the rest of the data set. If we have a lthologcal column, we can experment wth a number of dfferent and logcal attrbute combnatons to see whch clusters are most smlar to the lthologcal column. In ths elegant method, there s no restrcton to the dmenson of the nput. It could be one, two or many dmensonal nput vectors, defnng varous physcal attrbutes or geometrcal condtons. Ths means that, we may be able to use the SOM for vsual type classfcatons. Ths we wll have to experment wth n the near future. ACKNOWLEDMENTS: I would lke to express my thanks to James Schuelke and John Quren of Mobl R&D Corporaton for ther nvaluable help and drecton. To them and to Graham Jago goes many thanks for gentle edtng of the text. The value of the SOM technque n nterpretaton became more obvous n our conversatons. REFERENCES: Proceedngs of the IEEE, 1990, Specal ssue on Neural Networks; 1. Neural Networks, theory and modelng (September ssue).neural networks, analyss, technques and applcatons (October ssue) (These ssues have extensve lterature and background artcles) Bharath, R, and Drosen, J, 1994, "Neural Network Computng", Publshed b Wndcrest/McGraw-Hll, New York (Ths book contans examples of some prmtve functons used n Neural computaton. C-code programs are ncluded on a dskette) Chester, M, 1993, "Neural networks, A Tutoral", Publshed by PTR Prentce Hall, Englewood Clffs, New Jersey. (Ths book s recommended to the begnners of the Neural Networks) DeSeno, D., "Addng a Conscence to compettve learnng." IEEE Internatonal Conference on Neural Networks, Vol. 1, pp , San Dego CA. Haykn, S. 1994, "Neural Networks, A Comprehensve Foundaton", Publshed by Macmllan College Publshng Company, New York. (Ths book contans one of the most comprehensve dscussons of all types of Neural Networks ncludng the Self-Organzed Maps). Kohonen, T., 1988, "Self-Organzng and Assocatve Memory", 3 rd. ed., Publshed by Sprnger Verlag, New York. Pao Y. H., 1989, "Adaptve Pattern Recognton and Neural Networks", Addson-Wesley Publshng Company.