ANALYSIS OF THE FIRST LAYER IN WEIGHTLESS NEURAL NETWORKS FOR 3_DIMENSIONAL PATTERN RECOGNITION

Transcription

1 ANALYSIS OF THE FIRST LAYER IN WEIGHTLESS NEURAL NETWORKS FOR 3_DIMENSIONAL PATTERN RECOGNITION A. Váque-Nava * Ecuela de Ingeniería. CENTRO UNIVERSITARIO MEXICO. DIVISION DE ESTUDIOS SUPERIORES J. Figueroa Nauno & E. Varga Medina Laboratorio de Sitema Complejo UNIVERSIDAD DEL VALLE DE MEXICO - LOMAS VERDES Av. Paeo de la ave #1. Fracc. Loma Verde Naucalpan Edo. de México MEXICO FAX: (915) Publihed. CLEI ITESM Campu Etado de México. MEXICO September 19-23, 1994 ABSTRACT The behavior of the firt layer of a weightle artificial neural network i analyed in thi paper. The way in which the neural network receive external information change accordingly to different probability ditribution function that control data ampling from many different pattern. Thi paper decribe the architecture of thi ytem and how the effect of the different probability ditribution function over 3_dimenional pattern recognition. 1. INTRODUCTION Conidering contemporary reearch in neural network, it i known that combining a certain paradigm and hidden layer of neural network give a a reult a powerful tool for pattern recognition and claification. However, there ha been little development and interet in tudying the firt or receiving layer of artificial neural network. Implicitly, it ha been aumed that the firt layer ha an homogeneou (and almot an irrelevant) behavior which i aociated to the global performance of the neural network. Thi fact i againt a big amount of phyiological evidence indicating that the type and organiation of receiver of biological neural network have a great importance in their phyiological behavior. It i known that pecific phenomena of full generality are aociated to the behavior of the firt layer (or receiving layer) of neural network; the mot known cae i lateral inhibition. Earlier work have experimentally demontrated the effect on pattern recognition and claiffication when the way in which the firt layer of an artificial neural network receive external information i changed 4,5,6,7. Thi paper decribe a ytem compoed by a fixed weightle neural network (Alekander` model) 1,2,5,7. Thi ytem i capable of changing the way in which it receive external information. Pattern to be claified are ampled in many way through the ue of different probability ditribution function. Thee probability ditribution function determine the way in which the firt layer i activated. The effect of the different probability ditribution function over 3-dimenional pattern recognition i tudied in thi paper. 2. PROCEDURE The following are the neceary element of a geneneralied weightle Neural Network (baed on Alekander` Model) for 2D (2-dimenional) and 3D (3-dimenional) pattern learning and recognition. Thee element are: Pattern repreentation array, Control array and probability ditribution function for data ampling, Mapping function, and Learning/Recognition array.

2 2D 2.1 Pattern Repreentation Array: Thi array tore the color code of the point that belong to the learned/recognied pattern. In the cae of pattern learning/recognition, thi array i repreented by: [ i, j] : c V = (1) where i, j are integer repreenting the coordinate of a point P(i,j) and c repreent the color code for point P. Generaliing for 3D, thi array i repreented by uing a et of 2D array. Thi et i alo an array: [ i, j, ] : c W = (2) thi array i made of p matrixe of the ame type a V : [ i, j, ] : = V [ i, j] : = c ( 1, 2, 3,..., p) W = (3) 2.2 Control Matrixe and probability ditribution function for data ampling: Thee matrixe are required to perform the pattern learning and pattern recognition phae. The point of a pattern are ampled by uing the value tored in the Control Matrixe. Different probability ditribution function ( P(X) ) compute thee value that determine the way in which the firt layer of the neural network i activated. The effect of the different probability ditribution function over 2D & 3D pattern recognition i tudied in the next ection. For 2D pattern learning/recognition, 2 matrixe ( & MJ ) tore the coordinate of the point to be ampled. Matrix tore the value of coordinate i and matrix MJ tore the value of coordinate j of ampled element from the Repreentation array V. For 3D, there are 2 Control Matrixe ( & MJ ) that are aociated to each array V belonging to et W. The following expreion compute the value tored by Control Matrixe: MJ [ i, jmj ]: = INT( ( m + 1) * P( X )) [ i, j ]: = INT (( n+ 1) * P( X) ) MJ (4) & (5) where m repreent the number of row of V and n repreent the number of column of the ame array. P(X) i a probability ditribution function with X a a random variable and 0 <=P(X)<= 1. The dimenion of matrixe and MJ i computed a a function of the dimenion of matrix V. Given V whoe capacity i m*n, then: m n = m = n MJ MJ : = m * 2 : = m / 2 (6) & (7) where m i the number of row belonging to and MJ ; n i the number of column belonging to thee matrixe. 2.3 Mapping : The main purpoe of the Control Matrixe i to control the ampling of element belonging to matrix V (for 2D pattern learning and recognition) or matrixe V (for 3D). Thee ampled element are the input for computing a Mapping f. The Mapping f map the ampled element of V to element belonging to the Learning/Recognition array. Thi Mapping compute the addre of the a th element of the Learning/Recognition array. Thi mapping i performed with value k, l and matrixe, MJ & V a input. The Mapping ha the following tructure: ( V [ [ k, l], MJ[ k, l], maxcod, k l] ) Addre := f, For 2D pattern learning and recognition, the Mapping f i propoed a: (8)

3 a : = 1 + n l ( * maxcod ) n ( k 1) * maxcod + V[ [ k, l], MJ[ k, l ] l= 0 (9) where k, l are the coordinate of the element of matrixe and MJ. maxcod i the maximun number of color code ued for pattern repreentation. V[ [k,l], MJ[k,l] ] i an element which i ampled from V uing the pecified coordinate in matrixe & MJ. For 3D pattern learning and recognition, the Mapping f i propoed a: a : = 1 + n l ( * maxcod ) n ( k 1) * maxcod + V [ [ k, l], MJ [ k, l ] l= 0 where k, l are coordinate of element belonging to matrixe & MJ. maxcod repreent the maximum number of color code for pattern repreentation. V [ [k,l], MJ[k,l] ] i a ampled element from the " th " matrix V which belong to et W. (10) 2.4 Learning/Recognition Array and Learning Phae: During the learning phae, the Mapping f i computed in uch a way that the content of the mapped element of the Learning/Recognition Array (array A ) will be updated. Thee element are referenced by the addree which are computed by the Mapping f. For 2D, each element of thi array i compoed by a et of bit, each one i related to a pattern cla. If the k th bit of an element i et on, then thi fact implie that thi element belong to the pattern cla k. A ingle element of A may have everal bit et on at different poition. Thi ituation mean that a ingle element of A may belong to different pattern clae. For 3D, the ame approach i followed. However, the Learning/Recognition array i compoed by column, each column repreent a ingle array A. Therefore, for 3D pattern learning and recognition, the Learning/Recognition array B i a et of array of type A. The value a,which are computed by the mapping function, repreent the addree of mapped element belonging to the th column of B (ee fig. 1). 2.5 Mechanim for pattern learning: The pattern, which i to be learned by the Weightle Neural Network, i tored in the Pattern Repreentation Array ( V for 2D, W for 3D). The coordinate of ampled element belonging to the pattern are computed uing a probability ditribution function P(X). Thee coordinate are tored in the Control Matrixe for data ampling ( & MJ for 2D, & MJ for 3D). Thee matrixe control the ampling of element from the Pattern Repreentation Array. The ampled element are ued a input for the Mapping ( f for 2D, f for 3D), o the addree of element belonging to the Learning/Recognition Array are computed. The content of thee mapped element are updated by etting on the aociated bit to the cla. The learned pattern belong to thi cla. In cae of 2D pattern learning, n element of the Learning/Recognition Array will be updated (ee fig. 1 & 2). For 2D, thi update i performed by uing the following expreion: [] a:= Aa [] OR A 2 (11) where repreent the cla to which the learned pattern belong, and a i the addre which i computed by the Mapping f. In cae of 3D pattern learning, p * n element of the Learning/Recognition Array will be updated. For 3D, thi update i performed by: [ a, ] : = B[ a, ] OR B 2 (12) where repreent the cla to which the learned pattern belong, and a repreent the addre of the element to be updated in the th column of array B.

4 2.6 Mechanim for pattern recognition: The pattern, which i to be claiffied by the neural network, i tored in the Pattern Repreentation Array ( V for 2D, W for 3D). Uing the ame Control Matrixe for data ampling ( & MJ for 2D, & MJ for 3D), which were computed during the learning phae, the element of the pattern are elected to be input for the Mapping (ee fig. 2). For each mapped element belonging to the Learning/Recognition Array, whoe addre i computed by the Mapping, it content i analyed. Thi analyi conit in checking the bit which are et on in each mapped element. The number of bit of a given cla i counted. For 2D, the following expreion count the number of bit et on for each cla: count count [] [] : : = p n = n r= 1 = 1 r= 1 B [ ] A a r [ a ], r AND 2 2 AND 2 2 For =0, 1,, maxcla-1 (13) where a i computed by uing the Mapping f. maxcla i the maximum number of pattern clae pecified during learning phae. For 3D, the following expreion compute the count: For =0, 1,, maxcla-1 (14) where p i the number of matrixe V contained in et W, and B i the Learning/Recognition array. a, r i the addre of the mapped element which i computed by the Mapping f. Once the count for each pattern cla ha been computed, the count are compared among each other. The highet count for a given pattern cla k implie that the weightle neural network recognice the input pattern a belonging to cla k. Pattern recognition may be meaured by a core which range from 0 to 1. Each core i computed a a function of each count. For 2D, the core related to each pattern cla i computed by the following expreion: count core : = [] n [] For =0, 1,, maxcla-1 (15) For 3D, the following expreion compute each core: core [] count := p * [] n For =0, 1,, maxcla-1 (16) 3. EXPERIMENTS AND RESULTS A factorial experimental deign 3 * 3 * 4 (3 learned clae, 3 training condition and 4 probability ditribution function) wa implemented for teting the decribed artificial neural network. Each pattern wa drawn with 3 different color and uing a 16 * 16 * 5 Pattern Repreentation Array. The percentage and mean core of properly recognied pattern, and alo tandard deviation were meaured for each experimental condition. The reult of thi analyi are hown in table 1. The content of table 1 were ued a input for a Three-way Analyi of Variance (ANOVA). The reult of thi analyi clearly how that pattern recognition (performed by the weightle artificial neural network in thi paper) i affected by the number of learned clae (F=6.717, p<0.0001), the number of training pattern (F= , p<0.0001) and the Probability Ditribution ued for data ampling (F=15.014, p<0.0001). Thi analyi ha alo hown highly ignificant effect over pattern recognition due to the number of training pattern that interact with the type of probability ditribution (F=6.716, p<0.001); minor ignificant effect are aociated with interaction between number of learned clae and number of training pattern (F=2.188, p<0.05). Thee effect can clearly be een in fig. 3,4,5.

5 The number of learned pattern (for each cla) ha a direct relationhip with the performance aociated with pattern recognition and with the mean core of recognied clae. A the number of learned pattern increae, the percentage and mean core of recognied clae alo increae. It alo can be noted that the percentage and mean core are not affected (in a negative way) a the number of ued clae during learning phae increae. 4. CONCLUSIONS AND DISCUSSION The reult, which are decribed above, how in a very clear manner that different type of data ampling (ued for activating the firt layer of neural network) have important and ytematic effect on the global behavior of neural network. It ha been aumed that the firt layer ha a paive and unimportant behavior aociated to the performance of the neural network. However, thi and earlier work 4,5,6 how the way in which experimental manipulation of the firt layer affect and make eaier pattern recognition made by a neural network. Thi fact not only ha experimental importance but alo it how that thee theorie and formal analyi about neural network hould be reconidered in a theoretical way.

6 Mechanim for 3D pattern learning in a weightle artificial neural (1) Generating coordinate of element to be ampled from Pattern Repreentation Array V P(X) Probability Ditribution 7 Control Matrixe MJ for data ampling (2) Storing pattern to be learned and belonging to a pecific cla (3) Sampling element from the 3D Pattern Repreentation Array Mapping 5 3D Pattern Repreentation Array V (4) Calculating mapped element into Learning/Recognition Array by uing Mappging Control Matrixe MJ for data ampling (5) Updating mapped element Pattern Learning/Recognition Array B FIG. 1 Cla 0 Cla 1 Cla maxcla-1

7 Mechanim for 3D pattern recognition in a weightle artificial neural Control Matrixe MJ for data ampling 7 (1) Storing pattern to be claified (2) Sampling element from the 3D Pattern Repreentation Array Mapping 5 3D Pattern Repreentation Array V (3) Calculating mapped element into Learning/Recognition Array by uing Mappging Control Matrixe MJ for data ampling (4) Calculating Hamming ditance from each cla, according to each addre (5) Claifying input pattern a belonging to the cla with the highet core Pattern Learning/Recognition Array B FIG. 2 Cla 0 Cla 1 Cla maxcla-1

8 Effect over pattern recognition due to different Probability Ditribution under variou experimental condition. % of properly claified pattern % of properly claified pattern % of properly claified pattern Number of pattern for neural network training Effect over pattern recognition when 2 clae are learned Fig Number of pattern for neural network training Effect over pattern recognition when 3 clae are learned Fig Number of pattern for neural network training Effect over pattern recognition when 6 clae are learned Fig. 5 Simbology: Normal Ditribution Uniform Ditribution Cauchy Ditribution Beta Ditribution

9 Probability Ditribution NORMAL UNIFORM CAUCHY BETA Amount of tet pattern to be claified % % n=20 Number of learned clae: % % % n= % % % % n= % % % % n=30 Number of learned clae: % % % n= % % % % n= % % % % n=60 Number of learned clae: % % % % % % n=70 n=70 Note: - Each cell of the table ha the following tructure: 92.85% Percentage of tet pattern that were uccefully claified Mean Score of recognied pattern clae Standard Deviation - The lat column repreent the number of pattern to be claified in each Probability Ditribution Table 1

10 5. REFERENCES 1.- Alekander, I. Microcircuit Learning Computer, Mill & Boon Limited Alekander, I. & Marton, H. An Introduction to Neural Computing, Chapman & Hall Barlow, H.B. "Retina and central factor in human viion limited by noie." In. H.B. Varlow & P. Fatt. Vertebrate photoreception, pp London, Academic Pre Figueroa Nauno, J.; Flore, C.; Varga, E. & Romero, M. " mapping and it relationhip with the pychopyical function in the theory of neural network." International Joint-Conference on Neural Network. Wahington, D.C. January 15-19, Figueroa-Nauno, J.; Váque-Nava, A.; Varga-Medina, E. "3-Dimenional Pattern Recognition Uing Wightle Neural Network (Alekander' Model)." International Joint-Conference on Neural Network. Baltimore, Maryland. June 7-11, Flore Garc!a, C. ; Varga Medina, E. & Figueroa Nauno, J. "Modelo formale que predicen el reconocimiento de patrone en humano." Congreo Mexicón '95. México D.F. del 18 al 22 de eptiembre, Foryth Richard and Naylor Chri. The Hitch Hiver' guide to Artificial Intelligence, Chapman and Mall/Methven, London Geiler, W.S. "Sequential Ideal-Oberver Analyi of Viual Dicrimination." Pychological Review, vol. 96, 2, pp William, D.R. "Topography of the foveal cone moaic in the living human eye". Viion Reearch, 28, pp