Proceedings of the 8th WSEAS International Conference on Neural Networks, Vancouver, British Columbia, Canada, June 19-21,

Transcription

1 Proceedings of te 8t WSEAS International Conference on Neural Networks, Vancouver, Britis Columbia, Canada, June 9-2, Neural Network Structures wit Constant Weigts to Implement Dis-Jointly Removed Non-Convex (DJRNC) Decision Regions: Part A - Properties, Model, and Simple Case CHE-CHERN LIN National Kaosiung Normal University Department of Industrial Tecnology Education 6 Ho Ping First Road, Kaosiung, TAIWAN, R.O.C. cclin@nknucc.nknu.edu.tw Abstract - Tis is a series of studies to discuss te partitioning capabilities of multi-layer perceptrons on dis-jointly removed non-convex (DJRNC) decision regions. Tere are two papers proposed in te series of studies including part A and part B. In part A, we propose a network structure to implement DJRNC decision regions using multi-layer perceptrons. In te proposed structure, all weigts and te parameters of te activation functions are pre-determined wen a DJRNC decision region is establised. No constructive algoritm is needed for implementing te DJRNC decision regions and eac weigt determined by tis paper is eiter or. Tis makes te ardware implementations of te proposed network structures easy. Tree cases are discussed in tis paper including single, nested, and disconnected decision regions. Te first case is sown in part A, and te rest of two are demonstrated in part B. We also provide tree multi-layer perceptrons to implement te tree decision regions and prove te implementation feasibilities of te proposed model Key-Words: - Multi-layer perceptron; Nested decision region; Non-convex decision region; Disconnected decision region.. Introduction A multi-perceptron is one of popular neural network structures for implementing classification problems. Adjustable weigts are used to connect te nodes between adjacent layers and optimized by training algoritms to get te desired classification results. However, using training algoritm to optimize te weigts probably needs tousands of iterations and ence spends a lot of computational time. Related studies about partitioning capabilities of multi-layer perceptrons on vertical and orizontal partitioning ave been discussed in [-4]. Te rest of studies on te partitioning capabilities of multi-layer perceptrons, see [5-3]. Te implementation feasibility of convex recursive deletion regions as been discussed in [9] were te autors proposed a constructive algoritm to determine te parameters (weigts and tresolds) of two layer perceptrons.to implement te convex recursive deletion decision regions. Non-convex decision regions are more complicated case. In tis paper, te autor presents a network structure to implement Dis-Jointly Removed Non-Convex (DJRNC) decision regions using multi-layer perceptrons. In te proposed structure, all weigts and te parameters of te activation functions are pre-determined wen a DJRNC decision region is establised. No constructive algoritm is needed for implementing te DJRNC decision regions and eac weigt determined by tis paper is eiter or. In te following, we present an illustrative example to explain ow a two-layer perceptron implements a convex decision region. For te visual reason, in tis paper we use two-dimensional examples. It is easy to generalize tem to multi-dimensional cases. Fig. (a) is a convex decision region containing a convex polyedron (C) wic is bounded by 5 yper-planes: z,, z 3, z 4, and z 5. In tis figure, te dotted rectangular box indicates te input space of te convex decision region were te saded area belongs to class A wile te blank area belongs to class B. We call te yper-planes bounding te convex polyedron bounding yper-planes. A bounding yper-plane divides te input space into two linearly separable alf yper-planes. Te side of a bounding yper-plane is te alf yper-plane containing te convex polyedron, and 0 side is te oter alf yper-plane, as indicated in Fig. (a). In a convex decision region, a pattern is said to be in te convex polyedron if and only if it is on te sides of all bounding yper-planes of te convex polyedron. It as been known tat a convex decision region can be implemented by a two-layer

2 Proceedings of te 8t WSEAS International Conference on Neural Networks, Vancouver, Britis Columbia, Canada, June 9-2, perceptron were te first layer forms te convex decision region and te second layer performs te classifications. Te first layer weigts are pre-determined wen te decision region is formed. Te second layer weigts are all s. We use a ard limiter as an activation function in te second layer node to get te desired classification results. Te ard limiter is of te form [, 8] if θ θ y= g( θ ) = () 0 if θ < θ wereθ is te tresold and set to be equal to te number of te bounding yper-planes of te convex polyedron. For example, te convex decision region sown in Fig. (a) can be implemented by te two-layer perceptron sown in Fig. (b) were te tresold ( θ ) of te ard limiter in te second layer is set to 5 since te convex polyedron is bounded by 5 bounding yper-planes (z to z 5 ). Nested convex decision region (or convex recursive deletion region) problems ave been solved by a constructive algoritm via two-layer perceptrons [5]. Next, we give some definitions necessary for te paper. A DJRNC polyedron D is a polyedron obtained by dis-jointly removing a series of convex polyedrons (C, C 2,, C n ) from an original convex polyedron (C 0 ), given by D= C0 C C 2 C n (2) were C 0 (te original convex polyedron) is called te minimum containing convex polyedron (MCCP) of D; C 0, C,, and C n are called te removed convex polyedrons (RCPs) of D. Te word dis-jointly means tat any two of te RCPs do not ave any common bounding yper-planes except for te bounding yper-planes of te MCCP. Fig. 2(a) sows a DJRNC polyedron (D) in a single DJRNC decision region (defined in te next section). Fig. 2(b) sows te MCCP and te tree RCPs of te DJRNC polyedron (RCP, RCP2, and RCP3). Fig. 2(c) demonstrates te bounding yper-planes of te MCCP and te tree RCPs. Te 0 sides and sides of tese bounding yper-planes are also displayed in Fig. 2(c). 2. Model and Feasibility 2. Single DJRNC decision region Model A single DJRNC decision region is a region containing only one DJRNC polyedron. Fig. 2 (a) is also an example of a single DJRNC decision region. A single DJRNC decision region can be implemented by a tree-layer perceptron were all of te parameters (weigts and tresolds) are constants. In te tree-layer perceptron, te first layer serves to form te decision region, te second layer detects weter a pattern resides in te MCCP or a particular RCP, and te tird layer makes te final classification. Eac node in te second layer is associated wit te MCCP or a particular RCP wit one-to-one correspondence. We use te same notation to present a node of te second layer and its corresponding MCCP or RCP. Te weigts of te second layer are all s. Eac of te tird layer weigts is determined as follows: if it is connecting te MCCP node wit te tird layer node, te weigts is set to ; if it is connecting any of te RCPs nodes wit te tird layer node, te weigt is set to. Te tresold for a particular node of te second layer is equal to te number of te bounding yper-planes of te MCCP or an RCP associated wit te particular node. Te tresold ( θ ) for te tird layer node (te output node) is. 2.2 Feasibility We prove te implementation feasibility by considering te following cases: () If te pattern doesn t reside in te MCCP, te final classification result is 0 (class B). (2) If te pattern resides in te DJRNC polyedron, only te MCCP node produces a. Te final classification result is (class A). (3) If te pattern resides in a particular RCP, bot te MCCP and te particular RCP produce s. Te weigt connecting te MCCP node wit te tird layer node is, and te weigt connecting te particular RCP wit te tird layer node is. Te sum of tem is 0. Terefore te final classification is 0 (class B). Fig. 3 is te tree-layer perceptron to implement te single DJRNC decision region sown in Fig. 2 (a). 3. Conclusions We proposed a neural network model to solve te DJRNC decision regions using multi-layer perceptrons. In te proposed multi-layer perceptrons, all weigts and te parameters of te activation functions are pre-determined wen a DJRNC decision region is establised. No constructive algoritm is needed for implementing te above tree decision regions. Eac weigt determined by

3 Proceedings of te 8t WSEAS International Conference on Neural Networks, Vancouver, Britis Columbia, Canada, June 9-2, tis paper is eiter or. We presented tree cases to discuss, in practical, te capabilities of multi-layer perceptrons on DJRNC decision regions including single, nested, and disconnected DJRNC decision regions. Tere are two papers proposed in te series of studies including part A and part B. In tis paper (part A), we defined single DJRNC decision regions and presented te implementing multi-layer perceptrons for te DJRNC decision regions. We also proved te implementation feasibility for te single DJRNC decision region. References [] C. Lin, Partitioning Capabilities of Multi-layer Perceptrons on Nested Rectangular Decision Regions Part I: Algoritm, WSEAS Transactions on Information Science and Applications, Issue 9, Volume 3, September, 2006, pp [2] C. Lin, Partitioning Capabilities of Multi-layer Perceptrons on Nested Rectangular Decision Regions Part II: Properties and Feasibility, WSEAS Transactions on Information Science and Applications, Issue 9, Volume 3, September, 2006, pp [3] C. Lin, Partitioning Capabilities of Multi-layer Perceptrons on Nested Rectangular Decision Regions Part III: Applications, WSEAS Transactions on Information Science and Applications, Issue 9, Volume 3, September, 2006, pp [4] C. Lin, A Constructive Algoritm to Implement Celled Decision Regions Using Two-Layer Perceptrons, WSEAS Transactions on Information Science and Applications, Issue 9, Volume 3, September, 2006, pp [5] J. Makoul, A. El-Jaroudi, R. Scwartz, Partitioning Capabilities of Two-layer Neural Networks, IEEE Trans. on Signal Processing, Vol. 39, No. 6, 99, pp [6] R. P. Lippmann, An Introduction to Computing wit Neural Nets, IEEE ASSP Mag., Vol.4, 987, pp [7] R. Sonkwiler, Separating te Vertices of N-cubes by Hyperplanes and its Application to Artificial Neural Networks, IEEE Trans. on Neural Networks, Vol. 4, No. 2, 993, pp [8] C. Lin, A. El-Jaroudi, An Algoritm to Determine te Feasibilities and Weigts of Two-Layer Perceptrons for Partitioning and Classification, Pattern Recognition, Vol. 3, No., 998, pp [9] C. Cabrelli, U. Molter, R. Sonkwiler, A Constructive Algoritm to Solve Convex recursive deletion (CoRD) Classification Problems via Two-layer Perceptron Networks, IEEE Trans. On Neural Networks, Vol., No 3, 2000, pp [0] W. Fan, L Zang, Applying SP-MLP to Complex Classification Problems, Pattern Recognition Letters, 2, 2000, pp [] V. Deolalikar, A Two-layer Paradigm Capable of Forming Arbitrary Decision Regions in Input Space, IEEE Trans. On Neural Networks, Vol. 3, No., 2002, pp [2] G. Huang, Y Cen, H. A. Babri, Classification Ability of Single Hidden Layer Feedforward Neural Networks, IEEE Trans. On Neural Networks, Vol., No. 3, 2000, pp [3] S. Dragici, Te Constraint Based Decomposition (CBD) Training Arcitecture, Neural Networks 4, 200, pp

4 Proceedings of te 8t WSEAS International Conference on Neural Networks, Vancouver, Britis Columbia, Canada, June 9-2, z =0 z z = z 5 z 5 =0 z 5 = z z 5 z 3 z 3 = Convex polyedron C z 3 =0 z 4 = z 4 z 4 =0 Te input space of te convex decision region z 3 z 4 (a) Te convex decision region. y Te second layer node (output node) tresold (θ ) = 5 Te second layer weigts z z 3 z 4 z 5 Te first layer nodes Te first layer weigts x x 2 Te inputs (b) Te two-layer perceptron to implement te convex decision region sown in (a). Fig. : An illustrative example of a convex decision region implemented by a two-layer perceptron.

5 Proceedings of te 8t WSEAS International Conference on Neural Networks, Vancouver, Britis Columbia, Canada, June 9-2, DJRNC polyedron D Te input space of te single DJRNC decision region (a) Te single DJRNC decision region RCP DJRNC polyedron D RCP2 RCP3 Te MCCP (te solid lines) (b) Te MCCP and te tree RCPs (RCP, RCP2, and RCP3) of DJRNC polyedron D. z z =0 z = =0 = z 6 = z 6 z 6 =0 z RCP z 7 = z z 7 =0 7 z 8 =0 z8 = z 8 DJRNC polyedron D z 5 z 5 = z 5=0 RCP2 z 9 = z 9 =0 z 9 z 5 z 3 z 3 = z 3 =0 z 0 z z 0 =0 z 0 = RCP3 z 3 z =0 z = z 4 (c) Te bounding yper-planes z 4 = z 4 =0 z 4 Fig. 2: An illustrative example of a single DJRNC decision region

6 Proceedings of te 8t WSEAS International Conference on Neural Networks, Vancouver, Britis Columbia, Canada, June 9-2, y Te 3 rd layer node (output node),θ = Te 3 rd layer weigts MCCP θ = 5 RCP θ = 3 RCP2 θ = 3 RCP3 θ = 3 Te 2 nd layer nodes Te 2 nd layer weigts z z 3 z 4 z 5 z 6 z 7 z 8 z 9 z 0 z Te st layer nodes Te st layer weigts x x 2 Te inputs Fig. 3: Te tree-layer perceptron to implement te single DJRNC decision region sown in Fig. 2(a).