An Efficient Algorithm of Polygonal Fuzzy Neural Network 1

Transcription

1 An Efficient Algorithm of Polygonal Fuzzy Neural Network PUYIN LIU Department of Mathematics National University of Defense Technology Changsha 40073, Hunan, P R CHINA Abstract: Symmetric polygonal fuzzy number is employed to build an improved feedforward fuzzy neural network (FNN) First, a novel fuzzy arithmetic and extension principle for the polygonal fuzzy numbers is derived Second, the topological architecture of a three layer feedforward FNN is presented, and the input-output law of this network is systematically studied Third, a fuzzy BP learning algorithm for the polygonal fuzzy number connection weights and thresholds of the FNN is developed Finally a simulation example is illustrated to analyze the adaptive three layer feedforward FNN to realize data pairs consisting of real numbers and symmetric polygonal fuzzy numbers, approximately Keywards: algorithm n-polygonal fuzzy number; Fuzzy arithmetic; Feedforward fuzzy neural network; Fuzzy BP Introduction The research on FNN has attracted many scholars attention [, 3 8, 0] Recent years have witnessed a surge of interest in the combination of neural network and fuzzy theory The achievements in the field have found useful in many applied areas, such as pattern recognition [4], knowledge engineering [8], system identification [4, 9], clustering [5] and so on One of major points at issue in FNN field is learning algorithms for the structure parameters, including connection weights and thresholds of FNN Two major problems in learning for fuzzy parameters related to FNN are the derivative of the fuzzy function in which some fuzzy operators are included [0], and the error function by which the error between the desired values and real outputs of FNN is measured [4, 8] As some recent results with regard to this subject, Ishibuchi et al in [4] develop a BP learning algorithm for the FNN s whose weights and thresholds can be triangular or trapezoidal fuzzy numbers Li et al [5] present a similar BP type learning scheme of FNN s based on fuzzy number operations and level sets of fuzzy numbers To use gradient information of the error function, they employ the following derivatives of functions: { (x a), x a, x 0, x < a; (x a) x {, x a, 0, x > a However, Above formulas are only valid for x a If x a, they are no longer valid Based on these two derivative formulas, the chain rules for differentiation are only in form, and lack rigorous mathematical sense To solve this problem, authors in [0] develop an efficient approach to define differentiation for functions including and operations Thus, it is possible to develop learning algorithms for FNN s whose input, connection weights are general fuzzy number, based on gradient information In the paper, to generalize triangular or trapezoidal fuzzy numbers, we introduce symmetric polygonal fuzzy numbers, which can provide with approximation of general fuzzy numbers with arbitrary accuracy [8] By modifying the internal operations of FNN s, we build a novel FNN with symmetric polygonal fuzzy inputs and weights, which can be universal approximator of increasing fuzzy functions [7] A new fuzzy BP learning algorithm for the FNN is developed based on rigorous mathematical sense [0] Our approach is illustrated by a simulation of realizing given fuzzy pattern pairs Let us now introduce some notations used in the study By R we denote the set of all real numbers, and N the collection of natural numbers Define F bc (R) the set of fuzzy numbers with the following conditions: for each A F bc (R), (i) α (0, ], α cut Aα R is a nonempty bounded and closed interval; (ii) supp( A) {x R A(x)>0} is bounded; (iii) If supp( A) [a 0, a 0], ker( A) [e 0, e 0], then A( ) is strictly increasing on [a 0, e 0], and strictly decreasing on [e 0, a 0] From now on, by A0 we denote support supp( A), obviously A0 is a bounded and closed interval and let A0 [a 0, a 0], then we can easily show that A( ) is right continuous and strictly increasing on [a 0, e 0], left continuous and strictly decreasing on [e 0, a 0] d H (A, B) for A, B R (A, B ) is the Project supported by grants for National Natural Science Foundation of China (No ; No )

2 Hausdorff metric between A, B So [a, b], [c, d] R d H ([a, b], [c, d]) a c b d If A, B F bc (R), define the metric D( A, B) as follows [4]: D( A, B) (d H ( Aα, Bα)) It may easily α [0,] prove by [] that (F bc (R), D) is a metric space Define the Euclidean metric d E ([a, b], [c, d]) as follows: d E ([a, b], [c, d]) { (a c) + (b d) } For intervals [a, b], [c, d] R, we may easily obtain d H ([a, b], [c, d]) d E ([a, b], [c, d]) d H ([a, b], [c, d]), ie the metrics d E and d H are equivalent Polygonal Fuzzy Number Triangular and trapezoidal fuzzy numbers are widely applied in many real fields [3 5] As the generalizations of these fuzzy numbers we in the section introduce symmetric polygonal fuzzy numbers, whose structures and properties are similar with ones of triangular and trapezoidal fuzzy numbers Definition Let A F 0 (R) If there exist n N, and a 0, a,, a n, a n, a n,, a 0 R : a 0 a n a n a 0, so that (i) supp( A) [a 0, a 0], ker( A) [a n, a n]; (ii) k {,, n}, A( ) is linear on [a k, a k ] and [a k, a k ], respectively; (iii) Let k {,, n} Then we have a k < a k, A(a k ) k n, A(a k 0) k n, a k < a k, A(a k ) k n, A(a k + 0) k n ; where A (a k 0) is left limit lim t 0+ A (a k t), and A(a k + 0) is right limit lim A(a t 0+ k + t) A is called n symmetric polygonal fuzzy number, and denote A ( [a n, a n]; a 0,, a n, a n,, a 0) The collection of all n symmetric polygonal fuzzy numbers is denoted by Fbc tn (R) Specifically, let n, then an symmetric polygonal fuzzy number is triangular or trapezoidal A( ) n n n n x a a n a n a a 0 a 0 Fig Curve of n-symmetric polygonal fuzzy number If A is an n-symmetric polygonal fuzzy number, A is shown as Fig Fbc tn (R) means the set of all n-symmetric polygonal fuzzy numbers, and Fbc tn(r +) is the set of all non-negative elements in Fbc tn (R), that is, A (x) 0 if x < 0 for all A Fbc tn(r +) For given A ([a n, a n]; a 0,, a n, a n,, a 0), B ([b n, b n]; b 0,, b n, b n,, b 0) Fbc tn (R), we have D( A, B) n {d H ([a i, a i ], [b i, b i ])}; i0 A B i 0,,, n, b i a i a i b i () In Fbc tn (R) we define fuzzy arithmetic operations +,, and some extensions of monotone function σ : R R respectively as follows: Let A ([a n, a n]; a 0,, a n, a n,, a 0) Fbc tn(r), B ([b n, b n]; b 0,, b n, b n,, b 0) Fbc tn(r) Then A + B ( [a n + b n, a n + b n]; a 0 + b 0,, a n +b n, a n +b n,, a 0+b0) ; A B ( [a n b n, a n b n]; a 0 b 0,, a n b n, a n b n,, a 0 b0) ; A B ( [c n, c n]; c 0,, c n, c n,, c0), () where c i, c i (i 0,,, n) are determined by the interval arithmetic []: [c i, c i ] [a i, a i ] [b i, b i ] For σ : R R +, we define when σ is increasing, σ( A) ( [σ(a n), σ(a n)]; σ(a 0),, σ(a n ), σ(a n ),, σ(a 0) ) ; when σ is decreasing, σ( A) ( [σ(a n), σ(a n)]; σ(a 0),, σ(a n ), σ(a n ),, σ(a 0)), (3) Let a R, A Fbc tn (R),we call a A or A a the scalar product of a and A: when a 0, a A ( [aa n, aa n]; aa 0,, aa n, aa n,, aa0) ; when a < 0, a A ( [aa n, aa n]; aa 0,, aa n, aa n,, aa0) (4) By Eqs () (4),Fbc tn (R) is closed under the extension operations +,, and the scalar product And +, are identical with the ones based Zadeh s extension principle [9] Compared with ones of Zadeh s fuzzy multiplication and extension, the calculations of or extension based on Eq (3) are strikingly simple Take as a example: let A ([0, ];, 05, 5, ), B ([ 05, 05];,,, ) (5)

3 n n y A( ) B( ) A B a 0 a a a a a 0 b 0 b b b b b 0 c 0 C( ) c c c Fig The multiplication A B of A and B x c c 0 Then C A B ([ 05, 05]; 4, 5, 5, 4), which is shown in Fig 3 Three-layer Feedforward Fuzzy Neural Network In the section, we characterize the three-layer feedforward FNN with one input neuron and one output neuron In this network, the internal operations are based on Eqs () (4); the input and output neurons are linear; and there is a nonlinear operation which is an activation function σ (for example the Sigmoidal function σ(x) / ( + e x ), x R) in the hidden neurons Fig 3 gives the topological structure of a three-layer feedforward FNN whose input X in F tn bc (R +), output Y, and weights U i, V i and threshold Θi in Fbc tn (R) The input-output(i/o) relationship of the FNN is X Y Γ( X) p j h U j V j σ( U j X + Θj) (6) h V j h j p Input layer Hidden layer Output layer Fig 3 Three-layer feedforward FNN Given I/O pairs ( X, Y ),, ( Xm, Y m), our aim is to realize the I/O relationships by learning connection weights U j, V j and threshold Θj (j,, p) For j,, p; k,, m, let U j ( [u n(j), u n(j)]; u 0(j),, u n (j), u n (j),, u 0(j) ) ; V j ( [vn(j), vn(j)]; v0(j),, vn (j), vn (j),, v0(j) ) ; Θj ( [θn(j), θn(j)]; θ0(j),, θn (j), θn (j),, θ0(j) ) ; Xk ( [x n(k), x n(k)]; x 0(k),, x n (k), x n (k),, x 0(k) ) ; Y k ( [yn(k), yn(k)]; y0(k),, yn (k), yn (k),, y0(k) ) (7) The FNN shown in Fig 3 may conveniently handle triangular fuzzy information [4], further, by the fact Y that the space n N F tn bc (R) is dense in F bc(r) [8], this network also has the ability to process the general fuzzy data We at first study the I/O laws of FNN Theorem Let p N, U j, V j, Θj F bc tn (R) (j,, p) X ([x n, x n]; x 0,, x n, x n,, x 0), then Γ( X) p j V j σ( U j X + Θj) ( [γ n(x n, x n), γ n(x n, x n)]; γ 0(x 0, x 0),, γn (x, x n ), γn (x n, x n ),, γ0(x 0, x 0) ) (8) where γi k(x i, x i ) may be represented as follows for i 0,,, n : [ γ i (x i, x i ), γ i (x i, x i [ )] {v i (j)σ(s i (j))} {v i (j)σ(s i (j))}), j {v i (j)σ(s i (j))} {v i (j)σ(s i (j))})] (9) j ri (j) min{ u i (j)x i, u i (j)x i, u i (j)x i, } u i (j)x i, ri (j) max{ u i (j)x i, u i (j)x i, u i (j)x i, } u i (j)x i, s i (j) r i (j) + θ i (j), s i (j) r i (j) + θ i (j) (0) If X F tn bc (R +), Eq (0) will be a simple form: s i (j) min{u i (j)x i, u i (j)x i } + θ i (j), s i (j) max{u i (j)x i, u i (j)x i } + θ i (j) Assume that X degenerate to the real number x R, then s i (j), s i (j) are determined as follows: s i (j) min{u i (j)x, u i (j)x} + θ i (j), s i (j) max{u i (j)x, u i (j)x} + θ i (j), moreover let x R +, consequently s i (j) u i (j)x + θi (j), s i (j) u i (j)x + θ i (j), and Eq (9) becomes as [γi (x i, x i ), γ i (x i, x i [ )] {v i (j)σ(s i (j))} {v i (j)σ(s i (j))}), j {v i (j)σ(s i (j))} {v i (j)σ(s i (j))})] j 4 Learning Algorithm Two key steps to design learning algorithm for the connection weights of feedforward FNN are to define the derivatives of the function in which fuzzy operators or are included, and to derive the cost (error) function of fuzzy output from output neuron and the corresponding fuzzy target (desired value) [8, 0] To this end we write lor( ) : R R, lor(x), x > 0,, x 0, 0, x < 0 3

4 To develop a self-adaptive learning algorithm, the operation laws for derivatives of functions will first be presented Proposition [0] Let the real functions f, g be differentiable on R, h f g, h f g Then h, h are almost everywhere differentiable on R, moreover dh (x) d(f(x) g(x)) lor(f(x) g(x)) df(x) + lor(g(x) f(x))dg(x) ; dh (x) d(f(x) g(x)) lor(f(x) g(x)) dg(x) Especially if a R,the following hold: d(a f(x)) d(a f(x)) + lor(g(x) f(x))df(x) lor(f(x) a) df(x), lor(a f(x)) df(x) Let Ok be real output of output neuron when input is Xk, ie Ok Γ( Xk) (k,, m) Our goal is to make Ok approximate even equal Y k for each k,, m by adjusting V k, U k and Θk Let Ok, Y k F tn bc (R) defined respectively as ([o n(k), o n(k)]; o 0(k),, o n (k), o n (k),, o 0(k)), ([y n(k), y n(k)]; y 0(k),, y n (k), y n (k),, y 0(k)) The second step for designing the algorithm is to define the error function E By Eq (7) and the equivalence of metrics d H and d E, E is defined as E m ( D E Ok, ) Y k m n { ( de k( ) [o i (k), o i (k)], [y i (k), y i (k)])} i0 () Obviously E 0 k,, m, Ok Y k Since symmetric polygonal fuzzy number weights are specified by their finite parameters, we can develop some update rules respectively for these parameters Finally the polygonal fuzzy numbers can be reconstructed respectively by the updated parameters Theorem Let E be error function defined as Eq (), and for k,, m; i 0,,, n; j,, p, write t i (k) o i (k) y i (k), t i (k) o i (k) y i (k); Γ i (j, k) σ (s i (j, k))v i (j)lor(v i (j)), Γ i (j, k) σ (s i (j, k))v i (j)lor( v i (j)), Γ 3 i (j, k) σ (s i (j, k))v i (j)lor( v i (j)), Γ 4 i (j, k) σ (s i (j, k))v i (j)lor(v i (j)); i (j, k) ( {u i (j)x i (k)} {u i (j)x i (k)}) ( {u i (j)x i (k)} {u i (j)x i (k)}) ; i (j, k) ( {u i (j)x i (k)} {u i (j)x i (k)}) ( {u i (j)x i (k)} {u i (j)x i (k)}) ; () Then for each i 0,,, n; j,, p; l,, the following partial derivative formulas hold: (i) vi l(j) m ( t l i (k) lor( vi l(j))σ(s3 l i (j, k))+ k +lor(vi l(j))σ(sl i ); ( (j, k)) (ii) u l i (j) m ( t i (k)γ i (j, k) + t i (k)γ3 i ) (i, j, k) ( k lor( i (j, k))lor(( ) l+ x i (k))x i (k)+ +lor( i (j, k))lor(( ) l+ x i (k))x i )+ ) (k) + (t i (k)γ i (j, k) + t i (k)γ4 i (j, k) ( lor( i (j, k))lor(( ) l x i (k))x i (k)+ ) ) +lor( i (j, k))lor(( ) l x i (k))x i (k) ; (iii) θi l(k) m ( t i (k)lor( ( ) l+ vi (j)) vi (j)+ k +t i (k)lor( ( ) l vi (j)) vi )σ (j) (s l i (j, k)) By the gradient descent method, we can build the iteration laws of the fuzzy weights and thresholds of the FNN as Eq (6) Let t be iteration step, η > 0 be the learning rate, and α > 0 be the momentum constant Write for i 0,,, n; j,, p; l,, a l i (j) vl i (j)[t] η v vi l(j) +α l i (j)[t] vl i[t ], b l i (j) ul i (j)[t] η u u l i (j) +α l i (j)[t] ul i[t ], c l i (j) θl i (j)[t] η θ θi l(j) +α l i (j)[t] θl i[t ], (3) where vi l[t ] vl i [t] vl i [t ], ul i [t ] ul i [t] u l i [t ], θl i [t ] θl i [t] θl i [t ] For each j,, p, re-array the sets {a 0(j),, a n(j), a n(j),, a 0(j)}, {b 0(j),, b n(j), b n(j),, b 0(j)}, {c 0(j),, c n(j), c n(j),, c 0(j)} as {a (j),, a n+ (j)}, {b (j),, b n+ (j)} and {c (j),, c n+ (j)} respectively in such a way that the following inequalities hold: a (j) a (j) a n+ (j) a n+ (j); b (j) b (j) b n+ (j) b n+ (j); c (j) c (j) c n+ (j) c n+ (j) (4) Define V j [t+], U j [t+], Θj [t+] F tn bc (R) : V j[t+] ([ v n(j)[t+], v n(j)[t+] ] ; v 0(j)[t+],, v n (j)[t+], v n (j)[t+],, v 0(j)[t+] ), U j[t+] ([ u n(j)[t+], u n(j)[t+] ] ; u 0(j)[t+],, u n (j)[t+], u n (j)[t+],, u 0(j)[t+] ), Θj[t+] ([ θ n(j)[t+], θ n(j)[t+] ] ; θ 0(j)[t+],, θ n (j)[t+], θ n (j)[t+],, θ 0(j)[t+] ), 4

5 respectively as follows, i 0,,, n, vi (j)[t+] a i+(j), vi (j)[t+] a n+ i(j); u i (j)[t+] b i+(j), u i (j)[t+] b n+ i(j); θi (j)[t+] c i+(j), θi (j)[t+] c n+ i(j) (5) Thus, we obtain fuzzy weights V j [t+], U j [t+] Fbc tn(r) and fuzzy threshold Θj[t+] Fbc tn (R) (j,, p) when the iteration step is t + The learning algorithm determined by Eqs (3) (5) is called the fuzzy BP algorithm We may realize the algorithm by the following steps: Step Initialize the fuzzy weights and thresholds Choose randomly the initial fuzzy weights and threshold: V j[0], U j[0], Θj[0] (j,, p) And let t 0 Step For j,, p, according to Eqs (3) (5), let V j[t] ([ v n(j)[t], v n(j)[t] ] ; v 0(j)[t],, v n (j)[t], v n (j)[t],, v 0(j)[t] ) ; U j[t] ([ u n(j)[t], u n(j)[t] ] ; u 0(j)[t],, u n (j)[t], u n (j)[t],, u 0(j)[t] ) ; Θj[t] ([ θ n(j)[t], θ n(j)[t] ] ; θ 0(j)[t],, θ n (j)[t], θ n (j)[t],, θ 0(j)[t]) Step 3 For j,, p; i 0,,, n; l,, complete the following value assignment: v l i(j)[t] v l i(j), u l i(j)[t] u l i(j), θ l i(j)[t] θ l i(j) Step 4 For k,, m; j,, p; i 0,,, n; l,, calculate t l i (k), sl i (j, k), i(j, k), i (j, k) and Γ q i (j, k) (q,, 4) by Eqs () (4), respectively Step 5 By Theorem, compute / vi l(j) at vi l(j)[t], / θi l(j) at θl i (j)[t], and / u l i (j) at u l i (j)[t], respectively By Eqs (3) (5) we obtain vi l(j)[t+], θl i (j)[t+] and ul i (j)[t+] for j,, p; i 0,,, n; l, Step 6 Discriminate analysis: either t > M or E < ε, go to Step 7; otherwise let t t +, go to Step Step 7 Stop and output V j [t], Θj [t], U j [t] for j,, p, and Ok (k,, m) In Step 6, M is a given upper-bound of iteration steps and ε is an error bound 5 Simulation Example In the section, the proposed fuzzy BP algorithm is demonstrated by computer simulations on a numerical example The proposed approach is employed to realize approximately a given fuzzy valued function that maps a real number in [0, ] to a symmetric polygonal fuzzy number in Fbc tn (R) (n ) We choose the upper-bound of iteration steps M as 0 3, and the error bound ε 0 In the hidden layer of FNN there are four hidden neurons, that is, p 4 The activation function is chosen as σ : R R, σ(x) /(+e x ) for x R Obviously this is a continuously differentiable function By Eq () we have, i (j, k) i (j, k) 0 and Γ i (j, k) σ (u i (j)x(k) + θ i (j))v i (j)lor(v i (j)), Γ i (j, k) σ (u i (j)x(k) + θ i (j))v i (j)lor( v i (j)), Γ 3 i (j, k) σ (u i (j)x(k) + θ i (j))v i (j)lor( v i (j)), Γ 4 i (j, k) σ (u i (j)x(k) + θ i (j))v i (j)lor(v i (j)), u i (j) m x(k) ( t i (k)γ i (j, k)+t i (k)γ3 i (j, k)), k u i (j) m x(k) ( t i (k)γ i (j, k)+t i (k)γ4 i (j, k)), k vi (j) m t i (lor( v (k) i (j))σ(u i (j)x(k)+ k θi (j)) + lor(v i (j))σ(u i (j)x(k) + θ i ), (j)) vi (j) m k t i (k) (lor( v i (j))σ(u i (j)x(k)+ +θi (j)) + lor(v i (j))σ(u i (j)x(k) + θ i ), (j)) θi (k) m ( t i (k)lor(v i (j))v i (j)+ k +t i (k)lor( v i (j))v i )σ (j) (u i (j)x(k) + θ i (j)), θi (k) m ( t i (k)lor( v i (j))v i (j)+ k +t i (k)lor(v i (j))v i )σ (j) (u i (j)x(k) + θ i (j)), where i 0,, ; j,, 4 and let m 5, so k,, 5 In Eq (9), when X x(k) R +, then Theorem implies that γi l(x(k), x(k)) ol i (k) for i 0,,, n; l, So by Eqs (9)(0) it follows that o i (k) p ( {v i (j)σ(s i (j, k))} { vi (j)σ(s i (j, k))}), j o i (k) p ( {v i (j)σ(s i (j, k))} { vi (j)σ(s i (j, k))}), j s l i (j, k) ul i (j)x(k) + θl i (j) (l, ) For given fuzzy patterns ( x(), Y ),, ( x(5), Y 5 ) as Tab, where Y,, Y 5 belong to Fbc tn (R) (n ) Five pairs of real inputs and fuzzy outputs shown in Fig 4 are our training samples Xk Tab Training patterns Y k x() 0 Y ([0, 05]; 005, 007, 07, 08) x() 04 Y ([035, 04]; 0, 03, 045, 06) x(3) 06 Y 3 ([04, 05]; 03, 04, 05, 06) x(4) 08 Y 4 ([05, 055]; 04, 045, 06, 07) x(5) 0 Y 5 ([07, 08]; 06, 065, 085, 09) 5

6 With the training data as given in Tab, we can train the three-layer feedforward FNN according to the fuzzy BP algorithm by executing iteratively Step to Step 7 After trained, the real I/O relationship of the FNN can be established Tab shows the real outputs of the symmetric polygonal fuzzy numbers with respect to the respective input values These pairs of real inputs and fuzzy number outputs are shown in Fig 4 Xk Tab Real input output patterns Ok x() 0 O ([008, 0]; 005, 006, 07, 09) x() 04 O ([03, 04]; 06, 08, 047, 06) x(3) 06 O3 ([039, 048]; 03, 038, 05, 056) x(4) 08 O4 ([05, 057]; 04, 046, 06, 07) x(5) 0 O5 ([07, 08]; 057, 064, 088, 095) Fig6 demonstrates the relational curve of the energy E with respect to iteration t when t,,, 000 It is not difficult to see that the instantaneous mode adaption indeed minimize the energy function E defined as Eq() k k k3 k4 k5 Fig 4 Desired outputs ( ) and real outputs ( ) In this example, the iteration process shows that the convergence rate of the fuzzy BP algorithm is moderate, which is shown in Fig 5 However the convergent speed is sensitive to the variations of the values of learning rate η and momentum constant α From Fig 4 we may see that the error bound ε is guaranteed Learning error E Iteration step Fig 5 The variation curve of E 6 Conclusions By modifying the internal operations, this paper defines a novel FNN model by which general fuzzy information may be handled Our study generalizes Y k O k the conclusions in [4, 5] The simulation example demonstrates the efficiency of our approach The further problems related to the subject are as follows: First, prove in theory the convergence of the fuzzy BP algorithm; Second, develop some improved algorithms, in which the learning rate η and the momentum constant α may be adaptively adjusted, with quicker convergence speed and minimal computation complexity; Finally generalize the polygonal fuzzy numbers to more general cases References: [] J J Buckley and Y Hayashi, Fuzzy neural networks: A survey, Fuzzy Sets and Systems, Vol 66, No, 994, pp 3 [] P Diamond and P Kloeden, Metric spaces of fuzzy sets, Singapore: World Scientific Press, 994 [3] T Feuring and W M Lippe, The fuzzy neural network approximation lemma, Fuzzy Sets and Systems, Vol, 0, No, 999, pp 7 36 [4] H Ishibuchi and M Nii, Numerical analysis of the learning of fuzzified neural networks from fuzzy if then rules, Fuzzy Sets and Systems, Vol 0, No 3, 00, pp [5] Zhenquan Li, V Kecman and A Ichikawa, Fuzzified neural network based on fuzzy number operations, Fuzzy Sets and Systems, Vol 30, No 3, 00, pp [6] Puyin Liu, On the approximation realization of fuzzy closure mapping by multilayer regular fuzzy neural networks, Multiple Valued Logic, Vol 5, No, 000, pp [7] Puyin Liu, A novel fuzzy neural network and its approximation property, Science in China (Series F), Vol 44, No 3, 00, pp [8] Puyin Liu and Hongxing Li, Efficient learning algorithms for three-layer regular feedforward neural networks, IEEE Trans on Neural Networks, Vol 5, No, 004, pp 88-0 [9] H T Nguyen, A note on the extension principle for fuzzy set, J Math Anal Appl, Vol 64, No 5, 978, pp [0] X H Zhang, C C Huang and S H Tan, et al, The min-max function differentiation and training of fuzzy neural networks, IEEE Transon Neural Networks, Vol 7, No 6, 996, pp