Memory Convergence and Optimization with Fuzzy PSO and ACS

Transcription

1 Joural of Computer Sciece 4 (2): , 2008 ISSN Sciece Publicatios Memory Covergece ad Optimizatio with Fuzzy PSO ad ACS 1 Subhash Chadra Padey ad 2 Dr. Pueet Misra 1 Birla Istitute of Techology, Extesio Cetre: Naii, Allahabad, , Idia 2 Departmet of Computer Sciece, Luckow Uiversity, Luckow, Idia Abstract: Associative eural memories are models of biological pheomea that allow for the storage of patter associatios ad the retrieval of the desired output patter upo presetatio of a possibly oisy or icomplete versio of a iput patter. I this study, we itroduce fuzzy swarm particle optimizatio techique for covergece of associative eural memories based o fuzzy set theory. A Fuzzy Particle Swarm Optimizatio (FPSO) cosists of clusterig of swarm s particle by applyig fuzzy c-mea algorithm to attai the eighborhood best. We preset a sigular value decompositio method for the selectio of efficiet rule from a give rule base required to attai the global best. Fially, we illustrate the proposed method by virtue of some examples. Further, at coloy system ACS algorithm is used to study the Symmetric Travelig Salesma Problem TSP. The optimum parameters for this algorithm have to foud by trial ad error. The ACS parameters workig i a desiged subset of TSP istaces has also bee optimized by virtue of Particle Swarm Optimizatio PSO. Key words: Artificial eural etwork, covergece, particle swarm optimizatio, Fuzzy c-mea INTRODUCTION A artificial eural etwork (ANN) is a aalysis paradigm that is a simple model of the brai ad the back-propagatio algorithm is the oe of the most popular method to trai the artificial eural etwork. Recetly there have bee sigificat research efforts to apply evolutioary computatio techiques for the purposes of evolvig oe or more aspects of artificial eural etworks. The efficiet supervised traiig of feedforward eural etworks (FNNs) is a subject of cosiderable ogoig research ad umerous algorithms proposed to this ed. The back propagatio (BP) algorithm [1] is oe of the most commo supervised traiig methods. Although BP traiig has proved to be efficiet i may applicatios, its covergece teds to be slow ad yields to suboptimal solutios [2]. Attempts to speed up traiig ad reduce covergece to local miima have bee made i the cotext of gradiet descet [3,4,5]. However, these methods are based o variable weight, learig rate, step size ad bias to dyamically adapt BP algorithm ad use a costat gai for ay sigmoid fuctio durig its traiig cycle. Evolutioary computatio methodologies have bee applied to three mai attributes of eural etworks: etwork coectio weights, etwork architecture (etwork topology, trasfer fuctio) ad etwork learig algorithm. Particle swarm optimizatio (PSO) is a populatio based stochastic optimizatio techique [6,7] ispired by social behavior of bird flockig or fish schoolig. This is modeled by particles i multidimesioal space that have a positio ad a velocity. These particles are flyig through a hyperspace ad have two essetial reasoig capabilities: the memory of their ow best positio ad kowledge of the swarm s best, best simply meaig the positio with the smallest objective fuctio value. Members of a swarm commuicate good positios to each other ad adjust their ow positio ad velocity based o good positios. There are two mai ways this commuicatio is doe: A global best that is kow to all ad immediately updated whe a ew best positio is foud by ay particle i the swarm. A Neighborhood best where each particle oly immediately commuicates with a subset of the swarm about best positios. I this study we have desiged a algorithm usig a Particle Swarm Optimizatio (PSO) framework, to optimize the parameters of the ACS algorithm workig Correspodig Author: Subhash Chadra Padey, Departmet of Computer Sciece, Birla Istitute of Techology, Extesio Cetre: Naii, Allahabad, Idia

2 o a sigle Symmetric Travellig Salesma Problem (TSP) istace. For each istace the algorithm computes a optimal set of ACS parameters, their performace o all istaces (ot oly their related istace) ad the characteristics of their related istace for the purpose of fidig correlatios. The first ACO algorithm, called At-System, was proposed i [8-10]. A full review of ACO algorithms ad applicatios ca be foud i [11]. ACS is a versio of the At System that modifies the updatig of the pheromoe trail [12,13]. We have chose this ACS algorithm to work with because of the theoretical backgroud we have foud o it [11,12] ad the previous fie-tuig research o the parameters by [14]. We have chose PSO because it has a easy implemetatio for iteger ad real parameters ad, as geetic algorithms, it performs a blid search o all the possible sets of parameters. I our algorithm the domai of the PSO will be all possible sets of parameters for ACS. For a positio of a particle we compute the fitess by ruig the ACS algorithm with the parameters give by the positio o a TSP istace. THE PARTICLE SWARM OPTIMIZATION PSO s precursor was a simulator of the social behavior that was used to visualize the movemet of a birds flock. Several versio of the simulatio model were developed, icorporatig cocepts such as earest eighbor velocity matchig ad acceleratio by distace [6,15]. Two variats of the PSO algorithm were developed, Oe with a global eighborhood ad aother with local eighborhood [16]. Suppose that the search space is D-dimesioal ad the the i th particle of the swarm ca be represeted by a D-dimesioal vector X i = (x i1, x i2,...,x id ). The velocity (positio chage) of this particle ca be represeted by aother D-dimesioal vector V i = (v i1, v i2,...,v id ). The best previously visited positio of the i th particle is deoted as P i = (p i1, p i2,...,p id ). Defiig g as the idex of the best particle i the swarm (i.e., g th particle is the best) ad let the superscript deote the iterative umber, the the swarm is maipulated accordig to the followig two equatios [6] : Z 1 Z C r( p X ) C r ( p X id id + 1 id id + 2 id id ) + = (1) X 1 1 X + + Z id id id + = (2) where, d = 1,2,...D, i = 1,2,...,N ad N is the size of the swarm, C is a positive costat called, acceleratio costat r 1, r 2 are the radom umbers, uiformly J. Computer Sci., 4 (2): , distributed i [0,1], ad = 1,2,,determies the iteratio umbers. Equatios 1 ad 2 defie the iitial versio of the PSO algorithm. Sice there was o actual mechaism for cotrollig the velocity of a particle, it was ecessary to impose a maximum value V max o it. If the velocity exceeded this threshold, it was set equal to V max. This parameter is proved to be crucial, because large values could results i the particles movig past good solutios, while small values could result i isufficiet exploratio of the search space. This lack of cotrol mechaism for the velocity resulted i low efficiecy for PSO. Various attempts have bee made to improve the performace of the base lie PSO with varyig success. I [16] emphasis has bee give o optimizig the update equatios for the particles. Some researcher used a selectio mechaism i a attempt to improve the geeral quality of the particles i swarm. I [6] cluster aalysis techique is used to modify the update equatio, so that particles attempt to cofirm to the cetre of their clusters rather tha attemptig to coform to a global best. The aforemetioed problem was addressed by icorporatig a weight parameter for the previous velocity of the particle. Thus i the latest versio of the PSO, Eq. 1 ad 2 are chaged to the followig oes [17] : X 1 1 X W C r X ) C r (p X Z 2 ) id + + = X + (p id id + id id id 2 id id X 1 1 X id + + = X + id id I our proposed model both the approaches have bee cosider together. First, we clustered the swarm by applyig fuzzy c-mea algorithm to attai the eighborhood best ad the we reduce the umber of rules required to attai the global best by virtue of sigular value decompositio method. NEIGHBORHOOD BEST USING FCM The Fuzzy C-Meas algorithm geeralizes the hard C-meas algorithm to allow a particle of swarm to partially belog to a multiple clusters. Therefore, it produces a soft partitio for a give swarm. To do this, the objective fuctio J of hard c-meas has bee exteded i two ways Fig 1 ad 2. The fuzzy membership degrees i clusters were icorporated ito the formula ad: A additioal parameter P is itroduced as a weight expoet i the fuzzy membership.

3 f(x) Weights of Fuzzy c-meas, p= Theorem: A costraied fuzzy partitio (C 1, C 2,...,,C k ) ca be a local miimum of the objective fuctio J oly if the followig coditios are satisfied: ( ) 1 / p k ci (X ) 1 X V i / X V j (3) j= 1 µ = p p ( ) ( ) (4) V = µ (x) x / µ (x) i ci ci x X x X Based o this theorem, FCM updates the prototypes ad the membership fuctio iteratively usig 3 ad 4, util a covergece criterio is reached. The algorithm of FCM ca be described as: FCM (X, c, m, ε) x Fig.1: Weight of the FCM algorithm for p = 3 f(x) Weights of Fuzzy c-meas, p= x Fig.2. Weight of the FCM algorithm for p = 2 The exteded objective fuctio, deoted J, is k 2 J(P, V) = p ( µ c (Xk ) Xk Vi i= 1X k X where, P is a fuzzy partitio of the swarm X formed by C 1, C 2,...,,C k. The parameter p is a weight that determies the degree to which partial members of a cluster effect the clusterig result. 141 X: A ulabeled swarm size C: The umber of clusters to form p: The parameter i objective fuctio ε: A threshold for the covergece criteria. Iitial prototype V = {v 1, v 2,...,v c } Repeat V previous V Compute membership fuctio usig 4 Update the prototype, v i i usig 3 Util c V previous V ε i = 1 i i GLOBAL BEST USING SVD I our proposed model, after clusterig the particles of swarm, orthogoal trasformatio method is used for selectig importat fuzzy rules from a give rule base [18-21]. Ulike covetioal methods where multiple iteratios are usually required to fid optimal umber of fuzzy rules, orthogoal trasformatio methods are a o iterative procedure. Therefore, orthogoal trasformatio methods are computatioally less expesive compared to the covetioal methods especially whe the umbers of particles i the swarm are too large. I this sectio we itroduce how to use sigular value decompositio (SVD) to select the most importat fuzzy rules from a give rule base ad costruct compact fuzzy models with better geeralizatio ability. Sigular value decompositio takes a rectagular - by-p matrix A, i which the rows represets the gees ad the colums represets the experimetal coditio [22]. The SVD theorem states: A = U S V T p p p p

4 Where T U U = I J. Computer Sci., 4 (2): , 2008 Similarly, we obtai the followig: T V V I p p = (i.e. U ad V are orthogoal) S = diag (σ 1, σ 2, σ 3,..,σ m ) R p (m = mi{,p}) is a diagoal matrix with σ 1 σ 2 σ 3. σ m 0. The colums of U are the left sigular vectors has sigular values ad is diagoal (mode amplitudes), ad V T has rows that are the right sigular vectors (expressio level vectors). I the basic priciple of usig SVD for fuzzy rule selectio, we ca use fuzzy model with costat cosequet costituets as a example. This type of fuzzy model, which is usually referred to as the zero order TSK model, has the followig form [23]. R i: If x 1 is A i1 ad x 2 is A i2 ad adxm is A im The y is C i, i = 1,2,..,M. Where, C i is the costat costituets. The total output of the model is computed by. M M = i i i i 1 i 1 Y w c / w where, w i is the matchig degree. The SVD starts with a oversized rule base ad the remove redudat or less importat fuzzy rules through a oe pass operatio. Fially the efficiet rule obtaied is obeyed by all the swarm s cluster to approach the global best. Now we will illustrate the method by takig a example. Suppose we are give a swarm of size six particles, each of which has two features F 1 ad F 2.We list the particle i give Table 1. Assumig that we wat to use FCM to partitio the swarm i two clusters [23], suppose we set the parameter p i FCM at 2 ad the iitial prototypes to v 1 = (5,5) v 2 = ( 10,10 ). The iitial membership fuctios of the two clusters are calculated usig 3: c2 (x 1 ) = 1/[(68/58)+(68/68)] = c1 (x 2 ) = 1/[(17/17)+(17/37)] = c2 (x 2 ) = 1/[(37/17)+(37/37)] = c1 (x 3 ) = 1/[(68/68)+(68/18)] = c2 (x 3 ) = 1/[(18/68)+((18/18)] = c1 (x 4 ) = 1/[(36/36)+(36/26)] = c2 (x 4 ) = 1/[(26/36)+(26/26)] = c1 (x 5 ) = 1/[(53/53)+(53/13)] = c2 (x 5 ) = 1/[(13/53)+(13/13)] = c1 (x 6 ) = 1/[(82/82)+(82/52)] = c2 (x 6 ) = 1/[(52/82(+(52/52)] = Therefore, usig three prototypes of the two clusters, the membership fuctio idicates that x 1 ad x 2 are more i the first cluster, while the remaiig particles i the swarm are more i the secod cluster. The FCM algorithm the updates the prototypes accordig to 4. V 1 = 6 k=1 ( ci (x k )) 2 x k / 6 k=1 ( ci (x k )) 2 = [ (2.12) (4.9) (7.13) (11.5) (12.7) (14.4)]/[ ] = [(7.2761/1.0979), (10.044/1.0979)] = (6.6273, ) V 2 = 6 k=1( c2 (x k )) 2 x k / 6 k=1( c2 (x k )0 2 [ (2.12) (4.9) (7.13) (11.5) (12.7) (14.4)]/[ ] = [(22.326/2.2928), ( /2.2928)] = (9.7374, ) The update prototype V 1, as show i Fig. 3, is ci (x) =1 / 2 j=1( x 1 -v i / x 1 - v j ) 2 x 1 -v 1 2 = = = 58 x 1 -v 2 2 = = = 68 c1 (x 1 ) = 1/ [(58/58) + (58/68)] Table 1: A swarm to be partitioed F 1 F 2 X X2 4 9 X X X X Fig.3 A example of Fuzzy c-mea Algorithm 142

5 moved closer to the ceter of the cluster formed by X 1, X 2 ad X 3, while the updated prototype V 2 is moved closer to the cluster formed by X 4, X 5 ad X 6. Now we illustrate how to solve for SVD to obtai efficiet rule for approachig the global best, let s take the example of the matrix. A = For a matrix W, the ozero vector X is the eigevector of W if: WX = λx, λ is the eigevalue of A ad X is the eigevector of A correspodig to λ. So to fid the eigevalues of the etity we compute matrices AA T ad A T A. The eigevectors of AA T make up the colums of U so we ca do the followig aalysis to fid U. AA T = = 1 3 X1 = 0.82 ad X2 = ad X3 = X4 = 0 (this is the first colum of the U matrix). Combiig these we obtai: U = Similarly, we ca fid the value of V 2 4 T A A = 1 3 Similarly, we obtai the expressio V = Fially, the S is square root of the eigevalues from AA T or A T A ad ca be obtaied directly givig us: Sice WX = λx, the (W-λΙ)X = 0. Hece 20 λ λ X = ( W λ I ) X = 0 λ 0 0 λ Thus, from the solutio of characteristic equatio, we obtai λ = 0, λ = 0, λ = , λ = This value ca be used to determie the eigevector that ca be placed i the colums of U. Thus, we obtai the followig equatios X1 +14X2 = 0, 14X X2 = 0, X3 = 0, X4 = 0 Upo simplifyig the first two equatios we obtai a ratio which relates the value of X1 ad X2. The values of X1 ad X2 are chose such that the elemets of S are the square roots of the eigevalues. Thus a solutio that satisfies the above equatio X1 = ad X2 = 0.82 ad X3 = X4 = 0 (this is the secod colum of the U matrix). Substitutig the other eigevalues we obtai: X1+14X2 = 0, 14X X2 = 0, X3 = 0, X4 = 0. Thus a solutio that satisfies this set of equatios is S = It is obvious that σ 1 >σ 2 >σ 3. This is what the study was idicatig. ANT COLONY SYSTEM The ACS works as follow, it has a populatio of ats. Let deote for each arc e = (i, j) i the TSPistace graph a iitial heuristic value η e ad a iitial pheromoe value τ e is origially set to the iverse of the cost of traversig the edge e. τ e is iitially set to τ 0 = 1\L for all edge e, where L is equal to the iverse of the tour legth computed by the earest-eighborheuristic algorithm. Let q 0, α, ρ [0,1], be real values ad ϕ,β iteger values betwee 0 ad 8. For each vertex s V a eighbor set is defied amog the earest vertices, N(s). For a give at r, let NV(r) be the set of o-visited vertices. We deote j r (s) = N (s) NV (r) the set of o-visited vertices amog the eighbour set for a give vertex s ad a give at r.

6 I every iteratio, each at costructs a tour solutio for the TSP-istace. The costructios phase works as follows: Each at is iitially set i a radomly vertex, the at each step the etire ats make a movemet to a ovisited vertex. Give a at r with a actual positio (vertex)s, p krs is computed as a referece value for visitig or ot vertex k, where: [ τ( s,k )] ϕ[ η( s,k )] β if k j( s) p [ ( s,u )] [ ( s,u )] = τ ϕ η β (5) krs u j( s) 0 otherwise This formula icludes a small modificatio respect to the origial ACS algorithm icludig ϕ as expoet of the pheromoe level, this will allow us a deeper research o the effects of the possible combiatios of, ϕ, β parameters. A sample radom value q is computed. If q q 0 we visit the city k V with maximum p krs (exploitatio of the kowledge) otherwise ACS follows a radom - proportioal rule o p krs for all k V (biased exploratio). If there are o o-visited vertex o the eighbor of vertex s, we exted 5 to all vertices i NV (r)/n (s) (those ot visited by at r ad ot icluded i the eighbor of s) ad visits the vertex with maximum p krs. After a arc is iserted ito a route (a ew vertex is visited), its pheromoe trail is updated. This phase is called Local update ad for a iserted e E: τ = ( 1 α ) τ + ατ (6) e e 0 This reduces the pheromoe level i the visited arcs ad the exploratio i the set of possible tours is icreased. Whe all the tours have bee computed a global update phase is doe. For each edge e pertaiig to the global-best-tour foud: τ = ( 1 α ) τ + α τ (7) e e e τ 1/ L e = gb (8) iteratios are fiished, is the best solutio foud by the trial. A feasible set of parameters for ruig ACS is a combiatio of feasible q 0, ϕ, β, α, ρ umber of ats (a) ad a cocrete eighbor defiitio. PSO-ACO ALGORITHM AND IMPLEMENTATION The algorithm is ru each time o a sigle TSPistaces. The set of parameters of ACS that defie a poit i the PSO domai are q 0, ϕ, β, φ, α, ρ ad the umber of ats (a). Most of them have already bee explaied. Let φ deotes the percetage of vertices that will be icluded i to N (v) for ay vertex v V, so for a give v V ad φ = 0.5 N (v) = [φ* V ]. The rages of each parameter are show i Table 2, where each parameter pertais to its related] miimum, maximum]. DPSO = ]0,1]x]-1,8]x]-1,8]x]0,1]x]0,1]x]0,1]x]0,40] R 7. is the domai of the PSO. We defie the fitess value of a give positio (poit) as the legth of the best tour computed by a ACS usig the related parameters i the give istace. If comparig two differet positios they have the same legth value the computig time is cosidered. We cosider better of those parameters that miimize the legth of the tour ad secodly the time of computig. For computig the fitess of a give positio, first iteger parameters (a, ϕ ad β) are rouded up as show i Fig. 4, secodly the algorithm rus five trails of the ACS algorithm usig the rouded parameters i the TSP-istace ad returs the best value obtaied from the trails. For each particle of the populatio its iitial velocity is set radomly. For half of the populatio the iitial positio is set usig predefied parameters assurig that for every parameter there will be a Table 2: Rage of acs parameters q 0, ϕ, β, ρ, α, φ, a are the parameters used i ACS q 0 ϕ β ρ α φ a Miimum Maximum Poit i the PSO domai (reflects a set of parameters) ( ) where, L gb is the legth of the global-best-tour foud. I the origial at algorithm ad i most of the later versios, pheromoe global update is performed i all the edges; ACS oly updates pheromoe level i the set of edges pertaiig to the best tour. We cosider a trial as a performace of 1000 iteratios. The lowest legth tour foud after all 144 Modified values to ru o ACS: ( ) Fig. 4: Modificatio of PSO poits. I bold are the modified values.

7 particle cotaiig a value coverig the full rage. The positios of the other half of the iitial populatio is set radomly. Parameters for the PSO have bee set followig [6,8] C1 = C2 = 2, χ = 0.729, the iertia weight is set iitially to 1 ad gradually decreasig from (at each PSO iteratio w = 0.99w). Maximum umber of iteratios is set to 500 due to computig time costraits (for 1000 PSO-iteratios more tha 1 day of computig time was ecessary). The algorithm PSO-ACS pseudo-code is as follows: Select TSP-Istace. Iitialize particles. Do 500 For all the set of particles Positio_Fitess PF = INFINITE Do 5 Perform a trial ACS with particle parameters. If New Value < PF PF = New value Ed if Ed Do Ed for Compute w = 0.99w Update Best Parameters Foud by each Particle Update Best Parameters Foud by the Populatio Compute Velocity Movemet of Particles Ed Do Retur the set of parameters related to the best tour legth foud ad the tour legth. The algorithm is based i a PSO framework, where particles are iitialized ad iteratively are movig though the domai of the set of parameters. The goal of the algorithm is, for a give istace to compute the tour with lowest legth ad to compute the set of ACS parameters, amog those i DPSO, which gets the best ACS performace. Those fial parameters are related with the TSP-istace selected. RESULTS AND ANALYSIS Algorithm was coded i C++. Algorithm has bee ru o six of the most widely used TSP-istaces. Computatioal results are give i four parts: PSO- ACS behavior, PSO-ACS optimum values obtaied, best set of parameters ad compariso amog sets of parameters performace. Computatioally, each PSO - ACS iteratio shows a clear covergece: whe the optimum (defied by the algorithm) umber of ats ad φ are early fixed, the J. Computer Sci., 4 (2): , 2008 Fig. 5: Average time of the swarm at first 100 iteratios for istaces eil101 a ad rat 99 b. AVG_TIME is average time of the iteratio give a fixed umber of particles ad ITERATION_PSO is the umber of the iteratio i the PSO-ACS algorithm Table 3: Sets of parameters α β ρ ϕ a q 0 φ Fitess P_eil P_eil P_eil P_kroA P_kroB P_rat ACS a b ACS_GA a b a have bee tested for φ= b there is o fitess value related. Values i bold mea optimum. Computatioal time is also fixed (Fig. 5). I less tha 100 iteratios algorithm computes a optimum for iteger parameters ad i 200 iteratios there are small differeces amog the optimum foud ad the particle s positio for real parameters. I Fig. 6 we ca see the evolutio of the algorithm i the first 100 iteratios. For the average of the fitess of the swarm(i a give iteratio),there is a decreasig global tedecy ad after iteratio 75 we ca see the average of the fitess is kept o a fixed rage, the size of this rage is variable as show i ( c) ad (d). For the miimum value obtaied by the swarm i a give iteratio, computatioal results show that at the begiig there are icreasig ad decreasig phases, whe the particles are explorig their local optimums ad movig also to the global oe, but ear iteratio 100 the miimum is maitaied as i (a) or frequetly visited as i (b). This fast covergece ca be a advatage as well as a drawback because it ca lead to a fast o-desirable covergece. We set the reasos of this fast covergece i the PSO framework used ad maily i the method for evaluatig a set of parameters: i a stochastic algorithm there is the probability that a bad set of parameters could perform well, if all the particles move ito this 145

8 (a) (b) (c) (d) Fig. 6: First 100 iteratios of the PSO-ACS algorithm. A ad b are related to the miimum tour obtaied at each iteratio, c ad d are related to the average of the particles fitess values. A ad d are related to the istace kroa 100. b is related to krob 100, ad c to eil51. Those are the examples of typical behaviors i the 100 first iteratios of the algorithm area ad the umber of iteratios i this area icreases leadig to probably good solutios that will cause the algorithm to remai i this o-optimal area. Table 3 shows the optimum set of parameters foud ruig PSO-ACS o each oe of the istaces selected. CONCLUSION I this research, a ew approach is proposed for the covergece of associative eural memories by usig the Fuzzy Particle Swarm Optimizatio techique (FPSO). The approach focuses o the eighborhood best ad global best to icrease the speed of covergece. I additio, this proposed model overcomes the local miima problem which is major drawback with the PSO techique. The example illustrated suggests that our ew approach ca be used successfully as real time memory covergece techique for the artificial eural etwork. 146 Computatioal results seem to show that there is o uiquely optimal set of ACS parameters yieldig best quality solutios i all the TSP istaces. Nevertheless the PSO-ACS has bee able to fid a set of ACS parameters that work optimally for a majority of istaces ulike others kow so far. PSO-ACS algorithm works well across differet istaces because it adapts itself to the istace characteristics. But it has a high computatioal overhead. A future work will try to modify the algorithm framework to reduce this cost. PSO-ACS also has a fast covergece that ca lead to a bad set of parameters. This may be due to two reasos: first is the specific PSO framework used ad i modifyig it we expect to obtai better results. Secodly the way the sets of parameters are evaluated may have to be reviewed as a bad set of parameters could lead to a o-desired covergece.

9 REFERENCES 1. Armij, L., Miimizatio of fuctios havig Lischitz cotiuous first partial derivatives. Pacific J. Math., 16: Baladi ad K. Horik, Neural etworks ad pricipal compoet aalysis: Learig from examples ad local miima. Neural Networks, 2: Bello., M.G., Ehaced traiig algorithms ad itigrated traiig/architecture selectio for multi layer perceptro etworks. IEEE Tras. Neural Networks, 3: Atsaklis, P.J ad J.O. Moody, The depedece idetificatio eural etwork costructio algorithm. IEEE Tras. Neural Networks, 1: Baro Robert, H. ad P. Coughli James, Neural Computatio i Hopfield Networks ad Boltzma Machies. Newyark: Uiversity of Delaware Press. 6. Keedy, J. ad R. C.Eberhart, Particle swarm optimizatio. Proceedig IEEE Iteratioal Coferece o Neural Networks. 4: Eberhart, R.C. ad Y. Shi, Particle swarm optimizatio: Developmet, applicatios ad resources. Proceedig Cogress o Evolutioary Computatio, IEEE Service Cetre, Korea. 8. Dorigo, M., Optimizatio, Learig ad Natural Algorithms. Ph.D Thesis. Dip Elettroica. Politecico di Milao. 9. Dorigo, M., V. Maiezzo ad A. Colori, Positive Feedback as a Search Strategy. Techical Report. Dip Elettroica. Politecico di Milao. pp: Dorigo, M., V. Maiezzo ad A. Colori, The At System: Optimizatio by a Coloy of Cooperatig Agets. IEEE Trasactio o Systems, Ma ad Cyberetics. pp: Dorigo, M. ad T. Stutzle, At Coloy Optimizatio. MIT Press. 12. Dorigo, M. ad L.M. Gambardella, At Coloy System: A Cooperative Learig Approach to the Travellig Salesma Problem. IEEE Trasactio o Evolutioary Computti. pp: Gambardella, L.M ad M. Dorigo, At-Q: A reiforcemet Learig Approach to the Symmetric ad Asymmetric Travellig Salesma Problems. Proceedig of the IEEE Iteratioal Coferece o Evolutioary Computatio. pp: Pilat, E.C. ad T. White, Usig geetic algorithms to optimize ACS-TSP. Proceedigs of the Third Iteratioal Workshop o At Algorithms. pp: Keedy, J. ad R. Eberhart, Swarm Itelligece. Morga Kaufma Publishers. 16. Shi, Y. ad R. Eberhart, Parameter selectio i particle swarm optimizatio. IEEE Iteratioal Coferece o Evolutioary Computatios. Lecture otes i Computer Sciece. Spriger. pp: Shi, Y. ad R. Eberhart, A modified particle swarm optimizer. IEEE Iteratioal Coferece o Evolutioary Computatio. Achorage, Alaska. 18. Mouzouris, G.C. ad J.M. Medel, Desigig fuzzy logic systems for ucertai eviromets usig a sigular-value- QR decompositio method. Proceedigs of the 5th IEEE Iteratioal Coferece o Fuzzy Systems, New Orleas. pp: Ye, J. ad L. Wag, A SVD based fuzzy model reductio strategy. Proceedig of the 5th IEEE Iteratioal Coferece o Fuzzy System, New Orleas. pp: Ye, J ad L. Wag, Graule-based Models. Hadbook of fuzzy computatio. IOP Publishig. 21. Ye, J. ad L. Wag, Simplifyig fuzzy rulebased models usig orthogoal trasformatio methods. IEEE Trasactio o Systems, ma ad Cyberatics. 29: Alter, O.O. Brow ad D. Botstei, Sigular value decompositi for geome-wide expressio data processig ad modelig. Proc. Natl Acad Sci USA. 23. Ye, J. ad R. Lagari, Fuzzy logic: Itelligece, cotrol ad iformatio. Pearso Editio. 147