STABILITY OF SIMULTANEOUS RECURRENT NEURAL NETWORK DYNAMICS FOR STATIC OPTIMIZATION

Transcription

1 STABILITY OF SIMULTANEOUS RECURRENT NEURAL NETWOR DYNAMICS FOR STATIC OPTIMIZATION Grsel Serpen and Yifeng X Electrical Engineering and Compter Science Department, The University of Toledo, Toledo, OH 4366 USA Abstract A new trainable and recrrent neral optimiation algorithm, which has potentially sperior capabilities compared to existing neral search algorithms to compte high qality soltions of static optimiation problems in a comptationally efficient manner, is stdied. Specifically, local stability analysis of the dynamics of a relaxation-based recrrent neral networ, the Simltaneos Recrrent Neral networ, for static optimiation problems is presented. The reslts of theoretical as well as its correlated simlation stdy lead to the conjectre that the Simltaneos Recrrent Neral networ dynamics appears to demonstrate desirable stability characteristics. Dynamics often converge to fixed points pon conclsion of a relaxation cycle, which facilitates adaptation of weights throgh one of many fixed-point training algorithms. The trainability of this neral algorithm reslts relatively high qality soltions to be compted for large-scale problem instances with comptational efficiency, particlarly when compared to soltions compted by the Hopfield networ and its derivative algorithms inclding those with stochastic search control mechanisms. I. INTRODUCTION Combinatorial optimiation is a field where crrently there are nmeros effective algorithms drawn from varios fields of science and engineering, among which most notable inclde heristic search within the domain of Artificial Intelligence, and Operations Research. Relatively large-scale problem instances were sccessflly addressed for near-optimal soltions by these non-neral algorithms. However, as the problem sie grows, the comptation time of a near-optimal soltion by almost all of these algorithms increases prohibitively. Artificial neral networs as a recent arrival in the field of combinatorial optimiation offers nparalleled scaleability for real-time comptation of soltions, and hence can deliver soltions in constant time for all practical prposes if implemented in hardware. This premise is inherently lacing in non-neral approaches de to facts that they lac any significant amont of parallelism and their implementation is realied on architectres that can offer large-grain parallelism at best. Hopfield family algorithms inclding the Boltmann Machine and the Mean-Field Annealing networ are significant neral algorithms for static optimiation [, 4, 7, 5]. The Simltaneos Recrrent Neral networ (SRNN was empirically demonstrated to offer sperior qalities to address large-scale instances of static optimiation problems compared to the Hopfield and its stochastic derivatives [5, 6,, 4]. Accordingly, a qestion that natrally follows is how SRNN is different than Hopfield family neral search algorithms to facilitate sch comptational capability. In a typical search scenario, a Hopfield networ and its derivatives wold be randomly initialied and relaxed towards a fixed point. If the fixed point the networ converges to is not a soltion, then another relaxation period with fresh and randomly initialied neron otpts is started discarding any experience that may have been associated with the prior relaxation. On the other hand, the SRNN is able to leverage the experience following an nsccessfl relaxation for the next one throgh its learning process. The SRNN individally adapts each and every weight in the networ sing a well-defined error fnction (in order to associate soltions of the problem with the fixed points of the networ dynamics in a spervised mode [4]. Extensive simlation stdies were performed to solve classical static optimiation problems inclding the Traveling Salesman problem (TSP and the Graph Path Search problem (GPSP with the Simltaneos Recrrent Neral networ [6,, 4]. Initially, the SRNN was configred as a relaxation networ and trained with the recrrent bacpropagation (RBP algorithm [2, 8, 2, 6]. Simlation reslts demonstrated that the SRNN was able to locate high qality soltions for instances of the TSP and GPSP p to 6 cities/vertices. The comptational cost of simlating SRNN/RBP algorithm on serial compting architectres was relatively significant yet considerably less than what it wold cost if sing a Hopfield family neral optimier [4]. Frther improvements for the SRNN by replacing the recrrent bacpropagation with the resilient propagation (RPROP algorithm broght abot very promising reslts inclding frther drastic improvements in the qality of soltions with no observable increase in the comptational cost [5]. In smmary, SRNN demonstrated two important advantages over what the Hopfield family algorithms cold offer: comptational ability to address large-scale static optimiation problems and, secondly, to locate high qality soltions for sch problems at a comparable, if not better, comptational cost.

2 The same simlation stdies also indicated throgh empirical means that the networ dynamics demonstrated globally asymptotically stable behavior in general. Specifically stable eqilibrim points were rotinely observed in the state space of the nonlinear networ dynamics. It wold be very comforting to identify a Liapnov fnction to theoretically validate the indications of the empirical stdy, which sggests stable nonlinear dynamics for the SRNN. However, this tas is not trivial and reqires mch trial-and-error. In the absence of a Liapnov fnction, sefl insight can be developed throgh a local stability analysis of the SRNN dynamics. A mathematical insight into stability properties of SRNN nonlinear dynamics is needed to assess its comptational promise and this paper presents sch a stdy. II. SIMULTANEOUS RECURRENT NEURAL NETWOR A mathematical characteriation of the comptation performed by a Simltaneos Recrrent Neral Networ (SRNN is given by [7] ˆ = f ( x, W, where x and W are the external inpts and weights, respectively and ẑ is the eqilibrim vale of following a relaxation cycle: ( n ˆ = lim, n which can be compted by the following iteration ( n+ ( n = f (, x,w, where f is a feedforward networ and n is the iteration index. Figre presents the high level topology of the SRNN. x FIGURE I SIMULTANEOUS RECURRENT NEURAL NETWOR f(,w The networ is provided with the external inpts and initial otpts, which are typically assmed randomly in the absence of a priori information. The otpt of previos iteration is fed bac to the networ along with the external inpts to compte the otpt of next iteration. The networ is allowed to iterate ntil it reaches a stable eqilibrim point assming at least one exists. External inpts are applied throghot the complete relaxation cycle. When a stable eqilibrim point is reached, the ( n+ otpts stop changing (i.e. the otpt vale of is (n almost eqal to or very close to. It is important to note that the feedbac from the otpt layer to inpt layer in SRNN is not delayed: the feedbac is, theoretically speaing, simltaneos. An SRNN exhibits complex temporal behavior: it follows a trajectory in the state space to relax to a fixed point. One relaxation of the networ consists of one or more iterations of otpt comptation and propagation along the feedforward and feedbac paths ntil the otpts converge to a stable eqilibrim vale. III. DYNAMIC SYSTEM ANALYSIS OF SRNN A two-layer topology for the SRNN was shown to be sfficient to represent soltions of problems in the domain of graph search problems inclding those in combinatorial optimiation i.e., the Traveling Salesman Problem (TSP. These two layers will be called the hidden layer and the otpt layer, where the hidden layer is a one-dimensional array of nodes and the otpt layer is a two-dimensional array of nodes[4]. This optimiation specific topology derives from the assmption that a mlti-layer continosdynamics perceptron networ with one hidden layer and no external inpts is instantiated for the feedforward mapping in Figre, which coincidentally reslts in the minimal strctre for SRNN. Symbols U and V will be employed to represent the and weight matrices associated with forward and bacward signal propagations, respectively. A. Local Stability Analysis for SRNN Dynamics Important nowledge abot the eqilibrim points for the dynamics of an SRNN can be dedced throgh a characteriation of the set of eqilibrim points and lineariation of the SRNN dynamics abot these eqilibrim points. More specifically, reqirements on the weight matrices to maintain stability of those eqilibrim points of interest for a given problem can be derived. This nderstanding is also liely to help infer conditions on the weight pdate rle nder a fixed-point training algorithm for SRNN dynamics. Let y = [ y y ] T and = [ ] T 2 y represent the node otpt vectors for the hidden and otpt layers, respectively, while the internal state of the otpt layer nerons is given by the vector s. Derivation of eqilibrim points for the SRNN dynamics will reqire an eqation of the form =, where is the vector of neron otpts in the otpt layer, which typically represents soltions for a given optimiation problem. This eqation reqires the 2

3 neron dynamics in the otpt layer to be represented by partial differential eqations. A minimm complexity model wold dictate these eqations to be of first-order and employ ideal integrators. A similar neron model was shown to address a set of important deficiencies of the neral networs composed of nerons employing lossy integrators []. However, comptational model for the nerons in the hidden layer can tilie continos perceptron eqations for the sae of simplicity and withot sacrificing generality. Hence, neron dynamics of SRNN are given by ds = f ( s with = dt for the nodes in the otpt layer, and j= y j = f ( net j with net j = v = for the nodes in the hidden layer, while noting that j y j j y j is the otpt of j-th neron in hidden layer, is the otpt of -th neron in otpt layer, is the node cont in hidden layer, is the nmber of nerons in the otpt layer, is the forward weight from j-th neron in hidden layer to - th neron in otpt layer, v is the bacward weight from j -th neron in otpt layer to j-th neron hidden layer, and f is continos, differentiable fnction typically a sigmoid with a steep slope (for combinatorial optimiation problems. For convenience, the activation fnction is instantiated as nipolar sigmoid with slope coefficient + λ R the same for both the hidden and otpt layers withot loss of generality. According to the chain rle, we have = s s for =, 2,,. Sbseqent to a series of algebraic maniplations, the matrix form of above eqation maps to [7]: = λ ( ( λ i = /( + e λ i = /( + e vi i v i i j ( Eqilibrim points of the networ dynamics are the soltions of the eqation determined by setting the vector of partial derivatives in Eqation to ero. Upon setting the leftmost matrix to a ero vector with appropriate dimensions, it is noted that if = or =, then ( = for all. Therefore, Eqation shows that the corners of the nit hypercbe are eqilibrim points in the state space of the SRNN dynamics. A second set of eqilibrim points can be obtained by solving the eqation that is obtained by setting the remaining part of above eqation to ero. Since the activation fnction of the nerons is assmed nipolar sigmoid, the otpt is continos. Soltions, which are also the eqilibrim points of SRNN dynamics, can be any points within the nit hypercbe in the otpt space. Assme that the high-gain mapping is employed for the sigmoid fnction, effectively forcing otpts of nerons to acqire limiting vales pon convergence. For a nipolar high-gain sigmoid fnction, the neron otpts are forced towards vales approaching. or.. In that case, the eqilibrim point set will inclde mostly (bt not necessarily only corners of the hypercbe. Next, the SRNN dynamics will be linearied at eqilibrim points, which inclde the corners of the hypercbe in the state space of the SRNN dynamics, to determine the stability properties of the same eqilibrim points. In order to derive the eigenvales at a corner of the hypercbe for the linearied dynamics, the SRNN dynamics are linearied arond a given hypercbe corner. be sch vertex with [ ] T Let = 2, = or, for =,2,,. Near the area of the corner of the hypercbe, consider a small deviation from by an amont sch that = +, for =,2,,. Sbseqently, it follows that eigenvales of the acobian matrix of the linearied dynamics are given by [7] j γ = λ( 2, (2, j= λ v jii i= + e where =,2,, and = or. The reqirement for linearied dynamics of the SRNN to be stable at a hypercbe corner is that all the eigenvales of the acobian matrix evalated at that eqilibrim point (hypercbe corner be negative:

4 γ < for =, 2,,., Referring to Eqation 2, we obtain T ( 2 y < for =,2,,, where = [ ] T and y is the otpt vector 2 of the hidden layer while noting that λ R +. In more specific terms, following conditions shold be satisfied in order to establish the stability of this eqilibrim point: and T j y < or < when j= λv jii + e i= i T j y > or > when j= + e λ v jii + v j i= i = =, where =, 2,,. The reslt is a set of simltaneos ineqalities, where the nnowns are weights for the forward and bacward signal propagations, and the soltion is non-trivial in order to determine bonds on these weights. However, the insight gained as a reslt of this analysis leads s to the following theorem. The theorem sggests the feasibility of establishing a given hypercbe corner in the state space of the SRNN dynamics as a stable eqilibrim point de to existence of real weight matrices U and V, which satisfy the set of ineqalities above for the given hypercbe corner. Theorem: Assme the SRNN dynamics is configred for static combinatorial optimiation: a two-layer topology, no external inpt, and high-gain sigmoid mapping for otpt layer nerons. Then, for a given hypercbe corner, there exist real weight matrices U and V to establish that hypercbe corner as a stable eqilibrim point in a local sense in the state space of the SRNN dynamics. Proof: A detailed derivation of the proof is presented in [8]. The proof for this theorem provides valable insight into the process of training the SRNN dynamics configred for static combinatorial optimiation sing a fixed-point learning algorithm. It demonstrates, with relative ease, the feasibility of associating the set of hypercbe corners that typically represent soltions of a combinatorial optimiation problem with stable eqilibrim points of the networ dynamics as the fixedpoint training algorithm searches throgh the weight space. The proof frther sggests hypercbe corners can easily be made stable eqilibrim points in the state space of SRNN dynamics throgh a relatively nconstrained selection of weights, which was also observed throghot a comprehensive set of simlation stdies [5, 6,, 4]. These reslts sggest that the SRNN as a combinatorial optimiation algorithm is robst with respect to changes in the adaptable parameters, namely weights, for its stability characteristics and therefore, facilitating reasonably nconstrained application of a learning algorithm inclding the recrrent bacpropagation as well as standard bacpropagation and its comptationally-efficient derivatives QicProp and resilient propagation [4, 3]. Some frther thoghts on global stability characteristics of the SRNN are detailed in the next section to demonstrate striing similarities among the SRNN, the Bidirectional Associative Memory (BAM, and the Adaptive Resonance Theory (ART networ cores. B. Global Stability and Liapnov Fnction for SRNN It is easy to show that Bidirectional Associative Memory (BAM is a special case of the SRNN, where the operational mode is associative memory and the feedforward networ for the SRNN is a single layer of perceptrons [9]. A BAM networ consists of two layers of nodes, labeled as a and b, and two weight matrices, M and M T, to facilitate the signal flow from nerons in one layer to another and vice versa. Nerons in a given layer do not interact among themselves, and employ bipolar threshold (or very high-gain sigmoid fnction for their otpts. A Liapnov fnction in the form of E T ( a, b = a b exists for the dynamics of a BAM networ and is valid for any real matrix M. T Consider setting U = V, where the forward weight matrix is eqal to the transpose of the bacward weight matrix, for the SRNN configred for static optimiation, which then leads to a typical BAM architectre. Frthermore, following simplifying assmptions are needed to transform an SRNN to a BAM networ: The forward mapping bloc in Figre is a twolayer feedforward perceptron networ, Perceptrons employ very high gain sigmoid mapping fnctions with bipolar otpts, and The external inpt for every neron/perceptron is ero. These observations sggest BAM, for which a Liapnov fnction exists to demonstrate global stability, is a special case of SRNN. Along similar lines, the Adaptive Resonance Theory & 2 (ART and ART2 neral networs are shown to possess a Liapnov fnction [3]. There exists a similarity between

5 the topological strctres and node dynamics of ART series networ cores and the SRNN configred for static optimiation. Given this similarity, well-established stability reslts for the ART series networs cold possibly pave the way to a more mathematically sond nderstanding and exploration of the stability properties of the SRNN dynamics. An empirical observation based on extensive simlation analysis performed on the TSP and graph search problems indicated that the SRNN is stable: it tends to converge to a fixed point starting with random initial vales for the weight matrices and neron otpt vales. Frthermore, the same simlation stdies also demonstrated that the application of the recrrent bacpropagation algorithm to train the weights does not appear to lead to instability in general. The SRNN algorithm is biologically inspired althogh it is most liely a very high-level and crde mathematical model at this stage [7]. However, this biological connection is an asset and motivating factor for the existence of a Liapnov fnction for the SRNN. These observations collectively form a sond platform to motivate the exploration of a Liapnov fnction for the SRNN. IV. SIMULATION STUDY The theorem presented earlier sggests that there exist real weight matrices U and V that establish a given hypercbe corner as a stable eqilibrim point in the state space of the nonlinear networ dynamics. The simlation stdy explores specific and niformly-sampled instances of U and V matrices in the agmented U-V space and assesses the set of stable eqilibrim points for a given instance of the U and V weight matrices with particlar emphasis on hypercbe corners as being sefl entities to represent soltions of combinatorial optimiation problems. If sfficient nmber of instances for the U and V matrices are generated throgh niform sampling for a given SRNN networ topology, it is reasonable to expect that the cmlative set of stable eqilibrim points encontered wold inclde the complete set of hypercbe corners. Ths, this finding wold validate the claim made by the theorem indicating that for a given hypercbe corner, U and V matrices exist to mae that hypercbe corner stable eqilibrim point. Consider a SRNN with the maximally redced topology for combinatorial optimiation: a single hidden layer with nodes, nodes in the otpt layer, and no external inpt. A high-gain nipolar sigmoid activation fnction with a steepness coefficient vale of is employed. The networ weights are randomly initialied to a vale between and with a niform probability distribtion. The initial otpt of the SRNN is set to be a hypercbe corner, which is also an eqilibrim point. The simlation is performed sing MATLAB version For the SRNN with small vales of, it is possible to test if all hypercbe corners are stable eqilibrim points for a given instance of U and V weight matrix pair. In the simlation stdy, a 2 4 SRNN, which is a networ topology with 2 nodes in the hidden layer and 4 nodes in the otpt layer, was stdied with respect to 75 different U and V weight matrix instances. The 2 4 SRNN has 2 4 possible hypercbe corners since there are 4 otpt nodes. Stability of all 6 hypercbe corners for the 75 instances of the U and V matrices was individally checed. Simlation reslts indicated that, ot of the 75 instances, in most cases at least one hypercbe corner was a stable eqilibrim point. All 6 hypercbe corners were observed to be stable eqilibrim points at least once for the 75 instances of weight matrices U and V. In most cases, the SRNN dynamics had a niqe stable eqilibrim point that was also a hypercbe corner. For some instances of U and V matrices, two hypercbe corners trned ot to be stable eqilibrim points. In a small portion of the 75 instances, limit cycles with a length of two appeared instead of a stable eqilibrim point. The otpt vector [ ] T was also a stable eqilibrim point for a small nmber of instances of U and V matrices. In smmary, 48 stable eqilibrim points that are also hypercbe corners, 2 limit cycles and 9 stable eqilibrim points that are non-hypercbe corners appeared throghot testing with 75 instances of U and V matrices in the simlation. Since three types of stable eqilibrim points (hypercbe corners, non-hypercbe corners, and limit cycles coexisted for some instances of U and V, the total nmber of stable eqilibrim points inclding limit cycles for this SRNN topology is larger than 75. Second simlation stdy was performed on a larger SRNN topology of 2 nodes. In the 2 SRNN case, since there are 2 hypercbe corners it is practically impossible to test the stability of every hypercbe corner eqilibrim point. Therefore, the simlation was condcted by randomly selecting hypercbe corner eqilibrim points among the set of 2 possibilities. Stability of these randomly selected eqilibrim points given a specific instance of weight matrices U and V was checed by forcing the SRNN throgh a relaxation while initialiing the otpt layer nerons to one of hypercbe corner eqilibrim points. The nmber of instances for the weight matrices U and V was, which translated into relaxations in total. In 997 ot of relaxations, a hypercbe corner eqilibrim point proved to be stable. The limit cycle, which had a cycle length of two, happened only once in the cases. The otpt vector

6 [ 5. 5 ] T..5 was also a stable eqilibrim point for two cases. Stdies on larger sie SRNNs involved the 5 and the 5 25 topology instances. A set of hypercbe corners was identified for each problem sie sing a niform random probability distribtion for stability testing given a particlar instance of the weight matrices. Nmber of instances of U and V matrices evalated for the 5 and the 5 25 instances of SRNN topology was. In total, cases were rn for each networ topology. In majority of cases, hypercbe corners appeared as stable eqilibrim points. In a fraction of cases, limit cycles were observed. More specifically, 99 hypercbe corners appeared as stable eqilibrim points while one limit cycle was observed for the 5 networ topology. The nmber of hypercbe corners as stable eqilibrim points was 97 with three limit cycles for the 5 25 networ topology. Simlation stdy indicated that any and all hypercbe corners cold potentially be stable eqilibrim points, which is controlled by the vales of weight matrices U and V. Two other types of eqilibrim points were also observed as attractors for the networ dynamics throghot simlations. These are the limit cycle and the eqilibrim point [ ] T for an SRNN with a - node otpt layer. The nmber of limit cycles and nonhypercbe eqilibrim points, which are attractors, is only a small fraction of the set of eqilibrim points, which can be established as stable points throgh the U and V matrices. V. CONCLUSIONS Theoretical local stability analysis of the Simltaneos Recrrent Neral Networ dynamics copled with correlated simlation stdy indicated that stable eqilibrim points, which inclde hypercbe corners as well as points on the edges, srfaces and interior of the hypercbe, exist in the state space of the neral networ dynamics. This facilitates the networ to be employed as a fixed-point attractor sitable for combinatorial optimiation. Frthermore, presence of sch stable eqilibrim points also facilitates employment of a fixedpoint training algorithm lie the recrrent bacpropagation to gide the networ towards the promising regions of the soltion space. The SRNN networ is comptationally mch more powerfl than the Hopfield networ and its derivatives to address challenging and large-scale combinatorial optimiation problems. Frther enhancement to the SRNN algorithm by incorporating a comptationally efficient stochastic search component is liely to be very valable to improve its ability to locate mch higher qality soltions for challenging problems. Acnowledgement - Fnding provided throgh United States National Science Fondation (NSF Grant (ECS for this research project is grateflly acnowledged. Opinions, views and conclsions expressed are athors only and do not reflect those of the NSF. BIBLIOGRAPHY [] Acley, D. H., Hinton, G. E., and Sejnowsi, T.., A Learning Algorithm for Boltman Machine, Cognitive Science, Vol. 9, pp , 985. [2] Almeida, L. B., A Learning Rle for Asynchronos Perceptrons with Feedbac in a Combinatorial Environment, Proceeding of IEEE -st International Conference on Neral Networs, San Diego, CA, ne 2-24, pp , 987. [3] Carpenter, G. and Grossberg, S., Adaptive Resonance Theory (ART, The Handboo of Brain Theory and Neral Networs, MIT Press, 995. [4] Cichoci, A. and Unbehaen, R., Neral Networ for Optimiation and Signal Processing, Wiley, 993. [5] Corra,, Training Simltaneos Recrrent Neral Networ with Non-Recrrent RPROP Training Algorithm, Internal Technical Report, The University of Toledo, Toledo, OH, 2. [6] Geib,., The Simltaneos Recrrent Neral Networ Applied to Large-Scale Traveling Salesman Problems, Internal Technical Report, Department of Electrical Engineering and Compter Science, The University of Toledo, Toledo, OH, 2. [7] Hopfield,.., and Tan, D. W., Neral Comptation of Decision in Optimiation Problems, Biological Cybernetics, Vol. 52, pp. 4-52, 985. [8] in, L., and Gpta, M. M., Stable Dynamic Bac-propagation Learning in Recrrent Neral Networs, IEEE Transactions on Neral Networs, Vol., No. 6, pp , 999. [9] oso, B., Bi-directional Associative Memories, IEEE Trans. on SMC, Vol. 8, pp. 49-6, 988. [] Li, -H., Michel, A. N., Porod, W., Analysis and Synthesis of A Class of Neral Networs: Linear Systems Operating on A Closed Hypercbe, IEEE Trans. On Circits and Systems, Vol. 36, No., pp , 989. [] Patwardhan, A., The Simltaneos Recrrent Neral Networ for Static Optimiation Problems, Master of Science in Engineering Science Thesis, The University of Toledo, 999. [2] Pineda, F.., Generaliation of Bac-Propagation to Recrrent Neral Networs, Physical Review Letters, Vol. 59, pp , 987. [3] Riedmiller, M. and Bran, H., A Direct Adaptive Method for Fast Bacpropagation Learning: The RPROP Algorithm, Proc. Of the IEEE International Conference on Neral Networs, Vol. 5, pp , 993. [4] Serpen, G., Patwardhan, A. and Geib,., The Simltaneos Recrrent Neral Networ Addressing the Scaling Problem in Static Optimiation, International ornal of Neral Systems, Vol., No. 5, pp , 2. [5] Serpen, G. and Livingston, D. L., Determination of Weights for Relaxation Recrrent Neral Networs, Nerocompting, Vol. 34, pp , 2. [6] Werbos, P.., Generaliation of Bacpropagation with Application to A Recrrent Gas Maret Model, Neral Networs, Vol., No. 4, pp , 988. [7] Werbos, P.., and Pang, X., Neral Networ Design for Fnction Approximation in Dynamic Programming, technical report available online at 8-/CSHCN_MS_98-.pdf, 998. [8] X, Y., A Dynamic System Analysis of Simltaneos Recrrent Neral Networ, Master of Science in Electrical Engineering Thesis, The University of Toledo, 2.