ITERATIOAL JOURAL OF MATHEMATICS AD COMPUTERS I SIMULATIO A eural etwork Moel Base on Graph Matching an Annealing :Application to Han-Written Digits Recognition Kyunghee Lee Abstract We present a neural network moel base on graph matching an an annealing technique, one-variable stochastic simulate annealing(ossa) which makes it possible to evaluate the spin average value effectively by Markov process in case of many real applications. In orer to emonstrate the capability of our moel we implemente a program that can segment an recognize han-written igits. Input an object igits, memorize objects, are represente by graph, whose eges are labele by geometrical istance, an whose vertices are labele by position in the graph omain. Pattern recognition can be formulate as elastic graph matching, which is performe here by stochastic optimization of a matching cost function. Our approach provies not only the function of recognition but also the segmentation ability such that input characters are correctly recognize an segmente even if they are touching, connecte, an efecte by noise. Some results of computer experiments are reporte to show the feasibility of our approach. Keywors Pattern recognition, Segmentation, Optimization, Annealing neural networks H I. ITRODUCTIO AD-WRITTE character recognition is an important application fiel of neural networks because the conventional algorithms have much ifficulties in this area[][][3]. Especially, it is har to segment characters correctly by conventional, rule-base segmentation algorithms if they are touching, efecte or noisy. In general, most recognition moels use a preprocessor to remove meaningless variations an to capture meaningful features. However, its recognition accuracy has been limite by the accuracy of the segmentation algorithm it has. One of the well known problems in this situation is that one can not properly segment a character until it is recognize an yet one can not properly recognize a character until it is segmente correctly. This means that high ranks of recognition can be achieve by the integration of segmentation an recognition occurring simultaneously in the system. Combinatorial optimization ranks among the first application of moern neural networks an its application to recognition of characters an objects have been numerous in the literatures[4][6]. In this paper, we propose a neural network Manuscript receive May 3, 007; Revise receive December 6, 007. Kyunghee Lee is with the Department of Information an Communications, PyungTaek University,, YoungI-Dong, PyeongTaek Si, KyungKi-Do, Korea (phone: +8-3-655-7565; fax: +8-3-659-80; e-mail: khlee@ptu.ac.kr). Issue 4, Volume, 007 403 approach that simultaneously segments an recognizes the han-printe igits by graph matching, which is formulate as one of optimization problems. An it is shown that it is possible to map the graph matching problem onto a one-variable annealing neural network with an appropriate energy function. We reuce the size of graph moel without loss of recognition quality an use a one-variable stochastic simulate annealing(ossa) algorithm as an optimization algorithm, which evaluates effectively the spin average value in the mean fiel annealing(mfa) networks with continuous variables[5]. II. GRAPH MATCHIG FOR SEGMETATIO AD RECOGITIO A. Moel Parameters To characterize each feature three measurements are extracte. Their measurements or moel parameters were chosen for their properties of invariance with respect to size, translation incluing shift of the writing. The important factor to select the moel parameters is that the moel parameters can recreate the shape of a stroke an generally of a igit as well except for size an translation[6]. We choose the following three moel parameters to characterize the graph for segmentation an recognition. The first one is the istance, Eucliean istance, between the noe in the input graph an that of reference graph, object graph. This parameter can be invariant to shift of the input. The secon one is the acute angle between the two noes in the graph. This angle parameter is invariant to rotation of the input by the proper angle measure function. The thir one is the number of cross points between the two noes in the graph. This crossing parameter is invariant to the size of the input. B. Graph Matching Approach Pattern recognition in our system consists of a ynamic assignment proceure, which is performe uner the constraint that vertices(noes) in the input graph shoul have approximately the same topological relationship as the vertices in one of the store object graphs. During recognition, patterns are encoe as graphs (V,E), with vertex set V an ege set E. Vertices refer to noes to be assigne. An neighboring vertices are connecte by eges(links) with each other which encoe information about the local topology. In our graph matching approach each noe has relational properties with
neighboring noes an is connecte each other by a link. The relational properties are represente by the moel parameters(istance, acute angle, an the number of cress points between the noes). An not only the relations between the near neighboring noes are use as the compatibility constraints but also relations between all the far neighboring noes are use in orer to increase the robustness of the matching. The graph matching we present in this paper is elastic matching base on an energy of cost function. The minima of which provie the solution of the matching problem. This leas to consiering a neural network system where the state of the system is a set of connectivity rather than a vector of neural activities an computation is a connectivity-ynamics instea of an activity-ynamics[8]. In this approach, matching one graph with another graph consists in fining a connectivity state which satisfies at best many local requirements an minimize the cost C. Cost Function In the graph matching problem for pattern recognition an segmentation we formalize, an actually quantify, with the help of the best possible matching between input an object graph, the egree of matching by a mathematical formulation. To fin out the best-matching we characterize the following energy function to be minimize, which measures the quality of noe-to-noe matching as well as conservation of neighboring relations between noes in the input an object graph. min E = λ E + λ E + λ E () = # ) Vik i= k = j=, j ( i) l=, l k E E = i= k = j=, j ( i) l, l k 3 3 ( V () ( θ) VikVjl, if θ θth, = ( α θ ) V V, if θ θ 0, if min ( i, δ p ma ( i, (4) max + min E3 = β (( δ ( )) i= k = j=, j ( i) l=, l k max min ( ) ), otherwise # = # # (5) ik( i) { θ θ, π ( θ θ )} θ = min (6) ik ( i) ik ( i) δ = δ δ (7) ik ( i) ITERATIOAL JOURAL OF MATHEMATICS AD COMPUTERS I SIMULATIO Where V ik is analogous to a state variable of a neuron, an enotes the probability that noe i is place at position k. Therefore, it takes the value of when the noe i in the input graph is place at position k an it is matche perfectly to the noe i in the object graph. λ, λ, λ 3, are weight factors that etermine the relative weightings of the terms. # is the number of cross points between the noe i an noe j when noe i is at position k an noe j is at position l. θ is the acute angle between the interconnecting ege an horizontal line. δ ik jl Issue 4, Volume, 007 404 jl th (3) is a istance between noe i an noe j when noe i is at position k an noe j is at position l. min max enote minimum an maximum istance between noe i an noe j, that are allowe to move without penalty, respectively an that are neee to assist to prouce a fine-look matching graph. (i), a set of neighboring noes of i, consists of near neighbors an far neighbors. ear neighbors are upper, lower, left, right, upper-left, upper-right, lower-left, an lower-right sie noes aroun the noe i within a unit istance. K(i) an L( are positions, that are ecie when the object graph is constructe, of noe i an noe j in the object graph, respectively. θ th is a threshol value of angle ifference, Upper inices I an O refer to the input an store object graph, respectively.α is a penalty factor to prevent angle ifference between tow noes not to excee the θ th. If we use a small value of α we may get the matching graph consisting of the noes with much egree of isorer ue to the many crossovers. But if too large value of α is use the matching graph looks like rectangular gris of regular shapes. β is another penalty factor to make the noe i an noe j resie in a given range between min an max. The λ term is to be zero if the number of cross points between every two eges in the input graph is same as that in the object graph. The λ an λ 3 terms will have minimum value when all the neighboring noes have the same topology in view of the angle an istance similarity between input an object graph. Graph matching in out moel consists of two types of matching, global an local topology matching. Global topology matching with far neighbor noes are very robust with regar to noise an variance but their performance ecreases in the case of patterns with partially overlapping features. Local topology matching with near neighbor noes is usually istinct an easy to locate an ientify but they coul be occlue by another pattern. III. OE-VARIABLE STOCHASTIC SIMULATED AEALIG In the real worl application, there are many P-har optimizing problems with continuous variables. These inclue quantification analysis, D-optimal esign problem, an fuzzy clustering[5][7]. In the case of neural network with continuous variables of [a,b], the situation is ientical to that of the network with iscrete variables except that the state of each spin has a continues value, real value. An the equilibrium spin average of the continuous network is escribe as the following integral equation[6]. a si exp < si > = b exp b a Hs / T Hs / T Though (8) coul be calculate by the integration metho there are still some weaknesses. For instance, It is time consuming to calculate the integration in case of the large state space. In many cases, a floating point uner-flow error might occur by the harware limitation of the computer when the component of the integral goes s s i i (8)
to zero To overcome the above weakness we presents the way how to approximate (8) by the stochastic metho using Markov process. Monte Carlo(MC) techniques are methos of estimating the values of many-imensional integrals by sampling with the help of ranom numbers an regare as the methos appropriate to equilibrium statistical mechanics. We will concern ourselves with estimating the average potential energy of a simple flui system, where the potential energy is epenent on the configuration variables s =(s,,s ) of the particles: < E > = E( s ) p( s ) s (9) where the probability ensity p( s ) is given by E s kbt p s ( ) ( ) = exp (0) Z with Z the configuration integral Z = E ( s ) kbt exp s () In MC estimation of an average like (9), ranom numbers are use to generate approximately istribute configuration(s ) of a system of particles. In practice the computations are fairly expensive to carry out the integral, an it is impossible to calculate the configuration integral, (), because of the unlimite integral space. Importance Sampling(IS) is a promising metho for reucing run-time in MC which has long been recognize as a powerful tool for simulating low probability events. An most of the statistical mechanics applications of MC involve IS rather than inepenent samples. In the MFA neural network with continuous variables, the equilibrium spin average is the same form as the average potential energy with configuration integral except that the system consists of only one variable. An the average of the perturbe spin at a given temperature might be regare as an expecte value of that spin in the mean fiel from the Boltzmann istribution[6]. We propose a new stochastic algorithm, OSSA algorithm, to realize (8) as the following:. Select a spin i in the current state s, an perturb it into a new state s.. Compute the energy E(s ) an compare it with the E(s) of the current state s, an then let the spin i take the perturbe value with probability Pr( s s) =.0, ITERATIOAL JOURAL OF MATHEMATICS AD COMPUTERS I SIMULATIO ( E( s ) E( s)) if E( s ) E( s) () = exp T if E( s ) E( s) Where T is a temperature. 3. Repeat an a number of times 4. Calculate the average of the accepte perturb e values an regar it as an equilibrium spin average, <s i >, at a given temperature T. 5. Anneal with an annealing scheule. 6. Repeat step to 5 until the final temperature is Issue 4, Volume, 007 405 reache In the above algorithm, step to 3 are the same steps as in the original simulate annealing(sa) algorithm except that each single spin is perturbe an evaluate as its own value, which represents as element of the configuration vector. Step 4 is not necessary but recommene to be use because it can prevent a spin from being evaluate as a fluctuate value even though a small number of iteration is use in OSSA algorithm. As a result, OSSA algorithm reuces a multi-boy problem into a single boy problem uner the mean fiel an makes it possible to evaluate the spin average of the network with continuous variables effectively. By the above OSSA algorithm, a goo approximation of (8) can be obtaine an thus spin average with real value can be estimate. IV. SEGMETATIO AD RECOGITIO A. Graph Multi-partitioning The problems with iscrete variables arise often in engineering esign, LSI esign, an also have importance in pattern recognition, speech processing, vision an computing in general. One of the typical problems with iscrete variables is a graph partitioning problem that involves partitioning a graph, which consists of a set of noes an E interconnecting eges, into K equally size sub-graphs, each with /K noes an minimal number of eges. An graph multi-partitioning is a problem with noes that must be resie only on of B bins in the graph. Our graph matching problem is similar to the graph multi-partitioning in that it has exclusivity constraints that in the final solution a given noe n i must resie in only one bin among the B bins. In our case B correspons to the number of possible positions at which a noe can resie. B. Segmentation an Recognition using OSSA When the number of states of a spin is large, we can use OSSA algorithm to approximately calculate the spin values, Our segmentation an recognition algorithm using OSSA an graph matching is as follows:. Select one of the object graphs in sequence an repeat step to step 4 until all the object graphs are selecte.. Initialize every noe in the input graph to be positione aroun the center of its winow area. 3. Determine starting temperature as the following: (a) Perturb the noe i in the current s into a new state s with a starting temperature. (b) Let the noe i take the perturbe value with probability accoring to () an repeat this step a number of times. If the average of the accepte perturbe values has a small value, raise the starting temperature, otherwise lower the starting temperature. (c) Repeat step 3(a) to 3(b) until the average of the accepte perturbe values of every noe has the value of near.0.
ITERATIOAL JOURAL OF MATHEMATICS AD COMPUTERS I SIMULATIO 4. Apply OSSA algorithm an rank the cost at the final temperature as the cost of the graph matching with the selecte object graph. 5. Output the graph matching result that has the minimum cost. In step an, one of the object graphs is selecte an input graph is constructe from the input pattern with every noe place aroun the center of its winow area as an initialization. In step 3, the temperature at witch noe starts to stop the ranom movement is regare as the starting temperature, This enables that OSSA algorithm fins the proper starting temperature empirically without a complicate estimation of the critical temperature. In step 4, optimization is performe by OSSA algorithm as escribe in section 3, an the final solution of segmentation an recognition is obtaine. V. EXPERIMETAL RESULTS This section escribes some results of a preliminary experiment with computer simulation to investigate the ability of this moel to recognize an segment the han-written igits. In the experiments, the input omain which consists of 0 0 cells is use. Each object graph with 3 noes is mae to have its own characteristics in the object omain having the same size as input omain(see Fig ). segment an recognize characters we use the input stimulus consisting of two igits that are istorte or touching each other. The segmentation of connecte or touching characters is more ifficult problem than recognition of a single character because there are many local topologies similar to that of object graph aroun the touching or connecte area in the input graph. We applie graph matching an OSSA to segment han-written igits with the starting temperature(t)=.5, the final T=.0*0-4, number of bins(b)=*, α=.0, β=.0, θ th =0.8, an ε=.0*0-5. The weight factors use were λ =500.0, λ =50.0 an λ 3 =50.0. Fig 3 shows how the graph matching proceure when input pattern is 5. It is shown that at the initial T isorere graphs are mae(fig. 3(a)) but as the T ecrease refine graphs are outcome(fig 3.(b)). At the final temperature, the ientification an segmentation is complete an cost function efine in () is minimize an best matching graphs are isplaye(fig 3.(c)). Fig. An example of 5 as a given input graph To fin the near global minimum of the cost function, the solution of the matching problem. We implement an optimization algorithm, OSSA, for segmentation an recognition as escribe in the previous section. An we also use a winow mechanism, bounary noes(see Fig ) an a quench mechanism to spee up the computer simulation. (a) (b) (c) Fig 3. Progress in the graph matching process (input is 5 ) Fig. Bounary noes fixe to spee up the simulation To investigate the ability of our moel to simultaneously Fig 4 an Fig 5 shows the left-sie an right-sie proceure when a stimulus consisting of touching characters ( 5, 3 ) is presente. In these figures, same in case of Fig 3, as the T Issue 4, Volume, 007 406
ITERATIOAL JOURAL OF MATHEMATICS AD COMPUTERS I SIMULATIO ecrease refine graphs are outcome(fig 4(b), 5(b)). An at the final temperature, the ientification an segmentation is complete an best matching graphs are isplaye(fig 4(c), 5(c)). Left an right sie graph matching can be accomplishe simultaneously. As a result, we can see that graph matching with object graph 5 an 3 prouces the best matching that look like solutions of segmentation of han-written input 53. (a) (b) (c) Fig 5. Right-sie progress in the graph matching process (input is 53 ) (a) (b) (c) Fig 4. Left-sie progress in the graph matching process (input is 53 ) Fig 6 an Fig 7 shows some examples of han-written igits which have been successfully recognize an segmente by OSSA. It can be seen from the figure that input igits are correctly segmente an recognize without preprocessing such as noise reuction, thinning, an scaling even if they are touching, connecte, an efecte by noise, in which case most conventional systems can not hanle well. However, some problems still remain to be solve. For example, when a local topology in some part of input is similar to that of topology of Issue 4, Volume, 007 407
ITERATIOAL JOURAL OF MATHEMATICS AD COMPUTERS I SIMULATIO the object, or the topology of the input graph can be matche with more than one object graphs there is a possibility of failure in recognition an segmentation. Fig. 8 shows the examples of typical segmentation error. As can be seen from the figure 8(a), our moel faile to segment an recognize the igit. An another typical error occurre as in the figure 8(b). Because 7 inclues the topology of our moel recognize 7 as with some noise - an fails to segment an recognize. To overcome the problems we can use the object graphs with ifferent numbers of noes each other, that make it possible to represent the characteristics of the objects effectively accoring to their shapes. (a) (b) Fig 7. Some examples of igits successfully segmente an recognize (left-sie(a), right-sie(b) output graphs) (to be continue) Fig 6. Some examples of eforme numeric single igit that is recognize correctly Issue 4, Volume, 007 408
ITERATIOAL JOURAL OF MATHEMATICS AD COMPUTERS I SIMULATIO (a) (b) Fig 7. Some examples of igits successfully segmente an recognize (left-sie(a), right-sie(b) output graphs) REFERECES [] Keeler,J.D., Rumelhart,D.E., Keow,W.K., Integrate Segmentation an Recognition of Han-Printe umerals, Avance eural Information Processing System, Vol.3, 99, pp.557-563. [] Lee,S., Pan,J.C., Hanwritten umeral Recognition base on Hierarchically Self-Organizing Learning etworks, Proceeings of the International Joint Conference on eural etworks, Singapore, II, 99, pp. 33-3. [3] Zhang, B., Fu, M., Yan, H., A on Linear eural etwork Moel of Mixture of Local Principal Component Analysis : Application to Hanwritten Digits Recognition, Pattern Recognition, Vol.34, Issue, 00, pp. 03-4. [4] Tappert,C.C., Cursive Script Recognition by Elastic Matching, IBM Journal of Research an Development, Vol 6(6), 98, pp. 765-77. [5] Lee,K., Cho,K., Lee,W.D., Lee,S., Mean Fiel Annealing with Continuous Variables an Its Application to the Quantification Analysis Problem, Proceeings of the International Joint Conference on eural etworks, 99, pp. 43-435. [6] Von er Malsburg, C., Pattern Recognition by Labele Graph Matching, eural etworks, Vol., 988, pp. 4-48. [7] Kim,M.H., Choi,H.S., Lee,W.D., Fuzzy Clustering Extene MFA for Continuous Value State Space, Proceeings of the International Joint Conference on eural etworks, 99, pp. 733-738. [8] asrabai,.m., Li,W., Object Recognition by Hopfiel eural etwork, IEEE Transactions on Systems, Man, an Cybernetics, (6), 99, pp 53-535. [9] Izumi, M., Asano,T., Matching of Ege-Line Image Using Relaxation, IEICE Transactions on Information an System, E75-D(6), 99, pp 90-908. [0] Hopfiel,J.J., Tank,D.,W., eural Computation of Decision in Optimization Problems, Biological Cybernetics, 5, 985, pp. 4-5. []Kirkpatrics,S.,Gelatt,C.,Vecchi,M., Optimization by Simulate Annealing, Science, 0, 983, pp. 67-680.. [] Van en Bout, Miller III,T.,K., Graph Partitioning Using Anneale eural etworks, IEEE, Transactions on eural etworks, (), 990, 9-03. IPUT MATCHIG GRAPH IPUT MATCHIG GRAPH (a) (b) Fig 8. Examples of characters our moel fails to segment an recognize correctly VI. COCLUSIO The system presente here emonstrates that neural networks can, in fact, be use for segmentation as well as recognition. We have by no means emonstrate that this metho is better than conventional segmentation an recognition system in over all performance. However, most conventional system can not eal with touching, broken, or noisy characters very well at all. Whereas our system presente here hanles all of these cases an recognition in a integrate fashion. The major iea is that pattern recognition an segmentation problem often requires invariance that can not be easily obtaine in conventional neural network architecture an relational escriptions provies a powerful theoretical tool for solving this kin of problem which requires invariance with respect to various transformation of the image. In our approach, a constraint such as the angle, cross points an istance between two noes are use. These three types of constraints are simple to compute an aitional constraints an/or high-orer constraints may also be inclue. An we can see that it is promising for complex character, such as Korean an Chinese, recognition. Kyunghee Lee was born in Seoul, Korea, in 96. He receive B.S. egree in electrical engineering from Kyungpook ational University in 983 an the M.S. an Ph.D. egree in computer science from Choongnam ational University, Daejeon, Korea, in 990, in 993, respectively. In 983, he joine Electronics an Telecommunications Research Institute(ETRI). From 983 to 997, he was engage in the research an evelopment of high-spee neteork services an he was a project manager of KIIPD(Korea Information Infrastructure Pilot project in Daeuck science town) supporte by KT(Korea Telecom) in 996. He is now an associate professor at PyongTaek University. His main research interests inclue high spee networking services, neural networks, optimization, an classification in unconventional ways. Issue 4, Volume, 007 409