Image segmentation using an annealed Hopfield neural network

Size: px

Start display at page:

Download "Image segmentation using an annealed Hopfield neural network"

Abraham Richardson
5 years ago
Views:

neural network Yungsik Kim, North Carolina State University Sarah A.

1 Iowa State University From the SelectedWorks of Sarah A. Rajala December 16, 1992 Image segmentation using an annealed Hopfield neural network Yungsik Kim, North Carolina State University Sarah A. Rajala, North Carolina State University Available at:

2 Image segmentationusing an annealed Ilopfield neural network Yungsik Kim Sarah A. Rajala Department of Electrical and Computer Engineering North Carolina State University Raleigh, NC , USA kys ececlr.ncsu.edu or ABSTRACT Good image segmentation can be achieved by finding the optimum solution to an appropriate energy function. A liopfleld neural network has been shown to solve complex optimization problems fast, but it only guarantees convergence to a local minimum of the optimization function. Alternatively, mean field annealing has been shown to reach the global or the nearly global optimum solution when solving optimiiation problems. Furthermore, it has been shown that there is a relationship between a Hopfield neural network and mean field annealing. In this paper, we combine the advantages of the Hopfield neural network and the mean field annealing algorithm and propose using an annealed liopfield neural network to achieve good image segmentation fast. Here, we are concerned not only with identifying the segmented regions, but also finding a good approximation to the average gray level for each segment. A potential application is segmentation-based image coding. This approach is expected to find the global or nearly global solution fast using an annealing schedule for the neural gains. A weak continuity constraints approach is used to define the appropriate optimization function. The simulation results for segmenting noisy images are very encouraging. Smooth regions were accurately maintained and boundaries were detected correctly. 1. INTRODUCTION Digital computers are remarkably successful at handling many types of problems, such as computing missile trajectories or calculating the roots of very complicated equations. But they have great difficulty doing things such as taking dictation or recognizing a face, which humans do without any effort. Therefore, neural network architectures have been proposed as new computer architectures, which, in some sense, are based on knowledge about a "biological computer" such as the human brain. A neural network is a very complex network of very simple processing units that operate asynchronously, but in parallel. It can find good solutions to optimization problems quickly. Hopfield' proposed a neural network architecture composed of neurons with graded responses (sigmoid input-output relation). He showed that the Hopfield neural network can solve complex optimization problems such as a traveling salesman problem fast once a proper energy function has been chosen. Because Hopfield neurons are natural in that they resemble real biological neurons which have graded, continuous outputs as a function of their inputs, they can be implemented by real physical devices such as operational amplifiers. It should be noted, however, that a Hopifeld neural network can solve optimization problems fast when a proper energy function is given, but it only guarantees convergence to a local minimum, not to the global minimum solution. O /92/$4.OO SP1E Vol Neural and Stochastic Methods in Image and Signal Processing (1992)1 333

3 Image segmentation, like other engineering problems, can be formalized as an optimization problem and the desired image segmentation can be achieved by finding the global optimum solution to the objective function. Performing image segmentation by solving the optimization problem consists of two major parts: 1. Finding the proper objective function whose global minimum value represents the desired segmented image. 2. Choosing the relaxation method to find the global solution from the objective function as efficiently as possible. Blake and Zisserman9 proposed an algorithm in which they used the weak continuity constraints ("weak string" or "weak membrane" ) as the proper objective function for detecting the discontinuities in an image composed of piecewise constant regions with step edges. The power of weak continuity constraints lies in its rability to detect discontinuities and localize them accurately and stably without any prior information about their existence or location, even in the presence of substantial noise. We adopt their approach for defining an appropriate objective function for the image segmentation. Once the proper energy function is defined, the next step is to find the global minimum of the objective function. Relaxation algorithms such as stochastic simulated annealing (SSA)6 and deterministic mean field annealing (MFA)4'5 have been used to obtain nearly global optimum solutions to such problems. Stochastic simulated annealing is guaranteed to find the global minimum under certain conditions, but it converges very slowly. Mean field annealing4 can reach nearly global optimum solutions faster than stochastic simulated annealing, while retaining many of the advantages of stochastic simulated annealing. Furthermore, Van Den Bout el al.8 showed that under certain conditions, a Hopfield neural network and mean field annealing are equivalent if the slope of the sigmoid is increased over the running of the algorithm. In this paper, we incorporate an annealing schedule with a Hopfield neural network to find a more optimal segmentation of images. Furthermore, this annealed Hopfield neural network is expected to solve other kinds of optimization problems in which a fast approach to finding the global or the nearly global minimum is desired. A potential application of this annealed Hopfield neural network is segmentation-based image coding. 2. IMAGE SEGMENTATION USING AN ANNEALED HOPFIELD NEURAL NETWORK 2.1 Image segmentation procedure The procedure for image segmentation using the annealed Hopfield neural network is as follows: 1. Convert the image segmentation problem to an optimization problem by defining the proper energy function. The desired segmented image should be the minimum of the energy function. 2. Determine the inter-neuron connection strength A(j,j,m)(k,1,n) between the input of the image neuron (i,, m) and the output of the image neuron (k, 1, n), the external input ej,j,m to each image neuron (i, m), and the inputs V2,,, H1, to each vertical edge neuron (i, i) and horizontal edge neuron (i, i). Here (i, i) and (k, 1) define the pixel locations and m and n represent the gray levels. 334 / SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992)

4 3. Initialize the network parameters. 4. Simulate the network with an annealing schedule for the neural gains. 2.2 Energy function for image segmentation where Blake and Zisserman proposed the following energy function for image restoration problems. E=D+S+P (1) D = >.:(f, d,3)2 = d,,)2 (2) S = b2 >.[(fi,j+i f)2(1 v1,) + (f+i, f)2(1 = b2 [{(0i,j+1,m Oi,j,m)}2(1 +{>(Oi+i,j,m _ oi,j,m)}2(1 h,)} (3) P=a(v, +h,) (4) D is a measure of faithfulness of the restored pixel value fe,, to the input pixel value d13. S is a measure of how severely the restored image f is deformed, and P is the penalty function that is levied for the step edges in the restoration. In the previous equations, f1,, is the gray level of pixel (i, i) in the output image, d2,, is the gray level of pixel (i, ) in the input image, Oj,j,m 5 the output of the (i, m)th neuron ( (i, i) defines the pixel location and in defines the gray level), is the (i, j)th value of vertical edge neurons, h is the (i, j)th value of horizontal edge neurons, and b and c are constants. b controls the smoothness of the segmented image, and a controls the number of the edges in the segmented image. We believe that the same energy function can be used for image segmentation by adjusting weighting of each term. For image segmentation it is important to have more emphasis on S. Using a larger b forces the segmented regions to be smoother than appropriate for image restoration, and guarantees that the average gray levels are close to the actual gray levels in the regions. Since is a gray level value and Oi,j,m has the value 0 or 1 at convergence, f2,3 can be represented by summing the outputs of L neurons, where L is the maximum gray level in the image, i.e., 256 neurons/pixel in an 8 bit monochrome image. By minimizing D + S + F, the segmented image has smoothing regions which are close to original regions in gray level and has step edges between segmented regions. SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992)! 335

5 2.3 Connection strength between image neurons, external inputs to image neurons, and inputs to edge neurons In a Hopfield neural network, the energy function Eh0 which guarantees convergence to stable states is, Eh0 E E A(i,j,m)(k,l,n)Oi,j,mOk,l,n :ej,j,moi,j,m E[Vs,jvj,j + H,3h1,} (5) i,j,rnk,l,r& i,j,m i,j where i,j,m S the output of the (i, i, m)th image neuron, A(j,j,m)(kj,n) represents the connection strength between the input of the (i, ' rn)th image neuron and the output of the (k, 1, n)th image neuron, and e,j,m S the external input applied to the (i, m)th image neuron. V1 and H1,, are the inputs to the (1, j)th vertical edge neuron and to the (i, j)th horizontal edge neuron. By comparing E in Eq. 15 and Eh0 in Eq. 5, we can determine the image neuron connection strength A(i,j,m)(k,l,n), the external input to each image neuron ej,j,m, and inputs to edge neurons V1,1, He,,. Substituting in for D, S, and P, Eq. 15 becomes E = (>Oi,j,m_di,j)2 +b2 [{( i,j+i,m Oi,j,m)}2(1 v,1) + {( z+i,j,m Oi,j,m)}2(1 h,,)} +a + h,,) As a result, parameters become A(i,j,m)(i,j,n) = 2 2b2{(1 v,_i) + (1 h_1,) + (1 v1,) + (1 h1,)} (6) A(i,j,m)(i,j_l,n) = 2b2(1 v,_i) (7) A(i,j,m)(i_l,j,n) = 2b2(1 h1..1,) (8) A(i,j,m)(i,j+l,n) = 2b2(1 v22) (9) A(i,j,m)(i+l,j,n) = 2b2(1 hi,,) (10) ej,j,m = 2d,3 (11) vi,j = b2{(oi,j+i,m Oi,j,m)}2 a (12) = b2{(oi+i,j,m i,j,m)}2 a (13) As mentioned before, pixel gray level f2,, of the output image is found by summing the outputs of L neurons, where L is the maximum gray level value in the image. f2,j '2'' (14) 336 / SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992)

6 2.4 Initialization of the network The initial output value of the each neuron in the network is set to 0.5 E i where is a small uniform random variable. This output value comes from statistical mechanics where the mean value of each particle on a regular lattice is about 0.5 at a high temperature if a particle can only move between 0 and Annealing of the neural gains The annealing schedule for the neural gains needs to be determined. The initial neural gain \jnjt for each neuron is arbitrarily set to a very low value which corresponds to a high value of temperature T in MFA, because the optimal starting gain has not yet been determined. The final gain does not need to be determined. Instead, the neural network stops iterating when there is no output change in the network. The rate for increasing the gain, ij, at each iteration is -, that is Anew 7)A01d old At each iteration all of the image and the edge neurons are updated. 3. SIMULATION RESULTS We tested the annealed Hopfield neural network with three synthetic images, I, '2, and 13. I is a 10 x 10 8-bit monochrome image, 12 is a 32 x 32 8-bit monochrome image, and 13 is a 20 x 20 8-bit monochrome image. For each test image, the step edges have a height of 7 in gray level. First, we applied the noise-free '1 image to the annealed Hopfield neural network to see if the annealed neural network escapes from the - local minima and converges to the global or the nearly global minimum with an arbitrary initial image configuration. Next, the simulation was done with Ii, 12 and 13 at various noise levels. Initial output values for the image, the horizontal edge, and the vertical edge neurons are assumed to be around 0.5. The initial gain for each neuron is and the rate for increasing the neural gain is i = 3.1 Results for noise-free images When the input is the noise-free I image, perfect image segmentation was achieved, i.e., edges were correctly found and the gray level for each region was correctly identified. The results shows that the weak continuity constraints approach for the energy function works well and the annealed Hopfield neural network successfully escaped from the local minima and converged to the global minimum solution as an relaxation method. The simulation result is shown in Fig. 1. In addition, a plot of the energy evolution of the annealed Hopfield neural network vs. numbers of iterations is shown in Fig. 2. Energy values on the left side of the rightmost mode in the energy evolution figure are larger than the ones on th right side, although the difference is slight. SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992)1337

7 3.2 Results for noisy images The three input images were contaminated with the various levels of additive, white Gaussian noise. The standard deviations for the noise processes were 3, 3.5, and 4.0 in gray level (SNR = 2.33, 2, 1.75) for the noisy input images. For test images I and 12, boundaries were detected correctly and the gray level for each region was reasonably smooth. Some variation occurred at the different noise levels, but in general, the results were good. The simulation results for the images I and '2 are shown in Fig 3 and Fig 4 respectively for a SNR of 2. The segmentation results were not as good for images 13. As can be seen in Fig 5, sharp edges were not detected correctly in the segmented image. The difficulties arise with the presence of diagonal edges. To overcome this problem, we made an improvement to the algorithm by adding one more constraint to the energy function. 4. IMPROVEMENT ON THE TRIANGULAR PATTERN In order to better handle diagonaly-oriented edges, the energy function was extended. The new energy function Eezt is given by: EeD+S+P+N (15) where = >2(f d2,,)2 = >(0i,j,m d1,,)2 (16) S = b2 1)2(1 + (f+i, f,)2(1 = b2 oi,j,m)}2(l +{>(Oi+i,j,m oi,j,m)}2(1 (17) P = a + (18) D = r + + v1,3 + h2,)2(v1, + + vj1, + h1, 2)2 (19) The D, S, and P terms are the same as in the previous energy function E. The N term is new. The N term include interactions between the edge neurons and makes the detection of sharp edges better. A Comparison between the previous result using E and the improved result using Ee is shown in Fig / SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992)

8 5. CONCLUSION The annealed Hopfleld neural network was proposed in this paper to achieve good image segmentation fast. It was shown that the annealed llopfleld neural network successfully escaped from the local minima and converged to the global minimum for the noise-free input image. Simulation results showed that this neural network achieved good image segmentations for the input images contaminated with the various levels of noises. In addition to that, average values of the segmented regions were good approximations to the region values of the noise-free images. Because this network operates in parallel, it can be used in the real-time and it is expected to solve optimization problems such as image segmentation quickly, and to reach the global minimum or the nearly global minimum solution. Also, this network has the possibility of being implemented by physical devices such as operational amplifiers. A potential application is segmentationbased image coding applications. 6. REFERENCES 1. J. J. Ilopfield, "Neurons with Graded Response have Collective Computational Properties like Those of Two-state Neurons," Proc. Nail. Acad. Sci. USA, vol. 81, pp , May, J. J. Hopfield and D. W. Tank, "Neural Computation of Decisions in Optimization Problems," Biol. Cybern., vol. 52, pp , J. J. llopfield and D. W. Tank, "Computing with Neural Circuits: A Model", Science, vol. 233, pp , Aug 8, II. P. llariyannaiah, G. L. Bilbro, and W. E. Snyder, "Restoration of Piecewise-constant Images by Mean-field Annealing," Journal of Optical Society of America A, vol. 6, no. 12, pp , December, G. Bilbro, W. Snyder, and R. Mann, "Mean Field Approximation Minimizes Relative Entropy," Journal of Optical Society of America A, 8(2), February, S. Geman and D. Geman, "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images," IEEE Transactions on PAMI, vol. PAMI-6, no.6, pp , November, S. Kirkpatrick, C. D. Gelatt, Jr., M. P. Vecchi, "Optimization by Simulated Annealing," Science, vol. 220, No. 4598, pp , May 13, D. E. Van Den Bout, T. K. Miller, III., "Graph Partitioning Using Annealed Neural Networks," IEEE Transctions on Neural Networks, vol. 1, no. 2, pp , June, A. Blake and A. Zisserman, Visual Reconstruction, The MIT Press, Cambridge, MA, SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992) / 339

9 Figure 1: The left-hand image is a clean I input image and the right-hand image is the segmented image of the neural network E Iterations Figure 2: Energy evolution of the annealed Hopfield neural network vs. iterations. Input image is a noise-free I image. 340 / SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992)

Figure 3: The left-hand image is the noise-free I image. The center image is a noisy I input image and the right-hand image is the segmented Ii image of the neural network.

10 Figure 3: The left-hand image is the noise-free I image. The center image is a noisy I input image and the right-hand image is the segmented Ii image of the neural network. The standard deviation of the noise in the input image is 3.5 and SNR is 2. Figure 4: The left-hand image is the noise-free 12 image. The center image is a noisy 12 input image and the right-hand image is the segmented 12 image of the neural network. The standard deviation of the noise in the input image is 3.5 and SNR is 2. SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992)1341

11 Figure 5: The left-hand image is the noise-free 13 image. The center image is a noisy 13 input image and the right-hand image is the segmented 13 image of the neural network. The standard deviation of the noise in the input image is 3.5 and SNR is 2. Figure 6: The left-hand image is a noisy 13 input image. The center image is the segmented 13image with E = D+S+P and the right-hand image is the segmented 13 image with Ee = D+S+P+N. The standard deviation of the noise in the input image is 3.5 and SNR is / SPIE Vol Neural and Stochastic Methods in Image and Signal Processing (1992)

Detecting Salient Contours Using Orientation Energy Distribution. Part I: Thresholding Based on. Response Distribution

Detecting Salient Contours Using Orientation Energy Distribution The Problem: How Does the Visual System Detect Salient Contours? CPSC 636 Slide12, Spring 212 Yoonsuck Choe Co-work with S. Sarma and H.-C.