Application of the topological gradient method to color image restoration and classification

Transcription

1 Application of the topological gradient method to color image restoration and classification Didier Auroux a Lamia Jaafar Belaid b a Institut de Mathématiques de Toulouse, Université Paul Sabatier Toulouse 3, Toulouse cedex 9, France. b Ecole Nationale d Ingénieurs de Tunis & LAMSIN, B.P 37, 2 Le BELVEDERE, Tunis, Tunisie. Abstract The goal of this note is to generalize the topological gradient method, applied to restoration and classification problems for grey-level images, to color images. We illustrate our approach with numerical tests. Résumé Application de la méthode du gradient topologique au traitement d images couleur. Le but de cette note est de généraliser l approche du gradient topologique, appliquée au problème de la restauration et de la classification d images en niveaux de gris, à des images couleur. Quelques tests numériques sont présentés pour illustrer cette approche. Key words: Topological gradient; topological asymptotic expansion; image restoration; image classification; color vision. Mots-clés : Gradient topologique; développement asymptotique topologique; restauration d images; classification d images; vision couleur. 1. Introduction The goal of this note is to propose a new method for color image restoration and classification problems. This method is based on the topological gradient approach that has been introduced for topological optimization purpose [6,1]. The basic idea is to adapt the topological gradient approach applied to diffusive image restoration [4,5] and grey level classification problem [3]. In fact, color or multispectral images can be represented or modeled in various ways [7]. In this note we shall focus on the RGB model in which a color image I is usually represented by a 3D vector. For each pixel (x,y), the vector I(x,y) represents the intensity of the three colors: Red, Green, Blue. Since each component can be treated as a grey level image, then the topological gradient approach can easily be used. addresses: auroux@mip.ups-tlse.fr (Didier Auroux), lamia.belaid@esstt.rnu.tn (Lamia Jaafar Belaid). Preprint submitted to Elsevier Science February 28, 7

2 Let v be a given noisy color image defined as a vectoriel function v : (x,y) Ω R 2 v(x,y) = (v 1,v 2,v 3 ), where Ω represents the support of the image, and v k is the value of the pixel (x,y) in each color subspace (red, green, blue). Let u = (u 1,u 2,u 3 ) be the restored image. To find u, we solve the following PDE problem in each subspace (see [4,5] for more details about this equation) div ( c k u k) + u k = v k in Ω, (1) n u k = 0 on Ω, k {1,2,3}, where n denotes the outward unit normal to Ω, and c k is a positive coefficient. In topological optimization approach, c k takes only two values: c k 0 in the smooth part of the image and a small value ε k > 0 on edges. Inspired by the work of G. Aubert et al. [2] in which the authors propose a classification model coupled with a restoration process, we propose to use the topological gradient approach applied to color image restoration for regularized classification problem. Since the classification problem consists in finding a partition of Ω into a family of open sets {Ω i } i=1,..,n, the constant c k 0 in the restoration process, which controls the regularity of the restored image and then the length of the interfaces, will be considered as a regularity term in the classification process. The structure of this note is the following. We review in section 2 the topological gradient approach for image restoration and generalize it to color images. The classification problem and its generalization to color image is presented in section 3. Some numerical tests are presented and discussed in both sections. 2. Application of the topological asymptotic expansion to color image restoration This section is an extension to color images of the topological gradient approach applied to grey level restoration problem [4,5]. First, we recall the principle of the topological asymptotic expansion [6]. Let Ω be an open bounded domain of R 2. The topological asymptotic expansion consists in minimizing a functional j(ω) = J(Ω,u Ω ), where u Ω is the solution to a given PDE defined in Ω. For ρ > 0, let Ω ρ = Ω\(x 0 + ρω) be the set obtained by removing a small part x 0 + ρω from Ω, where x 0 Ω and ω R 2 is a fixed open bounded subset containing the origin. The topological sensitivity theory provides an asymptotic expansion of j when ρ tends to zero. It takes the general form j(ω ρ ) j(ω) = f(ρ)g(x 0 ) + o(f(ρ)), f(ρ) > 0, lim ρ 0 f (ρ) = 0. (2) The topological sensitivity g(x 0 ) provides an information for creating a small hole located at x 0. Hence the function g will be called the topological gradient. In the following, we assume that the perturbation of Ω is done by removing a crack σ ρ (n) containing x 0, where n is the unit normal to the crack and ρ its characteristic size. Let v be a given noisy color image defined as a vectoriel function v : (x,y) Ω R 2 v(x,y) = (v 1,v 2,v 3 ). In the RGB representation, v k,k = 1,2,3, represent the intensity of the primary colors separately. Since each monochromatic component v k,k = 1,2,3, of v can be seen as a grey level image as it represents only one scalar component of the image, the idea is then to apply the topological gradient approach used for the grey level image restoration problem. In order to denoise and enhance v, we consider the following problem: for each component k = 1,2,3, find u k ρ H 1 (Ω ρ ) such that div ( c k u k ρ) + u k ρ = v k in Ω ρ, n u k ρ = 0 on Ω ρ. 2 (3)

3 In order to achieve an edge preserving effect, we look for a subdomain of Ω where the energy is small. In the literature, two classical methods can be used: Channel by channel method: We first study the channel by channel approach, in which each component v k is assumed to be contaminated by a noise, so we have to minimize for each k = 1,2,3, the energy norm outside the edges j k (ρ) = J k (u k ρ) = u k ρ 2, (4) Ω ρ where u k ρ is the solution to problem (3). By considering for each k the solution p k to the adjoint problem of equation (1) div(c k p k ) + p k = u kj k (u k ) in Ω, (5) n p k = 0 on Ω, we obtain the following topological asymptotic expansion [1,4,5] j k (ρ) j k (0) = ρ 2 g k (x 0 ) + o(ρ 2 ), with g k (x 0 ) = π u k (x 0 ). p k (x 0 ) π u k (x 0 ) 2. (6) For each component k, the topological gradient could be written as g k (x) = M k (x)n,n, M k (x) = π uk (x) p k (x) T + p k (x) u k (x) T 2 π u k (x) u k (x) T, (7) where M k (x) is a 2 2 symmetric matrix. For a given point x, g k (x) takes its minimal value when n is the eigenvector associated to the lowest eigenvalue λ k min of M k. This value will be considered as the topological gradient associated to the optimal orientation of the crack σ ρ (n). We consider then the following algorithm Algorithm 1 Initialization : c k = c 0 for each k {1,2,3}; Calculation of u k 0 and p k 0 : solutions of the direct (3) and adjoint (5) problems for each k; Computation of the 2 2 matrices M k and their lowest eigenvalue λ k min at each point of the domain; Set c k ε k if x Ω such that λ k min < α k < 0, ε k > 0 1 = (8) elsewhere; c k 0 Compute u k 1, solutions to the problems (3) with c k = c k 1 for each k. In this algorithm, ε k > 0 is assumed to be small, and α k is a negative (small) threshold. The restored color image u 1 is then obtained by reassembling the three restored components u k 1. Figure 1 presents this color restoration process. We have considered a perturbed image with an additive Gaussian noise on each channel (signal to noise ration: SNR=7.09), and the restored image presented in this figure has a SNR equal to We note here that the topological gradient process requires only 3 resolutions of a PDE (direct and adjoint with c = c 0 and then direct with c = c 1 ) for each channel. Moreover, using a discrete cosine transform for the two first resolutions (c = c 0 ) and then a preconditioned conjugate gradient for the third one (c = c 1 ), the authors already showed that the computational cost of this algorithm is in O(n.log(n)) where n is the size of the image [4,5,3]. 3

4 Original image Noisy image Restored image Figure 1. left: original image, middle: noised image (SNR=7.09), right: restored image (SNR=16.96) Vectoriel method: In the vectoriel denoising approach, the cost function j is calculated directly on the vectoriel function v = (v 1,v 2,v 3 ) by using the following color gradient of the image u 2 color := u u u 3 2. (9) This color gradient will allow us to detect the common discontinuities to the three color channels of the image, and perform a common diffusion of its spectral components. So, we have to minimize the energy norm outsides the edges of the image, and the cost function we now consider is the following j(ρ) = J(u ρ ) = u ρ 2 ( color = u u u 3 2). (10) Ω ρ Ω ρ The corresponding topological asymptotic expansion can easily be deduced from the previous work j(ρ) j(0) = ρ 2 g(x 0 ) + o(ρ 2 ), with g(x) = M(x)n,n, (11) and the corresponding 2 2 symmetric matrix is given by M(x) = 3 [ π uk (x) p k (x) T + p k (x) u k (x) T k=1 2 π u k (x) u k (x) T ]. (12) One can notice that this topological gradient is the sum (or mean) of the three previous topological gradients defined by (7). The algorithm remains unchanged, except the point that we define only one common function c 1, which is used for the three direct resolutions. The computational cost of this algorithm is exactly the same as in the previous paragraph, we still have to solve three PDEs for each channel. The restored image we obtain with this vectoriel approach is visually similar to the one obtained with the channel by channel approach (and that is why we did not produce it here). This is also the case for the error distribution. But we should mention that the SNR of the restored image is Application of the topological gradient to color image classification The goal of this section is to solve a color image restoration-classification coupled problem. First, let us recall the principle of classification. Let v = (v 1,v 2,v 3 ) be the original image defined in an open set Ω of R 2 into R 3. The classification problem consists in classifying each component v k,1 k 3, with respect to each channel, by using the channel intensity as a classifier. The goal of color image classification is then to find three different partitions of the same domain Ω into subsets { } Ω k i, such that 4 i=1,...,n k, k=1,2,3

5 the original channel v k is close to C k i in Ω k i, where Ck i are given values of the channel intensities. The classified image u = (u 1,u 2,u 3 ) will then be defined by u k (x) = C k i x Ω k i := { } x Ω; x belongs to the i th class of the k th channel,k {1,2,3}. (13) Finding a classified image u = (u 1,u 2,u 3 ) close to the original image v is equivalent to minimizing for each channel k {1,2,3}, the following cost function j k with respect to the partition ω k = { } Ω k 1,...,Ω k n j k (ω k ) = Ω u k v k 2 dx = n k i=1 Ω k i C k i v k 2 dx. (14) We consider in the following a small region B (x 0,ρ) = x 0 + ρb, where ρ > 0 is small and B is an open bounded set containing the origin. From a numerical point of view, this small region represents the pixel x 0. We initialize our algorithm with constant images: u k (x) = C k n k x Ω. By switching the function u k around the pixel x 0, in B (x 0,ρ), from class n k to class i,i n k, the asymptotic variation of the cost function j k is given by its topological gradient g k i (x 0 ) = ( C k n k Ck i ) 2 ( 2 C k n )( k Ck i Cn k v k (x 0 ) ). (15) The implementation of this method is quite easy, because for each channel k, we only have to compute each g k i, i {1,...,nk 1}, which is an affine function of the original component v k, and then find the pixels x where g k i (x) < 0, in order to minimize the cost function jk, and reassign them to their optimal class. Finally, the classified color image u is obtained by recomposing the three classified component images u k. For each k {1,2,3}, the algorithm is then the following Algorithm 2 Initialization with Ω k n = Ω and all others Ω k k i = ; For all 1 i n k 1, compute gi k (x) for each pixel x, and find i 0 such that gi k 0 (x) gi k (x) i; If gi k 0 (x) < 0, reassign x to class Ci k 0 ; the classified images u k are still given by equation (13). We note that algorithm 2 converges in one iteration. At the end of the classification process, each pixel has been reassigned to its closest class, i.e. to the class C k i 0 with i 0 = arg min{ C k i v k 2}. Since this algorithm corresponds to the unregularized classification model, we can add a regularization term to the cost function. In [3], the authors have considered the case of the regularized classification process, and have given the expression of the topological gradient in this case. In this note, we propose another way to solve the regularized classification problem for color images. In fact, since algorithm 1 finds the contours of the image and smoothes the image elsewhere, we propose to apply the classification algorithm to the smooth image u 1 provided by the restoration algorithm. We note here that the constant c k 0 in the restoration process controls the regularity of the restored image and then the length of the interfaces, so it can be considered as a regularity term in the classification process. Figure 2 shows the results of our classification model, without regularization (algorithm 2) and with regularization using a restoration process (algorithm 1 before algorithm 2). The original image has = different colors, whereas the two restored images (without or with regularization) only have = 120 different colors. 5

6 Original image Unregularized classification Regularized classified image Figure 2. left: original image, middle: unregularized classified image, right: coupled restoration-classification process 4. Conclusion In this note, we have considered the topological gradient approach for color image restoration and classification problems. The model we proposed in this note is an extension to color images of the method introduced in [4] and [3]. The numerical tests show that our method can successfully remove the noise and preserve the global features of a color image. The numerical classification results are also satisfactory. As some coupling between the red, green and blue channels is important and should be considered, we propose in a future work to explore other color spaces (such as 2D and 3D histograms on HSV and CB spaces) and some coupling methods such as the Di Zenzo gradient [8]. References [1] S. Amstutz, I. Horchani and M. Masmoudi, Crack detection by the topological gradient method, Control and Cybernetics, 34(1) (5) [2] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing, Applied Mathematical Sciences, 147, Springer-Verlag, 1. [3] D. Auroux, L. Jaafar Belaid and M. Masmoudi, A topological asymptotic analysis for the regularized grey-level image classification problem, submitted to Mathematical Modelling and Numerical Analysis. [4] L. Jaafar Belaid, M. Jaoua, M. Masmoudi and L. Siala, Image restoration and edge detection by topological asymptotic expansion, C. R. Acad. Sci. Paris Sér. I, 342(55) (6) [5] L. Jaafar Belaid, M. Jaoua, M. Masmoudi and L. Siala, Application of the topological gradient to image restoration and edge detection, J. of Boundary Element Methods, to appear. [6] M. Masmoudi, The Topological Asymptotic, Computanional Methods for Control Applications, R. Glowinski, H. Kawarada and J. Périaux Eds, International Series GAKUTO, Tokyo, Japan, 16 (1) [7] J. Serra, Espaces couleur adaptés au traitement d images, Rapport technique CMM-Ecole des Mines de Paris, C- 03/03/MM, Mars 3. [8] D. Zenzo, A note on the gradient of a multi-image, Computer Vision, Graphics and Image Processing, 33 (1986)