AN IMPROVED NON-LOCAL DENOISING ALGORITHM

Transcription

1 AN IMPROVED NON-LOCAL DENOISING ALGORITHM Bart Goossens, Hiêp Luong, Aleksandra Pižurica and Wilfried Pilips Gent University, Department of Telecommunications and Information Processing (TELIN-IPI-IBBT Sint-Pietersnieuwstraat 4, B-9000 Gent, Belgium ABSTRACT Recently, te NLMeans filter as been proposed by Buades et al. for te suppression of wite Gaussian noise. Tis filter exploits te repetitive caracter of structures in an image, unlike conventional denoising algoritms, wic typically operate in a local neigbourood. Even toug te metod is quite intuitive and potentially very powerful, te PSNR and visual results are somewat inferior to oter recent state-of-te-art non-local algoritms, like KSVD and BM-3D. In tis paper, we sow tat te NLMeans algoritm is basically te first iteration of te Jacobi optimization algoritm for robustly estimating te noise-free image. Based on tis insigt, we present additional improvements to te NLMeans algoritm and also an extension to noise reduction of coloured (correlated noise. For wite noise, PSNR results sow tat te proposed metod is very competitive wit te BM-3D metod, wile te visual quality of our metod is better due to te lower presence of artifacts. For correlated noise on te oter and, we obtain a significant improvement in denoising performance compared to recent wavelet-based tecniques.. INTRODUCTION Digital imaging devices inevitably produce noise, originating from te analog circuitry in tese devices. Noise suppression by means of digital post-processing is often desirable, but also very callenging. In tis paper, we focus on te design of a noise reduction metod for stationary Gaussian noise, tat preserves original image details and tat as a ig visual quality. During te past decade, numerous and diverse denoising metods ave been proposed to tis end. Many metods, like total variation [], bilateral filtering [] or wavelet-based tecniques [3 9] estimate te denoised pixel intensities based on te information provided in a limited surrounding neigbourood. Tese metods only exploit te spatial redundancy in a local neigbourood and are terefore referred to as local metods. Recently, a number of non-local metods ave been developed. Tese metods estimate every pixel intensity based on information from te wole image tereby exploiting te presence of similar patterns and features in A. Pižurica is a postdoctoral researcer of te Fund for te Scientific Researc in Flanders (FWO Belgium. an image. Tis relatively new class of denoising metods originates from te Non-Local Means (NLMeans, introduced by Buades at al. [0, ]. Basically, te NLMeans filter estimates a noise-free pixel intensity as a weigted average of all pixel intensities in te image, and te weigts are proportional to te similarity between te local neigbourood of te pixel being processed and local neigbouroods of surrounding pixels. Oter non-local denoising metods are exemplar-based (KSVD [], or group similar blocks by block-matcing and ten apply 3D transform-domain filtering to te obtained stacks (BM-3D [3]. Te NLMeans filter, despite being intuitive and potentially very powerful, as two limitations at tis moment: first, bot te objective quality and visual quality are somewat inferior to te oter recent non-local tecniques and second, te NLMeans filter as a complexity tat is quadratic in te number of pixels in te image, wic makes te tecnique computationally intensive and even impractical in real applications. For tis reason, improvements for enancing te visual quality and for reducing te computation time ave been proposed by different researcers. Some autors investigate better similarity measures [4 6], use adaptive local neigbouroods [7], or refine te similarity estimates in different iterations [8]. Oter autors propose algoritmic acceleration tecniques [6,9 ], based for example on neigbourood preclassification [6,9] and FFT-based computation of te neigbourood similarities [0]. In tis paper, we sow te connection between te NLMeans filter and robust estimation tecniques, similar to te connection made for te bilateral filter in []. It turns out tat te NLMeans filter is te first iteration of te Jacobi algoritm [] (also known as te Diagonal Normalized Steepest Descent algoritm for robustly estimating te noise-free image using te Leclerc loss function. By tis observation, it becomes possible to investigate oter robust cost functions tat are commonly used for robust estimation. Also, tis suggests tat applying te NLMeans algoritm iteratively in a specific way, furter decreases te cost function. Anoter problem noted by Buades et al. is tat te NLMeans filter is not able to suppress any noise for non-repetitive neigbouroods. In tis work, we keep track of te local noise variance during NLMeans filtering and we apply a local post-processing

2 filter afterwards, to remove remaining noise in regions wit non-repetitive structures. Furtermore, we present an extension of te NLMeans filter to correlated noise and we also present a new acceleration tecnique tat computes te Euclidean distance by a recursive moving average filter. Te proposed modifications significantly improve te NLMeans filter, bot in PSNR as in computation time, and make te filter competitive to (or even better tan recent non-local metods suc as BM-3D of Dabov et al. [3]. Te remainder of tis article is as follows: in Section, te NLMeans algoritm is briefly presented. In Section 3, we investigate te coice of te similarity weigting functions, on probabilistic reasoning and witin te robust estimation framework. In Section 4, we present our improvements to te NLMeans algoritm. We discuss ow te computation time of te filter can be furter reduced in Section 5. Numerical results and visual results togeter wit a discussion, are given in Section 6. Finally, Section 7 concludes tis paper.. NON LOCAL MEANS Suppose an unknown signal X i,i=,..., N is corrupted by an additive noise process V i,i=,..., N, wicresults in te observed signal: Y i = X i + V i ( In tis work, we stick to definitions for -d signals for simplicity of te notations. An extension to -d images is straigtforward. Te denoised value ˆX i of te pixel intensity at position j is computedas te weigted average of all pixels in te image: ˆX i = N j= w(i, jy j N j= w(i, j ( We will refer to tis filter as te pixel-based NLMeans. Alternatively, a vector (or block-based NLMeans filter does also exist [0]. Here overlapping blocks are used, resulting in multiple estimates for eac pixel in a block. To aggregate te different estimates, an additional weigting function b(k determines te weigt contribution of central pixel to its neigbour at relative position k: N K j= k= K ˆX i = b(kw(i + k, j + ky j N K j= k= K b(kw(i + k, j + k (3 Te weigts w(i, j depend on te similarity between te neigbouroods centered at positions i and j. In tis paper, neigbouroods of fixed predefined size are used (e.g Let us denote x i = (X i K,...,X i+k, y i =(Y i K,...,Y i+k and v i =(V i K,...,V i+k as vectors containing pixel intensities of te local neigbourood centered at position i (were for example symmetrical boundary reflection is used at te signal/image boundaries. In te original NLMeans algoritm [0, ], te weigting function is defined as follows: ( w(i, j =exp y i y j (4 wit y i y j = (y i y j T (y i y j te Euclidean distance between te vectors y i and y j. Te bilateral filter [] is closely related to te NLMeans filter. For te bilateral filter, te weigting function is given by: ( w(i, j =exp (Y i Y j exp ( (i j d (5 were te first factor (called potometric distance is inversely proportional to te Euclidean distance between te pixel intensities Y i and Y j and te second factor (called geometric distance, measures te Euclidean distance between te center sample i and te j-t sample. Te NLMeans weigting function can be interpreted as a vector-extension of te bilateral filter weigting function, omitting a geometric distance factor. In [, ], te weigting functions as given in equations (5 and (4 are defined on intuition. However it is not guaranteed tat tese coices are optimal for a given criterion. In [], it is sown tat equation (5 emerges from optimizing a robust M-function. In te next Section, we will derive expressions for w(i, j based on probabilistic reasoning and we will furter extend te result from [] to te NLMeans filter. 3. COMPUTING SIMILARITY WEIGHTS One of te key elements for designing a ig performance NLMeans filter, is te selection of te similarity weigts. Clearly, te similarity weigts sould be adapted to te image in order to acieve maximal improvement. Because only a noisy version of te image is available, te weigts sould also take te noise properties into account; te presence of noise generally degrades te estimate of te similarity between two neigbouroods in te image. Te question now becomes: ow to determine te similarity weigts in an optimal sense for a givencriterion? Luckily, estimation teory can give an answer to tis problem. 3.. Best Linear Unbiased Estimator (BLUE We now only consider te estimation of te noise-free patc x i as a function of surrounding noisy patces. To take te correlation between different x i into account, we model te residual r i,j between patces (centered at position i and j as: x i = x j + r i,j (6 wit r i,j a zero-mean Gaussian random variable wit position-dependent covariance matrix σi,j I (te disadvantages of tis coice will be discussed furter on and wit σ i,i =0. Due to te additivity of te noise, we ave: y i = x j + r i,j + v i (7 For wite Gaussian noise, v i N(0,σ w I. Te Best Linear Unbiased Estimator (BLUE for tis problem is given

3 by: ˆx i = argmin x = argmin x = N j= N j= N log f y (y j ; x (8 j= N j= σ w +σ i,j y j σ w +σ i,j y j x σw + σ i,j (9 (0 Te minimum can be found iteratively, e.g. by gradient descent. Applying only one iteration of tis algoritm yields: ˆx i = y i λ i N j= ρ (y i y j (4 wit ρ (x te gradient of ρ(x. An interesting case is obtained by considering a robust function wit derivative of te form: ρ (x =xg(x, In tis case, equation (4 becomes: wic ere also corresponds to te maximum likeliood (ML estimator. Te variances σi,j,i j are unknown and need also to be estimated. Noting tat from: (y j y i =(r j,j r i,j +(v j v i it follows tat Var [y j y i ]= ( σ w +σ i,j I,tefollowing estimate can be obtained: σ i,j =max (0, (K + y i y j σw ( Tis solution suggests te use of te following weigt matrix: 4K+ y i y j y w(i, j = i y j 4K+ >σw y i y j ( 4K+ σw σ w Tis means tat for i j te weigts are inversely proportional to te Euclidean distance between te vectors at position i and j. Unfortunately, te Gaussian distribution for modeling te residuals is not a good coice: for natural images we may expect tat for eac vector x i tere are many oter vectors x j tat are very similar to x i. Te residual or difference between x j and x i would from tis perspective rater ave a Laplacian distribution tan a Gaussian distribution. Moreover, dissimilar patces x i tend to cluster, wic causes multiple modes in te istogram of te residual. One possibility is e.g. to use a better suited multivariate (multimodal distribution for r i,j tat also incorporates correlations between te components of r i,j. Because te estimators for suc a distribution are muc more complicated and te model training is computationally more intensive, an attractive alternative is to use robust statistics instead and to treat te deviations from te model as outliers. 3.. Robust M-estimator Te Robust M-estimator is derived from te ML estimator by replacing te quadratic term in te negative loglikeliood functional by a multivariate robust loss function ρ(x, as follows: ˆx i =argmin x N ρ (x y j (3 j= ˆx i = y i λ i N = λ i λ i j=,j i N j=,j i N j=,j i g (y i y j (y i y j g (y i y j y i + g (y i y j y j (5 To speed up te first iteration, te step-size λ i can be adapted based on te Jacobi algoritm. Analogous to [], tis leads to: λ i = Equation (5 becomes: + N j=,j i g (y i y j ˆx i = y i + N j=,j i g (y i y j y j + N j=,j i g (y i y j (6 (7 wic is te NLMeans filter wit weigt function given by: { g (y i y j i j w(i, j = (8 i = j Example: for te Leclerc robust function defined by: ( ρ(r = exp rt r (9 we ave: and ( ρ (r =r exp rt r ( w(i, j =exp y i y j (0 ( Hence, te Leclerc robust function leads us to te weigting function from equation (4. Interestingly, te weigt function ( derived in Section (3. can also be interpreted to be associated to a robust function: ρ(r = { r +log r r > r (

4 Tis interpretation makes it easier to compare te properties of te resulting estimator wit oter robust estimators. Note tat te robust estimation framework gives us a muc wider variety of weigting functions. Different robust functions will be compared more in detail in Section IMPROVEMENTS TO THE NLMEANS FILTER Based on te teoretical framework explained in Section and Section 3, we are now able to present a number of improvements for te NLMeans algoritm: If only one iteration of te NLMeans filter is applied, te solution as generally not converged to a (local optimum. Practically, local neigbouroods tat only ave few similar neigbouroods may still contain noise after one iteration (unless is cosen large enoug; but tis would cause oversmooting in oter regions. In tis paper, we propose to keep track of te noise variance at every location in te image and to remove te remainder of te noise as a post-processing step using a local filter (see Section 4.. As discussed in Section 3., te NLMeans algoritm can be considered to be te first iteration of te Jacobian optimization algoritm for te cost function in equation (3. Tis would suggest tat applying te NLMeans algoritm iteratively would furter decrease te cost function. Te coice of te Leclerc robust function is somewat arbitrary. We will see tat furter improvements can be acieved by using te Bisquare robust function. Te existing NLMeans algoritms assume tat te image noise is wite (uncorrelated, wile in most practical denoising applications te noise is correlated. In tis case, computation of te similarity based on te Euclidean distance is ampered wic eventually leads to a poor denoising performance. By extending te reasoning from Section 3, it now becomes possible to devise a NLMeans filter for coloured (correlated noise. Tese modifications will be described more in detail in te remainder of tis Section. 4.. Local filtering of remaining noise In some circumstances (e.g. non-repetitive structures, one iteration of te NLMeans filter may not remove all of te noise. Teoretically, an infinite number of neigbouroods is required in order to completely suppress te noise, wic is not possible for finite image dimensions. However, it is possible to compute te noise variance of eac estimated vector, based on equations (5 and (7: σx,i = Var[ˆx i ] N = (λ i g (y i y j Var [y j ]+ j=,j i λ i Var [y i ] N = σw λ i j=,j i g (y i y j + (3 wit λ i given by equation (6. Clearly, te noise variance depends on te position in te image, wic means tat we are dealing wit non-stationary noise. In teory, any algoritm for non-stationary noise can be used, suc as te ones presented in [6,]. Because before te aggregation, te output of te NLMeans filter is a set of vectors, it is beneficial to use a vector-based denoising algoritm and to aggregate afterwards. In tis paper, we adopt a locally adaptive basis of Principle Components, as in [3]. Tis metod assumes tat te Gaussian noise is stationary, but an extension to non-stationary noise is straigtforward, as we will sow next. First, we define C y,i as te local covariance matrix of y i, estimated as follows : Ĉ y,i = wit ˆμ i = R + R + R (y i+n μ i (y i+n μ i T n= R R n= R y i+n To find te PCA basis, we apply te diagonalization: Ĉ y,i = U i Λ i U T i (4 Te local covariance matrix of x i, denoted as C x,i can be estimated as follows: Ĉ x,i = (Ĉy,i σwi + ( = U i Λi σw I + UT i (5 were ( + replaces possible negative eigenvalues by a small positive number, suc tat te resulting matrix is positive definite (due to estimation errors of Ĉy,i, it is possible tat te difference Ĉy,i σ wi as negative eigenvalues. Te linear MMSE estimator is given by: ˆx i = ˆμ i + Ĉx,i (Ĉx,i + σ x,ii (ˆxi ˆμ i = ˆμ i + U i A i U T i (ˆx i ˆμ i (6 wit A i = ( ( (Λi Λ i σw I Iσ + w + + σ x,i I a diagonal (Wiener filter matrix. Because a direct implementation of te above formulas as a ig computational cost (te diagonalization in equation (4 needs to Tis expressions given ere are for -d signals, but can be easily extended to images by using a double summation.

5 (a Original (b Noisy (0.6dB wit, following te Jacobi algoritm for determining te step size: λ (n i = + ( N j=,j i g ˆx (n i (7 ˆx (n j Implementation of tis estimate is analogous to te implementation of (7, except tat te estimates from te previous iterations are used as input. At first sigt, te reader may incorrectly ave te impression tat te iterative NLMeans filter increases te computational cost by te number of iterations. Instead, te iterative filter offers a number of advantages: (c NLMeans-W witout post-processing filter (9.8dB (d NLMeans-W wit post-processing filter (30.53dB Figure. Denoising example of Barbara: te effect of using te proposed post-processing filter (a Crop out of te original image (b Image wit wite Gaussian noise (σ =5. (c Te result of te NLMeans filter witout post-processing filter (d Te result of te NLMeans filter wit post-processing filter. PSNR values are between parenteses. be performed for every pixel in te image, we only estimate te covariance matrix Ĉy,i at subsampled positions i and assume tis covariance matrix is piecewise constant. Also, for neigbouroods wit a sufficient number of similar blocks, te variance σx,i will be very small after te NLMeans filtering stage. In tis case do not apply te local filtering step. An example for te Barbara image is given in Figure. It can be seen tat te NLMeans filter removes most of te noise, except in regions wit limited repetitivity. In tis regions, te post-processing filter works excellent in removing te remainder of te noise in Figure c, wile retaining te stripes. 4.. An iterative NLMeans filter An iterative NLMeans filter can be obtained by performing te optimization in equation (3 iteratively. Terefore, we coose te observed image as an initial estimate, i.e. ˆx (0 j = y j,j =,..., N. For te n-t iteration (n >0, we find: ˆx (n i = ˆx (n i N λ (n i j= g ( ˆx (n i ( ˆx (n j ˆx (n i ˆx (n j A small searc window (e.g. 3 3 can be used instead of te wole image, wic dramatically reduces te computation time. Wen applying more iterations, information from outside te searc window will also be used, resulting in a complete nonlocal denoising tecnique. Potentially a better end solution can be found, because te cost function is furter reduced. A limitation of te non-iterative NLMeans filter is tat te weigts are computed directly based on te observed noisy image. As a result, te weigts are very sensitive to te image noise. In [3], tis problem is avoided by applying a roug denoising preprocessing step before selecting similar neigbouroods. However, tis tecnique as te problem tat by roug denoising, some details may be lost, potentially resulting in an incorrect selection of dissimilar blocks. In te iterative NLMeans algoritm, every iteration reduces te average noise variance, resulting in better weigt estimates, tereby increasing te overall denoising performance Te coice of te robust loss function As said before, te NLMeans algoritm proposed in [0, ] can be interpreted as te first Jacobian iteration of a robust estimation metod, tat uses te Leclerc loss function. In our earlier work [6] we noted tat te exponential form of g(x still assigns positive weigts to dissimilar neigbouroods. Even toug tese weigts are very small, te estimated pixel intensities can be severely biased due to many small contributions. We terefore proposed a preclassification based on te first tree statistical moment to exclude dissimilar blocks [6]. An alternative is to cange te sape of te robust function. An overview of some relevant robust functions are given in Table. We will now look at te caracteristics of te robust weigting functions more in detail. Te weigting function associated wit te BLUE estimator derived in Section 3. and te Caucy weigting function ave a very slow decay (see Figure a. Tey assign larger weigts to dissimilar blocks tan te Leclerc robust function, wic will eventually lead to oversmooting.

6 Te Leclerc weigting function as a faster decay, but still assigns positive weigts to dissimilar blocks. Te Andrews weigting function imposes a ard tresold to compare neigbouroods (te weigt is 0 as soon as a given tresold is exceeded, wile te Tukey and Bisquare weigting functions rater use a soft tresold (Figure b. Experimentally we found tat applying a soft tresold often improves te visual quality, in analogy to wavelet tresolding. To furter improve upon te Tukey and Bisquare weigting functions, we also modified te Bisquare robust function in order to ave a steeper slope (see Table and Figure b. In Section 6 we will report te improvement in PSNR by using tis robust function NLMeans filter for correlated noise In te previous Sections, we assumed tat te Gaussian noise is uncorrelated (wite. Applying te NLMeans filter witout modifications to images corrupted wit correlated noise often yields a poor denoising performance (see Section 6. Fortunately, a robust estimator for correlated noise can be obtained by replacing te Euclidean distance y i y j by te Maalanobis distance (y i y j T C w (y i y j tat takes te noise covariance matrix C w into account (see e.g. [4], giving te following estimator: ˆx i =argmin x N j= ( ρ C / w (x y j wit ( / te square root of a positive definite matrix. Te weigt function becomes: { ( g Cw / (y i y j i j w(i, j = (8 i = j Clearly, te correlatedness of te noise only affects te weigt function and not te final averaging. However, te matrix multiplication in equation (8 is still computationally expensive, given te large number of weigts to be computed. Terefore, we apply a prewitening linear filter operation to te noisy image. If te noise Power Spectral Density is given by H(l, ten we compute te prewitened image Y prewit i,i=,..., N as: Ỹ prewit (l =Ỹ (l max(ɛ, H(l wit Ỹ prewit (l, Ỹ (l te discrete Fourier transform of respectively Y prewit i and Y i,andɛ a small positive number to ensure stability (for te results in Section 6, ɛ = 0 4. Next, tis prewitened image is used to compute te weigts based on te Euclidean distance (see also Figure 3. Y i Prewitening Y prewit i Weigt function w(i, j NLMeans Figure 3. Block diagram for te suppression of correlated noise 5. SPEEDING UP THE NLMEANS FILTER Because te algoritmic complexity of te brute force NLMeansfilteronimagesisO(N (K +, wit N te number of pixels in te image, an efficient implementation is desirable. Our previous analysis in [6] revealed tat te main part of te computation time is taken by te weigt computation. 5.. Exploiting weigt symmetry As in [6], we reduce te computation time by approximately a factor by exploiting te fact tat weigt functions are symmetrical (i.e. w(i, j =w(j, i. Terefore, we keep track of a weigt normalization matrix and a accumulated contribution matrix (bot of te same size as te input image, and at te beginning of te algoritm, initialized wit zeros. Wen processing a pixel i wit te contribution of a pixel j, we add te products w(i, jy i and w(i, jy j to te accumulated contribution matrix at te pixel positions j and i respectively. Te weigt normalization matrix is also accumulated at te same pixel positions wit w(i, j. As a result, we only need to compute weigts of neigbouroods for j > i. Finally, we normalize te accumulated contribution matrix via element-wise division by te weigt normalization matrix, in order to obtain te estimated image. Tese concepts can also be applied to te vector-based NLMeans filter: ere te products w(i, jb(ky i+k and w(i, jb(ky j+k are added to te accumulated contribution matrix at positions j + k and i + k respectively, for k = K,..., K. At te same positions, te products w(i, jb(k are added to weigt normalization matrix. 5.. Fast computation of te weigt functions based on te Euclidean distance For te robust functions in Table, w(i, j is a function of te Euclidean distance y i y j. Wen considering a constant position difference Δi (i.e. j = i +Δi, te function β(i = y i y i+δi can be practically implemented using a moving average filter applied to te squared difference between te signals ˆXi

7 Type g(r ρ(r { { / r r > +log r r > BLUE = / = r r r Caucy =/ (+ r / = log ( + r [ ] {( r Tukey = 3 r 6 ( r r = 0 r > 6 r > { sin(π r / { π r / r Andrews = = π ( cos (π r / r 0 r > π r > Leclerc =exp ( r = exp ( r ( 3 ( 3 ( r Bisquare = r r = 6 r + r 0 r > 0 r > 8 ( 9 ( 9 ( r Modified Bisquare = r r = 8 r + r 0 r > 0 r > Table. Overview of multivariate robust M-functions for extending te NLMeans algoritm, wit a smooting parameter Caucy Le Clerc Andrews "BLUE" Le Clerc Modified bisquare Tukey Bisquare (a (b Figure. Comparison of te weigting functions g(r for commonly used robust functions. Here, r = y i y j is te Euclidean distances between two vectors y i and y j

8 Y i and Y i+δi, requiring only pixel accesses instead of K +(in -d. For images, we use te straigtforward - d extension of te moving averaging filter. Togeter wit te weigt symmetry, tis brings te complexity down to approximately O(N, yielding a speed up of a factor (K + /. Wen considering square neigbouroods of size, K =5and te overall speedup is a factor / compared to our previous NLMeans filter in [6] Limited searc window Te limited searc window strategy assigns zero weigts to neigbouroods tat are too far away from eac oter, i.e. w(i, j =0if i j >N w. Tis tecnique may decrease te strengt of te NLMeans, especially for images wit many repetitive structures, but for many real images, te denoising performance is not significantly affected. Tis stems from te fact tat many similar blocks can be found in te local neigbourood. In tis paper, we use a limited searc window of 3 3 pixels, in order to keep te computational cost of te algoritm low. We remark again tat in combination wit te iterative approac from Section 4., te effective searc window is extended by N w in eac dimension, ence wen running at least N/ (N w iterations, te complete image will be searced Pseudo-code For illustrative purposes, te pseudo-code of te pixelbased -d implementation is sown in Figure 4. In te pseudo-code, te boundary andling is omitted for clarity, as well as te initialization (warm-up operations of te moving average filter. Te algoritm uses two loops : te first loop iterates on Δi in a limited searc window, te second loop runs over te wole signal. Te similarities corresponding to zero displacements (Δi =0are treated differently, because te weigts are easier to compute in tis case. It can be seen tat te tree acceleration tecniques proposed in tis Section, can be elegantly and efficiently combined in tis algoritm. We furter remark tat for te vector-based implementation (see Section, additional canges are required: a different weigt accumulation matrix needs to be used for eac position in te local neigbourood. 6. RESULTS AND DISCUSSION To assess te improvement gained by using te modified Bisquare cost function over te Leclerc cost function (see Table, we set up a denoising experiment wit 8 images and 9 noise levels. All images are corrupted wit artificially generated wite Gaussian noise wit varying noise levels, and subsequently te non-iterative NLMeans algoritm is applied to tem (including te post-processing filter. In Figure 5, we report te PSNR improvement by using te modified Bisquare cost function. We note tat on For images, four loops are needed (two outer loops and two inner loops for te x and y-directions. % outer loop (limited searc window weigt_cum = zeros(,n; weigt_norm = zeros(,n; for di=:nw % some initialization of te % moving average operation... % inner loop for j=:n i=j+di; % moving average (sum based computation % of te Euclidean distance eucl_dist=eucl_dist+(y(i+k-y(j+k^; eucl_dist=eucl_dist-(y(i-k--y(j-k-^; % weigt computation weigt=g(eucl_dist; % weigt accumulation weigt_cum(i = weigt_cum(i+weigt*y(j; weigt_norm(i = weigt_norm(i+weigt; % symmetry weigt_cum(j = weigt_cum(j+weigt*y(i; weigt_norm(j = weigt_norm(j+weigt; end; end; di=0; % zero displacement for i=:n weigt=; weigt_cum(i=weigt_cum(i+weigt*y(i; weigt_norm(i=weigt_norm(i+weigt; end; % final estimate x_at = weigt_cum./ weigt_norm; Figure 4. Pseudo-code for te improved NLMeans algoritm in -d (pixel-based implementation average, te improvement is approximately 0.dB. However te gain depends bot on te image as on te noise level, and is maximal for images wit many structures or strong edges, suc as Barbara and ouse and for large noise levels (e.g. σ 60. Because of te vast improvement for most images and/or noise levels, we will furter use te Bisquare cost function. Te results for te proposed NLMeans filter are generated using a neigbourood size of (i.e. K =5, a searc window of 3 3 (i.e. N w =5. Te Bisquare cost function is used and te -parameter of te robust function is selected experimentally as: =.σ. Te post-processing filter uses a subsampling factor 8 in te x and y direction for estimating te local covariance matrix (see Section 4.. To furter save computation time and to improve te numerical stability, te post-processing is done on neigbouroods of size 5 5, selected from te center of te neigbourood (ence te aggregation weigting function b(k = I( k /5 is used, wit I( te indicator function. In Table, te denoising performance of te proposed metod for wite noise (NLMeans-W is compared to BLS-GSM [5] (a state-ofte-art wavelet denoising tecnique and BM-3D (a stateof-te-art non-local denoising tecnique [3]. We give

9 Δ PSNR Average increment in PSNR Maximal increment in PSNR Minimal increment in PSNR σ Figure 5. Comparison of using te Leclerc cost function versus te modified Bisquare cost function, for a denoising experiment on a set of 8 images (Barbara, Lena, boats, couple, ill, man, peppers, ouse. Te values are te average (over te test set, maximal and minimal gain in PSNR by using te modified Bisquare cost function compared to te Leclerc cost function, for different noise levels σ. results for bot te non-iterative NLMeans (i.e. only one iteration and te iterative NLMeans filter as presented in Section 4., wit 5 iterations. Te iterative NLMeans filter brings an additional improvement compared to te non-iterative NLMeans. Tis is because te similarity weigts are refined iteratively, eventually resulting in a cleaner image wit more details. Te iterative NLMeans algoritm clearly outperforms te BLS-GSM filter and seems competitive to te BM-3D metod. Te visual performance of tese metods is compared in Figure 6-Figure 9. Here, te difference is astonising: te NLMeans filter reconstructs smooter images wit remarkably less artifacts tan te oter metods. Some details are better reconstructed, suc as te fine stripes in te at of Lena and te featers, even toug te PSNR of te NLMeans filter is sligtly lower compared to BM- 3D. Furter investigation revealed tat te inferior PSNR is caused by te reduced contrast (or oversmooting of fine structures, tat are usually visually ardly noticeable (for example, te tick stripes in te at of Lena located above te featers in Fig.6f. We also compared te denoising performance of te NLMeans extension to correlated noise (NLMeans-C to recent tecniques for correlated noise: BLS-GSM [5] and our recently proposed MP-GSM [5]. Te NLMeans filter brings a significant improvement bot visually as in PSNR: tere are almost no ringing artifacts and obviously no wavelet artifacts. We also compare to NLMeans-W (tat is not adapted to te correlated noise, just to illustrate tat te NLMeans filter, tat uses weigting functions based on te Euclidean distance, does not perform well in te presence of correlated noise. Te computation time for te non-iterative NLMeans filter is approximately 0s. on a Pentium IV processor and for a 5 5 grayscale image. Te iterative filter takes approximately 8 minutes, wen 5 iterations are used. Noise standard deviation LENA BLS-GSM [5] BM-3D [3] NLMeans-W (non-it NLMeans-W (5 it BARBARA BLS-GSM [5] BM-3D [3] NLMeans-W (non-it NLMeans-W (5 it Table. PSNR [db] results for wite noise 7. CONCLUSION By exploiting te repetitivity of structures in an image, a significant gain in denoising performance is obtained compared to purely local metods. We ave sown tat te NLMeans algoritm is te first iteration of te Jacobi optimization algoritm for robustly estimating te noisefree image, based on te Leclerc loss function. Next, we ave proposed several improvements to te NLMeans filter tat affect bot te visual quality and te computation time. Our proposed metod now compares favourably to state-of-te-art non-local denoising tecniques in PSNR and even offers a superior visual quality: te denoised images contain less artifacts witout sacrificing sarpness. Te proposed improvements can also be applied to oter image processing tasks suc as intra-frame superresolution, non-local demosaicing and deinterlacing. 8. REFERENCES [] L. Rudin and S. Oser, Total variation based image restoration wit free local constraints, in Proc. of IEEE International Conference on Image Processing (ICIP, vol., Nov. 994, pp [] C. Tomasi and R. Manduci, Bilateral filtering for gray and color images, in Proceedings International conference on computer vision, 998, pp [3] D. L. Donoo, De-Noising by Soft-Tresolding, IEEE Trans. Inform. Teory, vol. 4, pp , May 995. [4] L. Şendur and I. Selesnick, Bivariate srinkage wit local variance estimation, IEEE Signal Processing Letters, vol. 9, pp , 00. [5] J. Portilla, V. Strela, M. Wainwrigt, and E. Simoncelli, Image denoising using scale mixtures of gaussians in te wavelet domain, IEEE Transactions on image processing, vol., no., pp , 003. [6] J. Portilla, Image restoration using Gaussian Scale Mixtures in Overcomplete Oriented Pyramids (a review, in Wavelets XI, in SPIE s International Symposium on Optical Science and Tecnology, SPIE s 50t Annual Meeting, Proc. of te SPIE, vol. 594, San Diego, CA, aug 005, pp

10 [7] A. Pižurica and W. Pilips, Estimating te probability of te presence of a signal of interest in multiresolution single- and multiband image denoising, IEEE Transactions on image processing, vol. 5, no. 3, pp , 006. [8] F. Luisier, T. Blu, and M. Unser, A New SURE Approac to Image Denoising: Interscale Ortonormal Wavelet Tresolding, IEEE Trans. Image Processing, vol. 6, no. 3, pp , Mar [9] J. A. Guerrero-Colon, L. Mancera, and J. Portilla, Image restoration using space-variant gaussian scale mixtures in overcomplete pyramids, IEEE Trans. Image Proc., vol. 7, no., pp. 7 4, 008. [0] A. Buades, B. Coll, and J. Morel, A review of image denoising algoritms, wit a new one, SIAM interdisciplinary journal: Multiscale Modeling and Simulation, vol. 4, no., pp , 005. [] A. Buades, B. Coll., and J. Morel, A non local algoritm for image denoising, in Proc. Int. Conf. Computer Vision and Pattern Recognition (CVPR,vol., 005, pp [] M. Elad, On te Origin of te Bilateral Filter and Ways to Improve it, IEEE Transactions on Image Processing, vol., no. 0, pp. 4 5, 00. [3] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3d transform-domain collaborative filtering, IEEE Transactions on image processing, vol. 6, no. 8, pp , 007. [4] N. Azzabou, N. Paragias, and G. F., Image Denoising Based on Adapted Dictionary Computation, in Proc. of IEEE International Conference on Image Processing (ICIP, San Antonio, Texas, USA, Sept. 007, pp. 09. [9] M. Mamoudi and G. Sapiro, Fast image and video denoising via nonlocal means of similar neigboroods, IEEE Signal Processing Letters, vol., no., pp , Dec [0] J. Wang, Y. Guo, Y. Ying, Y. Liu, and Q. Peng, Fast non-local algoritm for image denoising, in Proc. of IEEE International Conference on Image Processing (ICIP, 006, pp [] B. R. C. and M. Vevilainen, Fast nonlocal means for image denoising, in Proc. SPIE Digital Potograpy III, R. A. Martin, J. M. DiCarlo, and N. Sampat, Eds., vol. 650, no.. SPIE, 007. [] B. Goossens, A. Pižurica, and W. Pilips, Wavelet domain image denoising for non-stationary and signal-dependent noise, in IEEE International Conference on Image Processing (ICIP, Atlanta, GA, USA, oct 006, pp [3] D. D. Muresan and T. W. Parks, Adaptive Principal Components and Image Denoising, in Proc. Int. Conf. on Image Processing (ICIP, 003. [4] N. A. Campbell, H. P. Lopuaä, and P. J. Rousseeuw, On te calculation of a robust s-estimator of a covariance matrix. Stat Med, vol. 7, no. 3, pp , Dec 998. [5] B. Goossens, A. Pižurica, and W. Pilips, Image Denoising Using Mixtures of Projected Gaussian Scale Mixtures, IEEE Transactions on Image Processing, 008, (submitted. [5] C. Kervrann, J. Boulanger, and P. Coupé, Bayesian Non-Local Means Filter, Image Redundancy and Adaptive Dictionaries for Noise Removal, in Proc. Int. Conf. on Scale Space and Variational Metods in Computer Visions (SSVM 07, Iscia, Italy, 007, pp [6] A. Dauwe, B. Goossens, H. Luong, and W. Pilips, A Fast Non-Local Image Denoising Algoritm, in Proc. SPIE Electronic Imaging, vol. 68, San José, USA, Jan 008. [7] C. Kervrann and J. Boulanger, Optimal spatial adaptation for patc-based image denoising, IEEE Trans. Image Processing, vol. 5, no. 0, pp , 006. [8] T. Brox and D. Cremers, Iterated Nonlocal Means for Texture Restoration, in Proc. Int. Conf. on Scale Space and Variational Metods in Computer Visions (SSVM 07, vol Iscia, Italy: Springer, LNCS, 007.

11 (a Original (b Noisy (0.9dB (c SV-GSM [9] (3.59dB (d MP-GSM [5] (3.8dB (e BM-3D [3] (3.06dB (f NLMeans-W (3.74dB Figure 6. Visual results for te Lena image corrupted wit wite noise (σ =5

12 (a Original (b Noisy (0.0dB (c BM-3D [3] (6.47dB (d NLMeans-W (6.59dB Figure 7. Visual results for te peppers image corrupted wit wite noise (σ =80

13 (a Original (b Noisy (8.95dB (a Original (b Noisy (8.95dB (c BLS-GSM [5] (9.33dB (d NLMeans-W (8.5dB (c BLS-GSM [5] (9.33dB (d NLMeans-W (8.5dB (e MP-GSM [5] (30.0dB (f NLMeans-C (30.74dB (e MP-GSM [5] (30.0dB (f NLMeans-C (30.74dB Figure 8. Visual results for correlated noise: two different crop-outs of te ouse image. PSNR values are between parenteses.

14 (a Original (b Noisy (6.09dB (a Original (b Noisy (6.09dB (c BLS-GSM (4.76dB (d NLMeans-W (.5dB (c BLS-GSM (4.76dB (d NLMeans-W (.5dB (e MP-GSM (5.0dB (f NLMeans-C (5.44dB (e MP-GSM (5.0dB (f NLMeans-C (5.44dB Figure 9. Visual results for correlated noise: two different crop-outs of te flinstones image. PSNR values are between parenteses.