AS a classical problem in low level vision, image denoising. Group Sparsity Residual Constraint for Image Denoising

Transcription

1 1 Group Sparsty Resdual Constrant for Image Denosng Zhyuan Zha, Xnggan Zhang, Qong Wang, Lan Tang and Xn Lu arxv: v6 [cs.cv] 31 Jul 2017 Abstract Group-based sparse representaton has shown great potental n mage denosng. However, most exstng methods only consder the nonlocal self-smlarty (NSS) pror of nosy nput mage. That s, the smlar patches are collected only from degraded nput, whch makes the qualty of mage denosng largely depend on the nput tself. However, such methods often suffer from a common drawback that the denosng performance may degrade quckly wth ncreasng nose levels. In ths paper we propose a new pror model, called group sparsty resdual constrant (GSRC). Unlke the conventonal group-based sparse representaton denosng methods, two knds of pror, namely, the NSS prors of nosy and pre-fltered mages, are used n GSRC. In partcular, we ntegrate these two NSS prors through the mechansm of sparsty resdual, and thus, the task of mage denosng s converted to the problem of reducng the group sparsty resdual. To ths end, we frst obtan a good estmaton of the group sparse coeffcents of the orgnal mage by pre-flterng, and then the group sparse coeffcents of the nosy mage are used to approxmate ths estmaton. To mprove the accuracy of the nonlocal smlar patch selecton, an adaptve patch search scheme s desgned. Furthermore, to fuse these two NSS pror better, an effectve teratve shrnkage algorthm s developed to solve the proposed GSRC model. Expermental results demonstrate that the proposed GSRC modelng outperforms many state-of-the-art denosng methods n terms of the objectve and the perceptual metrcs. Index Terms Image denosng, group sparsty resdual constrant, nonlocal self-smlarty, adaptve patch search, teratve shrnkage algorthm. I. INTRODUCTION AS a classcal problem n low level vson, mage denosng has been wdely studed over the last half century due to ts practcal sgnfcance. The goal of mage denosng s to estmate the clean mage X from ts nosy observaton Y = X + V, where V s addtve whte Gaussan nose (AWGN). In the past three decades, extensve studes have been conducted on developng varous methods for mage denosng [1 11, 35, 36, 40, 60, 61, 63, 64, 67, 68]. Due to the ll-posed nature of mage denosng, t has been wdely recognzed that the pror knowledge of mages plays a key role n enhancng the performance of mage denosng methods. A varety of mage pror models have been developed, such as transform based [1 3], total varaton based [4, 5], sparse Z. Zha, X. Zhang and Q. Wang are wth the department of Electronc Scence and Engneerng, Nanjng Unversty, Nanjng , Chna. E-mal: zhazhyuan.mmd@gmal.com. L. Tang s wth the department of Electronc Scence and Engneerng, Nanjng Unversty, and Natonal Moble Commun. Research Lab., Southeast Unversty, Nanjng , Chna. L. Xn s wth the Center for Machne Vson and Sgnal Analyss, Unversty of Oulu, 90014, Fnland. representaton based [6, 7] and nonlocal self-smlarty based ones [8 10, 64]. Transform based methods assume that natural mages can be sparsely represented by some fxed bass (e.g., wavelet). Motvated by the fact, many wavelet shrnkage based methods have been proposed [1 3]. For nstance, Chang et al. [1] proposed a method called Bayes shrnk algorthm to model the wavelet transform coeffcents as a generalzed Gaussan dstrbuton. Remeny et al. [3] attempted to use 2D scale mxng complex-value wavelet transform to mprove denosng performance. In the total varaton based methods [4, 5], the mage gradent s modeled as Laplacan dstrbuton for mage denosng. Instead of modelng mage statstcs n some transform doman (e.g., gradent doman, wavelet doman), sparse representaton based pror assumes that mage patch can be precsely modeled as a sparse lnear combnaton of basc elements. These elements, called atoms, compose a dctonary [6, 11 13]. The semnal work of KSVD dctonary [11] has not only confrmed promsng denosng performance, but also been successfully used n varous mage processng and computer vson tasks [14 16, 65]. Nonetheless, there exst two man ssues for typcal patch-based sparse representaton methods. Frst, t s computatonally expensve to learn an offthe-shelf dctonary; second, ths knd of sparse representaton model usually neglects the correlaton between sparsely-coded patches. Recently, a flurry of methods have exploted nonlocal selfsmlarty (NSS) pror based on the fact that natural mages contan a large number of mutually smlar patches at dfferent locatons. The semnal work of nonlocal means (NLM) [8] utlzed the NSS pror to mplement a form of the weghted flterng for mage denosng. Snce then, a flurry of nonlocal regularzaton methods were proposed to solve varous mage nverse problems [17 21]. By contrast wth the local regularzaton based methods (e.g., total varaton method [4]), nonlocal regularzaton based methods can effectvely generate sharper mage edges and preserve more mage detals. However, there are stll lots of mage detals and structures that cannot be accurately recovered. One mportant reason s that the above nonlocal regularzaton terms rely on the weghted graph [22], and thus t s unavodable that the weghted manner leads to dsturbance and naccuracy [23]. Inspred by the success of the NSS pror, recent studes [9, 10, 24 27, 40, 56, 59, 63, 64, 71, 72] have revealed that structured or group sparsty can provde more promsng performance for nose removal. For nstance, Dabov et al. [9] proposed block matchng and 3-D (BM3D) method to

2 2 combne NSS pror and transform doman flterng, whch s stll one of the state-of-the-art denosng methods. Maral et al. [10] further advanced the dea of NSS by group sparse codng. Some other methods [26, 27, 59, 71, 72] also have acheved hghly compettve denosng results based on low rank property of the matrx formed by nonlocal smlar patches n a natural mage. Though group sparsty has verfed ts great success n mage denosng, most exstng group-based sparse representaton methods only consder the NSS pror of the nosy nput. For example, LPG-PCA [40] utlzed nonlocal nosy smlar patches as data samples to estmate statstcal parameters for PCA tranng. NLGBT [63] extracted the nonlocal smlar patches from a nosy mage and performed an teratve theresholdng procedure to enforce group sparsty n the graph-based transform (GBT) doman. In SSC-GSM [25], the nonlocal smlar patches are extracted from a nosy mage by smultaneous sparse codng (SSC) and the group sparsty meets Gaussan scale mxture (GSM). However, such methods often suffer from a common drawback that the denosng performance may degrade quckly wth ncreasng nose levels. Wth the above queston kept n mnd, ths paper proposes a new pror model for mage denosng, called group sparse resdual constrant (GSRC). Dfferent from the prevous groupbased sparse representaton denosng methods that only consder the sngle NSS pror of the nosy nput, two knds of NSS pror are (.e., NSS pror of nosy and pre-fltered mages) exploted for mage denosng. The contrbuton of ths paper s as follows. Frst, to enhance the performance of group-based sparse representaton denosng methods, the group sparsty resdual s proposed, and thus the problem of mage denosng s transformed nto one that reduces the group sparsty resdual. Second, to reduce the resdual, we frst obtan some good estmaton of the group sparse coeffcents of the orgnal mage by pre-flterng and then the group sparse coeffcents of the nosy mage are used to approxmate ths estmaton. Thrd, we desgn an adaptve patch search scheme to mprove the accuracy of the nonlocal smlar patch selecton. Fourth, to fuse these two NSS prors better, we present an effectve teratve shrnkage algorthm to solve the proposed GSRC model. Expermental results show that the proposed GSRC modelng outperforms many current state-of-the-art schemes such as BM3D [9] and WNNM [27]. The remnder of ths paper s organzed as follows. Secton II provdes a bref survey of the related work. Secton III presents the modelng of group sparsty resdual constrant (GSRC), adaptve patch search scheme, and dscusses the man dfference among the proposed GSRC method, the BM3D method [9], the NCSR method [19] and most exstng NSS pror-based denosng methods. Secton IV ntroduces the teratve shrnkage algorthm for solvng the GSRC model. Secton V presents the expermental results. Fnally, Secton VI concludes ths paper. The prelmnary work has appeared n [39]. II. RELATED WORK Image denosng s a classcal ll-posed nverse problem where the goal s to restore a latent clean mage from ts nosy observaton. It has been wdely recognzed that the statstcal modelng of natural mage prors s crucal to the success of mage denosng. Many mage pror models have been developed n lterature to characterze the statstcal feature of natural mages. Early models manly consder the pror on level of pxels, such as the local structures used n Tkhonov regularzaton [28] and total varaton (TV) regularzaton [4, 5]. These methods are effectve n removng the nose artfacts but smear out detals and tend to over-smooth the mages. Another popular pror s based on mage patch, whch has shown promsng performance n mage denosng [2, 6, 7, 11]. The well-known work s sparse representaton based model, whch has been successfully exploted for mage denosng [6, 7, 11]. Sparse representaton based model assumes that each patch of an mage can be precsely represented by a sparse coeffcent vector whose entres are mostly zero or close to zero based on a bass set called a dctonary. The dctonary s usually learned from a natural mage dataset and the representatve dctonary learnng (DL) based methods (e.g., ODL [12] and task drven DL [13]) have been proposed and appled to mage denosng and other mage processng tasks. Image patches that have smlar patterns can be spatally far from each other and thus can be gathered n the whole mage. The nonlocal self-smlarty (NSS) pror characterzes the repettveness of textures and structures reflected by natural mages wthn nonlocal regons, whch can be exploted to retan the edges and the sharpness effectvely. The semnal work of nonlocal means (NLM) denosng [8] has motvated a wde range of studes on NSS and a flurry of NSS methods (e.g., BM3D [9], LSSC [10] and NCSR [19]) have been proposed and appled to mage denosng tasks. Low rank modelng based methods have been wdely used and acheved great success n mage or vdeo denosng [26, 27, 59, 71, 72]. A representatve work was proposed by J et al. [26], to remove the flaws (e.g., nose, scratches and lnes) n a vdeo, the damaged pxels are frst detected and demarcated as mssng. The smlar patches are grouped, satsfyng that the patches n each group have smlar underlyng structure and carry out a low rank matrx approxmately for each group. Fnally, the matrx completon s conducted by each group to restore the mage. Snce the tradtonal low rank models tend to over-shrnk the rank components and treat dfferent rank components equally, Gu et al. [27, 59] proposed the weghted nuclear norm mnmzaton (WNNM) model for mage denosng, whch can acheve state-of-the-art denosng performance. Recently, deep learnng based technques for mage denosng have been attractng consderable attentons due to ts favorable denosng performance [32, 34, 37, 38, 58, 62]. For nstance, Jan et al. [32] proposed to use convolutonal neural networks (CNNs) for mage denosng and clamed that CNNs have smlar or even better representaton power than Markov random feld (MRF) model [33]. In [34], the mult-layer perceptron (MLP) was successfully exploted for mage denosng. Chen et al. [62] proposed a tranable nonlnear reacton dffuson (TNRD) model for mage denosng,

3 3 whch learned a modfed felds of experts [66] mage pror by unfoldng a fxed number of gradent descent nference steps. Zhang et al. [37] nvestgated the constructon of feedforward denosng convolutonal neural networks (Dn-CNN) to embrace the progress n very deep archtecture, learnng algorthm and regularzaton method nto mage denosng. Lu et al. [38] consdered the denosng problem as recursve mage flterng va a hybrd neural network. (a) sparsty (b) group sparsty III. MODELING OF GROUP SPARSITY RESIDUAL CONSTRAINT A. Group-based Sparse Representaton Recent studes have revealed that structured or group sparsty can offer more promsng performance for mage restoraton [9, 10, 24 27, 56, 57, 59, 69, 70]. Snce the unt of our proposed sparse representaton model s group, ths subsecton wll gve brefs to ntroduce how to construct the groups. To be concrete, mage X wth sze N s dvded nto n overlapped patches x of sze b b, = 1, 2,..., n. Then for each exemplar patch x, ts most smlar k patches are selected from an L L szed searchng wndow to form a set S (For the detals of smlar patch selecton operator, please see subsecton III-D ). After ths, all the patches n S are stacked nto a matrx X R b k, whch contans every element of S as ts column,.e., X = {x,1, x,2,..., x,k }. The matrx X consstng of all the patches wth smlar structures s called as a group, where x,k denotes the k-th smlar patch (column form) of the -th group. Fnally, smlar to patch-based sparse representaton [6, 7, 11], gven a dctonary D, whch s often learned from each group, such as DCT, PCA-based dctonary, each group X can be sparsely represented as B = D 1 X and solved by the followng l p -norm mnmzaton problem, B = arg mn B { X D B 2 F + λ B p } (1) where 2 F denotes the Frobenous norm, λ s the regularzaton parameter, and p characterzes the sparsty of B. Then the whole mage X can be represented by the set of group sparse codes B. Fg. 1 shows the dfference between sparsty and group sparsty. In mage denosng, the goal s to explot group-based sparse representaon model to recover X from nosy observaton Y and solve the followng mnmzaton problem, A = arg mn A { Y D A 2 F + λ A p } (2) Once all group sparse codes A are acheved, the latent clean mage can be reconstructed as ˆX = DA, where A ncludes the set of group sparse codes A. Although group sparsty has demonstrated ts effectveness n mage denosng, most exstng group-based sparse representaton denosng methods only use the NSS pror of nosy mage for nose removal (e.g., Eq. (2)), What s more, the denosng performance may degrade quckly wth ncreasng nose levels, makng t challengng to recover the latent clean mage drectly from ts nosy observaton. Fg. 1. Comparson between sparsty (where columns are sparse, but do not algnment) and group sparsty (where columns are sparse and algned). B. Group Sparsty Resdual Constrant Let us revst Eq. (1) and Eq. (2), due to the nfluence of nose, t s very dffcult to estmate the true group sparse code B from nosy mage Y. In other words, the group sparse code A obtaned by solvng Eq. (2) s expected to be close enough to the true group sparse code B of the orgnal mage X n Eq. (1). As a consequence, the qualty of mage denosng largely depends on the level of the group sparsty resdual, whch s defned as the dfference between group sparse code A and true group sparse code B, R = A B (3) Therefore, to reduce the group sparsty resdual R and boost the accuracy of A, we propose a new pror model to mage denosng, called group sparse resdual constrant (GSRC) [39], and thus Eq. (2) can be rewrtten as A = arg mn A { Y D A 2 F + λ A B p } (4) However, t can be seen that the true group sparse code B and p are unknown snce the orgnal mage X s not avalable. Therefore, we wll dscuss how to obtan B and p. In addton, one mportant ssue of the proposed GSRC based mage denosng s the selecton of the dctonary. To adapt to the local mage structures, nstead of learnng an overcomplete dctonary for each group Y as n [10], we learn the prncple component analyss (PCA) based dctonary [19] for each group Y. C. Estmaton of the Unknown Group Sparse Code Eq. (3) shows that by reducng the group sparsty resdual R, we could mprove the performance of mage denosng. In general, the orgnal mage X s not avalable n practce, and thus the true group sparse code B s unknown. However, the true group sparse code B can be estmated based on pror knowledge of the orgnal mage X we have. For example, f we have many example mages smlar to the orgnal mage X, then a good estmaton of B could be learned from the example mage set. However, under many practcal stuatons, the example mage set s smply and unsutable. The strategy of pre-flterng s a popular means to mage denosng. The basc dea s smlar to many denosng algorthms such as BM3D [9] where a frst stage plot denosng s exploted before gong to the second stage of the actual denosng. In past few years, a varety of mage denosng

4 4 Patch y Group Y Group Sparsty Group Sparsty Resdual Nosy Image PSNR=18.56dB Re-estmate nose Matchng and ntlzaton by pre-flterng NSS pror of nosy mage PCA Operator A D Y 1 - Dstrbuton of Recoverd Image Xˆ Aggregatng PSNR=32.63dB Dctonary D -2 Patch z Group Z Group Sparsty -6 Xˆ D ˆ Recovered Group Xˆ Extractng Matchng B D Z 1-10 Dstrbuton of A -B The pre-flterng precessng by BM3D PSNR=32.09dB NSS pror of pre-fltered mage Fg. 2. Flowchart of mage denosng by group sparsty resdual constrant (GSRC) model. methods based on pre-flterng have been developed, such as LPG-PCA [40], TID [41], SOS [42], and agmm [43] methods, etc. Based on the above analyss, we frst apply pre-flterng (e.g., BM3D [9], EPLL [44]) to nosy mage Y, and then the ntalzaton result of pre-flterng s defned as Z. Snce the pre-flterng has an deal denosng performance, Z could be regarded as a good approxmaton of the orgnal mage X. Therefore, n ths paper the group sparse code B s acheved by the pre-flterng Z. The flowchart of the proposed GSRC s llustrated n Fg. 2. Specfcally, to reduce the group sparsty resdual, we frst obtan a good estmaton of the group sparse coeffcents of the orgnal mage by pre-flterng Z and then the group sparse coeffcents of nosy nput mage are used to approxmate ths estmate. D. Adaptve Patch Search Scheme k Nearest Neghbors (knn) method [45] has been wdely used to nonlocal smlar patch selecton. Gven a nosy reference patch and a target dataset, the am of knn s to fnd the k most smlar patches. However, snce the gven reference patch s nosy, knn has a drawback that some of the k selected patches may not be truly smlar to gven reference patch. For nstance, the nosy smlar patches va knn and the clean patches matched wth these nosy smlar patch ndexes are shown n Fg. 3(a) and Fg. 3(b), respectvely. It can be seen that the 7-th patch (red box) s obvously devatng from gven reference patch (green box) n Fg. 3(b). Snce the pre-fltered mage s regarded as a good estmaton of the orgnal mage, n ths paper we frst adopt pre-flterng result as the target mage to fetch the k most smlar patch ndexes. Fg. 3(c) shows the smlar patches of BM3D-based pre-fltered mage searched by knn and Fg. 3(d) shows the clean patches matched wth the pre-fltered mage smlar patch ndexes. It can be seen that the smlar patch selecton of the pre-fltered mage s more accurate than that of the nosy mage. Therefore, to obtan an effectve smlar patch ndexes va knn, an adaptve patch search scheme s desgned. We defne the followng formula, = SSIM(Z, ˆX l+1 ) SSIM(Z, ˆX l ) (5) where SSIM represents structural smlarty [46] and ˆX l represents the l-th teraton denosng result. We emprcally defne that f < τ, ˆX l+1 s regarded as target mage to fetch the k smlar patch ndexes, otherwse Z s regarded as target mage. Z s the pre-fltered mage and τ s a small constant. Nosy mage Pre-fltered mage based on BM3D (a) Nosy mage patches (b) Orgnal mage patches matched by nosy mage patches (c) Image patches by pre-flterng based on BM3D (d) Orgnal mage patches matched by BM3D patches Fg. 3. Patch selecton between nosy mage and pre-fltered mage based on BM3D va knn method (where green box represents the reference patch). E. Dscusson Ths subsecton wll provde detaled dscusson about the man dfference among the proposed GSRC method, the BM3D mehod[9], the NCSR method [19] and most exstng NSS pror-based denosng methods. It can be seen that the proposed GSRC method s smlar to BM3D method, both of them are two stage-based denosng methods. Nonetheless, the BM3D s based on flterng method (e.g., DST, DCT), whle the proposed GSRC s based on sparse codng method, along wth dctonary learnng. Compared wth the analytcally desgned

5 0.01 dctonares (e.g., DCT/wavelet dctonary), dctonares learned from mage patch/group have an advantage of beng better adapted to mage local structures [6, 11], and thus could enhance the sparsty whch leads to better performance. Moreover, n the second stage of BM3D, they are mxng up the pre-fltered mage group wth the nosy mage group unmannerly, and the Wnner flterng s utlzed. However, we propose a new pror model, called group sparsty resdual constrant (GSRC). Unlke the BM3D method, we do not mx up the preflterng mage group wth the nosy mage group, nstead, we adopt an teratve shrnkage algorthm [49] to solve the proposed GSRC model, whch can better ntegrate these two NSS prors of nosy and pre-fltered mages (Secton IV for more detals). Natural mages often possess smlar repettve patterns,.e., a large number of nonlocal redundances [8]. By searchng many nonlocal patches smlar to gven reference patch, NCSR [19] frst obtaned good estmates of the sparse codng coeffcents of the orgnal mage by the prncple of NLM, and then centralzed the sparse codng coeffcents of the observed mage to those estmates to mprove the performance of denosng. However, due to the fact that NLM depends on the weghted graph [22], t s unavodable that the weghted manner leads to dsturbance and naccuracy [23]. It s worth mentonng that the proposed GSRC model does not nvolve n the weghted graph. In addton, NCSR s actually a patch-based sparse representaton method, whch usually neglects the relatonshps among smlar patches [24, 72]. NSS pror has shown great success n mage denosng. Most exstng denosng methods only explot the NSS pror of nosy mage [10, 19, 25 27, 47, 59], and few methods use the NSS pror from natural mages [48]. Actually, dfferent from the most exstng NSS prorbased denosng methods, n ths work we consder two knds of NSS pror,.e., NSS prors of nosy and pre-fltered mages. Expermental results show that the proposed GSRC scheme outperforms many state-of-theart methods, such as BM3D [9] and WNNM [27] (See Secton V for more detals) (a) (a) The dstrbuton of R Fttng Gaussan Fttng Laplacan Fttng hyper-laplacan The dstrbuton of R Fttng Gaussan Fttng Laplacan Fttng hyper-laplacan (b) The dstrbuton of R Fttng Gaussan Fttng Laplacan Fttng hyper-laplacan Fg. 5. The dstrbuton of the group sparsty resdual R for mage House wth σ=50 and fttng Gaussan, Laplacan and hyper-laplacan dstrbuton n (a) lnear and (b) log doman, respectvely (pre-flterng based on EPLL [44]). A. Settng of the Parameter p IV. ALGORITHM OF GSRC In Eq. (4), except for estmatng B, we also need to set the value of p. Here we perform some experments to nvestgate the statstcal property of the group sparsty resdual R, where R represents the set of R = A B. In these experments, two mages lena and House are used as examples, where Gaussan whte nose s added to the mages lena and House wth standard devaton σ= 30 (pre-flterng based on BM3D) and σ= 50 (pre-flterng based on EPLL), respectvely. We plot the hstogram of R as well as the fttng Gaussan, Laplacan and hyper-laplacan dstrbuton of R n Fg. 4(a) and Fg. 5(a). To better observe the fttng of the tals, we also plot these dstrbutons n the log doman n Fg. 4(b) and Fg. 5(b). It can be seen that the hstogram of R can be well characterzed by the Laplacan dstrbuton. Therefore, we set p = 1 and the l 1 -norm s adopted to regularze each group sparsty resdual R, and Eq. (4) can be rewrtten as (b) A = arg mn A { Y D A 2 F + λ A B 1 } = arg mn α { ỹ D α λ α β 1 } where ỹ, α, and β denote the vectorzaton of the matrx Y, A and B, respectvely. Each column d j of the matrx D = [ d 1, d 2,..., d J ] denotes the vectorzaton of the rank-one matrx, where J denotes the number of dctonary atoms. 5 (6) (a) The dstrbuton of R Fttng Gaussan Fttng Laplacan Fttng hyper-laplacan The dstrbuton of R Fttng Gaussan Fttng Laplacan Fttng hyper-laplacan Fg. 4. The dstrbuton of the group sparsty resdual R for mage lena wth σ=30 and fttng Gaussan, Laplacan and hyper-laplacan dstrbuton n (a) lnear and (b) log doman, respectvely (pre-flterng based on BM3D [9]). (b) B. Iteratve Shrnkage Algorthm to Solve the Proposed GSRC Model For fxed β, λ, Eq. (6) s convex and can be solved effcently. We adopt an teratve shrnkage algorthm n [49] to solve Eq. (6). In the l + 1-teraton, the proposed shrnkage operator can be calculated as α l+1 = S λ ( D 1 l ˆ x β ) + β (7) where S λ ( ) s the soft-thresholdng operator, ˆ x represents the vectorzaton of the -th reconstructed group ˆX. In fact, accordng to Eq. (7), one can observe that these two NSS prors can be better ntegrated nto ths surrogate algorthm. The above shrnkage operator follows the standard surrogate algorthm, from whch more detals can be seen n [49].

6 6 (a) (b) (c) (d) (e) (f) (g) (h) () (j) (k) (l) Fg. 6. The 12 test mages for denosng experments. (a) Barbara; (b) Elane; (c) flower; (d) foreman; (e) Hll; (f) House; (g) lena; (h) ln; () Monarch; (j) Parrot; (k) pentagon; (l) peppers. C. Adaptve Group Sparsty Regularzaton Parameter Settng The parameter λ for each group that balances the fdelty term and the regularzaton term should be adaptvely determned for better denosng performance. In ths subsecton, nspred by [1], we propose a more robust method for computng λ of Eq. (6) by formulatng the group sparse estmaton as a Maxmum A-Posteror (MAP) estmaton problem. For a gven B, the optmal soluton of Eq. (6) s ˆR = arg max log P(R Y ). By Bayes formula, t s equvalent to R ˆR = arg max R {log P(R Y )} = arg mn R { log P(Y R ) log P(R )} The log-lkelhood term log P(R Y ) s characterzed by the statstcs of nose V, whch s assumed to be addtve whte Gaussan nose wth standard devaton σ, and thus we have P(Y R ) = P(Y A, B ) = exp( 1 2σ 2 Y D A 2 F ) (9) where R and B are assumed to be ndependent. Snce the group sparsty resdual R can be well characterzed by the Laplacan dstrbuton from Fg. 4 and Fg. 5. Thus, the pror dstrbuton P(R ) s characterzed by an..d Laplacan dstrbuton, P(R ) = (8) c 2σ exp( c 2 R σ ) (10) Then we substtute Eq. (9) and Eq. (10) nto Eq. (8), and thus we can readly derve the desred regularzaton parameter λ for each group, λ = c 2 2σ 2 σ (11) where σ denotes the estmated varance of each group sparsty resdual R, and c s a small constant. Wth the soluton A n Eq. (7), the clean group X can be reconstructed as ˆX = D A. Then the latent clean mage ˆX can be reconstructed by aggregatng all the groups {X }. In practcal, we could perform the above denosng procedures for better results by several teratons. In the l-th teraton, the teratve regularzaton strategy [50] s used to update the estmaton of nose varance. Then the standard devaton of nose n l-th teraton s adjusted as σ l = γ (σ 2 Y ˆX l 2 2 ) (12) where γ s a constant. The complete descrpton of the proposed method for mage denosng based on GSRC model s exhbted n Table I. TABLE I GROUP SPARSITY RESIDUAL CONSTRAINT FOR IMAGE DENOISING. Input: Nosy mage Y. Intalzaton: ˆX = Y, Z, c, k, b, L, σ, τ, γ, δ; For l = 1, 2,..., K do End for Iteratve regularzaton Y l+1 = ˆX l + δ(y ˆX l ); Re-estmate σ l+1 computng by Eq. (12); If l = 1 Smlar patch ndexes selecton based on Z. Else If SSIM(Y l+1, Z) SSIM(Y l, Z) < τ Smlar patch ndexes selecton based on Y l+1. Else Smlar patch ndexes selecton based on Z. End f End f For each patch y and z do Fnd a group Y l+1 va knn. Fnd a group Z l+1 va knn. Constructng dctonary D l+1 by Y l+1 by PCA operator. Update B l+1 computng by B = D 1 Z. Update λ t+1 computng by Eq. (11). Update A l+1 computng by Eq. (7). Get the estmaton X l+1 =D l+1 A l+1. End for Output: ˆX l+1. Aggregate X l+1 to form the recovered mage ˆX l+1. V. EXPERIMENTAL RESULTS In ths secton, extensve expermental results are presented to evaluate the denosng performance of the proposed GSRC. For the test mages, we use two dfferent test datasets for thorough evaluaton. One s a test dataset contanng 200 natural mages from Berkeley segmentaton dataset (BSD200) [51] and the other one contans 12 mages whch are shown n Fg. 6. We consder two versons of pre-flterng: (1) a prefltered mage Z generated by the BM3D method [9], denoted as GSRC-BM3D; (2) a pre-fltered mage Z generated by the EPLL method [44], denoted as GSRC-EPLL. To evaluate the qualty of denosed mage, both PSNR and SSIM [46] metrcs are used. A. Parameter Settng Parameters used n the algorthm are emprcally chosen n consderaton of the nose levels n order to acheve relatvely good performance. The basc parameter settng s as follows: the searchng wndow L L s set to be The sze of patch b b s set to be 6 6, 7 7, 8 8 and 9 9 for σ 20, 20 < σ 50, 50 < σ 75 and 75 < σ 100, respectvely. The searchng matched patches k s set to be 60, 80, 90 for σ 50, 50 < σ 75 and 75 < σ 100, respectvely. The

7 7 Orgnal Image Nosy Image BM3D EPLL GSRC-BM3D GSRC-EPLL (a) (b) =20 (c) db (d) db (e) db (f) db (a) (b) =40 (c) db (d) db (e) db (f) db (a) (b) =50 (c) db (d) db (e) db (f) db Fg. 7. Denosng results of BM3D, EPLL, GSRC-BM3D and GSRC-EPLL on test mage Monarch, pentagon and peppers wth σ = 20, 40 and 50, respectvely. (a) Orgnal mage; (b) Nosy Image; (c) Pre-flterng BM3D [9] results. (d) Pre-flterng EPLL [44] results; (e) GSRC-BM3D results; (f) GSRC-EPLL results. detaled settng of the nvolved parameters c, δ, γ and τ are shown n Table II. We run denosng experments for a large range of nose standard devatons (σ= 20, 30, 40, 50, 75 and 100). TABLE II THE DETAILED INVOLVED PARAMETERS SETTING OF c, δ, γ, τ. Nose level GSRC-BM3D GSRC-EPLL σ c δ γ τ c δ γ τ σ e e-4 20 < σ e e-4 30 < σ e e-4 40 < σ e e-4 50 < σ e e-4 75 < σ e e-4 B. Performance Comparson wth the State-of-the-Art Methods In ths subsecton, we valdate the performance of the proposed GSRC and compare t wth recently proposed stateof-the-art denosng methods, ncludng BM3D [9], EPLL [44], NCSR [19], GID [52], LINC [53], MS-EPLL [54], AST-NLS [55] and WNNM [27]. For all the competng methods, the source codes are obtaned from the orgnal authors. We used the default parameters n ther software packages. Frst, we compare GSRC-BM3D, GSRC-EPLL wth BM3D and EPLL method, respectvely. In Table III, we report the PSNR results for dfferent nose varances for the 12 test mages n Fg. 6. It can be seen that GSRC-BM3D, GSRC- EPLL are sgnfcantly better than BM3D and EPLL wth an average gan of about 0.40dB and 0.79dB, respectvely. The vsual qualty comparsons n the case of σ = 20, 40 and 50 for test mages Monarch, pentagon and peppers are provded n Fg. 7, respectvely. It can be found out that the over-smooth phenomena and undesrable artfacts are generated by BM3D and EPLL methods, respectvely. In contrast, the proposed GSRC not only reduces most of the artfacts, but also provdes better denosng performance on both edges and textures than BM3D and EPLL methods. Therefore, these results valdate the usefulness of the proposed GSRC model through the preflterng BM3D and EPLL. Second, to further verfy the performance of the proposed GSRC n mage denosng, we compare t wth sx representatve algorthms: NCSR [19], GID [52], LINC [53], MS- EPLL [54], AST-NLS [55] and WNNM [27]. Gaussan whte nose wth standard devaton σ=20, 30, 40, 50, 75 and 100 s added to the 12 test mages. The PSNR results by the competng denosng methods are shown n Table IV. It can be seen that the proposed GSRC has acheved hghly compettve denosng performance to other leadng methods. Based on the pre-flterng BM3D [9], the proposed GSRC acheves 0.61dB, 1.68dB, 0.39dB, 0.52dB, 0.32dB and 0.11dB mprovement on average over NCSR, GID, LINC, MS-EPLL, AST-NLS and WNNM, respectvely. Meanwhle, based on the pre-flterng

8 8 TABLE III PSNR (db) VALUES OF DENOISING RESULTS FOR FOUR COMPETING STATE-OF-THE-ART IMAGE DENOSING METHODS. TOP LEFT: BM3D [9]; TOP RIGHT: EPLL [44]; BOTTOM LEFT: GSRC-BM3D; BOTTOM RIGHT: GSRC-EPLL. σ Barbara Elane flower foreman Hll House lena ln Monarch Parrot pentagon peppers Average (a) Orgnal Image (b) Nosy Image (c) NCSR (d) GID (e) LINC (f) MS-EPLL (g) AST-NLS (h) WNNM () GSRC-BM3D (j) GSRC-EPLL Fg. 8. Denosng mages of ln by dfferent methods (σ = 30). (a) Orgnal mage; (b) Nosy mage; (c) NCSR [19] (PSNR= 30.65dB, SSIM=0.8632); (d) GID [52] (PSNR= 29.63dB, SSIM=0.8287); (e) LINC [53] (PSNR= 31.03dB, SSIM=0.8670); (f) MS-EPLL [54] (PSNR= 30.96dB, SSIM=0.8688); (g) AST-NLS [55] (PSNR= 30.83dB, SSIM=0.8465); (h) WNNM [27] (PSNR= 31.07dB, SSIM=0.8657); () GSRC-BM3D (PSNR= 31.18dB, SSIM =0.8743); (j) GSRC-EPLL (PSNR = 31.13dB, SSIM=0.8704).

9 9 TABLE IV PSNR (db) COMPARISON OF NCSR [19], GID [52], LINC [53], MS-EPLL [54], AST-NLS [55], WNNM [27], GSRC-BM3D AND GSRC-EPLL. σ = 20 σ = 30 NCSR GID LINC MS- AST- WNNM GSRC- GSRC- MS- AST- GSRC- GSRC- NCSR GID LINC WNNM EPLL NLS BM3D EPLL EPLL NLS BM3D EPLL Barbara Elane flower foreman Hll House lena ln Monarch Parrot pentagon peppers Average σ = 40 σ = 50 NCSR GID LINC MS- AST- WNNM GSRC- GSRC- MS- AST- GSRC- GSRC- NCSR GID LINC WNNM EPLL NLS BM3D EPLL EPLL NLS BM3D EPLL Barbara Elane flower foreman Hll House lena ln Monarch Parrot pentagon peppers Average σ = 75 σ = 100 NCSR GID LINC MS- AST- WNNM GSRC- GSRC- MS- AST- GSRC- GSRC- NCSR GID LINC WNNM EPLL NLS BM3D EPLL EPLL NLS BM3D EPLL Barbara Elane flower foreman Hll House lena ln Monarch Parrot pentagon peppers Average EPLL [44], the proposed GSRC acheves 0.59dB, 1.66dB, 0.37dB, 0.50dB, 0.30dB and 0.09dB mprovement on average over NCSR, GID, LINC, MS-EPLL, AST-NLS and WNNM, respectvely. The vsual comparsons of the competng methods at nose level 30 and 75 are shown n Fg. 8 and Fg. 9, respectvely. TABLE V AVERAGE PSNR (db) RESULTS OF DIFFERENT DENOISING ALGORITHMS FOR GAUSSIAN DENOISING WITH NOISE LEVEL 20, 30, 40, 50, 75 AND 100 ON BSD200 DATASET [51]. σ NCSR [19] GID [52] LINC [53] MS-EPLL [54] AST-NLS [55] WNNM [27] GSRC-BM3D GSRC-EPLL TABLE VI AVERAGE PSNR (db) RESULTS OF APS AND NO-APS SCHEME ON 12 TEST IMAGES. Pre-flterng BM3D σ No-APS APS Pre-flterng EPLL σ No-APS APS To further demonstrate our performance, we comprehensvely evaluate the proposed GSRC on 200 test mages from the BSD dataset [51]. Table V lsts the average PSNR comparson results for a collecton of 200 test mages among eght competng methods at sx nose levels (σ=20, 30, 40, 50, 75 and 100). The vsual comparsons of the denosng methods for test mages and wth σ = 50 and 100 are shown n Fg. 10 and Fg. 11, respectvely. Obvously, one can

10 10 TABLE VII AVERAGE RUN TIME (s) ON THE 12 TEST IMAGES (SIZE: ) WITH DIFFERENT METHODS. Methods NCSR [19] GID [52] LINC [53] MS-EPLL [54] AST-NLS [55] WNNM [27] GSRC-BM3D GSRC-EPLL Average Tme (s) (a) Orgnal Image (b) Nosy Image (c) NCSR (d) GID (e) LINC (f) MS-EPLL (g) AST-NLS (h) WNNM () GSRC-BM3D (j) GSRC-EPLL Fg. 9. Denosng mages of House by dfferent methods (σ = 75). (a) Orgnal mage; (b) Nosy mage; (c) NCSR [19] (PSNR= 27.16dB, SSIM =0.7749); (d) GID [52] (PSNR= 25.23dB, SSIM=0.7052); (e) LINC [53] (PSNR= 27.56dB, SSIM=0.7850); (f) MS-EPLL [54] (PSNR= 27.45dB, SSIM=0.7738); (g) AST-NLS [55] (PSNR= 28.06dB, SSIM=0.7720); (h) WNNM [27] (PSNR= 28.25dB, SSIM=0.7883); () GSRC-BM3D (PSNR= 28.48dB, SSIM=0.7992); (j) GSRC-EPLL (PSNR = 28.53dB, SSIM=0.7998). observe that the proposed GSRC acheves very compettve denosng performance compared to WNNM. In addton, we apply the proposed GSRC to some real nosy mages. Fg. 12 shows the denosed mages yelded by BM3D and our approach. It can be seen that the proposed GSRC can not only reduce the nose effectvely, but also preserve the fner detals. The results ndcate the feasblty of the proposed GSRC for some practcal mage denosng tasks. To sum up, t can be easly found that BM3D, EPLL, NCSR, GID, LINC, MS-EPLL, AST-NLS and WNNM stll generate some undesrable artfacts and some detals are lost. By contrast, the proposed GSRC s able to preserve the sharp edges and suppress undesrable artfacts more effectvely than other competng methods. Such expermental fndngs clearly demonstrate that the GSRC model s a stronger pror for the class of photographc mages contanng large varatons n edges/textures. (a) Orgnal Image (b) Nosy Image (c) NCSR (d) GID (e) LINC (f) MS-EPLL (g) AST-NLS (h) WNNM () GSRC-BM3D (j) GSRC-EPLL C. Comparson between APS and No-APS Scheme In ths subsecton, n order to demonstrate the desgned adaptve patch selecton (APS) scheme effectvely, we compare t wth No-APS scheme. Table VI shows the average PSNR results of APS and No-APS schemes on 12 test mages. It can be seen that the average PSNR results of APS scheme are better than No-APS. Therefore, the proposed APS scheme can boost the accuracy of nonlocal smlar patch selecton under the task of mage denosng. D. Computatonal Cost Effcency s another key factor n evaluatng an algorthm. We then compare the speed of the proposed GSRC and sx representatve algorthms. All experments are conducted under the Matlab 2012b envronment on a machne wth Intel (R) Core (TM) wth 3.56Hz CPU and 4GB memory. The average run tme (s) of the competng methods on the 12 test mages (sze: ) s shown n Table VII. Denosng 12 test mages, NCSR, GID, LINC, MS-EPLL, AST-NLS and WNNM take, on average, roughly 348s, 346s, 257s, 191s, 459s and 202s, respectvely. For the test mages, the proposed GSRC requres only 72s and 148s on average based pre-flterng BM3D and EPLL, respectvely. Obvously, t can be seen that that the proposed GSRC used less computaton tme than these representatve methods. Note that the run tme of the proposed GSRC ncludes the pre-flterng process. VI. CONCLUSION In ths paper, we proposed a novel pror model named group sparsty resdual constrant (GSRC) that exploted two knds of nonlocal self-smlar (NSS) pror and explored ts applcaton nto mage denosng. To boost the performance of group sparse-based mage denosng, the group sparsty resdual was proposed, whch s defned as the dfference between the group sparse code of nosy mage and the group sparse code of the orgnal mage. Therefore, the problem of mage denosng was translated nto one that reduces the group sparsty resdual.

11 11 (a) Orgnal Image (b) Nosy Image (c) NCSR (d) GID (e) LINC (f) MS-EPLL (g) AST-NLS (h) WNNM () GSRC-BM3D (j) GSRC-EPLL Fg. 10. Denosng mages of by dfferent methods (σ = 50). (a) Orgnal mage; (b) Nosy mage; (c) NCSR [19] (PSNR= 25.69dB, SSIM=0.7800); (d) GID [52] (PSNR= 25.40dB, SSIM=0.7152); (e) LINC [53] (PSNR= 26.51dB, SSIM=0.7938); (f) MS-EPLL [54] (PSNR= 25.67dB, SSIM=0.7833); (g) AST-NLS [55] (PSNR= 26.27dB, SSIM=0.7715); (h) WNNM [27] (PSNR= 26.43dB, SSIM=0.7888); () GSRC-BM3D (PSNR= 26.69dB, SSIM=0.7937); (j) GSRC-EPLL (PSNR = 26.62dB, SSIM=0.7908). (a) Orgnal Image (b) Nosy Image (c) NCSR (d) GID (e) LINC (a) Orgnal Image (b) Nosy Image (c) NCSR (d) GID (e) LINC (f) MS-EPLL (g) AST-NLS (h) WNNM () GSRC-BM3D (j) GSRC-EPLL (f) MS-EPLL (g) AST-NLS (h) WNNM () GSRC-BM3D (j) GSRC-EPLL Fg. 11. Denosng mages of by dfferent methods (σ = 100). (a) Orgnal mage; (b) Nosy mage; (c) NCSR [19] (PSNR= 23.73dB, SSIM=0.6882); (d) GID [52] (PSNR= 22.66dB, SSIM=0.5997); (e) LINC [53] (PSNR= 23.68dB, SSIM=0.6756); (f) MS-EPLL [54] (PSNR= 24.21dB, SSIM=0.6869); (g) AST-NLS [55] (PSNR= 24.24dB, SSIM=0.6861); (h) WNNM [27] (PSNR= 24.48dB, SSIM=0.7025); () GSRC-BM3D (PSNR= 24.70dB, SSIM =0.7182); (j) GSRC-EPLL (PSNR = 24.67dB, SSIM=0.7157).

12 12 (a) Nosy (b) BM3D (c) GSRC-BM3D (a) Nosy (b) BM3D (c) GSRC-EPLL Fg. 12. Vsual comparsons of denosng results on real nosy mages wth unknown nose characterstcs. Snce the orgnal mage was unknown, to reduce the group sparsty resdual, we frst obtaned some good estmaton of the group sparse coeffcents of the orgnal mage by pre-flterng and then the group sparse coeffcents of the nosy mage were used to approxmate the estmaton. To enhance the accuracy of nonlocal smlar patches selecton, an adaptve patch search scheme was desgned. In addton, to fuse these two NSS prors better, an teratve shrnkage algorthm was adopted to solve the GSRC model. Extensve expermental results have shown that the proposed GSRC can not only leads to vsble PSNR mprovements over many state-of-the-art methods such as BM3D and WNNM, but also preserves the mage local structures, suppresses undesrable artfacts and results n a compettve speed. In the future, we wll extend the proposed GSRC to other applcatons such as mage deblurrng, mage super-resoluton and mage deblockng. REFERENCES [1] Chang S G, Yu B, Vetterl M. Adaptve wavelet thresholdng for mage denosng and compresson[j]. IEEE transactons on mage processng, 2000, 9(9): [2] Starck J L, Cands E J, Donoho D L. The curvelet transform for mage denosng[j]. IEEE Transactons on mage processng, 2002, 11(6): [3] Remeny N, Ncols O, Nason G, et al. Image denosng wth 2D scalemxng complex wavelet transforms[j]. IEEE Transactons on Image Processng, 2014, 23(12): [4] Rudn L I, Osher S, Fatem E. Nonlnear total varaton based nose removal algorthms[j]. Physca D: Nonlnear Phenomena, 1992, 60(1-4): [5] Chambolle A. An algorthm for total varaton mnmzaton and applcatons[j]. Journal of Mathematcal magng and vson, 2004, 20(1): [6] Elad M, Aharon M. Image denosng va sparse and redundant representatons over learned dctonares[j]. IEEE Transactons on Image processng, 2006, 15(12): [7] Protter M, Elad M. Image sequence denosng va sparse and redundant representatons[j]. IEEE Transactons on Image Processng, 2009, 18(1): [8] Buades A, Coll B, Morel J M. A non-local algorthm for mage denosng[c]//computer Vson and Pattern Recognton, CVPR IEEE Computer Socety Conference on. IEEE, 2005, 2: [9] Dabov K, Fo A, Katkovnk V, et al. Image denosng by sparse 3-D transform-doman collaboratve flterng[j]. IEEE Transactons on mage processng, 2007, 16(8): [10] Maral J, Bach F, Ponce J, et al. Non-local sparse models for mage restoraton[c]//computer Vson, 2009 IEEE 12th Internatonal Conference on. IEEE, 2009: [11] Aharon M, Elad M, Brucksten A. rmk-svd: An algorthm for desgnng overcomplete dctonares for sparse representaton[j]. IEEE Transactons on sgnal processng, 2006, 54(11): [12] Maral J, Bach F, Ponce J, et al. Onlne dctonary learnng for sparse codng[c]//proceedngs of the 26th annual nternatonal conference on machne learnng. ACM, 2009: [13] Maral J, Bach F, Ponce J. Task-drven dctonary learnng[j]. IEEE transactons on pattern analyss and machne ntellgence, 2012, 34(4): [14] Zhang Q, L B. Dscrmnatve K-SVD for dctonary learnng n face recognton[c]//computer Vson and Pattern Recognton (CVPR), 2010 IEEE Conference on. IEEE, 2010: [15] Maral J, Elad M, Sapro G. Sparse representaton for color mage restoraton[j]. IEEE Transactons on mage processng, 2008, 17(1): [16] Jang Z, Ln Z, Davs L S. Label consstent K-SVD: Learnng a dscrmnatve dctonary for recognton[j]. IEEE Transactons on Pattern Analyss and Machne Intellgence, 2013, 35(11): [17] Dong W, Zhang L, Sh G, et al. Image deblurrng and super-resoluton by adaptve sparse doman selecton and adaptve regularzaton[j]. IEEE Transactons on Image Processng, 2011, 20(7): [18] Jung M, Bresson X, Chan T F, et al. Nonlocal Mumford-Shah regularzers for color mage restoraton[j]. IEEE transactons on mage processng, 2011, 20(6): [19] Dong W, Zhang L, Sh G, et al. Nonlocally centralzed sparse representaton for mage restoraton[j]. IEEE Transactons on Image Processng, 2013, 22(4): [20] Zha Z, Lu X, Zhang X, et al. Compressed sensng mage reconstructon va adaptve sparse nonlocal regularzaton[j]. The Vsual Computer, 2016: [21] Elmoataz A, Lezoray O, Bougleux S. Nonlocal dscrete regularzaton on weghted graphs: a framework for mage and manfold processng[j]. IEEE transactons on Image Processng, 2008, 17(7): [22] Peyr G. Image processng wth nonlocal spectral bases[j]. Multscale Modelng & Smulaton, 2008, 7(2):