A Study on Clustering for Clustering Based Image De-Noising

Transcription

1 Journal of Informaton Systems and Telecommuncaton, Vol. 2, No. 4, October-December A Study on Clusterng for Clusterng Based Image De-Nosng Hossen Bakhsh Golestan* Department of Electrcal Engneerng, Sharf Unversty of Technology, Tehran, Iran h.b.golestan@gmal.com Mohsen Joned Department of Electrcal Engneerng and Computer Scence, Unversty of Central Florda, Orlando, USA Joned@knghts.ucf.edu Mostafa Sadegh Department of Electrcal Engneerng, Sharf Unversty of Technology, Tehran, Iran m.saadegh@gmal.com Receved: 24/Aug/2013 Revsed: 28/May/2014 Accepted: 07/July/2014 Abstract In ths paper, the problem of de-nosng of an mage contamnated wth Addtve Whte Gaussan Nose (AWGN) s studed. Ths subject s an open problem n sgnal processng for more than 50 years. methods suggested n recent years, have obtaned better results than global methods. However by more ntellgent tranng n such a way that frst, mportant data s more effectve for tranng, second, clusterng n such way that tranng blocks le n low-rank subspaces, we can desgn a dctonary applcable for mage de-nosng and obtan results near the state of the art local methods. In the present paper, we suggest a method based on global clusterng of mage constructng blocks. As the type of clusterng plays an mportant role n clusterng-based de-nosng methods, we address two questons about the clusterng. The frst, whch parts of the data should be consdered for clusterng? and the second, what data clusterng method s sutable for de-nosng.? Then clusterng s exploted to learn an over complete dctonary. By obtanng sparse decomposton of the nosy mage blocks n terms of the dctonary atoms, the de-nosed verson s acheved. In addton to our framework, 7 popular dctonary learnng methods are smulated and compared. The results are compared based on two major factors: (1) de-nosng performance and (2) executon tme. Expermental results show that our dctonary learnng framework outperforms ts compettors n terms of both factors. Keywords: Image De-Nosng; Data Clusterng; Dctonary Learnng; Hstogram Equalzaton and Sparse Representaton. 1. Introducton We consder the problem of estmatng a clean verson of an mage contamnated wth Addtve Whte Gaussan Nose (AWGN). A general approach to ths am s dvson of the nosy mage nto some (overlappng) small blocks, then de-nosng of each block and fnally obtanng the overall estmaton of the clean mage by averagng the de-nosed blocks. The model s as follows: where s the vector form of the th block of the nosy mage, s the vector form of the th block of the orgnal mage, and s a zero-mean AWGN wth varance. Throughout the paper, the blocks are, thus the vector space dmenson s. Image de-nosng s stll an open problem and numerous methods have been suggested up to now. The methods are based on defnng a neghborhood for each block and weghted averagng accordng to sutable weghts. The weghts are computed n each neghborhood, as n [1-4] whch are some relatvely successful approaches. All of them are n the spatal doman. The method n [5] can be consdered as same as [1-4], where processng s conducted n frequency doman. Ths method constructs a three-dmensonal matrx by (1) groupng those blocks that are smlar (n some senses, e.g. norm) wth a block of the mage. Correspondng to each block of the mage a group of smlar blocks should be found. In ths way, a three-dmensonal matrx s obtaned correspondng to each block. Then, a 3D collaboratve sgnal flterng n the frequency doman s performed whch provde a good estmaton of the clean verson of each block. Ths method can be consdered as the state of the art method of mage de-nosng; however t suffers from hgh computatonal complexty due to local processng. The work n [6] has the same approach and appled flterng n the Prncpal Component Analyss (PCA) transform doman. Elad and Aharon [7] have suggested a new approach. They have used K-Sngular Value Decomposton (K-SVD), whch s a dctonary learnng algorthm, to produce a global dctonary usng the nosy mage blocks. Ths method uses the representaton n terms of the dctonary to de-nose mage. The estmate of each de-nosed block can be estmated by analyzng nosy blocks n ths dctonary and applyng a sparse recovery algorthm. and global methods have some advantages and dsadvantages. A global dctonary can recover general characterstcs of an mage, whch are repeated n ts several regons. However, these methods are not able to recover specal local textures and detals n an mage. * Correspondng Author

2 Journal of Informaton Systems and Telecommuncaton, Vol. 2, No. 4, October-December Whle local methods ndcate hgher effcency n recoverng local detals of mage, they encounter overlearnng rsk leadng from nose learnng and ncorporatng nose nto the fnal result. Defcency of learnng n some regons s another problem of local methods. In [8], a clusterng-based method was suggested. Ths method produces a local dctonary by clusterng feature vectors from all nosy mage blocks and conducts de-nosng usng decomposton of nosy blocks n terms of representatves of the found clusters. Smlar to K-SVD, ths method s based on dctonary but t uses a local dctonary. patchng and smlar blocks clusterng are effectve factors n success of methods ncludng [5], [6] and [8]. Dctonary learnng based de-nosng methods also perform some type of blocks clusterng, for example K-SVD s a generalzaton of K-means clusterng algorthm. So t s necessary to consder the clusterng for the de-nosng applcaton more closely. In ths paper, we propose an approach for constructng a global dctonary and de-nosng based on sparse decomposton of nosy blocks over the dctonary. Ths global dctonary s constructed by ad of the optmzed clusterng that wll be presented. In the followng sectons clusterng of mage blocks s studed wth more detals n secton 2. An analytcal comparson between local and global clusterng s addressed n secton3. Secton 4 studes the effect of equalzaton of data accordng to ther varance n order to have an approprate clusterng. Learnng the dctonary s explaned n secton 5 based on representatves of the found clusters. Secton 6 studes applyng of de-nosng usng dctonary. Fnally, the local and global methods are evaluated n secton 7. wth data correspondng to hgh energy areas as outler data. So, these blocks have mnor effect on the tranng by common clusterng methods and the fnal desrable result wll not be obtaned. To solve the problem, frst lmtatons of clusterng-based de-nosng methods are examned. The MSE error lower bounds for mage de-nosng have been examned n [9] and [10]. Ths lower bound for one cluster block s calculated as follow. [ ] [( ) ] (2) (3) 2. Clusterng of Image Blocks In the case of methods ncludng LPG-PCA, KLLD, BM3D ([6], [8] and [5], respectvely), groupng of smlar blocks s ther crtcal factor of success. So, blocks groupng may has detals whch should be consdered specfcally. BM3D and LPG-PCA perform de-nosng by clusterng of the set of mage blocks. K-LLD method performs clusterng on feature vector extracted from surroundng blocks (Correspondng to each block). Consderng the number of pxels and feature vector dmenson, ths clusterng s of hgh computatonal load. In addton to hgh computatonal load, unbalanced clusterng s one of the problems of global clusterng of blocks. Ths problem s shown n Fgure 1. Assume that n Fgure 1-bottom, the goal s to fnd 2 means. K-means algorthm fnds two datacenters ndcated by volet crcles. These ponts are not good representatves of the blocks correspondng to the mage edges. However, clusterng objectve functon s mnmzed by these centers. Dense (hgh number data) correspond to mage smooth parts and scattered (low number data) correspond to blocks contanng edge or specal texture. Tradtonal clusterng algorthms behave Fg 1. In natural mages, number of smooth blocks are more than hgh energy ones. where, s the Fsher nformaton matrx and s the estmated covarance matrx for the group of vectors that are smlar to th block. For zero mean Gaussan nose, [10] assumed matrx as follow: where, s the number of smlar vectors of the th block. Assumng that smlar vectors for each pxel are of many members and nose level s not hgh, the rght hand of nequalty s smplfed: (4) ( ) ( ) (5) [ ] ( ) (6)

3 198 Bakhsh Golestan, Joned & Sadegh, A Study on Clusterng for Clusterng based Image De-nosng [ ] ( ) (7) where s the th egenvalue of covarance matrx of estmated data : ( ) ( ) (8) Assumng that the number of smlar patches of each block and the nose level s the same for all blocks; thus de-nosng bound s related to covarance matrx. Hgh detaled clusters (havng hgh covarance matrx egenvalues) are more dffcult to de-nose. So for blocks correspondng to low complex areas, lower bound wll be decreased for MSE of the estmated verson and the orgnal mage. However the result s predctable; because n smooth areas of an mage, a smple averagng can obtan good result but f a block conssts of more complexty, specfc texture and hgh varance, would lmt de-nosng performance. For such blocks, more precse smlar block groupng s needed. The more the number of same blocks causes the more approprate characterstcs of groupng. So we suggest that for detaled and textured blocks, more tranng data should be used. Let us generalze the concept presented n (2) to clusters (rather than groups for each block). Assume varable s allocated for clusters rather than blocks n (2). In other words, s a block from the th cluster and s the number of members of the th cluster. s the estmated covarance matrx of the th cluster. Frst queston that ths paper s gong to answer s "whch blocks should be consdered for clusterng?" As stated before, usng all blocks for clusterng not only have hgh computatonal load but also leads to unbalanced clusterng. Fgures 4 and 5 llustrate the dea of equalzed clusterng. Fgure6 s the equalzed clustered of Fgure 1 provdng good propertes for de-nosng applcaton. Dctonary learnng-based methods such as K-SVD decrease tranng data n a random way to reduce computatonal load. But as have been seen, removng valuable blocks from tranng data has negatve effect on the de-nosng lower bound. In Fgure 6, only data correspondng to smooth blocks are removed and the obtaned cluster centers are more approprate for de-nosng. In secton 3 tranng data equalzaton wll be studed. Second queston that the paper s gong to answer s "how do the clusterng?" Now we state the problem of clusterng. Frst we rewrte (2) as follow: s the set of ndces of tranng data that shows membershp of the tranng data to clusters. The problem of the optmum clusterng can be stated as follows: ( ) (11) The above problem s dependent of Egenvalues of each cluster, so ts computatonal burden s very hgh. Thus, exact soluton of the problem s not achevable. Egen values of the clusters correspondng to smooth or constant regons of are about zero so they can be neglected from ( ). So, only hgh varance blocks affect the cost functon. ( ) (12) In other words, smooth tranng data can be gnored n the clusterng. At the frst glance ths smplfcaton just makes the clusterng fast but t has an effect on the accuracy of the clusterng. In fact, less explotaton of nonmportant blocks causes n more affecton of mportant blocks n the clusterng problem (compare fgure 1 and fgure 6). Eq. (12) can be nterpreted as a hard threshold for selecton of blocks n clusterng. In the next secton varance of blocks wll be ntroduced as a crteron for smoothness and then varance hstogram equalzaton wll be presented as the soft threshold verson of (12) for selecton of data that partcpate n clusterng. Problem (11) can be vewed from another pont of vew. The cost functon encourages clusters to have a sparse vector of Egen values. Fgure 2 shows how (11) encourages Egen values to be zero. In other words problem (11) clusters data nto low-rank subspaces and guarantees that many of Egen values wll be zero for each cluster. [ ] (9) Let us wrte the rght sde of ths nequalty for all clusters as a cost functon: Fg 2. Contour of cost functon of (11) for a cluster 1. ( ) (10) 1 The fgure s contour of s true for contour of (11), as values of are postve, fgure 2

4 Journal of Informaton Systems and Telecommuncaton, Vol. 2, No. 4, October-December Hgh dmensonal data that le n low-rank subspaces have hgh correlaton wth each other (see Fgure 3). An alternatve for subspace clusterng may be correlaton clusterng [11] that has much less computatonal load. As can be seen n Fgure 3, the obtaned clusters by correlaton clusterng le n a rank-1 subspace that agrees wth problem (11) because only one Egen value of the covarance matrx of ths cluster s none-zero. In secton 6 smulatons has been done by correlaton clusterng. also contans domnant PCs spannng detals of each cluster). Those PCs would be added to the dctonary f ther correspondng egenvalues are greater than nose varance. The nosy mage blocks are then de-nosed nspred by the framework used n [7]. Ths leads to a fast and effcent de-nosng algorthm (algorthm1). It wll be shown n Secton 7 that the proposed algorthm outperforms tradtonal K-SVD. Fg 3. Comparson of correlaton clusterng and tradtonal clusterng. 3. Clusterng vs. Clusterng A well-known clusterng method s the famly of K- means clusterng algorthms [12], whch have been used by K-LLD [8] for mage de-nosng. K-means clusterng algorthm solves the followng problem K 2 yj dk D 2 (13) k =1 j k mn where, [ ]. Ths problem can be wrtten n the followng form whch s a factorzaton mn DX, Y DX j x x 2 j F,, : 0 = 1, 0,1 (14) where, Y=[ ] (L s the number of blocks), s the th column of X, and s the jth entry of. Ths problem mples that all entres of each must be equal to zero except one of them. The non-zero element s forced to be 1. Ths restrcton does not exst n the socalled gan-shaped varant of K-means [12], whch solves the followng problem DX, 2 Y DX subject to : x 0= 1 (15) mn F Ths problem s a K-rank1 subspace (K-lnes) clusterng. As can be seen n Fg. 4 (b) and (d), the obtaned clusters by gan-shaped K-means s n agreement wth problem (11). Ths s because only one egenvalue of each cluster s covarance matrx s non-zero. Inspred by the smple approach (15), a suboptmal soluton for (11) can be obtaned. We propose to construct the proper bass usng the obtaned cluster centrods and domnant prncpal components (PCs) of each cluster (generally, natural mages are not perfectly le on rank-1 subspace as n Fg. 4,.e., thus the proposed dctonary Another approach for clusterng s dctonary learnng n sparse sgnal representaton, whch ams to solve the followng problem 2 mn Y DX F subject to : x 0 (16) DX, K-SVD s a well-known dctonary learnng algorthm. Low-rank subspaces found by K-SVD have overlaps. It means that correspondng to each subset of the columns of D, there s a low-rank subspace that K-SVD learns. Data that used the same subset le on a low-rank subspace but K-SVD learns a very large number of low-rank subspaces for a set of tranng data such that many of them are empty or low populated (refer to Fg. 5, top). Actually, clusters found by K-SVD nclude the data that have used the same dctonary columns. Note that these clusters are not guaranteed to be low-rank. In the smulaton results we wll see that our proposed method based on gan-shaped K-means outperforms K-SVD. The derved problem (11) descrbes a sutable global clusterng problem, whle the state of the art algorthms do not perform global clusterng, but nstead use local patch-groupng. Translatng global clusterng to local groupng converts the problem to, G = mn G,G W, G G G 0 subject to (17) where, s group of blocks correspondng to the th block, s the egenvalues of covarance matrx of and s a wndow around the th block. The last constrant mples that the th block must be member of. An equvalent form of (17) can be stated as, G = max G subject to,g W, G G G 0 (18) BM3D, a hgh performance mage de-nosng algorthm, mplctly uses (18) n order to perform local groupng. The smlarty crteron used n BM3D for

5 200 Bakhsh Golestan, Joned & Sadegh, A Study on Clusterng for Clusterng based Image De-nosng performng local groupng s novel, n whch frstly blocks are transformed usng an orthonormal transformaton (e.g., DCT and DFT), then a projecton on a low-rank subspace s performed usng hard-thresholdng of the coeffcents of each block. In the new transformed space, a smple Eucldean dstance determnes smlar blocks wth the th block. Truncated coeffcents of the smlar blocks wth the th one also le on a low-rank subspaces near to the th one, thus many of are about zero and the constrant of (18) s satsfed. that have the same sparse representaton (structure) n one cluster. Fg 5: Top: K-SVD approxmates data by a unon of rank-2 subspaces. No rank-2 cluster can be found. Bottom: Group sparsty constrant on X. There are three rank-2 clusters. Another local groupng based method s a novel approach, called learned smultaneous sparse codng (LSSC) [14], that smultaneously performs group sparse codng [15] and groupng the smlar patches. Group sparse codng mples that the blocks wthn a group have smlar sparse representatons, lke CSR. Ths s acheved by jontly decomposng groups of smlar sgnals on subsets of the learned dctonary (as prevously explaned, K-SVD fals to acheve ths goal. See Fg. 5 for comparson). They proposed the followng cost functon, Fg 4. Comparson of clusterng n raw data doman and n the sparsedoman transformed data (as used n CSR and LSSC) for some 2D data. (a) Raw data. (b) K-means clusterng on raw data (K=3). (c) K-means clusterng on sparse-doman transformed data usng an over-complete dctonary havng 3 atoms. (d) Reconstructon of the data from ther sparse representatons n (c), n the case of these data Gan-shaped K- means drectly results n (d). The dea behnd (18) can be used n another way dfferent from what BM3D has used. These de-nosng algorthms frst perform groupng usng a rough crteron, e.g. Eucldean dstance, then n the man de-nosng algorthm obtan a low-rank representatve for each group and use t. The algorthm suggested by Dong et al. (clusterng based sparse representaton or CSR) [13] whch solves the followng problem, s an example of these types of algorthms mn X, B K 2 2 F j k 2 k =1 jg k (19) Y DX x x b where [ ], and s the centrod of the kth group. Note that (19) does not optmze the dctonary. In fact, frstly a global dctonary usng K-means and PCA s learned whch s then used by ths problem to smultaneously perform local groupng and sparse codng, n an teratve procedure. The frst and second terms n (19) are smlar to K-SVD problem, but the last term clusters the sparse-doman transformed data. Fgure 5 llustrates the effect of clusterng data n the sparse doman rather than the raw data. Contrary to K-SVD, n whch the members of a cluster have used one column of D, problem (19) encourages the clusterng to put data X k K k k p, q.. : 2 k=1 G k mn X s t k y Dx (20) where, s the coeffcent matrx of the kth cluster data, s the jth column of, and [ ], wth [ ] the th row of X. Mnmzng wth p=1 and q=2 (that s, the norm of the vector contanng the norms of the rows) mples that the number of engaged rows of X wll be lmted. In other words, ths cost functon encourages the data to have the same support of sparse representaton n a cluster. As the data n the same cluster can be decomposed by few bases, the rank of the data matrx n the same cluster wll be mnmzed. Thus a soluton for (20) tres to mnmze (17)..e, approxmates. At the smulaton results secton, numercal performances of the explaned local and global methods are compared, separately. 4. Block Varance Hstogram Equalzaton For the reasons prevously stated some ponts should be consdered. Frstly clusters wth dfferent complextes have approxmately the same number of members. Secondly, members of complcated clusters should not have hgh dstance from cluster subspace so that covarance matrx egenvalues would not become hgh and many of them would be zero. Thrd, members of hgh complex clusters should not be neglected for dctonary learnng. Blocks varance s consdered as a complexty measure. In natural mages, the number of hgh complex blocks s

6 Journal of Informaton Systems and Telecommuncaton, Vol. 2, No. 4, October-December lower than low complex blocks. Fgure 6 ndcates blocks varance hstogram of an orgnal mage and ts nosy verson. As can be seen, n the orgnal mage, concentraton s n lower values of varance and n nosy mage concentraton s n the pont correspondng to nose varance representng smooth blocks of orgnal mage. Those blocks that ther varances are approxmately the same as nose varance are not useful for tranng. Usng these blocks not only ncreases computatonal load but also causes unbalance clusterng and reduces the effect of mportant clusters. So ther number n fnal clusterng should be reduced. To equalze blocks varance hstogram, an equalzaton transform functon must be used. The followng functon s an example: T( ) { ( ) ( ) ( ) (21) where, ( ) s densty functon of blocks varance probablty and s a threshold. ( ) s the probablty of enterng a block wth varance nto tranng data to be used for clusterng. Fgure 6, ndcates an example of the transform functon and equalzed hstogram of nosy mage n Fgure 7. In ths hstogram, the effect of blocks wth varance 25 s reduced consderably. Fgure 8 shows equalzed clusterng of fgure 1. and ther presence for tranng not only mslead the clusterng algorthm but also have hgh computatonal load. Now, subspace clusterng should be done on remanng tranng data whch agrees wth Eq. (11). Fg 8. Equalzed clusterng of fgure 1 5. Dctonary Learnng Dctonary learnng s performed usng the blocks selected n the prevous stage. The fnal dctonary ncludes domnant prncpal components from each cluster (equal to non-zero Egen values of matrx explaned n Secton 2). In the next stage, SVD transform s derved from covarance of data matrx of each cluster: (22) where, s the number of clusters. Sngular values on the man dagonal are equal to whch are arranged n ascendng order by subscrpt. For each cluster, s the number of prncpal components that wll be ncluded n the fnal dctonary and s obtaned by the followng equaton: Fg 6. two clear and nosy mages wth hstogram. and ther blocks varance ( ) ( ) (23) The prncpal components hgher than have learned nose for each cluster n matrx. Actually, s the dmenson of nose-free data on the th cluster (or s the rank of subspace that th cluster les n t). It means that f the nose power s zero, autocorrelaton matrx of th cluster has only non-zero egenvalues. In presence of nose, all autocorrelaton matrx egenvalues of each cluster of nosy data wll be nonzero; from the component to the end are due to nose. By addng the frst prncpal component to, the dctonary s completed and we can perform denosng by ths desgned dctonary. Fg 7. Equalzng transform functon Regardng that the varance of smooth blocks s approxmately the same as nose varance. It can be sad that there s not valuable nformaton about orgnal mage, 6. Denosng Operaton Usefulness of the unon of subspaces model has been proved n many applcatons of sgnal processng. As

7 202 Bakhsh Golestan, Joned & Sadegh, A Study on Clusterng for Clusterng based Image De-nosng llustrated n secton 2 and 3, ths model s approprate for the analyss of sgnal de-nosng. Ths model assumes that mage blocks are lnear combnaton of few bases of a dctonary: (24) In the prevous secton a dctonary was defned. Denosed mage should also meet ths model whereas nosy mage cannot, because n the dctonary learnng stage, nose s not traned. In other words, to represent nose, many bases combnaton should be nvolved and no sparse representaton n equaton (25) can be found. (25) The model must be reformed to model the nose of data: (26) Assumng Gaussan nose wth zero mean n ths model, MAP estmaton for s (27) Optmum threshold s related to of a cluster where belongs to t. Ths can be replaced by the followng problem: (28) where, s a functon of nose varance. Now we can estmate de-nosed verson by ths estmaton of sparse coeffcents. We just need to project nto the nearest low-rank subspace spanned by the columns of the learned dctonary. 7. Smulaton Results In ths secton, de-nosng results of proposed method and some other recent approaches are presented and dscussed. Frst, the global and local methods are evaluated, then a comparson between global and local approaches s presented and fnally these methods are compared n term of total executon tme. K-SVD and our smple gan-shaped K-means (proposed method) are presented as global methods. The presented local methods nclude those ntroduced n [5], [8], [13], [14], [17] and [18]. Performance comparson of these algorthms can be seen Table 1. We have used the Peak Sgnal to Nose Rato (PSNR 1 ) as the performance crteron. The PSNR values were averaged over 5 experments, correspondng to 5 dfferent realzatons of AWGN. The varance was neglgble and not reported. Our method s smulated smlar to the framework of [7]. Both algorthms have the same amount of error for the tranng set (dependng on the nose varance) but ther sze of dctonary s dfferent. Table 1 shows that the proposed method surpasses the K-SVD [7] and ts results are 1 PSNR s defned as 10log 10(255 2 /MSE) and measured n db comparable wth the tme consumng local methods. As wll be tabulated, the executon tme of the proposed method s about 70% of K-SVD, 8% of LSSC [14] and 4% of CSR [13]. Recently [16] nvestgated a comprehensve comparson of dfferent mage de-nosng methods. They have shown numercally that BM3D, SCR and LSSC studed n ths paper have the best results. Fgure 9 shows an example of de-nosng results by our proposed method. Table 1. Image de-nosng performance of the and methods n PSNR (db) for 4 dfferent mage and varous Lena SNR 5/ / /22.11 Proposed K-SVD [7] K-LLD [8] LSSC [14] CSR [13] BM3D [5] LSC [17] SSMS [18] Barbara SNR 5/ / /22.11 Proposed K-SVD [7] K-LLD [8] LSSC [14] CSR [13] BM3D [5] LSC [17] SSMS [18] House SNR 5/ / /22.11 Proposed K-SVD [7] K-LLD [8] LSSC [14] CSR [13] BM3D [5] LSC [17] SSMS [18] Boat SNR 5/ / /22.11 Proposed K-SVD [7] K-LLD [8] LSSC [14] CSR [13] BM3D [5] LSC [17] SSMS [18]

8 Journal of Informaton Systems and Telecommuncaton, Vol. 2, No. 4, October-December (a) Orgnal mage (b) Nosy mage (c) Recovered mage by proposed method Fg 9. an example of denosng results by our method In natural mages, far away block have generally dfferent patterns, so, usng all blocks may result n napproprate clusterng. Moreover, non-overlapped clusters obtaned by global methods are not as flexble as the overlapped groups. On the other hand, local groupng assgn approprate groups to each block. Although local methods have better performance, global methods are able to extract salent features of mages and use t easly for de-nosng. Accordng to comparson of local and global methods n Table 1, the performance of the proposed global method s just about 0.2dB lower than promsng local methods (LSSC and CSR), whch s not a hgh dfference. However, a common good property of both global and local methods s that they explot the lowdmensonal characterstcs of clusters/groups n order to desgn a sutable de-nosng algorthm. To understand the effect of ths method on the dctonary, n the table 2 the results are compared only wth K-SVD method, whch s global a method lke our proposed method. However, n the results of local methods n table 1, the suggested method used about 27% less blocks for tranng and the tme requred for dctonary learnng s less than K-SVD method. Ths table studes the effect of data equalzaton on K-SVD. As t can be seen equalzaton mproves K-SVD about 0.4dB. Table 2. comparng the suggested method and K-SVD method. left: K- SVD + Equalzaton of data, mddle: KSVD, rght: the proposed clusterng σ/snr House Peppers 20/ / / σ/snr Lena Cameraman 20/ / / As mentoned, the proposed method s based on dctonary learnng and ts tme effcency should be compared wth other dctonary learnng based approaches e.g. [7], [13] and [14]. Table 4 compares the relatve executon tme of [13], [14], [7] and the proposed method n varous mage szes. Our experments were averaged on 5 dfferent runs carred out on a Personal Computer wth a 3.6-GHz AMD 2 Core CPU and 4 GB RAM. As can be seen, the global de-nosng methods (KSVD and proposed) are more effcent n term of executon tme and our proposed method surpasses KSVD. In fact, dctonary learnng runnng tme of proposed method (for dentfcaton of K-rank1 subspaces) s about 40% of K-SVD for 20,000 blocks extracted from a mage, but ts overall executon tme s about 72% of KSVD. Table 3. Relatve executon tme of dctonary learnng based methods (n mnutes) Image Sze (QCIF) (CIF) (4CIF) LSSC [14] CSR [13] KSVD [7] Proposed Conclusons methods suggested n recent years, have obtaned better results than global methods. However by more ntellgent tranng n such a way that frst, mportant data s more effectve for tranng, second, clusterng n such way that tranng blocks le n low-rank subspaces, we can desgn a dctonary applcable for mage de-nosng and obtan results near the state of the art local methods. As was seen, we have obtaned acceptable results by a relatvely smple method based on constructon of an approprate global dctonary.

9 204 Bakhsh Golestan, Joned & Sadegh, A Study on Clusterng for Clusterng based Image De-nosng References [1] D. van de Vlle and M. Kocher, SURE-Based non-local means, IEEE Sgnal Process. Letter, Vol. 16, No. 11, Nov. 2009, pp [2] T. Tasdzen, Prncpal neghborhood dctonares for nonlocal means mage denosng, IEEE Trans. Image Process., Vol. 18, No. 12, Dec. 2009, pp [3] R. Vgnesh, B. T. Oh, and C.-C. J. Kuo, Fast non-local means (NLM) computaton wth probablstc early termnaton, IEEE Sgnal Process. Letter, Vol. 17, No. 3, Mar. 2010, pp [4] H. Takeda, S. Farsu, and P. Mlanfar, Kernel Regresson for Image Processng and Reconstructon, IEEE Transactons on Image Processng 16, 2007, pp [5] K. Dabov, A. Fo, V. Katkovnk, and K. O. Egazaran, Image denosng by sparse 3-D transform-doman collaboratve flterng, IEEE Trans. Image Processng, Vol. 16, No. 8, Aug. 2007, pp [6] L. Zhang, W. Dong, D. Zhang, and G. Sh, Two-stage mage denosng by prncpal component analyss wth local pxel groupng, Pattern Recognton,Vol. 43, 2010, pp [7] M. Elad and M. Aharon, Image denosng va sparse and redundant representatons over learned dctonares, IEEE Trans. Image Processng,Vol. 15, No. 12, Dec. 2006, pp [8] P. Chatterjee and P. Mlanfar, "Clusterng-based denosng wth locally learned dctonares," IEEE Trans. on Image Processng, No. 18, Vol. 7, 2009, pp [9] P. Chatterjee, and P. Mlanfar, Patch-Based Near-Optmal Image Denosng, Image Processng, IEEE Transactons on, vol. 21, no. 4, Aprl 2012, pp [10] P. Chatterjee, and P. Mlanfar, Practcal Bounds on Image Denosng: From Estmaton to Informaton IEEE Transactons on Image Processng, Vol. 20, No. 5, May 2011, pp [11] C. Bohm, K. Kalng, P. Kroger, and A. Zmek, "Computng clusters of correlaton connected objects" In Proc. SIGMOD, 2004, pp [12] A. Gersho and R. M. Gray, Vector Quantzaton and Sgnal Compresson, Sprnger Press, [13] W. Dong, X. L, D. Zhang and G. Sh, Sparsty-based mage denosng va dctonary learnng and structural clusterng, n proceedngs of IEEE Conference on Computer Vson and Pattern Recognton (CVPR), June 2011, pp [14] J. Maral, F. Bach, J. Ponce, G. Sapro and A. Zsserman, Non-local sparce models for mage restoraton, n Proceedngs of IEEE Internatonal Conference on Computer Vson, 2009, pp [15] J. A. Tropp, Algorthms for smultaneous sparse approxmaton, Elsever Sgnal Processng journal, Vol. 86, No. 3, 2006, pp [16] L. Shao, R. Yan, X. L, and Y. Lu, From Heurstc Optmzaton to Dctonary Learnng: A revew and comprehensve comparson of mage denosng algorthms, IEEE Transacton on Cybernetcs, Vol. 44, No. 7, 2014, pp [17] A. Adler, M. Elad, Yacov Hel-Or, Probablstc subspace clusterng va sparse representaton, IEEE Sgnal Processng Letter, Vol. 20, No. 1, 2013, pp [18] G. Yu, G. Sapro, and S. Mallat, Image modelng and enhancement va structured sparse model selecton, In Proceedng of ICIP, 2010, pp Hossen Bakhsh Golestan receved the B.Sc. and M.Sc. n Electrcal engneerng from Ferdows unversty of Mashhad, Mashhad, Iran and Sharf Unversty of Technology, Tehran, Iran, n 2010 and 2012 respectvely. Hs research nterests nclude multmeda sgnal processng, statstcal sgnal processng, vdeo/speech codng and compresson, sgnal qualty assessment and robotcs. Mohsen Joned receved the B.Sc. and M.Sc. n Electrcal engneerng from Ferdows unversty of Mashhad, Mashhad, Iran and Sharf Unversty of Technology, Tehran, Iran, n 2010 and 2012 respectvely. He s currently workng toward the Ph.D degree n department of electrcal engneerng and computer scence at the Unversty of Central Florda, Orlando, USA. Hs research nterests nclude multmeda sgnal processng, statstcal sgnal processng, sgnal qualty assessment and compressve sensng. Mostafa Sadegh receved the B.Sc. and M.Sc. n Electrcal engneerng from Ferdows unversty of Mashhad, Mashhad, Iran and Sharf Unversty of Technology, Tehran, Iran, n 2010 and 2012 respectvely. He s currently workng toward the Ph.D degree n department of electrcal engneerng at the Sharf Unversty of Technology, Tehran, Iran. Hs research nterests nclude statstcal sgnal processng, sparse representaton/codng and compressve sensng.