Research Article. A Novel Spectral Clustering and its Application in Image Processing. Gu Ruijun*, Chen Shenglei and Wang Jiacai

Transcription

1 Jestr Journal of Engneerng Scence and echnology Revew 6 (3 ( Research Artcle JOURNAL OF Engneerng Scence and echnology Revew A Novel Spectral Clusterng and ts Applcaton n Image Processng Gu Rujun*, Chen Shengle and Wang Jaca School of Informaton Scence, Nanjng Audt Unversty,Nanjng, 85, Chna Receved 5 June 0; Accepted 3 January 03 Abstract hs paper proposes an mproved spectral clusterng algorthm based on neghbour adaptve scale, who fully consders the local structure of dataset usng neghbour adaptve scale, whch smplfes the selecton of parameters and makes the mproved algorthm nsenstve to both densty and outlers. hs paper llustrates the proposed algorthm not only has nhbton for certan outlers but s able to cluster the data sets wth dfferent scales. Experments on UCI data sets show that the proposed method s effectve. Some experments were also performed n mage clusterng and mage segmentaton to demonstrate ts excellent features n applcaton. Keywords: Spectral graph theory, Spectral clusterng, Neghbour adaptve scale, Image segmentaton. Introducton Clusterng s one popular data analyss method and has been wdely used n pattern recognton, mage processng and data mnng. Based on varable densty of dataset, DBSCAN[] can deal wth datasets wth arbtrary shape, but an napproprate choce of parameters may yeld poor result. Spectral clusterng algorthms have seen an explosve development over the past years and been successfully used n mage clusterng and mage segmentaton[]. hey can deal wth arbtrary dstrbuton dataset and easy to mplement. However, spectral clusterng also has some weaknesses[3][4], for example, t s senstve to the datasets wth dstnctly dfferent denstes and the parameters of algorthm must be selected cautously. hs paper proposes an mproved spectral clusterng algorthm who fully consders the local structure of dataset lke DBSCAN. Neghbour adaptve scale smplfes the selecton of parameters and makes the mproved algorthm nsenstve to both densty and outlers. o extend ts applcaton doman, a robust mage clusterng usng mage dstance was proposed. Experments on mages returned by search engne show that the proposed method s effectve. he rest of the paper s organzed as follows. In Secton, Spectral clusterng algorthm s summarzed brefly. In Secton 3, the proposed adaptve spectral clusterng algorthm s descrbed. In Secton 4, experments are presented and the results are dscussed. Fnally, a concluson s provded n Secton 5.. Spectral clusterng * E-mal address: grj79@hotmal.com ISSN: Kavala Insttute of echnology. All rghts reserved. Spectral clusterng can be nterpreted n mult-vew, such as graph cut theory [][5], random walks pont of vew [6] and perturbaton theory [7]. Spectral clusterng n the end s the problem of fndng egenvectors of Laplacan matrx, and then clusterng egenvectors nto clusters. Normalzed Cut based-on spectral clusterng algorthm [8] s a representatve one [9] (.e. standard spectral clusterng. Gven a set of n l ponts X={x,,x n } n R, cluster them nto c clusters as follows: n n computer affnty matrx A R, n whch A =0 and j = exp( x xj / σ A ( j ( Construct Laplacan matrx L=D -/ AD -/, n whch D s dagonal matrx defned as D = j= A j. 3 Fnd f,, f c, the c largest egenvectors of matrx L n c and form the matrx F = [ f,... f c ] R (normalzaton when requred. 4 Normalze the rows of F to be unt length,.e. c / j = j /( j= j F F F. k 5 reatng each row of F as a pont n R, cluster nto clusters usng k-means algorthm or any other sensble clusterng algorthm. 6 Assgn the orgnal pont x to cluster j f and only f row of F was assgned to cluster j. We can see that two parameters nfluence the fnal clusterng result and they are scale parameter σ n affnty functon and clusters number c. In [8] the parameterσ s chosen based on that value of σ that gves the least dstorted clusters by repeated experments. It s obvously that the value of σ depends on specfc problem and n

2 Gu Rujun, Chen Shengle, Wang Jaca /Journal of Engneerng Scence and echnology Revew 6 (3 ( detectng the range of σ s dffcult. Moreover, repeated experments wll ncrease the compute complexty. In concluson, standard spectral clusterng s senstve to σ and specally not robust to dataset wth varable denstes. Besdes, spectral clusterng can fnd clusters n nonsphercal-shape dstrbuton wth approprateσ, however, k-means etc can not do. 3. Adaptve Spectral clusterng algorthm based on neghbour adaptve scale A = exp( x x / σσ (3 j j j because scaleσ vares wth neghbour dstrbuton and s adaptve to local structure. hs trat enlarges the affnty between two ponts n the same cluster and reduces that n dfferent clusters as Fg. shows. For convenence, assume that x t -x p = x t -x q. If constant scale σ s used, accordng formula (, we can obtan tp = exp( xt xp / σ = exp( xt xq / σ = tq A A, whch shows that f arbtrary par ponts have the same dstances they have the same affntes. In fact, x t and x p are located n one cluster whereas x t, and x q n dfferent clusters, so f Atp > A, spectral clusterng wll beneft tq from ths. If neghbour adaptve scale s adopted, then σ > σ and p q A = exp( x x / σσ = exp( x x / σσ tp t p t p t q t p > exp( x x / σσ = A. t q t q tq Fg. heory of spectral clusterng From above analyss we know that maybe a varyngσ s more approprate for spectral clusterng. Inspred by DBSCAN, for each pont x an adaptve scaleσ s set, whch consders the local dstrbuton of neghbours of x. Let σ k = x xm k m= ( where σ s the average dstance between x and ts k nearest neghbours, called the Neghborhood Adaptve Scale (ASC of pont x. However, self-tunng spectral clusterng n [3] only consders the kth neghbour (usually k=7 and tends to be affected by outlers. Smlar to [3], affnty between pont x and x j s defned as It s obvous that neghbour adaptve scale makes ponts closer n one cluster and farther n dfferent clusters. Replace the frst step of standard spectral clusterng wth above algorthm, the mproved spectral algorthm s named Adaptve Spectral Clusterng (ASC. ASC can dstngush clusters wth dfferent denstes. ASC utlzes average dstance to tune affnty by means of the dea of DBSCAN. ASC s resstant to outlers, because σ s determned by the average dstance, whch s more relable than the nearest neghbour dstance or kth neghbour [3] dstance. ASC does not have a bas towards a partcular cluster shape or sze compared wth k-means. 4. Expermental results and analyss o valdate the effcency of the ASC, we conducted extensve experments usng two publcly avalable datasets, and then compared ASC wth k-means, standard spectral clusterng and self-tunng spectral clusterng. able I. Clusterng Comparson on UCL Database Standard spectral Datasets k- clusterng Self-tunng ASC spectral Dataset Sample Categores Attrbutes means σ =0.d r σ =0.d r clusterng k=3 k=5 sze Glass WDBC Image segment Irs Ionosphere average

3 Gu Rujun, Chen Shengle, Wang Jaca /Journal of Engneerng Scence and echnology Revew 6 (3 ( Experment on the UCI dataset Next, fve real datasets selected from UCI [] database were used to evaluate ASC. o evaluate clusterng performance quanttatvely, we adopt popular Rand ndex as evaluaton, he Rand ndex has a value between 0 and, wth 0 ndcatng that the two data clusters do not agree on any par of ponts and ndcatng that the data clusters are exactly the same. he value of σ s typcally set to 0 to 0 percent of the total range of the feature dstance functon d r (d r =max(d- mn(d accordng to []. Wthout loss of generalty, the parameters of these algorthms were set as follows: ( standard spectral clusterng: σ =0.d r, σ =0.d r ; ( self-tunng spectral clusterng: k=7; (3 ASC: k=3, k=5. ABLE shows the attrbutes of UCI database and the expermental results of four clusterng algorthms. As can be seen, ASC almost acheved all optmal RIs, moreover, the values are very near when k=3 or k=5. We can conclude that ASC s the most stable algorthm of them and nsenstve to k. Although the RIs of standard spectral clusterng vared wdely, at most one RI s larger than that of k-means clusterng. hat s to say standard spectral clusterng s certan to outperform k-means clusterng f a sutable σ s set. Self-tunng spectral clusterng s also not stable, because t just consders the seventh neghbour and s sensble to outlers. 4. Experment on mage clusterng he goal of mage clusterng s to fnd a mappng of the archve mages nto clusters such that the set of clusters provde nearly the same nformaton about the entre mage collecton [3][4]. o mprove the accuracy of mage clusterng, we ntroduce a novel mage dstance to measure the smlarty between the mages. hen neghbour spectral clusterng s performed on the mage smlarty matrx. Unlke the tradtonal Eucldean dstance, IMD [5] takes nto account the spatal relatonshps of pxels. herefore, t s robust to small perturbaton of mages. Let vetor x, y be two m by n mages, x =(x, x,, x, y =( y, y,, y, the Eucldean dstance d E(x, y s gven by E(, = ( = d xy x y (4 Suppose e, e,, e s a base of -dmensonal mage space, the metrc coeffcents g j s defned as g =< e, e >= < e, e > < e, e > cosθ (5 j j j j j where <,> s the scalar product and θ s the angle j between e and e j. hen, the IMD of two mages x, y s wrtten by where G = ( s a symmetrc postve defnte g j matrx. g j can be represented as a Gaussan functon gj = exp{ / } P Pj σ (6 πσ, j= where P -P j s the pxel dstance between P and P j. Let σ =, then IMD between mage x and mage y s defned as d (, exp{ P P /}( x y ( x y IMD xy = j j π, j = o reduce computaton complexty, G s decomposed by / / / / G = ΓΛΓ = ( ΓΛ Γ ( ΓΛ Γ = G G (8 where Λ s a dagonal matrx whose elements are egenvalues of G and Γ s an orthogonal matrx whose / colu vectors are egenvectors of G. Let u = G x, / v = G y, so d (, xy = ( xy G G ( x y = ( uv( uv / / IMD whch has the form of Eucldean dstance. Because G s only assocated wth the sze of an mage, so can be computed before transform s performed. Utlzng the property of IMD and spectral clusterng, we propose a robust mage clusterng method named IMD-ASC, by replacng Eucldean dstance wth IMD. IMD- ASC can be descred as follows [6]. Step: For N mages (m by n, compute G / accordng to equaton (7 and equaton (8; Step: For mage x, x j (,j=,,,n, let u =G / x, u j =G / x j then d ( x, x = ( x x G( x x = ( u u ( u u IMD j j j j j Step3: Replace the Eucldean dstance n ASC by IMD to perform spectral clusterng. Utlzng Google search engne, mages were collected by nputtng keywords lke apple, Afrca, Nanjng separately. hen the former 500 mages were selected as the nput of IMD-ASC. Clusterng results were partally fgured out n Fg.-3. In Fg., mages were clustered nto three man clusters dentfed by apples for eatng, Apple logo and Apple products. In Fg.3, elephants, landscape and eagles are the typcal Afrca elements. IMD j j, j= d (, xy = g ( x y ( x y = ( xy Gx ( y,

4 Gu Rujun, Chen Shengle, Wang Jaca /Journal of Engneerng Scence and echnology Revew 6 (3 ( Fg.. Image clusterng result for keyword apple by IMD-ASC (a (b (c (d Fg. 4 Gray mage segmentaon result by ASC( k=5 Fg. 3. Image clusterng result for keyword Affrca by IMD-ASC 4.3 Experment on mage segmentaton Image segmentaton [7] s the process of dvdng an mage nto multple parts, whch s a dffcult problem n computer vson. Image segmentaton s typcally used to dentfy objects or other relevant nformaton n dgtal mages. ASC can also be used on mage segmentaton. Fg.4 shows the result usng ASC( k=5 on two typcal gray mages. Notce that for both mages, our method can clearly segmented the object from the background. 5. Conlusons and future work ASC algorthm makes each pont wth a dfferent neghbour adaptve scale computed accordng to the mean dstance to ts k nearest neghbors. Neghbour adaptve scale smplfes the selecton of parameters and makes the mproved algorthm nsenstve to both densty and outlers. Expermental results show that, compared wth k-means and standard spectral clusterng, our algorthm can acheve better clusterng effect on artfcal datasets and UCI publc databases. However, ASC s sensble to noses and the number of clusters s dffcult to select. Besdes, computatonal complexty s ncreased. For NAS, some quanttatve comparson has been performed. But for mage clusterng and segmentaton, only qualtatve results were concluded. It s expected that usng evaluaton to mprove the performance further. References Acknowledgements hs research was funded by a grant (No.777/ G0 from the Natonal Natural Scence Foundaton of Chna and the Prorty Academc Program Development of Jangsu Hgher Educaton Insttutons (Audtng Scence and echnology.. M. Ester, H. P. Kregel, J. Sander, and X. Xu, A Densty-based Algorthm for Dscoverng Clusters n Large Spatal Databases wth Nose, n Proc. of the nd Internatonal Conference on Knowledge Dscovery and Data mnng. Portland, Oregon: AAAI Press, pp.6 3, J. Sh, and J. Malk, Normalzed Cuts and Image Segmentaton, IEEE rans. on Pattern Analyss and Machne Intellgence, vol., pp , Aug Z. M. Lh, and P. Petro, Self-tunng Spectral Clusterng, n Advances n Neural Informaton Processng Systems 7, Cambrdge, MA: MI Press, pp ,

5 Gu Rujun, Chen Shengle, Wang Jaca /Journal of Engneerng Scence and echnology Revew 6 (3 ( U. von Luxburg, O. Bousquet, and M. Belkn, Lmts of Spectral Clusterng, n Advances n Neural Informaton Processng Systems 7. Cambrdge, MA: MI Press, pp , F. R. K. Chung, Spectral Graph heory, Provdence, R.I.: Amercan Mathematcal Socety, M. Mela, and J. Sh, A Random Walks Vew of Spectral Segmentaton, n Proc. of 8th Internatonal Conference on Artfcal Intellgence and Statstcs. Key West, FL, U. von Luxburg, A utoral on Spectral Clusterng, Journa Statstcs and Computng, vol.7,pp , Dec A. Ng, M. Jordan, and Y. Wess. On Spectral Clusterng: Analyss and an Algorthm, n Advances n Neural Informaton Processng Systems 4. Cambrdge, MA: MI Press, D. Verma, and M. Mela, A Comparson of Spectral Clusterng Algorthms, Unversty of Washngton, ech Rep: UW-CSE , 003. [Onlne]. Avalable: ftp://ftp.cs.washngton.edu/tr/003/05/ UW-CSE PS.Z 9. B. Fel, and J.Abony, Geodesc Dstance Based Fuzzy Clusterng, n Proc. of th Onlne World Conference on Soft Computng n Industral Applcatons, 006. [Onlne]. Avalable: Armstrong.edu /wsc/ndex.html. 0. C. Blake E. Keogh, and C. J. Merz, UCI Repostory of Machne Learnng Databases, Dept. Inf. Comp. Sc., Unv. Calforna, Irvne, 998. [Onlne]. Avalable: MLRepostory. html. W. M. Rand, Objectve Crtera for the Evaluaton of Clusterng Methods, Journal of the Amercan Statstcal Assocaton, pp , 97.. S. Gordon, H.Greenspan,"Unsupervsed Image-set Clusterng Usng an Informaton heoretc Framework", IEEE rans. on Image Processng,vol 5,pp ,Feb H. L,X.Huang, A Herarchcal Image Clusterng Cosegmentaton Framework, n Proc. of the IEEE CVPR, pp , L. Wang, Y. Zhang and J. Feng, "On the Eucldean Dstance of Images", IEEE rans. on Pattern Analyss and Machne Intellgence, vol. 7, pp , Aug C. Pughneanu, I. Balan. Parallel Algorthm Evaluaton n the Image and Clusterng Processng, Electroncs and Electrcal Engneerng. Vol0,No 4,pp.89-9, D. Batra, A. Kowdle, D. Parkh, J. Luo, and. Chen. Icoseg: Interactve Co-Segmentaton wth Intellgent Scrbble Gudance, n Proc. of the IEEE CVPR, pp , Y, Fang, L. an and B. Han. An Improved Watermarkng Algorthm to Colour Image Based on Wavelet Doman,Journal of Engneerng Scence and echnology Revew. vol.6,pp.39-44,0. 4