Image segmentation by using the localized subspace iteration algorithm

Transcription

1 Image segmentaton by usng the localzed subspace teraton algorthm Jnlong Wu and Tejun L May 8, 28 Abstract An mage segmentaton algorthm called segmentaton based on the localzed subspace teratons (SLSI) s proposed n ths paper. The basc dea s to combne the strateges n Ncut algorthm by Sh and Malk [Normalzed cuts and mage segmentaton, IEEE Trans. Pattern Anal. Mach. Intel. 22 (2), ] and the LSI by E, L and Lu [Localzed bass of egensubspaces and operator compresson, submtted to Proc. Nat. Acad. Sc., 27]. The LSI s appled to solve an egenvalue problem assocated wth the affnty matrx of an mage, whch makes the overall algorthm lnearly scaled. The choces of the partton number, the supports and weght functons n SLSI are dscussed. Numercal experments for real mages show the applcablty of the algorthm. Keywords: Image segmentaton; Localzed subspace teraton; Ncut; Graph Laplacan 1 Introducton We are nterested n egenvector-based methods for mage segmentaton. Ths knd of deas have attracted a great deal of attenton, see for example the Perona and Freeman algorthm (PF) [4], the Scott and Longuet-Hggns algorthm (SL) [7] and the Sh and Malk algorthm (SM) [8]. These algorthms use egenvectors of dfferent but smlar systems to segment mages and yeld satsfactory results n many cases. Related deas are also appled to perform dmensonalty reducton [6]. Of partcular nterest to us n ths paper s the normalzed cuts algorthm (Ncut) proposed n [8]. Accordng to [8], f the colors n one mage are well clustered, the affnty matrx W, whch wll be defned n the next secton, has a block structure. Correspondngly, the normalzed graph Laplacan D 1 (D W) has pecewse constant egenvectors, where D s the degree matrx of W. The proposal n [8] s to apply the K-means algorthm to these egenvectors to partton orgnal network. For general affnty matrces, the egenvectors wll be approxmately pecewse constant, and smlar procedures can also be appled. Notce that the key step n Ncut s to get the leadng egenvectors of the generalzed egenvalue problem (D W)x = λdx. A new method the localzed subspace teraton (LSI) LMAM and School of Mathematcal Scences, Pekng Unversty, Bejng 1871, Chna (wujnlong1984@math.pku.edu.cn, tel@pku.edu.cn). 1

2 s developed to solve the above problem wth lnear scalng computatonal effort provded that the matrx s sparse and the requred egen-subspace has localzed bass representaton [2]. LSI s a localzed verson of the standard subspace teraton algorthm (SI). Instead of obtanng the egenvectors as n standard subspace teraton (SI), LSI s desgned for obtanng a set of approxmate, but localzed bass of the egen-subspace spanned by the egenvectors correspondng to the leadng egenvalues. The man dea s to replace the orthogonalzaton step n SI by a localzaton step, followed by a truncaton of the bass vectors,.e. outsde of a support set whch s dfferent for dfferent bass vectors, the components of the vector are set to be zero. Ths has the effect of reducng the computatonal complexty to O(N). For the mage segmentaton problem, the localzed bass representaton of the leadng egen-subspace s approxmately the set of ndcator vectors for the dfferent clusters. At a frst sght, the logc may seem crcular. In order to localze the bass vectors, we have to know frst the support sets, whch are closely related to the ndvdual segments that we are lookng for. However, the support sets are n general larger than the segments and can be chosen adaptvely as the algorthm proceeds. Ths degree of freedom can be used to mprove an exstng segmentaton. Therefore, our strategy s to start the algorthm usng a crude segmentaton procedure such as the K-means algorthm, and then teratvely mprove the result usng LSI. The man steps of the algorthm are as follows: Algorthm descrpton: Step 1: Fnd a coarse pre-partton of the mage wth smple clusterng algorthm, for example, K-means algorthm. Step 2: Set up the overlappng sets among dfferent parttons, whch are not sure to be classfed, but wll be reparttoned to ts correct class later. Step 3: Perform LSI step to the partton vectors to get the localzed bass. Step 4: Repartton the mage pxels accordng to the results obtaned by LSI. Our method has the vrtue that t s an O(N) algorthm. Thus t can be appled to large pctures. The rest of the paper s organzed as follows. In secton 2 we brefly ntroduce LSI and Ncut algorthm. Then we apply LSI to a toy example wth well-separated structure to show the essence of the algorthm n Secton 3. Our new algorthm segmentaton based on LSI (SLSI) s formulated n secton 4 followed by some numercal results for real mages. Fnally we dscuss some numercal ssues and future works n the last secton. 2 The normalzed cuts and LSI 2.1 The Normalzed Cuts Let W j be the smlarty measure between the th and j th pxel n an mage (for example the defnton (4) n Secton 3) and all the W j form an N N affnty matrx W (or smlarty matrx), where N s the number of the pxels. For smplcty, we suppose W j = W j for all 2

3 , j = 1, 2,...,N and so W s symmetrc. In [8] Sh and Malk show the optmal bpartton of the mage n some sense s equvalent to mnmze the so-called normalzed cuts: NCut(A, B) = cut(a, B) cut(a, B) + asso(a, V ) asso(b, V ), (1) where cut(a, B) = A,j B W j, asso(a, V ) = A,j V W j, V s the set of all pxels and A, B are the subsets of V. Denote the degree matrx of W to be the dagonal matrx D wth D = N j=1 W j. Then the mnmzaton of (1) can be rewrtten equvalently as mn y y T (D W)y y T Dy = 1 max y y T Wy y T Dy, (2) wth the constrants that y {1, b} and y T D1 = (see [8] for detals). As dscussed n [8], the second constrant y T D1 = can be satsfed automatcally. However, the frst constrant wll make the optmzaton (2) NP-complete. Hence they smply gnore ths constrant. As a result the soluton to approxmate (2) after relaxaton s the generalzed egenvector correspondng to the second largest generalzed egenvalue of the system W x = λdx, snce the frst largest egenvalue s always 1 and the related egenvector s a constant vector. Ths type of problem s closely related to the extensvely studed graph Laplacan [1]. Sh and Malk show that the second egenvector could be used to bpartton the mage. Although ths egenvector does not guarantee to be the optmal soluton to (2) because of gnorng the frst constrant, t usually gves good parttons for many real-world examples. Usually the egenvectors related to the several largest egenvalues have approxmately pecewse constant property and can be used to partton the mage smultaneously wth the K-means algorthm, whch s called Modfed NCut algorthm n [9]. 2.2 The Localzed Subspace Iteraton Algorthm (LSI) Subspace teraton (SI) s a typcal technque for computng leadng egenvalues and egenvectors of a matrx. It ncludes two steps: the matrx-vector multplcaton step and the orthogonalzaton step [3]. The localzed subspace teraton (LSI) was proposed n [2] to replace the orthogonalzaton step by a localzaton step and thus t decreases the computatonal complexty from O(M 2 N) to O(N), where N s the dmenson of the matrx and M s the number of the leadng egenvectors whch one tres to obtan. Besdes the reducton of computatons, the other vrtue of LSI s that t produces a set of sparse bass of the leadng egen-subspace. In many problems sparsty s pvotal for reducng the computatons. The key dea of the localzed subspace teraton s that the subspace s exactly what we want, whch let us free to choose the bass representaton. For the mage segmentaton problem, the output of the LSI s a set of localzed vectors whch are used to partton the mage fnally. The mplementaton of LSI are descrbed n the followng Algorthm 1. For a more detaled nformaton of LSI, the readers may be referred to [2]. 3

4 Algorthm 1 (Localzed Subspace Iteraton). Gven the supports S of the bass functons and the weght functons w for each n {1, 2,...,M}. We fnd the bass functons Q by teraton tll convergence: Purfcaton: Compute Q n+1 = p(a)q (n) ; n+1 Localzaton: Denote by V = span{ Q whose support overlaps wth that of Qn+1. Fnd varatonal problem Q n+1 } the lnear space spanned by n V by solvng the ˆQ n+1 mn ˆQ n+1 n+1 V, ˆQ 2 =1 ˆQ n+1 2,w. (3) Here 2,w s the weghted l 2 -norm wth weght functon w centered on S. Truncaton: Q n+1 s the truncaton of ˆQ n+1 on S. Here Q s the localzed subspace bass of matrx A to be found. p(a) s some polynomal of A to accelerate the convergence. A good choce s the Chebyshev polynomal [5]. Note that n LSI, the supports S must be known a pror. 3 A toy example for mage segmentaton In ths secton we consder an example for strongly separated clusters to show the essence of our basc partton strategy. We wll frst present the relaton between the egenvectors and the parttons to show the applcablty of the spectral type methods. Then we consder the LSI appled to the computaton of the localzed bass. Fnally we gve the numercal result. The synthetc data set s composed of sx types of data ponts n R 2. Each type are sampled from a normal dstrbuton wth the same covarance matrx K = dag(1 4, 1 4 ) but wth dfferent means. The numbers of data ponts n each type are 25, 2, 15, 22, 12, and 28 respectvely. The left panel n Fgure 1 dsplays the whole data set. As n [8] and [1], we assocate these data wth the N N affnty matrx W as W j = e x x j 2 2 /2σ2, (4) where N s the sze of the whole data set and x l s the spatal poston of the l th pont (l = 1, 2,...,N). Here N = 122 and σ =.5. We suppose that the data ponts are ordered accordng to ther types, and the number of the ponts n each type s assumed to be known a pror. Agan, ths s an deal smplfcaton for the toy example. For the real mages, t should be found automatcally from the data (see the next secton). Here we show the ten largest generalzed egenvalues and the egenvectors related to the nne largest egenvalues n Fgures 1 and 2. As expected, the frst egenvalue s 1 and the correspondng egenvector s constant. Several egenvectors followed are approxmately pecewse constant. As Sh and Malk noted, the data set can be parttoned by these leadng egenvectors wth K-means algorthm. From the computatons shown above, a set of localzed bass of the leadng generalzed egenspace of Wx = λdx can be found by lnear combnaton of the egenvectors. Ths wll gve a 4

5 .4 The orgnal data set 1.2 The ten largest generalzed egenvalues of system Wx=λ Dx y x Fgure 1: The data set shown n the left panel s composed of sx types of ponts whose numbers are 25, 2, 15, 22, 12, and 28 respectvely. Dfferent types of data ponts are llustrated wth dfferent colors. All data ponts are sampled from normal dstrbutons wth the same covarance matrx K = dag(1 4, 1 4 ) but wth dfferent means. The rght panel shows the ten largest generalzed egenvalues of the system Wx = λdx. The other egenvalues are all zeros. set of approxmate ndcator vectors whch do not vansh only n one cluster. Ths s the base for applyng LSI. Let us defne the normalzed affnty matrx V = D 1/2 WD 1/2, (5) whch s also symmetrc. If x s a generalzed egenvector of the system Wx = λdx correspondng to the egenvalue λ, namely Wx = λ Dx, then y = D 1/2 x s an egenvector of V related to the same egenvalue. Ths means all the egenvectors of V can be obtaned from the generalzed egenvectors multpled by D 1/2. The exstence of the localzed bass for W ensures the smlar structure for V. Thus LSI s appled to V nstead of the generalzed egen-system n ths paper. Before runnng LSI, the followng three problems should be solved. (1) How to choose the partton number M? (2) How to choose the supports S? (3) How to defne the weght functons w? Let us answer these problems one by one for ths toy model, whch s much easer than for the real mages n the later sectons. Evdently M should be 6 for ths example. For the choce of supports, t s clear that the th support S must nclude all the data ponts n the th type. However, the supports should overlap to ensure the convergence of the LSI. For smplcty, we take the overlappng sze between the neghborng types to be 8. That s, we mark S j = 1 f the j th data pont belongs to the th support, and S j = otherwse 1. Then, the th support S s { 1, j {l + 1,...,r }, S j = (6), otherwse, 1 In ths paper we denote the th support set and the th ndcator vector produced by the th support set wth the same token S snce they are equvalent. Ths knd of notaton s also used for the weght w. 5

6 x 1 3 the 1 th egenvector 2 x 1 3 the 2 th egenvector 5 x 1 3 the 3 th egenvector x 1 3 the 4 th egenvector x 1 3 the 5 th egenvector x 1 3 the 6 th egenvector the 7 th egenvector.1 the 8 th egenvector.1 the 9 th egenvector Fgure 2: The leadng nne generalzed egenvectors of the system Wx = λdx for the data set s shown. The components of these egenvectors are sorted accordng to the second egenvector n descendng order. The pecewse constant property for the frst sx egenvectors can be observed clearly. where 1 l = max( N j 8, ), j=1 r = mn( N j + 8, N) and N j s the number of the j th type of the data ponts (j = 1,...,6). For the th weght w, we take w j = f the j th pont belongs to the th type, and make the weght ncrease by (x/1) 2 once the pont gets out of type. Ths quadratc penalzaton performs farly well n our examples. The ntal value Q can be sampled randomly or smply let Q be S, where S = (S 1, S 2,...,S M ) s the N M support matrx. Accordng to our experments, the result of LSI s not senstve to the choce of Q. Now we take Q = S here. LSI converges after seven teratons wth the prevous parameters. If we wrte the fnal Q 7 as ˆQ, then we can check the precson of ths result usng the error Error M ( ˆQ) ( := Tr ( ˆQ T ) M 1 ˆQ) ˆQT V ˆQ λ (7) j=1 =1 and the relatve error RError M ( ˆQ) := Error M( ˆQ) M =1 λ, (8) where λ, = 1,...,M stands for the M largest egenvalues of V. For ths synthetc data set, Error M ( ˆQ) =.18 and RError M ( ˆQ) =.34% n MATLAB. Ths precson s acceptable for most of real-world problems. The fnal localzed bass of W can be obtaned from the results of LSI multpled by D 1/2. They are llustrated n Fgure 3. 6

7 2 the 1 th vector after localzed 5 the 2 th vector after localzed the 3 th vector after localzed ndex of data set 2 the 4 th vector after localzed ndex of data set 4 the 5 th vector after localzed ndex of data set 2 the 6 th vector after localzed ndex of data set ndex of data set ndex of data set Fgure 3: The sx localzed vectors of the leadng generalzed egen-space of system W x = λdx are shown. They are obtaned from the results of LSI multpled by D 1/2. As expected, the localzed vectors are approxmate ndcator vectors multpled by a constant. These vectors can be used to partton the orgnal data set. Therefore, LSI s applcable to well-separated data sets. But how to generalze t to real mages? 4 LSI for real mages From the prevous secton, LSI s successful for well-separated data sets. It s almost mpossble that the real-world data set s so well-separated. Moreover, the supports for the toy example are chosen from the well-sorted data ponts. The partton number and the boundary between the neghborng parttons are all assumed known, whch are too deal for most cases. For real mages, all of these nformaton s unknown. Hence our frst job s to dg them out. Before dong so, let us defne some new notatons. We denote the new affnty matrx W as W j = e z z j 2 x x j σc e σ d, f x x j 2 < r, otherwse for real mages as n [8], where N s the number of pxels n the mage, x s the spatal poston of the th pxel, and z s a vector related to ntensty or color nformaton at that pxel. σ c, σ d s the scale parameters and r s the truncaton radus. In the whole paper we obtan z as n [8]. For example, we take vector z = [v, v s sn(h ), v s cos(h )] (1) for color mages as n [8], where h, s, v are the HSV expresson of the color of pxel ( = 1, 2,...,N). Ths truncaton strategy s very mportant for handlng large pctures because otherwse W s full, whch s formdable to be manpulated. Denote the th kernel support to be composed of all pxels whch are n support S but not n (9) 7

8 any other support S j (j ). That s, Ker(S ) = S \ j S j. (11) The th overlappng set s defned as Ovlap = S \Ker(S ), (12) where = 1,...,M. In our numercal experments the results of LSI are very robust to the ntal value Q as n the prevous example. So we take Q = S for the rest of the paper as before. The alternatve s to choose Q randomly. Moreover, we gnore the dfference of Ovlap and S c (the complementary set of S ) and smply let the th weght w be w j = {, f pxel j Ker(S ), 1, otherwse. (13) In the subsequent subsectons, a new segmentaton algorthm based on LSI and the Modfed NCut wll be proposed, n whch the problems stated before wll be solved n turn. 4.1 Segmentaton based on LSI (SLSI) Fndng M automatcally and pre-parttonng mages Once M and supports are acqured, the other parameters are easy to be obtaned accordng to our prevous dscussons. But unfortunately, unlke the above example, no a pror nformaton s avalable for the supports of dfferent parttons n the real mages. Hence some coarse pre-partton strategy should be appled at frst and then t s used to be refned later. Typcally K-means s a good pre-partton method. The other vrtue of K-means s that t can be used to fnd the number of parttons M automatcally as noted n [11]. Denote y = [h, s, v ] the vector of pxel n the HSV color representaton. The vector y s used to fnd M nstead of z n (1). The varance of one partton for an mage can be wrtten as Var(K) = 1 N K =1 j Seg y j c 2 2, (14) where c s the th center from K-means, and Seg represents the th part of the segments. K s the number of the segments and N s the number of the mage pxels. It s obvous that a better partton wll produce a smaller varance. The left panel n Fgure 4 presents dfferent values Var(K) of a baby mage from Sh s homepage when K takes dfferent values. A good K can be justfed when a satsfactory knk pont occurs. Typcally a knk pont s judged based on the rato of the slope of the fold lne for Var(K) versus K. For clarty, we denote a varable ISlope(K) as Var(K) Var(K + 1) ISlope(K) = (15) Var(K 1) Var(K) 8

9 .35 Varance vesus k 3 nverse of slope vesus k.3 Var 2.5 ISlope γ Varance nverse of slope number of cluster k number of cluster k Fgure 4: The left panel shows dfferent values Var(K) versus the number of segments K for a baby mage from Sh s homepage. The values of ISlope are llustrated on the rght panel. The horzontal magenta lne shows the threshold γ =.5 for knk detecton. to judge the goodness of a knk. As an example, dfferent values of ISlope(K) are shown n the rght panel of Fgure 4 based on Var(K). For a gven postve parameter γ, we take the smallest K whch satsfes ISlope(K) < γ as the number of segments. In our examples, γ =.5 s usually a good choce. For ths baby mage, K wll be 2 f γ =.5, whch concdes wth our ntuton. After M s fxed, K-means algorthm s appled to vectors z to obtan a coarse pre-partton. Then we can take the th part of the pre-partton as the th support S. However, S needs to be extended n order that they have overlaps wth some other S k (k ). Intutvely t s reasonable to take the overlaps to be the boundares of the segments whch are easer to be wrongly parttoned. Next we show how to get the overlappng regon n more detal Modfyng centers and obtanng supports As well known, the fnal results of K-means strongly rely on the ntal chosen centers. Usually the bad choce of ntal centers wll brng bad fnal results ncludng fnal centers. And the resultng centers obtaned by K-means are not the real pxels of the orgnal mage. Here we choose a real pxel n the th segment whch s closest 2 to the th fnal center of K-means and take t as the new center. We also call these new centers the fnal centers of K-means. Snce the fnal overlappng sets of LSI are selected based on the fnal centers of K-means, thus the bad choce of the fnal centers wll produce unreasonable supports, whch can not even ensure the convergence of LSI. Hence the centers are crucal for LSI and thus for our new algorthm. On the other hand, the fnal centers from K-means can not always represent dfferent features of the mage. Some of them even have smlar color and texture. Take mage acaleph as an example. The fnal centers of K-means are marked by the sold red crcles n the mddle and rght panels n Fgure 7. The deal centers for ths mage should be two pxels whch mrror the yellow acalephs and the blue background respectvely. Obvously the two centers c 1 and c 2 are not 2 closest means the smallest dstance between the vectors z. 9

10 good representatves. So the modfcaton of the centers s also necessary so as to obtan better representatve centers before choosng supports. Now we state our modfyng centers strategy. Frst we choose one fnal center as the reference center whch wll yeld no modfcaton. Wthout loss of generalty, we assume that the frst center c 1 s the reference center. Suppose c 1,...,c 1 have been modfed so far, now we hope to fnd a new c. If the old c s not too close to all c 1,...,c 1, that s, all the dstances between c and c 1,...,c 1 are larger than a value θ c, whch s set by the user, then c does not need to be changed. Otherwse c should be modfed as follows. Denote F the ndex set of all pxels n Seg whch are not too close to all c 1,...,c 1, namely, F = { k Seg : } mn z k c j 2 2> θc 2, (16) j {1,..., 1} where z s defned n (1) and θ c s a user-defned parameter. After obtanng F, we average y n F and then fnd a closest pxel n F to ths average. Ths pxel s chosen as the new center c. The modfcaton procedure contnues untl all k centers are obtaned. If the procedure can not be contnued n one step because of empty F, then we choose another center of K-means as the reference center and restarts. If no reference centers can be chosen to fnsh the procedure, then we decrease parameter θ c and restart the procedure. We call the whole prevous procedure one loop. Once one loop s ended, a set of new centers wll be avalable. But f the resultng centers of one loop s not satsfactory, we can restart the procedure agan based on the centers of the prevous loop. Actually we do not stop the loops untl one loop can not gve any change for the centers from the prevous loop n our examples. Based on the above modfyng procedure, the orgnal centers of K-means shown wth red sold crcles n the frst row of Fgure 7 are modfed and the new centers are dsplayed wth whte sold crcles. They are better representatves of dfferent features n mage acaleph. Wth these modfyng centers, we choose the supports for dfferent features of the mage. As before, we frst take S as the th part of the segments obtaned by K-means ntally. Then S needs to be extended so that they overlap. Intutvely the boundares between the segments may be good canddates as the overlappng regons. After obtanng the new center c, we can compute the maxmal dstance Max between c and all the other pxels n Seg. Then we take the pxels n Seg to be n the boundary f the dstance between c and these pxels s comparable wth Max. Mathematcally we defne the boundary of Seg as B = { j Seg : z j c 2 2> (1 θ ovlap )Max 2 }, (17) where parameter θ ovlap s between and 1. It determnes the sze of B. Now the overlappng regon Ovlap s smply taken to be Ovlap = M B j (18) for = 1, 2,...,M. Incorporatng Seg and Ovlap, we have the fnal support j=1 S = Seg Ovlap. (19) 1

11 After all the supports are acqured, we perform LSI. Applyng K-means agan to the resultng localzed vectors, we can get the fnal partton of mages. Now we summarze the prevous steps n the followng algorthm. Algorthm 2 ( SLSI ). Gven an mage and the parameters θ ovlap, σ c, σ d and r. 4.2 Examples Pre-partton: If gven the number of segments M, apply K-means to the vectors z defned n (1) to obtan a coarse pre-partton; Or fnd M and the pre-partton by usng K-means for a gven parameter γ. Modfcaton of centers: Modfy the centers of K-means as stated n secton so that the new centers are more representatve for the dfferent features. Supports: Use the modfed centers and (17) (19) to compute all the supports S (=1,...,M). LSI: Set up weghts accordng to (13) and let the ntal value Q be S = [S 1, S 2,...,S M ] or chosen n random. Form the affnty matrx W based on σ c, σ d, r and (9). Perform LSI. Partton: Apply K-means to the localzed vectors of LSI to partton the mage fnally. Example 1 (a baby mage). Ths baby mage s from J. Sh s homepage. The number of segments M s chosen automatcally accordng to K-means. The results are shown n Fgure 4. Hence M should be 2. K-means are appled to the vectors z whch are defned n (1) so as to obtan the pre-partton. Then more sutable centers are generated based on the procedures n Secton The new centers are marked wth the whte crcles n the mddle and rght panels at the frst row of Fgure 5. The two boundares of the segments B s are llustrated at row three. The pxels n the boundares are shown wth whte and others wth black whle keepng the orgnal postons. From these two plots, the boundares are reasonablely rough snce they all belong to the edges of the baby. Thus the whole overlappng set S s the unon of B 1 and B 2 ( = 1, 2). The overlappng set s small when θ ovlap =.1. The weghts are obtaned easly accordng to (13), and the ntal value Q s taken to be the supports S. After obtanng the localzed vectors of LSI whch are dsplayed at row four of Fgure 5, we use K-means to reclassfy them to acheve the fnal partton of ths mage. The results are shown at the second row of Fgure 5. Evdently the fnal partton from the localzed vectors are better than the ntal partton usng K-means. Ths example confrms effectveness of localzed vectors for partton even wth very small overlappng sets. It s nstructve to compare the pre-partton and the fnal partton of the mage. In the prepartton step, only the color nformaton s consdered. The par of eyes are msclassfed nto the same segment wth the background because the eyes has smlar color as that of the background. In the LSI step, because the spatal nformaton s added n the normalzed affnty matrx V and the pxels of eyes are selected n the overlappng regon, the fnal localzed vectors correctly classfy 11

12 the eyes back nto the same partton as the man body of the baby. Example 2 (a cactus mage). The orgnal cactus mage shown n the top leftmost panel of Fgure 6 s suppled by Dr. Le L. All of the parameters and steps are the same as the prevous example. The frst center s good enough so no modfcaton s necessary. It s reasonable snce ths center s n the branch of the cactus and thus t represents the cactus feature. Intutvely the deal partton s to separate the cactus from the background. From the frst row of Fgure 6 we see the pre-partton does not gve a good partton. The whtest part of the background splts off from the other background. SLSI overcomes ths problem wth a small overlappng set. It pulls back the whtest part to the background segments. The fnal partton shown at the second row of Fgure 6 s perfect. 5 Dscussons SLSI s essentally a combned verson of MNcut [9] and LSI [2] for mage segmentaton. Theoretcally, the classfed result of SLSI would not be better than MNcut. But SLSI has the vrtue of havng the computatonal complexty O(N), where N s the number of pxels n one mage. And the supports of the localzed vectors show dfferent parttons naturally. The fnal effect of LSI s to classfy the boundary ponts among dfferent features characterzed by the center pxels, whch s the key for a successful classfcaton. For the computatonal complexty, as well known, the K-means s O(MN) where M s the number of segments. Snce M s usually small n one real mage, t s reasonable to thnk that K-means s a lnear-scalng algorthm for our present segmentaton problems. Hence the prepartton step n SLSI can be fnshed n O(N) cost. As noted n secton 4.1.2, modfyng centers and computng supports can also be done wth O(N) operatons. The affnty matrx W and the normalzed matrx V are usually very sparse from ther constructon. Thus one matrxvector computaton typcally takes O(N) operatons. Based on the dscusson n [2], LSI has the complexty of O(N). Snce the fnal localzed vectors of LSI are only used for partton, the precson requred for these vectors s rather low. As a result, LSI converges very quckly. To sum up, the complexty of the new algorthm SLSI s O(N), whch can be appled to large pctures. Now let us consder some ssues of SLSI whch should be handled n the future. Take Fgure 7 as a typcal example. The perfect partton for the acaleph mage s to separate all the acalephs from the background. But the fnal partton of SLSI msclassfes some part of the background near the boundary as the K-means pre-partton does, though SLSI has a bt mprovement on dggng the acalephs out more completely. From the thrd row of Fgure 7, the overlappng sets nclude almost all msclassfed pxels and pxels near boundary surface. Hence they should be very approprate snce we want to pull out the background part near the boundary to the other segment. However, SLSI fals. That s probably because we do not dstngush pxels n overlappng sets and the outsde, whose penalzed weghts are assumed to take the same values such as n (13). Maybe t s the man reason why the result n Fgure 7 s not good. So far t s not a trval problem to determne the weghts more adaptvely. Another ssue s that the fnal partton depends on the choce of the vector representaton of the pxels used to obtan the pre-partton and the supports. Emprcally dfferent choces wll 12

13 The orgnal mage the 1 th preclassfed seg the 2 th preclassfed seg The orgnal mage The 1 th, LSI, K Means The 2 th, LSI, K Means the 1 th overlap the 2 th overlap The 1 th localzed vector The 2 th localzed vector Fgure 5: A baby mage from Sh s homepage. The frst column of the top two rows dsplays the orgnal mage. The frst row shows the pre-classfed results usng K-means. The red sold crcles show the postons of the fnal centers from K-means. The whte ones show the correspondng modfed centers. The results of partton usng localzed vectors of LSI and K- means are shown n the second row. The correspondng overlappng sets are gven on row three. The left and rght panels show the overlappng sets n the frst and second segments. The pxels n the overlappng sets are shown wth whte color. Both of the overlappng sets are very small snce parameter θ ovlap takes a small value. Row four shows the two resultng localzed vectors of LSI. They are rescaled such that the components wth maxmal absolute values take value 255, whch are related to whte pxels n the panels. The parameters σ c =.4, σ d = 5, r = 1, θ ovlap =.1. 13

14 The orgnal mage the 1 th preclassfed seg the 2 th preclassfed seg The orgnal mage The 1 th, LSI, K Means The 2 th, LSI, K Means the 1 th overlap 2 the 2 th overlap The 1 th localzed vector The 2 th localzed vector Fgure 6: A cactus mage. The results of the pre-partton and the fnal partton are shown on the top two rows. Note that the old and new centers for segment 1 overlaps. Overlappng sets n the th ( = 1, 2) segments are presented on row three. The two localzed vectors of LSI are llustrated on row four. The parameters σ c =.4, σ d = 5, r = 1, θ ovlap =.1. 14

15 produce dfferent parttons. For example, the parameters for the prevous mages are acqured usng vectors z. But f we choose y nstead of z, the fnal partton s worse for the cactus mage but better for the acaleph mage (The comparson s made wth the best performance). But ths ssue exsts for any segmentaton algorthm. A further comment on SLSI s about the pre-partton. If we apply K-Medods through smlartes between pxels of mages to obtan the pre-parttons, the result of SLSI were usually better than those produced by K-means n our experments. However, K-Medods typcally costs more than O(N), whch s dffcult to be appled to large pctures. That was the reason why we choose K-means nstead of K-Medods n the fnal algorthm. In summary, though SLSI s successful for many cases we consdered, t s not mature enough to deal wth all knds of mages at present. More researches are necessary. But the lnear-scalng cost makes SLSI a promsng segmentaton algorthm. 6 Acknowledge We thank Dr. Le L for the supply of the mages and many helpful dscussons. T. L acknowledges the support by the Chna Natonal Basc Research Program under the grant 25CB References [1] F.R.K. Chung, Spectral graph theory, Amercan Mathematcal Socety, Provdence, Rode Island, [2] W. E, T. L and J. Lu, Localzed bass of egen-subspaces and operator compresson, submtted to Proc. Nat. Acad. Sc., 27. [3] G.H. Golub and C.F. Van Loan, Matrx computatons (3rd edton), The Johns Hopkns Unversty Press, Baltmore and London, USA, [4] P. Perona and W. T. Freeman. A factorzaton approach to groupng. In H. Burkardt and B. Neumann, edtors, Proc ECCV, , [5] Y. Saad, Numercal methods for large egenvalue problems, Manchester Unversty Press, Manchester, UK, [6] L.K. Saul, K.Q. Wenberger, F. Sha, J. Ham and D.D. Lee, Spectral methods for dmensonalty reducton, Semsupervsed learnng, MIT Press, Massachusetts, USA, 26. [7] G.L. Scott and H.C. Longuet-Hggns, Feature groupng by relocalsaton of egenvectors of the proxmty matrx. In Proc. Brtsh Machne Vson Conference, 13-18, 199. [8] J. Sh and J. Malk, Normalzed cuts and mage segmentaton, IEEE Trans. Pattern Anal. Mach. Intel. 22 (2), [9] M. Mela and J. Sh, A random walks vew of spectral segmentaton, AI and STATISTICS (AISTATS),

16 The orgnal mage the 1 th preclassfed seg the 2 th preclassfed seg The orgnal mage The 1 th, LSI, K Means The 2 th, LSI, K Means the 1 th overlap the 2 th overlap The 1 th localzed vector The 2 th localzed vector Fgure 7: A acaleph mage. The results of the pre-partton and the fnal partton are shown on the top two rows. Overlap sets n the th ( = 1, 2) segments are presented on row three. The two localzed vectors of LSI s llustrated on row four. The parameters σ c =.1, σ d = 2, r = 1, θ ovlap =.1. 16

17 [1] Y. Wess, Segmentaton usng egenvectors: a unfyng vew, Internatonal Conference on Computer Vson, [11] T. Haste, R. Tbshran and J. Fredman, The elements of statstcal learnng, Sprnger-Verlag, New York, Berln and Hedelberg,