Small Network Segmentation with Template Guidance

Transcription

1 Small Network Segmentaton wth Template Gudance Krstn Dane Lu Department of Mathematcs Unversty of Calforna, Davs Davs, CA Ian Davdson Department of Computer Scence Unversty of Calforna, Davs Davs, CA ABSTRACT The mportance of segmentng socal networks has grown along wth the usage of socal network stes such as Facebook. Socal networks provde a wealth of nformaton about people ncludng frendshps and nterests. Informaton can be added to segmentng algorthms such as spectral clusterng as n the form of a sngle constrant. Takng advantage of multple nstances of nformaton, such as a template composton, for clusterng s not trval snce addng multple constrants to spectral clusterng technques s dffcult. We present an approach to ncorporate template constrants to the sem-defnte program formulaton of spectral clusterng. We descrbe dfferent types of template constrants and ther nterpretaton. Also, we present a randomze roundng scheme to ensure the rounded soluton to the SDP formulaton satsfes the template segmentaton. Keywords Spectral Clusterng, Semdefnte Programmng, Randomzed Roundng, Constraned Clusterng 1. INTRODUCTION Socal networks have grown n popularty n the last decade where webstes such as Facebook, Myspace, and Twtter have over 100 mllon users each. Wth growng use, t s mportant to have useful segmentatons of these networks. Segmentatons based just on graph topology (edge structure) have ther uses but n ths paper we shall look at segmentatons of ego networks wth template constrants that allow doman experts to encode segment expectatons as a template structure. Ths work s well suted to socal networks so as to allow users to descrbe themselves along lnes of nterests, locaton, and work afflaton, for example. Extendng segmentaton of socal networks to factor n these descrptons allows for a varety of drectons ncludng creatng segments of people who are frends and also share (or do not share) smlar nterests. Permsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. To copy otherwse, to republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. MLG 11 San Dego, CA, USA Copyrght 2011 ACM $10.00 A key problem s then how to represent ths addtonal node nformaton, or background nformaton, n a meanngful way so that exstng clusterng algorthms can take advantage of t. Spectral clusterng s an mportant technque used to fnd segmentatons of a sngle graph [19][5][23]. The quadratc programmng formulaton (whch spectral clusterng s an example of) does not lend tself to easly addng more constrants on ths extra nformaton [26]. In ths earler work, we could only add one preference for the segmentaton. Sem-defnte programmng (SDP) s an alternatve convex programmng formulaton [27][3] (whose objectve functon s lnear, not quadratc) that easly allows the addton of many constrants. Usng the SDP formulaton, we can segment small networks wth templates. Templates are naturally modeled as postve sem-defnte matrces (kernels) that can be added as lnear constrants n the SDP formulaton. As a constrant, we can dctate whether the segmentaton wll have strct or loose adherence to the template composton. For example, we can conduct pragmatc operatons such as, Segment a network so we have n the same clusterng: all my close frends who have smlar musc tastes for strct adherence to the constrant or, Segment a network so we have n the same clusterng: some of my close frends who have smlar musc tastes for loose adherence to the constrant. More complex operatons could nvolve segmentng a network so that people n the same cluster have smlar musc tastes but dd not go to school together. In ths way, we are placng a compostonal restrcton on who can be n the same cluster together. Consder the example shown n Fgure (1). The left graph wth black nodes shows the segmentaton (ndcated by the dotted lne) from normalzed spectral clusterng [19]. Spectral clusterng s dscussed n detal n Secton 3.1. The rght graph shows the clusterng result from template clusterng. The red square nodes have dfferent labels from the black crcle nodes. The template clusterng formulaton yelds a segmentaton whch consders both the normalzed clusterng segmentaton and the gudance from the template constrants. Templates can take any number of forms whch nclude: 1. Topologcal Structures - preserve x% of a group afflaton such as frendshps or co-workers. 2. Characterstcs of People - preserve x% of people who

2 We present a novel approach usng a SDP formulaton for normalzed spectral clusterng to segment a socal network wth template compostons. We present lnear constrants that represent desred template compostons for segmentaton. We can place lmts on how strctly or loosely the templates are adhered to. Fgure 1: Example of normalzed spectral clusterng (left) and spectral clusterng wth a template constrant (rght). Normalzed spectral clusterng seeks a balanced cut of the graph. The template constrant seeks a clusterng that also ncludes known nodes wth the same label (shown as red squares). share a certan characterstc such as people who enjoy rock musc. 3. Alternatve Segmentatons - preserve x% of a known segmentaton of the graph [25][7]. 4. Multple Dstance Metrcs or Vews - Segment a network based on multple metrcs and vews An mportant contrbuton of our work s the ablty to add multple templates to the one clusterng. Examples of the dfferent types of template compostons and ther constrant forms are outlned n Secton 4. We base our work on the SDP formulaton of normalzed spectral clusterng smlar to Xng and Jordan [27] to segment a socal network wth templates based on background knowledge. Our SDP formulaton s shown n Secton 4. The SDP s easly mplementable n MATLAB by usng the CVX package developed by Grant and Boyd [11] for convex optmzaton. The SDP formulaton guarantees an optmal soluton to the relaxed objectve functon and template composton constrants can be added to the optmzaton problem as addtonal constrants. Another advantage of the SDP formulaton s that we are not lmted n the number of constrants that are used so multple sources of background knowledge can be used for segmentaton resultng n a desred template for clusterng. However, usng templates n an SDP formulaton has a sgnfcant challenge. The template constrants can be wrtten so that a certan quantty of the structure must be preserved, such as 80% of people wthn each segment must be from the same school. Whle the soluton from the SDP formulaton wll satsfy ths constrant, due to the relaxed formulaton of the membershp assgnments (whch allows fractonal values) the soluton obtaned wth roundng up and down membershp assgnments to ntegers may not satsfy the template. We present a randomzed roundng scheme n Secton 5 to ensure the template constrants are satsfed. For example, let there be a network wth four people: C = {w, x, y, z} and a desred template composton s to have nodes w, x, y together. A possble segmentaton s c = [0.7, 0.6, 0.08, 0.3] but roundng the vector c may result n ĉ = [1, 1, 1, 1] and the template constrant s no longer satsfed. Our contrbutons are: We present a randomzed roundng scheme to ensure the rounded partton soluton stll satsfes the template segmentaton. Our expermental results show the usefulness of our work. It should be noted that a well known lmtaton of SDP formulatons s handlng large problems. In future work we shall extend our work by consderng low rank SDP solvers [16][3]. 2. PREVIOUS WORK Prevous work that s closely related to our own ncludes modfyng graph Laplacans wth gudance from a pror knowledge by Tollver et al. [22] and addng must lnk and cannot lnk lnear constrants to an SDP for graph segmentaton by Heler et al. [12]. Tollver et al. performed mage segmentaton by modfyng the mage s normalzed laplacan matrx by ncorporatng external nformaton of the mage n order to perform the segmentaton. A pror knowledge s generally pxel-level nformaton of the mage such as the desred structure of the segmentaton. The Felder space s teratvely modfed by the vector representng the a pror knowledge untl the vectors are algned. Whle we also utlze a pror knowledge n the forms of labels and alternatve cuts of the graph, our work dffers n that we do not modfy the graph Laplacan. Heler et al. also ncorporates a pror knowledge, specfcally about class membershp nformaton, by addng lnear constrants to the SDP formulaton of spectral clusterng. The lnear constrants for nodes and j are of the form: trace ( (e e T j + e je T )X ) = 2, where e = (0,..., 0, 1, 0,..., 0) s the elementary vector wth 1 n the th poston. The constrant encodes a must-lnk constrant between nodes and j and can be changed to a cannot-lnk constrant by settng the constrant to 2 nstead. Moreover, multple ( constrants can be combned ) as a sngle constrant: trace (e e T j + e je T )X = 2 P k where P k s the set (,j) P k of nodes that are n the same class. Whle the constrants we propose are smlar n nature, the formulaton of the constrants allows for a more flexble nterpretaton of the types of membershps (known structures, class membershps, and alternatve segmentatons). Addtonally, there s more flexblty to the adherence to the a pror nformaton. For example, nodes and j that are known to have the same class membershp are not necessarly requred to be n the same segmentaton. The Heler et al. constrant forces the cut to put nodes and j n the same or dfferent clusters. Another method that seeks to ncorporate a pror knowledge s segmentatons wth multple graphs. A straghtfor-

3 ward approach to clusterng wth multple vews or representatons of a graph s to lnearly combne kernels [13][31] or convexly combne graph Laplacans [20][1] and apply exstng clusterng technques. Tang et al. proposed Lnked Matrx Factorzaton (LMF) for clusterng wth multple vews [21]. LMF seeks a common factor matrx P that s the low dmensonal embeddng of enttes characterzed by the multple graphs and matrx Λ that captures the characterstcs of each graph. None of the methods are able to utlze the extra nformaton provded by the multple representatons. Instead, they seek to average the multple vews so standard clusterng technques can be used. For template segmentaton, we want to use the extra nformaton to fnd nodes that satsfy a template. Thus, prevous work n segmentaton wth multple vews, or kernels, do not lend themselves to template segmentaton. 3. SEGMENTING GRAPHS 3.1 Spectral Formulaton Let G(V, E) be a weghted undrected graph wth node set V (G) and edge set E(G). Each edge has an edge weght a j such that a j > 0 f there s an edge between nodes and j and a j = 0 f there s no edge between nodes and j. The affnty matrx A s defned as the non-negatve symmetrc matrx A = {a j}. Let the degree of a node be the sum of the edge weghts of the node: d = j aj. Then let the degree matrx D be a dagonal matrx wth the node degrees on the dagonal. The graph partton problem seeks to partton graph G nto k dsjont sets (S 1, S 2,..., S k ) by mnmzng the total sum of the edge weghts removed to create the parttons. Ths total sum s also known as the cut of the graph whch for two sets s defned as: cut(s 1, S 2) = S 1,j S 2 a j. The drawback of the graph partton problem s that found parttons tend to favor separatng a sngle node from the graph. To prevent cuts of the graph whch are unfavorable, Sh and Malk developed the normalzed cut, or Ncut [19] whch s defned as Ncut(S 1,..., S k ) = k =1 cut(s, S ), (1) vol(s ) where vol(s) s the sum of the edge weghts n set S. The defnton ams to fnd balanced clusters based on the edge weghts n the cluster. Solvng ths normalzed partton problem s known to be NP hard. However, the problem can be relaxed to a normalzed spectral clusterng problem. Let the Laplacan matrx be defned as L = D A and the normalzed Laplacan as L = D 1/2 LD 1/2. Now equaton (1) can be solved by solvng the relaxed optmzaton problem: argmn z z T Lz z T z such that z T z = 1 whch s known as normalzed spectral clusterng. Ths problem s easly solved by fndng the k top egenvectors of L. These k egenvectors are then used to fnd the k clusters of the graph. It s dffcult to add constrants to the spectral clusterng formulaton. To add constrants, some methods alter the affnty matrx or the graph Laplacan [14][24][28] such as the method by Lu et al. [17] whch ncorporates par-wse constrants nto the affnty matrx. Other methods use the constrants to restrct the feasble soluton space [6][8][29][30]. Wang and Davdson provde a formulaton for flexble constraned spectral clusterng n whch a constrant s added to the objectve functon for spectral clusterng to create a novel constraned optmzaton problem. However, the formulaton s only able to ncorporate a sngle set of labels as a sngle constrant. 3.2 SDP Formulaton A sem-defnte program (SDP) s a convex optmzaton problem that seeks to mnmze (or maxmze) a lnear objectve functon over the convex cone of symmetrc and postve semdefnte matrces subject to lnear constrants. The canoncal form of a SDP s: argmn trace(kx) X such that trace(a X) = b for = 1,..., m X 0 where the constrant X 0 restrcts X to postve semdefnte matrces. The advantage of formulatng spectral clusterng problems as a SDP s that the relaxed soluton of the SDP formulaton s tghter than the relaxed soluton provded by spectral clusterng. Also, addtonal lnear constrants can be added to the SDP formulaton. Prevous work usng SDP to approxmate graph optmzaton problems ncludes Max-Cut and graph parttonng [9][10][15]. Peng reformulates the spectral clusterng problem as a 0-1 SDP [18] whch, as an nteger program, s NP hard but can also be relaxed. Normalzed spectral clusterng s stll more desrable because clusters are balanced and parttons of sngle nodes are avoded. Below s the relaxed SDP formulaton for normalzed spectral clusterng gven by Xng and Jordan [27]: argmn Z trace( LZ) such that Zdag(D 1/2 ) = dag(d 1/2 ) (2) Z 0 (elementwse) (3) trace (Z) = k (4) Z 0, Z = Z t (5) I Z 0 (6) L s the normalzed Laplacan defned earler. Z R n n s a postve semdefnte co-occurrence matrx and the soluton to the relaxed SDP formulaton. Constrant (2) ensures that each node s assgned to one cluster; (3) and (5) are from relaxng the SDP formulaton; (4) corresponds to the k clusters desred for segmentaton; and (6) s from the defnton of Y. Z s decomposed nto Z = Y Y T where Y R n k yelds the k clusters of the graph. The factor Y has the followng propertes: Y T Y = I and Y = D 1/2 X(X T DX) 1/2. Y s obtaned by applyng a sngular value decomposton to Z and usng the top k sngular vectors. We then apply

4 the randomzed roundng scheme wth restarts to obtan the segmentaton. A dsadvantage of usng the SDP formulatons s that the formulaton cannot handle large matrces. However, low-rank SDP formulatons do exst [16][3]. Another dsadvantage s that the soluton Z from the relaxed formulaton takes on real values n the nterval [-1,1] nstead of ntegers {0, 1}. We present a roundng scheme n Secton 5 that allows us to round the real values of the solutons to more nterpretable ntegers for segmentaton. 4. ADDING TEMPLATES TO SEGMENTING Background knowledge can take the form of par-wse smlartes va a Laplacan or a co-occurrence matrx. We can use the background knowledge we have to create templates for desred clusters. For example, we may want to fnd a segmentaton of the graph that ncludes the template of coworkers who enjoy the same outdoor actvtes. Background knowledge can be added to the SDP formulaton for normalzed spectral clusterng as addtonal lnear constrants. The template constrants can smply be added to the SDP formulaton: argmn Z such that trace( LZ) normalzed spectral clusterng constrants template constrant 1 template constrant 2 The template constrants are of the form trace(kz) β. The value of the constant β determnes whether the segmentaton follows strct or loose adherence to the template constrant. Examples of dfferent types of matrx K for template constrants and the effect of strct and loose adherence to the constrants on segmentatons are shown Secton 4.1. Explanaton and nterpretaton of the constant β for template constrants s n Secton Types of Templates We present four types of template constrants: topologcal groups, characterstcs of people, alternatve segmentatons, and multple metrcs. In each secton, we offer an llustratve example to explan the affect of each template constrant on the segmentaton. Whle the SDP formulaton for spectral clusterng can fnd a segmentaton of a graph for k-clusters, our examples wll be for k = 2. The same graph s used for all examples and the mage of the graph and ts normalze spectral clusterng segmentaton s shown n Fgure (1) Topologcal Groups/Structures It may be desrable to preserve known relatonshps n a socal network such as famly members, co-workers, or rval sports teams. Such relatonshps can be encoded n a matrx. Let the matrx M = {m j}, where m j { 1, 1}, be the n n matrx that holds the together and apart relatons. The template constrant s then trace(mz) β, where β > 0 s some constant (nterpretaton of trace(mz) and β are n Secton 4.2). Consder the networks n Fgure (2). The red square nodes ndcate nodes whch are known to have a relatonshp that the user would lke to preserve. The graph on the left shows strong adherence to the template constrant snce all the red square nodes are n the same cluster. The graph on the rght shows a looser adherence to the template constrant snce one red square node s n a dfferent cluster. Note that the rght graph (wth the looser adherence) has a segmentaton that s close to the normalzed segmentaton. Fgure 2: Example of template clusterng usng nformaton from known relatonshps between a few nodes. (Secton 4.1.1). The left graph shows a strong adherence to the template constrant. The rght graph shows a looser adherence. The red square nodes are assocated wth the template constrant for groups Characterstcs of People We can encode k bnary labels n k ndcator vectors. For ndcator vector v, v = 1 f node has the label and v = 1 f the node does not have the label. The labels may encode the lkes and dslkes of people n a socal network such as whch people enjoy mountan bkng, rock musc, or cookng. We can mplement the labels as a rank-1 matrx M = {m j}, where m j {0, 1}, whch s created from the outer product of the ndcator matrx. The template constrant s then trace(mz) β, where β > 0 s some constant. Consder the two graphs n Fgure (3). The red square nodes are nodes wth labels on them whle the labels for the black crcle nodes are unknown. The graph on the left shows the resultng segmentaton from strong adherence to the template constrant wth all the red square nodes n the same cluster. The rght network shows a looser adherence to the template constrant as red square and black crcle nodes are clustered together nstead of separately. Fgure 3: Example of template clusterng usng nformaton from labels on nodes (Secton 4.1.2). The left graph shows a strong adherence to the constrant. The rght graph shows a looser adherence. The red square nodes are assocated wth the template constrant for labels Alternatve Segmentatons There may be cases where we have a cut of a graph based on another kernel. We can then use the nformaton of the cut as a template constrant. Let M = {m j}, where m j { 1, 1}, be the co-occurrence matrx that encodes the other segmentaton. The template constrant s then trace(mz) β, where β > 0 s some constant. The hgher value of the β, the stronger the soluton Z wll adhere to the constrant. Consder the graphs n Fgure (4). The blue

5 trangle nodes represent an alternatve clusterng. The left graph shows strong adherence to the template constrant: the blue trangle nodes are clustered together, separate from the rest of the graph. The rght graph shows a looser adherence to the template constrant: the blue nodes are clustered together but a black crcle node s allowed as part of the cluster. Fgure 4: Example of template clusterng usng nformaton from another segmentaton (Secton 4.1.3). The left graph shows strong adherence to the template constrant. The rght graph shows a looser adherence. The blue trangle nodes are assocated wth the template constrant for an alternatve segmentaton Multple Dstance Metrcs or Vews The afflatons of a socal network can also be measured wth dfferent metrcs. For example, one metrc may represent work afflatons whle another metrc represents frendshps. We may wsh to fnd a segmentaton that partally adheres to the secondary metrc. The template constrant s trace(mz) β, where β > 0 s some constant. Consder the graphs n Fgure (5). The graph on the left represents the same set of nodes as the graph on the rght but the edges are determned by a some other metrc. The graph on the rght shows the resultng segmentaton when the secondary metrc s ncorporated as a template constrant. produce the normalze cut of the graph. Conversely, large values of β wll favor the template constrant. For example, the graphs n Fgure (3) have nodes wth known labels (shown as red square nodes). The graph on the left shows strct adherence to the template constrant by clusterng all the red square nodes together. The graph on the rght shows the same formulaton but wth a smaller β value for the template constrant. A majorty of the red square nodes are clustered together and the normalzed cut s more closely followed. However, the cut devates from the normalzed cut to cluster three of the red square nodes together. 4.3 Combnng Templates The SDP formulaton allows for the addton of more than one constrant. Thus, a user could construct a template composton of several dfferent nstances of each type of template constrant presented n Secton 4.1. However, one can do more than add multple nstances of one type of constrant such as usng a characterstc constrant wth a groupng constrant. Consder the followng example. Two template constrants are used: Constrant 1 for labels on nodes represented by red square nodes (Secton 4.1.2) wth bound β 1 and Constrant 2 for alternatve segmentatons represented by blue trangle nodes (Secton 4.1.3) wth bound β 2. The affect of changng the values for β 1 and β 2 are shown n Fgure (6). The graph on the left s the result of enforcng a stronger adherence to the alternatve clusterng template constrant va a larger β 2 value. The graph on the rght s the result of enforcng a stronger adherence to the labels on nodes template constrant va a larger β 1 value. Fgure 5: Example of template clusterng usng nformaton from another metrc for the same graph (Secton 4.1.4). The left graph represents the same graph but wth a dfferent metrc. The rght graph s the resultng cut. 4.2 Interpretng constrants/constants The four types of template constrants we have presented take the form of trace(kz) β where K s a matrx representng the addtonal nformaton of the graph, such as known relatonshps or labels, and β > 0 s a constant. An nterpretaton of ths constrant s that we want the segmentaton suggested by Z to match the assgnments gven by K at least β tmes. That s, at least β nodes of the graph are clustered together and match the nformaton gven by K. Note that β [0, m] where m = max n Kj. Note that for small values of β, the segmentaton wll favor the normalzed spectral clusterng. That s, the template constrant wll be loosely adhered to. In fact, β = 0 wll Fgure 6: Example of template clusterng usng nformaton from two types of constrants: alternatve segmentaton and labels. The left graph shows a stronger adherence to the template constrant for the blue trangle nodes. The rght graph shows a stronger adherence to the template constrant for red square nodes. 5. MAKING TEMPLATES INTERPRETABLE 5.1 Roundng Schemes The soluton to the relaxed SDP formulaton for spectral clusterng must be rounded from real values to ntegers for cluster assgnments. Roundng schemes are used to recover the feasble soluton partton matrx X = {x j} where x j {0, 1} A drawback of roundng s that the rounded soluton may no longer satsfy the template constrants. Some roundng schemes nclude: 1. Drectonal cosne method [4]: Orthogonal projecton from n-dmensons to k dmensons

6 2. Randomzed Projecton Heurstc Method [9]: Take rows of Z and project them to a random lower dmensonal space. 3. Roundng by Clusterng: Whle treatng each row of Z as a vertex, use a clusterng method, such as k-mean clusterng, to obtan X. However, none of these roundng schemes provde bounds on the qualty of the rounded soluton. We now present, to our knowledge, some of the only work that attempts to do ths. We present a randomzed roundng scheme for recoverng the soluton X that satsfes the un-relaxed normalzed spectral clusterng problem wth template constrants. Let u : u j [ 1, 1] be the soluton to the relaxed SDP for normalzed spectral clusterng. We can apply the followng randomzed roundng scheme to vector u to get soluton v, v { 1, 1} where v s determned probablstcally as a functon of u : { 1 wth probablty (1 + u)/2 v = 1 wth probablty (1 u )/2 We note the followng well known results: E[v ] = u V ar[v ] = 1 u 2 We now wsh to determne bounds to answer two questons: 1. If we apply the roundng scheme how wll the qualty of the soluton be affected? 2. How many tmes should the roundng scheme be appled n order to satsfy the bounds. One nsght we shall emprcally valdate s that for queston (1) the mean qualty of the soluton after applyng the roundng scheme wll be the same as the qualty of the soluton whch s found, as shown below. [ ] E[trace(KV )] = E l v T = [ ] l E v T = l u T = trace(ku) where K s some kernel However, an answer to queston (2) s not so straght forward. As mentoned before, trace(kv ) = n =1 K V s essentally nterpreted as the agreement between the labelng (kernel K) and segmentaton (soluton V ) schemes. Ths can be broken down nto n summatons whose total value s greater than β. Consder the nstance gvng the worst such agreement whch we know wll have numercal value less than β/n, that s argmn jk j V j < β/n. Let q be the ndex that mnmzes ths expresson. We wsh to determne how many tmes we should apply our randomzed roundng scheme so that ts value s greater than β/n. Let ɛ = K q V q β/n. Note that ɛ cannot exceed one snce β s at most the number of nodes havng the label whch s always less than n and the least that K q V q can be s 0. We wsh our randomzed roundng scheme, when appled to each pont, to have a sum greater than ɛ. Gven that our data set conssts of n ponts, we requre that 1 n K V > ɛ: that s, the roundng scheme wll on average need to exceed the worst possble case. We can now defne the Chernoff nequalty to determne the chance our roundng scheme wll satsfy the template constrant nequalty: [ ] P θ 1 u > ɛ e 2nɛ2 n [ ] P K q V q 1 tr (LU ) > ɛ e 2nɛ2 (7) n Where n s the number of nstances n the data set. Ths gves us the chance that one applcaton of the roundng scheme wll satsfy the bound. We wsh to obtan at least one such success (we can easly emprcally verfy the success) whch s gven by the bnomal dstrbuton 1 f(0; n, p) = ( n 0) p 0 (1 p) n 0 = (1 p) n. We can then determne the number of tmes one must use the randomze roundng scheme so that V satsfes template constrant wth a certan confdence. 6. EXPERIMENTAL RESULTS The purpose of ths expermental secton s two fold: To verfy the correctness of our randomzed roundng scheme. Emprcally verfy the bound n Equaton (7). The frst purpose s to show that the composton of the clusterng does ndeed satsfy the rght hand sde of the bound. For example, for the constrant trace(mv ) β, where matrx M encodes the characterstcs of people (Secton 4.1.2), we are statng our clusterng must contan a cluster wth at least β people wth the characterstc represented by M. The second purpose s requred to determne the tghtness of our bound n Equaton (7). Snce ths s an upper bound we wsh to test how tght the bound s. In ths paper, we use a subset of a Facebook data set 1 (analyss of data set n [2]) consstng of 92 ndvduals. Each ndvdual has a number of possble labels assocated wth hm/her: current locaton, undergraduate unversty, graduate unversty, and current employer. For our frst experment, we label those people who lve n Washngton, DC. We wsh to cluster the network accordng to the frendshp lnks but also requre all nne people who lve n Washngton, DC to be n the same cluster. To acheve ths we created a ndcator label vector l of 0 s and 1 s to dentfy those people who lve n Washngton, DC. We then take the outer product M = ll T to obtan a matrx that s postve sem-defnte. Snce we want all nne people to be n the same cluser we enforce the template constrant: trace(mv ) β 9. Accordng to the bound n Equaton (7), we can generate the probablty of the randomzed roundng scheme satsfyng the requrement that all 9 people 1 Obtaned from the Davs Socal Lnks project ucdavs.edu/lab_webste/)

7 Label set From Went to Wash., DC UVA Upper bound of roundng scheme 82.81% 88.63% success from Eqn (7) Emprcally how often the roundng scheme 65% 43% satsfed trace(mv ) β Emprcally how often roundng scheme clustered 3% 25% labeled nodes together Table 1: Results of usng our randomze roundng scheme on a Facebook data set wth labels. 100 trals of the rounded scheme were used and successes were measured by whether the rounded soluton satsfed trace(mb) β and f the labeled nodes were clustered together. from Vrgna are n the same cluster. We can then emprcally compare what proporton of tmes the roundng scheme when appled satsfed the constrant to verfy the correctness of the bound. For each tme the roundng scheme s successful we can then verfy the composton of the clusterng does ndeed have the nne people from Vrgna n the same cluster. Results are shown n Table 1. We repeat ths experment for another label, people who went to the Unversty of Vrgna (UVA) of whch there are 10 such people n our data set. trace(mv ) β 10. The template constrant s Expermental results are shown n Table 1. Based on our experments, we can conclude that for small β, the bound s loose. For the frst case where the number of labeled nodes s nne, the predcted upper bound of success s 82.81% and was emprcally verfed to be 65%. For the second case where the number of labeled nodes s 10, the predted upper bound of success s 88.63% but was verfed emprcally to be 43%. Furthermore, we emprcally verfed than when the bound was satsfed after randomzed roundng that ndeed all 9 or 10 people from Washngton, D.C. and who went to UVA respectvely were n the same cluster. 7. CONCLUSION AND FUTURE WORK Templates allows us to mpose our background expectatons on the clusterng of a socal network. It allows us to specfy the composton of the segmentatons wth respect to the propertes of the people n the network not just on ther frendshp structure. However, addng multple constrants to spectral clusterng s dffcult. Methods whch utlze the nformaton of other kernels of a network can only do so by takng a lnear combnaton of those kernels and then applyng spectral clusterng technques to the new kernel. We presented a novel approach to clusterng small socal networks wth template compostons usng a relaxed SDP formulaton for normalzed spectral clusterng. The SDP formulaton allows the addton of multple constrants and provdes solutons that satsfy template compostons. We showed the possble types of template constrants, the ways n whch they are mplemented n the SDP, and the ways n whch the user can enforce strct or loose adherence to the constrant. We also addressed the drawback of the SDP formulaton, whch s that the rounded soluton may no longer satsfy the template constrants, by presentng a randomzed roundng scheme. We showed how the SDP for spectral clusterng wth template constrants and the randomzed roundng scheme can be appled to a small socal network. Future work ncludes mplementng other types of template constrants such as groupng constrants and usng alternatve cut nformaton and also mplementng combnatons of two or more of dfferent template constrant types such as a constrant for characterstcs of people and for alternatve cut nformaton. Fnally, applyng the template constrants wth a fast SDP or low-rank SDP formulatons to test constrants for larger data sets. 8. REFERENCES [1] A. Argyrou, M. Herbster, M. Herbster, M. Pontl, and M. Pontl. Combnng graph laplacans for sem-supervsed learnng. In Advances n Neural Informaton Processng Systems 18, pages MIT Press, [2] P. Bhattacharyya, A. Garg, and S. F. Wu. Analyss of user keyword n onlne socal networks. In In Socal Networks Analyss and Mnng Journal (by Sprnger), [3] T. D. Be, N. Crstann, P. Bennett, and E. Parrado-hernández. Fast sdp relaxatons of graph cut clusterng, transducton, and other combnatoral problems. JMLR, 7: , [4] P. K. Chan, M. D. F. Schlag, and J. Y. Zen. Spectral k-way rato-cut parttonng and clusterng. In Proceedngs of the 30th nternatonal Desgn Automaton Conference, DAC 93, pages , New York, NY, USA, ACM. [5] F. R. K. Chung. Spectral Graph Theory (CBMS Regonal Conference Seres n Mathematcs, No. 92). Amercan Mathematcal Socety, Feb [6] T. Coleman, J. Saunderson, and A. Wrth. Spectral clusterng wth nconsstent advce. In Proceedngs of the 25th nternatonal conference on Machne learnng, ICML 08, pages ACM, [7] I. Davdson and Z. Q. Fndng alternatve clusterngs usng constrants. In ICDM 08, pages , [8] T. De Be, J. Suykens, and B. D. Moor. Learnng from general label constrants. In SSPR/SPR, pages , [9] A. M. Freze and M. Jerrum. Improved approxmaton algorthms for max k-cut and max bsecton. In Proceedngs of the 4th Internatonal IPCO Conference on Integer Programmng and Combnatoral Optmzaton, pages 1 13, London, UK, Sprnger-Verlag. [10] M. X. Goemans and D. P. Wllamson. Improved approxmaton algorthms for maxmum cut and satsfablty problems usng semdefnte programmng. JACM, 42: , November [11] M. Grant and S. Boyd. CVX: Matlab software for dscplned convex programmng, verson Apr [12] M. Heler, J. Keuchel, and C. Schnörr. Semdefnte clusterng for mage segmentaton wth a-pror knowledge. In DAGM-Symposum, pages ,

8 2005. [13] T. Joachms, T. J. De, N. Crstann, and N. R. A. Uk. Composte kernels for hypertext categorsaton. In In Proceedngs of the Internatonal Conference on Machne Learnng (ICML, pages Morgan Kaufmann Publshers, [14] S. Kamvar, D. Klen, and C. Mannng. Spectral learnng. In In IJCAI, pages , [15] S. E. Karsch and F. Rendl. Semdefnte programmng and graph equpartton. In In Topcs n Semdefnte and Interor-Pont Methods, pages AMS, [16] B. Kuls. Fast low-rank semdefnte programmng for embeddng and clusterng. In n Eleventh Internatonal Conference on Artfcal Intellgence and Statstcs, AISTATS 2007, page 2007, [17] Z. Lu and M. Carrera-Perpnan. Constraned spectral clusterng through affnty propagaton. In Computer Vson and Pattern Recognton, CVPR IEEE Conference on, pages 1 8, june [18] J. Peng and Y. We. Approxmatng k-means-type clusterng va semdefnte programmng. SIAM J. on Optmzaton, 18: , February [19] J. Sh and J. Malk. Normalzed cuts and mage segmentaton. In Proceedngs of the 1997 Conference on Computer Vson and Pattern Recognton (CVPR 97), CVPR 97, pages 731, Washngton, DC, USA, IEEE Computer Socety. [20] V. Sndhwan and P. Nyog. A co-regularzed approach to sem-supervsed learnng wth multple vews. In Proceedngs of the ICML Workshop on Learnng wth Multple Vews, [21] W. Tang, Z. Lu, and I. S. Dhllon. Clusterng wth multple graphs. In Proceedngs of the 2009 Nnth IEEE Internatonal Conference on Data Mnng, ICDM 09, pages , Washngton, DC, USA, IEEE Computer Socety. [22] D. Tollver, G. L. Mller, and R. Collns. Corrected laplacans: closer cuts and segmentaton wth shape prors. In Computer Vson and Pattern Recognton, CVPR, volume 2, pages IEEE, June [23] U. von Luxburg. A tutoral on spectral clusterng. Statstcs and Computng, 17: , December [24] F. Wang, C. Dng, and T. Lz. Integrated kl (k-means - laplacan) clusterng: A new clusterng approach by combnng attrbute data and parwse relatons. SDM, pages 38 48, [25] X. Wang and I. Davdson. Dscoverng contexts and contextual outlers usng random walks n graphs. In ICDM 09, pages , [26] X. Wang and I. Davdson. Flexble constraned spectral clusterng. In Proceedngs of the 16th ACM SIGKDD Internatonal Conference on Knowledge Dscovery and Data Mnng, KDD 10, pages , New York, NY, USA, ACM. [27] E. P. Xng and M. I. Jordan. On semdefnte relaxaton for normalzed k-cut and connectons to spectral clusterng. Techncal Report UCB/CSD , EECS Department, Unversty of Calforna, Berkeley, Jun [28] Q. Xu, M. desjardns, and K. Wagstaff. Constraned spectral clusterng under a local proxmty structure assumpton. In FLAIRS Conference, pages , [29] S. X. Yu and J. Sh. Groupng wth bas. In In Advances n Neural Informaton Processng Systems. MIT Press, [30] S. X. Yu, I. C. Socety, J. Sh, and I. C. Socety. Segmentaton gven partal groupng constrants. IEEE Transactons on Pattern Analyss and Machne Intellgence, 26: , [31] T. Zhang, A. Popescul, and B. Dom. Lnear predcton models wth graph regularzaton for web-page categorzaton. In Proceedngs of the 12th ACM SIGKDD nternatonal conference on Knowledge dscovery and data mnng, KDD 06, pages , New York, NY, USA, ACM.