Evolutionary Clustering and Analysis of Bibliographic Networks

Transcription

1 Evolutionary Clustring and Analysis of Bibliographic Ntworks Manish Gupta Univrsity of Illinois at Urbana-Champaign Charu C. Aggarwal IBM T. J. Watson Rsarch Cntr Jiawi Han Univrsity of Illinois at Urbana-Champaign Yizhou Sun Univrsity of Illinois at Urbana-Champaign Abstract In this papr, w study th problm of volutionary clustring of multi-typd objcts in a htrognous bibliographic ntwork. Th traditional mthods of homognous clustring mthods do not rsult in a good typd-clustring. Th dsign of htrognous mthods for clustring can hlp us bttr undrstand th volution of ach of th typs apart from th volution of th ntwork as a whol. In fact, th problm of clustring and volution diagnosis ar closly rlatd bcaus of th ability of th clustring procss to summariz th ntwork and provid insights into th changs in th objcts ovr tim. W prsnt such a tightly intgratd mthod for clustring and volution diagnosis of htrognous bibliographic information ntworks. W prsnt an algorithm, ENtClus, which prforms such an agglomrativ volutionary clustring which is abl to show variations in th clustrs ovr tim with a tmporal smoothnss approach. Prvious work on clustring ntworks is ithr basd on homognous graphs with volution, or it dos not account for volution in th procss of clustring htrognous ntworks. This papr provids th first framwork for volutionsnsitiv clustring and diagnosis of htrognous information ntworks. Th ENtClus algorithm gnrats consistnt typdclustrings across tim, which can b usd for furthr volution diagnosis and insights. Th framwork of th algorithm is spcifically dsignd in ordr to facilitat insights about th volution procss. W us this tchniqu in ordr to provid novl insights about bibliographic information ntworks. I. INTRODUCTION Information ntworks hav bcom ubiquitous in rcnt yars bcaus of th larg numbr of ntworkd applications such as social ntworks, th wb and othr linkd ntitis. For xampl, acadmic ntworks such as DBLP, biological ntworks, and massiv ntity-rlation modls ar all xampls of information ntworks. Such ntworks hav th common proprty that thy contain diffrnt kinds of ntitis which intract with on anothr. Som of th xampls such as social ntworks and th wb ar inhrntly homognous, sinc thy contain ntitis of th sam typ. For xampl, a social ntwork contains actors that ar linkd by frindship rlationships, whras th wb contains documnts which ar linkd by hypr-links. Most work in th ara of graph and ntwork mining has focusd on th homognous domain. Howvr, a htrognous rprsntation is much richr; for xampl a richr rprsntation of a bibliographic ntwork may contain nods corrsponding to diffrnt ntitis such as author, confrnc, papr and trm. Edgs may dnot mor divrs rlationships such as writtn-by btwn an author nod and a papr nod, contains btwn a trm nod and a papr nod, publishd-in btwn a papr nod and a confrnc nod. Clarly, th richr rprsntation of a htrognous ntwork maks it powrful; on th othr hand it is also much mor challnging for mining purposs. In rcnt yars, thr has bn an incrasing intrst in th ara of htrognous information ntworks. In this papr, w will xamin th problm of clustring and volution diagnosis in massiv information ntworks. Htrognous information ntworks ar oftn ncountrd in dynamic nvironmnts which ar continuously volving. Th problm of clustring has bn studid rcntly in th contxt of non-volving and static information ntworks. E.g., Sun t al. [5] prsnt NtClus, which is a clustring algorithm for star schma-basd htrognous information ntworks. Th cntr of th star is trmd as th targt typ whil othr typ nods ar connctd to this cntr typ and ar calld as attribut typ nods. E.g. for th DBLP graph, papr is a targt typ whil authors, confrnc, trms ar attribut typ nods. Figur illustrats a nt-clustr viw of th DBLP ntwork. Th star-schma is a particularly important cas, bcaus of its rprsntational powr in a varity of scnarios. Fig.. Trms Papr Conf Clustr Conf Authors Papr Papr 3 Trms Authors Conf Nt-clustr viw of DBLP ntwork Whil NtClus is a powrful algorithm for dtrmining htrognous clustrs, it incorporats no notion of volution. NtClus, if usd ovr multipl snapshots, would produc clustrs which would hav no corrspondnc with th ons in th prvious snapshot. This ffct bcoms much mor prominnt bcaus NtClus clustrs dpnd immnsly on th initial sds. Th problm of volutionary clustring has bn studid in th contxt of homognous ntworks [4]. Th basic principl is to crat a clustring which focuss on both maintaining high quality clustrs, and on crating clustrs in which a natural corrspondnc can b maintaind among th clustrs across diffrnt snapshots. This broad tchniqu is rfrrd to as tmporal smoothing. A varity of tchniqus

2 [5], [] hav bn dsignd for dtrmining volutionary clustrs using tmporal smoothing. Howvr, ths tchniqus ar inhrntly dsignd for th homognous cas. For htrognous ntworks, w wish to clustr ntir ntity (group of rlatd objcts) as a whol rathr than clustring of individual typs sparatly. It is challnging to clustr th ntitis (ach consisting of multipl typs of nods) with tmporal smoothnss such that th snapshot quality is maintaind. In this papr, w will study th problm of volutionary clustring and diagnosis in information ntworks. W will tak a broadr viw of clustring and volution analysis as two tightly intgratd problms which can b usd in ordr to driv intrsting insights from data. This is spcially tru in th cas of htrognous information ntworks, sinc on can study how th trnds in th diffrnt kinds of ntitis ar affctd by on anothr. For xampl, in an authorship ntwork, volutionary clustring can b combind with carful volutionary diagnosis and mtrics to dtrmin mrgs and splits of diffrnt topical aras, authorship volution and topical volution. Such insights ar critical in undrstanding th natur of th changs which occur in dynamic and rapidly volving information ntworks. Sinc clustring can b viwd as a ntwork summarization tchniqu, it is a natural approach for intgrating with th problm of volutionary diagnosis in ordr to undrstand and summariz th ky changs which may occur in a ntwork ovr tim. In this papr, w mak th following contributions:. Whil th problm of volutionary clustring has bn studid for th homognous cas, th problm is much mor challnging for th htrognous cas, bcaus of th diffrnt ntity-typs which may volv ovr tim. This papr is th first to prsnt an volutionary clustring algorithm for htrognous ntworks. Our algorithm rturns tmporally smoothd, high quality agglomrativ clustrs, and lvrags on som concpts drivd from th NtClus framwork.. W tightly intgrat th problms of volutionary clustring and diagnosis; th volutionary diagnosis is achivd by dfining mtrics and tchniqus to charactriz th clustring bhavior ovr tim. For xampl, w can dsign tchniqus to idntify th birth, continuation and slow disapparanc of a community. W also study th influnc of on community onto anothr and flux btwn two diffrnt kinds of communitis. Our tchniqus ar gnral nough to dal with diffrnt tim granularitis and ntity typs. 3. On of th additional rsults of this ffort is to provid novl insights into th volution of bibliographic ntworks with th us of th tchniqus proposd in this papr. As a spcific xampl, w us th DBLP datast in ordr to provid novl insights about th volution procss. Th papr is organizd as follows. In Sction II, w first prsnt our xtnsion to th NtClus framwork. In Sction III, w dfin volution diagnosis mthods and mtrics. W prsnt xprimntal rsults on th DBLP datast in Sction IV. W thn prsnt an ovrviw of rlatd work in Sction V. W conclud with a summary and futur work in Sction VI. II. THE ENETCLUS ALGORITHM In this sction, w prsnt our algorithm for clustring of an volutionary information ntwork. Th broad approach is to us a probabilistic gnrativ modl in which w modl th probability of gnration of diffrnt objcts from ach clustr. A maximum liklihood tchniqu is usd to valuat th postrior probability of prsnc of an objct in a clustr. Th conditionals (i.., probability of th prsnc of an objct in a clustr) ar computd using ranking of objct within currnt clustrs and rprsntativnss from prvious clustring. Th priors (probability of clustrs) ar stimatd using an Expctation Maximization approach. W first dscrib th problm formulation and th mthods for computing th undrlying probabilitis. Thn w dscrib th ranking and th clustring parts of th algorithm, with a spcial mphasis on how th dynamic volution affcts diffrnt parts of th NtClus framwork. A. Problm Formulation Givn diffrnt snapshots of a graph, ach of which contain nods of multipl typs, our aim is to find a consistnt agglomrativ clustring of th graph snapshots across tim. Consistncy rfrs to our ability to rlat th clustrs to on anothr in diffrnt snapshots, so that it may b bttr possibl to diagnos th volution procss. Lt GS dnot th graph squnc {G i } N i= whr ach of th graphs G i is a snapshot takn at th tim instant {y i } N i=. Givn th numbr of lvls L and th numbr of clustrs K, w would lik to obtain a nt-clustr tr squnc CT S for th graph squnc GS. Dfinition. (Nt-Clustr): A nt-clustr c of a graph G(V,E) is a subgraph G (V,E ) such that V V and E E and E, wight W (E) = W (E ). Lt b c : V [,] dnot th probability with which an objct o V blongs to clustr c. If o is a targt typ objct, b c (o) is ithr or ; for attribut typ objcts o, b c (o) [,]. Dfinition. (Nt-Clustr Tr): A nt-clustr tr CT for a graph G is a tr with L lvls (lvl bing th root and lvl L bing th lavs) and branching factor K. Root of CT corrsponds to th graph G. Childrn nods {nc i } K i= of a nod n stor th K nt-clustrs {c i } K i= obtaind as a rsult of th clustring of th subgraph G n (V n,e n ) at nod n. Th distribution of th targt objcts within ach of th childrn nods of a nod n forms a disjoint partition of targt objcts in nod n. An attribut typ objct o V n blongs to th child nod nc i with probability b i (o). Childrn nods of a nod n ar ordrd. Dfinition.3 (Nt-clustr Tr Squnc): A nt-clustr tr squnc CT S corrsponding to a graph squnc GS is a squnc of nt-clustr trs {CT i } N i= whr CT i corrsponds to tim instant y i. Each of th trs in th squnc hav th sam branching factor and sam numbr of lvls. Th childrn of vry nod in ach of th trs is ordrd in th sns that first child of a nod n i in CT i corrsponds to th first child of a similar nod n j in CT j. W aim at gnrating such a nt-clustr tr squnc CTS for a graph squnc GS such that th trs ar consistnt and rprsnt high-quality clustrs.

3 B. ENtClus Framwork To prform volutionary clustring in a htrognous ntwork, on could us any of th homognous clustring algorithms to clustr ach typ of nods individually. But that would not guarant that all th objcts rlatd to sam ntity li in th sam clustr and also th mutual information btwn diffrnt typs of objcts would not b xploitd. To achiv agglomrativ tmporally smoothd clustrs, w xploit a natural variation of th NtClus algorithm. NtClus prforms itrativ ranking and clustring. W us th knowldg from th currnt snapshot clustring to initializ th clustrs for th nxt snapshot and also to influnc th ranking of objcts in ths nw clustrs using th priors. Not that th priors ar propagatd in th forward dirction as tim progrsss. This mans that th algorithm can b xcutd as an onlin volutionary clustring algorithm. Supplying good priors actually improvs th quality of NtClus algorithm. Thus, with gratr consistncy, w achiv bttr quality unlik othr volutionary clustring algorithms. Algorithm shows our framwork. W xplain ach of th stps in furthr subsctions. Algorithm NtClus with Evolution-Awar Priors : Priors: Initializ prior probabilitis {P(o c k )} K k=. : Initializ: Gnrat initial nt-clustrs. {c k }K k=. 3: Rank: Build probabilistic gnrativ modl for ach ntclustr, i.., {P(o c t k )}K k=. 4: Clustr-targt: Comput (p(c t k o)) for targt objcts and adjust thir clustr assignmnts. 5: Itrat: Rpat stps 3 and 4 until th clustrs don t chang significantly. 6: Clustr-attribut: Calculat p(c k o) for ach attribut objct in ach nt-clustr. 7: rturn p(c k o) C. Initialization of Priors and Nt-Clustrs At th first snapshot, which is dnotd by y, prior probabilitis ar dfind intuitivly. E.g., if w bliv that th data has 4 clustrs, w can dfin high prior probabilitis for th trms rprsntativ of ach clustr. For othr tim instants, th prior probabilitis {P(o c k )} K k= ar dfind as follows. Th prior probability of an objct o blonging to clustr c k is dfind as its rprsntativnss in th corrsponding clustr within th nt-clustr tr for th prvious tim instant (stp ). Th us of ths priors nsurs tmporal smoothnss, bcaus th computation of clustr mmbrship in a particular snapshot is affctd by th mmbrship bhavior in prvious snapshots. Th rprsntativnss of an objct o in clustr c dpnds on th probability of gnrating that objct (givn clustr c), and is invrsly proportional to th ntropy of th clustr mmbrship distribution of th objct o. An objct with a distribution pakd at clustr c will hav a high rprsntativnss valu for clustr c. Subsction II-D illustrats how th ranking part of th algorithm uss ths priors. Th initialization of clustrs should b don in a way that provids a smooth transition from th clustring in th prvious snapshot. Hnc, th algorithm gnrats initial partitions for targt objcts as follows. Lt {o i } L i= b th L attribut typ objcts connctd to an objct o. Considr a probability distribution using th priors mntiond in stp, {p k = L i= P(o i c k )} K k=. A targt objct o is assignd to clustr c k with max probability p k. Thn initial nt-clustrs ar inducd from th original ntwork according to ths partitions, i.., {c k }K k=. This corrsponds to stp of th algorithm. Initializing clustrs using rprsntativnss valus nsurs fastr convrgnc to a bttr local maxima. D. Ranking and Clustring In stp 3, th algorithm builds ranking-basd probabilistic gnrativ modl for ach nt-clustr, i.., {P(o c t k )}K k=. Th ranking procss constructs rprsntativ objcts from th diffrnt clustrs. Th corrsponding probability can b dcomposd by conditioning on th typ of th objct in th corrsponding clustr. In othr words, w hav P(o c k ) = P(T o c k ) P(o T o,c k ). P(T o c k ) is stimatd as th maximum liklihood stimat of typ T o in clustr c k. P(o T o,c k ) can b computd using two diffrnt notions of ranking. For som attributs, w can us a frquncy-basd approach of stimating this probability. For xampl, in a bibliographic information ntwork, th probability for a trm is th wightd ratio of numbr of paprs containing this trm to th total numbr of paprs, whr wights ar associatd with vry papr. P(o T o,c k ) can also b computd using authority-basd ranking. In this cas, ranking of an objct o is stimatd by propagating authority scors from objcts of othr attribut typs via th targt typ. Finally, th ovrall probability P(o T o,c k ) is computd as a wightd sum of ranking-basd P(o T o,c k ) and th priors gnratd in stp. Th prior wight (λ P ) controls how much th currnt ranking and thrfor th currnt clustring dpnds on th clustring at th prvious tim instant. Thus, λ P controls th tradoff btwn th snapshot clustring quality and tmporal smoothnss. By using th rankd attribut objct probabilitis, w can comput th conditional probability of a targt objct o as P(o c k ) = x N Gk (o) P(T x c k ) W(o,x) P(x T x,c k ) W(o,x), whr N Gk is th nighborhood st of objcts in subgraph G k. In stp 4, th algorithm calculats th postrior probabilitis for ach targt objct. This is don by itrating ovr th EM quations p t (c k o) p(o c k ) p t (c k ) and p t+ (c k ) = O i= pt (c k o i )/ O. Onc th postrior probabilitis P(c k o) hav bn computd, ths can b usd to xprss th objct o as a vctor v o = (p(c o),p(c o),...p(c K o)). This nw vctor spac can b lvragd for similarity computation and objct assignmnt. By using prvious clustr assignmnts, vctor v for th clustr cntroids is computd as an avrag of objcts blonging to that clustr. Th objct o is r-assignd to a clustr, by using th cosin similarity valu btwn v o and clustr cntroids. Th stps 3 and 4 ar rpatd until th clustrs do not chang significantly, i.., {c k }K k= = {ct k }K k= {ct k } K k= in th K-dimnsional vctor spac. Finally, in stp 6, postrior probabilitis for ach attribut objct (p(c k o)) in ach nt-clustr ar computd using th postrior probabilitis of its nighboring targt objcts. Not that th way th priors ar propagatd automatically nsurs

4 matching of th clustrs at all lvls in th hirarchy. This is anothr advantag of our maximum liklihood basd modl compard to othr volutionary clustring schms whr grdy mthods ar usd to find corrsponding clustrs across diffrnt snapshots. E. Complxity Analysis As mntiond in [5], clustring onc rquirs O(c E + c N) tim, whr N is th numbr of targt objcts. A ntclustr tr with L lvls is cratd aftr clustring of all intrnal nods i.., KL K nods. But, on an avrag, th siz of th graph at a nod dcrass K tims pr lvl. Hnc, th cration of a clustr tr rquirs O(L (c E +c N)) tim. Th tim rquird to comput th ntir CTS would also dpnd on th numbr of tim granularitis and th numbr of tim instants pr tim granularity. Th xact complxity would dpnd on how dns th graph bcoms at diffrnt instants and intrvals of tim. III. EVOLUTION DIAGNOSIS AND METRICS In this sction, w discuss mthods for volution diagnosis and mtrics. W dfin mtrics for volution quantification such as apparanc and disapparanc rat, continu/mrg/split rat, stability and sociability of objcts. W furthr not that an algorithmic tradoff xists btwn clustring quality and th consistncy of th clustrs ovr tim. Whil most of th proprtis studid in this sction ar proprtis of th data, th lattr ar th proprtis of th algorithm in trms of th lvl of smoothnss. W xamin mthods to quantify and undrstand this tradoff at th algorithmic lvl. A. Quantifying Consistncy ENtClus prforms a clustring of th attribut typ nods, in which mmbrship probabilitis ar assignd to nods. Lt th currnt data st bing clustrd blong to th tim instant y and lt th typ w ar intrstd in b t. Lt th prior wight b fixd to λ P, numbr of clustrs b K and currnt lvl b l. Thn, th mmbrship probability of objct o of typ t to clustr c i is dnotd by {b i (o)} K i=. Intuitivly, consistncy btwn two sts of clustr mmbrship distributions is th dgr of similarity btwn distributions of th intrscting objcts. This implis that th insights drivd from on st of th clustrs should continu to hold valid ovr th nxt st, unlss a major volution has occurrd. W can dfin consistncy of th clustring c as th avrag cosin similarity btwn th clustr mmbrship probability distributions of an objct at tim y and tim y. consistncy(clustring c, y, y)= O o O K K b k(o) k= y b k (o) y K b k(o) y b k(o) y k= k= Such a comparison of consistncy btwn two sts of clustrs should b basd only on th objcts prsnt in both tim instants. Thrfor, th st O usd for th computation procss dnots th st of intrscting objcts at tim y and tim y. Furthrmor, w can dfin consistncy for a particular lvl of th hirarchical clustring as th avrag consistncy of sts of clustrs at that lvl, ach wightd by th numbr of objcts in that st of clustrs. Ovrall consistncy is thn an avrag of th consistncis at ach lvl. Finally, w can xprss consistncy across diffrnt typs as a wightd sum of consistncy with rspct to ach of th typs. Th abov dfinition dfins th consistncy only ovr succssiv snapshots. Howvr, it can b asily gnralizd to th cas of arbitrary intrvals, by using th objcts in th corrsponding intrvals. Furthrmor, w can dfin a chaind path consistncy ovr a squnc of intrvals as th product of consistncis ovr ths corrsponding intrvals. B. Quantifying Snapshot Clustring Quality ENtClus rprsnts ach objctoin ak-dimnsional spac whn prforming clustring. W could us th avrag ratio of intra-clustr similarity to intr-clustr similarity as a masur of th quality of a clustr. This is also calld compactnss. Highr valus of compactnss usually imply that th clustring is of bttr quality. Th compactnss C is dfind as follows: Ok i= C = K s(o ki,c k ) O k= k s(o ki,c k k )/(K ) whr O is th st of th targt objcts, c k is th cntroid for clustr k and s(a, b) masurs th cosin similarity btwn K-dimnsional vctors a and b. W can dfin avrag ntropy of a clustr as E = K O k= Ok o= b k(o) log(b k (o)). Lowr ntropy mans that on an avrag th objcts blong to a particular clustr with high probability and so th clustring is of highr quality. C. Clustr Mrg and Splits As th ntwork volvs, diffrnt clustrs can mrg into a singl clustr or a clustr can split into multipl clustrs. Our soft clustring procss has a fixd numbr of clustrs for ach tim priod. Howvr, w can still study th mrging and splitting of clustrs as follows. Considr a st of clustrs c at lvl l at tim y. If at tim y +, a substantial part of th mmbrship probabilitis movs out of a clustr c i to othr clustrs at th sam lvl, thn w can say that th clustr c i has split into multipl parts. Similarly, if clustr c i has obtaind a substantial part of mmbrship probabilitis from othr clustrs at th sam lvl at tim y, thn w can claim that clustr c i has bn formd from a mrg of othr clustrs. Whil masuring th amount of mrg or split of a clustr c i, w should considr only thos objcts which occur in th ntwork at both th tims i.., at y and y whn studying th mrg phnomna and at y and y + whn studying th split phnomna. Continu rat: This is th rat at which th objcts appar to continu in th clustr. If th mmbrship probability of an objct blonging to clustr c i dcrass, ratio of nw probability to old probability is th continu rat of th objct in clustr c i. If th mmbrship probability incrass, th objct is said to continu in th clustr with continu rat of. Continu rat of clustr c i = O o O min( bi(o)y b i(o) y,) whr O is th st of th objcts that occur in th ntwork both at y and y.

5 Mrg rat: This is th rat at which th objcts appar to mrg into a particular clustr c i from all othr clustrs. An objct is said to b mrging into a clustr c i if its clustr mmbrship probability for c i incrass ovr tim. If th clustr mmbrship probability dcrass for th objct wrt clustr c i, it contributs to th mrg rat of clustr c i. Mrg rat of clustrc i = O o O max(bi(o)y bi(o)y b i(o) y,) Split rat: This is th rat at which th objcts appar to split out of a particular clustr c i to all othr clustrs. An objct is said to b splitting out from a clustr c i if its clustr mmbrship probability for c i dcrass ovr tim. If th clustr mmbrship probability incrass for th objct wrt clustr c i, it contributs to th split rat of clustr c i. Split rat of clustr c i = O o O max(bi(o)y bi(o)y b i(o) y,) Such rats can hlp us dfin intrsting charactristics of th volving data. E.g., in th cas of bibliographic data, th mrg and split rats can hlp us dtrmin th influnc of diffrnt rsarch aras on anothr. Th ability to dtrmin both th volutions of th clustrs as wll as th intractions of diffrnt parts of th ntwork is ky to th infrnc of intrsting volutionary insights in a multi-typd ntwork. D. Clustr Apparanc and Disapparanc A clustr can b considrd nw, whn most of its objcts wr not prsnt in th prvious tim priod. This can b formally dfind in trms of th mmbrship probabilitis as bc(o)y follows: Apparanc rat = o O o O bc(o)y Hr, th st O consists of objcts which wr not prsnt at tim y and ar prsnt at tim y. Th st O consists of all objcts in th clustr at tim y. Similarly, a clustr can b considrd to b disapparing if most of th objcts in it ar absnt at tim y+. Disapparanc rat = o O bc(o)y o O bc(o)y whr th st O consists of objcts which wr prsnt at tim y and ar not prsnt at tim y + and st O consists of all objcts in th clustr at tim y. E. Stability of Objcts Whil many of th afor-mntiond dfinitions quantify th bhavior of clustrs, it is also intrsting to quantify th volutionary bhavior of objcts. Th stability of an objct quantifis th lvl to which th objct is stabl wrt its clustr or th ntwork. ) Tmporal Stability: An objct may appar continuously ovr multipl tim instants or may appar intrmittntly. Simpl tmporal stability can b dfind as th ratio of th numbr of tim instants th objct appars to th numbr of tim instants in th obsrvd tim intrval. Squntial tmporal stability can b dfind as th ratio of th numbr of tim instants th objct disappars to th numbr of tim instants in th obsrvd tim intrval. Maximum squntial tmporal stability can b dfind as th ratio of th maximum tim intrval for which th objct is prsnt in th ntwork to th numbr of tim instants in th obsrvd tim intrval. ) Simpl Social Stability: Th concpt of social stability is basd on clustr mmbrship, and how frquntly objcts shift clustrs. W dfin simpl social stability as th ratio of numbr of tims th objct is rtaind in th sam clustr to numbr of tim instants it appars in th data. For soft clustrs, w can assign vry objct to th clustr for which it has th maximum mmbrship probability. W can also adapt th dfinition as th ratio of similarity btwn clustr mmbrship distribution for th objct ovr conscutiv tim instants to th numbr of tims th objct appars in th data. 3) Rankd Social Stability: Th rankd social stability dfins th lvl of stability among th most rprsntativ objcts in th clustr. This is a natural mtric for a clustring procss which incorporats multi-typd ranking into th undrlying algorithm. Lt L y and L y+ b th list of th top k rprsntativ objcts in a clustr at tim y and y +. Rankd social stability of clustr c can b formally valuatd by Ly Ly+ L y. By varying th valu of k, it is possibl to gain dpr insights into th ffcts of th volution on th most rprsntativ objcts in th clustr. Sinc th most rprsntativ objcts in th clustr can provid intuitiv insights into th natur of th clustrs, this provids a diffrnt prspctiv on how th clustrs may hav changd ovr tim. F. Sociability of Objcts Sociability of an objct is th dgr to which it intracts with diffrnt clustrs. An objct which blongs to many clustrs is mor sociabl compard to on which blongs to a singl clustr. It is a masur of th ntropy of th mmbrship of th objct to diffrnt clustrs. Thrfor, social stability i ovr a tim intrval I can b dfind as pi log(pi) log(k) whr K is th numbr of clustrs and p i is th ratio of numbr of tims th objct blongs to clustr i to th numbr of tim instants in intrval I for which th objct was prsnt in th data. For soft clustrs, p i can b dfind as th ratio of sum of mmbrship probability of th objct in clustr i ovr I to th numbr of tim instants in intrval I for which th objct was prsnt in th data. This can oftn provid novl insights in a multi-typd ntwork. E.g., in a bibliographic ntwork, it can provid insights about how th trms or authors may volv across diffrnt topical aras. G. Effct of Social Influnc Multipl authors may mov out of on rsarch ara to anothr. But thr would still b som authors who may not. W nd a mtric to quantify th dgr to which an objct follows th clustr trnd. Considr a vctor V which has siz K. Lt V (i,j) dnot th movmnt of mmbrship probabilitis of an objct o from clustr c i to clustr c j at tim priod y + i.. th influnc that clustr c i has on clustr c j via objct o. Intuitivly if clustr c i influncs c j through objct o, b i (o) y+ should b lss than b i (o) y and b j (o) y+ should b mor than b j (o) y, othrwis V (i,j)=. In th first cas, V (i,j) can b computd as V (i,j) = (b i (o) y b i (o) y+ ) (b j (o) y+ b j (o) y ). Finally, w normaliz V (i,j) so that all lmnts add up to.

6 Considr anothr similar vctor V of siz K. W comput V (i,j) for vry objct o O and stor th avrag influnc valus in V. Th cosin similarity btwn th vctors V and V provids a good mtric to masur th dgr to which th objct follows th trnd. This can b calld normality. IV. EXPERIMENTS In this sction, w will study th powr of our tightly intgratd approach of studying volution and clustring in finding intrsting cass in bibliographic ntworks. W us th DBLP ntwork. Such a ntwork affords th ability to xamin apparanc of nw rsarch aras, clustr apparanc rat for trms, authors and confrncs. Th goal of this sction is to illustrat th powr of our tchniqus in dtrmining intrsting changs in such a data st. Sinc th focus of this papr is to study clustring as a tool for volutionary analysis of information ntworks, th focus of this sction is also to mak intrsting obsrvations about th undrlying volution with th us of such an approach. A. Data St W prform clustring and study of volution on DBLP data from 993 to 8. This data st contains approximatly 654K paprs, 484K authors, 7K titl trms and 39 confrncs. Th numbr of clustrs was st to 4. W varid th prior wight btwn and. Th priors wr spcifically usd for trms. Th diffrnt nod typs in th graph wr paprs, authors, confrncs and trms. Edgs xist btwn papr and author nod typs, papr and confrnc nod typs and btwn papr and trm nod typs. Our algorithm xplicitly assigns papr nods to particular partitions, and maintains a mmbrship probability distribution for othr nods. W also us a four ara data st, which has bn usd arlir in [5]. This data st focuss on four information procssing rlatd aras. This is a subst of th DBLP data st containing approximatly 4K paprs (with K titl trms) writtn by 6K authors in confrncs in th four aras of data mining, databass, information rtrival and machin larning ovr th yars 993 to 8. W study th ntir DBLP datast in trms of slics and four ara datast in trms of snapshots. B. Evolutionary Analysis of DBLP Data St W prsnt som intrsting rsults for th DBLP data st. Figur shows that th #authors pr papr has bn incrasing ovr tim. W conjctur that this gnral trnd in bibliographic ntworks is a rsult of gratr collaborativ fforts in rcnt yars as a rsult of bttr communications and ntworking abilitis, as wll as bttr softwar support and nablmnt of collaborativ fforts. W also obsrvd that th #trms in th titl of a papr has also incrasd ovr tim. This is bcaus th incrasing complxity of rsarch has brought in nw trms into th vocabulary, whil th old trms also continu to b usd. With incrasing maturity of rsarch aras, authors hav bn writing mor dtail-orintd paprs which nd to us both th old trms and th nw trms in ordr to dscrib th undrlying topic. W also studid th powr law bhavior of th bibliographic ntwork. Figur shows that th numbr of paprs using a % 8% 6% 4% % % (a) #authors=5 #authors=4 #authors=3 #authors= #authors= (b) 5 5 Fig.. (a) Evolution in th numbr of authors pr papr (b) Powr laws in th DBLP ntwork: #paprs vs. rank of a trm(lft), #paprs vs. rank of an author(right) TABLE I CONSISTENCY VERSUS PRIOR WEIGHT Prior wt Author Trm Conf particular trm in th titl or th numbr of paprs publishd by an author follow th xpctd powr laws. On intrsting obsrvation was that th powr law curv bcam mor gntl ovr tim. This is a rsult of th fact that th siz of th ntwork has incrasd ovr tim. This suggsts that largr bibliographic ntworks tnd to hav gntlr powr law curvs. C. Effct of Prior Wights Tabl I shows th ffct of varying th prior wight whn prforming clustring. Th us of a highr prior-wight rsults in mor consistncy and smoothnss in th clustring ovr diffrnt tim-priods. W not that th consistncy valus ris as w incras th prior wight. Th incras in prior wight incrass th influnc of th prvious clustring on th currnt clustring. Introduction of nw nods in th ntwork can rsult in fluctuations of th undrlying consistncy valus. Thus, compard to th original NtClus, w achiv much mor consistnt clustrs. Tabl II shows th variation in compactnss (dfind in subsction III-B) with incrasing prior wight. Notic that th quality dcrass initially and thn improvs as w incras th prior wight. Th dcras happns bcaus for lowr prior wight, th clustring is confusd btwn th prior information and th currnt information whil whn th prior wight is high, th clustring itrations align th currnt information around th prior information and tnd to convrg to a bttr local maxima of log liklihood. Th prior information provid a firm initial clustring. This natur is quit diffrnt from othr clustring algorithms whr it has bn shown that th snapshot quality dcrass as th consistncy incrass. Also not that as prior wight is incrasd, th corrspondnc btwn two clustrs btwn two snapshots incrass. Hnc, as against volutionary K-mans clustring, our clustring automatically rsults into matchd clustrs. Thus a prior wight of.8 hlps provid both good consistncy as wll as good clustring quality. It would b intrsting to s how quality and consistncy chang whn priors ar dfind ovr othr typs lik authors and confrnc also and at diffrnt tim granularitis. W lav it as part of futur work. D. Continu, Mrg and Split Rats Figur 3 shows th continu, mrg and split rats for diffrnt typs of nods whr prior wight is fixd at.8. Th

7 t t r s TABLE II QUALITY VARIATION WITH PRIOR WEIGHT Prior wt Compactnss rg/split Continu/Mr Rats C Lvl Lvl Lvl 3 r r y Author continu Author Mrg Author Split Trm Continu Trm Mrg Trm Split Conf Continu Conf Mrg Conf Split App aranc/ Diappa aranc rat s Apparan nc author Apparan nc trm Apparan nc conf Dis sapparan nc author Dis sapparan nc trm Dis sapparan nc conf Lvl Lvl Lvl 3 Lvl 4 Numbr o f objcts.e+6.e+5.e+4.e+3.e+.e+.e+ Numbr of yars 3 5 Simpl (trm) Simpl (author) Simpl (conf) Max sq (trm) Max sq (author) Max sq (conf) Sq (Trm) Sq (author) Sq (conf) Stability l Social S Simp author trm conf (a) (b) (a) (b) Fig. 3. (a) Variation of th mrg, continu and split rats (b) Variation of th apparanc and disapparanc rats rat valus ar avragd ovr all th yars from 993 to 8. Also, in ach histogram, ach bar rprsnts valus for on lvl of th agglomrativ clustring. Notic that th continu rat is mor than mrg and split rats. Also, mrg rat is gnrally lowr than th split rat. Looking out for outlirs, w did obsrv a high split rat in DB clustr of.89 and a high mrg rat of.8578 for DM clustr for authors in (dnoting th ris of data mining from databass). Thr is a high mrg rat for trms in IR clustr in tim priod (possibly du to publications rlatd to ranking tchniqus). E. Clustr Apparanc and Disapparanc Figur 3 illustrats th apparanc and disapparanc rats for ntir DBLP datast for diffrnt typs. Th rat valus ar avragd ovr all th yars from 993 to 8. A bar in ach histogram rprsnts valus for on lvl of th agglomrativ clustring. On avrag, th apparanc and th disapparanc rats incras as w go dpr into th lowr lvls of clustring, which rprsnt finr graind topics of th bibliographic ntwork. An intuition for this is as follows. Authors oftn publish in diffrnt sub-aras in diffrnt yars, as a rsult of which thy can appar to hav disappard from that sub-ara in that tim priod. Howvr, th author s major ara usually rmains th sam, and hnc th disapparanc rat for authors would b highr in sub-aras than in major aras. This broad intuition is tru across diffrnt kinds of volution of th clustring procss. In th four ara datast, ML is th most dominant clustr in th first fw yars. W obsrv ML confrncs at th top in DM and IR clustrs for thos yars. But slowly in lat 9s, w s IR and DM confrncs apparing at th top. F. Evolution of Individual Nods Whil our afor-mntiond obsrvations discuss th volution of clustrs, w will now study th volution of individual nods. W prform ths xprimnts on th ntir DBLP datast. W study th volution of individual nods in trms of th stability mtrics. Figur 4 shows th diffrnt typs of tmporal stability valus in trms of th numbr of yars. Th figur shows th numbr of objcts vrsus th tmporal stability xprssd in trms of numbr of yars. Not that th confrncs and trms ar mor stabl than th authors. A stability valu of 4 implis that th objct disappard from al Stability nkd Socia Ran Fig. 4. k= k= k= (a) Tmporal stability (b) Simpl social stability At lvl= and prior wt= (a) flnc Social Inf author trm conf IR >IR IR >DM IR >DB IR >ML DM >IR DM >DM DM >DB DM >ML DB >IR DB >DM DB >DB DB >ML ML >IR ML >DM ML >DB ML >ML Fig. 5. (a) Rankd social stability (b) Influncs among th four aras th graph 4 tims in th 6 yars in which it was rprsntd. Th trnd for squntial stability is quit diffrnt compard to th simpl and maximum squntial stability valus. Nxt, w plot th simpl social stability valus for th most tmporally stabl objcts (i.., objcts which wr prsnt in our data for all th 6 yars). Figur 4 shows that on an avrag objcts maintain thir clustr mmbrship distribution upto a dgr of 7%. Th mmbrship bhavior of trms and confrncs in clustrs is much mor stabl as compard to th mmbrship bhavior of authors. This is rasonabl to xpct, bcaus th broad topics in th clustrs volv rlativly slowly, whras th authors may mov in and out of diffrnt topical aras mor rapidly. W not that such obsrvations about th volutionary bhavior of information ntworks can b usful in ordr to idntify th objct typs which show th most intrsting volution trnds ovr tim. Furthr, w study rankd social stability for th nods of typ trm with a prior wight of.8. Th rsults ar illustratd in Figur 5(a). Th numbr of rprsntativ objcts in th ranking was varid at top-k =,,. Whil thr is som variation in th rsults across diffrnt yars, th rsults show that highr stability valus ar achivd by fixing k= as compard to k= or. This suggsts that only th most rprsntativ objcts in th clustr continu to b stabl, whras th modstly rprsntativ objcts may vary mor significantly. Figur 5(b) shows avrag social influnc among diffrnt rsarch aras using th four ara datast across 6 yars. Diffrnt bars rprsnt diffrnt typs. W can clarly s th influnc btwn th DB and IR aras. W also notic ML to IR influnc which is somwhat countr-intuitiv. W think that this happns bcaus in th first fw yars, sinc IR was not much dvlopd, ML authors, confrncs and trms occupy th IR clustr. Mutual influnc btwn DM and ML is quit natural. (b)

8 V. RELATED WORK Traditionally, clustring has bn prformd using mincut, min-max cut, normalizd cut, spctral and dnsity-basd mthods in homognous graph ntworks. Sun t al. prsnt a systm calld RankClus [4] and thn NtClus [5] for clustring ovr htrognous information ntworks. W xtndd NtClus to prform agglomrativ volutionary clustring and thn providd mtrics to analys ths clustrs and masur volution. Our mthod could b xtndd by building on clustr tr squnc pr typ similar to [3], [], which hav diffrnt numbr of clustrs pr typ. Evolutionary clustring has bn studid in som of th works [5], [4]. Chakrabarti t al. [4] proposd huristic solutions to volutionary hirarchical clustring problms and volutionary k-mans clustring problms. Thy introduc th concpts of consistncy of clustrs and clustr corrspondnc. Chi t al. [5] incorporat tmporal smoothnss in volutionary spctral clustring which provids stabl and consistnt clustring rsults. Thy also handl th cas whn nw data points ar insrtd and old ons ar rmovd ovr tim. Whil our framwork automatically taks car of th nw and old data points, w incorporat thm sparatly whn valuating th similarity btwn clustrings. Also, unlik ths works, w focus on volution of htrognous ntworks. Mi t al. [] discovr and summariz th volutionary pattrns of thms in a txt stram. Kumar t al. [7] study th volution of structur within larg onlin social ntworks. Thy prsnt a sgmntation of th ntwork into thr rgions and study th volution of ths rgions. Th ara of volutionary clustring is also closly rlatd to aras lik clustring data strams. W lav storag and clustring of ntwork data strams as futur work. Sun t al. [3] propos a systm, GraphScop, which idntifis communitis in a paramtr-fr way, using th MDL principl. Kim and Han [6] prform volutionary clustring using dnsity-basd mthods. W us NtClus to idntify clustrs. Similar to thir work, w can also track changs in clustrs, apparanc and disapparanc of various clustrs ovr tim. Backstrom t al. [] prsnt an analysis of group formation and volution in LivJournal and DBLP. Som of our volution mtrics ar influncd by thir work. Howvr, thy dfin confrncs in DBLP as clustrs whil w hav typd-clustrs obtaind using NtClus. Lskovc t al. [8], [9] prsnt a dtaild study of ntwork volution. Howvr, thy do not dal with clustring of ths graphs or study of th volution of clustrs. Tang t al. [6] study community volution in a multi-mod ntwork using a spctral framwork. FactNt [] provids a framwork for analysing communitis and thir volution. W study volution of clustrs in much mor dtail. Apart from that th clustrs obtaind using th itrativ NtClus algorithm hav bn shown to b mor maningful and hnc studying thir volution is intrsting. Asur t al. [] charactriz complx bhavioral pattrns of individuals and communitis ovr tim. Thy do not prform any tmporally smoothd clustring. VI. CONCLUSION AND FUTURE WORK In this papr, w dsignd a clustring algorithm for volution diagnosis of htrognous information ntworks. This approach tightly intgrats th volution and clustring procss, and provids novl insights into th volution both at th objct lvl and th clustring lvl. W studid th application of our approach on bibliographic information ntworks. W providd novl insights for volution diagnosis on th DBLP data st, and showd th ffctivnss of th volutionsnsitiv clustring approach for htrognous information ntworks. W can furthr modify th tchniqu to incorporat variabl numbr of clustrs at diffrnt tim priods. Also, it would b intrsting to study th ffct on compactnss for diffrnt tim granularitis and whn priors ar dfind for othr nod typs. Such an volutionary clustring ovr htrognous information ntworks can also b hlpful in idntifying outlirs in th ntwork both in th static as wll as volutionary sns. VII. ACKNOWLEDGEMENTS Th work was supportd in part by NSF IIS-9-55, and th U.S. Army Rsarch Laboratory undr Cooprativ Agrmnt Numbr W9NF (NS-CTA). Th viws and conclusions containd in this documnt ar thos of th authors and should not b intrprtd as rprsnting th official policis, ithr xprssd or implid, of th Army Rsarch Laboratory or th U.S. Govrnmnt. Th U.S. Govrnmnt is authorizd to rproduc and distribut rprints for Govrnmnt purposs notwithstanding any copyright notation hr on. REFERENCES [] S. Asur, S. Parthasarathy, and D. Ucar. An vnt-basd framwork for charactrizing th volutionary bhavior of intraction graphs. KDD 7 [] L. Backstrom, D. Huttnlochr, J. Klinbrg, and X. Lan. Group formation in larg social ntworks: mmbrship, growth, and volution. KDD 6 [3] R. Bkkrman, R. El-Yaniv, and A. McCallum. Multi-way distributional clustring via pairwis intractions. ICML 5 [4] D. Chakrabarti, R. Kumar, and A. Tomkins. Evolutionary clustring. KDD 6 [5] Y. Chi, X. Song, D. Zhou, K. Hino, and B. Tsng. Evolutionary spctral clustring by incorporating tmporal smoothnss. KDD 7 [6] M. Kim and J. Han. A Particl-and-Dnsity Basd Evolutionary Clustring Mthod for Dynamic Ntworks. VLDB 9 [7] R. Kumar, J. Novak, and A. Tomkins. Structur and volution of onlin social ntworks. KDD 6 [8] J. Lskovc, L. Backstrom, R. Kumar, and A. Tomkins. Microscopic volution of social ntworks. KDD 8 [9] J. Lskovc, J. Klinbrg, and C. Faloutsos. Graphs ovr tim: dnsification laws, shrinking diamtrs and possibl xplanations. KDD 5 [] Y. Lin, Y. Chi, S. Zhu, H. Sundaram, and B. Tsng. Factnt: a framwork for analyzing communitis and thir volutions in dynamic ntworks. WWW 8 [] B. Long, Z. Zhang, X. Wú, and P. Yu. Spctral clustring for multi-typ rlational data. ICML 6 [] Q. Mi and C. Zhai. Discovring volutionary thm pattrns from txt: an xploration of tmporal txt mining. KDD 5 [3] J. Sun, C. Faloutsos, S. Papadimitriou, and P. Yu. Graphscop: paramtr-fr mining of larg tim-volving graphs. KDD 7 [4] Y. Sun, J. Han, P. Zhao, Z. Yin, H. Chng, and T. Wu. Rankclus: intgrating clustring with ranking for htrognous information ntwork analysis. EDBT 9 [5] Y. Sun, Y. Yu, and J. Han. Ranking-basd clustring of htrognous information ntworks with star ntwork schma. KDD 9 [6] L. Tang, H. Liu, J. Zhang, and Z. Nazri. Community volution in dynamic multi-mod ntworks. KDD 8