Who Thinks Who Knows Who? Socio-cognitive Analysis of Networks. Technical Report

Transcription

1 Who Thinks Who Knows Who? Socio-cogniive Analysis of Neworks Technical Repor Deparmen of Compuer Science and Engineering Universiy of Minnesoa EECS Building 200 Union Sree SE Minneapolis, MN USA TR Who Thinks Who Knows Who? Socio-cogniive Analysis of Neworks Nishih Pahak, Sandeep Mane, and Jaideep Srivasava July 21, 2006

2

3 Who Thinks Who Knows Who? Socio-cogniive Analysis of Neworks Nishih Pahak Dep. of Compuer Science, Universiy of Minnesoa, Minneapolis, USA Sandeep Mane Dep. of Compuer Science, Universiy of Minnesoa, Minneapolis, USA Jaideep Srivasava Dep. of Compuer Science, Universiy of Minnesoa, Minneapolis, USA Absrac 1. Inroducion Inerpersonal ineracion plays an imporan role in organizaional dynamics, and undersanding hese ineracion neworks is a key issue for any organizaion, since hese can be apped o faciliae various organizaional processes. However, he approaches of collecing daa abou hem using surveys/inerviews are fraugh wih problems of scalabiliy, logisics and reporing biases, especially since such surveys may be perceived o be inrusive. Widespread use of compuer neworks for organizaional communicaion provides a unique opporuniy o overcome hese difficulies and auomaically map he organizaional neworks wih a high degree of deail and accuracy. This paper describes an effecive and scalable approach for modeling organizaional neworks by apping ino an organizaion s communicaion. The approach models communicaion beween acors as nonsaionary Bernoulli rials and Bayesian inference is used for esimaing model parameers over ime. This approach is useful for socio-cogniive analysis (who knows who knows who) of organizaional communicaion neworks. Using his approach, novel measures for analysis of (i) closeness beween acors percepions abou such organizaional neworks (agreemen), (ii) divergence of an acor s percepions abou organizaional nework from realiy (mispercepion) are explained. Using he Enron daa, we show ha hese echniques provide sociologiss wih a new ool o undersand organizaional neworks. Keywords Socio-cogniive nework, communicaion nework, belief divergence, Enron daa Organizaion dynamics plays an imporan role in he funcioning of an enerprise. Undersanding he dynamics of organizaional processes empowers managers and enables hem o effecively manage an enerprise's resources. Informal social and sociocogniive neworks in an organizaion play an imporan role in such processes and significan effor has been made o sudy hem. However, mos research has relied on daa colleced manually (e.g. using surveys and observing communicaion beween individuals in meeings) and hence is subec o a variey of noise (e.g. biased opinions). The emergence of compuer neworks has enabled new mehods of communicaion, e.g. and insan messaging, beween individuals in an organizaion, providing a unique opporuniy o sudy social neworks in a deailed and unbiased manner by collecing such daa. In addiion, he ease of use and small coss of elecronic communicaion have enabled geographically dispersed communicaion beween individuals, leading o he creaion of geographically-unresriced social neworks. The curren social nework analysis models like laen space model [3] and p* model [16] suffer from compuaional efficiency and scalabiliy issues. Thus, here exiss a need for new scalable, efficien compuaional echniques o sudy such organizaional neworks. In communicaion, an acor observes only hose s which are addressed o ha acor, i.e., he acor is in eiher To, Cc or Bcc fields of hose s. From a socio-cogniive perspecive, differen acors have differen percepions abou he communicaion nework. Thus, communicaion moivaes as well as enables he sudy of socio-cogniive neworks in an organizaion, i.e., undersanding who knows who knows who in a social nework. No prior research exiss for

4 such an analysis of communicaion neworks. Thus, his paper proposes a novel model for represening he communicaion beween acors in a social nework, using non-saionary Bernoulli probabiliies. Such probabiliies are derived based on he observed communicaion. A Markov ime window based approach is described o handle he nonsaionary naure of Bernoulli probabiliies. As agains a more sophisicaed model, he proposed simple model provides a scalable approach, in addiion o being less affeced by daa sparseness as well as providing reasonable performance on real daa. Daa sparseness exiss in communicaion neworks since an acor (on he average) communicaes wih only a few oher acors (limied social bandwidh observed in social neworks [5]). Thus, such a model can be used for boh socio-cenric as well as ego-cenric analysis of a social nework. Using he proposed non-saionary Bernoulli model, each acor s percepions abou he oal communicaion is modeled using he respecive subse of s observed by ha acor. To quanify he difference in percepions of acors, a novel measure, a- closeness, which uses KL-divergence, is proposed. This measure is similar o he percepual congruence measure in social science lieraure [1]. In addiion o he acors percepions, he server observes all he communicaion, which is also represened using he proposed model and hus forms he baseline for he real communicaion nework. The divergence of an acor s percepions from he real communicaion nework is quanified using a novel measure, called r- closeness. No counerpar for such a measure exiss in social science research due o lack of availabiliy of such real daa. Experimenal resuls using he proposed model and measures on real-world Enron daase show ineresing resuls and illusrae ha hese echniques provide a powerful compuaional ool for social nework analysis. The res of he paper is organized as follows: Secion 2 provides background on he problem addressed in his paper. Secion 3 describes he non-saionary Bernoulli model for consrucing a social nework from daa. Secion 4 explains wo differen socio-cogniive analyses of an communicaion nework using he proposed model and hen describes new measures for such analyses. Secion 5 presens experimenal resuls of socio-cogniive analyses on he Enron daase. Secion 6 summarizes he paper, explains he applicaions of his research and discusses fuure research direcions. 2. Background The firs sub-secion explains basic erminology on social and socio-cogniive nework analysis; he nex sub-secion analyzes he impac of communicaion on social nework analysis; and he final sub-secion describes he problem addressed in his research. 2.1 Social and Socio-Cogniive Nework Analysis Social nework analysis is an acive field of sudy in sociology as well as anhropology. A social nework is a social srucure of individuals (people) called acors, relaed (direcly or indirecly) o each oher hrough a common relaion of ineres [15]. A social nework plays an imporan role in he disseminaion of ideas, informaion or influences among he individuals. However, in any social nework, i is no possible for everyone o be conneced o everyone else, nor is i desirable [1]. Thus, he main moivaion of social nework analysis is o sudy who knows who in a social nework. There are wo ypes of social neworks analysis: (i) Socio-cenric (whole) nework analysis, where he ineracions beween he enire well-defined se of people are sudied; and (ii) Egocenric (personal) nework analysis, where one sudies he ineracions beween an acor (called ego ) and only hose acors relaed (direcly or indirecly) o he ego. Subsanial research has illusraed he imporance of such analyses in organizaions. In an organizaion, informal neworks are formed by relaionships beween employees across funcions and/or divisions in order o accomplish asks quickly [8]. Such informal neworks can cu hrough formal reporing procedures o ump sar salled iniiaives and mee exraordinary deadlines. Informal neworks can us as easily saboage companies' bes laid plans by blocking communicaion and fomening opposiion o change unless managers know how o idenify and direc hem. Social nework analysis enables he undersanding of which acors are perceived as "friends" or "adversaries" by an acor, and which acors are aware of he presence of which oher acors. Taking his a sep furher is socio-cogniive nework analysis, which analyzes who knows who knows who in he social nework. This analysis is useful as i affecs he percepions of an acor abou oher acors, and hence he behavior of acors owards oher acors. This is of prime imporance o a manager in an

5 organizaion. The beliefs for each acor are ranslaed ino a weighed digraph corresponding o he social nework ha exiss from ha acor's perspecive. Using hese digraphs, one can deermine who hinks who is influenial in he organizaion. This informaion is highly valuable for a manager o undersand he exising informal nework in he organizaion. Tradiionally, researchers have relied on acor inerviews and surveys for socio-cogniive daa. Hence, even hough echniques from simple graph-based o sophisicaed mulilevel models ([1], [13] and [14]) exis for analyzing hese responses, here has been no research on exracing ineresing socio-cogniive paerns from large observable communicaion logs Organizaional Communicaion One of he main reasons for compuer neworks (and Inerne) o come ino exisence is o foser collaboraive work beween geographically dispersed researchers. These compuer neworks have now urned ino an infrasrucure ha suppors social neworks; connecing people, organizaions as well as knowledge [13]. The widespread use of inerne and he growing online communiy of users have enabled he formaion of social neworks based on differen relaions of ineres. For example, Usene a widely used online newsgroup had more han 80,000 opic-oriened discussion groups (or social neworks) in These discussion groups allow individuals o form geographically dispersed, loosely-bound, social neworks. On he oher hand, compuer neworks also faciliae an acor o paricipae in differen social neworks (communiies), hus enabling he acor o know many more oher acors and increase his/her social capial. In an organizaion, an server logs all s exchanged beween employees, hus capuring an unbiased view of all communicaion beween hem. In an organizaion, i is possible o map he online acor (e.g. address) o a real-world acor (e.g. employee), and analysis of hese ineracions has he poenial of providing unbiased measures abou social relaionships beween real-world acors. However, o analyze such gigabyes of daa abou s exchanged beween employees (considering a medium scale organizaion) requires new scalable, compuaional echniques. Wih he availabiliy of he Enron corpus, here has been a growing ineres in applying compuaional echniques o analyze based social neworks. Iniial research on analysis of Figure 1. Acor s percepions of a social nework (Socio-cogniive nework). such daa has concenraed mainly on applying radiional social nework echniques and/or graphbased measures [3] [6], [12]. 2.3 Problem descripion This research akes a sep furher by providing novel compuaional echniques for socio-cogniive analysis of daa. Consider an sen by acor A o B, wih Cc o C and Bcc o D. The analysis of he header reveals he following: B and C know ha A and B communicaed, and ha all hree (A, B and C) know abou his communicaion. However, neiher B nor C know ha D was also sen his . Acors A and D know D received he , and boh also know ha B and C do no know ha D received ha . This illusraes ha an can creae differen beliefs abou communicaion among differen acors, depending on wheher and how hey are included in he recipien lis. Based on he observed s, an acor forms his/her beliefs of probabiliies of communicaion beween differen acors. An communicaion nework is defined using he acors as he nodes and edges beween acors represening communicaion beween hem. Each acor in he nework mainains his/her communicaion nework based on he s observed by him/her. Such a communicaion nework defines he acor s beliefs regarding he social nework and he se of such neworks for all acors is defined as a socio-cogniive nework in his paper. (see Figure 1). This paper hus addresses he problem of represening, using an inuiive, simple ye scalable model, he communicaion neworks in a sociocogniive nework and hen illusraes he use of ha model for novel, ineresing socio-cogniive nework analysis.

6 3. An Approach for Socio-Cogniive Nework Modeling This secion presens a novel approach for auomaed consrucion of a communicaion nework in a sociocogniive nework by analyzing of an organizaion s communicaion. 3.1 Basic conceps As explained in previous secion, an acor paricipaing in he communicaion nework mainains beliefs regarding he communicaions in ha nework, i.e. beliefs abou who communicaes wih whom, based on he s ha he acor observes. Basic conceps, which enable modeling of such communicaion probabiliies, are explained here. Consider an communicaion nework consising of N acors denoed by he se, {A i 1 i N }. Le P i = Pr(Sender = A i ) denoe he probabiliy ha an in he communicaion nework is sen by he acor A i. Thus, Number of s sen by Ai Pi = Toal number of s sen in he nework Since each has a unique (single) sender, he evens corresponding o an being sen by differen acors are muually exclusive. Hence, he following condiion mus always hold - P 0, A and P = 1 K (1) i i A i Le P i i = Pr(A Recipiens Sender = A i ) denoe he probabiliy of A being a recipien of an , given ha A i has sen ha , i.e., Number of s sen by Ai and received by A P i = Toal number of s sen by Ai Thus, P(, he probabiliy ha an acor A i sends an o an acor A, is defined as, Number of s sen by Ai and received by A Pi P i = P( = Toal number of s sen in he nework This represens he srengh of he acor A i s communicaion wih acor A. The evens corresponding o differen acors being recipiens of an are no muually exclusive since an may have muliple recipiens. Thus he marginal probabiliies of differen acors being recipiens, are dependen and so do no add up o one. Anoher approach is o consider i as poin o poin communicaion, i.e. an wih muliple recipiens assumed as muliple s wih one recipien for each such . Bu in ha case, he marginal probabiliies are forced o be independen, which may be a srong assumpion ha may no hold in mos cases. Hence, in order o preserve he dependencies beween marginals, he probabiliies P( of an acor A i being a sender and anoher acor A being a recipien are no muually i exclusive evens for differen pairs of senders and recipiens. 3.2 Modeling Communicaion Nework in a Socio-cogniive Nework The even of an acor A i being he sender and A being he recipien of an is muually exclusive o is complemen, i.e. he even where for an eiher A i is no he sender or A is no a recipien or boh. The probabiliies of hese wo evens are represened as P( and 1-P( respecively. We define a Bernoulli disribuion over he wo evens corresponding o communicaion beween acors A i and A, i.e., L( = [P(, 1-P(]. where P( is he parameer of he Bernoulli disribuion L(. For he communicaion nework perceived by an acor, here will N(N-1) such disribuions, one for every ordered pair of acors (A i,a ), A i A. Every exchanged in he nework is a Bernoulli rial, i.e. eiher a given is sen by an acor and he oher acor is one of he s recipien(s) or i s complemen (see Figure 2). Based on such observaions, every acor mainains a disribuion over all possible probabiliies P(y) for a given ordered pair (A x, A y ), i.e. a disribuion over all possible values for he parameer P(y) of each Bernoulli Figure 2. Communicaion beween acors expressed as Bernoulli disribuions. Figure 3. Belief Sae of acor A k, wih beliefs as Bea disribuions. disribuion L(y). For mainaining his disribuion over all possible parameers of a Bernoulli disribuion, a Bea

7 belief sae. To mainain he belief sae of an acor A k, N(N-1) couners, corresponding o α( k of each Bea disribuion, are mainained. In addiion, a couner is mainained for he oal number of s observed by each acor. The β( k parameer is compued by subracing he corresponding α( k couner from he oal number of s. Couners for each of he α( k parameers are iniialized wih heir corresponding priors and he oal number of s couner sars wih an iniial value of (α( k +β( k ), as will be explained laer. 3.3 Non-saionariy and Time Windows disribuion is used. As he Bea disribuion is he conugae prior for he Bernoulli disribuion, a Bayesian updae on he parameers of a Bea disribuion is used for mainaining acors beliefs. DEFINITION 1 (Belief Sae): A belief sae of an acor is defined as a se of N(N-1) Bea disribuions, where each Bea disribuion J( is defined over he corresponding Bernoulli disribuion L( represening communicaion beween acors A i and A. Thus, he belief sae B k for a given acor A k is given as, B k = {J( k ordered (A i,a ) such ha A i A } where J( k, is a Bea disribuion over he parameer of L( and is defined as A k s belief abou probabiliy of communicaion from A i (sender) o A (recipien) (see Figure 3). Each such Bea disribuion J( k in belief sae B k of an acor A k has wo parameers, α( k and β( k. Based on he communicaion A k observes, A k updaes he parameers for all J( k in B k. We associae he parameer α( k wih he number of successes, i.e. he number of s, observed by A k, ha have been sen by A i o A, and parameer β( k. wih failures, i.e. number of s observed by A k for which eiher A i is no he sender or A is no he recipien or boh. Thus, for each observed by A k o be sen from A i o A, he corresponding α( k parameer is incremened whereas for each failure, he parameer β( k. is incremened. Algorihm 1 provides he mehodology for updaing an acor s belief sae based on he se of s observed in a paricular ime window. An acor A k (1 k N) sars wih an iniial belief sae B k, wih parameers for all disribuions having defaul prior values. As acors observe communicaion, he acor updaes his/her As more s are exchanged over ime, he communicaion probabiliies may change. Thus, as A k observes more communicaion in he nework over ime, he/she updaes his/her belief sae using Bayesian inference. Since he underlying Bernoulli probabiliies are non-saionary in naure, we choose o capure his dynamic naure of he communicaion probabiliies using a ime window based approach. A he beginning of each ime window, he parameers for all Bea disribuions in a given acor s belief sae are scaled down by a parameer λ (0 λ 1). For each , he corresponding α( k and β( k are updaed, and hus an acor s belief sae of all communicaion relaions are mainained. A he beginning of he nex ime window, he poserior parameers from previous ime window are scaled down and are used as priors for he nex ime window. The model parameer λ regulaes how much of hisory is remembered by an acor, i.e. he degree of he Markovian chain. Higher he value of λ, more is he imporance given o hisory. If λ=1, each observaion is given he same imporance and all he hisory is remembered. If λ=0, hen he previous probabiliy esimaes are compleely washed ou a he beginning of each ime window and new priors (α( k > 0 and β( k > 0) are chosen. Thus, here is an exponenial decay of hisory, where he rae of decay is conrolled by he parameer λ. Anoher imporan parameer of ineres in his model is he lengh of he ime window. This problem is similar o he classical problem of segmening ime series in emporal daa analysis, since he vecor of communicaion probabiliies is analogous o a ime series daase and each segmen is analogous o a ime window. A Bayesian belief updae for each acor occurs a he end each ime window. The choice of lengh of ime window affecs he number of s observed in he ime window and hence he inerpreaion of resuls. This paper assumes ha he

8 lengh of he ime window is a user-specified parameer, bu i provides sufficien number of s wihin each ime window. Oher approaches such as varying ime window lengh and/or updaing differen acors belief saes a end of differen ime windows can also be adoped, bu hey are lef as open problems for fuure research. To model he emporally varying naure of beliefs, we denoe he belief sae of an acor a ime as B k,. DEFINITION 2 (Belief Sae a ime ): Formally, he belief sae for he given acor A k a he given ime, is defined as, B k, = {J( k, (A i, A ) such ha A i A } where, J( k, is he Bea disribuion for an ordered pair of acors (A i, A ), mainained by he acor A k a ime. The belief sae of a given acor a ime reflecs wha he acor believes o be he probabiliies of he possible srenghs of differen acor communicaions in he nework a ime. A socio-cogniive nework a a given ime is he se of belief saes of all acors a ha ime. 3.4 Priors Selecion This sub-secion addresses he issue of selecing priors for he parameers of each of he disribuions J(y) k in a given belief sae B k. The priors are chosen such ha α(=δ i ε i and β(=1-α(, where δ i is he prior probabiliy for A i being he sender of an and ε i is he prior probabiliy for A being a recipien given ha A i has sen he . Each probabiliy in an acor s belief sae is expressed as a fracion of he communicaion in he nework. Hence, he sum of he expeced probabiliies for all communicaions mus always be greaer han or equal o 1 and less han or equal o (N-1) (see appendix A). Since, he evens of differen acors being senders is muually exclusive, he following condiion mus hold, i δ i = 1. Thus, a simple soluion chosen is o use uniform priors, where each δ i =1/N, N being he number of acors. For ε i, a closed world assumpion is made, i.e. since an acor has no observed any communicaion in he prior sae, he iniial prior probabiliy for he even of A being a recipien given ha he has been sen by A i, is some small ε +. For example, assigning ε i =0.01 gives he following simple soluion for priors is α( k =0.01/N and β( k =1-(0.01/Ν). An advanage of small iniial values for boh α( k and β( k is he low influence of he priors in he updaed belief saes, because as he number of observaions ( s) is usually relaively large compared o he priors, i resuls in washing ou of priors. 3.5 Time Complexiy Analysis This sub-secion analyzes he compuaional complexiy for belief updae an acor needs o perform on observing an . Consider an sen or received by an acor in he communicaion nework. Le he number of recipiens in he be n. The acor needs o updae parameers for all senderrecipien pairs. For his purpose he acor incremens he oal mails observed couner and he α parameer couner for each sender-recipien pair observed by he acor. This requires a maximum of (n+1) updaes. Thus, he complexiy for belief updae for every an acor observes is O(n+1). In case of he sociocogniive nework, since n<<n (N is oal number of acors), he ime complexiy is pracically also approximaely linear. 4. Socio-cogniive Nework Analysis This secion presens wo useful socio-cogniive analyses which can be performed using he model described in he previous secion. 4.1 Divergence beween Beliefs Given he belief saes B and B y, for wo acors A x and A y a ime, here is a need o measure he similariy beween hese belief saes in order o quanify how similar he percepions of he wo acors. Since B and B y, are vecors of probabiliy disribuions, in his paper, for compuing he divergence beween B and B y,, he divergence beween respecive pairs of beliefs in he wo ses are compued and hen combined. In his paper, he divergence beween respecive beliefs of wo acors is defined as he KL-divergence [9] across he expeced Bernoulli disribuions for he wo respecive beliefs. The expeced Bernoulli disribuion for a belief is he expecaion of he Bea disribuion corresponding o ha belief. If J( is he Bea disribuion, hen he corresponding expeced Bernoulli disribuion is denoed as E[J( ], which is obained by normalizing he parameers of Bea disribuion J( as follows, α β E[ J ( ( ] = [ α ( + β ( (, α ( + β ( KL-divergence is an informaion-heoreic measure for quanifying direced divergence beween wo probabiliy disribuions. KL-divergence of a ]

9 probabiliy disribuion p from a probabiliy disribuion q, denoed as KL(q p), is given as, q( x) KL( q p) = q( x) log x p( x) Since i is an asymmeric measure, he symmeric KLdivergence KL sym (q p) is defined as, KL sym ( q p) = KL( q p) + KL( p q). Thus, DEFINITION 3. The similariy beween beliefs of communicaion from A i o A for acors A x and A y, expressed by he Bea disribuions J( and J( y,, a ime, is defined as, Sim( J ( where,, J ( 1 ) = and 1+ KL ( J ( J ( ) y, symm y, p 1 p KL( E[ J ( E[ J( y, ) = plog + (1 p)log K(4) q 1 q α( and = p α( y, + β ( α( y, q = α( + β ( y, This similariy beween wo beliefs ranges from 0 o 1, wih 0 and 1 indicaing minimum and maximum similariy respecively. Definiion 3 is used o measure he similariy beween belief saes of wo acors in he following secions. 4.2 a-closeness Measure An imporan analysis using belief saes for each acor is o measure he similariy beween acors percepions of communicaion neworks. This paper proposes a novel measure, a-closeness, o quanify such similariy in percepions of wo acors. This measure is based on he previous definiion 3 of divergence beween belief saes of wo acors a ime. DEFINITION 4 (a-closeness). The a-closeness measure is defined as he agreemen beween belief saes B and B y, of acors A x and A y respecively a ime, and is given by, a closeness( B, B y, ) = ( B B y, Sim( J ( n( B ) n( B J ( y, ) y, )...(5) where n(b, ) represens he number of beliefs (communicaion links) for which acor A x has observed a leas one and (B y,, ) represens he number of beliefs (communicaion links) for which acor A y has observed a leas one . The a-closeness for wo belief saes is symmeric and ranges beween 0 and 1, wih lower values represening lesser closeness and higher values represening more closeness. I aains a maximum similariy of 1 only when he wo belief saes are idenical. The numeraor in definiion 4 sums up he similariy beween only hose beliefs for which boh A x and A y have observed a leas one . 1 The inuiive reasoning for his is now explained. An communicaion nework is usually quie sparse, i.e. ou of all possible ordered pairs of acors, only a few of hem will acually communicae. Hence, he belief saes of he acors being compared will be even sparser and for boh he acors, he beliefs associaed wih maoriy of communicaions will indicae very low probabiliy of occurring (since no insances of hese ineracions have been observed). In such a case, i is desirable o disregard such beliefs while measuring similariy beween acors belief saes. The siuaion analogous o compuing documen similariy, where one compues similariy based only on hose words ha are presen in boh he documens. Also, if he whole se of beliefs is considered for every acor, one implicily assumes ha he every acors is equally aware of he presence of all acors as well as all relaions in he social nework, which may be quie unrealisic. The denominaor normalizes he numeraor using he geomeric mean of he number of beliefs for which each acor has observed a leas one . The firs applicaion of a-closeness measure is o use i o consruc a graph, called agreemen graph, where nodes represen acors while an edge exiss beween wo nodes if he a-closeness measure beween hose acors is greaer han a user-specified hreshold µ. This graph capures informaion abou which pairs of acors have similar percepion abou communicaion nework. Classical social nework analysis echniques can be applied o such a graph. For example, cliques represen groups of acors having similar beliefs of communicaion neworks, bow-ies represen ariculaion poins, sar srucures idenify he cenral acors, whereas bridges idenify acors wih similar beliefs o wo or more oher groups. A second applicaion of a-closeness is o compue he mean a-closeness across all ordered pairs of acors. This represens he consensus among he acors. Lower mean a-closeness indicaes lower agreemen wihin he social nework while higher mean a-closeness represens higher agreemen beween he acors. In 1 Oher inerpreaions of closeness beween belief saes are possible and remains an ineresing open research problem

10 addiion, he sandard deviaion of across all acors quanifies he variance in agreemen of acors in nework. 4.3 r-closeness Measure For second analysis, his paper inroduces he concep of a super-acor, i.e. an acor who observes all he communicaion in he nework. An server is an example of a super-acor. A closed world assumpion is made wherein all communicaion is said o be sen hrough he server, hence observed by i and no oher communicaion occurs beween he acors. 2 Thus, he super-acor s belief sae for communicaion is a benchmark for realiy, under he closed world assumpion and he sudy of similariy beween an acor s belief sae and he super acor s belief sae (realiy) is a novel and ineresing analysis. To quanify his divergence, his paper proposes r- closeness measure as defined below. DEFINITION 5 (r-closeness). The r-closeness measure is defined as he closeness of an acor A x s belief sae B o super-acor s belief sae (realiy) B S, a a ime and is given by, r closeness A ) = a closeness( B, B ) (6) ( x S, x, K Higher is he r-closeness for an acor, more realisic are he acor s percepions abou communicaion in he nework. The mean r-closeness across all acors provides an aggregae measure of he overall knowledge or level of percepion in he nework. Higher is he mean r-closeness, hen more acors in he nework acually know abou oher acors communicaions, i.e. he communicaion is ransparen. A lower mean value for r-closeness indicaes ha acors generally have mispercepions regarding oher acors communicaions. The laer is usually expeced o be observed for a large social nework consising of various diverse groups, where i is difficul for a single acor o capure all communicaion in he nework.. The sandard deviaion for r-closeness across differen acors indicaes he variance in he levels of percepion. Oher applicaion includes esing new hypoheses for socio-cogniive neworks. For example, Krackhard [6] explains ha an acor s percepion of who communicaes wih whom is a funcion of he acor s social posiion. In an organizaional environmen, i is believed ha op acors in he formal organizaional hierarchy have beer knowledge abou communicaion 2 This assumpion will be relaxed in fuure research. han lesser acors and hence beer percepions abou he social nework, i.e execuive managemen have a beer percepion of he social nework as compared o employees. In addiion, inuiively i is expeced ha, more is he communicaion an acor observes, he beer are acor s percepions abou he social ineracions occurring in organizaion. Such hypoheses can be esed using he r-closeness measure compued for all acors. 5 Experimenal Work This secion describes he experimenal resuls for socio-cogniive nework analysis of Enron daase using he proposed model and measures. 5.1 Enron Corpus The Enron corpus (hp:// is a se of s beween 151 users, mosly senior managemen of Enron, exchanged beween mid-1998 and mid-2002 (approximaely 4 years), which includes he Enron crisis ha broke ou in Ocober In he curren experimenal seup, a cleaned version is chosen, in which duplicae, erroneous and unk s have been removed (Shey and Abidi [11]). The daa consiss of 252,759 messages for he se of 151 users. For his experimenal analysis, firs he enire se of 151 users is chosen, and hen only hose s (approx. 20,311) which are exchanged beween hese 151 users were seleced. The lengh for he ime window was chosen o be one monh. Resuls for differen values of λ {0,0.5,1} are compiled, where λ=0 represens no hisory, λ=1 includes all hisory and λ=0.5 represens an exponenial decay of hisory. 5.2 Experimenal Resuls a-closeness An agreemen graph for socio-cogniive nework is consruced using he a-closeness of acors (employees) a he end of Ocober 2000 and Ocober An edge is drawn beween wo acors only if he a-closeness beween hem was more han a cerain hreshold µ (µ {0.25, 0.5, 0.7}). The a-closeness values beween acors are observed o be low in general and ineresing rends are observed only for µ=0.25. Figures 4 (a), (b) and (c) show he agreemen graph for Ocober, 2000, for differen values of λ and µ=0.25. I is observed ha each graph consiss of many small, disoin componens of users. A possible reason for his is because big

11 organizaions like Enron usually have many organizaional groups wih high inra-group communicaion and low iner-group communicaion. Ineresing srucures like cliques, bowies and sars are observed in he agreemen graph. Excep for a few changes in edges, no significan changes are observed for differen values of λ and almos he same clusers of acors are observed. Bu he reason for such lack of changes wih λ may be because of he naure of he underlying daase, he naure of analysis (looking mainly a macro level saisics and rends) as well as he choice of he ime window lengh. For smaller ime windows or for oher daases, ineresing, unexpeced changes migh be observed for differen values of λ. Figures 5 (a), (b) and (c) show he agreemen graph for Ocober 2001, for µ=0.25 and λ {0, 0.5, 1}. Each one of hem mainly consiss of one large, conneced componen (excep for λ=0 where he large componens breaks up ino wo large componens, however, his does no affec he general conclusions drawn regarding he Ocober 2001 a-closeness rends). This indicaes ha here is a considerable exen of he overlap in social percepions during he crisis period. The conneciviy of he Ocober 2001 agreemen graphs also indicaes ha communicaion (and hence informaion) is shared among various acors and pairs of acors are few hops away from each oher in erms of cogniive overlap. Such a nework is highly conducive owards disseminaion of ideas in a social nework. Indeed, in case of Enron daase, he Enron crisis was a ho opic ha was ofen discussed in he underlying social nework. Also, noe ha he number of nodes in he Ocober 2001 graphs is much more han ha of he Ocober 2000 graphs. An acor is included in he agreemen graph only if i s a-closeness wih a leas one acor, crosses he hreshold. In Ocober 2000, many acors are isolaed from he res of he nework due o less communicaion beween mos acors while in Ocober 2001 almos all acors are par of one big componen due o high overlap of communicaion. We also observe some ineresing srucures such as - cliques of acors having similar r-closeness and persisen cliques (cliques ha exis in boh Ocober 2000 and Ocober 2001). For example, a clique of raders such ha all raders had similar low r-closeness measure shows here was agreemen in he percepions of he group, bu he enire group is far removed from realiy. The second example is a clique of employees which is a persisen clique (disconneced clique in he op righ corner of Figure 9). For acors presen in such a persisen clique, here is probably a srong correlaion in heir roles, like all such acors worked on he same proec. Though insufficien knowledge regarding he domain of daa limis he undersanding of causes for such srucures, he proposed mehodology holds promise in finding ineresing paerns/srucures of he socio-cogniive aspecs of Enron daa, which radiional approaches fail o capure r-closeness The r-closeness across acors is examined for wo differen monhs, Ocober, 2000, a monh wih normal aciviy in he organizaion, and Ocober, 2001, a monh during he Enron crisis. In each case, users are ranked in he decreasing order of r-closeness. For Ocober, 2000, he acors can be roughly divided ino hree caegories. The firs caegory consiss of acors who are communicaively acive and observe a lo of diverse communicaions. These acors occupy he op posiions in he rankings. These are followed by he second caegory acors who also observe a lo of communicaion; however, heir observaions are skewed which in urn leads o skewed percepions. The hird caegory consiss of acors who are communicaively inacive and hardly observe any of he communicaion. These acors have low r-closeness values and are a he boom of he rankings able. Table 1 summarizes he percenages of various acors (according o heir formal posiions) in he differen ranges of r-closeness rankings. Using he rankings for Ocober 2000, wo socio-cogniive nework hypoheses of ineres o sociologiss are sudied. H1. Higher is an acor in he organizaional hierarchy, beer is his/her percepion of he social nework. From he r-closeness rankings, i is observed ha maoriy of he op posiions are no occupied by higher level execuive employees. The op 50 ranks consis of a large chunk of he employee populaion (around 46.4% of he employees) along wih 21.4% of he higher managemen and 34.4% of he execuive managemen acors (see Table 1). A relaed observaion is ha mos of he higher level execuives are communicaively inacive and herefore have fewer percepions. H2. The more communicaion an acor observes, he beer will be his/her percepion regarding he social nework. I is observed ha even hough some acors observe a lo of communicaion, hey are sill ranked low in erms of r-closeness. A main reason for his is ha acors end

12 o paricipae in only cerain communicaions and paricipae less in oher communicaions. This resuls in percepions abou he social nework ha are skewed owards hose favored communicaions. Execuive managemen acors who observed a lo of communicaion showed a endency for his skewed percepion behavior. Table 1. Users in differen rank ranges of r- closeness (Ocober 2000, λ=0.5) No Higher Execuive Employees Ranks Available -men Manage Managemen % (4) % (7) % (28) 4.9% (2) 41.5% (17) 53.6% (22) 7.1% (2) 14.3% (4) 31.0% (9) 78.6% (22) Ohers 3.4% (1) 7.1% (1) 65.6% (19) 21.4% (3) 71.5% (10) s were exchanged across differen levels of formal hierarchy in he organizaion hus exposing managemen level acors o more diverse communicaion [3]. Anoher possible and inuiively appealing reason [3] is ha during Ocober 2000, on an average, managemen people sen abou 80% and received only 20% of he oal communicaion hey were exposed o. In he Ocober 2001, here was a reversal and managemen people sen only 20% and received abou 80% of heir oal communicaion. Since hey observed a lo more communicaion during he laer period, here was a significan increase in he r-closeness ranks of managemen level acors during Ocober Finally, managemen level acors were also lo more communicaively acive in Ocober 2001 han in Ocober 2000 (i.e. hey were exposed o a lo more communicaion during he crisis period and so he 80% of Ocober 2001 is greaer han he 20% of Ocober 2000). Table 2. Users in differen rank ranges of r- closeness (Ocober 2001, λ=0.5) No Higher Execuive Empl Ohe Ranks Available emen Manag Manageoyees rs men % (2) % (9) % (28) Oher observaions from his socio-cogniive nework analysis of Enron daa are discussed below. Table 2 summarizes saisics for r-closeness rankings for he monh of Ocober The rankings for he crisis monh Ocober 2001 are significanly differen from hose of Ocober For boh he monhs, he disribuion of various acors among he r-closeness rankings was only slighly differen for differen λ values. 3 For all values of λ, i was observed ha he percenage of managemen saff among he op 50 ranks increased significanly a he cos of employees being pushed down. Thus, a shif from he normal behavior is observed, indicaing ha communicaion perceived by mos managemen level acors is more diverse and evenly disribued as compared o he skewed or no percepions in Oc A possible reason for his may be ha during he crisis monh, 3 Due o space consrains, only resuls for λ = 0.5 are illusraed here. 2.5% (1) 26.8% (11) 70.7% (29) 3.6% (1) 28.6% (8) 67.8% (19) 20.7% (6) 37.9% (11) 41.4% (12) 0% (0) 7.1% (1) 92.9% (13) Figure 6 is a plo of mean r-closeness of all acors over ime for differen values of λ. An ineresing observaion is ha, for λ=0, he mean r-closeness peaks during he crisis monh of Ocober 2001, indicaing a general increase in he percepion of social ineracions during he crisis period. Afer he crisis period, mean r- closeness drops down. For λ > 0, he plos are almos idenical and i is observed ha r-closeness increases unil he crisis period and afer ha i sabilizes. This can be aribued o increased communicaion among acors. Since almos each acor in he nework was involved in some communicaion, as a resul, he general awareness of an acor increased. The difference in observaion for λ= 0 and λ > 0 is due o he memory effec inroduced by aking λ > 0. 6 CONCLUSIONS AND FUTURE DIRECTIONS The growing populariy of compuer nework-based social neworks and he abiliy o collec gigabyes of unbiased social informaion provides a unique opporuniy for compuer scieniss o develop new compuaional echniques for mining social nework paerns. This paper makes imporan conribuions o his research by (i) providing a scalable compuaional for modeling socio-cogniive neworks for communicaion nework, (ii) proposing a measure o quanify similariies in individual acors percepions of social nework in such a socio-cogniive nework and using i o consruc agreemen graphs beween acors, (iii) idenifying a novel analysis, enabled by social nework on compuer neworks, for quanifying how well an acor s percepions reflec realiy and proposing

13 a new measure for he same, and (iv) illusraing he use of hese echniques using a real-world Enron daa. These echniques provide a handy compuaional ool for sociologiss o analyze large daases and will enable in advancing he undersanding of such social neworks. This paper will moivae research in developing new compuaional ools (e.g. more sophisicaed, scalable approaches) for based social neworks. Fuure research direcions include (i) incorporaing semanic informaion abou he conens of and (ii) differen weighs of imporance for acors in To, Cc and Bcc fields of he Acknowledgemens Nishih Pahak s research was suppored by he Army High Performance Compuing Research Cener (AHPCRC) under he auspices of he Deparmen of he Army, Army Research Laboraory (ARL) under Cooperaive Agreemen number DAAD Sandeep Mane s research was suppored by NSF gran No. IIS The auhors would like o hank Dr. Lyle Ungar for his helpful commens on his research. 8. References [1] N. Conracor (1998) Formal and Emergen Predicors of Coworkers Percepual Congruence on an Organizaion s Social Srucure. Human Communicaions Research, 24, [2] R. Cross, N. Nohria, A. and A. Parker. Six Myhs Abou Informal Neworks and How To Overcome Them. Sloan Managemen Review, 43(3), pp , [3] J. Diesner, and K. Carley. (2005). Exploraion of Communicaion Neworks from he Enron Corpus. Proc. of Workshop on Link Analysis, Counererrorism and Securiy, SIAM Inernaional Conference on Daa Mining 2005, pp Newpor Beach, CA, April 21-23, [4] P. Hoff, A. Rafery and M. Handcock. (2002) Laen Space Approaches o Social Nework Analysis. Journal of American Saisical Associaion, Vol. 97(460), [5] E.M. Jin, M. Girvan, M.E.J. Newman (2001) The srucure of growing social neworks. Phys. Rev., E 64, [6] B. Klim and Y. Yang. (2004). Inroducing he Enron corpus. Firs Conference on and Ani-Spam (CEAS). [7] D. Krackhard. (1990). Assessing he poliical landscape: srucure, cogniion, and power in organizaions. Adminisraive Science Quarerly 35, [8] D. Krackhard and J. Hanson. Informal Neworks: The Company behind he Char. Harvard Business Review, , July-Augus, [9] S. Kullback and R. Leibler. (1951) On informaion and sufficiency. The Annals of Mahemaical Saisics, 22(1): [10] N. Pahak, S. Mane and J. Srivasava. (2006) Who Thinks Who Knows Who? Socio-Cogniive Analysis of Nework. CSE Technical Repor, Universiy of Minnesoa, Minneapolis, USA. [11] J. Shey and J. Adibi (2004). The Enron daase daabase schema and brief saisical repor. Technical Repor, ISI, Universiy of Souhern California. [12] J. Shey and J. Adibi (2005) Discovering Imporan Nodes hrough Graph Enropy - The Case of Enron Daabase. In Proc. of LinkKDD, in conuncion wih he 11h ACM SIGKDD. [13] M. Van Duin, J. Van Busschbach, T. Snider (1999) Mulilevel Analysis of Personal Neworks as dependen Variables. Social Neworks, 21, [14] J. Vermun, M. Kalmin (2006) Random Effecs models for personal neworks, An applicaion o marial saus homogeneiy. Mehodology, 2, [15] S. Wasserman and K. Faus. (1994) Social Nework Analysis Mehods and Applicaions. Cambridge Universiy Press. [16] S. Wasserman and P. Paison (1996) Logi Models and Logisic Regression for Social Neworks: I An Inroducion o Markov Graphs and p*. Psychomerika, 61, [17] B. Wellman. (2001). Compuer Neworks as Social Neworks. Science, 293(14). APPENDIX A. VALID BELIEF STATES Consider he expeced Bernoulli disribuion E[J(y)] using he Bea disribuion J(y), in an acor s belief sae. The parameer of E[J(y)] is he expeced communicaion probabiliy E[P(y)], according o he acor, given by, α( y) E[ P( y)] = where α(y) and β(y) α( y) + β ( y) are he parameers of Bea disribuion J(y). Since, communicaion probabiliies are defined as fracions of he oal communicaion, We mus have, E [ P( ] 1 Recall, (, P ( i, ) = P P i i Each P i has a maximum value of 1, which gives P ( N 1) i, Since, Pi = 1 i (, 1, we mus have, E[ P( ] ( N 1), where N is he number of acors If he expeced communicaion probabiliies in he belief sae of an acor do no saisfy he above inequaliy hen we say ha he acor s belief sae is

14 invalid (i.e. he paricular se of expeced communicaion probabiliies inferred by he acor canno acually exis). PROPOSITION 1. If he prior probabiliies of a belief sae are such ha he belief sae is valid, hen he poserior probabiliies will also resul in a valid belief sae. Le he communicaion probabiliy P( have prior probabiliy x i. Suppose α( = x i and β( = 1-x i. Then he expeced communicaion probabiliy will be, α ( E [ P( ] = = x i α( + β ( Also assume ha he priors correspond o a valid belief sae i.e. 1 x ( N 1) (8) i (, Suppose, in a ime inerval, an acor observes M s ou of which k i are s from acor A i o A. We have, M k i (, M + 1 ( x i + k i ) (9) (from 8) k i (, (, is maximum when every , from some acor, is addressed o every oher acor. ki M ( N 1) (, ( x i (, + k ) M ( N 1) + ( N 1) (10) (from 8) From (9) and (10) we have, i 1 (, xi + ki ( ) ( N 1) M + 1 Bu, xi + ki = x, where x i is he expeced poserior i M + 1 probabiliy, E[P(] poserior. 1 x N 1. i (, The above proof also holds for he case when priors for Bea disribuion parameers do no sum up o 1 i.e. α(y) = rx i and β(y) = r- α(y), where r is some real valued scaling facor indicaing he confidence in he prior probabiliy x i The priors x i can be expressed a produc δ i ε i, where δ i is prior for P i and ε i is prior for P i. In some cases insead of direcly working wih x i, i migh be easier o fix δ i and ε i such ha (8) is saisfied.

15 Figure 4 (a). Agreemen graph for Ocober 2000 (µ = 0.25 and λ = 0)

16 Figure 4 (b). Agreemen graph for Ocober 2000 (µ = 0.25 and λ = 0.5)

17 Figure 4 (c). Agreemen graph for Ocober 2000 (µ = 0.25 and λ = 1)

18 Figure 5 (a). Agreemen graph for Ocober 2001 (µ = 0.25 and λ = 0)

19 Figure 5 (b). Agreemen graph for Ocober 2001 (µ = 0.25 and λ = 0.5)

20 Figure 5 (c). Agreemen graph for Ocober 2001 (µ = 0.25 and λ = 1)

21 mean r-closeness across acors versus ime mean r-closeness _1999 8_ _1999 2_2000 5_2000 8_ _2000 2_2001 5_2001 8_ _2001 2_2002 5_2002 monh_year Figure 6. Mean r-closeness across acors Lambda = 0 Lambda = 0.5 Lambda = 1