Interfaces for networked media exploration and collaborative annotation

Transcription

1 Interfaces for networked meda exploraton and collaboratve annotaton Preetha Appan Bageshree Shevade Har Sundaram Davd Brchfeld Arts Meda and Engneerng Program, AME-TR Arzona State Unversty Tempe, AZ emal: {preetha.appan, bageshree.shevade, har.sundaram, ABSTRACT In ths paper, we present our efforts towards creatng nterfaces for networked meda exploraton and collaboratve annotaton. The problem s mportant snce onlne socal networks are emergng as conduts for exchange of everyday experences. These networks do not currently provde meda-rch communcaton envronments. Our approach has two parts collaboratve annotaton, and a meda exploraton framework. The collaboratve annotaton takes place through a web based nterface, and provdes to each user personalzed recommendatons, based on meda features, and by usng a common sense nference toolkt. We develop three meda exploraton nterfaces that allow for two-way nteracton amongst the partcpants (a spato-temporal evoluton, (b event cones and (c vewpont centrc nteracton. We also analyze the user actvty to determne mportant people and events, for each user. We also develop subtle vsual nterface cues for actvty feedback. Prelmnary user studes ndcate that the system performs well and s well lked by the users. Categores and Subject descrptors H.5.1 [Multmeda Informaton Systems]: Artfcal, augmented, and vrtual realtes, H.5.4 [Hypertext/Hypermeda]: Archtectures, Navgaton, H.5.2 [User Interfaces]: Theory and methods, User-centered desgn General Terms Algorthms, Desgn, Human Factors Keywords Networked meda, communcaton, collaboratve annotaton, meda exploraton, personalzed meda nteracton 1. INTRODUCTION In ths paper we develop a system (ref. Fgure 1 that allows for a network of users to explore the actvtes of a group of frends. The system ncorporates a collaboratve annotaton and a meda rch exploraton framework. Ths s an emergng problem n several contexts: (a sharng of meda s mportant n onlne socal-networks such as Frendster [2], where a set of frends share a set of multmeda experences. (b Wth the ready avalablty of dgtal stll and vdeo cameras, t has become easy Permsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. To copy otherwse, or republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. IUI 05, January 9-12, 2005, San Dego, Calforna, USA. Copyrght 2004 ACM /05/ $5.00. Upload Annotaton Clent Recommendaton System Database User Presentaton System Presentaton Clent Actvty Analyzer Fgure 1: Overall system dagram. Users upload meda to database and annotate usng the recommendaton system that ncorporates user-context. The actvty analyzer updates the user-context whch n turn affects the presentaton system. The dark arrows ndcate data flow from server to clent and the grey arrows, vce versa. to archve events n our daly lves. The popularty of socal networks such as Frendster, as well the formaton of moblogs suggests that users are wllng to share personal meda experences onlne. There has been pror work n creatng collaboratve annotaton systems [7,9]. In [7], the authors explore a collaboratve annotaton system for moble devces. There they used appearance based recommendatons to suggest annotatons to moble users. In [9], the authors descrbe a collaboratve annotaton procedure for scentfc vsualzaton tasks, that can done remotely. A key nnovaton n our approach s to augment the feature based recommendaton systems wth a common sense toolkt [12], thus makng the recommendatons more useful. There has been pror work n explorng mage collectons. Pror work n [10,11], provdes a summary vew of meda through thumbnal dsplay of mages. Temporal arrangement of meda can be explctly quered for, and smlarly other types of Boolean queres can be used to search for meda effectvely. However, these vsualzatons provde no nformaton on spato-temporal or semantc relatonshps amongst meda. Dsplayng mages as clusters around locatons n a map as done n [13], gves an dea of how the meda are dstrbuted across space. In ths work temporal relatonshps between mages are lost. We now present our approach to both the collaboratve annotaton and networked exploraton. In our approach, our collaboratve recommendaton system conssts of the followng components: (a meda and ts features,

2 (b user / group context, (c common sense based recommendatons. The user annotates the mages usng a webbased nterface. As the user begns to annotate mages, the system provdes personalzed recommendatons usng a combnaton of low-level features and a common sense toolkt [12]. After the user has fnshed annotatng an mage, the system creates postve example mage sets (or clusters for the assocated annotaton words wthn each feld (who, when, where, what. The clusters are based on annotaton words/concepts entered by the users and not on automatc groupng of low-level features. These clusters wll help the annotaton process mprove for all users of the network. Our goal s to develop an nteractve exploratory framework that enables easy and ntutve exploraton of shared meda, for a group of users n a socal network, through mplct rather than explct queryng. Our nteracton framework has two key components (a a vsualzaton subsystem (b an nteracton analyss module. The user nteracts wth the shared meda set, through three novel vsualzaton schemes. Her nteractons get captured by the nteracton analyss module, whch then sends nformaton back to the presentaton system. Then, the presentaton system provdes vsual cues to the user that ndcates how she nteracts wth other members. Prelmnary user studes ndcate that the annotaton / exploraton system s well lked by members of the network. The rest of ths paper s as follows. In the next secton, we present a bref overvew of our collaboratve recommendaton system. In Secton 3, we shall descrbe the features extracted, and how we use ConceptNet. In Secton 4, we descrbe our recommendaton system algorthm n detal. In Secton 5, we descrbe the webbased nterface for collaboratve annotaton and show how collaboratve annotaton s related to networked exploraton. Secton 6, dscusses our networked exploraton framework vsualzatons. Secton 7 dscusses detals on actvty analyss and vsual feedback. Fnally, we present our experments n Secton 8 and conclusons n Secton COLLABORATIVE ANNOTATION The motvaton behnd collaboratve annotaton s to share the process of annotatng shared personal meda wthn a socal network. Ths s ntutve as members of a small socal network share the same meda and partcpate n the same events. The shared personal meda n our framework conssts largely of mages and audo assocated wth everyday events. Ths meda needs to be annotated before beng used n the nteractve exploraton framework. Our system requres mnmal authorng the users provde meta-data for the mages n the form of who, where, when and what felds. The system takes advantage of the fact that other users n the network have entered annotaton for ther meda for shared events. 2.1 Context User context models are crucal to collaboratve annotaton as they help to gve personalzed recommendatons to each user. The dctonary defnton of context s gven as: the nterrelated condtons n whch somethng exsts or occurs. These condtons could be the physcal locaton, tme, user s actvty and past actons, envronment etc [14]. In our system, the user s context model comprses (a the ntal user profle entered by the user whch ncludes demographc nformaton lke age, background, hobbes/nterests etc. (b statstcal nformaton lke number of mages contrbuted n the shared socal network and (c usage statstcs whch ncludes the words she has used for annotaton and ther frequency. Mantanng a frequency count of annotatons s ntutve as the meda that s shared by the members of the socal network conssts of everyday events (people, places, what that recur. We also model the group context usng (a the mages uploaded by all the members of the group and (b the annotaton words used by all the members of the group. The group context s therefore, the unon of the user context of all the members of the group. Our system uses the group context to provde group recommendatons usng low-level features. 3. MEDIA ANALYSIS In ths secton, we dscuss the low-level features used and the use of semantcs through ConceptNet. 3.1 Low-level features In ths secton, we gve a bref overvew of the low-level features as well as feature-based mage dstance used n our recommendaton system. In ths work, our feature vector comprses of color, texture and edge hstograms. The color hstogram comprses of 166 bns n the HSV space. The edge hstogram conssts of 71 bns and the texture hstogram conssts of 3 bns. We then concatenate these three hstograms wth an equal weght to get the fnal composte hstogram. The low-level Eucldean feature dstance d between two mages and j s now gven as follows: N b b j 2 <1> b= 1 dj (, = ( h h where N s the total number of bns n the fnal feature vector and h s the hstogram for mage. 3.2 Meda Semantcs Semantcs are ncorporated n our framework through the use of ConceptNet. ConceptNet s a large repostory of commonsense concepts and ts relatons [12]. The repostory represents twenty semantc relatons between concepts lke effect-of, capable-of, made-of etc. In our system, semantcs are ncorporated through the use of ConceptNet. Snce the shared personal meda conssts of everyday events and actvtes; we beleve that usng a commonsense knowledge base such as ConceptNet wll enhance the qualty of the recommendatons. 3.3 Semantc dstance In ths secton, we determne a procedure to compute semantc dstance between any two concepts usng ConceptNet. The ConceptNet toolkt allows three basc operatons on a concept (a fndng contextual neghborhoods that determne the context around a concept or around the ntersecton of several concepts, (b fndng analogous concepts, that returns semantcally smlar concepts for a source concept and (c fndng paths n the semantc network graph between two concepts. In our system, we use the ConceptNet toolkt to determne the semantc dstance between two gven concepts e and f n the followng manner: Context of Concepts: Let us assume that the toolkt returns the contextual neghborhood sets C e and C f for the concepts. Then the context-based semantc dstance d c (e,f s now defned as follows:

3 Ce Cf dc (, e f =, <2> Ce Cf where s the cardnalty operator. Analogous concepts: Let us assume that the returned analogous concept sets are A e and A f. Then the semantc dstance d a (e,f based on analogous concepts s defned as follows: Ae Af da (, e f =. <3> Ae Af Number of paths between two concepts: Gven two concepts, the system extracts the total number of paths between them as well as the number of hops n each path. The path-based semantc dstance d p (e,f s then gven as follows: N 1 1 dp( a, b = N h <4> = 1 where N s the total number of paths between concepts e and f n the semantc network graph of ConceptNet and h s the number of hops n path. The fnal semantc dstance between concepts e and f s then computed as the weghted sum of the above dstances. We have defned equal weght for each of the above dstances. The fnal semantc dstance s gven as follows: d( e, f = w d ( e, f + w d ( e, f + w d ( e, f. <5> c c a a p p 4. RECOMMENDATION ALGORITHM In ths secton, we dscuss the algorthm for the recommendaton system. The goal s to provde recommendatons as the user s tryng to annotate mages uploaded by her. Let us consder the followng scenaro. Let us assume that the user has entered an ntal user profle and the system has been seeded wth a few annotated mages. When the user chooses to annotate an mage, the system provdes recommendatons based on low-level features, user-context, group-context and ConceptNet. As the user annotates mages, the system creates postve example mage sets for the assocated annotatons. The system forms clusters for each dstnct annotaton ntroduced n the system. These clusters grow n number and sze as the users n the network annotate ther shared meda. Thus wth a larger partcpaton by the members of the group, the recommendatons get more refned and reflect the group as a whole. 4.1 Algorthm Detals In ths secton, we present the detals of the algorthm for the recommendaton system. Fgure 2 shows the flow chart of the algorthm. Let us assume that the user wshes to annotate an mage a wth the who, where, when, and what felds. Let us also assume that the database contans N clusters for annotatons wthn each feld Frequency based Personal Recommendaton When the user chooses mage a for annotatng, the system provdes two knds of recommendaton lst for each of the above mentoned felds (a personal recommendaton lst and (b group recommendaton lst. The personal lst s obtaned from the frequency count of the annotaton words used by the user. As the user annotates mages, the system mantans a frequency count wthn each feld for each annotaton word used by the user for Augment personal recommendatons based on ConceptNet. Select mage to annotate Generate group recommendaton based on low-level features. Are group recommendatons semantcally close to user profle? Yes No Annotate mage by choosng recommendatons or addng new annotaton words. Treat the mage as a postve example of the annotaton words. Update cluster centers and create new clusters as needed. Generate personal recommendatons based only on frequency count. Fgure 2: Flow Chart for Recommendaton System Algorthm annotatng her mages. The system then pcks the three most frequently used words wthn each feld to generate the personal lst for each feld Group Recommendaton and Concept Flterng The group recommendaton for each feld s obtaned by computng the low-level feature dstance between uploaded mage Event snowboardng User Context {skng, ce-skatng} {game, snow, break leg} Fgure 3: Flterng of group recommendaton by usercontext usng ConceptNet.

4 and the cluster centers. The system then presents the top three closest cluster center words as recommendatons n the group lst. The system also flters ths group recommendaton lst by the user profle of the currently logged n user to get addtonal personal recommendatons for the what feld of the mage. Ths s done by computng the semantc dstance gven by equaton <5> between every concept n the user s profle and the concepts returned n the group recommendaton lst. When the semantc dstance between the user profle concept and the group recommendaton concept s less than one, the system uses the ConceptNet toolkt to Table 1: Example recommendatons obtaned by flterng group recommendatons wth user-profle usng ConceptNet. The arrow ndcates the sequence of actons taken for the annotatons to be personalzed. Photo Group dnner cake golf User Profle flm musc tenns ConceptNet move, people, char at party, actvty, group, fun game, ball, sports, actvty, fun get a lst of concepts whch are n the context of user profle concept based by group recommendaton concept. For e.g. Suppose the user profle concept contaned the word skng and the group recommendaton contaned the word snowboardng. We now use ConceptNet to get a lst of concepts n the context of skng whch are based by the context of the concept snowboardng. An example of such a lst would contan concepts lke game, snow, break leg etc. The system then pcks the top fve concepts n the lst and adds t to the personal recommendaton lst for the what feld. Ths s done for every concept n the user profle that s semantcally close to the concept n the group recommendaton lst. Ths s shown n Fgure 3. Table 1 shows the recommendatons that are obtaned by flterng group context usng user-context Updatng the System When the user has annotated mage a wth the recommendatons provded or by enterng her own annotatons, the system treats mage a as a postve example of all the annotatons assocated wth t. The system thus creates semantc clusters correspondng to all annotatons that exst n the system. If the user has ntroduced a new annotaton nto the system, then system creates a new cluster for the annotaton wth only mage a as the postve example. The system then computes the semantc dstance between the newly ntroduced annotatons and all the concepts n all the user profles present n the system. Ths s done as a background process to ensure real tme nteractvty and to make the system scalable. The user can now contnue annotatng other mages and ncrease the meta-data n the system. 5. THE ANNOTATION INTERFACE In ths secton, we descrbe the web nterface used for annotatng mages. Let us assume that the user has logged nto the system and has created a sesson. The user now uploads some mages to the shared meda repostory. When the user uploads the mages, the system scales these mages to a fxed sze and computes the color, texture and edge hstograms on these scaled mages. Image chosen for annotaton Personal and group recommendaton for who, where and what. Fgure 5: Web nterface that provdes users wth personal and group recommendatons that facltates authorng of meda. When the user has uploaded the mages, she s requred to group them nto events [3]. In our framework, events have the followng propertes assocated wth them: name, locaton, tme, meda elements (set of mages, sounds, text, as well as partcpants. Once the user has uploaded the mages, the system presents the user wth all the mages that she has uploaded n the system so far but has not grouped nto events. The user can then group mages and creates a new event or she can add mages to an already exstng event. After the user has created events, the user can now choose to annotate mages. Then, the system presents the user wth all the mages that have been grouped by the user nto events but have not been annotated. When she chooses an mage for annotaton, the system provdes her wth recommendaton for the who, where, and what felds of the mage. Ths s shown n Fgure 5. The recommendatons are provded based on the algorthm descrbed n the prevous secton. Collaboratve Annotaton Networked Exploraton Tempe, 11/2/2004, Harn wth frends Harn uploads Preetha recommends Tempe, 11/2/2004, Harn wth frends Harn browses photos Fgure 4: Meda flow through Collaboratve Annotaton and Networked Exploraton framework. Collaboratve Annotaton system helps users annotate meda whch can then be explored wth Networked Exploraton framework. Our web nterface also allows the user to add vewponts to events. Vewpont s defned as a personal narratve about an

5 event [3]. When the user chooses to add vewpont, she s asked to select the event to whch she wants to add vewponts. When the event s selected, the system presents all the mages whch are grouped n that event but have been uploaded and annotated by other users of the network. The user can now choose an mage and add her own personal narratve to that mage. When the user annotates mages, the system updates the cluster centers as descrbed n the prevous secton. These updates get reflected n the recommendatons n the next sesson. Our web nterface also allows the user to browse photographs va the vsualzaton tool. Fgure 4 shows an example of how the collaboratve annotaton framework affects a specfc user s event exploraton. 6. NETWORKED MEDIA EXPLORATION In ths secton, we descrbe the nteractve vsualzaton framework that we have developed, to allow users to explore events and actvtes of frends n ther socal network. 6.1 Desgn Goals It s a challengng task to buld a vsualzaton framework that presents meda to users n novel ways, as well as enables them to understand the relatonshps between people and events. We begn by enumeratng our desgn goals, for ths framework: 1. The framework should be nteractve, and preferably webbased. Ths would requre new vsualzatons and nteracton mechansms that dffer from current query based photo browsng systems. 2. In addton to browsng, the framework should also allow users to perform certan tasks through mplct queryng. Three such tasks are lsted below. In the followng, we shall use magnary users Alce and Bob: Event condtonng: What has Alce done snce a partcular event? (e.g. party at Bob s house Event support: What was the sequence of events that led Alce to meet Bob at partcular locaton? (e.g. Alce meets Bob at the mall. Interestng events: What are the set of events of nterest to Alce? Note that nterest s user context dependent. 3. The authorng envronment should place mnmal burden on the user. Ths ssue was addressed by our annotaton framework. 4. The system should be able to provde feedback to users about ther socal network, by analyzng user nteracton. 5. The communcaton between users should be two way,.e. users should be able to provde feedback to events authored by other users through the system. We have dealt wth desgn goals one and two n pror work [3]. We are addressng the other three goals n ths paper. 6.2 Meda Exploraton framework We now present our meda exploraton framework. Whle we had presented a prelmnary nterface n pror work [3], t now sgnfcantly augmented by real-tme user actvty analyss, a new novel vsual feedback cues Spato-temporal Evoluton Ths vsualzaton addresses the task of event condtonng, as well the nterestng events task descrbed n the desgn goals secton. Pror work n [11] show collectons of dgtal meda organzed ether by space or tme. Our vsualzaton ntroduces the dea of evoluton of meda through both space and tme. We use map data avalable n XML format [1], on whch events unfold over tme and space. The nteracton begns by the user selectng a frend. The system responds by showng the spato-temporal sldeshow of events, over tme and by locaton (e.g. GPS.The temporal transtons between locatons s ndcated usng arcs, and saturaton of the arc color s used to ndcate tme. Ths s llustrated n Fgure 6. Ths vsualzaton addtonally addresses the problem of vsualzng nterestng events. It s reasonable to assume that users are nterested n knowng who n ther group of frends went to the same places as themselves. Hence, when the user moves her mouse over a specfc event locaton, then all the frends of the selected user who partcpated n the same event are hghlghted n green. The other frends of the selected user, who partcpated n other events at the same locaton, are hghlghted n purple. Users can then clck on hghlghted members, to see a spatotemporal sldeshow, startng at that event of nterest Event Cone temporal transton arcs selected user sldeshow Fgure 6: Spato-Temporal Evoluton. For every user, the system shows the events that the user partcpates n over space and tme. events tme selected event two frends meet agan Fgure 7: Ths s a snapshot of all the events assocated wth the selected users. The events are ordered along the vertcal axs and the horzontal axs shows tme. The event cone vsualzaton addresses the task of event support. Ths vsualzaton attempts to provde a summarzed snapshot of the event relatons over tme and space. We call such a summary event cone, as seen n Fgure 7. Note that, whle the spatotemporal evoluton does enable users to understand what happens to a person over tme, t doesn t allow a proper understandng of the event relatonshps. When the user selects her event of nterest n the spato-temporal browser, an event cone of that event s shown, whch shows the

6 tmelne of events of all the partcpants of that event, pror to and after that event. Each partcpant has a dstnct path and s ndvdually colored. When the paths ntersect, they mply that the frends meet. Users can contnue nteractng wth ths vsualzaton by choosng other events from the currently dsplayed event cone, whch wll redraw the event cone wth the chosen event as the center Vewpont Based Evoluton supports Current Event vewponts next event (vewpont dependent Sldeshow Fgure 8: The user can dynamcally change the vewpont, thus changng the sldeshow assocated wth the vewpont, as well as the future event. We present our vsualzaton scheme that allows for vewpont (.e. personalzed narratves based exploraton of events. The dea that users desre agency [5] users proactvely nteract wth the system to control the sequence of events they see, s ncorporated n ths vsualzaton. Intally, crcles representng events assocated wth the selected user appear on locatons n the map. As seen n Fgure 8, the vewponts assocated wth an event, are dsplayed around t. Also the set of precedng events for the current event and one succeedng event are shown as red and blue crcles respectvely. The set of precedng events for each event are calculated as the unon of all events that precede each of the vewponts of the event. The dea behnd ths s to provde context for the user to understand how the event happened, through each vewpont s perspectve. Now users can select vew ponts around the current event, to see a sldeshow of meda assocated wth the event (and the selected vewpont. Also the succeedng event for each event, changes accordng to the chosen vewpont. Thus, the user can contnue to explore any event/vewpont of ther nterest. Thus ths s a very dynamc envronment that allows users to explore events/vewponts n a non lnear manner. 7. ANALYSIS AND FEEDBACK In [8], the authors dscuss why t s vtal for the desgn of user nterfaces for onlne communtes to provde the means to communcate socal cues and nformaton. We attempt to provde such cues to users about ther socal network as they nteract wth our system. 7.1 Actvty Our system allows users to upload meda about ther events and experences, and share them wth frends n ther socal network, through the vsualzaton nterface. Hence the term actvty has two dfferent meanngs assocated wth t (a User actvty by partcpaton and (b User actvty by nteracton. The former refers to actual socal actvty.e. users partcpatng n events and sharng them through our system wth other frends. Ths nformaton s lmted to the extent that the user uploads to the system. The latter refers to the way users nteract wth the system,.e. how they actually vew the meda n the system, who they pay attenton to, etc. Ths can be captured by keepng detaled logs of user nteracton n our system. We would lke to use avalable nformaton about user actvty (.e. by both partcpaton and nteracton to provde vsual feedback to users through the system. Two questons that we attempt to answer for each user through actvty analyss are: (a Who are the people (wthn my network, that I nteract the most wth? and (b Whch are the most mportant events to me? In the followng sub-secton, we descrbe the capture and computaton of user actvty. 7.2 Actvty Capture The three vsualzaton schemes descrbed n the prevous secton complement each other. The functonalty of each vsualzaton wll determne the type of user actvty data we collect from them. We shall descrbe the nformaton collected from each n detal n the followng sectons. Note that the actvty results wll be n general dfferent for each member of the network Spato-Temporal Evoluton Snce ths vsualzaton provdes users choce to watch the actvtes of the socal network members they are nterested n, we conjecture that the tme spent by the user on each member s a good ndcator of the user nterest n that member. Snce ths tme s based by the number of photographs uploaded by that member, we use a normalzed tme score ( t ST. See <6>. To calculate the score of the user wth respect to each member of the socal network, we defne an exponental growth functon on the normalzed tme spent on each member, by the user. The ntuton behnd usng a functon that vares exponentally wth tme s that concepts added to memory also grow at an exponental rate. The equaton used s gven below: np ( tst ( U, P = ts( U, P, NP ( ST W ( U, P = 1 e αt, ST <6> where W ST (U, P s the weght assgned to partcpant P wth respect to the currently logged user U n the spato temporal vsualzaton. n(p s the number of photographs of partcpant P s spato temporal sldeshow that the user saw, N(P s the total number of photographs for P, and t s s the total tme spent by the user on P Event-Cones Ths vsualzaton allows users to see a snapshot of how an event occurred. Thus, n ths vsualzaton users focus on the event tself, rather than ts partcpants. Consequently, we conjecture that the tme spent by users vewng each event s a good ndcator of that event s mportance. We proceed as follows. We record the tme spent on the event by the user, and apply the exponental growth equaton to get the mportance score of each event for the gven user. Snce every event n the event cone s represented by a sngle photograph, we do not need to normalze the tme score here. te W ( U, E = 1 e α <7> EC Where W EC (U, E s the score of the event E for the gven user U, n the event cone vsualzaton, te s the tme spent by the gven user on event E n the event cone vsualzaton.

7 7.2.3 Vewpont Centrc Vsualzaton Whle browsng n ths vsualzaton, users can ether follow all events of a partcular user, or choose a completely non-lnear path, by followng dfferent vewponts of arbtrary events. Hence, user actvty n ths vsualzaton can gve us nformaton about both event mportance and mportance of the members to the user. We measure event mportance as a fracton gven by the rato of number of vewponts of that event that have been seen, to the total number of vewponts for that event. Thus, events, for whch all vewponts have been vewed by the user, would have the maxmum weght. NVseen( E WVP( U, E = NVtotal ( E <8> Where W VP (U,E s the weght of the event E for user U n the vewpont centrc vsualzaton, NV seen (E s the number of vewponts of that event that have been seen, and NV total (E s the total number of vewponts of that event. For member mportance, we record the aggregate of tme spent by the user n browsng that partcpant s vewpont and normalze t. We also use a smlar exponental functon as n <6> to arrve at the weght of the frend wth respect to the gven user, descrbed as follows. np ( t ( U, P (,, VP = ts U P N( P tvp W ( U, P = 1 e α, VP <9> Where W VP (U, P s the weght assgned to partcpant P wth respect to the currently logged user U n the vewpont centrc vsualzaton. n(p and N(P are the total number of photographs of P s vewpont that the user saw, and total number of photographs for P s vewpont respectvely, and t s s the total tme spent by the user on P. 7.3 Aggregate Actvty Scores We descrbe how we arrve at aggregate event mportance scores and user mportance scores, from the data collected on user actvty. Snce we calculate both event scores and people scores for a gven user from dfferent vsualzatons, we need a way to compare the scores across vsualzatons. Hence we perform another normalzaton step, where we obtan the mean and standard devaton of each score per vsualzaton, across sessons. Let us assume that we are calculatng the normalzed score for the vewpont centrc vsualzaton. We use the raw scores to obtan a re-weghted score, as follows: E ( µ e En( =, 3σ e Pm ( µ p Pn ( m =, 3σ p <10> Where E N corresponds to the score of the event, µ e s the mean of all event scores for that vsualzaton, and σ e s the standard devaton of the event scores dstrbuton over all sessons. The correspondng normalzed people score P N s also calculated n the same way as gven above. Dong ths step ensures that the sgnfcance of scores obtaned across vsualzatons s comparable. We repeat ths calculaton for the other two vsualzatons as well. To calculate event mportance scores for each user, we use data about user actvty collected from two vsualzatons the event cone and the vewpont centrc vsualzaton. The average of the normalzed event mportance from these two vsualzatons (E Ne (, E Nv (, gves the fnal event mportance. Here E Ne ( and E Nv ( corresponds to event scores from event cone and vewpont vsualzatons respectvely, computed usng <10>. To calculate mportance scores per user, wth respect to all the other users n the network, we use actvty nformaton collected from spato-temporal evoluton and vewpont centrc vsualzatons. Smlar to event mportance, partcpant mportance s also calculated as average of the normalzed scores obtaned from both the above mentoned vsualzatons (P Ns (m and P Nv (m. 7.4 Vsual cues We descrbe n ths secton, how we use mportance scores obtaned above, to provde vsual feedback to each user, through the system. The purpose of ths s two fold (a provdng feedback to the user about her nteracton, gves her nterestng nsghts and (b the system can learn from user behavor and adapt vsualzaton and nteracton to sut user preferences and resource constrants. Most mportant Least mportant Fgure 9: The above fgure shows vsual cues that ndcate mportant people and mportant events. Color ndcates mportance n people, and saturaton ndcates mportance n events. Red dots above photographs also ndcate partcpant mportance. Our goal s to provde subtle vsual feedback to users nteractng wth the system, rather than explct dsplay of the nformaton calculated. We are motvated by ethnographc studes on Frendster [4]. We use color rather than text, as the prmary form of provdng feedback, guded by general prncples of graphc desgn gven n [15]. We use varyng levels of color to ndcate mportance levels amongst frends n the network, for each user. Photographs of mportant frends n the network appear the brghtest, and frends wth whom the user s actvty ndex s low, appear grayed out. The contrast perceved when the network members photographs vary from color to grey scale, communcates ther mportance or rank very well. We addtonally use red dots above member photographs to ndcate the top fve mportant people to the user. See Fgure 9. To ndcate mportant events to users as they are browsng n the system, we use color saturaton. The saturaton of the color of the event beng rendered ncreases (as the events become less mportant. Thus the most mportant events (accordng to that user s nteracton hstory, would appear darkest. (See Fgure 9.

8 8. EXPERIMENTS We had conducted some prelmnary experments n [3] to only evaluate our vsualzatons. Users were asked to evaluate each of the three vsualzaton schemes.e. vewpont based evoluton, spato-temporal evoluton and event cones. We found our vsualzaton schemes were well lked. In order to evaluate the current system, we asked a group of nne members, seven students and two faculty members to partcpate n the evaluaton. Partcpants could use the web nterface to upload ther everyday personal meda, or send photos to the system through emal from ther cell phone. The partcpants nteracted wth the annotaton and presentaton systems for a perod of one week and recorded ther observatons of partcpaton n the socal network. We opted for a observerpartcpant user-study analyss as opposed to a plot user study, to understand the practce of networked meda use. Most users found the dea of collaboratve annotaton to be nterestng. Some of the postve comments are as follows: The recommendatons make thngs easer for annotaton and are often approprate, recommendatons of names, usng frequency worked well, as mages related to a sngle event are uploaded one after another at one go. Comments about the vsualzaton nterface the event cone vsualzaton s awesome really deal for seeng whose paths crossed when and where, vewpont centrc vsualzaton s a good dea, as many tmes we see a photographs of frends and would lke to add your vewpont about them. Some of the negatve feedback were the sldeshow n the spato temporal evoluton was a lttle too fast, the event cone becomes cluttered when there are more photographs, we should not be forced to create new events, for events that occur perodcally, that mean the same thng but at dfferent tme spans e.g. drvng to work, takng dance lessons. We found these observatons very helpful and encouragng we also plan on conductng a long-term user study to gan further nsght. 9. CONCLUSIONS AND FUTURE WORK We now present our conclusons. In ths paper, we descrbed our system to enable users n a socal network communcate ther everyday experences. Our framework provdes (a an easy to use web nterface to upload meda, b a novel recommendaton system to help users author the uploaded meda, (c presentaton and nteractons schemes to share and vsualze the meda and (d vsual cues to provde feedback to users about ther nteracton. Our collaboratve annotaton system uses meda features, usercontext, group-context and a common sense knowledge base (ConceptNet, to provde recommendatons that enable users to author shared meda. Our presentaton schemes enable users n the socal network to vsualze and nteract wth the shared meda. The actvty analyss subsystem provdes vsual cues that reflect users nteracton wth each other. Our ntal study ndcates that our nterfaces were well lked. We plan to address ssues such as allowng users to structure events and create semantc relatonshps between them. We also plan to conduct extensve user studes to evaluate the system further as well as add more vsualzatons and use Support Vector Machnes [6] to mprove feature based group recommendatons. 10. REFERENCES [1] TIGER maps [2] Frendster [3] P. APPAN and H. SUNDARAM (2004. Networked event exploraton and nteracton summarzaton, Proc. ACM Multmeda 2004, New York, New York, also AME TR , Oct [4] D. BOYD (2004. Frendster and Publcly Artculated Socal Networks, Conference on Human Factors and Computng Systems, Venna, [5] K. M. BROOKS (1997. Do story agents use rockng chars? The theory and mplementaton of one model for computatonal narratve, Proceedngs of the fourth ACM nternatonal conference on Multmeda, , [6] N. CRISTIANINI and J. SHAWE-TAYLOR (2000. An ntroducton to support vector machnes : and other kernelbased learnng methods. Cambrdge, U.K. ; New York, Cambrdge Unversty Press. [7] M. DAVIS, N. GOOD and R. SARVAS (2004. From Context to Content: Leveragng Context for Moble Meda Metadata, MobSys 2004 Workshop on Context Awareness, Boston, MA, [8] J. S. DONATH (1997. Inhabtng the vrtual cty:the desgn of socal envronments for electronc communtes, MIT, PhD Thess. [9] S. E. ELLIS and D. P. GROTH (2004. A collaboratve annotaton system for data vsualzaton, Proceedngs of the workng conference on Advanced vsual nterfaces, ACM Press, , [10] J. GEMMELL, G. BELL, R. LUEDER, et al. (2002. MyLfeBts: fulfllng the Memex vson, Proceedngs of the 10 th ACM nternatonal conference on Multmeda, Juan Les-Pns, France, pp , Dec [11] H. KANG and B. SHNEIDERMAN (2000. Vsualzaton Methods for Personal Photo Collectons : Browsng and Searchng n the PhotoFnder, (ICME2000, New York:, IEEE, , [12] H. LIU and P. SINGH (2004. ConceptNet: a practcal commonsense reasonng toolkt. BT Technology Journal 22(4: pp [13] C. SHEN, B. MOGHADDAM, N. LESH, et al. (2001. Personal Dgtal Hstoran: User Interface Desgn, In Proceedngs of Extended Abstracts of Human Factors n Computng Systems (CHI 2001, ACM Press, [14] H. SRIDHARAN, H. SUNDARAM and T. RIKAKIS (2003. Context, memory and Hyper-medaton n Experental Systems, 1st ACM Workshop on Experental Telepresence, n conjuncton wth ACM Multmeda 2003,, Berkeley CA, AME-TR , Nov [15] A. WHITE (2002. The elements of graphc desgn : space, unty, page archtecture, and type. New York, Allworth Press.