User-centric Cross-network Social Multimedia Computing

Transcription

1 Part III User-centric Cross-network Social Multimedia Computing Jitao Sang Multimedia Computing Group National Lab of Pattern Recognition, Institute of Automation Chinese Academy of Sciences

2 Big Data & Multimedia Big Data:any collection of data sets so large and complex that is difficult to process using traditional techniques. --- Wikipedia Enterprise & government data According to IDC, in 5 years, the data storage will reach 18EB (10 18 ), in fields of telecommunication, financial services, health care, public safety, transportation, education, etc. Internet data BAT(Baidu Alibaba Tencent)possess data in the scale of 10EB(10 18 ),and increase at a speed of PB per day. Personal data EMC2 estimated that an individual contributes to average 45 GB personal data (public service, credit record, video surveilliance, social media data, etc.) 2

3 Big Data & Multimedia Big Data:any collection of data sets so large and complex that is difficult to process using traditional techniques. --- Wikipedia 45% else 60% else else 70% Enterprise & government data Internet data Personal data Multimedia 3

4 Big Data & Social Multimedia Social Multimedia has significant big data 4V characteristics: YouTube: #[videos] > 2 billion; Facebook: #[pics] > 300 billion huge volume Volume Variety sources formats source:desktop/mobile, official/individual; format:traditional photo/video/audio, new media-pic tweet/audio pic/geo-tagged media YouTube: uploading 72 hour video per min. Skype: up to 1.4 million mins chat per min exponential growth Velocity Veracity low value density format: 1 hour video with few semantics; generation:open environment -> low quality, duplicate data; demands:personalized 4

5 Big Data & Social Multimedia Social Multimedia has significant big data characteristics: capacity in data storage YouTube: #[videos] > 2 billion; Facebook: #[pics] > 300 billion huge volume Volume Variety sources formats source:desktop/mobile, official/individual; format:traditional photo/video/audio, analysis new media-pic tweet/audio pic/geo-tagged media complexity in data efficiency in data capture & computing YouTube: uploading 72 hour video per min. Skype: up to 1.4 million mins chat per min exponential growth Velocity Veracity low value density format: 1 hour video with few semantics; data accuracy and quality generation:open environment -> low quality, duplicate data; demands:personalized 5

6 Variety in Social Multimedia Multiple Modalities received extensive attentions in the small data era Multiple Sources

7 Variety in Social Multimedia beyond multiple modalities: the heterogeneous data created same modality, different information. and consumed in various social media networks upload favorite playlist rate share comment +1 Content + Context. Multiple Sources

8 Variety in Social Multimedia beyond multiple modalities the heterogeneous data created and consumed in various social media networks Multiple Sources

9 Multisource in Social Multimedia Macro-level analysis: Characteristics of different social media networks. - degree distribution, clustering coefficiency [Ahn et al. 2007], - degree centrality, shortest path [Magnani and Rossi, 2011]; User activity patterns in macro-level. - user tagging patterns [Guo et al. 2009]; - user participation motivations [Choudhury and Sundaram, 2011]. Diffusion dynamics between social media networks. - cite and influence correlation [Leskovec et al. 2007]; - diffusion and evolution patterns [Rodriguez et al. 2013]; - jointly analyze network characteristics, user activity patterns, and diffusion dynamics [Kim et al. 2014]

10 Multisource in Social Multimedia Micro-level analysis and applications: Concept: different perspectives for the same concept/event, e.g., the distribution and evolution of social events among Twitter, Facebook, etc. Jasmine Revolution User: different domains involved by the same individual, e.g., unique user registers and participates into several social media websites.

11 User-centric Solution Heterogeneous data among different social media networks share the unique user space:

12 Cross-network User Account Collection Identical user account among different social media services. Google account Tencent account Users are voluntary to discover their accounts in multiple networks. User account linkage mining is a separated research topic.

13 User-centric Cross-network Dataset Identify cross-network user accounts for unique individual API API API API API API Use corresponding APIs to collect user data Cross-network Dataset

14 User-centric Cross-network Dataset ,000 registered users in About.me. # User Over 50% users share at least 4 accounts.

15 User-centric Cross-network Dataset

16 User-centric Cross-network Social Multimedia Computing User-centric Cross-network Social Multimedia Computing From Users: On Users: For Users: Cross-network Knowledge Association Mining Mining the correlation based on overlapped users perceptions. Cross-network User Modeling Integrating heterogeneous user data for comprehensive user understanding. Cross-network Collaborated Multimedia Applications Exploring user-centric crossnetwork characteristics to design collaborated solutions. 16

17 User-centric Cross-network Social Multimedia Computing User-centric Cross-network Social Multimedia Computing From Users: On Users: For Users: Cross-network Knowledge v Association Mining Mining the correlation based on overlapped users perceptions. Cross-network User Modeling Integrating heterogeneous user data for comprehensive user understanding. Cross-network Collaborated Multimedia Applications Exploring user-centric crossnetwork characteristics to design collaborated solutions. Ming Yan, Jitao Sang, and Changsheng Xu. Mining Cross-network Association for YouTube Video Promotion. ACM Multimedia,

18 Background: Heterogeneous Knowledge Association Heterogeneous Knowledge Association Cross-network Application UGC behavior, e.g., tweeting history Consuming pattern Targeted advertising Video browsing behavior Following social network Cross-network video promotion 18

19 Challenge: Cross-network Knowledge Gap No explicit association exists between different social media networks. The association is not necessarily semantic-based. Traditional semantic-based solution cannot address all scenarios. A data-driven cross-network association mining solution is needed. 19

20 Motivation: Overlapping User Collaboration Assumption: If abundant users heavily involve with pattern in social media network and pattern in network, it is very likely that pattern and pattern are closely associated. pattern pattern pattern pattern We refer to this associated pattern pairs as crowd-perceptive correlated.

21 Cross-network Knowledge Association Mining YouTube video Twitter followee Social Media Network A Topic Discovery Topics A Overlapping User-based Topic Association Cross-network Application Social Media Network B Topic Discovery Topics B 21

22 Twitter user friend network ( ) Following ACM Multimedia Bill Britney LDA Association Mining ( ) YouTube videos ( ) icorr-lda Aggregation ( ) Twitter user tweet stream Tweets Nic@nicing-Aug 28 #VoteObama, presidential debate Nic@nicing-Aug 28 I like LDA ( ) Association Mining Heterogeneous Topic Modeling Cross-network Topic Association 22

23 Cross-network Topic Association Mining

24 Cross-network Topic Association Mining 1 Transition Probabilitybased Association 2 Regression-based Association 3 Latent Attribute-based Association

25 Cross-network Topic Association Mining 1 Transition Probabilitybased Association = over all the overlapped users = ( ) ( Twitter Distribution Transfer Matrix YouTube Distribution the same overlapped user

26 Cross-network Topic Association Mining 1 Noisy user topic Transition Probabilitybased distributions will deteriorate the derived Association association. 2 Regression-based Association 3 Latent Attribute-based Association

27 Cross-network Topic Association Mining 2 Regression-based Association 1 norm: Lasso problem 2 norm: ridge regression problem min Overlapped user Twitter distribution 2 Overlapped user YouTube distribution 1 or 2

28 Cross-network Topic Association Mining 1 Noisy user topic Transition Probabilitybased distributions will deteriorate the derived Association association. 2 (1) Non-linear association Regression-based is not allowed. (2) Non-overlapped Associationusers are not exploited. 3 Latent Attribute-based Association

29 Cross-network Topic Association Mining 3 Latent Attribute-based Association shared representation coefficients : YouTube distribution Twitter distribution Age gender education location occupation Latent user attributes

30 Cross-network Knowledge Association Mining 3 Latent Attribute-based Association Not only coupled to unique user attributes over the overlapped users, but minimizing the reconstruction error over all the non-overlapped users. min,,,.. 1, 1,, : base vector in latent attribute space; : shared latent user attribute representation. + =, =,, ; =,, =,. =min = 30

31 Experiments: Cross-network Topic Association Quantitatively calculate Mean Absolute Error (MAE) over half of the overlapped users. prediction error over all topics in Twitter topic space = 31

32 Experiments: Association Mining between Twitter Tweet & YouTube Video digital devices US presidential election 32

33 Experiments: Association Mining between Twitter Tweet & YouTube Video horse riding social media marketing beer US presidential debate 33

34 Experiments: Association Mining between Twitter Network & YouTube Video game video semantic game- Visualization of discovered Twitter topics related correlated Berlin popular followees geographical correlated German TV show 34

35 Experiments: Association Mining between Twitter Network & YouTube Video famous actor war & political Australian official account cute animal 35

36 User-centric Cross-network Social Multimedia Computing User-centric Cross-network Social Multimedia Computing From Users: On Users: For Users: Cross-network Knowledge Association Mining Mining the correlation based on overlapped users perceptions. Cross-network User Modeling Integrating heterogeneous user data for comprehensive user understanding. Cross-network Collaborated Multimedia Applications Exploring user-centric crossnetwork characteristics to design collaborated solutions. Zhengyu Deng, Jitao Sang, and Changsheng Xu. Cross-network User Modeling with Local Social Regularization. Submitted for publication. 36

37 Background: User Data are Heterogeneous Heterogeneity is beyond modalities. tagged photo audio photo image tweet geo-tagged video 37

38 Background: User Data are Heterogeneous Heterogeneity is obvious within the same modality. upload favorite playlist rate share comment +1 Complex social interactions aggravate the heterogeneity. 38

39 Motivation: Integrating Heterogeneous Data How to unify different behaviors? Cross-network user behavior quantification upload favorite playlist rate share comment How to integrate social relation with behaviors? Collaborative filtering with local social regularization Latent user space 39

40 Cross-network user behavior quantification upload share playlist comment favorite rate +1 40

41 Cross-network user behavior quantification user favorite matrix upload matrix 1 Users Users 1 Users playlist matrix Users share matrix 1 1 Users Users 1 1 comment matrix Users 1 video 1 rate matrix Users 1 1 Users +1 matrix 1 Users

42 Cross-network user behavior quantification Multiple-kernel learning: Users share matrix 1 Users Users 1 1 favorite matrix Users 1 +1 matrix 1 Users

43 Cross-network user behavior quantification upload matrix 1 Users 1 share matrix User s 1 1 fused matrix favorite matrix Users 1 rate matrix playlist matrix Users Users 1 1 User s 1 +1 matrix 1 User s

44 Collaborative filtering with local social regularization A B C?? 44

45 Collaborative filtering with local social regularization?? 45

46 Collaborative filtering with local social regularization video space latent user space Social interaction on Google+ Derived user latent representation 46

47 Experiments: Evaluation on Video Recommendation +1 47

48 User-centric Cross-network Social Multimedia Computing User-centric Cross-network Social Multimedia Computing From Users: On Users: For Users: Cross-network Knowledge Association Mining Mining the correlation based on overlapped users perceptions. Cross-network User Modeling Integrating heterogeneous user data for comprehensive user understanding. Cross-network Collaborated Multimedia Applications Exploring user-centric crossnetwork characteristics to design collaborated solutions. Zhengyu Deng, Ming Yan, Jitao Sang, Changsheng Xu. Twitter is Faster: Personalized Time-aware Video Recommendation from Twitter to YouTube, TOMCCAP,

49 Challenge: Sparsity in Personalization Long-term interest Application: Real-time Personalized video search /recommendation Key problem: Dynamic Interest Modeling Challenge: Sparse user data in single network,difficult to capture the interest drifting Short-term interest Stable and on general topics, e.g., sports, politics Evolve with time, vulnerable to transient events, e.g., focuses on FIFA World Cup around July,

50 Motivation: Twitter is Faster Twitter has been recognized as an efficient platform for information sharing and spread. Virginia earthquake tweets heat map (08/23/2011) 50

51 Motivation: Twitter is Faster Twitter is faster than many social media services Twitter is faster than Wikipedia. Twitter is faster than Digg. Will this conclusion apply to micro-level? Is the time interval unique for different topics? 51

52 Data Analysis: Statistics The examined 22 trending events. The involved user number for each event. 52

53 Data Analysis: Cross-network Temporal User Behavior Analysis Twitter responses faster than YouTube in macro level Twitter YouTube #Users Wild-Card Round Division Round AFC and NFC championship Super bowl XLVII Elapsed days from Jan.1 st,

54 Data Analysis: Cross-network Temporal User Behavior Analysis Twitter responses faster than YouTube in individual level 54

55 Data Analysis: Cross-network Temporal User Behavior Analysis The cross-network temporal dynamic characteristic is topic-sensitive 55

56 Cross-network Collaborated Video Recommendation Data analysis conclusion: for specific user, his/her short-term interest change emerges first on Twitter Basic idea: exploit the Twitter behavior towards short-term interest modeling User tweets YouTube profile Short-term user modeling Long-term user modeling 56

57 Cross-network Collaborated Video Recommendation Dataset: evaluate on 10 of the 22 trending events. Ground-truth: user s favorite videos on YouTube. Baselines: only considering user interested topics on Twitter, or profiles on YouTube. 57

58 User-centric Cross-network Social Multimedia Computing Cross-network Knowledge Association Cross-network User Modeling Cross-network Collaborated Multimedia Applications Mining the correlation based on overlapped users perceptions. MM 2014 TMM under review Integrating heterogeneous user data for comprehensive user understanding. ICME 2013 TMM under review Exploring user-centric crossnetwork characteristics to design collaborated solutions. ICME 2013 ICIAP 2013, TOMCCAP TKDE under review