Web Science Your Business, too!

Transcription

1 Web Science & Technologies University of Koblenz Landau, Germany Web Science Your Business, too!

2 Agenda What is Web Science? An explanation by analogy What do we do about it? Understanding collective effects arising from individual behavior Behavior deviating from the model Models of collective administration Why should you watch out?

3 Night sky Flickr, cc Oct 14, 2007, Michael Donough

4 From Cerf, WWW2010 keynote

5 How to approach the Web as an object of scientific investigation?

6 Web Science applied to Online Communities Is my model correct? Data Mining Value Added Motivation Models for community growth and decline New Ethical Norms Political Influence Copyright vs Censorship

7 Bild eines schwarzen Lochs Flickr cc, Jan by thebadastronomer

8 Agenda What is Web Science? What do we do about it? Understanding collective effects arising from individual behavior Behavior deviating from the model Models of collective administration Why should you watch out?

9 Better understanding of the tagging process Cooperative classification of resources Which factors influence the tagging process? Background knowledge of the user? Tag assignments of other users? Hypothesis: Tagging involves imitation of other users AND selection of tags from background knowledge of users.

10 Overall Scheme User interface Something else? Conceptualization Tagging Behavior Own Knowledge Model of User Interface Influence Shared terminology Joint Stochastic Model Comparison of Statistics Simulated Tagging Behavior Model of Own Knowledge Model of Sharing

11 Components of Analysis Properties of Tag Streams Stream view of Folksonomies Co-occurrence streams Resource streams Observations in the real world Dynamic model for Tagging Systems Simulating background knowledge Simulating tag imitation Stochastic models of influence Simulation Results Co-occurrence streams Resource streams Which models best fit the reality?

12 Stream Views of a Folksonomy Folksonomies: Vertices: Users, tags, resources Edges: Tag assignments Postings: Tag assignments of a user to a single resource Can be ordered according to their time-stamp

13 Co-occurrence Streams Co-occurrence Streams: All tags co-occurring with a given tag in a posting Ordered by posting time Co-occurrence stream for 'apple': {mackz, r1, {apple, tree}, 13:25} {klaasd, r2, {apple, mac, ibook}, 13:26} {mackz, r2, {apple, macintosh, stevejobs}, 13:27} tree, mac, ibook, macintosh, stevejobs Tag Y U T R ajax blog xml

14 Properties of Co-occurrence Streams Tag Growth linear growth

15 Properties of Co-occurrence Streams Tag Frequencies power law

16 Resource Streams Resource Streams: All tags assigned to a resource Ordered by posting time Resource stream for 'r2': {mackz, r1, {apple, tree}, 13:25} {klaasd, r2, {apple, mac, ibook}, 13:26} {mackz, r2, {apple, macintosh, stevejobs}, 13:27} apple, mac, ibook, apple, macintosh, stevejobs

17 Properties of Resource Streams Tag Frequencies

18 Properties of Resource Streams Tag Frequencies

19 Web Science & Technologies University of Koblenz Landau, Germany Simulating the Evolution of Tag Streams

20 Simulating tag streams Which of my concepts represent this web page? How do I tag this web page? Inspiration for conceptualization from: 1. Most popular tags 2. Most recently used tags 3. Tags used for this resource 4. Tags co-occuring with similar text documents 5. Creating completely new tags 6. Which combination of inspirations develop the same statistics as the one observed for delicious?

21 The Delicious User Interface Imitating previous tag assignments: Recommended tags: Intersection of tags of a user and tags already assigned to the resource. Your tags: Tags of the user. Popular tags: 7 most popular tags assigned to the resource.

22 Simulating a Tag Stream Start with empty tag stream Each simulation step appends a new tag assignment Simulation of a single tag assignment: P BK p(w t): Probability of selecting word w for topic t. Modeled by word distributions in a topic centered text corpus. 1-P BK n: Number of visible previous tags. h: Maximal number of previous tag assignments used for determining ranking of the n distinct tags.

23 Modeling Background Knowledge Text Corpora Text Corpora Del.icio.us P BK : Probability of selecting from background knowledge p(w t): Probability of selecting word w for topic t. Modeled by word distributions in a topic centered text corpus. p(w r): Probability of selecting word w for resource r.

24 Modeling Tag Imitation P BK t t-1 t-2 t-3 t-4 t-5 t-h 1-P BK n P I = 1 P BK : Probability of imitating a previous tag assignment n: Number of visible top-ranked tags h: Maximal number of previous tag assignments used for determining ranking of the n distinct tags

25 Web Science & Technologies University of Koblenz Landau, Germany Simulation Results

26 Overall Scheme User interface Something else? Conceptualization Tagging Behavior Own Knowledge Model of User Interface Influence Joint Stochastic Model Shared terminology Comparison of Statistics Simulated Tagging Behavior Model of Own Knowledge Model of Sharing

27 Simulating Co-occurrence Streams Tag growth: Influenced by P BK and p(w t) Tag Frequencies: Influenced by P BK, p(w t), n, h n: Semantic breadth of a topic (blog: 100 tags, ajax: 50 tags, xml: 50 tags; Cattuto et al. 2007) h: No hint for realistic values. Good guesses may be 500 and 1000.

28 Co-occ. Streams Simulated Tag Growth

29 Co-occ. Streams Simulated Tag Frequencies

30 Co-occ. Stream Simulated Tag Frequencies

31 Simulating Resource Streams P I and P BK : Values comparable to co-occurrence streams p(w r): Approximated by p(w t) n: 7 tags are visible (cf. Delicious user interface) h: Smaller value than for co-occurrence streams

32 Res. Streams Simulated Tag Frequencies

33 Res. Streams Simulated Tag Frequencies

34 Lessons learned [Dellschaft+Staab, ACM Hypertext 2008] Black holes do not only eat mass they also dissolve by emitting radiation Imitation AND background knowledge are needed for explaining properties of tag streams Probability of imitating previous tag assignments: ~70-90% Frequency Rank Co-occur. Streams Resource Streams Tag Growth Polya Urn Model o o fixed size Simon Model o o linear YS Model w/ Memory + o linear Halpin et al. Model o o linear Epistemic Our Model Model + + power-law

35 Agenda What is Web Science? What do we do about it? Understanding collective effects arising from individual behavior Behavior deviating from the model Models of collective administration Why should you watch out?

36 Solar System Jupiter Saturn Neptun Uranus Flickr, cc Sep by Image Editor

37 Overall Scheme User interface Something else? Conceptualization Tagging Behavior Own Knowledge Model of User Interface Influence Joint Stochastic Model Shared terminology Comparison of Statistics Simulated Tagging Behavior Model of Own Knowledge Model of Sharing

38 What is our Uranus? What is this?

39 Uranus = Spam [Dellschaft+Staab, WebSci 2010] Effect of removing 257 spammers of users from the bookmark stream

40 Why care? The Bibsonomy Example Complete snapshot of Bibsonomy system Manually labeled ground truth of spammers in the data set Users Tags Resources TAS Spammers 29, ,846 1,197,354 13,258,759 Non-Spammers 2,467 61, , ,196

41 Why care? The Delicious Example Crawled during the TAGora Project Users Tags Resources TAS 532,938 2,482,850 18,778, ,305,446 Amount of spammers not known exactly Estimation based on random sample of 500 users: With 95% probability: Between and spammers Delicious most likely already applies spam detection Why care about ~ 1.5% spammers in Delicious?

42 Filtering Results (Users) Number of Spammers and Non-Spammers Spammer Non-Spammer

43 Filtering Results (Tag Assignments) Filtered and unfiltered number of TAS Spam Non-Spam

44 That s why Effect of removing 257 spammers of users from the bookmark stream

45 How statistically significant is the epistemic model for normal users?

46 Lessons learned Uranus was discovered because it affected Neptun Pluto was discovered because it affected Uranus! Spammers can be discovered by their behavior, even if you do not know what kind of spam they are producing!

47 Agenda What is Web Science? What do we do about it? Understanding collective effects arising from individual behavior Behavior deviating from the model Models of collective administration Why should you watch out?

48 Why has the sky the density it has? Flickr, cc Oct 14, 2007, Michael Donough

49 Why do tagging systems have so little spam? [Schwagereit et al.,websci 2010] Administrative Process Content Quality Community Policy User Roles Content Process

50 Yahoo Answers Ensure quality of user generated content Use of administrators and community moderators How? Policy influences community processes

51 Methodology Principle 1. Define a Web Community model (Lycos IQ, Yahoo Answers ) 2. Adapt this model to an existing community 3. Estimate parameters 4. Define quality measure 5. Simulate community behaviour 6. Compare simulation results with real data 7. Analyze quality measures wrt variations of CoSiMo parameters

52 Dataset Lycos IQ Time Period: 909 days Users: Administrators: 36 Questions: Answers: Deleted non-compliant Answers:

53 Observed parameters (input to simulation)

54 Example Behaviors and Example Policies Behaviors of Ordinary Users: Create new postings Read existing postings Report non-compliant postings OR give bonus points to poster Moderator Users: Create new postings Read existing postings Delete non-compliant posting OR give bonus points to poster Administrators: Read existing postings Delete non-compliant postings Reading Policies for Administrators: PA: random selection of postings PB: random selection of postings that no other administrator has examined so far PC: selection of postings that were most often reported by users for being non-compliant Promotion Policy: PM-X : ordinary users become moderators (who can delete postings) when having at least X bonus points

55 How many administrators are needed?

56 Fighting spam with administrators Variation of policies and number of administrators Efficient policies result in high quality content A minimum of 18 administrators are needed Many moderators are needed to bring the quality to a high level

57 Fighting spam with user moderators Variation of policies and posting quality A limited number of administrators has a limited capacity of filtering a surge of non-compliant postings Moderators are helping to increase quality

58 Lessons Learned Strategy of selecting questionable postings is crucial Reporting by normal users is the most effective strategy Moderators are not so effective as expected, if they hunt only incidentally for non-compliant content Sufficiently strong requirements regarding moderator profiles lead to high quality of moderators Policies for promoting users need to be based on a criterion that is time dependent

59 Agenda What is Web Science? What do we do about it? Understanding collective effects arising from individual behavior Behavior deviating from the model Models of collective administration Why should you watch out?

60 SAP Business Partner Use Case SAP Developer Network Posts per day Size of user generated content (posts) Number of users SAP M 4M 10.0 M 1M 1.7M 4.8M

61 ROBUST: IBM Employee Use Case Business Data Created per day Number of users IBM Activities Entry IBM Blogs Entries IBM Communities IBM Bookmarks IBM Wikis NA NA IBM Files NA NA IBM Overall * * *

62 ROBUST: Public domain use case Cloud computing communities & photo communities Based on 1.4 million relevant, up-to-the minute, far-reaching News and Social Media posts every day (46 million monthly) 780,000+ editorially-vetted, spam-free Web sources (Social Media White List refined from 12 million+ sources including Flickr, Twitter, Digg and Reddit) Double-digit monthly growth in source additions 800+ Searchable Industry Categories 100+ Countries and 50+ Languages

63 References K. Dellschaft, S. Staab. An Epistemic Dynamic Model for Tagging Systems. HYPERTEXT 2008, Proceedings of the 19th ACM Conference on Hypertext and Hypermedia, June 19-21, Pittsburgh, Pennsylvania, USA. K. Dellschaft, S. Staab. On Differences in the Tagging Behavior of Spammers and Regular Users. In: Proc. of WebSci-2010, Raleigh, April, F. Schwagereit, S. Sizov, S. Staab. Finding Optimal Policies for Online Communities with CoSiMo. In: Proc. of WebSci- 2010, Raleigh, US, April, 2010.

64 Web Science: A wide open field highly relevant for BIS Is my model correct? Data Mining Value Added Motivation Models for community growth and decline New Ethical Norms Political Influence Copyright vs Censorship

65 WebSci 2011 Koblenz, Germany June 15-17