Technische Universität Dresden Fakultät Informatik. Wikidata. A Free Collaborative Knowledge Base. Markus Krötzsch TU Dresden.

Transcription

1 Technische Universität Dresden Fakultät Informatik Wikidata A Free Collaborative Knowledge Base Markus Krötzsch TU Dresden IBM June 2015

2 Where is Wikipedia Going? Wikipedia in 2015: A project that has shaped the Web Huge global reach (> 500M unique visitors/month) Stable, reliable, loosing momentum? Criticized on a regular basis

3 Wikipedia's Challenges (selection) Community of Contributors Editing experience Content Size and Quality Maintenance effort Language diversity User engagement Mobile markets Content reuse Integration with external sources

4 Example: Language Diversity There is no one Wikipedia: over 280 language editions 15 languages: 1 Mio+ 40+ languages: 100, languages: 10,000+ Great differences in Size Goals ( What is encyclopaedic? ) Community Coverage Quality

5 English (950,277 points)

6 German (356,904 points)

7 Russian (298,215 points)

8 Italian (249,666 points)

9 Chinese (218,602 points)

10 Japanese (112,344 points)

11 English (950,277 points)

12 Example: Content Reuse Wikipedia as an information cul-de-sac Extremely restricted access paths (main access method: reading lengthy pages of text) Information extraction is hard Question answering is hard Adapting to new contexts is hard Example: What are the world's largest cities with a female mayor?

13 Markus Krötzsch: Wikidata Toolkit Kickoff

14

15 Wikidata Official Wikipedia Database Live at Data used by most Wikimedia projects All 285 language editions of Wikipedia Wikivoyage, Wikiquote, Wikimedia Commons,... Large, active community More than 80K editors so far Among the most active Wikimedia projects by edits

16 Wikidata Development Based on free software Wikibase Ongoing development led by Wikimedia Germany Funded by Wikimedia Foundation Original funding by donations (ai², Google, Moore Foundation, Yandex)

17 Important note All data is entered by volunteers. The community decides what to enter and how. Wikimedia provides infrastructure, not data.

18 Freebase integration Google's open collaborative knowledge base Used directly in Google Knowledge Graph As announced on 16 Dec 2014 by Google: Freebase will be retired by 30 June 2015 soon Data should be migrated to Wikidata where useful Google develops Wikidata import review tool Hoped for: Freebase community joins Wikidata Google sources KG data from Wikidata

19 Data Model

20 The Content of Wikidata

21 Terms and Languages Three kinds of terms: labels, descriptions, aliases Term: string in a language ( monolingual text value ) Over 350 languages Used for labelling and searching Terms as keys: Label-description pair globally unique

22 Site Links Links to other Wikimedia projects Used in all Wikipedias to create language links Site links as keys: At most one link per project (functional) At most one item per site link (inverse functional)

23 Statements The richest part of Wikidata's data Property Value Reference(s)

24 Statements The richest part of Wikidata's data

25 Statements The richest part of Wikidata's data Value Property Rank List of references List of qualifiers Reference = List of propertyvalue pairs

26 Properties and Datatypes Properties have datatypes Datatype fixed after creation Datatypes: Item, Property, String, URL, CommonsMedia, Time, Geo Coordinates, Quantity, Monolingual Text Properties can have statements (since late 2014)

27 Some Statistics

28 Size (22 June 2015) Items: Entities: Statements: with references: Labels: Aliases: Descriptions: Site links: Properties: 17,881, ,596 64,964, ,791 32,439, ,721, ,714 9,343, , ,065, ,833 43,712,591

29 Language information (22 June 2015) Aliases Descriptions Labels

30 Language information (top 20) Aliases Descriptions Labels en de fr es it nl sv ru pl vi war nb ceb pt ja da zh nn

31 Labels vs. Wikipedia Articles Labels Articles Total Labels: 78,721,424 Total Articles: 43,712, en de fr es it nl sv ru pl vi war nb ceb pt ja da zh nn fa

32 English (950,277 points)

33 Wikidata (2,165,028 points)

34 Wikidata and the Semantic Web

35 Semantic Technologies and Wikidata Wikidata is not using RDF internally but it was always clear that it should export it Why? Integration with datasets from other sources Tight connection with linked data web Taking advantage of existing tools In January 2015, the Wikimedia Foundation decided to use RDF and SPARQL as the technological basis for developing a public Wikidata query service.

36 Exporting Wikidata Statements to RDF URIs for items:

37 Classification Properties subclass of (P279) and instance of (P31) P31 is the most used property on Wikidata Often (but not always) used without qualifiers Interesting class hierarchy: Entities used as classes: 182,798 Subclass of: 152,515 (without qualifiers) Instance of: 12,331,092 (without qualifiers) Property hierarchy now also emerging

38 Available RDF Data RDF/OWL file exports at: Dumps of Jan 5, 2015: 506M triples RDF dumps (main serializations) 81M triples simplified statements 12.5M triples unqualified instanceof/subclassof Several third-party SPARQL endpoints: (Virtuoso) (BlazeGraph)

39 Wikidata and DBpedia: A Superficial Comparison Wikidata Data related to Wikipedia Online since late 2012* Manual editing One multilingual dataset Based on statements About 1.5k properties Wikipedia integration Unique community *) influenced by Semantic MediaWiki (started 2005) DBpedia Data related to Wikipedia Started in 2006 Automated extraction One dataset per language Based on triples (RDF) >10k properties Stand-alone dataset Unique community

40 Usage & Applications

41

42 Primary Use Case: Wikipedia

43 Primary Use Case: Wikipedia Infobox

44 Primary Use Case: Wikipedia Look! No inline data!

45 Primary Use Case: Wikipedia Move inline data to Wikidata In progress since 2.5 years, still incomplete Process changes (Example: English Wikipedia PersonData) Populate Infoboxes from Wikidata Only just starting (technical support since May 2015) First examples in production, first community decisions made on how to use this Create list articles (semi)automatically with queries Technology still under development Community discussions have started

46 Other Wikimedia Use Cases Wikidata is used in other projects too WikiQuote WikiVoyage Wikimedia Commons Adoption of Wikibase in other projects planned Wikimedia Commons: manage multimedia data Wiktionary: structured language resource

47 Third-Party Application Areas Labels and descriptions Use as a language resource Identifiers data integration, authority control, linking Data access Embed Wikipedia-related data anywhere Answer complex questions Advanced analytics Data science Analyse Wikipedia community processes

48 Third-party applications: Examples Wikipedia mobile apps

49 Third-party applications: Examples Reasonator (by Magnus Manske)

50 Third-party applications: Examples Wikidata Game (by Magnus Manske)

51 Third-party applications: Examples Mix'n'Match (by Magnus Manske)

52 Third-party applications: Examples Vizidata (by Georg Wild)

53 Third-party applications: Examples Histropedia

54 Getting the Data See Direct access per item (Web API, JSON, RDF, ) Database dumps (JSON) Use Wikidata Toolkit to parse dumps in Java RDF dumps SPARQL endpoints

55 Conclusions Wikidata is developing rapidly Data size Vocabulary size Technical features and community processes A platform for data integration Including links to many other databases Data access is easy, both legally and technically Extended exports and query facilities under way

56 And the Female Mayors? Madrid Ana Botella Warschau Hanna Gronkiewicz-Waltz München Christine Strobl Prag Adriana Krnáčová Valencia Rita Barberá Dresden Helma Orosz Zürich Corine Mauch Alicante Sonia Castedo Resistencia Aída Beatríz Máxima Ayala Gijón Carmen Moriyón Chemnitz Barbara Ludwig

57 Further reading Denny Vrandecic, Markus Krötzsch. Wikidata: A Free Collaborative Knowledge Base. Comm. ACM general first introduction to Wikidata Fredo Erxleben, Michael Günther, Markus Krötzsch, Julian Mendez, Denny Vrandečić. Introducing Wikidata to the Linked Data Web. Proc. Int. Semantic Web Conf. (ISWC) introduction of the Wikidata RDF export and data model