JSON-LD and MongoDB. Gregg Kellogg. Powering Linked Web Monday, August 20, 12

Transcription

1 JSON-LD and MongoDB Powering Linked Web Apps Gregg

2 JSON-LD and Mongo JSON-LD graph expression in JSON MongoDB easily query JSON documents Together efficient graph query optimized for client-side processing

3 Use Case WebApps: Single Page Applications (SPAS) Use JSON as native object representation REST access to web service mirroring objects MongoDB document store maintaining equivalent object representations Data is linked, internally and externally (RDF)

4 JSON-based syntax to express

5 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"]

6 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] Context defines terms

7 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] Context defines terms Identifier treated as IRI

8 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] Context defines terms Identifier treated as IRI Type same as rdf:type

9 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] Context defines terms Identifier treated as IRI Type same as rdf:type Data-typed values

10 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] Context defines terms Identifier treated as IRI Type same as rdf:type Data-typed values Simple string values

11 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] Context defines terms Identifier treated as IRI Type same as rdf:type Data-typed values Simple string values (unordered) arrays of values

12 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] Context defines terms Identifier treated as IRI Type same as rdf:type Data-typed values Simple string values (unordered) arrays of values subject reference links to other resources

13 JSON, as expected "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] Context defines terms Identifier treated as IRI Type same as rdf:type Data-typed values Simple string values (unordered) arrays of values subject reference links to other resources Arrays may be ordered

14 JSON-LD context "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "description": "Primary game entry point", "genre": ["First-person shooter" "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " "wikia:platform": ["Xbox 360","PS3","PC","Wii","DS"] "@context": "wikia": " "xsd": " "@vocab": " "@language": "en", "datecreated": "@type": "xsd:datetime", "datemodified": "@type": "xsd:datetime", "event": "@id": "wikia:event", "@container": "@list", "startdate": "@type": "xsd:datetime", "width": "@type": "xsd:integer"

15 Syntactic Conventions Keys represent unique properties Values may be singular or multiple May use strings for most value representations, with typing information maintained in a context.

16 Unique properties Expanded Form: Compact Form: Context: " [ "@id": " e7f ", "@id": " a6e7f18464d000001" ] "publisher": [ " e7f ", " a6e7f18464d000001" ] "@context": "@vocab": " "publisher":"@type": "@id",...

17 Language tagged values Expanded Form: " "Call of Duty: Black Ops", "en" Compact Form: "name": "Call of Duty: Black Ops" Context: " "en",...

18 Typed values Expanded Form: " " ", "xsd:datetime" Compact Form: "datepublished": " " Context: " "xsd": " "xsd:datetime",...

19 Subject references Expanded Form: " [ "@id": " e7f ", "@id": " a6e7f18464d000001" ] Compact Form: Context: " e7f ", " a6e7f18464d000001" ] "@context": "@vocab": " "publisher":"@type": "@id",...

20 Nesting Flattened Form "/contexts/wikia.jsonld", [ "500f1a1b6e7f1827ba000001", "wikia:videogame", " e7f ", " a6e7f18464d000001" ], " e7f ", "name": "Activision", "url": " "schema:corporation", " a6e7f18464d000001", "schema:corporation", "name": "Square Enix (Japan)" ] Nested Form: "@context": "/contexts/wikia.jsonld", "@id": "500f1a1b6e7f1827ba000001", "@type": "wikia:videogame", "@id": " e7f ", "name": "Activision", "url": " "@type": "schema:corporation", "@id": " a6e7f18464d000001", "@type": "schema:corporation", "name": "Square Enix (Japan)" ]

21 Nesting Flattened Form "/contexts/wikia.jsonld", [ "500f1a1b6e7f1827ba000001", "wikia:videogame", " e7f ", " a6e7f18464d000001" ], " e7f ", "name": "Activision", "url": " "schema:corporation", " a6e7f18464d000001", "schema:corporation", "name": "Square Enix (Japan)" ] Nested Form: "@context": "/contexts/wikia.jsonld", "@id": "500f1a1b6e7f1827ba000001", "@type": "wikia:videogame", "@id": " e7f ", "name": "Activision", "url": " "@type": "schema:corporation", "@id": " a6e7f18464d000001", "@type": "schema:corporation", "name": "Square Enix (Japan)" ]

22 Nesting Flattened Form "/contexts/wikia.jsonld", [ "500f1a1b6e7f1827ba000001", "wikia:videogame", " e7f ", " a6e7f18464d000001" ], " e7f ", "name": "Activision", "url": " "schema:corporation", " a6e7f18464d000001", "schema:corporation", "name": "Square Enix (Japan)" ] Nested Form: "@context": "/contexts/wikia.jsonld", "@id": "500f1a1b6e7f1827ba000001", "@type": "wikia:videogame", "@id": " e7f ", "name": "Activision", "url": " "@type": "schema:corporation", "@id": " a6e7f18464d000001", "@type": "schema:corporation", "name": "Square Enix (Japan)" ]

23 In-memory linking Flattened Form Linked Form: "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", "schema:name": "Call of Duty: Black Ops", "wikia:event": [ "@id": "500f28856e7f ", "@id": "500f44556e7f18f7ef000001", "@id": "500f449e6e7f18f94b000001", "@id": "500f44e26e7f18fa " ], "@id": "500f28856e7f ", "type": "cod:mission", "schema:name": "Operation 40", "schema:startdate": " ", "wikia:eventin": [ "@id": "500f1a1b6e7f1827ba000001" 500f1a1b6e7f1827ba type wikia:videogame name "Call of Duty: Black Ops" event 500f28856e7f type cod:mission name "Operation 40" startdate " " event 500f1a1b6e7f1827ba000001

24 Turtle Mapping "/contexts/wikia.jsonld", "500f1a1b6e7f1827ba000001", "wikia:videogame", "contentrating": [ "ESRB: Mature (DS version is Teen)", "PEGI: 18+ (DS version is 16+) ", "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)" "datepublished": " ", "@id": " e7f ", "@id": " a6e7f18464d000001" "wikia:event": [ "@id": "500f28856e7f ",... rdf: schema: wikia: xsd: < <500f1a1b6e7f1827ba000001> a wikia:videogame; schema:contentrating "ESRB: Mature (DS version is Teen)"@en, "PEGI: 18+ (DS version is 16+) "@en, "USK: 18 CERO: Z OFLC: MA15+ (DS version is M)"@en; schema:datepublished: " "^^xsd:dateTime; schema:name "Call of Duty: Black Ops"@en; schema:publisher < e7f >, < a6e7f18464d000001>. wikia:event <500f28856e7f >,...;.

25 More Compact IRIs, Unlabeled Nodes Language maps definitions Named Graphs Property generators *

26 History 2010 Started as a way to project from the RDFa API Developers want solutions to work within HTML applications Desire for JSON-idiomatic way of representing RDF (Linked Data) Adapt existing JSON APIs to Linked Data

27 History 2011 W3C Community Group launched Broad Participation Separation of Syntax from API Proposed to W3C RDF 1.1 Working Group

28 History 2012 RDF WG abandons other JSON serialization efforts (RDF/JSON) JSON-LD accepted as official work item Community Group drafts final report RDF WG publishes JSON-LD (Syntax and API) as FPWD

29 MongoDB JSON-like document store BSON supports extra datatypes Certain key patterns are restricted e.g.,. not allowed in a key Query on value = pattern or value includes pattern

30 Use Case Wikia Structured Data API

31 Wikia Application Express Wiki content as structured data Wiki markup doesn t naturally include semantic cues (Semantic Media Wiki aside). Add semantic markup to HTML generated from Wikis in RDFa Aggregate structured content through JSON-LD/MongoDB powered data service

32 Service Architecture MongoDB persistence JSON document representation JSON-LD describedby link header Ruby/Sinatra web service Content-negotiated access to data JSON-LD, HTML, Turtle,...

33 Document Model Use aliases keywords Simplifies use within client-side MVC frameworks (e.g. backbone.js) "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", " e7f ", " a6e7f18464d000001" "wikia:event": [ "500f28856e7f " ]

34 Document Model Individual subject definition as Mongo document "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", " e7f ", " a6e7f18464d000001" "wikia:event": [ "500f28856e7f " ]

35 Document Model Use expanded subject references This allows simple traversal for graphify "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", "id": " e7f ", "id": " a6e7f18464d000001" "wikia:event": [ "id": "500f28856e7f " ]

36 Document Model Gather subject references under a hidden key This allows easy querying for objects referencing another object "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", "id": " e7f ", "id": " a6e7f18464d000001" "wikia:event": [ "id": "500f28856e7f " "_references": [ " e7f ", " a6e7f18464d000001", "500f28856e7f " ]

37 Document Model GET 500f1a1b6e7f1827ba Return context as HTTP Link Header HTTP/ OK Content-Type: application/json;charset=utf-8 Link: </contexts/wikia.jsonld>; rel="describedby"; type="application/ld+json" "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", "id": " e7f ", "id": " a6e7f18464d000001" "wikia:event": [ "id": "500f28856e7f " ]

38 Restful Endpoints GET withtype="schema:corporation" Accept: application/json index query collection by type return objects having (or containing) a type HTTP/ OK Content-Type: application/json;charset=utf-8 Link: </contexts/wikia.jsonld>; rel="describedby"; type="application/ld+json" [ "id": " e7f ", "type": "schema:corporation", "name": "Activision", "url": " "id": " a6e7f18464d000001", "type": "schema:corporation", "name": "Square Enix (Japan)" ]

39 Restful Endpoints index query collection by referencing return objects referencing another object Build out space around an object Similar to SPARQL describe GET referencing="500f1a1b6e7f1827ba000001" Accept: application/json HTTP/ OK Content-Type: application/json;charset=utf-8 Link: </contexts/wikia.jsonld>; rel="describedby"; type="application/ld+json" [ "id": "500f28856e7f ", "cod:game": "id": "500f1a1b6e7f1827ba000001", "schema:name": "Operation 40", "schema:startdate": " T00:00:00", "wikia:eventin": ["id": "500f1a1b6e7f1827ba000001" "wikia:next": "id": "500f44556e7f18f7ef000001", "wikia:objective": [ "id": " e7f1847c ", "id": "50143e1e6e7f18256d000001",...,...]

40 Restful Endpoints GET 500f1a1b6e7f1827ba Accept: application/json GET return single object HTTP/ OK Content-Type: application/json;charset=utf-8 Link: </contexts/wikia.jsonld>; rel="describedby"; type="application/ld+json" "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", "id": " e7f ", "id": " a6e7f18464d000001" ]

41 Restful Endpoints POST Accept: application/json Content-Type: application/json POST Create new object Good practice to maintain datecreated and datemodified equivalents "type": "wikia:videogame", "name": "Call of Duty: Black Ops" HTTP/ Created Content-Type: application/json;charset=utf-8 Link: </contexts/wikia.jsonld>; rel="describedby"; type="application/ld+json" Location: 500f1a1b6e7f1827ba "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", "datecreated": " T14:00:00-08:00", "datemodified": " T14:00:00-08:00",

42 Restful Endpoints PUT 500f1a1b6e7f1827ba Accept: application/json Content-Type: application/json PUT Update object Good practice to update datemodified equivalent HTTP/ OK Content-Type: application/json;charset=utf-8 Link: </contexts/wikia.jsonld>; rel="describedby"; type="application/ld+json" "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", "id": " e7f ", "id": " a6e7f18464d000001" "datecreated": " T14:00:00-08:00", "datemodified": " T14:15:00-08:00",

43 Restful Endpoints DELETE Destroy object DELETE 500f1a1b6e7f1827ba HTTP/ No Content

44 Restful Endpoints GET 500f1a1b6e7f1827ba000001/publisher/ Accept: application/json path relational query Return objects having a property relationship with another object HTTP/ OK Content-Type: application/json;charset=utf-8 Link: </contexts/wikia.jsonld>; rel="describedby"; type="application/ld+json" [ "id": " e7f ", "type": "schema:corporation", "name": "Activision", "url": " Activision", "id": " a6e7f18464d000001", "type": "schema:corporation", "name": "Square Enix (Japan)" ]

45 Restful Endpoints GET 500f1a1b6e7f1827ba Accept: application/ld+json HTTP/ OK Content-Type: application/ld+json;charset=utf-8 GET as explicit JSON- LD "/contexts/wikia.jsonld", "id": "500f1a1b6e7f1827ba000001", "type": "wikia:videogame", "id": " e7f ", "id": " a6e7f18464d000001" ]

46 Working with data Application Structure Ruby/Sinatra REST service JavaScript/Backbone.js client Alias JSON-LD keywords for convenience "id": "type":

47 MVC Client Natural access from client-side MVC (e.g., backbone.js) Collection // Object collection, handles objects constructed from OWL classes and properties. // Initial fetch is only for VideoGame types, as referenced objects are // fetched, they are added to the collection. Wikia.ObjectCollection = Backbone.Collection.extend( initialize: function() this.model = Wikia.ObjectModel;, url: function() return "/api/#@vers/#@name";, comparator: function(model) return model.get('schema:name')...

48 MVC Client Natural access from client-side MVC (e.g., backbone.js) Model // OWL model class and sub-classes for dealing with owl:class, // owl:datatypeproperty, owl:objectproperty and owl:datatype Wikia.ObjectModel Backbone.Model.extend( referencedobjects: function() return this.resolve(_.flatten(_.toarray(this.attributes)));, getmodel: function(key, options) return this._resolve this.get(key)), setmodel: function(key, value)..., _resolve: function(value)...,...

49 Summary JSON-LD is a light-weight layer for representing linked data in JSON The MongoDB document model is a natural fit for JSON-LD Not optimized for SPARQL; consider alternatives Single Page Applications benefit from linked data principles RDF with OWL can be powerful when used appropriately in the client

50 Resources JSON-LD Syntax JSON-LD API Git Repo Information/Playground IRC #json-ld on Freenode

51 More Information json-ld.org Gregg Kellogg JavaScript Ruby Python PHP Java