A Primer on Converting Analysis Results Data to RDF Data Cubes using Free and Open Source Tools

Transcription

1 TT03 A Primer on Converting Analysis Results Data to RDF Data Cubes using Free and Open Source Tools PhUSE 2014 Tim Williams Principal Statistical Solutions Analyst Global Statistical Sciences UCB BioSciences, Inc.

2 The Semantic Web (circa 2011) 2

3 "I want to take the clinical trials results..." 3 Placebo LowDose HighDose Baseline N=28 N=30 N= Sex F 12 (42.9) 14 (46.7) 16 (55.2) M 16 (57.1) 16 (53.3) 13 (44.8) "..and put them in an RDF Data Cube!" ds:obs1 a qb:observation ; prop:treatment "Plc" ; prop:sex "F" ; prop:statistic "count" ; prop:result "12"^^xsd:double ; qb:dataset ds:dataset-demog. ds:obs2 a qb:observation ;...

4 How to start? Triple 4 Homer Simpson hasspouse Marge Simpson Tribble ts:i7832 ts:firstname Homer ; ts:lastname Simpson ; ts:hasspouse ts:i5628. Turtle ts:i5628 ts:firstname Marge ; ts:lastname Simpson ; Turtle "op": "rdf-extension/save-rdf-schema", "description": "Save RDF schema skeleton", "schema": { "baseuri": " "prefixes": [ { "name": "dccs", "uri": " demog/dccs/" }, { "name": "rdfs", "uri": " }, { "name": "prov", "uri": " },... JSON Jason

5 In Scope Simplified RDF Data Cube Two creation methods (overview) 5 Out of Scope Introduction to Semantic Web, RDF... PhUSE Wiki "PhUSE Semantic Technology Curriculum" Detailed tutorial

6 PhUSE Wiki: Companion Documents 6

7 What is an RDF Data Cube? 7

8 8

9 3 Main Components in the Cube Model 9 Attributes metadata status=final,issued=" t00:00:00"^^xsd:datetime ; Measure (or Primary Measure) the observed value of primary interest count=12 Dimensions value keys or indices that identify the measure treatment="plc", sex="f", statistic="count"

10 10 Treatment Baseline Placebo LowDose HighDose Characteristic N=28 N=30 N= Sex Statistic count F 12 (42.9) 14 (46.7) 16 (55.2) percentage M 16 (57.1) 16 (53.3) 13 (44.8) percentage S e x count F M Plc LowD HighD Treatment

11 count S e x F Plc Treatment Measure Dimensions treatment="plc", It's a hit!! count=12 sex="f", statistic="count"

12 Publisci OpenRefine 12 X X X

13 Publisci Method 13 Map table CSV Ruby Script Baseline Placebo LowDose HighDose Characteristic N=28 N=30 N= Sex Statistic count F 12 (42.9) 14 (46.7) 16 (55.2) percentage M require 16 (57.1) 'publisci' 16 (53.3) 13 (44.8) include PubliSci::DSL Treatment,Sex,Statistic,Result data do Plc,F,count,12 source 'demog3dimsource.csv' Plc,F,percentage,42.9 dimension 'Treatment', 'Sex', 'Statistic' Plc,M,count,16 measure 'Result' Plc,M,percentage,57.1 option :base_url, ' LowD,F,count,14, option 'base', ' LowD,F,percentage,46.7 option 'label_column', 'Statistic' LowD,M,count,16 end LowD,M,percentage, metadata do etc. ns:obs1 a qb:observation dataset 'Demographics ; Analysis Results' qb:dataset ns:dataset-demog3dimsource title 'Demographics' ; rdfs:label "1" ; creator 'Your-Name-Here' prop:treatment <code/treatment/plc> description 'Table example ; for Demographics and Baseline Characteris prop:sex <code/sex/f> date ' T00:00:00' ; prop:statistic <code/statistic/count> end ; prop:result 12 ; open('demog3dim_p.ttl','w'){ file file.write generate_n3}....

14 Publisci Method 14 Advantages Simple, quick, easy Minimal cube knowledge Automatic code list generation Disadvantages Limited support* Harder to extend unless you are a Ruby and Cube expert Not as flexible as OpenRefine RDF Data Cube

15 OpenRefine Method 15 Map table CSV/ XLS Create Project Import Construct Cube Skeleton Components Attributes Dimensions Measure Values Attach Export

16 OpenRefine 16

17 17 Save & Re-use JSON from OpenRefine

18 OpenRefine method 18 Advantages Flexibility in cube design Incremental development Cube components available within interface Data reconciliation Disadvantages Greater cube knowledge required Steep Learning curve Labour-intensive, manual steps Measures in the same cube all receive the same data type Example: count and percentage as xsd:double

19 Query the data with SPARQL 19 PREFIX prop: < demog/prop/> SELECT?value WHERE {?obs prop:treatment "Plc"; prop:sex "F"; prop:statistic "count"; prop:result?value. } SPARQL Protocol and RDF Query Language Where did I go wrong with this child? I blame The Internets, honey.

20 Cube Construction: an Evolution. 20 Publisci My first cube! Codelists OpenRefine Structure and Skeletons Customization Data reconciliation rrdf, rrdflibs Production solution

21 Data Transparency? 21 Semantic Interoperability: "The ability for computer systems to exchange data with unambiguous, shared meaning". - Wikipedia. Metadata embedded with the data Standardization data reconciliation with online vocabularies & thesauri translation between different coding systems and data models Merge data similar and dissimilar sources Machine readable Reasoning, logic, intelligent search

22 Thank you! 22 Acknowledgements Will Strinz - Publisci Author OpenRefine team Ian Fleming, Marc Andersen - PhUSE WG Leads PhUSE WG team members Open Source Movement The Internets Contact: Tim Williams UCB Biosciences, Inc. Raleigh, NC USA tim.williams@ucb.com

23 Copyright & Source Attributions All images are copyrights of their creators and respected owners. LOD Cloud Diagram as of September 2011CC BY-SA 3.0. Anja Jentzch, own work Paramount Pictures My life with Fly Ball dogs / _3c9d6b56be.jpg 23 Hasbro Inc. Ron Leishman. Image illustrations Of.com Davidson University Dept. of Biology Herpetology Lab Research Daily Tombstone Photo MAUSOLEUM OF CHARLES "LUCKY" LUCIANO - St. John's Cemetery, Middle Village, New York Image modifications by TW, Aug 2014

24 Copyright & Source Attributions All images are copyrights of their creators and respected owners. 24 Nickelodeon