Optique Pilot for Oil & Gas and Energy; Statoil

Transcription

1 Scalable End-user Access to Big Data Optique Pilot for Oil & Gas and Energy; Statoil Martin G. Skjæveland and Dag Hovland University of Oslo 1 / 21

2 The Problem of Data Access predefined queries Engineer Application answers 2 / 21

3 The Problem of Data Access information need IT-expert specialised query Engineer Application answers 3 / 21

4 Data Access: The Optique Solution ontology-based query translated query Engineer Application answers 4 / 21

5 Statoil use case Work package in the Optique project Interface between industry and academia Exploration domain Well and stratigraphy data Early phase project prospect evaluation Complexity and variety of Big Data Complex and large data models Many data sources from different vendors Specialised, but partially overlapping content 5 / 21

6 Statoil use case: Stratigraphy Simplified example Chronostrat. Interval C C C Lithostrat. interval L L L Cores interval C C C Content interval Oil Oil Pressure interval P P Wellbores Chrono Litho content W1 C1 L1 & L2 oil W2 C1 AS-IS approach a) Copy from DB into Excel and start analyse b) Ask an IT/DB expert to create a SQL query 8 Security Classificati on: Internal - Status:

7 Statoil use case: Implementation Collected 100 exploration information needs Subsurface exploration domain ontology Mappings towards EPDS, NPD FactPages, OpenWorks, GeoChemDB, CoreDB, Recall, Compass, Petrel Studio, DBR, GoldFinder Parallel query execution in private 8 node cluster ArcGIS and Petrel client tool integration 7 / 21

8 Benefits for Statoil Query across multiple data sources Ad-hoc queries over complex data models Optimised parallel query execution Implementation independent, knowledge driven domain model User-friendly front-end Visual query formulation Statoil FactPages web portal combining data from multiple sources Standardised model and mapping format Web standards compliant interface RDF, SPARQL, OWL, R2RML Communicate with Optique platform from existing client tools 8 / 21

9 Query: Wellbores with cores that overlap log curves 9 / 21

10 Optique Architecture Visualisation & Analysis End-user Query Formulation IT-expert Ontology & Mapping Management Data models Std. ontologies results Queries central Ontology repository Query Transformation Query Planning Mappings Query Execution Query Execution Query Execution streaming data temporal data static data 10 / 21

11 Optique Architecture Visualisation & Analysis End-user Query Formulation IT-expert Ontology & Mapping Management Data models Std. ontologies results Queries central Ontology repository Query Transformation Query Planning Mappings Query Execution Query Execution Query Execution streaming data temporal data static data 10 / 21

12 Intuitive visual query interface 11 / 21

13 Cores overlapping log curves PREFIX ns1: < PREFIX ns2: < SELECT DISTINCT?Wellbore_c2?WKT_a1 WHERE {?Well_c1 ns1:type ns2:well.?wellbore_c2 ns1:type ns2:wellbore.?place_c7 ns1:type ns2:place.?wellboreinterval_c3 ns1:type ns2:wellboreinterval.?wellboreinterval_c4 ns1:type ns2:wellboreinterval.?core_c6 ns1:type ns2:core.?logcurve_c5 ns1:type ns2:logcurve.?well_c1 ns2:haswellbore?wellbore_c2.?well_c1 ns2:locatedin?place_c7.?place_c7 ns2:wkt?wkt_a1.?wellbore_c2 ns2:haswellboreinterval?wellboreinterval_c3.?wellboreinterval_c3 ns2:overlapswellboreinterval?wellboreinterval_c4.?wellboreinterval_c3 ^ns2:extractedfrom?core_c6.?wellboreinterval_c4 ns2:haslogcurve?logcurve_c5. } 12 / 21

14 overlapswellboreinterval select from where and and and LOG.LOG_CURVE_ID CURVE_ID, WELLBORE_INTERVAL.WELLBORE_ID BRONNAVN, WELLBORE_INTERVAL.WELLBORE_INTV_S WELLBORE_INTERVAL_S OPENWORKSBRAGE_LOG_CURVE_HEADER LOG, OPENWORKSBRAGE_WELL_MASTER WM, SLEGGE1_WELLBORE_INTV WELLBORE_INTERVAL WELLBORE_INTERVAL.WELLBORE_ID = WM.WELL_UWI WM.WELL_ID = LOG.WELL_ID LOG.TOP_DEPTH <= WELLBORE_INTERVAL.BOTTOM_DEPTH LOG.BASE_DEPTH >= WELLBORE_INTERVAL.TOP_DEPTH rr:subjectmap rr:template "se:loggedinterval-{curve_id}" rr:predicateobjectmap rr:predicate se:overlapswellboreinterval rr:objectmap rr:template "se:wellboreinterval-{wellbore_interval_s}" 13 / 21

15 SELECT * FROM ( SELECT 1 AS "Wellbore_c2QuestType", NULL AS "Wellbore_c2Lang", UNION SELECT... (' QVIEW2.WELLBORE_S) AS "Wellbore_c2", 3 AS "WKT_a1QuestType", NULL AS "WKT_a1Lang", (((('POINT(' QVIEW5.DATA_VALUE_2_O) ' ') QVIEW5.DATA_VALUE_1_O) ')') AS "WKT_a1" FROM ADP.SLEGGE1_WELLBORE QVIEW2, ADP.SLEGGE1_WELL_SURFACE_PT QVIEW3, ADP.SLEGGE1_WELL QVIEW4, ADP.SLEGGE1_P_LOCATION_2D QVIEW5, ADP.SLEGGE1_WELLBORE_INTV QVIEW6, ADP.RECALL_LOG_VW QVIEW1, ADP.RECALL_LOG_VW QVIEW7, ADP.RECALL_LOG_VW QVIEW8 FROM ADP.SLEGGE1_WELLBORE QVIEW2, ADP.SLEGGE1_WELL_SURFACE_PT QVIEW3, ADP.SLEGGE1_WELL QVIEW4, ADP.SLEGGE1_P_LOCATION_2D QVIEW5, ADP.SLEGGE1_WELLBORE_INTV QVIEW6, ADP.OPENWORKSBRAGE_LOG_CURVE_HEADER QVIEW1, ADP.OPENWORKSBRAGE_WELL_MASTER QVIEW7, ADP.OPENWORKSBRAGE_LOG_CURVE_HEADER QVIEW8 WHERE QVIEW1.MIRROR_ID IS NOT NULL (QVIEW2.R_EXISTENCE_KD_NM = 'actual') QVIEW2.WELL_S IS NOT NULL QVIEW2.WELLBORE_S IS NOT NULL (QVIEW2.WELL_S = QVIEW3.WELL_S) (QVIEW2.WELL_S = QVIEW4.WELL_S) (QVIEW5.DATA_VALUE_1_OU = 'dega') (QVIEW5.DATA_VALUE_2_OU = 'dega') (QVIEW3.WELL_SURFACE_PT_S = QVIEW5.WELL_SURFACE_PT_S) QVIEW5.P_LOCATION_2D_S IS NOT NULL (QVIEW2.WELLBORE_S = QVIEW6.WELLBORE_S) (QVIEW6.R_WELLBORE_INTV_NM = 'cored interval') QVIEW6.WELLBORE_INTV_S IS NOT NULL (QVIEW1.MIRROR_ID = QVIEW7.MIRROR_ID) (QVIEW6.WELLBORE_ID = QVIEW7.WELL_NAME) ((QVIEW7.BOTTOM_DEPTH >= QVIEW6.TOP_DEPTH) (QVIEW7.TOP_DEPTH <= QVIEW6.BOTTOM_DEPTH)) WHERE QVIEW1.LOG_CURVE_ID IS NOT NULL (QVIEW2.R_EXISTENCE_KD_NM = 'actual') QVIEW2.WELL_S IS NOT NULL QVIEW2.WELLBORE_S IS NOT NULL (QVIEW2.WELL_S = QVIEW3.WELL_S) (QVIEW2.WELL_S = QVIEW4.WELL_S) (QVIEW5.DATA_VALUE_1_OU = 'dega') (QVIEW5.DATA_VALUE_2_OU = 'dega') (QVIEW3.WELL_SURFACE_PT_S = QVIEW5.WELL_SURFAC QVIEW5.P_LOCATION_2D_S IS NOT NULL (QVIEW2.WELLBORE_S = QVIEW6.WELLBORE_S) (QVIEW6.R_WELLBORE_INTV_NM = 'cored interval') QVIEW6.WELLBORE_INTV_S IS NOT NULL (QVIEW6.WELLBORE_ID = QVIEW7.WELL_UWI) (QVIEW1.LOG_CURVE_ID = QVIEW8.LOG_CURVE_ID) (QVIEW7.WELL_ID = QVIEW8.WELL_ID) ((QVIEW8.BASE_DEPTH >= QVIEW6.TOP_DEPTH) (QVIEW1.MIRROR_ID = QVIEW8.MIRROR_ID) (QVIEW8.TOP_DEPTH <= QVIEW6.BOTTOM_DEPTH QVIEW5.DATA_VALUE_1_O IS NOT NULL QVIEW5.DATA_VALUE_2_O IS NOT NULL QVIEW5.DATA_VALUE_1_O IS NOT NULL QVIEW5.DATA_VALUE_2_O IS NOT NULL) SUB_QVIEW 14 / 21

16 RESULT UNION PROJECT PROJECT alias27.top_depth <= alias22.bottom_depth alias17.mirror_id = alias24.mirror_id alias27.base_depth >= alias22.top_depth alias23.top_depth <= alias22.bottom_depth alias24.mirror_id IS NOT NULL RECALL_LOG_VW alias24 alias26.well_id = alias27.well_id alias23.bottom_depth >= alias22.top_depth alias25.log_curve_id = alias27.log_curve_id alias22.wellbore_id = alias26.well_uwi alias22.wellbore_id = alias23.well_name OPENWORKSBRAGE_WELL_MASTER alias26 alias27.log_curve_id IS NOT NULL alias25.log_curve_id IS NOT NULL alias18.wellbore_s = alias22.wellbore_s alias17.mirror_id = alias23.mirror_id OPENWORKSBRAGE_LOG_CURVE_HEADER alias27 OPENWORKSBRAGE_LOG_CURVE_HEADER alias25 alias19.well_surface_pt_s = alias21.well_surface_pt_s alias22.r_wellbore_intv_nm = 'cored interval', alias22.wellbore_intv_s IS NOT NULL, alias22.wellbore_s IS NOT NULL alias23.mirror_id IS NOT NULL alias17.mirror_id IS NOT NULL SLEGGE1_WELLBORE_INTV alias22 RECALL_LOG_VW alias23 RECALL_LOG_VW alias17 alias21.data_value_1_o IS NOT NULL, alias21.data_value_1_ou = 'dega', alias21.p_location_2d_s IS NOT NULL, alias21.data_value_2_o IS NOT NULL, alias21.data_value_2_ou = 'dega' alias18.well_s = alias20.well_s SLEGGE1_P_LOCATION_2D alias21 alias20.well_s IS NOT NULL alias18.well_s = alias19.well_s SLEGGE1_WELL alias20 alias19.well_s IS NOT NULL alias18.well_s IS NOT NULL, alias18.wellbore_s IS NOT NULL, alias18.r_existence_kd_nm = 'actual' SLEGGE1_WELL_SURFACE_PT alias19 SLEGGE1_WELLBORE alias18 15 / 21

17 Statoil FactPages Easy access to information about central Statoil assets Requires: Integration Visualisation Web portal Export possibilities Maintenance Wellbores 16 / 21

18 NPD FactPages 17 / 21

19 Why the Optique platform? Ontology-based data access (OBDA) Integration architecture SPARQL queries: end user interface OWL ontology: information model R2RML mappings: federation of multiple data sources Unobtrusive to existing information systems Virtual OBDA Leave data in existing stores Data analysis and computation with R Visualisations with wiki templates Modular and layered approach Using declarative/functional languages 18 / 21

20 Example: Wellbore trajectory 19 / 21

21 Example: Wellbore trajectory SELECT * {?well subexp:haswellbore?? ; subexp:haswellbore [ subexp:haswellborepoint?pt ] ; subexp:hasconvergence [ subexp:valueinstandardunit?conv_deg ] ; subexp:hassloteastwest [ subexp:valueinstandardunit?slot_ew_m ] ; subexp:hasslotnorthsouth [ subexp:valueinstandardunit?slot_ns_m ] ; subexp:haswaterdepth [ subexp:valueinstandardunit?water_depth_m ] ; subexp:hasdatumelevation [ subexp:valueinstandardunit?datum_elev_m ].?pt subexp:hasdepthmeasurement [ subexp:valueinstandardunit?md_m ], [ subexp:valueinstandardunit?tvd_m ] ; subexp:hasnorthoffset [ subexp:valueinstandardunit?north_m ] ; subexp:haseastoffset [ subexp:valueinstandardunit?east_m ]. FILTER(?? =?this_wlb). } 19 / 21

22 Example: Wellbore trajectory SELECT * {?well subexp:haswellbore?? ; subexp:haswellbore [ subexp:haswellborepoint?pt ] ; subexp:hasconvergence [ subexp:valueinstandardunit?conv_deg ] ; subexp:hassloteastwest [ subexp:valueinstandardunit?slot_ew_m ] ; subexp:hasslotnorthsouth [ subexp:valueinstandardunit?slot_ns_m ] ; subexp:haswaterdepth [ subexp:valueinstandardunit?water_depth_m ] ; subexp:hasdatumelevation [ subexp:valueinstandardunit?datum_elev_m ].?pt subexp:hasdepthmeasurement [ subexp:valueinstandardunit?md_m ], [ subexp:valueinstandardunit?tvd_m ] ; subexp:hasnorthoffset [ subexp:valueinstandardunit?north_m ] ; subexp:haseastoffset [ subexp:valueinstandardunit?east_m ]. FILTER(?? =?this_wlb). } north_grid_calc <- function(north_offset, east_offset, conv_deg){ return ((slot_ns + north_offset) * cos (conv_deg * pi / ) + (slot_ew + east_offset) * sin(conv_deg * pi / 1 } east_grid_calc <- function(north_offset, east_offset, conv_deg){ return ( (slot_ew + east_offset) * cos (conv_deg * pi / ) + (slot_ns + north_offset) * sin(conv_deg * pi / } north_grid = north_grid_calc(north_offset, east_offset, res[,'conv_deg']) east_grid = east_grid_calc(north_offset, east_offset, res[,'conv_deg']) northeast_lims = c(min(c(east_grid,north_grid,0)), max(c(east_grid,north_grid,depth_high))) depth_order = order(res[,'wlb'],res[,'md_m']) s3d <- scatterplot3d(east_grid[depth_order], north_grid[depth_order], res[depth_order,'tvd_m'], xlim=northea 19 / 21

23

24

25 Ontology-based data access Ontology Conceptual model of some domain Defines a vocabulary and its semantics using formal logic Supports reasoning tasks: consistency, entailments Mappings Relates the data sources to the ontology Q O Q DB Ontology-based data access (OBDA) Access data via queries over the ontology Virtual approach, using query rewriting All ontology queries are rewritten into SQL queries DL-Lite (OWL2 QL), GAV mappings, UCQs 22 / 21

26 Ontology-based data access Database: wlb_dev(name,...) wlb_exp(name, purpose,...) Query: List all wellbores. Ontology: Field Wellbore DevWellbore ExpWellbore Wildcat Mappings: DevWellbore(name) SELECT name FROM wlb_dev ExpWellbore(name) SELECT name FROM wlb_exp Wildcat(name) SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' 23 / 21

27 Ontology-based data access Database: wlb_dev(name,...) wlb_exp(name, purpose,...) Query: List all wellbores. Ontology: Field Wellbore DevWellbore ExpWellbore Wildcat Mappings: DevWellbore(name) SELECT name FROM wlb_dev ExpWellbore(name) SELECT name FROM wlb_exp Wildcat(name) SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' 24 / 21

28 Ontology-based data access Database: wlb_dev(name,...) wlb_exp(name, purpose,...) Query: List all wellbores. q: Wellbore(x) Ontology: Field Wellbore DevWellbore ExpWellbore Wildcat Mappings: DevWellbore(name) SELECT name FROM wlb_dev ExpWellbore(name) SELECT name FROM wlb_exp Wildcat(name) SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' 25 / 21

29 Ontology-based data access Database: wlb_dev(name,...) wlb_exp(name, purpose,...) Ontology: Field Wellbore Query: List all wellbores. q: Wellbore(x) q O : Wellbore(x) DevWellbore(x) ExpWellbore(x) Wildcat(x) DevWellbore ExpWellbore Wildcat Mappings: DevWellbore(name) SELECT name FROM wlb_dev ExpWellbore(name) SELECT name FROM wlb_exp Wildcat(name) SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' 25 / 21

30 Ontology-based data access Database: wlb_dev(name,...) wlb_exp(name, purpose,...) Ontology: Field Mappings: DevWellbore Wellbore ExpWellbore Wildcat Query: List all wellbores. q: Wellbore(x) q O : Wellbore(x) DevWellbore(x) ExpWellbore(x) Wildcat(x) q SQL : SELECT name FROM wlb_dev UNION SELECT name FROM wlb_exp UNION SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' DevWellbore(name) SELECT name FROM wlb_dev ExpWellbore(name) SELECT name FROM wlb_exp Wildcat(name) SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' 25 / 21

31 Ontology-based data access Database: wlb_dev(name,...) wlb_exp(name, purpose,...) Ontology: Field Mappings: DevWellbore Wellbore ExpWellbore Wildcat Query: List all wellbores. q: Wellbore(x) q O : Wellbore(x) DevWellbore(x) ExpWellbore(x) Wildcat(x) q SQL : SELECT name FROM wlb_dev UNION SELECT name FROM wlb_exp UNION SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' DevWellbore(name) SELECT name FROM wlb_dev ExpWellbore(name) SELECT name FROM wlb_exp Wildcat(name) SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' 26 / 21

32 Ontology-based data access Database: wlb_dev(name,...) wlb_exp(name, purpose,...) Ontology: Field Mappings: DevWellbore Wellbore ExpWellbore Wildcat DevWellbore(name) SELECT name FROM wlb_dev ExpWellbore(name) SELECT name FROM wlb_exp Wildcat(name) SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' Query: List all wellbores. q: Wellbore(x) q O : Wellbore(x) DevWellbore(x) ExpWellbore(x) Wildcat(x) q SQL : SELECT name FROM wlb_dev UNION SELECT name FROM wlb_exp UNION SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' q SQL : SELECT name FROM wlb_dev UNION SELECT name FROM wlb_exp 26 / 21

33 Ontology-based data access Database: wlb_dev(name,...) wlb_exp(name, purpose,...) Ontology: Field Mappings: DevWellbore Wellbore ExpWellbore Wildcat DevWellbore(name) SELECT name FROM wlb_dev ExpWellbore(name) SELECT name FROM wlb_exp Wildcat(name) SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' Query: List all wellbores. q: Wellbore(x) q O : Wellbore(x) DevWellbore(x) ExpWellbore(x) Wildcat(x) q SQL : SELECT name FROM wlb_dev UNION SELECT name FROM wlb_exp UNION SELECT name FROM wlb_exp WHERE purpose = 'WILDCAT' q SQL : SELECT name FROM wlb_dev UNION SELECT name FROM wlb_exp 26 / 21

34 se: < [] a rr:triplesmap ; rr:logicaltable [ a rr:r2rmlview ; rr:sqlquery """ select WELLBORE_ID IDENTIFIER, WELL_ID WELL_IDENTIFIER from SLEGGE_EPI.WELLBORE where R_EXISTENCE_KD_NM = 'actual' """ ] ; rr:subjectmap [ a rr:subjectmap, rr:termmap ; rr:template "se:well-{well_identifier}" ; rr:termtype rr:iri ] ; rr:predicateobjectmap [ a rr:predicateobjectmap ; rr:predicate se:haswellbore ; rr:objectmap [ a rr:objectmap, rr:termmap ; rr:template "se:wellbore-{identifier}" ; rr:termtype rr:iri ] ]. < se:well-{well_identifier}, se:haswellbore, se:wellbore-{identifier} > 27 / 21