Financial Dataspaces: Challenges, Approaches and Trends

Transcription

1 Financial Dataspaces: Challenges, Approaches and Trends Finance and Economics on the Semantic Web (FEOSW), ESWC 27 th May, 2012 Seán O Riain ebusiness Copyright All rights reserved.

2 Motivation Changing face of data interaction Financial Data spaces Properties / characteristics Interactive Data Challenges Querying / Searching Distributional semantic approach: terms, entities, vocabularies. Multilingual, cross jurisdictional knowledge access & querying - xebr Data Management Auditability / Provenance Analytics, OLAP / Multi-dimensional querying XBRL Comparability Final Remarks

3 Motivation

4 Changing Nature of Data Engagement Move towards large numbers of interrelated data sources dependencies Difficulty in managing in a principles or integrated fashion demands of data every where Big data, dataspaces, mashup s, one report Too much data, good enough answer, pay-as-yougo approach Dataspaces and support systems as a new agenda for data management

5 Data Space Properties Schema-less environments High semantic heterogeneity Open Web/closed datasets Diverse data formats (structured/unstructured) Massive data volume Franklin et al., From Databases to Dataspaces: A New Abstraction for Information Management, ACM SIGMOD Dec 2005

6 Challenges

7 Supporting Interactive Data Querying & Searching Data management

8 Querying & Searching Dataspaces

9 Linked Data Queries HTML CSV What German software companies have English customers with revenue > 59M? Investors / Analyst Need 360 o view, agile query capability Cross company querying Cross taxonomy querying Multi-dimensional querying

10 Current Interactive Approaches SPARQL.. Poor usability Entity-Centric Search Engines... Limited expressivity Visual Query Interfaces... Vocabulary scalability issues

11 The Vocabulary Problem Adapted from Kauffman et al (2009)

12 Query Approach Natural language queries Ranked query results Distributional semantics to allow vocabulary independency Two phase search process combining entity search with spreading activation search

13

14 Between Search Engines and Databases T-Space Vector Space Model for building RDF indexes Graph structure preserved Allows semantic matching /search behavior Embeds a distributional semantic model Distributional semantics is a key element for query-dataset semantic matching Simplified semantic model automatically built from large amounts of unstructured data Distributional hypothesis: words that co-occur is similar contexts tend to have similar meanings

15 Treo Components

16 Vocabulary Search Reuse of existing vocabularies: Which concepts are already out there? Dataset discovery and understanding: What is this dataset about? Search for vocabulary concepts Potential for regulatory compliance and auditability

17 Vocabulary Search Query: [justice] dbpedia-ontology:supremecourtoftheunitedstatescase (score = 0.329) dbpedia-ontology:judge (score = 0.129) dbpedia-ontology:showjudge (score = ) dbpedia-ontology:chiefeditor (score = ) dbpedia-ontology:appointer (score = ) dbpedia-ontology:criminal (score = ) dbpedia-ontology:department (score = ) dbpedia-ontology:retired (score = )

18 Query Trends Schema-independent Ranked list of results (semantic best-effort) Bluring of search and querying for the casual user Distributional semantics for semantic matching Semantics extracted from large text corpora Coexistence of multiple query paradigms Structured queries, entity search, natural language queries

19 Multilingual

20 Multilingual knowledge Access Use Case IFRS Statements listed companies xebr statements registered companies European Business Registry Need Production, enrichment and consumption of financial information at the semantic level, and in the languages of choice of the end users; Ease of data access (cross / multilingual) xebr Interoperability 22 of XYZ

21 International Financial Reporting Standard International Financial Reporting Standards

22 xebr Comparability Business Register Business Register Multiple rules, templates, GAAPs, organizations xebr Core Reference Taxonomy Interoperability + Comparability

23 xebr Core Reference Taxonomy Uses xebr taxonomy and mappings to xebr taxonomy from country taxonomies. xebr Taxonomy Map from Belgian to xebr

24 Monnet xebr Financial Demo 26 of XYZ

25 xebr Proof of Concept (1) Cross-border Storage and Access Filter on contexts Direct access to source documents Prepare and reuse report template

26 xebr Proof of Concept (2) Ratio Analysis

27 Interoperability & Comparability Initiatives DIRECTIVE OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL Amending Directives 89/666/EEC, 2005/56/EC and 2009/101/EC regarding the interconnection of central, commercial and companies registers Enhanced access to economic and legal information relating to European companies Provision of interconnective tools between local registers

28 Data Management and Dataspaces

29 Dataspace Challenges Integrate, repurpose and analyze heterogeneous data? Extract-Transform-Load (ETL) Guarantee data quality and trustworthiness? Provenance Management Integrating structured and unstructured data? Semantic best-effort extraction

30 ETL & Linked Data ETL tools for Linked Data (e.g. Pentaho Data Integration) Scalable data management environment Scheduled processes Interactive Data Transformation Platforms (e.g. Google Refine) Minimizes/eliminates programming efforts in the data transformation process Interaction-based

31 ETL & Linked Data

32 Provenance

33 Provenance Stakeholders Process documentation, reproducibility and reuse Data quality assessment and trustworthiness Consistency-checking, debugging and semantic reconciliation Auditability

34 Provenance Management Data generation applications need to become provenance-aware Generic Provenance Management Framework Provenance beyond scientific workflows Prov4J, Vistrails SDK Capture, representation and consumption Minimization of software adaptations Semantic interoperability across systems Using Linked Data standards

35 ETL Provenance Model Natural isomorphism between ETL and provenance (workflow structure) Provenance interoperability across different ETL platforms Cogs ETL Provenance vocabulary Vocabulary for the description of ETL elements Open Provenance Model complementary layer

36 Provenance Descriptor GRI Example

37 Provenance Descriptor - Sustainability

38 Linked Data Extraction from Text Structured, semi-structured and unstructured data should live together in a dataspace Semantic best-effort data extraction from text Entities, time contexts as semantic pivots Co-reference resolution No vocabulary normalization Not pattern-based

39 Extracted Graphs

40 Trends (Data integration) ETL for Linked Data management ETL facilitates provenance management Interactive data transformation as part of the ETL process Interoperability across ETL workflows Semantic best-effort graph extraction supporting unstructured/structured data integration RDF graphs as a common integration format

41 Analytics

42 Financial Linked Data Huge volume of interesting statistics Standards interoperability, comparable difficult Difficult to integrate XBRL data Need cross entity/financial instrument querying Need cross taxonomy querying (e.g. XBRL from SEC, FFIEC). Require multi-dimensional querying (Aggregation, Slice, Dice) Really require OLAP querying of Financial Linked Data

43 Key Ingredients 1. Modeling XBRL as Linked Data: Multidimensional model, disclosures, revenues, companies -> RDF Data Cube vocabulary 2. Crawling, preprocessing and storage: XBRL linked to Freebase -> data structure definitions, labels 3. OLAP querying: OLAP engine maps OLAP queries to sparql, API allows range of OLAP clients

44 Preliminary Results Preprocessing Using SPARQL CONSTRUCT queries Scalability issues for larger, more connected data sources and with changing information requirements? OLAP queries Basic OLAP queries mapped to SPARQL Scalability issues with OLAP queries for larger more dynamic multidimensional models Next steps XBRL dimensions, rules and calculations Additional OLAP functionality: Dimension hierarchy drill down, what-if-scenarios

45 FIOS, example for Rayonier Sales

46 XBRL Comparability Update

47 Enhance data comparability, targeting richer data analysis Standard API s to more easily and consistently connect with XBRL enabled systems Instrument search, ration analysis, OLAP, multidimensional querying Comparability Also includes some thing like Its as much about pools as instances Consider wider standards community, MG, W3C Cater for Software vendors (commercial and technology enabled), End users (e.g. banks, regulators)

48 Final Remarks

49 Dataspaces Moving from the traditional controlled schema world to a dataspace implies trade off between data consistency and data richness Coping with the dataspace heterogeneity Comprehensive and robust semantic model (distributional data semantics) More interactive systems (querying and integration) Semantic best-effort unstructured data extraction Structured ETL/provenance workflows

50 Interactive Data Agree approach for specification and use of common identifiers or at least their mappings Adhere to common publishing method Reduces integration effort and facilitates data reuse Linked Data principles Convergence between data providers requires coordination and time No need for Big Bang integration Follow a pay-as-you-go iterative approach to integration

51 Interactive Data (Open) Data innovation Europeana - Data Exchange Agreement Comparability Will drive interoperability, taxonomy alignment, URI Commercial Requirement API / OLAP type analytics capability Assertion, what if capability Search / Querying Natural language interfaces

52 References O Riain et al., XBRL and Open Data for Global Financial Ecosystems: A Linked Data Approach International Journal of Accounting Information Systems, 2012 O Riain et al., Linked Data Driven Information Systems as an Enabler for Integrating Financial Data in Information Systems for Global Financial Markets: Emerging Developments and Effects, 2010 Freitas et al., A Distributional Structured Semantic Space for Querying RDF Graph Data. In International Journal of Semantic Computing, 5(4): , Freitas et al., Representing Interoperable Provenance Descriptions for Web-based ETL Workflows. SWPM, ESWC, Monnet - Multilingual Ontologies for Networked Knowledge, EU FP7 Grant No Science Foundation Ireland under Grant No. SFI/08/CE/I1380 (Lion-2)

53 Acknowledgements Prof. Juan Miguel Gomez Berbis, Universidad Carlos III de Madrid, Monnet - Multilingual Ontologies for Networked Knowledge, EU FP7 Grant No Science Foundation Ireland under Grant No. SFI/08/CE/I1380 (Lion-2)