Mastering Enterprise Metadata with Seman2c Modeling

Transcription

1 Unlocking the Power of Seman4c Knowledge Mastering Enterprise Metadata with Seman2c Modeling 1 Enterprise Metadata: The descrip4on of the organiza4onal context processes, roles, policies, products and offerings, etc. that are implicitly part of the enterprise informa4on ecosystem. Seman2c Modeling: A data model linked to the real world through a conceptual model If we combine enterprise metadata with an enterprise seman4c web ini4a4ve, we can create a knowledge fabric that can completely change the way we think of enterprise sopware 1 November 2013 Proprietary and confiden4al informa4on MphasiS

2 Seman2c Model: A data model linked to the real world through a conceptual model Ted We need a legal name for this person: A name registered with the government body within whose legal jurisdic4on we are engaging with this person Cust. ID Name Address Drivers License 2 A person in the real world, who just happens to be playing the role of the customer at this point in 4me Data in & about a record also gives us technical context: which table in which database on which server. When was this record created & by whom The seman4c model is not just part of the metadata, it is the metadata SID: ORCL An address is not just a couple of lines. Is it a residence? Does the customer have in the town / in the country homes? What sort of neighborhood is the house in? The state that issued the license tells us the primary state of domicile of the person. The date of first issue tells us how long the person has been in the state

3 Modern Enterprise IT Environment: Ecosystem of meta driven systems Smart Devices Service Oriented Architecture Message and Data Integra4on Enterprise Content Management Modern Modular UI s Business Rules Engine Data marts and Warehouse Security & En4tlement Business Process Management Accoun4ng Pla_orm Analy4cs Ecosystems Local & Wide Area Networks Customer Rela4onship Management Master & Reference Data Management Enterprise Resource Planning On demand Data Centers & Virtual Desktops Modern development pla_orms are also heavily meta data driven: JEE, Spring,.Net, Cococa & ios, Android One primary role of designers and developers today is as translators between the dialects that these systems speak 3

4 Evolu2on of Metadata Driven Systems Current Generation Next Generation Metadata Driven Applica2on Context Parameter Driven Applica2on Logic R2RML Custom Extensions Logic Parameters Orchestra4on Rules Orchestra4on Rules Parameters Custom Extensions Context Logic Fragments Orchestrator Case specific Process Rule Fragments Dependency Tree Seman4c Model Rule Fragments Parameters Process Model Enterprise Context Logic Fragments Parameters 4

5 The business (problem) space has metadata too Scenario Involves History analyzed through Report Process Segmented into Swim Lane In a Realized as Performed by Belongs to Task Use Case Actor En2tlement Generates Requires Realized as Validated by Metrics Requires / generates Func2on Test Case Behavior driven by Data Business Rule 5

6 Taking this one step further express actual data as graphs as well Real world facts John Smith is a customer of the bank with a large investment porqolio. He is a cau4ous investor with a preference for energy stocks and commodity futures. PB banker is James M John Smith holds posi2ons in Venture Solar, an energy company, currently headquartered in California, USA John Smith currently works in Germany and is subject to German tax laws Venture Solar is corporate customer. Frank K from corporate advisory services is currently helping them put together a reverse merger with Benthik Petroleum Benthik Petroleum is authorized to operate in Syria <a country> (Germany) <a banker> [James M] Works in <LOB> [Private Bank] Tax Jurisdic4on Is advised by Is no4fied of Works with <LOB> [Corporate Advisory Services] John Smith Has name <a human > Public Informa4on <a corporate ac4on> <a banker> [Frank K] Has posi4ons in Operates ini4ates Is advised by plans plans Works in Non public Informa4on <a planned corporate ac4on> <a por_olio account> <a state> [California] Headquartered in <a company> [Venture Solar] Will become subsidiary of <a company> Benthik Petroleum Operates in <a country> (Syria) includes Issued By Ac4vity Area <a security holding> For Security <a security> Energy 6

7 If we put this all together 7 Process Por_olio Review Mobile Device ipad 3 Part of EU Prospect Policy Prospect informa4on cannot be held for more than 3 months Advise ac4on on a posi4on On device Task On app enforces Performed by Mobile Applica4on Applies to Advisor Desktop <a country> (Germany) Works in <LOB> [Private Bank] Customer Service Team Reference Pla_orm Customer Master <a banker> [James M] Personal Banker Tax Jurisdic4on Is advised by Is no4fied of Works with <LOB> [Corporate Advisory Services] John Smith Has name Public Informa4on <a corporate ac4on> Works in CIF Aqribute Legal Name <a human > <a banker> [Frank K] plans Operates Has posi4ons in ini4ates Is advised by plans Non public informa4on <a planned corporate ac4on> Table Column Customer Name <a por_olio account> <a state> [California] Headquartered in Will become subsidiary of Table CIF <a company> [Venture Solar] <a company> Benthik Petroleum Operates in <a country> (Syria) includes Issued By Ac4vity Area Data Pla_orm Db2 on Mainframe <a security holding> For security <a security> Energy

8 We now know Which processes update data in table X? SELECT?process WHERE {?table rdf:type db:table.?app app:canupdate?table.?process proc:hastask?task.? task proc:performedon?app } Which development teams query the customer master table? SELECT?team WHERE {?app app:canupdate <CustomerMaster>.?app app:maintainedby?team } Which func4ons are impacted by an outage of the DB2 mainframe? SELECT?process?func4on WHERE { {?process prod:supports?func4on.?process prod:runson prod:db2_mainframe } UNION?{ process prod:supports?func4on.?func4on prod:dependson prod:db2_mainframe } } All the possible provenance alterna4ves for customer phone numbers Which systems are impacted by this new EU policy? Who are the possible downstream consumers of the field I am capturing on this screen? 8

9 And change the way we write apps... Pick the right connec4on pool paqern based on table proper4es Invoke the right valida4ons based on the region of the customer and region of the user capturing the customer s data Enforce addi4onal valida4ons because of the needs of downstream processes Enforce addi4onal process steps because of the poor controls upstream Look across customers por_olios and iden4fy synergies across customers and foster collabora4ons Based on how open folks have to override an upstream automated decision, add criteria upstream at the decision point 9

10 GeWng There Overview of the Program

11 Enterprise Metadata Model an Ini2al Inventory Processes Actors Tasks Interac4ons 11 Roles Systems Func4onal Inventory Dependencies Inputs & outputs Informa2on + IT Asset Inventory Structured Data Use Cases Reusable modules Organiza2onal Context Products & Departments services Charter KPI s Business domain knowledge Common sense knowledge Opera4onal Analy4cal Dependencies Inputs & outputs Real World Overlap Geography News & events Interfaces Messages Unstructured Data Documents Knowledge Systems Legal Structure Legal Jurisdic4on Legal En4ty Business Regula4ons Opera4onal Regula4ons Transac2onal Systems CRM Core Systems ERP/corporate Process Orchestra2on Front office Back office Ops Rules and Policies Compliance R2RML Risk Regula4on Opera2onal Meta Data KPI s SLA s Taxonomy

12 Program Overview Possible func2onal End State Transac2onal Interac2on Governance / control / oversight Analy2cs Discovery My Task Context Policy Overlay My seman4c model My seman4c model Process Context + Current state Access Layer (message / service / file / Subject area specific ontologies External Ontologies Reference Data Customer Product Org details Transac2onal Systems Account Trades GL Process Orchestra2on Front office Back office Ops Opera2onal Meta Data Process Role Taxonomy Rules and Policies Compliance R2RML Risk Regula4on Eternal Data Sources 12

13 Approach Overview Build a context model Build a context model Process Organiza4on Models Structure Services & Products External context Current state: Ver 1.0 Future state: Ver 2.0 Applica2ons access the seman2c knowledge Applica2ons access the seman2c knowledge Encode the process models BPM Business rules Opera4onal Reports Actors & roles Build a sta2c business ontology Concepts Taxonomy Rela4onships Inferences Reveal data through the ontology + process context Unified View of the Enterprise Internal Data Encoded process Encoded Architecture Ontology Encoded rules External Data 13

14 Phase 1: Integra2on Seman2c Repository 14 Tasks Interrogate and discover business & IT knowledge across the en2re ecosystem Build repository & import meta data Boqom up seman4c modeling Import Business Process Models Import Enterprise Architecture models Import Interface specifica4ons Wrap data sources with SPARQL SPARQL query front end Business Process Models Database Schema Metadata Service Signatures + Data Types Enterprise Architecture Basic Ontologies for each Domain R2RML Map to Ontology Actual Data XMI 3 rd Party tools XSLT XMI JDBC JDBC Document Metadata BPMN DB En4ty Model WSDL / XML Schema / XML / JSON UML Database Tables Document Stores

15 Notes on Process Modeling Volume informa4on for the process shown in diagrams Actor for the use case Driver Attendant Driver Use cases are color coded and number labeled. Arrive at lot Volume: Max: 10 per hour Normal: 1 / 2 per hour Park Plus Keys Take key UC 1: Capture registration Screen Registration Hand over keys Return copy to customer UC2: Print tags SLA: Prints in < 10 seconds Where SLA s are cri4cal, show them Return copy to customer Screen Conformation of tags Tag Location time Serial number Attach copy to keys Tag Location Time Serial number Registration Leave for meetings / appointments Park car in designated location Data tags shown in each element All interac4ons must show data between users and systems Hang up keys in booth 15

16 Building Domain Models Driver Targeted process Arrive at Hand Return Leave for lot over keys copy to meetings / customer appointments Volume: Max: 10 per hour Normal: 1 / 2 per hour Indian Ci4zen Suresh Holds passport Indian Passport No: Z12345 Keys Tag Location time Serial number Driver Park Plus Take key UC 1: Capture registration Screen Registration Return copy to customer UC2: Print tags Screen Conformation of tags SLA: Prints in < 10 seconds Inventory of terms to be modeled Process context: Actor Task Attach copy to keys Tag Location Time Serial number Registration Interac4on Park car in designated location Hang up keys in booth Meta data s meta data Role Importance Interfaces Foreign ci4zen authorized to work in the US Suresh Person Suresh Prime holder on Acme Bank Account No: Is identified by Works at Has statement request US Social security number No: Z12345 US Address 460 Park Ave S, NYC Statement Request _ x _ Send to 16

17 Lessons Learned Phase 1 Avoid deep ontology modeling Deep, complete seman4c models are very difficult to manage in a project context, and do not add significant value in the integra4on phase Avoid shortcuts model the real world Tradi4onal database designs use a variety of short cuts to make real world complexity manageable. Le}ng these propagate into the seman4c model results in an ontology that is specific to the project context, and therefore does not survive well into later phases Avoid abstract classes Without a context to anchor them on, discussions on abstract classes tend to go into free fall, and added liqle to know value Evolve classes & taxonomy from defining characteris2cs Explicitly declared taxonomies proved difficult to reverse engineer in later phases of the project 17

18 Phase 2: Ontology based Inferencing 18 Applica4ons Tasks Interrogate and discover business & IT knowledge across the en4re ecosystem Applica4on logic driven by seman4c models Code generators (Java using Jena API + SPARQL) SPARQL query front end Enrich OWL with inference rules Add inferencing capability to repository Migrate business rules to RIF Encode IT paqerns into RDF + RIF Ontology Based Inferencing Seman4c Repository Business Process Models Database schema meta data Service signatures + data types Enterprise Architecture Enriched ontologies Meta data for code generators (RDF) + design paqerns (RDF + RIF) R2RML Map to ontology Actual data XMI 3 rd Party tools XSLT XMI JDBC JDBC Document meta data BPMN DB En4ty model WSDL / XML Schema / XML / JSON UML Database Tables Document Stores

19 Phase 3: Computer Intelligible Models (just star2ng) 19 Applica4ons Tasks Interrogate and discover business & IT knowledge across the en4re ecosystem Applica4on logic driven by seman4c models Code generators (Java using Jena API + SPARQL) SPARQL query front end Add backward chaining rules engine Add rule cura4on to govern learned rules Ontology Based Inferencing Event history Seman4c Repository Business Process Models Database schema meta data Service signatures + data types Enterprise Architecture Enriched ontologies Business Rules Meta data for code generators + design paqerns R2RML Reasoning Map to ontology Actual data XMI 3 rd Party tools XSLT XMI Rule Cura4on Process OWL JDBC JDBC Document meta data BPMN DB En4ty model WSDL / XML Schema / XML / JSON UML Public Ontolgies Database Tables Document Stores

20 Lessons Learned Phases 2 & 3 If a rule is not easy to define, check the model first Majority of cases where a reasonable rule was proving difficult to implement, the root cause was a poor model of the real world. Fixing the model made rule defini4on easier Retain graph models through all the layers When consuming seman4c models in Java or other non- seman4c languages, we learned to retain the graph models through all applica4on 4ers Majority of developers do not use Java objects as pure logical models of the real world. Instead they use families of classes to enable implementa4on of logic and ensure maintainability Retaining the seman4c model through all 4ers was the only way to retain the Ontology overlays and enable use of business logic in all 4ers Gaps in governance of ontology, rule and data cura4on will kill projects Errors in ontology and rule models are rela4vely painless to fix, they can require people to walk back mul4ple threads of thought, which can be painful 20

21 Elements in the Solu2on Data graphs Quads, not tuples (nothing in the default graph) Reified statements where provenance is important Ontology Encoding OWL 2.0 Tight version control All ontologies must be uploaded to the repository Repositories Single scalable repository for meta data En4tlement enforced at named graph level Federated front end only to merge meta data with actual data Inference levels Inferred rela4onships Inferred taxonomy Seman4c Web Rule Language = declara4ve rules. Rules produce new facts based on exis4ng facts in the model Rule Interchange Format = forward & backward chaining + other business rules variants: given an outcome, can figure out the star4ng configura4on that would lead to this result Interfaces Interac4ve SPARQL endpoint for power users Custom HTML 5.0 screens + canned queries + model naviga4on for occasional users Import + export through RDF / TTL / TRIG format. Convert to RDF / TTL / TRIG using custom Java code 21

22 THANK YOU Presenta2on by Suresh Nair Vice President & Chief Architect, Banking & Cap Markets About MphasiS MphasiS an HP Company is a USD 1 billion global service provider, delivering technology- based solu4ons across industries, including Banking & Capital Markets, Insurance, Manufacturing, Communica4ons, Media & Entertainment, Healthcare & Life Sciences, Transporta4on & Logis4cs, Retail & Consumer Goods, Energy & U4li4es and Governments around the world. MphasiS integrated service offerings in Applica4ons, Infrastructure Services and Business Process Outsourcing help organiza4ons adapt to changing market condi4ons and derive maximum value from IT investments. For more informa4on about MphasiS, log on to