Semantics In Action For Proactive Policing

Transcription

1 Semantics In Action For Proactive Policing Jen Shorten Technical Delivery Architect, Consulting Services Jon Williams Senior Sales Engineer, UK Public Sector

2 The Nature of Policing Is Changing The increasing availability of information and new technologies offers us huge potential to improve how we protect the public. It sets new expectations about the services we provide. Police IT systems need to adapt to keep up with those changes

3 NATIONAL POLICE OBJECTIVES Digital Transformation of Policing PEOPLE, THINGS EVENTS OUTCOMES Proactive Impact led Outcomes driven Data driven TIME SLIDE: 3

4 WHAT POLICE NEED TO DELIVER THAT VISION A Unified, Actionable 360 View of Data SLIDE: 4

5 THE REALITY Data Is In Silos Data is spread across disconnected databases Data quality issues are significant Data collection is a slow manual process SLIDE: 5

6 Current State SLIDE: 6

7 Traditional RDBMS Solutions ETL SEARCH ETL CRIME RECORDING ETL C2 DOC DOC ETL DOC ETL COTS DOC CHILD PROTECTION C2 INTEL CHILD PROTECTION CRIME RECORDING INTEL FEDERATED SEARCH COTS PRODUCTS SLIDE: 7

8 Federated Search SLIDE: 8

9 TRADITIONAL APPROACH WITH COTS The Promise: Easy ETL, Low Costs POLICE & PARTNER SOURCE SYSTEMS COTS PRODUCT SLIDE: 9

10 TRADITIONAL APPROACH WITH COTS The Reality: Extract, Transform, & Lose POLICE & PARTNER SOURCE SYSTEMS ETL COTS PRODUCT SLIDE: 10

11 Nigel partner suspect Sarah victim Crime THE IDEAL SOLUTION Use All of the Data Semantic linking to see relationships between people, locations, events and objects Wheelchair bound frequent 999 caller Extract context from narrative text Build a complete picture by exploiting the value in all of the data MULTI-MODEL: DOCUMENTS & TRIPLES TOGETHER JSON, XML, & RDF SLIDE: 11

12 Police Intelligence Platform Single point of entry to all force information sources for intelligence and safeguarding 10+ Data Feeds Emergency Calls Crimes Arrests Missing Persons Child Protection Intelligence Mapping & Addresses + MORE INMATE RISK ASSESSMENT INITELLIGENCE/ANALYSIS TOOLS CSE SOCIAL MEDIA VULNERABILITY DETECTION CRIME PATTERNS SEARCH ALERTS GEO WORKFLOW SECURTIY & PERMISSIONS TRANSFORMATION, DISAMBIGUATION & AGGREGATION AUTOMATED INGESTION THE PROJECT Single point of entry to all Police information sources for intelligence and safeguarding 4 applications built on top of a single unified set of data from 10 different police databases 12 weeks of development Data quality issues Disconnected data SLIDE: 12

13 Use Triples To Find Relationships SLIDE: 13

14 Combine Triples With Documents SLIDE: 14

15 Add Geospatial to Triples and Documents SLIDE: 15

16 Statistical Analysis of Linked Data SLIDE: 16

17 DATA PROCESSING Load As Is Export source data Load data as-is as documents XML/JSON Record provenance information PROV-O ontology Harmonize data envelope pattern Canonicalize POLE model <CRIME> <REFERENCE>HW/001900/15</REFERENCE> <CODE>MOPI GROUP 2 - ASSAULT A.B.H</CODE> <CRIMINCIDENT_DATE>08/11/2015</CRIMINCIDENT_DATE> <ADDRESS> <TYPE>SCENE OF CRIME</TYPE> <STREET>8 HAVEN ROAD, EXETER</STREET> <POSTCODE>EX2 8BP</POSTCODE> </ADDRESS> <PERSON> <AUTNPERSONTYPE>VICTIM</AUTNPERSONTYPE> <SURNAME>PHILLIPS</SURNAME> <FORENAME>MILDRED</FORENAME> <DATE_OF_BIRTH>14/01/1927</DATE_OF_BIRTH> <GENDER>F</GENDER> <OCCUPATION>UNEMPLOYED</OCCUPATION> <PERSONALIASLIST></PERSONALIASLIST>... SLIDE: 17

18 EXTRACT ENTITIES Harmonize People Generate a unique ID for the entity instance Harmonize element names and data formats Generate phonetic versions of names <envelope> <person> <uri>646569e5-5f6c c7-09ff84b3e08e</uri> <persontype>victim</persontype> <surname>phillips</surname> <surname_dm>flps</surname_dm> <forename>mildred</forename> <forename_dm>mltrt</forename_dm> <dob> </dob> <gender>f</gender> <occupation>unemployed</occupation>... SLIDE: 18

19 EXTRACT ENTITIES Harmonize Event Generate a unique ID for the entity instance Harmonize element names and data formats <envelope> <event> <uri>police.uk/event/crime/cms2_106600</uri> <eventtype>crime</eventtype> <eventdate> </eventdate>... SLIDE: 19

20 EXTRACT ENTITIES Harmonize Locations Utilize an authoritative reference data source for addresses e.g. Ordnance Survey Record the Unique Property Reference Number (UPRN) <envelope> <event> <uri>police.uk/event/crime/cms2_106600</uri> <eventtype>crime</eventtype> <eventdate> </eventdate> <location>key:3a001d392f0e819e a542391a96aa177e </event> <address> <key>key:3a001d392f0e819e a542391a96aa177e</key> </address>... SLIDE: 20

21 APPLY MATCHING RULES Calculate Hashes Compute hash codes for dimensional combinations that disambiguate the entity: - forename_dm && surname_dm && dob - forename_dm && surname_dm && address Leverage MarkLogic s universal index for resolving the keys <envelope> <person> <uri>646569e5-5f6c c7-09ff84b3e08e</uri> <persontype>victim</persontype> <surname>phillips</surname> <surname_dm>flps</surname_dm> <forename>mildred</forename> <forename_dm>mltrt</sp:forename_dm> <dob> </dob> <gender>f</gender> <occupation>unemployed</occupation> <key>key:21a7e3154a71311e07f277d d0bbd1bf94</key> <key>key:82b93e911bd18e2612fae64d1c81e889b3858f64</key>... SLIDE: 21

22 RECORD RELATIONSHIPS Store Triples Record relationships between entities: <envelope> <triple> <subject>7b9c5fb1-4a49-4d a84408da51c5<subject> <predicate>suspectof</predicate> <object>police.uk/event/crime/cms2_106600</object> </triple> - Person <suspectof> Crime SLIDE: 22

23 RECORD RELATIONSHIPS Store Triples Record relationships between entities: - Person <suspectof> Crime Record relationship between entity instance (i.e Person) and the disambiguation hash code <envelope> <triple> <subject>7b9c5fb1-4a49-4d a84408da51c5<subject> <predicate>suspectof</predicate> <object>police.uk/event/crime/cms2_106600</object> </triple> <triple> <subject>7b9c5fb1-4a49-4d a84408da51c5</subject> <predicate>sameas</predicate> <object>key:21a7e3154a71311e07f277d d0bbd1bf94 </triple> <triple> <subject>7b9c5fb1-4a49-4d a84408da51c5</subject> <predicate>sameas</predicate> <object>key:82b93e911bd18e2612fae64d1c81e889b3858f64 </triple> SLIDE: 23

24 Collapsing Entities Using Semantics Same As Same As Hash Code 1 Same As Person A Person B Hash Code 2 Hash Code N Person C Person X SLIDE: 24

25 Advantages of the Multi-Model Approach Fast - Fast search - limits joins as entities are documents, relationships are triples - Fast ingest - disambiguation effort is performed at query time - Fast disambiguation - sem:transitive-closure operation is very quick Flexible - Query time disambiguation allows rules to be changed or applied on a per user basis - Use different predicates for use-case sensitive deduplication and different degrees of confidence SLIDE: 25

26 TWO OPTIONS Multi-Vendor Approach vs. MarkLogic Approach MULTI-PRODUCT, MULTI-DATABASE STACK MARKLOGIC OPERATIONAL DATA HUB SLIDE: 26

27 Multi-Product, Multi-Database Stack Processing and Storage Search and Query Security and Audit Connection management Transaction handler In-memory Joins Data Marshalling & Unmarshalling Bespoke audit code Bespoke access controls Exception Handling Version Control Federated Search Query Optimisation Bespoke data constraints Data integrity management JSON Conversion ETL Cache Alert Reporting Data Governance Database/Search/Graph Information Synchronization Relational or Non-Relational Database Search Indexes Graph Database SLIDE: 27

28 OUR SOLUTION The Operational Data Hub SLIDE: 28

29 Benefits of a Document + Triple Store All the benefits of each, plus: Docs can contain triples, Triples can annotate docs, Graphs can contain docs DATA DOCUMENTS SEMANTICS Faster data integration using semantics as the glue Ideal model for reference data, metadata, provenance Ability to run really powerful queries Massive speed and scale Simplicity of a single unified platform Enterprise features (security, HA/DR, ACID transactions, ) SLIDE: 29

30 Why MarkLogic? True Multi-model Database For All of Your Data Enterprise-Ready Agility and Faster Time to Results Simplified Architecture SLIDE: 30

31 Q&A