Considerations For Defining A Lexicon

Transcription

1 Considerations For Defining A Lexicon Nov J.DeLay CSO/Architect Geospatial harris.com

2 Agenda What is a Lexicon Why have a Lexicon Example 1: NGA Briefed Lexicon Example 2: Mapping Example Example 3: Applied to Analytics Wrap Up and Feedback 11/14/2013 2

3 What is a Lexicon lex i con noun \ˈlek-sə-ˌkän also -kən\ : the words used in a language or by a person or group of peoplep plural lex i ca \-kə\ or lexicons Full Definition of LEXICON 1. A book containing an alphabetical arrangement of the words in a language and their definition ala: dictionary 2. The vocabulary of a language, an individual speaker or group of speakers, or a subject 3. The total stock of morphemes in a language 3

4 Why Have A Lexicon? Machine Language Human Language ABI Lexicon Actionable Intelligence Ad-Hoc Automated Context Driven Question? Derived Intelligence Data Process Big Data Analytics Question? Derived Products & Metadata Existing Databases Data Question? Process Production & Processing Raw Data Data Process WAM MI FMV Image ery Lida ar Sensors HSI MSI GMT TI Open Sour rce 4

5 ABI Lexicon Network Knowledge Object Knowledge Pattern of Pattern of Life Events Movement Change Represent exchange of information with a precise temporal start and stop Transactions Tie Relationship Interaction Dependence Associations Conceptual Representation Objective A conceptual representation of something Object Associations Entity Resolution Durability Proxy Associations Precise Temporal Component Entities Observation Proxies Data Sparsity Physical Cyber Person Unique Object Discreteness Incidental Collection Change In Location Observation Observable Identifier used as a entity substitute Change Of Environment Network Qualifiers Equally Important t 5

6 Mapping to NGA Lexicon NGA Lexicon Event Transaction Harris Graph Model Event (Tweet) Transaction Object Entity Proxy Observation Association Network Person Alias Not specifically represented in the model yet Represented by some of the connecting relationship lines in the model example Represented by sections of the graph example 6

7 Example Connection/Relationship Graph adding Data from Chat Analytics Organization Organization Islamic Brotherhood network Head of Ford Alias (ISR + Date/Time Al Qaeda Known Associate of Is Inside Alias (ISR chat) Pinto City / Capital Is Abuja Person Location of Ben-from-Nigeria Is Tweet Preceded by Alias Is City Inside revolution organizing cell grps /Event Tweet Most Recent Nigeria City Country Is City Inside occupy nigeria start cell grps Lat/Long Coord. Location Tweets mirc chat Geospatial facts Other Date/Time Mustang Alias (ISR chat) Participated in Is Transaction Face-to-Face Mtg Participated in Person Layode Adeoye Is Men ntions (in tweets) # of times Mentions (tweets) # of Alias (Twitter) Was Date/Time Confidence Lagos 7

8 Analytics Based Lexicon Structured Associations Event Super Event Time Un-Structured Conditions To Who Dependent On At When At Where Location Duration How Long ID Name Description Associations Entities Desired Result By Who Effects Require People what Result Sub Objective Resources Qualifiers Un-Lock Intelligence 8

9 Implementing a Lexicon Requires A Well Thoughout Tagging Model A tag assigns a keyword or term to an item or piece of data A tag creates metadata that describes an item so that it can be discovered later Tagging enables new items to be added to existing items both in space and time Tagging aides classification, ownership and rights markings, boundaries, content essence and consumption tracking Tags are extracted automatically or added manually from/to items or other data sources 9 9

10 How Tagging Supports a Lexicon Tagging helps us unlock intelligence associated with the data. Some common techniques include: Building taxonomies: the practice and science of tagging data for the purpose of classification Categorizing: a method for users to share, organize, search, and manage resources Collaborative folksonomies: systems of classifications derived from the practice of collaboratively creating and managing tags to annotate and categorize data Events: a method to associate data with other data using structured or unstructured data models Structured t associations: bi bring order to an otherwise unstructured set of data though the use of a controlled vocabulary ( or Lexicon) 10 10

11 Tagging Supports a Lexicon With All Kinds Of Useful Metadata Content metadata describes the structure such as tables, columns and indexes Essence metadata helps humans find specific items and is usually expressed as a set of keywords in a natural language Technical metadata is metadata about the content Business metadata is metadata that drives external processes Workflow Process metadata includes 3 types of metadata: Descriptive metadata drives the search and discovery of an item (examples: title, author, subject, keyword, publisher) Structural metadata describes how the components of the item are organized Administrative metadata provides technical information such as file type. Two sub-types of administrative metadata are rights management metadata and preservation metadata. Metadata can be stored either internally, in the same file (embedded metadata) as the item, or externally, in a separate file. Metadata is typically stored detached from the data in a data repository. Both have advantages and disadvantages

12 Bottoms Up vs. Top Down Tags are a "bottoms-up" classification which is random in nature There are an unlimited number of ways to classify an item (truck, red truck, lories), and there is no "wrong"" choice. Instead of belonging to one category, an item may have several different tags in time and space. Tags do not adhere to hierarchies, which use "top-down classifications Hierarchical systems typically adhere to a (taxonomy or potentially a Lexicon), which supports a limited number of terms to use for classification, and there is one correct way to classify each item The ideal combines both structured hierarchies and random tagging 12 12

13 Summary Having a Lexicon critical to set the foundations for ABI from a human perspective However the qualifiers around a Lexicon are critical to implement an effective tagging model to support an ABI Lexicon A Lexicon needs to support multiple qualifiers to generate usable intelligence from analytics An open architectural framework is a key enabler to building workflows that fully support a ABI Lexicon 13