IBM Initiate Master Data Service Version 10 Release 0. Hub Overview GI

Transcription

1 IBM Initiate Master Data Serice Version 10 Release 0 Hub Oeriew GI

2

3 IBM Initiate Master Data Serice Version 10 Release 0 Hub Oeriew GI

4 Note Before using this information and the product that it supports, read the information in Notices and trademarks on page 71. Copyright IBM Corporation 1995, US Goernment Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

5 Contents Figures Tables ii Chapter 1. IBM Initiate Master Data Serice IBM Initiate Master Data Serice terms Chapter 2. Components of IBM Initiate Master Data Serice software Master Data Engine Comparison algorithms APIs IBM Initiate Workbench Integration Suite Message Broker Suite Software Deelopment Kits IBM Initiate Web Reports Additional component offerings Chapter 3. Initial configuration and implementation Defining your Hub configuration Hub offerings Initial data load Weight assignment Chapter 4. Key Master Data Engine processes Core data creation and storage Vertical data storage Virtual attributes and sources Identifiers stored for a member Data deriation process Standardized data Bucket data Comparison data Matching process Candidate selection Comparison, scoring, and linking Chapter 5. Linkages and Enterprise ID assignment Linkage types and statuses Entities and linking styles Entity linker and relationship linker process Entity linker Relationship linker Enterprise ID assignment Merging records Chapter 6. Tasks Task categories Identification task types Data issue tasks System trigger tasks Relationship task types Custom task types Task promotion Task creation and life cycle Task status Task resolution flow Reiew Identifier task resolution Potential Linkage task resolution Potential Duplicate task resolution Potential Oerlay task resolution Chapter 7. Attributes Attribute types Attribute creation and life cycle mpi_segattr settings Member mode Rules for attribute updates Chapter 8. Summary Appendix. Feature Implementation Oeriew Adanced Issue Management (Custom Tasks) Understanding Adanced Issue Management.. 55 Oeriew of Adanced Issue Management implementation Householding Understanding the householding feature Oeriew of householding implementation IBM Initiate Master Data Serice and IBM InfoSphere Global Name Recognition integration.. 65 Configuring GNR Disabling GNRMETA Legal Statement Notices and trademarks Index Contacting IBM Copyright IBM Corp. 1995, 2011 iii

6 i Hub Oeriew

7 Figures 1. Data construction in IBM Initiate Master Data Serice software IBM Initiate Master Data Serice matching process Copyright IBM Corp. 1995, 2011

8 i Hub Oeriew

9 Tables 1. Attributes for John Public Vertical creation and storage of member data in the hub database Buckets for John Public Buckets for John Public Candidate selection list John Public comparison results Reiew identifier task records Potential linkage task records Potential duplicate task records Potential oerlay task records com.initiate.serer.task.cfg file Household algorithm definitions IBM resources Proiding feedback to IBM Copyright IBM Corp. 1995, 2011 ii

10 iii Hub Oeriew

11 Chapter 1. IBM Initiate Master Data Serice Whether your business is banking, security, healthcare, or perhaps consumer goods, understanding your customer is the key to your success. With olumes of data stored across multiple source systems and the often dynamic state of that data, organizations are faced with challenging integrity, profiling, and relationship issues. IBM Initiate Master Data Serice enables you to combine, compare, reiew, and resole potential data issues. Using IBM Initiate Master Data Serice adds alue to your organization by increasing data integrity and proiding a 360-degree iew of your customers and the relationships between your customers. IBM Initiate Master Data Serice is designed to store and manage data from multiple source systems. Using a sophisticated and flexible data model and attribute weighting, the probabilistic algorithms identify and link records across these source systems. The IBM Initiate Master Data Serice software receies updated transactions from the source systems. The deployed algorithms identify potential data issues and linkages across the systems that potentially represent the same member or object. The algorithms detect these issues based on arious data elements (member/object attributes specific to your business enironment). The algorithms standardize each data element and compare both similarities and differences to determine the likelihood of a match. As issues are detected, the IBM Initiate Master Data Serice software assigns comparison scores and manages the records based upon threshold settings. This process gies you a complete customer picture. This process does not affect the data stored within your local source systems; record data in the hub is a subset of information aailable from the sources. IBM Initiate Master Data Serice uses statistical and other processes to determine the likelihood that records may pertain to the same party. No process for linking records is perfect and, gien the imperfect nature of statistical analysis, inaccurate input data and other uncertainties, the results of the analysis performed by the IBM Initiate Master Data Serice may contain errors and other problems such as false posities (for example, records identified as possibly pertaining to the same party which actually relate to different parties) and false negaties (for example, separate records that are not identified as pertaining to the same party but actually do relate to the same party). A specific linkage weight does not necessarily hae equialent relatie importance across the comparison of different files. Independent erification of accuracy and completeness of linkage results and decisions based on linkage results produced by the IBM Initiate Master Data Serice is required. There are a number of operations and processes that occur within the IBM Initiate Master Data Serice software to proide an enterprise iew. To assist in telling the story of the IBM Initiate Master Data Serice, we introduce John Public. John Public is a fictional character whose daily experiences with organizations that hae implemented the IBM Initiate Master Data Serice help illustrate how his life is enhanced from a healthcare, financial, and consumer perspectie. John lies in Arizona and works for a diision of ReallyBigCorp. John owns a home, has started a home-based business with his wife, contributes to charity, and plays sports for recreation. Copyright IBM Corp. 1995,

12 IBM Initiate Master Data Serice terms Before diing in-depth to components and processes, it can be helpful to reiew some basic terms. This list is not a comprehensie list of terms; see the IBM Initiate Master Data Serice Glossary for additional terminology. Master Data Engine Hub Attributes Candidate Comparison Comparison Score Enterprise ID EntRecno Entity The core Master Data Engine contains the logic that is at the heart of IBM Initiate Master Data Serice software. Within each Master Data Engine installation, multiple hubs (or instances) can be configured to meet your business requirements. A hub is best described as the collection of data attributes specific to your domain. The hub contains the customized algorithms processed by the Master Data Engine and the rule definitions used by the engine to form entities and relationships between records. Information that describes the member object. For example name, date of birth, address, Social Security or other identifier number, part number, company location and business type, loan or account numbers. When a search request is made, the Master Data Engine returns a list of records that best match the search parameters. These returned records are called candidates. The action of the Master Data Engine applying the algorithm and comparison strategies to determine similarities and differences between two or more records. The score returned for records after a comparison is performed by the software. Scores are based on the defined threshold settings. The higher the score, the more likely that the records represent the same indiidual, organization, or object. The ID number used to represent the same indiidual (or entity) across multiple sources or within the same source. This number is initially assigned by the Master Data Engine, but can be changed by users of IBM Initiate Master Data Serice applications. The database field name for a member Enterprise ID. An entity is defined as something that exists as a particular and discrete unit. In terms of Master Data Management or identity management, an entity is the logical relationship between two or more records. Entities are represented in the Master Data Engine and hub as records sharing an Enterprise ID. An entity can also be called a linkage set. There can be an unlimited number of records in an entity or linkage set. 2 Hub Oeriew

13 Entity Types Glue Member Golden Record Interaction Linkages Linkage Set An entity type proides a distinction between the way members are iewed and linked within the Master Data Engine. Each entity type has a specific algorithm configuration. The following lists examples of how entities can be configured: Identity an indiidual. Records in an identity entity (multiple records representing the same person or object) are linked based on attribute similarity. Household multiple indiidual records who are associated with the same physical location by sharing a common address alue (for example, family members, non-family members sharing a home address). Members of a household entity share a common Enterprise ID, just as members of an identity entity. Organization multiple indiiduals who are associated with a single organization (for example, employees of a company or students at a uniersity) or companies under the same parent company umbrella. Members of an organization entity also share a common Enterprise ID. The term glue member is used to describe the record in a linkage set to which all other records in the set are linked. In other words, the record bringing the linkage set together. For example, if you had the following records: John Public, DOB = , Ph = John Quentin Public, , John Quentin, , Record 1 likely links with Record 2 and Record 3 likely links with Record 2; howeer Record 1 does not likely link with Record 3. Therefore, Record 2 is the glue member. A golden record is found in a transactional hub where the most releant/appropriate representation of the customer/entity is persisted instead of dynamically created. An interaction is best defined as a request from a client application to the Master Data Engine and the result of that request returned to the client. (See IBM Initiate Master Data Serice SDK Reference for Jaa and Web Serices for more details around interactions.) Two or more records grouped by a common Enterprise ID. A linkage set can hae an unlimited number of records. There are two types of linkages. 1) Auto-link, these are records automatically linked and assigned common Enterprise IDs by the Master Data engine. 2) Manual-link, records that are assigned a common Enterprise ID by user action. (In organizations that use hierarchy management, the linkage types can differ. Those types are described in detail in the IBM Initiate Inspector User's Guide and online help.) A grouping of two or more records that hae a high likelihood of representing the same person or object. An unlimited number of records can be included in a linkage set, which can contain records from a single source or multiple sources. Chapter 1. IBM Initiate Master Data Serice 3

14 Member Type Member MemRecno Relationships Source ID Segment SrcRecno SetRecno The IBM Initiate Master Data Serice software is configurable to operate in multiple types of business enironments. The data managed in an organization determines the member type (also called member object ). Member types are associated with algorithms designed to fit the data particulars. Examples of some ready-to-use member types include: Patient, Proider, Consumer, and Organization. Howeer, the flexibility of the software supports easy configuration of new member types or the adjustment of existing member types to fit any business requirement. Depending upon business needs, an organization can potentially implement multiple member types and associated algorithms. A member is defined as a set of demographic information that represents one indiidual (for example, a person), a group of indiiduals (organization), or object (for example, a car part). A member object is the representation of what a single source system asserts to be true about an indiidual or thing. The flexibility of the IBM Initiate Master Data Serice software permits member types/members to be configured for data representing something other than people-related data. For example, the software can be configured for use in equipment manufacturing enironments or product distribution warehouses. For discussion and illustration purposes, this guide uses people member types/members. Each record stored in the hub database is assigned a unique member record number. This number seres as the surrogate primary key for a member. (Not to be confused with SrcRecno.) A Relationship in the IBM Initiate Master Data Serice software is a type of association that can exist between two entities (they can be the same or different entity types). For example, a person can manage another person, or an organization can legally own another organization. The ID number assigned to a record to identify the originating source. Segments are the logical representation of a database table and the contents of that table. When an interaction requests information from the hub database, segments containing the retrieed data are returned by the same interaction. The IBM Initiate Master Data Serice is released with a set of pre-defined segments. Software implementers can add member-attribute implementation-defined segments. Implementation-defined segments are created at the time of implementation and therefore are not associated with an IBM Initiate Master Data Serice generated class. A record ID that identifies a data source. The SrcRecno is assigned by the engine and is used in indiidual records to identify the source from which the record originated. Record number internally assigned to records in a task set. 4 Hub Oeriew

15 Tasks Thresholds Trigger Member or Trigger Record Task Resolution flow Unit of work, created by the Master Data Engine, which facilitates user reiew and resolution. There are three types of tasks managed by the Master Data Engine: predefined tasks that address identity data issues (Potential Linkage, Potential Duplicate, Potential Oerlay, Reiew Identifier), custom tasks (also called implementation-defined tasks) and relationship tasks. Thresholds are scoring leels set by an organization to determine how records are managed by the Master Data Engine. When a task search is requested, the results are generated in real time at the moment the search is initiated. The record, or criteria, that the Master Data Engine uses to compare against other records (candidates) is called the trigger member or trigger record (that is, the record triggering the comparison). When results of a task search are returned, the trigger member typically has a higher score than the other records. The score is higher based on the trigger record being compared against itself during the comparison process. The other records returned are issued comparison scores based upon their comparison against the trigger record. A record is also considered to be the trigger member if an update to that member causes a cross-match that results in task creation. The stages or process in which a task moes from creation to resolution. A task status indicates where a task stands in the resolution process (for example, an unexamined status indicates the task has not been reiewed. A deferred status indicates that the task has been reiewed but not yet resoled) or, for resoled tasks, indicates the resolution determination (for example, same person or not the same person ). Task workflow statuses are often customized for each implementation. Chapter 1. IBM Initiate Master Data Serice 5

16 6 Hub Oeriew

17 Chapter 2. Components of IBM Initiate Master Data Serice software Master Data Engine The IBM Initiate Master Data Serice software is made up of a number of components. Master Data Engine and data model (hub) MPINET serer process Comparison algorithms APIs IBM Initiate Workbench Integration Suite Message Broker Suite IBM Initiate Master Data Serice Software Deelopment Kits IBM Initiate Web Reports The Master Data Engine comprises the database, business rules, and linkage logic (algorithms) that support required functions in modules and applications. The Master Data Engine supports all IBM Initiate Master Data Serice application serice requests. For example the functions aailable in IBM Initiate Workbench, IBM Initiate Inspector, or IBM Initiate Enterprise Viewer all use the Master Data Engine logic. The Master Data Engine is contained within a Jaa process wrapper (not an application serer or serlet engine) that enables greater flexibility of feature deelopment. Through TCP/IP or HTTP socket connections for the arious API calls, the MPINET serer receies messages from the Message Brokers and clients, performs application interactions, and stores the data in the hub database. MPINET is a multi-threaded process and implements database connection pooling to decrease response time. Information about MPINET interactions is written to.mlg log files. The Master Data Engine writes to the database through Open Database Connectiity (ODBC), a standard database access method. The DXI layer is a database-specific abstraction that is included to handle the ariations in accessing databases through ODBC. The IBM Initiate Master Data Serice software supports secure/encrypted communication between the Master Data Engine and clients through Socket Layer Security (SSL). Two-way SSL is the default configuration. For goernment installations, the engine can be configured for FIPS compatibility. For more information about security and enabling SSL and FIPS, see the IBM Initiate Master Data Serice Engine Installation Guide. As well, all authentications to the Master Data Engine are done through an internal Jaa-based directory serer that manages user, group, and permission information. The internal LDAP directory serer is installed during the normal Master Data Engine installation process. The directory serer can be configured to communicate with external corporate directory serers. Copyright IBM Corp. 1995,

18 Within each Master Data Engine installation, multiple hubs can be configured. Encompassed within your hub are: The data attributes specific to your domain The customized algorithms processed by the Master Data Engine Rule definitions used by the Master Data Engine to form entities and relationships between records Comparison algorithms The comparison algorithms proide the heart of IBM Initiate Master Data Serice. Combining specialized routines for data standardization, deriation and comparison, the algorithms enable candidate selection, member identification, and linking of enterprise records. The IBM Initiate Workbench application proides a isual tool for configuring custom algorithms and running queries to understand the effectieness of the configuration. Comparison algorithms The comparison algorithms proide the heart of IBM Initiate Master Data Serice. Combining specialized routines for data standardization, deriation and comparison, the algorithms enable candidate selection, member identification, and linking of enterprise records. The IBM Initiate Workbench application proides a isual tool for configuring custom algorithms and running queries to understand the effectieness of the configuration. APIs The IBM Initiate Master Data Serice APIs permit interaction between the engine database and the clients or other modules. IBM Initiate Workbench The core IBM Initiate Master Data Serice software package uses the APIs to interact with IBM Initiate Workbench, Message Broker Suite components and other applications. Through the use of software deeloper kits (SDKs), customers can deelop internal applications that take adantage of the APIs and engine functions. IBM Initiate Workbench, deliered with the base IBM Initiate Master Data Serice package, is a comprehensie enironment configuration and management application for the Master Data Engine and hub enironment. IBM Initiate Workbench is an Eclipse-based application that offers a number of perspecties from which to configure and manage your enironment. For example: The Configuration perspectie permits you to manage the hub dictionary data and to design and edit algorithms. From this perspectie, you can define your member and entity types, sources, applications, attributes, task threshold leels, relationship rules, algorithms, composite iews, and engine callouts. Job sets are used to run operations like member comparisons or a selection of engine utilities. 8 Hub Oeriew

19 Integration Suite Analytics perspectie enables you to analyze arious aspects of your configuration and ealuate it for errors or potential performance problems. Analysis categories consist of data, entities, buckets, and linkages. Mapping perspectie is used to create and edit data flows. LDAP perspectie supports the management of directory serer user authentication for interaction with the engine. The Cloer ETL function is used to design and run data extract, transfer, and load (ETL) operations in a graph format. Use of this perspectie is licensed through the Master Data Extract component. IBM Initiate Master Data Serice proides tools that enable you to configure the Master Data Engine and other components so that they work best for you. The Integration Suite includes inbound and outbound messaging capabilities and software deelopment kits (SDKs). Message Broker Suite The Message Broker Suite consists of six components. Inbound Brokers Outbound Brokers Mapping Message Manager (Mapping Broker) Routing Message Manager (Routing Broker) HL7 Query Adapter EID Synchronization Inbound Broker The Inbound Broker is a generic interface designed to manage messaging from source systems to the Master Data Engine. Supported message types are extensible markup language (XML), health leel 7 (HL7), fixed-width, or delimited messages. When information about a member is entered or updated in a source system, this information is made aailable to the hub database through the Inbound Broker. The Inbound Broker can communicate with the engine through TCP/IP connection or oer HTTP. The Inbound Broker has access to the Master Data Engine APIs and is made up of two parts: 1) a Message Reader that feeds the 2) Inbound Message Manager. The Inbound Broker receies incoming data through the Message Reader. The main function of the Message is to receie data and acknowledge the receipt of that data. The Message Reader works with the Inbound Message Manager and the APIs to parse incoming data and make any necessary transformations before the data is consumed by the engine. Outbound Broker The Outbound Broker enables the IBM Initiate Master Data Serice software to function as the central management point for all your customer data and keep your systems synchronized with the most accurate, up-to-date information using a single identifier. The Outbound Broker proides message deliery from the Master Data Engine to source systems. The Outbound Broker component has an Outbound Message Chapter 2. Components of IBM Initiate Master Data Serice software 9

20 Manager that sends messages to the Message Sender. The Outbound Broker triggers on specific eent types. These eent types are Has Shadow (indicates a change to a member attribute), Premerge (indicates two member records hae been merged), and Enterprise ID Update tasks. When these eents are newly created, the Outbound process identifies them, builds a customer-specific message, and writes it to outbound queues. The message is then picked up by the Message Sender process and sent to the client (source system) for processing. Note: In instances where a source system is unable to process messages from the Master Data Engine, a custom Outbound Screen Scrape can be implemented. This application uses screen scrape and emulator technology to transfer updates from the Master Data Engine to the source system. Mapping and Routing Message Managers Aside from the standard inbound and outbound messaging processes, the Message Broker Suite supports the configuration of brokers that can handle the mapping and routing of messages. The Mapping Message Manager, or Mapping Broker, is a flexible component that can be used to create outbound messages based on an incoming message. The Mapping Broker supports all four message types: HL7, XML, fixed-width, and delimited. The Routing Message Manager, or Routing Broker, enables the routing of messages from a single queue into multiple queues based on the information contained in the message. The Routing Broker supports all four message formats: HL7, XML, fixed-width, and delimited. Regarding sources, there are two types of source systems with which the IBM Initiate Master Data Serice software interacts: definitional and informational. A definitional source is one in which members (records) are created and usually updated. IBM Initiate Master Data Serice software, through the Outbound Broker, can send updates to a definitional source. Informational sources are thought of as look up data sources, typically proiding legal alues used to identify certain member attributes. For example, an informational source can proide alid credit card numbers or frequent flyer numbers. Member records are not created in informational sources, nor does the IBM Initiate Master Data Serice software send updates to an informational source. HL7 Query Adapter The HL7 Query Adapter supports Health Leel Seen (HL7) Query Response and supports easy integration of the IBM Initiate Master Data Serice software with healthcare enterprise applications. Also called Query Broker, the functions of this component satisfy the need for HL7 eent-drien message requests from customer applications that expect an immediate message-based response. You can configure the Member Get (MemGet) and Member Search (MemSearch) interactions based upon requests from external applications. In addition, response configuration can be customized. Message Broker EID synchronization The Message Broker Suite EID synchronization capabilities send Enterprise ID (EID) update messages from the Master Data Engine to sources systems as changes are made to member data. Configuration is done through the Outbound Broker configuration files. 10 Hub Oeriew

21 Software Deelopment Kits Programmers can use SDKs to deelop solutions that use the searching, scoring, and linking API capabilities of IBM Initiate Master Data Serice software. IBM Initiate Web Reports SDK offerings include Jaa Web Serices IBM Initiate Enterprise Serice Oriented Architecture (ESOA) Toolkit The web-based operational reports shipped with the IBM Initiate Master Data Serice software reports assist you by proiding information about tasks, linkages, and the eents causing them. The aailable reports are diided into the following categories: IBM Initiate Inspector proide you with iews about duplication rates, eents triggering tasks, task resolution and management, linkages, and merges. Operational reports help data managers collect trusted, accurate information on the states of the data issues, their distribution across the stewardship team, and how these issues split up when iewed from a business classification perspectie. IBM Initiate Workbench reports are designed to gie the end user a iew into the Master Data Engine with regard to tasks and eents. Additional component offerings There are a number of components that can be licensed separately from the IBM Initiate Master Data Serice software. IBM Initiate Master Data Serice Composer IBM Initiate Master Data Serice Composer is a unified deelopment enironment for quickly building lightweight but robust applications to display, access, and manipulate the data managed by the engine. For example, a deeloper might quickly assemble components into an application that allows a user to search for an entity, sort a list of results, and iew the relationships to other entities for any item in the list. IBM Initiate Enterprise Viewer IBM Initiate Enterprise Viewer is a web-based application that uses the robust search capabilities of Master Data Engine that assist you in locating member records in the hub. The returned results gie you both an enterprise-wide iew and source-specific iew of a member. Communication and interaction with the Master Data Engine and database occurs oer TCP/IP connection and the Jaa API. For detailed information, see IBM Initiate Enterprise Viewer User's Guide. IBM Initiate Master Data Extract IBM Initiate Master Data Extract uses CloerETL, an open source Extract Transfer Load utility, to extract data from the Master Data Engineto external files for use with reporting and analytical systems. Extracts are designed and run as graphs in CloerETL, and can be either full or incremental. Master Data Extract is accessed Chapter 2. Components of IBM Initiate Master Data Serice software 11

22 through the IBM Initiate Workbench CloerETL perspectie. The Serices team uses these functions to load your data into the hub database. IBM Initiate Inspector The IBM Initiate Inspector web-based client enables users to reiew member records and linkages and to resole tasks. IBM Initiate Inspector uses the Master Data Engine search capabilities to locate records of indiidual members, or multiple records as a linkage entity or task set. The functions include Enterprise and Source ID assignment, attribute edits, manual linkage of records, selection and comparison of records, and the ability to add members to the hub database. IBM Initiate Inspector communicates with the Master Data Engine through TCP/IP or HTTP. Detailed information is proided in the IBM Initiate Inspector User's Guide. IBM Initiate Pair Manager IBM Initiate Pair Manager is a stand-alone tool that reads the sample pair file (.xls) created from the Threshold Analysis Pair Generation job. The Threshold Analysis Pair Generation job supports diiding the generated sample pairs into multiple.xls files. This tool is primarily used during the implementation phase to speed the process of sample-pair ealuation. 12 Hub Oeriew

23 Chapter 3. Initial configuration and implementation During the implementation phase, the serices project team performs numerous actiities with the goal of creating an operational enironment that enables effectie and efficient use of the IBM Initiate Master Data Serice software. These actiities include reiewing your data and organizational processes, and creating and adjusting algorithms and attribute weighting so that the results you see are an accurate reflection of the state of your data. Defining your Hub configuration Your data and what you want to understand about that data determines the manner in which your hub is configured. Some of the items configured include: Member types to be managed (person, organization for example), Data attributes you want stored in the hub database and the sources from where this data is receied, Attributes you want used in record comparison, Cross-source comparison rules (for example, compare Source A to Source B and C, compare Source B to A but not to C), Rules that control relationships between members and entities, And finally, the algorithms that compare your data. Note: There are additional items that are configured (such as task and linkage statuses, or user group permissions). For a better understanding of the configuration of a hub, see the IBM Initiate Workbench User's Guide. Hub offerings The flexibility of IBM Initiate Master Data Serice architecture supports the design of hubs that meet your unique data and business needs. Our extensie experience in data analytics and data issue resolution has enabled the offering of some predefined hub solutions to support certain business enironments. The success reached by organizations implementing a hub is directly attributed to the ability to further customize the predefined hubs. A sampling of the offerings includes: IBM Initiate Master Data Serice Citizen Hub permits local, state, and national goernment organizations to create a comprehensie, secure single iew of citizens. This solution can increase the efficiency and aailability of the serices proided by the agency. For example, state goernments can implement this hub to assist in deliering programs that allow residents to obtain and renew licenses, register businesses, pay taxes, and obtain benefits through a single channel such as the internet. John Public uses the local state internet to renew automobile tags and update contact information with the business registrar for his new home-based business. Haing a complete iew of John Public, the state is able to identify a small business program that can help grow his company. IBM Initiate Master Data Serice Indiidual Hub supports software for business-to-consumer commercial organizations such as finance, insurance, Copyright IBM Corp. 1995,

24 hospitality, retail and high technology. By deliering a comprehensie single iew of indiidual consumers to users and automated business processes, organizations are able to meet their needs of efficiency and customer serice. A hotel chain can use this hub to manage its customers or guests. For instance, John Public is a member of an internet reseration system. When data is receied from a market research firm that sureyed a collection of stays at economically priced hotel chains, the Public family was the recipient of bonus frequent guest points. IBM Initiate Master Data Serice Patient Hub supports healthcare enterprises that deploy an Enterprise Master Person/Patient Index (EMPI). Using this hub, healthcare enterprises proide users and clinical software applications with a comprehensie iew of patient records across multiple sources. This iew enables the deliery of safe, responsie care to patients and increases productiity and cost effectieness for management. When John Public tore his ACL in a basketball game, his full medical picture was aailable to the doctors and facility staff as needed. IBM Initiate Master Data Serice Proider Hub meets the needs for healthcare payer and integrated deliery network companies that manage information for and about healthcare proiders. The information allows companies to manage proider information and understand proider relationships from a single application. For example, John Public's healthcare payer uses this hub to help erify that John's healthcare proiders are authorized to delier the serices described on insurance claims and that the appropriate negotiated payment terms are applied to the claim. IBM Initiate Master Data Serice Organization Hub proides relationship and hierarchy management capabilities to business-to-business commercial enterprises. This hub allows an enterprise to isually understand the relationships and hierarchies between records. This allows the enterprise to better manage the alue, risk profile, and account coerage of complex customers through diisions, departments, and physical locations. For example, John's employer, ReallyBigCorp (a high technology company deeloping and selling enterprise solutions), uses this hub to improe sales resource deployment, effectiely manage commission payments and selling costs, and ultimately delier quality serice to their customers. Hubs configured for relationships permit enterprises to fully understand the relationships between members and records across data domains and within the same domain. Some examples include: IBM Initiate Master Data Serice Patient Hub at IBM Initiate Master Data Serice Proider Hub IBM Initiate Master Data Serice Household Hub John Public is a customer of a large cellular phone company, Cell&Cell. John uses his Cell&Cell phone for business only and has arranged for his employer, ReallyBigCorp, to pay his cellular phone bill directly. Cell&Cell has implemented the Consumer at Organization solution to determine how much business ReallyBigCorp represents. Typically, because John Public is the customer, Cell&Cell would hae a hard time determining how much business it does with ReallyBigCorp. By implementing relationships, Cell&Cell can determine exactly how much business ReallyBigCorp and their subsidiaries represent. Through the householding solution, Cell&Cell is also able to see that both John and his wife hae cell phone serices. 14 Hub Oeriew

25 Initial data load Weight assignment Before the software is first used by your organization, the project team loads extracts of your data into the hub database. During this load, the algorithm is deployed and detects issues based on the configured attributes. A standard set of thresholds and weights are typically used in the initial processing. The algorithm standardizes each data element and compares both similarities and differences to determine the likelihood of a match. If potential issues are detected that compare aboe a set threshold (clerical reiew), the Master Data Engine assigns a comparison score and creates a task. If relationships are detected between records aboe a second threshold (auto-link), linkages are formed. The initial data comparison only generates Potential Duplicate, Potential Linkage, and Reiew Identifier tasks. Potential Oerlay tasks are generated after the system has been implemented and updates from the source are receied by the Master Data Engine. (See Chapter 6, Tasks, on page 35 for a detailed discussion on tasks). After the initial load is complete, the project team re-ealuates the results to determine appropriate threshold leels and weights for your specific data. Attribute weight assignment is an important part of the implementation process. In the early phases of the implementation process, the project team uses analysis tools to understand the intricacies of your data. This analysis considers the data demographics such as data type, ethnic/cultural locations, the frequency of certain data occurrence and data entry error rates. The project team uses the analysis results to define algorithms and to generate weight tables that reflect the exclusie nature of your data. Weight tables are computed by determining: 1. the frequency of occurrence of each attribute alue in the data and, 2. the frequency that attributes differ when a pair of records represents the same member. Attribute weight assignment supports accurate and efficient comparison. A comparison weight assigned to indiidual attributes proides a measure of eidence that two or more records represent the same member. The final score gien when records are compared is the culmination of the indiidual attribute weight scores. Using a standard Person member type implementation as an example, weights are computed and assigned for the attributes of last name, first name, gender, Social Security number, address, phone, birth day, birth month, and birth year. If the algorithm is comparing all of these attributes, why is indiidual weighting important? Because the frequency of an attribute alue has an oerall effect on accurate member determination. Some attribute alues become more important than others. Using the name attribute for example, Smith is considered a fairly common name in the United States. If you randomly compare records based on the last name of Smith, it is possible that two records can compare faorably while not being the same person (John Smith ersus Janna Smith). Howeer, when comparing two records with the less common name of Zukowski, a match on the last name is a stronger indication that the records might be the same person. In this instance, the last name attribute of Smith receies a lower weight (thus a lower Chapter 3. Initial configuration and implementation 15

26 affect on the total comparison score), while last name of Zukowski gets a higher weight. In some areas, howeer, the name Zukowski might be more common and analysis of data frequency might result in the name receiing a lower weight. Gender and Social Security number are two more examples of weight importance. Two records haing the same gender is a common occurrence and proides little eidence of a match. Howeer, if those records hae the same Social Security number which should be unique to an indiidual that proides strong eidence of a match. Therefore, gender receies a lower weight than Social Security number. 16 Hub Oeriew

27 Chapter 4. Key Master Data Engine processes The Master Data Engine receies data in real time or from batch files from indiidual records, internal data sources, business partners, or other sources. Data comes into the Master Data Engine as it exists in the source systems. After the engine instance (hub) has been configured to receie your data, that data begins to moe from your source systems to the engine. During the transfer of data, a number of actions occur to facilitate the comparison and linking process. These actions are part of three core engine processes: 1. Core data creation 2. Deriation process 3. Matching process The following illustration shows the internal actions taken by the software to create and store data in the hub database. Figure 1. Data construction in IBM Initiate Master Data Serice software Core data creation and storage The first stage of the Master Data Engine process is the creation and storage of your data. Data is sent from source systems to the Master Data Engine through an inbound message or API call. The Inbound Broker parses the source data and extracts releant information based on configured attributes. This data is passed to the Master Data Engine where member objects and core data are created. The actions Copyright IBM Corp. 1995,

28 taken with the data depend upon whether the specific data is used in comparison or is additional information about the member. The creation and storage of the data is done in a erticalized process. Vertical data storage The flexibility of the IBM Initiate Master Data Serice data model supports storage of member information in a ertical format using parent-child tables. This structure, rather than storing one row for a member, spreads the data among tables and permits more than one row for member information per table. By employing this data model, maintenance of attribute history and auditing of changed alues is enabled. Consider the following member information: Table 1. Attributes for John Public Attribute Value First name John Middle initial Q Last name Public Address 1043 West Easy Street City Phoenix State Arizona Zip code Phone Home Cell Home Social Security number Date of birth Gender Male address jp@xyz.com Frequent flyer number XYZ Airlines 1450 The storage structure of the data model is such that mpi_memhead, the parent table, stores the primary member identifier (MemRecno). Actual attribute alues are stored ertically across child tables. The ertical storage of data also supports historical tracking of indiidual attributes. The following table illustrates where the known attributes for John Q. Public are being stored in the hub database. Table 2. Vertical creation and storage of member data in the hub database Database table Attribute alue mpi_memhead (memrecno) 104 mpi_memname John Q Public 18 Hub Oeriew

29 Table 2. Vertical creation and storage of member data in the hub database (continued) Database table mpi_memaddr Attribute alue 1043 W Easy St Phoenix Arizona mpi_memident (SSN) 1450 (XYZ Airlines) mpi_memphone mpi_memdate mpi_memattr jp@xyz.com male Virtual attributes and sources When talking about attribute storage, the handling of irtual data must also be considered. A physical storage limitation can affect the decision of an organization regarding the amount and type of data they want to store. Virtual settings can be defined by attribute or by source. The use of irtual attributes enables the control of whether certain member attributes are physically stored in the hub database or only in deried data form. Howeer, as a result of implementing irtual attributes or irtual sources, accurate member comparisons and iews are affected because the full complement of data is unaailable. Note: Do not confuse irtual attributes with external attributes. Virtual attributes are stored in deried data, while external attributes are not stored anywhere in the hub database. See Chapter 7, Attributes, on page 47 for a description of attribute types. Identifiers stored for a member The Master Data Engine uniquely identifies each member so that selecting, comparing, scoring, and linking are done in an accurate manner. The software uses both unique identification keys proided by a source system and a set of keys assigned by the Master Data Engine. These key identifiers are: Source ID (SrcRecno). This number is assigned by the source system from where the record originated and identifies the source to the Master Data Engine. This identification is used by the engine for synchronization and enhances record updates. This identifier is displayed in IBM Initiate Master Data Serice applications. Chapter 4. Key Master Data Engine processes 19

30 Data deriation process Member ID (MemRecno). This number is an internal database identifier assigned to each member record by the Master Data Engine; thus, one member can hae multiple Member IDs. This identifier is displayed in IBM Initiate Inspector. Enterprise ID (EntRecno). Enterprise IDs are assigned by the Master Data Engine to records within an entity (linkage relationship). This identifier is displayed in IBM Initiate Master Data Serice applications. After core member data is created, the deriation process begins. Deriation results in the creation of: 1. Standardized data 2. Bucket data 3. Comparison data Standardized data All information from the source system is passed to the Master Data Engine to create a complete iew of member information (core data). Data (the attribute alues) used for bucketing and comparison goes through a standardization process to generate data chunks. Standardization transforms similar data receied in arious formats to a common format. Normally, this process includes standardizing all alphabetic characters, remoal of punctuation, anonymous alue checks, and data ordering. For example, Source A sends address information for John Public with a street name of West Easy St, while Source B sends it as W. Easy Street. The standardization function or routine can be configured to take all street names with the alue of West and format it as W and Street to St. By standardizing data, comparison accuracy is enhanced because the algorithm is looking at the same alue for that attribute. The standardized format of an attribute is not stored in the database. Using the John Public example, when reiewing data you would still see the addresses with West Easy St. and W. Easy Street, while the standardized address output might look like 1024_W_EASY_ST. Another example of standardization is a phone number. The number can be entered as The standardization routine might strip the dashes and the area code and format the number as During implementation of IBM Initiate Master Data Serice software, the project team identifies the attributes and the specific standardization routines (functions) to employ. Standardization is typically performed on attributes such as name, address, and phone. After being standardized, the data is stored as the comparison data components of the deried data and used in the generation of the bucketing data. Data not configured for use in bucketing and comparison does not participate in standardization; rather it is passed directly to the hub database. Bucket data Bucketing data assists in candidate selection by identifying groups of shared information. 20 Hub Oeriew

31 During initial configuration, buckets are identified for data comparison and each bucket can hae one or more attributes inoled. For example, buckets can be defined for name (first, last, middle), birth date + last name, address, and Social Security number or other identifiers. Any attribute with an anonymous alue is skipped in bucketing. The standardized string is conerted to a format that makes for quick access during candidate selection. Using the example of John Q. Public, the following strings might be generated: Table 3. Buckets for John Public Bucket Standardized string Name JohnQPublic Last name + DOB Public1024 Phone Address + Zip code 1043WEasySt85545 SSN Comparison data Comparison data consists of all the comparable attributes (or data chunks) used to identify a member grouped into a single delimited string. Matching process Both bucket data and comparison data are stored in the database. After the deriation process has completed, the actual matching process begins. Standardization, bucketing, and comparison functions are used by the algorithms to dictate how the engine handles the deried data output to complete the matching process. The matching process uses the deried and comparison data to determine if an incoming record is related to existing records in the hub database, or if an update to an existing record triggers any data issues. After the engine has constructed core member data and deried data, the matching process begins. These stages comprise the matching process. 1. Candidate selection 2. Comparison 3. Scoring 4. Linking The following graphic illustrates this process: Chapter 4. Key Master Data Engine processes 21