Informatica Data Quality for Siebel (Version HotFix 2) User Guide

Transcription

1 Informatica Data Quality for Siebel (Version HotFix 2) User Guide

2 Informatica Data Quality for Siebel User Guide Version HotFix 2 August 2011 Copyright (c) Informatica. All rights reserved. This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. This Software may be protected by U.S. and/or international Patents and other Patents Pending. Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as provided in DFARS (a) and (a) (1995), DFARS (1)(ii) (OCT 1988), FAR (a) (1995), FAR , or FAR (ALT III), as applicable. The information in this product or documentation is subject to change without notice. If you find any problems in this product or documentation, please report them to us in writing. Informatica, Informatica Platform, Informatica Data Services, PowerCenter, PowerCenterRT, PowerCenter Connect, PowerCenter Data Analyzer, PowerExchange, PowerMart, Metadata Manager, Informatica Data Quality, Informatica Data Explorer, Informatica B2B Data Transformation, Informatica B2B Data Exchange Informatica On Demand, Informatica Identity Resolution, Informatica Application Information Lifecycle Management, Informatica Complex Event Processing, Ultra Messaging and Informatica Master Data Management are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners. Portions of this software and/or documentation are subject to copyright held by third parties, including without limitation: Copyright DataDirect Technologies. All rights reserved. Copyright Sun Microsystems. All rights reserved. Copyright RSA Security Inc. All Rights Reserved. Copyright Ordinal Technology Corp. All rights reserved.copyright Aandacht c.v. All rights reserved. Copyright Genivia, Inc. All rights reserved. Copyright Isomorphic Software. All rights reserved. Copyright Meta Integration Technology, Inc. All rights reserved. Copyright Intalio. All rights reserved. Copyright Oracle. All rights reserved. Copyright Adobe Systems Incorporated. All rights reserved. Copyright DataArt, Inc. All rights reserved. Copyright ComponentSource. All rights reserved. Copyright Microsoft Corporation. All rights reserved. Copyright Rogue Wave Software, Inc. All rights reserved. Copyright Teradata Corporation. All rights reserved. Copyright Yahoo! Inc. All rights reserved. Copyright Glyph & Cog, LLC. All rights reserved. Copyright Thinkmap, Inc. All rights reserved. Copyright Clearpace Software Limited. All rights reserved. Copyright Information Builders, Inc. All rights reserved. Copyright OSS Nokalva, Inc. All rights reserved. Copyright Edifecs, Inc. All rights reserved. Copyright Cleo Communications, Inc. All rights reserved. Copyright International Organization for Standardization All rights reserved. Copyright ej-technologies GmbH. All rights reserved. Copyright Jaspersoft Corporation. All rights reserved. This product includes software developed by the Apache Software Foundation ( and other software which is licensed under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. This product includes software which was developed by Mozilla ( software copyright The JBoss Group, LLC, all rights reserved; software copyright by Bruno Lowagie and Paulo Soares and other software which is licensed under the GNU Lesser General Public License Agreement, which may be found at The materials are provided free of charge by Informatica, "as-is", without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. The product includes ACE(TM) and TAO(TM) software copyrighted by Douglas C. Schmidt and his research group at Washington University, University of California, Irvine, and Vanderbilt University, Copyright , all rights reserved. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (copyright The OpenSSL Project. All Rights Reserved) and redistribution of this software is subject to terms available at and This product includes Curl software which is Copyright , Daniel Stenberg, <daniel@haxx.se>. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. The product includes software copyright ( ) MetaStuff, Ltd. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at license.html. The product includes software copyright , The Dojo Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at This product includes ICU software which is copyright International Business Machines Corporation and others. All rights reserved. Permissions and limitations regarding this software are subject to terms available at This product includes software copyright Per Bothner. All rights reserved. Your right to use such materials is set forth in the license which may be found at kawa/software-license.html. This product includes OSSP UUID software which is Copyright 2002 Ralf S. Engelschall, Copyright 2002 The OSSP Project Copyright 2002 Cable & Wireless Deutschland. Permissions and limitations regarding this software are subject to terms available at This product includes software developed by Boost ( or under the Boost software license. Permissions and limitations regarding this software are subject to terms available at / This product includes software copyright University of Cambridge. Permissions and limitations regarding this software are subject to terms available at This product includes software copyright 2007 The Eclipse Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at This product includes software licensed under the terms at license.html, license.html, ; license.html; nanoxml.sourceforge.net/orig/copyright.html; and license.html.

3 This product includes software licensed under the Academic Free License ( the Common Development and Distribution License ( ) the Common Public License ( ), the Sun Binary Code License Agreement Supplemental License Terms, the BSD License ( the MIT License ( and the Artistic License ( This product includes software copyright Joe WaInes, XStream Committers. All rights reserved. Permissions and limitations regarding this software are subject to terms available at This product includes software developed by the Indiana University Extreme! Lab. For further information please visit This product contains runtime modules of IBM DB2 Driver for JDBC and SQLJ (c) Copyright IBM Corporation 2006 All rights reserved. This Software is protected by U.S. Patent Numbers 5,794,246; 6,014,670; 6,016,501; 6,029,178; 6,032,158; 6,035,307; 6,044,374; 6,092,086; 6,208,990; 6,339,775; 6,640,226; 6,789,096; 6,820,077; 6,823,373; 6,850,947; 6,895,471; 7,117,215; 7,162,643; 7,254,590; 7,281,001; 7,421,458; 7,496,588; 7,523,121; 7,584,422, 7,720,842; 7,721,270; and 7,774,791, international Patents and other Patents Pending. DISCLAIMER: Informatica Corporation provides this documentation "as is" without warranty of any kind, either express or implied, including, but not limited to, the implied warranties of noninfringement, merchantability, or use for a particular purpose. Informatica Corporation does not warrant that this software or documentation is error free. The information provided in this software or documentation may include technical inaccuracies or typographical errors. The information in this software and documentation is subject to change at any time without notice. NOTICES This Informatica product (the "Software") includes certain drivers (the "DataDirect Drivers") from DataDirect Technologies, an operating company of Progress Software Corporation ("DataDirect") which are subject to the following terms and conditions: 1. THE DATADIRECT DRIVERS ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. 2. IN NO EVENT WILL DATADIRECT OR ITS THIRD PARTY SUPPLIERS BE LIABLE TO THE END-USER CUSTOMER FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR OTHER DAMAGES ARISING OUT OF THE USE OF THE ODBC DRIVERS, WHETHER OR NOT INFORMED OF THE POSSIBILITIES OF DAMAGES IN ADVANCE. THESE LIMITATIONS APPLY TO ALL CAUSES OF ACTION, INCLUDING, WITHOUT LIMITATION, BREACH OF CONTRACT, BREACH OF WARRANTY, NEGLIGENCE, STRICT LIABILITY, MISREPRESENTATION AND OTHER TORTS. Part Number: IDQ-SEI HF2-0001

4 Table of Contents Preface.... iv Informatica Resources.... iv Informatica Customer Portal.... iv Informatica Documentation.... iv Informatica Web Site.... iv Informatica How-To Library.... iv Informatica Knowledge Base.... v Informatica Multimedia Knowledge Base.... v Informatica Global Customer Support.... v Chapter 1: Introduction to Data Quality for Siebel Overview Understanding Data Cleansing, Deduplication, and Address Validation Logical Architecture About Workflows, Mappings, and Mapplets Cleansing and Deduplication Process Flows Real-Time and Batch Cleansing Real-Time Deduplication Batch Deduplication Using Staging Tables in Batch Deduplication Address Validation Process Flows Realtime Address Validation Batch Address Validation Physical Architecture and Installable Components Siebel Components Informatica Components Data Quality for Siebel Components Bill of Materials System Requirements Chapter 2: Installing and Configuring Informatica Components Overview Install Sequence Install PowerCenter Client and Server Components Post-Installation Requirements Create a PowerCenter Web Services Hub Create Staging Tables for Batch Duplicate Analysis Pre-Installation Requirements Table of Contents i

5 Character Encoding Requirements Install Informatica Data Quality Reference Tables Importing PowerCenter Workflows Pre-Import Requirements Steps to Import Post-Import Requirements Configuring the Siebel Parameter File Verify the Index Database Type Update the Batch Deduplication Settings Installing Reference Data Chapter 3: Configuring Siebel Overview Adding Library Files Adding the JAR File Adding Configuration Files Editing Configuration Files Chapter 4: Configuring Siebel for Cleansing and Deduplication Overview Enable Data Quality Functionality for Siebel Register the INFADQSiebel Library Enable Siebel Data Quality Configure Siebel Vendor Parameters Set Siebel Vender Parameters Set Siebel Data Quality Parameters Configure Business Components for Data Quality Operations Select Informatica as the Data Quality Vendor Define Business Components and Data Quality Operations Generate Siebel Match Keys Enable Cleansing and Deduplication for a Thick Client Chapter 5: Configuring Siebel for Address Validation Overview SIF File Import Adding Picks Modifying the Address Business Component Adding Picks for Address Applets System Preferences Setup INFADQSiebel JAR Setup JAR Setup for the Siebel Thick Client JAR Setup for the Siebel Thin Client Batch Setup ii Table of Contents

6 Verifying Your Address Validation Implementation Verifying a Realtime Address Validation Implementation Verifying a Batch Address Validation Implementation Index Table of Contents iii

7 Preface The Informatica Data Quality for Siebel Guide is written for Siebel system administrators and other users who install and set up Data Quality for Siebel and who configure Siebel applications to communicate with Data Quality for Siebel. This guide assumes that you have an understanding of data cleansing, deduplication, and address validation capabilities in real-time and batch scenarios. Informatica Resources Informatica Customer Portal As an Informatica customer, you can access the Informatica Customer Portal site at The site contains product information, user group information, newsletters, access to the Informatica customer support case management system (ATLAS), the Informatica How-To Library, the Informatica Knowledge Base, the Informatica Multimedia Knowledge Base, Informatica Product Documentation, and access to the Informatica user community. Informatica Documentation The Informatica Documentation team takes every effort to create accurate, usable documentation. If you have questions, comments, or ideas about this documentation, contact the Informatica Documentation team through at We will use your feedback to improve our documentation. Let us know if we can contact you regarding your comments. The Documentation team updates documentation as needed. To get the latest documentation for your product, navigate to Product Documentation from Informatica Web Site You can access the Informatica corporate web site at The site contains information about Informatica, its background, upcoming events, and sales offices. You will also find product and partner information. The services area of the site includes important information about technical support, training and education, and implementation services. Informatica How-To Library As an Informatica customer, you can access the Informatica How-To Library at The How-To Library is a collection of resources to help you learn more about Informatica products and features. It iv

8 includes articles and interactive demonstrations that provide solutions to common problems, compare features and behaviors, and guide you through performing specific real-world tasks. Informatica Knowledge Base As an Informatica customer, you can access the Informatica Knowledge Base at Use the Knowledge Base to search for documented solutions to known technical issues about Informatica products. You can also find answers to frequently asked questions, technical white papers, and technical tips. If you have questions, comments, or ideas about the Knowledge Base, contact the Informatica Knowledge Base team through at Informatica Multimedia Knowledge Base As an Informatica customer, you can access the Informatica Multimedia Knowledge Base at The Multimedia Knowledge Base is a collection of instructional multimedia files that help you learn about common concepts and guide you through performing specific tasks. If you have questions, comments, or ideas about the Multimedia Knowledge Base, contact the Informatica Knowledge Base team through at KB_Feedback@informatica.com. Informatica Global Customer Support You can contact a Customer Support Center by telephone or through the Online Support. Online Support requires a user name and password. You can request a user name and password at Use the following telephone numbers to contact Informatica Global Customer Support: North America / South America Europe / Middle East / Africa Asia / Australia Toll Free Brazil: Mexico: North America: Toll Free France: Germany: Italy: Netherlands: Portugal: Spain: Switzerland: United Kingdom: Toll Free Australia: New Zealand: Standard Rate India: Standard Rate Belgium: France: Germany: Netherlands: United Kingdom: Preface v

9 vi

10 C H A P T E R 1 Introduction to Data Quality for Siebel This chapter includes the following topics: Overview, 1 Understanding Data Cleansing, Deduplication, and Address Validation, 2 Logical Architecture, 2 Cleansing and Deduplication Process Flows, 4 Address Validation Process Flows, 5 Physical Architecture and Installable Components, 7 Overview Siebel acts as a unified source of account, customer, and prospect data across an enterprise. Data Quality for Siebel enhances your ability to create and maintain reliable and duplicate-free data in the Siebel system. Data Quality for Siebel applies the data quality management capabilities of Informatica applications to new record data entering the Siebel system and to data stored in the system. It integrates with the Siebel Data Quality (SDQ) Universal Connector through its application programming interface to deliver enhanced data cleansing, deduplication, and address validation capabilities in realtime and in batch mode. On These Record Types... Account Customer List Management Prospect CUT/Business Address CUT/Business Address You Can Perform These Data Quality Operations Real-Time Cleansing Batch Cleansing Real-Time Deduplication Batch Deduplication Real-Time and Batch Cleansing Real-Time and Batch Address Validation The data cleansing processes in Data Quality for Siebel correct and standardize record data values. The data deduplication processes identify duplicate records in the system and return these records for evaluation. The address validation processes validate records against valid addresses in a reference dataset. You can customize the data cleansing, deduplication, and address validation rules to suit the requirements of your organization. 1

11 Note: Siebel provides data consolidation and record merging rules are part of its configuration and setup. Data Quality for Siebel provides match scores to Siebel to facilitate this functionality. Understanding Data Cleansing, Deduplication, and Address Validation The data cleansing, deduplication, and address validation functionality in Data Quality for Siebel focuses on name and address data. Data cleansing performs standardization operations on the input record so that it meets user standards and requirements. Standardization harmonizes variations in customer terms, such as Doctor/Dr and Street/St, and removes extraneous punctuation. Data Quality for Siebel performs data cleansing in real time and batch modes. In each case, the Siebel Universal Connector sends data to Data Quality for Siebel one record at a time. Data deduplication identifies potential duplicates in record data provided by Siebel to Informatica applications. Data Quality for Siebel performs data deduplication in real time and batch modes. When processing a new record in real time, Data Quality for Siebel performs data cleansing before data deduplication. The deduplication process begins when the cleansing process completes. Address validation enhances the address cleansing and verification capabilities of Data Quality for Siebel by adding Informatica address validation components to the installed solution. Full address validation requires software and reference datasets sourced by Informatica from third-party address reference specialists. In realtime mode, users can interactively verify addresses against an address validation reference dataset in order to choose from valid address matches. In batch mode, Data Quality for Siebel non-interactively compares input record addresses to the reference dataset and then returns a validated or corrected address with status or error code information appended. Logical Architecture Data Quality for Siebel communicates with the Siebel system through a library file called INFADQSiebel that Universal Connector:in logical architecture resides locally to the Universal Connector component of Siebel Data Quality. The library file wraps the record data and instructions it receives from the Universal Connector in a SOAP (Simple Object Access Protocol) envelope and sends them as XML documents to PowerCenter. PowerCenter Web Services Hub receives the data and sends it to a PowerCenter workflow associated with the required data process. Depending on the data process involved, PowerCenter may call one or several workflows to process the incoming data. Each workflow may contain one or several mappings that include mapplets implementing data quality processes. These mapplets define the low-level cleansing, deduplication, and address validation operations performed on the Siebel data. The Data Quality for Siebel installation fileset includes Data Quality for Siebel mappings. These mappings do not represent the only types of cleansing, deduplication, and address validation you can perform with Informatica applications, nor do they represent the complete extent of Informatica s data quality capabilities. The mappings are provided as a useful sample of these capabilities. To edit the mappings, or create mappings or mapplets specific 2 Chapter 1: Introduction to Data Quality for Siebel

12 to your business processes, use Informormatica Data Quality. For more information, see Data Quality for Siebel Mappings and Mapplets on page 4. The following graphic illustrates the architecture on the Siebel side: Figure 1. Data Quality for Siebel Architecture, Siebel Orientation The following graphic illustrates the architecture on the Informatica side: Figure 2. Data Quality for Siebel Architecture, Informatica Orientation About Workflows, Mappings, and Mapplets Three types of Informatica processes are involved in running data cleansing, deduplication, and address validation tasks on data received from Siebel: workflows, mappings, and mapplets. Workflow. A set of instructions informing PowerCenter to run one or more tasks. It can be triggered by a PowerCenter user, by the arrival of data in real time, or by a scheduler. Workflows are composed of tasks. Tasks that contain mappings are called sessions. Data Quality for Siebel provides several workflows for import to the PowerCenter repository. These workflows contain one or more session tasks that specify the mappings that run on the input data. Data Quality for Siebel can run one or several mappings in sequence to perform cleansing, deduplication, and address validation on input data. Workflows are created in PowerCenter Workflow Manager and saved in the PowerCenter repository. Some workflows run continuously to maximize speed of data throughput between Siebel and Informatica applications. Mapping. A set of data input (source) parameters, data movement or transformation instructions, and data output (target) parameters that can be applied to a dataset in PowerCenter. Mappings are created in PowerCenter Designer and saved in the PowerCenter repository. Mapplets. A mapplet contains a set of input (source) parameters, data analysis, enhancement, or deduplication instructions, and output (target) parameters. Mapplets are stored in the PowerCenter repository for use in mappings. Data Quality for Siebel includes pre-built mapplets containing data quality processes. Logical Architecture 3

13 Data Quality for Siebel Mappings and Mapplets Use Informatica Data Quality to generate Data Quality for Siebel mappings and mapplets. The Data Quality for Siebel installation fileset includes Data Quality for Siebel mappings. You can customize the mappings, or you can create mappings and mapplets specific to your business processes. Save Data Quality for Siebel mappings and mapplets into the PowerCenter repository. Cleansing and Deduplication Process Flows When you enter a new record to the Siebel system, Data Quality for Siebel cleanses the record and returns the cleansed record to Siebel. When Siebel receives the cleansed record, the data deduplication processes begin. Note: You cannot create a record with the same name as an existing record. If Siebel finds an record match with the new record name, it displays a message stating that new record names must be unique. Real-Time and Batch Cleansing The data cleansing process flow is identical for real-time and batch scenarios. In each case, records are passed one at a time to Informatica. Real-time data cleansing enhances the usability of new records as they enter the Siebel system. Batch cleansing enhances the usability of records present in the system. The data flow is as follows: 1. Real time. You enter or edit a record in the Siebel system. - or - Batch. Siebel begins a batch cleansing job. 2. Siebel calls its Data Quality Manager, which instructs Siebel Data Quality to initiate the data cleansing operation. 3. The Siebel Universal Connector sends the record data to the INFADQSiebel library file, which passes the record information to the PowerCenter Web Services Hub in a SOAP envelope. 4. PowerCenter Web Services Hub routes the input data to a task in a PowerCenter workflow. 5. The data is routed to a PowerCenter mapping containing a data cleansing mapplet. When the data reaches the mapplet instructions, PowerCenter cleanses the data according to these instructions. 6. The same workflow task returns the cleansed data to the PowerCenter Web Services Hub. 7. The PowerCenter Web Services Hub returns the cleansed data to the Siebel system through the INFADQSiebel library file. Real-Time Deduplication This process begins when Siebel receives the cleansed record from the INFADQSiebel library file. The process flow is as follows: 1. Siebel calls its Data Quality Manager, which instructs Siebel Data Quality to initiate the deduplication operation. Siebel Data Quality performs initial duplicate analysis on its data to identify possible matches for the record. These possible matches are called candidate records. The input record is called the driver record. 2. The Siebel Universal Connector sends the driver and candidate records to the INFADQSiebel library file, which passes them to the PowerCenter Web Services Hub in a SOAP envelope. 4 Chapter 1: Introduction to Data Quality for Siebel

14 3. PowerCenter Web Services Hub routes the input data to a task in a PowerCenter workflow. 4. The data is routed to a PowerCenter mapping containing a data deduplication mapplet. When the data reaches the mapplet instructions, PowerCenter performs duplicate analysis according to the instructions. Informatica Data Quality operations calculate a set of match scores indicating the level of similarity between each driver-candidate pair. Informatica Data Quality returns the match scores and a unique ID for each pair. It does not return the original record data. 5. A workflow task returns the data results to the PowerCenter Web Services Hub, which returns the data to the Siebel System. 6. Siebel displays onscreen the candidate records that meet or exceed the match threshold score set in the Siebel system. You can configure the columns presented on this screen within the Siebel application using Siebel Tools. You can edit the match threshold value in Siebel. 7. You review the results of the process and decide how to proceed. You can select the original record for addition to the database or a candidate record to be merged with the information entered on the new record. Batch Deduplication In batch duplicate analysis, the process flow is as follows: 1. Siebel begins a batch cleansing job and calls its Data Quality Manager, which instructs Siebel Data Quality to initiate the deduplication operation. 2. The Siebel Universal Connector sends records to the INFADQSiebel library file in one or more batches. The default upper limit of records per batch is 200. This value is set in Siebel. The INFADQSiebel library file passes these records to the PowerCenter Web Services Hub in a SOAP envelope. 3. PowerCenter Web Services Hub routes the input data to a task in a PowerCenter workflow. 4. The data is routed to a PowerCenter mapping that writes the data to staging tables. Informatica Data Quality performs duplicate analysis on the data in these tables. Informatica Data Quality analyzes the degrees of similarity between records that share a common Siebel Dedup Token. Informatica Data Quality compares each record against each other record with the same Dedup Token and calculates a match score for each record pair. PowerCenter writes the match scores and a unique ID for each record pair to the staging database. 5. A workflow task returns the data results to the PowerCenter Web Services Hub, which returns the scores and IDs to Siebel. It does not return the original record data. Using Staging Tables in Batch Deduplication In data cleansing and real-time deduplication PowerCenter operates on data in XML. In batch deduplication, PowerCenter writes the data to a staging table before Informatica Data Quality performs duplicate analysis. PowerCenter then returns the results of the analysis to Siebel as XML. You must create the staging tables that will hold this data. For more information, see Create Staging Tables for Batch Duplicate Analysis on page 11. Address Validation Process Flows Data Quality for Siebel uses AddressDoctor to perform address validation in both realtime and batch mode. Previously, Data Quality for Siebel used QAS Pro Web to perform all realtime address validation. Address Validation Process Flows 5

15 Data Quality for Seibel performs realtime and batch address validation for the countries indicated by the following ISO codes: AUS CAN DEU FRA GBR NLD USA Realtime Address Validation In realtime address validation, the process flow is as follows: 1. A Siebel user selects the pick icon in the Address field. 2. If the country is supported by and configured for address validation, the Address data is passed by Siebel to the INFADQSiebel JAR via a Java Business Service. 3. The INFADQSiebel JAR passes the data to the PowerCenter WebServices Hub using the INFADQSiebel library, which passes the record to the PowerCenter Web Services Hub in a SOAP envelope. 4. The PowerCenter Web Services Hub routes the input data to a task in the PowerCenter Workflow. 5. The data is routed to a PowerCenter mapping that uses an Address Validator transformation to analyze the data. 6. The response from the PowerCenter mapping is returned to the WebServices Hub, which returns a list of matching addresses and their scores to Siebel. 7. The matching addresses and scores are presented to the user as a list in Siebel. Batch Address Validation In batch address validation, the process flow is as follows: 1. The Informatica Address Validation workflow is executed using the Siebel 'Workflow Process Manager' job. 2. The Informatica Address Validation workflow searches for addresses matched by the 'Informatica Batch Org' or 'Informatica Batch Per' searches defined in System Preferences. The workflow sends matches to the INFADQSiebel JAR in one or more batches. The default upper limit of records per batch is configurable. 3. The INFADQSiebel JAR passes the data to the PowerCenter WebServices Hub using the INFADQSiebel library, which passes the record to the PowerCenter Web Services Hub in a SOAP envelope. 4. The PowerCenter Web Services Hub routes the input data to a task in the PowerCenter Workflow. 5. The data is routed to a PowerCenter mapping, which sends the data to an appropriate address validation engine based on the country that the address is from. 6. The address data is compared to reference data, and if possible, corrected by the address validation engine. The address validation engine generates a validation status and returns the updated address to the Web Services Hub. 7. The Web Services Hub returns the address and validation status to Siebel. 8. Siebel updates the address data only if the validation status is specified in the 'Informatica Batch Status' entry in System Preferences. 9. The MatchStatus column of the address is updated with the validation status value returned. 6 Chapter 1: Introduction to Data Quality for Siebel

16 Physical Architecture and Installable Components The system architecture that encompasses Data Quality for Siebel comprises applications and files from Siebel and Informatica. Siebel Components Data Quality for Siebel uses the following Siebel components: Siebel system. Server-side Siebel instance, including the database. Data hubs. Used for managing data in the database. Supported hubs are Siebel Universal Customer Master and Siebel Customer Relationship Management. Data entry client. A client data entry tool for Siebel. Thick client users must load library and configuration files to their machines to use Data Quality for Siebel. Thin client users do not need to install these files. Data Quality Business Services. Data hubs use business services to call data quality processes on input data. Siebel Data Quality. Application that manages data cleansing and deduplication operations and communicates with third-party applications such as Data Quality for Siebel. Informatica Components Data Quality for Siebel uses server-side and client-side components from Informatica PowerCenter. Data Quality for Siebel uses the following server-side components: Repository Service Integration Service Web Services Hub The Repository and Integration Services manage and run data processes in PowerCenter. The Web Services Hub is a gateway that makes PowerCenter functionality available to external client applications through web services. The Web Services Hub and the web services it hosts comprise the Web Services Provider. Data Quality for Siebel also uses the client-side Repository Manager component. Use the Repository Manager to import the required workflows to the PowerCenter repository. Ensure that your license includes the following PowerCenter options: Data Cleansing Option Real-Time Option Data Quality for Siebel Components Data Quality for Siebel provides the following components, which you install separately from the Informatica components above. Locate these components in the Data Quality for Siebel install folders. Informatica Data Quality Library and Configuration Files. Data Quality for Siebel uses library and configuration files to enable communication between the Universal Connector and the PowerCenter Web Services Hub. Locate the library files in the bin folder of the installable fileset. Locate the configuration files in the SDQConnector folder of the installable fileset. PowerCenter Workflows. The data quality mapplets that perform data cleansing, duplication, and address validation operations are embedded in PowerCenter mappings. PowerCenter runs these mappings through Physical Architecture and Installable Components 7

17 workflow tasks. The workflows and their constituent tasks, mappings, and mapplets are saved as a single XML file for import to the PowerCenter repository. Locate the workflow XML file in the Workflows folder of the installable fileset. Informatica Data Quality Reference Tables. Data quality mapplets use reference tables in cleansing operations. Locate the reference tables in the Dictionaries folder of the installable fileset. Informatica Data Quality Mappings. Backup copies of the PowerCenter mappings. You do not need to install these files unless you want to edit the mappings in Data Quality and generate new mapplets for your solution. Staging Database Table Schema. PowerCenter uses staging tables when performing duplicate analysis in batch mode. You must configure PowerCenter to use tables with the required schema. The schema is contained in a file named <DatabaseType>-ddl.txt, where DatabaseType refers to the installed database. Locate this file in the Database folder of the installable fileset. Bill of Materials Your receive your Data Quality for Siebel components in a compressed folder. The following table describes the contents of this folder: Sub-folder Name [root folder] bin Database Dictionaries Java Resources SDQConnector Sif Workflows Xsd File Type Readme.txt Library files Database schema files XML file containing reference tables Informatica JAR files XML file containing data quality mappings Configuration and property files Siebel installation file PowerCenter workflow XML Web service data description files System Requirements The following general requirements apply to Data Quality for Siebel: Platform requirements. Data Quality for Siebel installs on 32-bit Windows, Linux, and AIX platforms. Application requirements. Data Quality for Siebel comprises applications and files associated with Siebel, Informatica PowerCenter, and Informatica Data Quality. The system requirements for each of these applications also apply to the components installed on their respective machines. Consult the user documentation for these applications for full information. Character encoding requirements. The staging database used by Informatica for batch duplicate analysis must be configured to accept data encoded in the UTF-8 encoding of Unicode. 8 Chapter 1: Introduction to Data Quality for Siebel

18 Database requirements. Data Quality for Siebel requires one of the following databases to stage data during batch deduplication: Oracle, DB2, or Microsoft SQL Server. Address validation requirements. To perform address validation, a 32-bit Java VM (version 1.5 or higher) must be installed on the Siebel machine. Additional configuration requirements apply to installations of PowerCenter and Informatica Data Quality. For more information on PowerCenter requirements and Informatica Data Quality requirements, see Chapter 2, Installing and Configuring Informatica Components on page 10. Physical Architecture and Installable Components 9

19 C H A P T E R 2 Installing and Configuring Informatica Components This chapter includes the following topics: Overview, 10 Install PowerCenter Client and Server Components, 11 Create a PowerCenter Web Services Hub, 11 Create Staging Tables for Batch Duplicate Analysis, 11 Install Informatica Data Quality Reference Tables, 12 Importing PowerCenter Workflows, 12 Configuring the Siebel Parameter File, 13 Installing Reference Data, 14 Overview The steps to install and deploy Informatica software for Data Quality for Siebel include installing applications and loading files for use in the Data Quality for Siebel environment. You must consult other Informatica documents for complete install instructions. This chapter identifies the documents you need. Informatica product documentation is available from the Informatica Customer Portal site at Install Sequence Install and configure Informatica components in this order: 1. Install PowerCenter Client and Server. 2. Create a PowerCenter Web Services Hub. 3. Create staging tables for PowerCenter batch deduplication operations. 4. Install Informatica Data Quality reference tables. 5. Import PowerCenter workflows and mappings. 6. If you want to customize the provided Data Quality mappings, install Informatica Data Quality. 10

20 Install all Informatica software on the same machine. You need not install Informatica software on the same machine as Siebel software. Informatica and Siebel components communicate by Simple Object Access Protocol (SOAP). Note: Informatica recommends installing Siebel and PowerCenter in the same physical location or data center. Install PowerCenter Client and Server Components Install PowerCenter server-side and client-side components. PowerCenter Client components import workflows to the PowerCenter repository. PowerCenter Server components define the PowerCenter repository and execute the workflow tasks. Refer to the PowerCenter documentation for instructions on installing the server-side and client-side PowerCenter components. The product install document is the PowerCenter Installation Guide. Post-Installation Requirements Use the PowerCenter Admin Console to review the Maximum Processes setting. The Maximum Processes setting determines the maximum number of concurrent processes permitted on the PowerCenter node must be adequate for the quantity of processes that Data Quality for Siebel may use. The Maximum Processes setting must be no lower than 20. Informatica recommends a Maximum Processes setting of 50 for Data Quality for Siebel. Create a PowerCenter Web Services Hub Create a Web Services Hub on the PowerCenter Integration Service machine. Refer to the PowerCenter documentation for information on the Web Services Hub and Web Services Provider. The install document is the PowerCenter Administration Guide. For more information on the Web Services Hub, consult the PowerCenter Web Services Provider Guide. Create Staging Tables for Batch Duplicate Analysis Data Quality for Siebel writes data to staging tables for batch duplicate analysis. You must create these tables. 1. Copy the schema for these tables from the <DataBaseType>-ddl.txt file included in your install fileset for Data Quality for Siebel. Pre-Installation Requirements Ensure that the PowerCenter Integration Service machine has a client installed for your staging database type. The following staging database types are supported: Oracle, DB2, and Microsoft SQL Server. Install PowerCenter Client and Server Components 11

21 Character Encoding Requirements If any characters outside of the Latin1 character will be passed to Data Quality for Siebel for batch duplicate analysis, you must verify that the database can store Unicode characters. Contact the database administrator to configure the encoding for the staging database. Install Informatica Data Quality Reference Tables Informatica Data Quality uses reference tables in data cleansing. 1. Locate the Dictionaries folder in your Data Quality for Siebel install fileset. This folder contains a sub-folder named DQforSiebel. 2. Copy the DQforSiebel folder into the services folder of your Informatica installation directory. Importing PowerCenter Workflows Data Quality for Siebel uses pre-defined PowerCenter workflows, which you must import to PowerCenter. The workflows are saved in the INFADQSiebel.xml file located in the Workflows folder of your install fileset. Import this file to your PowerCenter repository using the PowerCenter Repository Manager. Pre-Import Requirements Before you import the workflows, verify the following: Create a relational connection in the PowerCenter Repository and configure it to use the staging database schema specified on Create Staging Tables for Batch Duplicate Analysis on page 11. Ensure that the code page for this connection is set to UTF-8 encoding of Unicode. Add the All.param parameter file to the PowerCenter system in the $PMRootDir of the integration service. In default installations, this directory is named infa_shared. Within this directory, create a new directory called Param and place All.param there. See Configuring the Siebel Parameter File on page 13 for instructions on editing this file to match your system settings. Steps to Import To import a mapping or workflow to a PowerCenter repository: 1. Start the PowerCenter Repository Manager and connect to a repository. 2. From the Repository menu, select Import Objects. 3. The Import Wizard opens. Click Browse and select the INFADQSiebel.xml file. Click Next. 4. On the Select objects to import screen, select the option to Add All. Click Next. 5. On the Match Folders page, click the Browse button in the Destination Folder field to open the Folder Selection dialog box. Select the destination folder for the objects you will import. Create a new repository folder if necessary. Click Next. 12 Chapter 2: Installing and Configuring Informatica Components

22 6. You are prompted to specify rules for conflict resolution. Conflicts will occur if any object you import has the same name as an object in the destination repository. Create a rule that replaces (overwrites) any such objects. Click Next. 7. Resolve any conflicts identified between the import file and the contents of the destination folder. If the status is Resolved, click Import. 8. The import wizard describes the import progress on its Output tab. Click Done when the import is complete. Post-Import Requirements After you import the workflows, use PowerCenter Workflow Manager to associate the imported workflows with your PowerCenter Integration Service. To assign a workflow to a PowerCenter Integration Service: 1. From the Workflow Manager menu, select Service > Assign Integration Service. 2. The Assign Integration Service dialog box opens. In this dialog box, use the menu options to select the folder containing your imported Data Quality for Siebel workflows and to select an Integration Service. 3. Select the workflows to assign to the Integration Service. To select all workflows in the folder, choose Select all displayed workflows. 4. Click Assign. 5. Restart the PowerCenter Web Services Hub. Configuring the Siebel Parameter File The Data Quality for Siebel install files contain a parameter file named All.param that you must add to your PowerCenter installation and edit to suit your system. The file consists of several settings that have blank values or values that are deselected by a prefixed # comment character. Review these settings and edit them as necessary for your system. The types of setting include: The database type. Batch deduplication settings. Verify the Index Database Type The parameter file contains $$DATABASE_TYPE entries for Oracle, Microsoft SQL Server, and DB2 as shown below. By default, Oracle is the selected database type, indicated by the absence of a # symbol at the start of the line entry. $$DATABASE_TYPE=Oracle #$$DATABASE_TYPE=Microsoft SQL Server #$$DATABASE_TYPE=DB2 To change the database type: u Remove the # character from its $$DATABASE_TYPE and add this character to the current line entry. Configuring the Siebel Parameter File 13

23 Update the Batch Deduplication Settings Relational Connection Settings The $DBConnectionDQBatch setting specifies the relational connection for batch deduplication operations. $DBConnectionDQBatch=SDQ_Batch_Oracle Update the value of this setting to match the relational connection configured for batch deduplication in the Workflow Manager. Debug Settings Set $$DEBUG_ON to 1 to store batch deduplication source rows after deduplication completes. Source rows are stored in the following locations: Account - sdq_account_bulk_match_bak Contact - sdq_contact_bulk_match_bak Prospect - sdq_prospect_bulk_match_bak Session ID Cache Settings $$SESSION_ID_CACHE specifies the number of session ids that the Informatica library caches between calls to the SDQ_GetSessionID web service. The default value is This value must be set to 2 or higher. If the value is set to lower than 2, the default value of 1000 will be used. Match Score Threshold The $$MATCHSCORE_LIMIT setting determines the batch deduplication records to be stored in the staging database and returned to Siebel. This value should be less than or equal to the matchscore threshold configured in Siebel. The default value for this setting is as follows: MPLT_SDQ_BatchDedup.$$MATCHSCORE_LIMIT=60 Record Removal Settings Use the $$RETAINED_DAYS setting to determine the number of days that records are stored in the batch deduplication staging database. Records older than the number of days specified for this setting are deleted on a nightly basis. Installing Reference Data Informatica provides the batch and realtime reference datasets for Data Quality for Siebel. Informatica also provides the Data Quality Content Installer. Use the Data Quality Content Installer to install address reference datasets after you install all applications. For instructions on running the Content Installer, see the Informatica Data Quality Content Installation Guide. 14 Chapter 2: Installing and Configuring Informatica Components