User Guide. PowerCenter Connect for Netezza. (Version )

Transcription

1 User Guide PowerCenter Connect for Netezza (Version )

2 Informatica PowerCenter Connect for Netezza User Guide Version March 2008 Copyright (c) Informatica Corporation. All rights reserved. This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. This Software is protected by U.S. and/or international Patents and other Patents Pending. Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as provided in DFARS (a) and (a) (1995), DFARS (c)(1)(ii) (OCT 1988), FAR (a) (1995), FAR , or FAR (ALT III), as applicable. Informatica, PowerCenter, PowerCenterRT, PowerCenter Connect, PowerCenter Data Analyzer, PowerExchange, PowerMart, Metadata Manager, Informatica Data Quality, Informatica Data Explorer, Informatica Complex Data Exchange and Informatica On Demand Data Replicator are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners. Portions of this software and/or documentation are subject to copyright held by third parties, including without limitation: Copyright Oracle. All Rights Reserved. This product includes Eclipse software which is Copyright (c) 2007 The Eclipse Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at This product includes software developed by the Apache Software Foundation ( which is licensed under the Apache License, Version 2.0 (the License ). You may obtain a copy of the License at Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. DISCLAIMER: Informatica Corporation provides this documentation as is without warranty of any kind, either express or implied, including, but not limited to, the implied warranties of non-infringement, merchantability, or use for a particular purpose. Informatica Corporation does not warrant that this software or documentation is error free. The information provided in this software or documentation may include technical inaccuracies or typographical errors. The information in this software and documentation is subject to change at any time without notice. Part Number: PCC-NZU

3 Table of Contents List of Tables v Preface vii New Features and Enhancements viii About This Book ix Document Conventions ix Informatica Resources x Informatica Customer Portal x Informatica Web Site x Informatica Knowledge Base x Informatica Global Customer Support x Chapter 1: Understanding PowerCenter Connect for Netezza Overview Integration Service and Netezza Integration Chapter 2: Installation and Configuration Overview Minimum System Requirements Prerequisites Installing and Configuring PowerCenter Connect for Netezza Before You Upgrade Upgrading PowerCenter Connect for Netezza Installing PowerCenter Connect for Netezza Installing the Client Component Installing the Repository Component Installing the Integration Service Component Registering the Plug-in Updating the Plug-in Registering the Plug-in Establishing ODBC Connectivity Configuring ODBC Connectivity on Windows Configuring ODBC Connectivity on UNIX Uninstalling PowerCenter Connect for Netezza iii

4 Uninstalling from Windows Uninstalling from UNIX Changed Behavior Connection Object Type Change Update Strategy Configuration Change Key Type Change Chapter 3: Working with Netezza Sources and Targets Overview Configuring Netezza Source and Target Definitions Configuring Source Qualifier Properties Chapter 4: Creating and Configuring Netezza Workflows Configuring Netezza Connections Configuring a Netezza Session Configuring a Session with a Netezza Source Configuring a Session with a Netezza Target Target Database Connection Target Properties Configuring Update Strategy Configuring Partitioning Configuring Multiple Targets for the Same Target Table Configuring Support for NPS 4.0 on Windows Configuring a Session for Optimal Performance Troubleshooting Appendix A: Datatypes Datatype Reference Appendix B: Error Messages Designer Error Messages Integration Service Error Messages iv Table of Contents

5 List of Tables Table 2-1. Library Variables Table 3-1. SQL Properties Table 4-1. Netezza Source Properties Table 4-2. Netezza Target Properties Table 4-3. Treating Source Rows as Updates Table 4-4. Treating Source Rows as Data-Driven Table A-1. Netezza and Transformation Datatypes List of Tables v

6 vi List of Tables

7 Preface Welcome to PowerCenter Connect, Informatica s family of packaged software products that helps you extract data and metadata from ERP and other third-party applications. PowerCenter Connect for Netezza is a natural extension to the open architecture of PowerCenter, which supports data extraction from a wide variety of operational data sources. PowerCenter Connect for Netezza extracts data stored in Netezza databases into PowerCenter, leverages it further for effective planning, analysis, and management, and loads data into Netezza database or other targets. vii

8 New Features and Enhancements PowerCenter Connect for Netezza includes the following new features and enhancements: Duplicate row handling. Enhanced ability to handle duplicate records for incoming data. Escape character. Handles special characters by adding an escape character to the data. Session load summary. Displays the load summary report in the session load summary for the number of rows applied, rejected, requested, and affected. Support for NPS 4.0 on Windows. Certified against NPS 4.0 and Netezza ODBC Driver. viii Preface

9 About This Book The Informatica PowerCenter Connect for Netezza User Guide provides information about extracting data from a Netezza source and loading data into a Netezza target. It is written for database administrators and developers who are responsible for extracting data from Netezza and loading data to Netezza. This book assumes you have knowledge of Netezza and PowerCenter. Document Conventions This guide uses the following formatting conventions: If you see italicized text boldfaced text italicized monospaced text Note: Tip: Warning: monospaced text bold monospaced text It means The word or set of words are especially emphasized. Emphasized subjects. This is the variable name for a value you enter as part of an operating system command. This is generic text that should be replaced with user-supplied values. The following paragraph provides additional facts. Supplies information, which may apply only in special cases. The following paragraph provides suggested uses. The following paragraph notes situations where you can overwrite or corrupt data, unless you follow the specified procedure. This is a code example. This is an operating system command you enter from a prompt to execute a task. Preface ix

10 Informatica Resources Informatica Customer Portal As an Informatica customer, you can access the Informatica Customer Portal site at The site contains product information, user group information, newsletters, access to the Informatica customer support case management system (ATLAS), the Informatica Knowledge Base, Informatica Documentation Center, and access to the Informatica user community. Informatica Web Site You can access the Informatica corporate web site at The site contains information about Informatica, its background, upcoming events, and sales offices. You will also find product and partner information. The services area of the site includes important information about technical support, training and education, and implementation services. Informatica Knowledge Base As an Informatica customer, you can access the Informatica Knowledge Base at Use the Knowledge Base to search for documented solutions to known technical issues about Informatica products. You can also find answers to frequently asked questions, technical white papers, and technical tips. Informatica Global Customer Support There are many ways to access Informatica Global Customer Support. You can contact a Customer Support Center through telephone, , or the WebSupport Service. Use the following addresses to contact Informatica Global Customer Support: support@informatica.com for technical inquiries support_admin@informatica.com for general customer service requests WebSupport requires a user name and password. You can request a user name and password at x Preface

11 Use the following telephone numbers to contact Informatica Global Customer Support: North America / South America Europe / Middle East / Africa Asia / Australia Informatica Corporation Headquarters 100 Cardinal Way Redwood City, California United States Toll Free Standard Rate Brazil: Mexico: United States: Informatica Software Ltd. 6 Waltham Park Waltham Road, White Waltham Maidenhead, Berkshire SL6 3TN United Kingdom Toll Free Standard Rate Belgium: France: Germany: Netherlands: United Kingdom: Informatica Business Solutions Pvt. Ltd. Diamond District Tower B, 3rd Floor 150 Airport Road Bangalore India Toll Free Australia: Singapore: Standard Rate India: Preface xi

12 xii Preface

13 Chapter 1 Understanding PowerCenter Connect for Netezza This chapter includes the following topic: Overview, 2 1

14 Overview PowerCenter Connect for Netezza provides bidirectional connectivity between Informatica PowerCenter and Netezza to extract and load data. The Designer uses ODBC drivers to connect to the Netezza database. You can import Netezza tables as sources and target definitions. You can connect to Netezza Performance Server to read data from Netezza tables and load data to Netezza tables. Netezza Performance Server integrates database, server, and storage in a single system. Integration Service and Netezza Integration The Integration Service reads and writes data through an external table. It uses the bulk load utility on the external table to extract and load data. This bulk loading utility increases the session performance. Figure 1-1 shows how the Integration Service writes data to the Netezza Performance Server using bulk loading utility: Figure 1-1. Integration Service and Netezza Performance Server Integration Repository Netezza Session Netezza Connection The Integration Service reads and writes data based on session and Netezza relational connection configuration. Integration Service Bulk extract/load using external table Netezza Performance Server The Integration Service reads and writes data through an external table. Configure a Netezza database connection to read data from and write to Netezza. 2 Chapter 1: Understanding PowerCenter Connect for Netezza

15 Chapter 2 Installation and Configuration This chapter includes the following topics: Overview, 4 Installing PowerCenter Connect for Netezza, 6 Registering the Plug-in, 9 Establishing ODBC Connectivity, 10 Uninstalling PowerCenter Connect for Netezza, 13 Changed Behavior, 14 3

16 Overview This chapter provides information about upgrading, installing, and configuring PowerCenter Connect for Netezza. Minimum System Requirements Ensure that the minimum system requirements are met. Prerequisites Before you upgrade or install PowerCenter Connect for Netezza, complete the following tasks: Install or upgrade PowerCenter. For more information about the supported PowerCenter versions, see the PowerCenter Connect for Netezza Release Notes. Install client and server components of the Netezza Performance Server. Verify that you have read and write permissions on the following directories: <PowerCenter Installation Directory>\server\bin <PowerCenter Installation Directory>\client\bin Install the 3.5 ODBC standard Netezza ODBC driver on Windows for the PowerCenter Client to fetch the metadata and 3.0 standard Netezza ODBC driver on the machine hosting the Integration Service to access the data in Netezza Performance Server. The Services installation must be able to add and overwrite files in this directory. Installing and Configuring PowerCenter Connect for Netezza To install and configure PowerCenter Connect for Netezza, complete the following steps: 1. Install the components. Install Client, Repository, and Integration Service components to access the Netezza Performance Server. For more information, see Installing PowerCenter Connect for Netezza on page Register the plug-in. Register the plug-in with the repository. For more information, see Registering the Plug-in on page Establish ODBC connectivity. Establish ODBC connectivity to connect to the Netezza Performance Server with the PowerCenter Client and Integration Service. For more information, see Establishing ODBC Connectivity on page 10. Before You Upgrade Before you upgrade PowerCenter Connect for Netezza, uninstall the previous version. When you uninstall PowerCenter Connect for Netezza, you do not need to unregister the plug-in. 4 Chapter 2: Installation and Configuration

17 Upgrading PowerCenter Connect for Netezza When you upgrade PowerCenter Connect for Netezza, complete the following tasks: 1. Install PowerCenter Connect for Netezza. When you install PowerCenter Connect for Netezza, you install the PowerCenter Client, Server, and Repository components. For more information, see Installing PowerCenter Connect for Netezza on page Update and register the plug-in. Register the PowerCenter Connect for Netezza plug-in to update the existing plug-in registration. For more information, see Registering the Plug-in on page 9. After you upgrade, review the behavior changes for Netezza. For more information, see Changed Behavior on page 14. Overview 5

18 Installing PowerCenter Connect for Netezza Step 1. Install Step 1. Upgrade When you install PowerCenter Connect for Netezza, you install components that allow PowerCenter to access the Netezza Performance Server. Install the following components: Client component. You can import definitions, create mappings, and create connection objects using the PowerCenter Client. Repository component. Allows the Repository Service access to the Netezza Performance Server metadata in the repository. Integration Service component. Allows the Integration Service to run Netezza Performance Server sessions. Install these components on the machines that you want to access Netezza. Installing the Client Component Install the Client component on each PowerCenter Client machine where you want to create or access Netezza Performance Server metadata. To install the Client component: 1. Run installer.exe from the installation package. 2. Choose the Client component. 3. Click Next. 4. Enter your name and organization name. 5. Select the directory where the PowerCenter Client executables are located. By default, these are in the following location: <PowerCenter Installation Directory>\client\bin 6. Click Next to begin the installation. 7. Click OK when the installation is complete. Installing the Repository Component Install the Repository component on any node configured to run a Repository Service process. If the Repository Service is configured to run on primary and backup nodes, install the Repository component on each node configured to run the Repository Service process. 6 Chapter 2: Installation and Configuration

19 To install the Repository component on Windows: 1. Run installer.exe from the installation package. 2. Choose the Repository component. 3. Click Next. 4. Enter your name and organization name. 5. Select the directory where you want to install the PowerCenter Connect for Netezza Repository component. By default, the Repository component is installed in the following location: <PowerCenter Installation Directory>\server\bin\Plugin 6. Click Next to begin the installation. 7. Click OK when the installation is complete. To install the Repository component on UNIX: 1. Go to the Repo directory. 2. Enter./install at the prompt. 3. Install the PowerCenter for Netezza Repository component in the following location: <PowerCenter Installation Directory>/server/bin/Plugin The Repository component is installed. Installing the Integration Service Component If the Integration Service is configured to run on primary and backup nodes, install the Integration Service component on each node configured to run the Integration Service process. If the Integration Service is configured to run on a grid, install the Integration Service component on each node configured to run on the grid. If you cannot install the Integration Service component on each node in the grid, create a resource in the domain and assign it to each node where you installed the Integration Service Component. When you create a session, configure the session to use the resource. For example, create a custom resource called Netezza. When you create a session, you can require the session to use the resource Netezza. The Load Balancer will only dispatch the session to nodes where the Integration Service component is installed. For more information about resources, see the PowerCenter Administrator Guide. To install the Integration Service component on Windows: 1. Run installer.exe from the installation package. 2. Choose the Server component. 3. Click Next. Installing PowerCenter Connect for Netezza 7

20 4. Enter your name and organization name. 5. Select the directory where the Integration Service executables are located. By default, the Integration Service component is installed in the following location: <PowerCenter Installation Directory>\server\bin 6. Click Next to begin the installation. 7. Click OK when the installation is complete. To install the Integration Service component on UNIX: 1. Go to the Server directory. 2. Enter./install at the prompt. 3. Install the Integration Service executables in the following location. <PowerCenter Installation Directory>/server/bin The Integration Service component is installed. 8 Chapter 2: Installation and Configuration

21 Registering the Plug-in Step 2. Install Step 2. Upgrade When you install PowerCenter Connect for Netezza, register the plug-in with the repository. A plug-in is an XML file that defines the functionality of PowerCenter Connect for Netezza. When you upgrade from PowerCenter Connect for Netezza 1.1 SP 5 or PowerCenter Connect for Netezza 1.2, perform the following tasks: 1. Update the plug-in using the pmrep fixnetezzaplugin command. 2. Register the plug-in using the pmrep RegisterPlugin command. To run the pmrep commands to upgrade and register the plug-in, connect to the repository and run the Repository Service in exclusive mode. If you do not have the Administration Console privilege on the Repository Service or the Admin Repository privilege on the repository, you can ask the Repository Service administrator to register the plug-in. Updating the Plug-in Update the plug-in when you upgrade. The fixnetezzaplugin pmrep command updates the Netezza plug-in in the repository. The fixnetezzaplugin command uses the following syntax: fixnetezzaplugin Registering the Plug-in Register the plug-in when you install or upgrade. The RegisterPlugin pmrep command registers the Netezza plug-in with the repository. The plug-in file for PowerCenter Connect for Netezza is Netezza.xml. When you install the Repository component, the installer copies Netezza.xml to the following directory: <PowerCenter Installation Directory>/server/bin/Plugin The RegisterPlugin command uses the following syntax: registerplugin -i <input_registration_file_name_or_path> -e The plug-in file for PowerCenter Connect for Netezza is Netezza.xml. When you install the Repository component, the installer copies Netezza.xml to the following directory: <PowerCenter Installation Directory>/server/bin/Plugin For more information about using the Administration Console to register a plug-in, see the PowerCenter Administrator Guide. For more information about the pmrep RegisterPlugin command, see the PowerCenter Command Line Reference. Registering the Plug-in 9

22 Establishing ODBC Connectivity Step 3. Install The Designer uses ODBC drivers to import metadata from the Netezza Performance Server. The Integration Service uses ODBC to read or write data to Netezza Performance Server. You specify an ODBC data source in the Designer when you import a Netezza definition. You specify an ODBC data source in the Workflow Manager when you create a Netezza relational database connection object. Configuring ODBC Connectivity on Windows To establish ODBC connectivity for the PowerCenter Client and Integration Service on Windows, create an ODBC data source on each PowerCenter Client machine that you want to access Netezza metadata. Also, create an ODBC data source on each node on which the Integration Service is configured to run. You configure the ODBC connection from the Control Panel. Use the NetezzaSQL driver for the ODBC connection. The NetezzaSQL driver installs with Netezza Performance Server client component. Note: Install NetezzaSQL drivers on PowerCenter Client and Integration Service machines. Configuring ODBC Connectivity on UNIX To establish ODBC connectivity for the Integration Service on UNIX, complete the following steps on each machine configured to run the Integration Service: 1. Create an ODBC data source. Configure the.odbc.ini file to create a Netezza Performance Server ODBC data source. 2. Set the environment variable. Set the environment variable to the ODBC driver manager. Creating an ODBC Data Source To create an ODBC data source, you create and configure a file called.odbc.ini. You add fields to the file that contain the connection information the Integration Service requires to connect to the Netezza Performance Server. Note: Place the.odbinst.ini file in the same directory as.odbc.ini. 10 Chapter 2: Installation and Configuration

23 To create an ODBC data source: 1. Open the.odbc.ini file in the $HOME directory. If the file does not exist in the $HOME directory, create it. 2. Add the following entries to the.odbc.ini file: Field Required/ Optional Description [DSN] Required Data source name used to identify database connection properties. Driver Required Path where the Netezza ODBC driver is installed. Description Required Enter a description of the NetezzaSQL ODBC data source. Servername Required Host on which the Netezza Performance Server is running. Port Required Port number on which the Netezza Performance Server is running. Database Required Database of external or target table. Username Required User name for the database. Password Required Password for the database. ReadOnly Optional Enter True or False. Only Select SQL statement will be allowed in case set to True. Other SQL statements will cause an error. ShowSystemTables Optional Enter True or False. When displaying available table, system tables should be included in the list. Optimizer Optional Enter True or False. This switch controls the Keyset Query Optimization. The following example shows fields in an.odbc.ini file to connect to Netezza Performance Server: [NZSQL] driver=/usr/local/nz/lib/libnzsqlodbc3.so Description=NetezzaSQL ODBC Servername=nzdev Port=5480 Database=dev Username=admin Password=pass ReadOnly=False ShowSystemTables=False Optimizer=False Setting the Environmental Variable to the ODBC Driver Manager Set the library path to the ODBC driver manager. When you set the library path, use the appropriate environment variable for each operating system. Establishing ODBC Connectivity 11

24 Table 2-1. describes environment variable for the library path on each operating system: Table 2-1. Library Variables Operating System Solaris Linux AIX HP-UX Variable LD_LIBRARY_PATH LD_LIBRARY_PATH LIBPATH SHLIB_PATH For example, use the following syntax on Solaris: export LD_LIBRARY_PATH:<PowerCenter Installation Directory>/ ODBC 5.1/lib:$LD_LIBRARY_PATH 12 Chapter 2: Installation and Configuration

25 Uninstalling PowerCenter Connect for Netezza To uninstall PowerCenter Connect for Netezza, uninstall it from each machine where you installed the Client, Repository, or Integration Service components. Uninstalling from Windows To uninstall PowerCenter Connect for Netezza from Windows, use the Windows Add/ Remove Programs. Uninstalling from UNIX You can uninstall the PowerCenter Connect for Netezza Repository and Integration Service components from UNIX. To uninstall the Repository component from UNIX: 1. Enter./install at the prompt. 2. Enter the absolute path to the installation directory. By default, the Repository component is in the following location: <PowerCenter Installation Directory>/server/bin/Plugin If PowerCenter Connect for Netezza is installed in this directory, the installer prompts you to overwrite the existing installation or uninstall the Repository component. 3. Use the uninstall option to uninstall the Repository component. To uninstall the Integration Service component from UNIX: 1. Enter./install at the prompt. 2. Enter the absolute path to the installation directory. By default, the Integration Service is in the following location: <PowerCenter Installation Directory>/server/bin/ 3. When the installer prompts you to overwrite the existing installation or uninstall the Integration Service component, choose to uninstall the Integration Service component. Uninstalling PowerCenter Connect for Netezza 13

26 Changed Behavior This section describes the behavior changes for PowerCenter Connect for Netezza from previous versions. Connection Object Type Change Upgrade Impact: Behavior Change, Upgraded Metadata Version Implemented: 1.2 Versions Affected: 1.1 SP5 Effective in version 1.2 for PowerCenter Connect for Netezza, the Netezza connection object type is changed from Application to Relational. The Netezza application connections you created in a previous version upgrade to Netezza relational database connections. Update Strategy Configuration Change Upgrade Impact: Behavior Change, Upgraded Metadata Version Implemented: 1.2 Versions Affected: 1.1 SP5 Effective in version 1.2 for PowerCenter Connect for Netezza, the options to configure update strategy are changed in the session properties. You can select Insert, Delete, and Update as Update, or Update Else Insert. In a previous version you could choose Insert, Update, Delete and Update Else Insert. When you upgrade, the sessions enable to Update are upgraded to Update as Update. Key Type Change Upgrade Impact: Upgraded Metadata Version Implemented: 1.2 Versions Affected: 1.1 SP5 Effective in version 1.2 for PowerCenter Connect for Netezza, the key types are changed to Primary Key and Foreign Key. In a previous version, the key types were Key or No Key. When you upgrade, the field in a source or target definition are configured as the key upgrades to the primary key. 14 Chapter 2: Installation and Configuration

27 Chapter 3 Working with Netezza Sources and Targets This chapter includes the following topics: Overview, 16 Configuring Netezza Source and Target Definitions, 17 Configuring Source Qualifier Properties, 19 15

28 Overview Netezza source and target definitions represent metadata for the Netezza tables. You can use the wizard to import Netezza source and target definitions into the PowerCenter repository. When you import Netezza source and target definitions, you can choose to preview data in the tables. When the Integration Service extracts from Netezza, it uses the bulk extract functionality of the external tables. When the Integration Service loads to Netezza, it uses the bulk loading functionality of the external tables. 16 Chapter 3: Working with Netezza Sources and Targets

29 Configuring Netezza Source and Target Definitions You can define a Netezza source or target by importing a Netezza table from the Netezza database. To import a Netezza table from the Netezza database, use the Import Netezza Table wizard. You need to specify an ODBC data source in the Designer when you import a Netezza definition. You can manually edit source and target definitions to configure the properties that you cannot import from Netezza. You can edit a Netezza definition to create key columns and key relationships. These relationships can be logical relationships. They do not have to exist in the database. To insert with primary key validation, update, or delete data from Netezza, you must define at least one primary key in the target definition. Configure the key type on the Columns tab of the target definition. To import a Netezza source or target definition: 1. In the Source Analyzer, click Sources > Import Netezza Sources. -or- In the Target Designer, click Targets > Import Netezza Target. The Import Netezza Table dialog box appears. 2. Select an ODBC data source from the list. 3. Enter a Netezza user name and password. 4. Click Connect to connect to Netezza. A list of tables appears. Configuring Netezza Source and Target Definitions 17

30 5. Optionally, select a table from the list and click Preview Data to preview the data in the selected table. The Preview Data dialog box appears. You can see the data for all the columns in the selected table. 6. Click OK. 7. Select the tables you want to import. 18 Chapter 3: Working with Netezza Sources and Targets

31 Configuring Source Qualifier Properties You can define the properties for the Application Source Qualifier associated with a Netezza source definition. You can configure properties to sort the number of input ports and to retrieve distinct data from a Netezza source. Figure 3-1 shows the SQL Properties tab to configure the Source Qualifier properties: Figure 3-1. SQL Properties Table 3-1 describes the SQL properties: Table 3-1. SQL Properties Source Options Required/ Optional Description Select Distinct Optional Selects unique rows. The Integration Service includes a SELECT DISTINCT clause if you choose Select Distinct option. Netezza ignores trailing spaces. As a result, when you use select distinct when extracting data from a Netezza source, the Integration Service might extract fewer rows than expected. Source Filter Optional Filter condition. The Integration Service applies filter condition when reading source data. Use the following syntax: <table name>. <field name> <operator> <value>. The filter condition is case sensitive. User Defined Join Optional Condition used to join data from multiple sources represented in the Application Source Qualifier transformation. Configuring Source Qualifier Properties 19

32 Table 3-1. SQL Properties Source Options Number of Sorted Ports Required/ Optional Optional Description A positive number to indicate the number of columns used when sorting rows queried from the source. If you enter a positive value, the Integration Service adds an ORDER BY clause to the default query when it reads source rows. The ORDER BY clause includes the number of ports specified, starting from the top of the transformation. When you specify the number of sorted ports, the database sort order must match the session sort order. Default is 0. SQL Query Optional Overrides the default query. When you enter an SQL query, enclose column names in double quotes. The query is case sensitive. 20 Chapter 3: Working with Netezza Sources and Targets

33 Chapter 4 Creating and Configuring Netezza Workflows This chapter includes the following topics: Configuring Netezza Connections, 22 Configuring a Netezza Session, 23 Configuring a Session for Optimal Performance, 30 Troubleshooting, 31 21

34 Configuring Netezza Connections Before you can run a Netezza session, you must configure a Netezza database connection for the Netezza sources or targets in the Workflow Manager. When you configure a Netezza database connection, you specify connection attributes that the Integration Service uses to connect to Netezza Performance Server. To configure Netezza relational connections: 1. In the Workflow Manager, connect to a PowerCenter repository. 2. Click Connections > Relational. The Relational Connection Browser dialog box appears. 3. Click New. The Select Subtype dialog box appears. 4. Select Netezza from the Select Subtype list. 5. Click OK. The Connection Object Definition dialog box appears. 6. Enter the following information in the Connection Object Definition dialog box: Connection Attribute Required/ Optional Description User Name Required User name to connect to Netezza. Password Required Password for the user name. Connect String Required ODBC data source to connect to Netezza Performance Server. Code Page Required Code page associated with Netezza Performance Server. Select UTF-8 encoding of Unicode as the code page for Unicode data movement mode. Select MS Windows Latin (ASNI), superset as the code page for ASCII data movement mode. Environmental SQL n/a The Integration Service ignores this value. 7. Click OK. The relational connection appears in the Relational Connection Browser dialog box. 22 Chapter 4: Creating and Configuring Netezza Workflows

35 Configuring a Netezza Session Before you can run a Netezza session, the Integration Service must connect to Netezza Performance Server. Netezza database connections must exist in the repository to appear on the source or target database list. You must define them prior to configuring a session. For more information about configuring a database connection, see Configuring Netezza Connections on page 22. Configuring a Session with a Netezza Source You can configure the session properties for Netezza source on the Mapping tab. Define the properties for each source instance in the session. When you click on the Mapping tab, you can view and configure the settings of a session associated with a mapping containing a Netezza source definition. Select the source under the Source node and specify the value for the following session to override the corresponding values in the Application Source Qualifier that are available only at the session level. Table 4-1 describes the session properties: Table 4-1. Netezza Source Properties Attribute Name Pipe Directory Path Required/ Optional Optional Description Define a path for the Integration Service to create the pipe. The defined directory path should exist. If you do not specify the path, the Integration Service uses <PC installation directory>/server/bin to create the pipe. Select Distinct Optional Selects unique rows. The Integration Service includes a SELECT DISTINCT clause if you choose Select Distinct option. Netezza ignores trailing spaces. As a result, when you use select distinct when extracting data from a Netezza source, the Integration Service might extract fewer rows than expected. Select to override the Select Distinct attribute in the Source Qualifier. Source Filter Optional Filter condition. The Integration Service applies filter condition when reading source data. Use the following syntax: <table name>. <field name> <operator> <value>. The filter condition is case sensitive. This filter condition overrides the filter condition in the Source Qualifier. User Defined Join Optional Condition used to join data from multiple sources represented in the Application Source Qualifier transformation. This value overrides the value in the Source Qualifier. Configuring a Netezza Session 23

36 Table 4-1. Netezza Source Properties Attribute Name Number of Sorted Ports Required/ Optional Optional Description A positive number to indicate the number of columns used when sorting rows queried from the source. If you enter a positive value, the Integration Service adds an ORDER BY clause to the default query when it reads source rows. The ORDER BY clause includes the number of ports specified, starting from the top of the transformation. When you specify the number of sorted ports, the database sort order must match the session sort order. Default is 0. This value overrides the value in the Source Qualifier. SQL Query Optional Overrides the default query. When you enter an SQL query, enclose column names in double quotes. The query is case sensitive. This query overrides the query in the Source Qualifier. Delimiter Optional Set the delimiter to any value supported by Netezza Performance Server. The delimiter separates successive input fields. The value that you enter in the Delimiter field can be a part of the data for the Netezza source. Default is. NullValue Optional Configure the NullValue parameter of an external table. The Integration Service uses the NullValue internally. Maximum value is one character. Default is blank. EscapeCharacter Optional Configure the escape character of an external table. If the data contains NULL, CR, and LF characters in the Char or Varchar field, you need to escape these characters in the source data before extracting. Enter a escape character before the data. The only supported escape character is backslash (\). Configuring a Session with a Netezza Target You can configure target properties for a session that writes data to Netezza targets: Target properties. You can define target properties, such as target update options and reject options. For more information, see Target Properties on page 25. Pipeline partitioning. You can create multiple partitions in a session to load target data concurrently to the target. For more information, see Configuring Partitioning on page 29. Update strategy. Configure the update strategy to determine how the Integration Service inserts, updates, and deletes rows. For more information, see Configuring Update Strategy on page 28. Multiple targets referring to the same table. You can configure multiple targets to refer data of the same Netezza table. For more information, see Configuring Multiple Targets for the Same Target Table on page 29. Target Database Connection You can choose the Netezza target connection in the Transformations view of the Mapping tab. Click either the Targets or Connections node and select the Netezza database connection from the list for each target instance. You must have the read permission for the target 24 Chapter 4: Creating and Configuring Netezza Workflows

37 database connection to configure the session to use it. The user starting the configured session must have the execute permission for target database connections. You can switch the writer type to Netezza writer for all the mappings that include relational targets. However, for the mappings that include Netezza targets, you cannot change the writer type to relational or file. Target Properties You can configure the session properties for Netezza targets in the Transformations view on the Mapping tab. Define the properties for each target instance in the session. Figure 4-1 shows the Netezza target properties you define in the Properties settings on the Mapping tab: Figure 4-1. Netezza Target Properties Configuring a Netezza Session 25

38 Table 4-2 describes the target properties available on the Mapping tab: Table 4-2. Netezza Target Properties Target Property Pipe Directory Path Error Log Directory Name Required/ Optional Optional Optional Description Define a directory path for the Integration Service to create the pipe. The defined directory path should exist. If you do not specify the directory path, the Integration Service uses <PC installation directory>/server/bin to create the pipe. The error log directory can reside on the machine where the Integration Service runs. For example, you can enter the following path: /home/persist/errorlogdir If you do not specify any value, the error log is created in the /tmp directory on the machine hosting NPS. The Integration Service creates a bad file in the error log directory if the data is not valid. Insert Optional The Integration Service inserts all rows flagged for insert. Default is enabled. Delete Optional The Integration Service deletes all rows flagged for delete. Default is enabled. Update (as Update) Update (else Insert) Truncate Target Table Option Optional Optional Optional The Integration Service updates all rows flagged for update. Default is enabled. For more information about Updates, see Table 4-3 on page 28. The Integration Service updates the existing rows and inserts the remaining rows flagged as update. Default is disabled. For more information about Updates, see Table 4-3 on page 28. The Integration Service truncates the target before loading. Default is disabled. Delimiter Optional Set the delimiter to any value supported by the Netezza Performance Server. The delimiter separates successive input fields. Ensure that the value that you enter in this field should not be a part of the input data. Default is Control Character Optional Configure the CTRLCHARS parameter of external table to transfer data containing control characters. You can enter control characters for Char and Varchar fields. If you enter a control character, you must escape NULL, CR, and LF fields. Default is TRUE. CRINSTRING Optional Set the CRINSTRING parameter to transfer data containing carriage returns (CR). You can enter a non escape CR in Char or Varchar fields. To load the control characters present in the Char and Varchar fields, set the CTRLCHARS and CRINSTRING parameters to TRUE in the session properties for the Netezza source. Default is TRUE. NullValue Optional Configure the NullValue parameter of an external table. The Integration Service uses the NullValue internally. Maximum value is one character. Default is blank. 26 Chapter 4: Creating and Configuring Netezza Workflows

39 Table 4-2. Netezza Target Properties Target Property Required/ Optional Description EscapeCharacter Optional Configure the escape character of an external table. If the data contains NULL, CR, and LF characters in the Char or Varchar field, you need to escape these characters before loading. Enter a escape character before the data. The only supported escape character is backslash (\). Quoted Value Optional You can configure the QUOTEDVALUE parameter of an external table. You can select SINGLE or DOUBLE to enclose the field in single or double quotes. Select NO to omit quotes. Default is NO. The quoted value is not a part of the data for this property. Ignore Key Constraints Duplicate Row Handling Mechanism Add Escape Character Optional Optional Optional Ignores constraints on primary key fields. When you select this option, the Integration Service can write duplicate rows with the same primary key to the target. Default is disabled. Select when you configure multiple partitions. The Integration Service ignores this value when you treat source rows as update. Select one of the following values to indicate how the Integration Service handles duplicate rows: - First Row. The Integration Service passes the first row to the target and rejects the rows that follow with the same primary key. - Last Row. The Integration Service passes the last duplicate row to the target and discards rest of the rows. Default is First Row. Adds an escape character to all the special characters in the data. The special characters include \n, \r, \0, delimiter, escape character, and null value character. Ensure that the value of the escape character is entered in the EscapeCharacter attribute. Duplicate Row Handling Netezza ignores trailing spaces. Therefore, when you treat duplicate row handling as first for a Netezza target, the Integration Service might load fewer rows than expected to the target. For example, in the following data ITEM NAME is the primary key: ITEM NAME Flashlight ITEM COLOR Blue Flashlight Red Vest Vest Camoflauge Pink When you treat duplicate row handling as first, the Integration Service loads the following values to the Netezza target: ITEM NAME Flashlight Vest ITEM COLOR Blue Camoflauge Configuring a Netezza Session 27

40 Configuring Update Strategy You can set session properties to specify how the Integration Service inserts, updates, and deletes rows. For a source, you can specify whether the Integration Service inserts, updates, or deletes source rows, or whether it treats rows as data-driven. If you treat source rows as data-driven, you must use an Update Strategy transformation to indicate how the Integration Service handles rows. When you configure row handling, you can treat source rows as updates or data-driven according to the following guidelines: Updates. If you treat source rows as updates, the behavior of the Integration Service depends on the target options you select. Table 4-3 describes how the Integration Service loads the target when you configure the session to treat source rows as updates: Table 4-3. Treating Source Rows as Updates Target Option Update as Update* Description The Integration Service updates all the rows as updates. If Update is set to None, the Integration Service rejects all the rows flagged for update. Update else Insert* The Integration Service updates existing rows and inserts remaining rows flagged as update. If Update is set to None, the Integration Service rejects all rows flagged for update. *The Integration Service updates all rows if you do not select one of the target update options. Data-driven. If you treat source rows as data-driven, you use an Update Strategy transformation to specify how the Integration Service handles rows. Depending on the row type, Integration Service will insert, update or delete the rows from the Netezza targets. However, the behavior of Integration Service also depends on the target options you select. Table 4-4 describes how Integration Service loads the target when you configure the session to treat source rows as data-driven: Table 4-4. Treating Source Rows as Data-Driven Target Option Insert Update as Update* Update else Insert* Description If enabled, the Integration Service inserts all rows flagged for insert. If disabled, the Integration Service rejects the Rows flagged for insert. Default is enabled. The Integration Service updates all the rows flagged for update. If Update is set as None the Integration Service rejects all the rows flagged for update. Default is enabled. The Integration Service updates the existing rows and inserts the remaining rows flagged as update. If Update is set as None the Integration Service rejects all the rows flagged for update. Delete If enabled, the Integration Service deletes all the rows flagged for delete. If disabled, the Integration Service rejects all the rows flagged for delete. *The Integration Service rejects rows flagged for update if you do not select one of the target update options. 28 Chapter 4: Creating and Configuring Netezza Workflows

41 Configuring Partitioning If you want to load a large amount of data to a Netezza target, you can use partitioning. When you configure multiple partitions the Integration Service creates a separate connection to the target for each partition at the target instance. It concurrently loads data for each partition into the target. Use the following rules and guidelines when you configure multiple partitions in a Netezza session: Set the partitioning type to pass-through for Netezza targets. For multiple partitions, configure the target to ignore key constraints. For more information, see Table 4-2 on page 26. Ensure that the Delete and Update attributes on the Targets view on the Mapping tab are enabled for one partition or less than one. If you enable these options for a partition, clear Delete and set Update to None for all other partitions. You cannot perform multiple updates, multiple deletes, or update and delete simultaneously on the same Netezza target table. Configuring Multiple Targets for the Same Target Table You can configure multiple targets to refer data of the same Netezza table. Configure the target to ignore key constraints. For more information, see Table 4-2 on page 26. When you configure multiple targets to write data to the same Netezza table, ensure that the Delete and Update attributes on the Targets view on the Mapping tab are enabled for one target or less than one. If you enable these options for a target, clear Delete and set Update to None for all other targets that write data to the same table. You cannot perform multiple updates, or multiple deletes, or update and delete simultaneously on the same Netezza target table. Configuring Support for NPS 4.0 on Windows For the NPS ODBC driver to work with NPS 4.0, modify the encryption from md5 to crypt on the last line of pg_hba.conf in the /nz/data directory. Configuring a Netezza Session 29