User Guide. PowerCenter Connect for Netezza. (Version )

Transcription

1 User Guide PowerCenter Connect for Netezza (Version )

2 Informatica PowerCenter Connect for Netezza User Guide Version October 2007 Copyright Informatica Corporation. All rights reserved. This software and documentation contain proprietary information of Informatica Corporation, and are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without the prior written consent of Informatica Corporation. Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as provided in DFARS (a) and (a) (1995), DFARS (c)(1)(ii) (OCT 1988), FAR (a) (1995), FAR , or FAR (ALT III), as applicable. Informatica, PowerCenter, PowerCenterRT, PowerExchange, PowerCenter Connect, PowerCenter Data Analyzer, PowerMart, Metadata Manager, Informatica Data Quality and Informatica Data Explorer are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners. U.S. Patent Pending. ICU is Copyright (c) International Business Machines Corporation and others. All rights reserved. Permission is hereby granted, free of charge, to any person obtaining a copy of the ICU software and associated documentation files (the Software ), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so. Copyright Persistent Systems Private Limited. DISCLAIMER: Informatica Corporation provides this documentation as is without warranty of any kind, either express or implied, including, but not limited to, the implied warranties of non-infringement, merchantability, or use for a particular purpose. The information provided in this documentation may include technical inaccuracies or typographical errors. Informatica could make improvements and/or changes in the products described in this documentation at any time without notice.

3 Table of Contents List of Figures v List of Tables vii Preface ix New Features and Enhancements x About This Book xi Document Conventions xi Other Informatica Resources xii Visiting Informatica Customer Portal xii Visiting the Informatica Web Site xii Visiting the Informatica Knowledge Base xii Obtaining Customer Support xii Chapter 1: Understanding PowerCenter Connect for Netezza Overview Integration Service and Netezza Integration Chapter 2: Installation and Configuration Overview Minimum System Requirements Prerequisites Installing and Configuring PowerCenter Connect for Netezza Before You Upgrade Upgrading PowerCenter Connect for Netezza Installing PowerCenter Connect for Netezza Installing the Client Component Installing the Repository Component Installing the Integration Service Component Registering the Plug-in Updating the Plug-in Registering the Plug-in Establishing ODBC Connectivity Configuring ODBC Connectivity on Windows iii

4 Configuring ODBC Connectivity on UNIX Uninstalling PowerCenter Connect for Netezza Uninstalling from Windows Uninstalling from UNIX Changed Behavior Connection Object Type Change Update Strategy Configuration Change Key Type Change Chapter 3: Working with Netezza Sources and Targets Overview Configuring Netezza Source and Target Definitions Steps to Import a Netezza Definition Configuring Source Qualifier Properties Chapter 4: Creating and Configuring Netezza Workflows Configuring Netezza Connections Configuring a Netezza Session Configuring a Session with a Netezza Source Configuring a Session with a Netezza Target Target Database Connection Target Properties Configuring Update Strategy Configuring Partitioning Configuring Multiple Targets for the Same Target Table Configuring a Session for Optimal Performance Troubleshooting Appendix A: Datatypes Datatype Reference Appendix B: Error Messages Client Error Messages Integration Service Error Messages iv Table of Contents

5 List of Figures Figure 1-1. Integration Service and Netezza Performance Server Integration Figure 3-1. SQL Properties Figure 4-1. Netezza Target Properties List of Figures v

6 vi List of Figures

7 List of Tables Table 2-1. Library Variables Table 3-1. SQL Properties Table 4-1. Netezza Source Properties Table 4-2. Netezza Target Properties Table 4-3. Treating Source Rows as Updates Table 4-4. Treating Source Rows as Data-Driven Table A-1. Netezza and Transformation Datatypes List of Tables vii

8 viii List of Tables

9 Preface Welcome to PowerCenter Connect, Informatica s family of packaged software products that helps you extract data and metadata from ERP and other third-party applications. PowerCenter Connect for Netezza is a natural extension to the open architecture of PowerCenter, which supports data extraction from a wide variety of operational data sources. PowerCenter Connect for Netezza extracts data stored in Netezza databases into PowerCenter, leverages it further for effective planning, analysis, and management, and loads data into Netezza database or other targets. ix

10 New Features and Enhancements PowerCenter Connect for Netezza includes the following new features and enhancements: Unicode. PowerCenter has the ability for Unicode data movement for the Integration Service. Bulk reader. PowerCenter Connect for Netezza has the ability to extract bulk data from Netezza Performance Sever and load data into different data sources including flat files, Microsoft Access, Oracle, and Microsoft SQL Server. Netezza load summary. PowerCenter Connect for Netezza displays a load summary report for number of rows applied, rejected, requested and affected in Netezza Load Summary printed in the session log. x Preface

11 About This Book The Informatica PowerCenter Connect for Netezza User Guide provides information about extracting data from a Netezza source and loading data into a Netezza target. It is written for database administrators and developers who are responsible for extracting data from Netezza and loading data to Netezza. This book assumes you have knowledge of Netezza and PowerCenter. Document Conventions This guide uses the following formatting conventions: If you see italicized text boldfaced text italicized monospaced text Note: Tip: Warning: monospaced text bold monospaced text It means The word or set of words are especially emphasized. Emphasized subjects. This is the variable name for a value you enter as part of an operating system command. This is generic text that should be replaced with user-supplied values. The following paragraph provides additional facts. Supplies information, which may apply only in special cases. The following paragraph provides suggested uses. The following paragraph notes situations where you can overwrite or corrupt data, unless you follow the specified procedure. This is a code example. This is an operating system command you enter from a prompt to execute a task. Preface xi

12 Other Informatica Resources In addition to the product manuals, Informatica provides these other resources: Informatica Customer Portal Informatica web site Informatica Knowledge Base Informatica Global Customer Support Visiting Informatica Customer Portal As an Informatica customer, you can access the Informatica Customer Portal site at The site contains product information, user group information, newsletters, access to the Informatica customer support case management system (ATLAS), the Informatica Knowledge Base, Informatica Documentation Center, and access to the Informatica user community. Visiting the Informatica Web Site You can access the Informatica corporate web site at The site contains information about Informatica, its background, upcoming events, and sales offices. You will also find product and partner information. The services area of the site includes important information about technical support, training and education, and implementation services. Visiting the Informatica Knowledge Base As an Informatica customer, you can access the Informatica Knowledge Base at Use the Knowledge Base to search for documented solutions to known technical issues about Informatica products. You can also find answers to frequently asked questions, technical white papers, and technical tips. Obtaining Customer Support There are many ways to access Informatica Global Customer Support. You can contact a Customer Support Center through telephone, , or the WebSupport Service. Use the following addresses to contact Informatica Global Customer Support: support@informatica.com for technical inquiries support_admin@informatica.com for general customer service requests xii Preface

13 WebSupport requires a user name and password. You can request a user name and password at North America / South America Europe / Middle East / Africa Asia / Australia Informatica Corporation Headquarters 100 Cardinal Way Redwood City, California United States Toll Free Standard Rate United States: Informatica Software Ltd. 6 Waltham Park Waltham Road, White Waltham Maidenhead, Berkshire SL6 3TN United Kingdom Toll Free Standard Rate Belgium: France: Germany: Netherlands: United Kingdom: Informatica Business Solutions Pvt. Ltd. Diamond District Tower B, 3rd Floor 150 Airport Road Bangalore India Toll Free Australia: Singapore: Standard Rate India: Preface xiii

14 xiv Preface

15 Chapter 1 Understanding PowerCenter Connect for Netezza This chapter includes the following topic: Overview, 2 1

16 Overview PowerCenter Connect for Netezza provides bidirectional connectivity between Informatica PowerCenter and Netezza to extract and load data. The Designer uses ODBC drivers to connect to the Netezza database. You can import Netezza tables as sources and target definitions. You can connect to Netezza Performance Server to read data from Netezza tables and load data to Netezza tables. Netezza Performance Server integrates database, server, and storage in a single system. Integration Service and Netezza Integration The Integration Service reads and writes data through an external table. It uses the bulk load utility on the external table to extract and load data. This bulk loading utility increases the session performance. Figure 1-1 shows how the Integration Service writes data to the Netezza Performance Server using bulk loading utility: Figure 1-1. Integration Service and Netezza Performance Server Integration Repository Netezza Session Netezza Connection The Integration Service reads and writes data based on session and Netezza relational connection configuration. Integration Service Bulk extract/load using external table Netezza Performance Server The Integration Service reads and writes data through an external table. Configure a Netezza database connection to read data from and write to Netezza. For more information about configuring a Netezza database connection, see Configuring Netezza Connections on page Chapter 1: Understanding PowerCenter Connect for Netezza

17 Chapter 2 Installation and Configuration This chapter includes the following topics: Overview, 4 Installing PowerCenter Connect for Netezza, 6 Registering the Plug-in, 9 Establishing ODBC Connectivity, 10 Uninstalling PowerCenter Connect for Netezza, 13 Changed Behavior, 14 3

18 Overview This chapter provides information about upgrading, installing, and configuring PowerCenter Connect for Netezza. Minimum System Requirements Ensure that the minimum system requirements are met. For more information about minimum system requirements, see the PowerCenter Installation and Configuration Guide. Prerequisites Before you upgrade or install PowerCenter Connect for Netezza, complete the following tasks: Install or upgrade PowerCenter. For more information about the supported PowerCenter versions, see the PowerCenter Connect for Netezza Release Notes. Install client and server components of the Netezza Performance Server. Verify that you have read and write permissions on the following directories: <PowerCenter Installation Directory>\server\bin <PowerCenter Installation Directory>\client\bin Install the Netezza ODBC 3.5 driver on Windows for the PowerCenter Client to fetch the metadata and Netezza ODBC 3.0 driver on the machine hosting the Integration Service to access the data in Netezza Performance Server. The Services installation must be able to add and overwrite files in this directory. Installing and Configuring PowerCenter Connect for Netezza To install and configure PowerCenter Connect for Netezza, complete the following steps: 1. Install the components. Install Client, Repository, and Integration Service components to access the Netezza Performance Server. For more information, see Installing PowerCenter Connect for Netezza on page Register the plug-in. Register the plug-in with the repository. For more information, see Registering the Plug-in on page Establish ODBC connectivity. Establish ODBC connectivity to connect to the Netezza Performance Server with the PowerCenter Client and Integration Service. For more information, see Establishing ODBC Connectivity on page 10. Before You Upgrade Before you upgrade PowerCenter Connect for Netezza, uninstall the previous version. When you uninstall PowerCenter Connect for Netezza, you do not need to unregister the plug-in. 4 Chapter 2: Installation and Configuration

19 Upgrading PowerCenter Connect for Netezza When you upgrade PowerCenter Connect for Netezza, complete the following tasks: 1. Install PowerCenter Connect for Netezza. When you install PowerCenter Connect for Netezza, you install the PowerCenter Client, Server, and Repository components. For more information, see Installing PowerCenter Connect for Netezza on page Update and register the plug-in. Register the PowerCenter Connect for Netezza plug-in to update the existing plug-in registration. For more information, see Registering the Plug-in on page 9. After you upgrade, review the behavior changes for Netezza. For more information, see Changed Behavior on page 14. Overview 5

20 Installing PowerCenter Connect for Netezza Step 1. Install Step 1. Upgrade When you install PowerCenter Connect for Netezza, you install components that allow PowerCenter to access the Netezza Performance Server. Install the following components: Client component. You can import definitions, create mappings, and create connection objects using the PowerCenter Client. Repository component. Allows the Repository Service access to the Netezza Performance Server metadata in the repository. Integration Service component. Allows the Integration Service to run Netezza Performance Server sessions. Install these components on the machines that you want to access Netezza. Installing the Client Component Install the Client component on each PowerCenter Client machine where you want to create or access Netezza Performance Server metadata. To install the Client component: 1. Run installer.exe from the installation package. 2. Choose the Client component. 3. Click Next. 4. Enter your name and organization name. 5. Select the directory where the PowerCenter Client executables are located. By default, these are at the following location: <PowerCenter Installation Directory>\client\bin 6. Click Next to begin the installation. 7. Click OK when the installation is complete. Installing the Repository Component Install the Repository component on any node configured to run a Repository Service process. If the Repository Service is configured to run on primary and backup nodes, install the Repository component on each node configured to run the Repository Service process. 6 Chapter 2: Installation and Configuration

21 To install the Repository component on Windows: 1. Run installer.exe from the installation package. 2. Choose the Repository component. 3. Click Next. 4. Enter your name and organization name. 5. Select the directory where you want to install the PowerCenter Connect for Netezza Repository component. By default, the Repository component is installed in the following location: <PowerCenter Installation Directory>\server\bin\Plugin 6. Click Next to begin the installation. 7. Click OK when the installation is complete. To install the Repository component on UNIX: 1. Go to the Repo directory. 2. Enter./install at the prompt. 3. Install the PowerCenter for Netezza Repository component in the following location: <PowerCenter Installation Directory>/server/bin/Plugin The Repository component is installed. Installing the Integration Service Component If the Integration Service is configured to run on primary and backup nodes, install the Integration Service component on each node configured to run the Integration Service process. If the Integration Service is configured to run on a grid, install the Integration Service component on each node configured to run on the grid. If you cannot install the Integration Service component on each node in the grid, create a resource in the domain and assign it to each node where you installed the Integration Service Component. When you create a session, configure the session to use the resource. For example, create a custom resource called Netezza. When you create a session, you can require the session to use the resource Netezza. The Load Balancer will only dispatch the session to nodes where the Integration Service component is installed. For more information about resources, see the PowerCenter Administrator Guide. To install the Integration Service component on Windows: 1. Run installer.exe from the installation package. 2. Choose the Server component. 3. Click Next. Installing PowerCenter Connect for Netezza 7

22 4. Enter your name and organization name. 5. Select the directory where the Integration Service executables are located. By default, the Integration Service component is installed in the following location: <PowerCenter Installation Directory>\server\bin 6. Click Next to begin the installation. 7. Click OK when the installation is complete. To install the Integration Service component on UNIX: 1. Go to the Server directory. 2. Enter./install at the prompt. 3. Install the Integration Service executables in the following location. <PowerCenter Installation Directory>/server/bin The Integration Service component is installed. 8 Chapter 2: Installation and Configuration

23 Registering the Plug-in Step 2. Install Step 2. Upgrade When you install PowerCenter Connect for Netezza, register the plug-in with the repository. A plug-in is an XML file that defines the functionality of PowerCenter Connect for Netezza. When you upgrade from PowerCenter Connect for Netezza 1.1 SP 5 or PowerCenter Connect for Netezza 1.2, perform the following tasks: 1. Update the plug-in using the pmrep fixnetezzaplugin command. 2. Register the plug-in using the pmrep RegisterPlugin command. To run the pmrep commands to upgrade and register the plug-in, connect to the repository and run the Repository Service in exclusive mode. If you do not have the Administration Console privilege on the Repository Service or the Admin Repository privilege on the repository, you can ask the Repository Service administrator to register the plug-in. Updating the Plug-in Update the plug-in when you upgrade. The fixnetezzaplugin pmrep command updates the Netezza plug-in in the repository. The fixnetezzaplugin command uses the following syntax: fixnetezzaplugin Registering the Plug-in Register the plug-in when you install or upgrade. The RegisterPlugin pmrep command registers the Netezza plug-in with the repository. The plug-in file for PowerCenter Connect for Netezza is Netezza.xml. When you install the Repository component, the installer copies Netezza.xml to the following directory: <PowerCenter Installation Directory>/server/bin/Plugin The RegisterPlugin command uses the following syntax: registerplugin -i <input_registration_file_name_or_path> -e The plug-in file for PowerCenter Connect for Netezza is Netezza.xml. When you install the Repository component, the installer copies Netezza.xml to the following directory: <PowerCenter Installation Directory>/server/bin/Plugin For more information about using the Administration Console to register a plug-in, see the PowerCenter Administrator Guide. For more information about the pmrep RegisterPlugin command, see the PowerCenter Command Line Reference. Registering the Plug-in 9

24 Establishing ODBC Connectivity Step 3. Install The Designer uses ODBC drivers to import metadata from the Netezza Performance Server. The Integration Service uses ODBC to read or write data to Netezza Performance Server. You specify an ODBC data source in the Designer when you import a Netezza definition. You specify an ODBC data source in the Workflow Manager when you create a Netezza relational database connection object. Configuring ODBC Connectivity on Windows To establish ODBC connectivity for the PowerCenter Client and Integration Service on Windows, create an ODBC data source on each PowerCenter Client machine that you want to access Netezza metadata. Also, create an ODBC data source on each node on which the Integration Service is configured to run. You configure the ODBC connection from the Control Panel. Use the NetezzaSQL driver for the ODBC connection. The NetezzaSQL driver installs with Netezza Performance Server client component. Note: Install NetezzaSQL drivers on PowerCenter Client and Integration Service machines. Configuring ODBC Connectivity on UNIX To establish ODBC connectivity for the Integration Service on UNIX, complete the following steps on each machine configured to run the Integration Service: 1. Create an ODBC data source. Configure the.odbc.ini file to create a Netezza Performance Server ODBC data source. 2. Set the environment variable. Set the environment variable to the ODBC driver manager. For more information, see Table 2-1 Library Variables on page 12. Creating an ODBC Data Source To create an ODBC data source, you create and configure a file called.odbc.ini. You add fields to the file that contain the connection information the Integration Service requires to connect to the Netezza Performance Server. Note: Place the.odbinst.ini file in the same directory as.odbc.ini. 10 Chapter 2: Installation and Configuration

25 To create an ODBC data source: 1. Open the.odbc.ini file in the $HOME directory. If the file does not exist in the $HOME directory, create it. 2. Add the following entries to the.odbc.ini file: Field Required/ Optional Description [DSN] Required Data source name used to identify database connection properties. Driver Required Path where the Netezza ODBC driver is installed. Description Required Enter a description of the NetezzaSQL ODBC data source. Servername Required Host on which the Netezza Performance Server is running. Port Required Port number on which the Netezza Performance Server is running. Database Required Database of external or target table. Username Required User name for the database. Password Required Password for the database. ReadOnly Optional Enter True or False. Only Select SQL statement will be allowed in case set to True. Other SQL statements will cause an error. ShowSystemTables Optional Enter True or False. When displaying available table, system tables should be included in the list. Optimizer Optional Enter True or False. This switch controls the Keyset Query Optimization. The following example shows fields in an.odbc.ini file to connect to Netezza Performance Server: [NZSQL] driver=/usr/local/nz/lib/libnzsqlodbc3.so Description=NetezzaSQL ODBC Servername=nzdev Port=5480 Database=dev Username=admin Password=pass ReadOnly=False ShowSystemTables=False Optimizer=False Setting the Environmental Variable to the ODBC Driver Manager Set the library path to the ODBC driver manager. When you set the library path, use the appropriate environment variable for each operating system. Establishing ODBC Connectivity 11

26 Table 2-1. describes environment variable for the library path on each operating system: Table 2-1. Library Variables Operating System Solaris Linux AIX HP-UX Variable LD_LIBRARY_PATH LD_LIBRARY_PATH LIBPATH SHLIB_PATH For example, use the following syntax on Solaris: export LD_LIBRARY_PATH:<PowerCenter Installation Directory>/ ODBC 5.1/lib:$LD_LIBRARY_PATH 12 Chapter 2: Installation and Configuration

27 Uninstalling PowerCenter Connect for Netezza To uninstall PowerCenter Connect for Netezza, uninstall it from each machine where you installed the Client, Repository, or Integration Service components. Uninstalling from Windows To uninstall PowerCenter Connect for Netezza from Windows, use the Windows Add/ Remove Programs. Uninstalling from UNIX You can uninstall the PowerCenter Connect for Netezza Repository and Integration Service components from UNIX. To uninstall the Repository component from UNIX: 1. Enter./install at the prompt. 2. Enter the absolute path to the installation directory. By default, the Repository component is in the following location: <PowerCenter Installation Directory>/server/bin/Plugin If PowerCenter Connect for Netezza is installed in this directory, the installer prompts you to overwrite the existing installation or uninstall the Repository component. 3. Use the uninstall option to uninstall the Repository component. To uninstall the Integration Service component from UNIX: 1. Enter./install at the prompt. 2. Enter the absolute path to the installation directory. By default, the Integration Service is in the following location: <PowerCenter Installation Directory>/server/bin/ 3. When the installer prompts you to overwrite the existing installation or uninstall the Integration Service component, choose to uninstall the Integration Service component. Uninstalling PowerCenter Connect for Netezza 13

28 Changed Behavior This section describes the behavior changes for PowerCenter Connect for Netezza from previous versions. Connection Object Type Change Upgrade Impact: Behavior Change, Upgraded Metadata Version Implemented: 1.2 Versions Affected: 1.1 SP5 Effective in version 1.2 for PowerCenter Connect for Netezza, the Netezza connection object type is changed from Application to Relational. The Netezza application connections you created in a previous version upgrade to Netezza relational database connections. Update Strategy Configuration Change Upgrade Impact: Behavior Change, Upgraded Metadata Version Implemented: 1.2 Versions Affected: 1.1 SP5 Effective in version 1.2 for PowerCenter Connect for Netezza, the options to configure update strategy are changed in the session properties. You can select Insert, Delete, and Update as Update, or Update Else Insert. In a previous version you could choose Insert, Update, Delete and Update Else Insert. When you upgrade, the sessions enable to Update are upgraded to Update as Update. Key Type Change Upgrade Impact: Upgraded Metadata Version Implemented: 1.2 Versions Affected: 1.1 SP5 Effective in version 1.2 for PowerCenter Connect for Netezza , the key types are changed to Primary Key and Foreign Key. In a previous version, the key types were Key or No Key. When you upgrade, the field in a source or target definition configured as the key upgrades to the primary key. 14 Chapter 2: Installation and Configuration

29 Chapter 3 Working with Netezza Sources and Targets This chapter includes the following topics: Overview, 16 Configuring Netezza Source and Target Definitions, 17 Configuring Source Qualifier Properties, 19 15

30 Overview Netezza source and target definitions represent metadata for the Netezza tables. You can use the Designer to import Netezza source and target definitions into the PowerCenter repository. When you import Netezza source and target definitions, you can choose to preview data in the tables. When the Integration Service extracts from Netezza it uses the bulk extract functionality of the external tables. When the Integration Service loads to Netezza, it uses the bulk loading functionality of the external tables. 16 Chapter 3: Working with Netezza Sources and Targets

31 Configuring Netezza Source and Target Definitions You can define a Netezza source or target by importing a Netezza table from the Netezza database. To import a Netezza table from the Netezza database, use the Import Netezza Table dialog box. You need to specify an ODBC data source in the Designer when you import a Netezza definition. You can manually edit source and target definitions to record the properties that you cannot import from Netezza. You can edit a Netezza definition to create key columns and key relationships. These relationships can be logical relationships. They do not have to exist in the database. To insert with primary key validation, update, or delete data when you write to Netezza, you must define at least one primary key in the target definition. Configure the key type on the Columns tab of the target definition. Steps to Import a Netezza Definition To import a Netezza source or target definition: 1. In the Source Analyzer, click Sources > Import Netezza. -or- In the Target Designer, click Targets > Import from Netezza. 2. Select an ODBC data source from the list. 3. Enter a Netezza user name and password. 4. Click Connect to connect to Netezza. A list of tables appears. Configuring Netezza Source and Target Definitions 17

32 5. Optionally, select a table from the list and click Preview Data to preview the data in the selected table. The Preview Data dialog box appears. You can see the data for all the columns in the selected table. 6. Click OK. 7. Select the tables you want to import. Click OK. 18 Chapter 3: Working with Netezza Sources and Targets

33 Configuring Source Qualifier Properties You can define the properties for the Application Source Qualifier associated with a Netezza source definition. You can configure properties, for number of input ports to sort, and to retrieve distinct data from a Netezza source. Figure 3-1 shows the SQL Properties tab: Figure 3-1. SQL Properties Table 3-1 describes the SQL properties: Table 3-1. SQL Properties Source Options Required/ Optional Description Select Distinct Optional Selects unique rows. The Integration Service includes a SELECT DISTINCT clause if you choose Select Distinct option. Source Filter Optional Filter condition. The Integration Service applies when reading source data. Use the following syntax: <Tablename>.<field_name> <operator> <value>. User Defined Join Optional Specifies the condition used to join data from multiple sources represented in the Application Source Qualifier transformation. Configuring Source Qualifier Properties 19

34 Table 3-1. SQL Properties Source Options Number of Sorted Ports Required/ Optional Optional Description A positive number to indicate the number of columns used when sorting rows queried from the source. If you enter a positive value, the Integration Service adds an ORDER BY to the default query when it reads source rows. The ORDER BY includes the number of ports specified, starting from the top of the transformation. When you specify the number of sorted ports, the database sort order must match the session sort order. Default is 0. SQL Query Optional Overrides the default query. 20 Chapter 3: Working with Netezza Sources and Targets

35 Chapter 4 Creating and Configuring Netezza Workflows This chapter includes the following topics: Configuring Netezza Connections, 22 Configuring a Netezza Session, 24 Configuring a Session for Optimal Performance, 31 Troubleshooting, 32 21

36 Configuring Netezza Connections Before you can run a Netezza session, you must configure a Netezza database connection for the Netezza sources or targets in the Workflow Manager. When you configure a Netezza database connection, you specify connection attributes that the Integration Service uses to connect to Netezza Performance Server. To configure Netezza relational connections: 1. In the Workflow Manager, connect to a PowerCenter repository. 2. Click Connections > Relational. The Relational Connection Browser dialog box appears. 3. Click New. The Select Subtype dialog box appears. 4. Select Netezza from the Select Subtype list. 5. Click OK. The Connection Object Definition dialog box appears. 22 Chapter 4: Creating and Configuring Netezza Workflows

37 6. Enter the following information in the Connection Object Definition dialog box: Connection Attribute Required/ Optional Description User Name Required User name to connect to Netezza. Password Required Password for the user name. Connect String Required The ODBC data source to connect to Netezza Performance Server. Code Page Required Code page associated with Netezza Performance Server. Environmental SQL n/a The Integration Service ignores this value. 7. Click OK. The relational connection appears in the Relational Connection Browser dialog box. Configuring Netezza Connections 23

38 Configuring a Netezza Session Before you can run a Netezza session, the Integration Service must connect to Netezza Performance Server. Netezza database connections must exist in the repository to appear on the source or target database list. You must define them prior to configuring a session. For more information about configuring a database connection, see Configuring Netezza Connections on page 22. Configuring a Session with a Netezza Source You can configure the session properties for Netezza source on the Mapping tab. Define the properties for each source instance in the session. When you click on the Mapping tab, you can view and configure the settings of a session associated with a mapping containing a Netezza source definition. Select the source under the Source node and specify the value for the following session to override the corresponding values in the Application Source Qualifier that are available only at the session level. Table 4-1 describes the session properties: Table 4-1. Netezza Source Properties Source Options Required/ Optional Description Delimiter Optional Set the delimiter to any value supported by Netezza Performance Server. The delimiter separates successive input fields. Ensure that the value that you enter in this field should not be a part of the data. Default is. NullValue Optional Configure the NullValue parameter of an external table. Maximum is four characters. If you set the NullValue as a for Char or Varchar field, the Integration Service replaces all the null values with a. Default is blank. EscapeCharacter Optional Configure the escape character of an external table. If the data contains NULL, CR, and LF characters in the Char or Varchar field, and you need to escape these characters in the source data before extracting, enter a escape character. Configuring a Session with a Netezza Target You can configure target properties for a session that writes data to Netezza targets: Target properties. You can define target properties, such as target update options and reject options. For more information, see Target Properties on page 25. Pipeline partitioning. You can create multiple partitions in a session to load target data concurrently to the target. For more information, see Configuring Partitioning on page 29. Update strategy. Configure the update strategy to determine how the Integration Service inserts, updates, and deletes rows. For more information, see Configuring Update Strategy on page Chapter 4: Creating and Configuring Netezza Workflows

39 Multiple targets referring to the same table. You can configure multiple targets to refer data of the same Netezza table. For more information, see Configuring Multiple Targets for the Same Target Table on page 29. Session attributes. You can configure session attributes, such as EscapeCharacter and NullValue. For more information, see Table 4-1, Netezza Source Properties, on page 24. Target Database Connection You can choose the Netezza target connection in the Transformations view of the Mapping tab. Click either the Targets or Connections node and select the Netezza database connection from the list for each target instance. You must have the read permission for the target database connection to configure the session to use it. The user starting the configured session must have the execute permission for target database connections. You can switch the writer type to Netezza writer for all the mappings that include relational targets. However, for the mappings that include Netezza targets, you cannot change the writer type to relational or file. Target Properties You can configure the session properties for Netezza targets in the Transformations view on the Mapping tab. Define the properties for each target instance in the session. Configuring a Netezza Session 25

40 Figure 4-1 shows the Netezza target properties you define in the Properties settings on the Mapping tab: Figure 4-1. Netezza Target Properties 26 Chapter 4: Creating and Configuring Netezza Workflows

41 Table 4-2 describes the target properties available on the Mapping tab: Table 4-2. Netezza Target Properties Target Property Error Log Directory Name Required/ Optional Optional Description The error log directory can reside on the machine where the Integration Service is running. For example, you can enter the following path: /home/persist/errorlogdir If you do not specify any value, the error log is created in the /tmp directory on the machine hosting NPS. The Integration Service creates a bad file in the error log directory if the data is not valid. Insert Optional The Integration Service inserts all rows flagged for insert. Default is enabled. Delete Optional The Integration Service deletes all rows flagged for delete. Default is enabled. Update (as Update) Update (else Insert) Truncate Target Table Option Optional Optional Optional The Integration Service updates all rows flagged for update. Default is enabled. For more information about Updates, see Table 4-3 on page 28. The Integration Service updates rows flagged for update if they exist in the target, then inserts any remaining rows marked for update. Default is disabled. For more information about Updates, see Table 4-3 on page 28. The Integration Service truncates the target before loading. Default is disabled. Delimiter Optional Set the delimiter to any value supported by the Netezza Performance Server. The delimiter separates successive input fields. Ensure that the value that you enter in this field should not be a part of the input data. Default is Control Character Optional Configure the CTRLCHARS parameter of external table to transfer data containing control characters. You can enter control characters for Char and Varchar fields. If you enter a control character, you must escape NULL, CR, and LF fields. Default is TRUE. CRINSTRING Optional Set the CRINSTRING parameter to transfer data containing carriage returns (CR). You can enter a non escape CR in Char or Varchar fields. To load the control characters present in the Char and Varchar fields, set the CTRLCHARS and CRINSTRING parameters to TRUE in the session properties for the Netezza source. Default is TRUE. NullValue Optional Configure the NullValue parameter of an external table. Maximum is four characters. If you set the NullValue as a, the Integration Service will treat a data as NULL. Default is empty. EscapeCharacter Optional Configure the escape character of an external table. If the data contains NULL, CR, and LF characters in the Char or Varchar field, and you need to escape these characters before loading, enter a escape character. Configuring a Netezza Session 27

42 Table 4-2. Netezza Target Properties Target Property Required/ Optional Description Quoted Value Optional You can configure the QUOTEDVALUE parameter of an external table. You can select SINGLE or DOUBLE to enclose the field in single or double quotes. Select NO to omit quotes. Default is NO. The quoted value is not a part of the data for this property. Ignore Key Constraints Optional Ignores constraints on primary key fields. When you select this option, the Integration Service can write duplicate rows with the same primary key to the target. Default is disabled. Configuring Update Strategy You can set session properties to specify how the Integration Service inserts, updates, and deletes rows. For a source, you can specify whether the Integration Service inserts, updates, or deletes source rows, or whether it treats rows as data-driven. If you treat source rows as data-driven, you must use an Update Strategy transformation to indicate how the Integration Service handles rows. For more information about the Update Strategy transformation, see Update Strategy Transformation in the PowerCenter Transformation Guide. When you configure row handling, you can treat source rows as updates or data-driven according to the following guidelines: Updates. If you treat source rows as updates, the behavior of the Integration Service depends on the target options you select. Table 4-3 describes how the Integration Service loads the target when you configure the session to treat source rows as updates: Table 4-3. Treating Source Rows as Updates Target Option Update as Update* Description The Integration Service updates all the rows as updates. If Update is set to None, the Integration Service rejects all the rows flagged for update. Update else Insert* The Integration Service updates existing rows and inserts remaining rows flagged as update. If Update is set to None, the Integration Service rejects all rows flagged for update. *The Integration Service updates all rows if you do not select one of the target update options. Data-driven. If you treat source rows as data-driven, you use an Update Strategy transformation to specify how the Integration Service handles rows. Depending on the row type, Integration Service will insert, update or delete the rows from the Netezza targets. However, the behavior of Integration Service also depends on the target options you select. 28 Chapter 4: Creating and Configuring Netezza Workflows

43 Table 4-4 describes how Integration Service loads the target when you configure the session to treat source rows as data-driven: Table 4-4. Treating Source Rows as Data-Driven Target Option Insert Update as Update* Update else Insert* Description If enabled, the Integration Service inserts all rows flagged for insert. If disabled, the Integration Service rejects the Rows flagged for insert. Default is enabled. The Integration Service updates all the rows flagged for update. If Update is set as None the Integration Service rejects all the rows flagged for update. Default is enabled. The Integration Service updates the existing rows and inserts the remaining rows flagged as update. If Update is set as None the Integration Service rejects all the rows flagged for update. Delete If enabled, the Integration Service deletes all the rows flagged for delete. If disabled, the Integration Service rejects all the rows flagged for delete. *The Integration Service rejects rows flagged for update if you do not select one of the target update options. For more information about the Update Strategy transformation, see Update Strategy Transformation in the PowerCenter Transformation Guide. Configuring Partitioning If you want to load a large amount of data to a Netezza target, you can use partitioning. When you configure multiple partitions the Integration Service creates a separate connection to the target for each partition at the target instance. It concurrently loads data for each partition into the target. Use the following rules and guidelines when you configure multiple partitions in a Netezza session: Set the partitioning type to pass-through for Netezza targets. For multiple partitions, configure the target to ignore key constraints. For more information, see Table 4-2 on page 27. Ensure that the Delete and Update attributes on the Targets view on the Mapping tab are enabled for one partition or less than one. If you enable these options for a partition, clear Delete and set Update to None for all other partitions. You cannot perform multiple updates, multiple deletes, or update and delete simultaneously on the same Netezza target table. For more information about partitioning, see the PowerCenter Workflow Administration Guide. Configuring Multiple Targets for the Same Target Table You can configure multiple targets to refer data of the same Netezza table. Configure the target to ignore key constraints. For more information, see Table 4-2 on page 27. Configuring a Netezza Session 29

44 When you configure multiple targets to write data to the same Netezza table, ensure that the Delete and Update attributes on the Targets view on the Mapping tab are enabled for one target or less than one. If you enable these options for a target, clear Delete and set Update to None for all other targets that write data to the same table. You cannot perform multiple updates, or multiple deletes, or update and delete simultaneously on the same Netezza target table. 30 Chapter 4: Creating and Configuring Netezza Workflows

45 Configuring a Session for Optimal Performance You can increase the performance of PowerCenter Connect for Netezza by setting the properties in the session. Set the following parameters to increase the session performance: Default buffer block size. You can increase or decrease the number of available memory blocks that are used to hold the source and target data in the session. Line sequential buffer length. You can improve the session performance by setting the number of bytes the Integration Service reads per line. Commit interval. You can increase or decrease the value of Commit Interval to determine the point at which the Integration Service commits data to the target. DTM buffer size. You can increase or decrease the value of DTM Buffer Size to specify the amount of memory the Integration Service uses as DTM buffer memory. For more information about these parameters see Optimizing Sessions in the PowerCenter Performance Tuning Guide. For example, to obtain optimal performance for 3 million rows and 32 KB row size, set the following parameters: Default buffer block size - 1,280,000 Line sequential buffer length - 202,400 Commit interval - 200,000 DTM buffer size - 28,000,000 Configuring a Session for Optimal Performance 31

46 Troubleshooting The session fails with the following error: NO KEY FIELD FOUND. You may not have specified the key fields for the Netezza target table in the Designer during update, delete, and insert operations. To perform these operations, you need to identify the key fields for the Netezza target table. Update, delete, and upsert are performed based on the key field comparison. For more information about specifying the key fields, see Configuring Source Qualifier Properties on page 19. The session fails with the following error: Failed to execute CREATE EXTERNAL TABLE query. This can happen in the following circumstances: You did not specify the absolute path for the error log directory name. The error log directory resides on the machine hosting the Netezza Performance Server or on the machine where the Integration Service is running. For example, you can enter the following path: /home/persist/errorlogdir PowerCenter Connect for Netezza creates a bad file in the error log directory if the data is invalid. You do not have sufficient database privileges. Contact your database administrator. When creating a DSN on Windows, you may not have cleared the ReadOnly checkbox in the advanced options of the ODBC data source administrator. The session fails with the following error: SQL Connect failed. On Windows, this error occurs if you specify an invalid server name, database name, user name, or password while creating a DSN. On UNIX, this error occurs if you specify an incorrect Netezza ODBC driver path, or an invalid server name, database name, user name, or password in the odbc.ini file. The session fails with the following error: Could not serialize - transaction aborted. This is a Netezza limitation. This error occurs when you try to perform an update, delete, or upsert simultaneously on the same table in Netezza. You can perform an insert and update or insert and delete on the same table simultaneously but not an update and delete. Perform these operations separately. The session fails with the following error: Limit of concurrent read/write transactions reached, update operations not permitted. This is a Netezza limitation. This happens when you try to perform more than 128 transactions concurrently in Netezza. You can perform a maximum of 128 transactions simultaneously for version greater than or equal to 3.1.x and a maximum of 31 transactions for version lesser than or equal to 3.0.x in Netezza. 32 Chapter 4: Creating and Configuring Netezza Workflows