How to Convert an SQL Query to a Mapping

Transcription

1 How to Convert an SQL Query to a Mapping 2014 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. All other company and product names may be trade names or trademarks of their respective owners and/or copyrighted materials of such owners.

2 Abstract This article explains how to convert SQL queries to Model repository mappings so that you can use the mappings in the Developer tool and PowerCenter. Supported Versions Informatica Data Services HotFix 4 PowerCenter HotFix 4 Table of Contents Supported Versions... 2 Table of Contents... 2 Overview... 2 infacmd sql preparemapping Command... 2 Converting SQL Query to a Model Repository Mapping... 3 Converting the Model Repository Mapping to a PowerCenter Mapping... 4 Example... 4 Converting the SQL Query to a Model repository mapping... 5 Restrictions... 7 Troubleshooting Tips... 7 Overview You can use the infacmd sql preparemapping command to reduce the amount of time it takes to convert complex SQL queries into mappings in PowerCenter or Informatica Developer. The infacmd sql preparemapping command converts an SQL query to a Model repository mapping. You can either import the mapping file into the Model repository or convert the mapping to a PowerCenter mapping with the infacmd ipc exporttopc command. You can then import the PowerCenter mapping file into the PowerCenter repository. infacmd sql preparemapping Command Converts an SQL query to a Model repository mapping. You can either enter the SQL query as a command parameter or enter the SQL query in a text file and provide the text file name as the command parameter. You can convert only one SQL query at a time. The infacmd sql preparemapping command uses the following syntax: preparemapping <-ConnectString -cs> connection_string [<-sql> sql_statement] [<-sf> sql_file] [<-tl> target_location] [<-convertmappingtoldo> -cml> convert_mapping_to_ldo] The following table describes infacmd sql preparemapping options and arguments: Option Argument Description -ConnectString connection_string Required. Enter an SQL data service connect string with the following 2

3 -cs syntax: jdbc:informatica:sqlds/<optional security domain\><optional user name>/<optional user host name>:<domain HTTP port>?dis=<data Integration Service name>&sqlds=<runtime SQL data service name> -sql sql_statement Optional. Enter the SQL query that you want to convert to a Model repository mapping. -sf Sql_file Optional. Enter the file name that contains the SQL query that you want to convert to a Model repository mapping. If you do not enter a statement or file name, the command prompts you to enter the SQL query after you run the command. -tl target_location Optional. Enter the target location for the Model repository mapping file. If you do not enter the fully qualified file path, the command creates the mapping file in the directory from which you run the command. -convertmappingtoldo -cml convert_mapping_to_ldo Optional. If you enter the option, the command converts the mapping to a logical data object. Converting SQL Query to a Model Repository Mapping To convert an SQL query to a Model repository mapping, perform the following steps: 1. In the Developer tool, create a connection to the relational data source that is used in the SQL query. 2. Create a physical data object or a logical data object with the connection in the Model Repository Service. 3. Deploy the physical data object or the logical data object as an SQL data service. Copy the JDBC connection string of the deployed SQL data service from the Administrator tool. 4. Remove unwanted spaces and new line characters from the SQL query if you use the sql option. You need can retain the query as is if you use the sf option. 5. Run the infacmd sql preparemapping command. The command creates a Model repository mapping file that contains the Model repository mapping corresponding to the SQL statement. 6. Import the mapping file to the Model Repository Service through the Developer tool. 7. Open the mapping in the Developer tool. 8. Edit the mapping to remove the output transformations and add the correct targets for the mapping. You must assign connections for the data sources and lookups. You may need to add transformations or edit the mapping to include any logic that the command did not convert. 3

4 Converting the Model Repository Mapping to a PowerCenter Mapping Run the infacmd ipc exporttopc command to convert the mapping file to the PowerCenter format. You must set the environment variable INFA_PARTIAL_EXPORT as true before you run the command. You can use the optimization option to optimize the mapping to reduce the complexity. By default, optimization is not applied while converting the mapping. Optionally, you can apply the following optimization levels: Optimization Level 1 (-o1). The command applies early projection optimization in the mapping. It identifies unused ports and removes the links between those ports. Early projection optimization improves performance by reducing the amount of data that the Integration Service moves across transformations. Optimization Level 2 (-o2). The command applies early projection, early selection, push-into, pushdown, and predicate optimization methods. Early projection optimization improves performance by reducing the amount of data that the Integration Service moves across transformations. Pushdown optimization method pushes transformation logic to the source database. Predicate optimization method examines the predicate expressions that a mapping generates. It determines whether it can simplify or rewrite the expressions to increase mapping performance. Early selection optimization method splits, moves, or removes the Filter transformations in a mapping. It moves the filters closer to the source definition. Push-into optimization moves the Filter transformation logic into the transformation immediately upstream of the Filter transformation in the mapping. Push-into optimization increases performance by reducing the number of rows that pass through the mapping. After you run the command, import the PowerCenter mapping into the PowerCenter repository. Open the mapping in Designer and add the proper target definitions. You may also need to edit the mapping to include any logic that the command did not convert. Example ABC Dealers uses the database tables, CS_FIN_C_PL_LKP_PC_CODE, CS_FIN_C_PL_PDE_RAW, and VW_CS_FIN_C_LKP_DIST_CC, to store data on sales and profitability for different regions. The following SQL query joins the tables and creates an output for the data warehouse: SELECT NULL PLACE, TABLENAME.label, TABLENAME.TABLENAME, TABLENAME.DATE_WEEKLY, (TABLENAME.C_AREA TABLENAME.PC_ID) C_AREA_NUMBER, (TABLENAME.TABLENAME / RESULTANT_TABLE.THE_C_AREA_SUM) TABLENAME_ALL, TABLENAME.C_AREA AREA, TABLENAME.PC_ID, 'LOAD_SRT' LOAD_BY, CAST(TO_CHAR(CURRENT_DATE, 'YYYYMMDDHHMISS') AS INTEGER) TIME_STAMP, 'UPDATE_STR' UPDATED_BY FROM ( SELECT TABLENAME.label, TABLENAME.DATE_WEEKLY, tab4.c_area, TABLE2.PC_ID, SUM(TABLENAME.TABLENAME) TABLENAME FROM TABLE3 TABLENAME JOIN TABLE4 tab4 ON 4

5 TABLENAME.PLACE = tab4.place_id LEFT JOIN TABLE5 TABLE2 ON TABLENAME.label = TABLE2.PC_CODE_D WHERE TABLENAME.DATE_WEEKLY = CURRENT_DATE GROUP BY TABLENAME.label, TABLENAME.DATE_WEEKLY, tab4.c_area, TABLE2.PC_ID ) TABLENAME LEFT JOIN ( SELECT TABLENAME.DATE_WEEKLY, tab4.c_area, SUM(TABLENAME.TABLENAME) THE_C_AREA_SUM FROM TABLE3 TABLENAME JOIN TABLE4 tab4 ON TABLENAME.PLACE = tab4.place_id GROUP BY TABLENAME.DATE_WEEKLY, tab4.c_area ) RESULTANT_TABLE ON TABLENAME.DATE_WEEKLY = RESULTANT_TABLE.DATE_WEEKLY AND TABLENAME.C_AREA = RESULTANT_TABLE.C_AREA ORDER BY TABLENAME.label, TABLENAME.C_AREA You want to convert the SQL query as a Model repository mapping and a PowerCenter mapping. The SQL query is available in the file, querywithjoins.sql. Converting the SQL Query to a Model repository mapping To convert the SQL query to a Model repository mapping, perform the following steps: 1. Create physical data objects for table3, table4, and table5, in the developer tool. 2. Deploy the physical data objects in the Data Integration Service. The deployed data service has the following JDBC URL: jdbc:informatica:sqlds/administrator/administrator@irl58pqa15:9005?dis=dis_e xample&sqlds=application.sql_data_service 3. Remove unwanted spaces and new line characters from the SQL query if you enter SQL query along with the command. Skip this step if you enter the SQL file as the parameter. 4. Run the infacmd sql preparemapping command. Enter the following command: infacmd sql preparemapping -cs "jdbc:informatica:sqlds/administrator/administrator@irl58pqa15:9005?dis=dis_ example&sqlds=application.sql_data_service" sf querywithjoins.sql -tl c:\demomapping\devmapping.xml The command creates the mapping file, devmapping.xml, in the following folder: C:\DemoMapping 5

6 5. Import the mapping file into the Model repository with the Developer tool. The following image shows the mapping in the Developer tool: 6. Run the infacmd ipc exporttopc command to convert the Model repository mapping to a PowerCenter mapping. Run the following command: infacmd ipc ExportToPC rel sf devtoolmapping.xml tl PCmapping.xml The following image shows the PowerCenter mapping: You can run the command with different optimization levels. The following image shows the PowerCenter mapping with optimization level 1: The following image shows the PowerCenter mapping with optimization level 2: 6

7 Restrictions The following restrictions exist when you convert an SQL query to a Model repository mapping in Informatica HotFix 4: Only SQL statements that follow SQL-92 standard is supported. INSERT INTO statement is not supported. If the SQL query contains an INSERT INTO <table name> clause, remove the clause and run the command. The following functions are not supported in expressions : - OCTET_LENGTH - BIT_LENGTH - WIDTH_BUCKET - SYSDATE (You can modify SYSDATE to use CURRENT_DATE) The infacmd sql preparemapping command fails if the SQL query contains unsupported statements or expressions. The following restrictions exist when you convert a Model repository mapping that you converted from an SQL query to a PowerCenter mapping: The Boolean value, UNKNOWN, is not supported. Sub-queries cannot exist without a join condition. Expressions are not supported in join conditions. For example, you cannot use ltrim(port1) = rtrim(port2). You can use port1= port2. DISTINCT, statements are not supported. Queries that contain outer joins are not converted in PowerCenter mappings if you run the preparemapping command with the converttoldo option. Troubleshooting Tips The following error occurs when you run the exporttopc command even when there are no multiple sources: [Converters_037] A conflict exists in the object of type [SOURCE_TYPE] with the following name: [NAME]. Run the exporttopc command again. If the error occurs again, you can try running the exporttopc command with optimization option set to 1. A join condition is invalid when you run the exporttopc command with optimization set as 2 even after you enter an equi-join condition. If your mapping has joiner followed by a Filter transformation, remove the Filter transformation from the Developer tool and try running the command again. You can add the Filter transformation from the Designer after you successfully convert the mapping. You can also try running the exporttopc command with optimization set as 1. The following warning appears when you run the exporttopc command: "Warning: Only 1 or 2 optimization Level allowed. Using No Optimization" If you enter an incorrect optimization level value, the command does not use any optimization while converting the mapping. The following warning appears when you run the exporttopc command: "Warning: No Mapping Object Found. Converting rest of the objects" You may have specified the convertmappingtoldo option when you ran the preparemapping command. The mapping contains a logical data object and it cannot be converted to the PowerCenter format. The output PowerCenter mapping XML will contain the rest of the objects in the query. 7

8 Author Narayan Sivaramakrishnan Senior Technical Writer Acknowledgements The author would like to acknowledge Peeyush Agrawal, Software Engineer, for the guidance while writing the article. 8