Aggregate Data in Informatica Developer

Transcription

1 Aggregate Data in Informatica Developer 2013 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. All other company and product names may be trade names or trademarks of their respective owners and/or copyrighted materials of such owners.

2 Abstract You can use an Aggregator transformation to perform aggregate calculations on data such as averages and sums. To increase mapping performance, sort data for the Aggregator transformation. This article describes how customers can use a Sorter and an Aggregator transformation to perform aggregate calculations on groups of sorted data to get information for business analysis. Supported Versions Informatica Data Services 9.x Informatica Data Quality 9.x PowerCenter Express Table of Contents Overview Scenario Source Data Sorter Transformation Configuration Aggregator Transformation Configuration Preview Transformation Output Overview You can use an Aggregator transformation to perform aggregate calculations on data such as averages and sums. An Aggregator transformation can perform calculations on groups of data. To increase mapping performance, you can sort the input data. When you sort the input data before aggregation, the Data Integration Service performs the aggregate calculations as it reads rows for a group. Use the Sorted Input option in the Aggregator transformation. The Sorted Input option reduces the amount of data cached and increases performance. You can design a mapping with a Sorter and an Aggregator transformation to perform aggregate calculations on groups of sorted data to get information for business analysis. Scenario HypoStores is a corporation that wants to calculate the total daily sales for each customer from the sales transaction entries. You can design a mapping that sorts the input sales transaction entries using a Sorter transformation. After you sort the input data, you can calculate the total daily sales using an Aggregator transformation. The sort key ports in the Sorter transformation and the group-by ports in the Aggregator transformation must be identical. In this scenario, the ports are customer number and transaction date. 2

3 The following figure shows a mapping that takes the sales transaction data and transforms the data to calculate the total daily sales for each customer: Source Data The input source data contains sales transaction entries. The entries are in random order. You want to calculate the total daily sales for each customer from the sales transaction entries The following table shows a sample of the sales transaction data: CUST_NO TRANSACTION_DATE SELLING_PRICE QUANTITY 21 01/10/ :00: /12/ :00: /14/ :00: /14/ :00: /14/ :00: /15/ :00: Sorter Transformation Configuration Use a Sorter transformation to sort sales transaction entries. Then use the result of the Sorter transformation as input to the Aggregator transformation to increase performance. When you create a Sorter transformation, you specify one or more ports as sort keys and configure each sort key port to sort in ascending or descending order. In the Sorter transformation, srt_transactions, the sort key ports are CUST_NO and TRANSACTION_DATE. The direction of sorting is ascending. 3

4 The following figure shows the properties on Ports tab for the Sorter transformation: Aggregator Transformation Configuration Use an Aggregator transformation to calculate the total daily sales by customer number. Since you are sorting the sales transaction entries, use the Sorted Input property to increase mapping performance. When you use Sorted Input option, the Data Integration Service assumes the data is sorted by group. The Sorted Input property decreases the use of Aggregate cache and stores the group information in memory. The Data Integration Service performs aggregate calculations as it reads rows for a group. To calculate the total daily sales by customer number, you can define groups for aggregation using group-by ports. The groupby ports in the Aggregator transformation agg_sum_transactions are CUST_NO and TRANSACTION_DATE. Since the data is also sorted by CUST_NO and TRANSACTION_DATE, the Data Integration Service performs the aggregate calculations as it reads rows for a group. You can use variables ports to store the results of a calculation. Variable ports can reference input ports and variable ports. The Data Integration Service evaluates variable ports after input ports. The variable port PURCHASE stores the result of SELLING PRICE * QUANTITY. 4

5 The following figure shows the properties on the Ports tab for the Aggregator transformation: The Aggregator transformation has the following port configurations: Input ports SELLING_PRICE and QUANTITY receive data from the Sorter transformation. Output port DAILY_PURCHASE contains the return value of an expression, SUM(PURCHASE). Pass-through ports CUSTOMER_NO and TRANSACTION_DATE pass data downstream without any change. Variable port PURCHASE contains the return value of the expression SELLING_PRICE * QUANTITY. Group-by ports CUSTOMER_NUMBER and TRANSACTION_DATE group the results of the aggregated data. You must also configure the Sorted Input property on the Advanced tab. The following figure shows the properties set on the Advanced tab: Preview Transformation Output You can preview the data in the Sorter and Aggregator transformation to ensure that the data is processed as expected. You can preview the data in a transformation from the Data Viewer view. The data in the Sorter transformation is sorted in ascending order by ports CUST_NO and TRANSACTION_DATE. The following table shows a sample of the processed data in the Sorter transformation: CUST_NO TRANSACTION_DATE SELLING_PRICE QUANTITY 21 01/10/ :00: /14/ :00:

6 CUST_NO TRANSACTION_DATE SELLING_PRICE QUANTITY 21 01/14/ :00: /14/ :00: /12/ :00: /15/ :00: The Aggregator transformation shows the total daily sales grouped by the ports CUSTOMER_NUMBER and TRANSACTION_DATE. The value of DAILY_PURCHASE port is rounded to nearest whole number The following table shows a sample of the processed data in the Aggregator transformation: CUSTOMER_NUMBER TRANSACTION_DATE DAILY_PURCHASE 21 01/10/ :00: /14/ :00: /12/ :00: /15/ :00: Author Srilakshmi Sitaraman Technical Writer Acknowledgements The author would like to acknowledge Sorabh Agarwal, Manager QA Engineering for his technical assistance. 6