Table of Contents. Eccella 1

Transcription

1 ECCELLA 22-Apr-14

2 Table of Contents Introduction... 2 About the tool... 2 Features... 2 Scope... 3 Components... 4 Input... 4 Outputs... 5 Points to Note... 5 Operation... 6 Installation... 6 Update Licensing Information Execution Appendix A. Velocity_Standards.xml Eccella 1

3 Introduction Manual code reviews and cleanup of Informatica PowerCenter mappings and workflows take a considerable amount of time and effort. Even then, there remains some scope for human errors and an impact on project deadlines. The purpose of this tool is to significantly reduce the time taken for mapping review and correction, to increase the review quality and effectiveness and more importantly to automate the entire process ensuring consistency and accuracy. About the tool The Eccella Mapping Cleanup Tool requires Informatica B2B Data Transformation (DT) and as such the studio must be installed on the local client machine where it is executed. The Informatica B2B DT studio is included in the standard client tool installer for Informatica PowerCenter. The tool takes an XML input and provides an XML output. The XML input is the exported XML of mappings or workflows from PowerCenter and the corrected XML output can be directly imported back into PowerCenter. The Eccella Mapping Cleanup Tool reviews the input XML, finds if any standard conventions are not followed, provides reports on the violations found, corrects and standardizes those violations as confirmed by the user and finally outputs the corrected XML. Features The tool has the following features: - The tool can review and clean up the input XML based on ten different rules. - Along with the cleaned up output XML, the tool also generates HTML reports summarizing the changes in the input XML. - User has the option to Review the input XML and verify the possible changes. - User may then select the rules which would be checked for while the input XML is being cleaned up. - User can customize the naming standards to be followed by the tool while reviewing and cleaning up the source XML. - User can selectively mark workflows, tasks, mappings, transformations or ports to be ignored by the tool in the cleanup process. Eccella 2

4 - The tool can cleanup multiple objects in one run. So the user may export all the mappings or workflows in a folder into one XML and clean all of them up in just one run. Scope The tool reviews and cleans up the input XML based on the following rules: R01. Naming Standards: Check if the mappings, transformations within the mappings, workflows and tasks within the workflows have standard names with the correct prefixes. If naming standards are not followed, the names are updated in the XML if possible. The naming standards followed by the tool during the cleanup process may be customized, if required, by updating a configuration file named Velocity_Standards.xml. The guidelines on customizing the standards are given in Appendix A. R02. Port Ordering: Check if the ports within the expression or aggregator transformations, are in the standard order (Input, Input-Output, Variable, Output). If the port ordering is not standard, it is updated in the mapping if possible. R03. Filters with Default True Condition: Check if a filter transformation has its filter condition defaulted, i.e. TRUE, or set to some non-zero number (which also means TRUE). If such a filter transformation is found, it is removed from the mapping if possible. R04. Unused Lookups/Stored-Procedures: Check if there are any unconnected lookups or stored-procedures (of 'normal' type) in a mapping which are not called or used in any other transformation in the mapping. If such an unused transformation is found, it is removed from the mapping if possible. R05. Tracing Level: Check the tracing level of each transformation in the mappings and sessions. The tracing level of each transformation can be updated to a desired tracing level. Note, verbose tracing levels lead to larger logs and decrease the performance in general. R06. Description: Report if the workflows, tasks within the workflows, mappings, tasks within the mappings do not have a description provided. R07. Push-down Optimization: Report if a filter/sorter/aggregator/joiner transformation is present right after source qualifier(s) reading from a database. In the above scenario, the logic implemented by Eccella 3

5 Note: the former may also be implemented in the SQL query override of the source qualifier in order to make the mapping more efficient. R08. Consecutive Transformations of Same Type: Report if there are connections between transformations of the same type. The types being Update Strategy, Expression, Aggregator, Filter, Router, Sorter, Transaction Control. The number of transformations used in a mapping should be reduced if possible. So the logic implemented in two consecutive transformations of the same type can also be implemented in a single transformation of that type. R09. Sorted Input in Joiners/Aggregators: Report if a joiner or an aggregator transformation does not have its sorted input option checked. It is always advisable to give sorted input to a joiner or an aggregator. R10. Port Attributes: Report if the datatypes of two connected ports do not match, or if the length of a port is greater than the length of the port it is connected to which might lead to data truncation. - Rules R01 to R05 make changes in the input XML and cleans it up. - Rules R06 to R10 only generate reports. - When the user is reviewing the input XML for possible cleanups, nothing is actually changed in the input XML. Components The tool has been packed into one single installer named Eccella_Mapping_Cleanup_Tool_v2.0-setup.exe. A license key file is separately sent out along with the installer. This key file should be used to update the licensing information for the installation. Input The tool accepts XML export of mappings or workflows, directly exported from PowerCenter. The mappings or workflows must be exported from a non-shared folder only. User may export multiple mappings or workflows together into one XML from the Repository Manager. Eccella 4

6 Outputs After the input XML is cleaned up by the tool, it creates a cleaned up output XML file which may be directly imported back into PowerCenter. The name of this file is Output_XML_for_re-Import.xml. The tool also generates the following html reports along with the cleaned up output XML file. Dashboard.html This file has the links to all other reports generated. When the input XML is reviewed, this report contains the links to all the other review reports listing the possible cleanups in the input XML. And when the input XML is cleaned up, this report contains the links to all the other cleanup reports listing the changes done in the input XML. All the other reports are generated rule-wise as listed below. R01-Mapping_Name_Updates.html R01-Task_Workflow_Name_Updates.html R01-Transformation_Name_Updates.html R01-Transformation_Port_Name_Updates.html R02-Port_Ordering_Check.html R03-Default_Filters_Check.html R04-Unused_Transformations_Check.html R05-Tracing_Level_Check.html R06-Description_Check_Mappings.html R06-Description_Check_Workflows.html R07-PushDownOptimization_Check.html R08-SameType_Transformation_Check.html R09-Sorted_Input_Check.html R10-Port_Attribute_Mismatches.html Points to Note 1. The input XML may have multiple mappings or workflows but they should be exported from a non-shared folder only. Multiple mappings or workflows may be exported together into one XML from the Repository Manager. 2. As a pre-requisite, the user must have Informatica B2B Data Transformation installed on their local machine. Eccella 5

7 3. A file with the default naming standards and prefixes (Velocity_Standards.xml) comes with the installation, and may be easily customized by the user if they want the tool to follow different naming standards. 4. None of the prefixes in the Velocity Standards XML should be NULL (other than for INPUT/OUTPUT and INPUT/OUTPUT/MASTER prefixes under the Port_Prefixes group). This is not checked in the tool, so user should ensure this. 5. All the prefixes in the Velocity Standards XML should be DISTINCT. Again this is not checked in the tool, so user should ensure this as well. 6. A text #Ignore# may be included in the description of a workflow, task, mapping, transformation or a port and that object will be ignored by the tool while cleaning up the input XML. Thus user may selectively mark objects which he doesn t want the tool to update. 7. If any mapping or transformation has the CRCValue attribute set, then it and its child elements are not updated as that would invalidate the XML. But the cleanups required in those mappings or transformations would show up in the reports. 8. Reusable mapplets and transformations are not reviewed nor cleaned (other than their instance name). Operation Installation The installation package consists of an installer. The Eccella Mapping Cleanup Tool may be installed as shown below: 1. Run the installer Eccella_Mapping_Cleanup_Tool_v2.0-setup.exe. 2. Click Next > in the welcome screen. Eccella 6

8 3. Choose the folder where the tool will be installed and click Next >. Note, you need to have administrative privileges in this folder for the tool to run. Eccella 7

9 4. Choose the Start Menu folder where the shortcuts will be created and click Next > on the next screen. 5. Select the checkbox to create a desktop shortcut and click Next > on the next screen. Eccella 8

10 6. Verify the summary and click Install on the final screen to install the tool. 7. Wait till the tool installs. Eccella 9

11 8. After it installs, click Finish. The tool will run, and you need to update the licensing information when it runs for the first time. Update Licensing Information When you purchased the Eccella Mapping Cleanup Tool from us, you must have received a license key file named Eccella_Mapping_Cleanup_Tool.key. This file contains your licensing information and you need to update this in the tool before it can be used. Follow the steps below to update the licensing information: 1. Procure the license key file (Eccella_Mapping_Cleanup_Tool.key) you received. 2. During the first run of the tool you ll be prompted to update the licensing information. Click OK. Eccella 10

12 3. Browse and select the license key file. 4. The tool will then update its licensing information and start if the license is valid. Now that you have updated the licensing information, you may now use the tool. Execution Before you start the tool, export the mappings or workflows you want the tool to cleanup into an XML file preferably from the Repository Manager. Now that you have the input source XML, start the tool from either the desktop shortcut created or from the start menu programs shortcut created during the installation. Eccella 11

13 1. By default the tool follows the velocity naming standards while cleaning up the input XML. But if you want the tool to use different naming standards, you may edit the Velocity_Standards.xml file and update it as required. 2. Under the third tab, clicking on the Edit button will open up the directory containing the Velocity_Standards.xml file. You may edit this file with any text editor. Please make sure you strictly follow the guidelines (Appendix A) while editing the Velocity_Standards.xml file, or the tool might behave unexpectedly. 3. Once you have set the naming standards, come back to the first tab. From this tab you can review the input source XML and see the possible changes. Make all the mandatory selections as marked by the *. 4. Now you have the option to review the input source XML for possible cleanups or you may skip review and go to the Execute tab to run the tool and cleanup the input source XML. Click Review to review the source XML. Eccella 12

14 5. After the tool has completed reviewing the source XML, click OK. 6. The Review Reports Dashboard will open in your default browser. The tool creates the output html reports rule-wise. When you click on the hyperlinks, you ll be able to see the possible changes in the input source XML. You may hover your mouse over each rule for a description of that rule. You should review each report listed in this dashboard and make sure we are comfortable with all the possible changes. If we are not comfortable with the tool making any of the possible changes under a rule, we may deselect that rule before cleaning up the input source XML. Eccella 13

15 7. After a successful review, the tool automatically switches to the second tab which is Execute. From this tab you can run the tool and cleanup the source XML. 8. Let s say you are comfortable with the possible changes under R01 and R02 only. Only keep these two rules checked and click Run. Eccella 14

16 9. If you get this warning that the reports from the previous run will be deleted from the output directory, click OK. Before that make sure you no longer need those reports. If you need those reports, you may copy them to another location before clicking OK. 10. After the tool has completed cleaning up the input source XML, click OK. 11. The Cleanup Reports Dashboard will open in your default browser. The tool creates the output html reports rule-wise. Rules that were not selected during the cleanup process are greyed out in the dashboard. When you click on the hyperlinks, you ll be able to see the list of changes done in the input source XML. Eccella 15

17 12. The output directory will open in a new explorer window. The file Output_XML_for_re- Import.xml may now be imported back into Informatica PowerCenter. Appendix A. Velocity_Standards.xml Guidelines on filling up Velocity_Standards.xml In the Task_Prefixes group, make sure of the following: Task_Type: The task type entered is a valid type of task in Informatica. Prefix: None of the prefixes are left null or blank. If it is left blank, that type of task s names will not be updated. These are case sensitive. A _ is added between this prefix and the current task name if the current name doesn t start with <prefix>_. As per Velocity standards, task prefixes should be in all lower case. Prefixes_to_be_Replaced: The prefixes which would be removed from this type of task (if found) are delimited by a comma and a space. Eccella 16

18 These are case insensitive. Task name till the first _ is extracted and compared to these prefixes, and removed if found in this list. In the Mapping_Prefix group, make sure of the following: Prefix: This is not left null or blank. If it is left blank, then mapping names will not be updated. This is case sensitive. A _ is added between this prefix and the current mapping name if the current name doesn t start with <prefix>_. As per Velocity standards, mapping prefix should be in all lower case. Prefixes_to_be_Replaced: The prefixes which would be removed from mapping names (if found) are delimited by a comma and a space. These are case insensitive. Mapping name till the first _ is extracted and compared to these prefixes, and removed if found in this list. In the Transformation_Prefixes group, make sure of the following: Transformation_Type: The transformation type entered is a valid type in Informatica. Prefix: None of the prefixes are left null or blank. If it is left blank, that type of transformation s names will not be updated. These are case sensitive. A _ is added between this prefix and the current transformation name if the current name doesn t start with <prefix>_. As per Velocity standards, transformation prefixes should be in all upper case. For mapplets, it should be in lower case. Prefixes_to_be_Replaced: The prefixes which would be removed from this type of transformation (if found) are delimited by a comma and a space. These are case insensitive. Transformation name till the first _ is extracted and compared to these prefixes, and removed if found in this list. In the Port_Prefixes group, make sure of the following: Port_Type: The Port_Type entered is a valid type in Informatica. Prefix: None of the prefixes are left null or blank (except for INPUT/OUTPUT and INPUT/OUTPUT/MASTER type ports). These are case sensitive. A _ is added between this prefix and the current transformation name if the current name doesn t start with <prefix>_. As per Velocity standards, port prefixes should be in all lower case. Prefixes_to_be_Replaced: The prefixes which would be removed from this type of port (if found) are delimited by a comma and a space. These are case insensitive. Port name till the first _ is extracted and compared to these prefixes, and removed if found in this list. Eccella 17

19 In the Shortcut_Prefix group, make sure of the following: Prefix: This is not left null or blank. This is case sensitive. A _ is added between this prefix and the current shortcut name if the current name doesn t start with <prefix>_. As per Velocity standards, shortcut prefix should be in all lower case. Prefixes_to_be_Replaced: The prefixes which would be removed from shortcuts (if found) are delimited by a comma and a space. These are case insensitive. Shortcut name till the first _ is extracted and compared to these prefixes, and removed if found in this list. In the Other_Inputs groups, make sure of the following: Skip_the_fields_of_these_ The fields of the transformations listed here would be skipped and Transformation_Types: wouldn't be standardized. Enter the transformation names delimited by <space>~<space>. Skip_these_Table_Attribu tes: Don t change this unless absolutely required to. The table attribute names listed here would be skipped and wouldn't be standardized. Enter the transformation names delimited by <space>~<space>. Don t change this unless absolutely required to. Velocity_Standards.xml The naming standards file can be found in [Installation directory]\dt_service\eccella_mapping_cleanup_tool\velocity_standards\velocity_standards.xml Eccella 18