ASG-Rochade Reconciliation Toolkit Release Notes

Transcription

1 ASG-Rochade Reconciliation Toolkit Release Notes Version January 29, 2016 RRT This publication introduces changes made to ASG-Rochade Reconciliation Toolkit (herein called Reconciliation Toolkit) since Version It also provides installation instructions as well as information on how to get started using the product. ISP Updates and Service Packs ASG encourages you to visit the Intelligent Support Portal (ISP): The ISP allows you to verify whether any product or documentation revisions, new maintenance, or service packs apply to this product release. Product Overview Reconciliation Toolkit enables you to transform and transport metadata in a consistent form in ASG-Rochade (herein called Rochade) from a tool-specific source subject area to an integration subject area. The metadata in the integration subject area are stored in compliance with the information model DWRRIM10. Reconciliation Toolkit is implemented as a Java application using the Scriptable Rochade Application Platform (SRAP) as the programming interface. You can think of it as a framework that realizes the required mappings on the basis of rules and JavaScript procedures that are stored in special control data within Rochade. In addition to the fixed rules defined in the framework, you can implement complex mappings by means of JavaScript via SRAP and ASG-Rochade Java API. For a detailed description of the control data, see the ASG-Rochade Reconciliation Toolkit User s Guide. You invoke Reconciliation Toolkit through a command file. ASG recommends that you use ASG-Metability Scanner Configuration (herein called ScannerConfig) for generating the command files. Copyright 2016 ASG GmbH & Co. KG, a wholly owned subsidiary of Allen Systems Group, Inc. All rights reserved. All names and products contained herein are the trademarks or registered trademarks of their respective holders. 1

2 Enhancements and Changes Version Version Version includes this enhancement: The mapping for Apache Sqoop has been improved to provide more comprehensive lineage information. Version includes these enhancements: An error message now displays if an item cannot be created because the item name is an empty string. A new mapping has been added for ASG-Rochade Scanner for MySQL (herein called SCANMYSQL). The ScannerConfig template has been extended to support SCANMYSQL. The mapping for MicroStrategy now maps the attribute MCS_EXPRESSION_TEXT to the attribute DWR_ANL_QUERY for these item types: MCS_FACT MCS_FILTER MCS_METRIC The item types are transformed to DWR_ANL_DIMENSION_MAP or DWR_ANL_RPT_MAP respectively. The DWRRIM10 information model has been updated to Version The model includes these enhancements: The new item type DWR_ARC_BUSINESS_APPLICATION has been added to support application dependency. The new abstract item type EIM BUSINESS_RELEVANT_ITEM has been added as a supertype for the abstract item type DWR ITEM to integrate with issue management and application dependency support. The new abstract item type RWF GOVERNED_CONTEXT has been added to support issue management. The new attribute DWR_ANL_QUERY has been added to these item types: DWR_ANL_DIMENSION DWR_ANL_DIMENSION_MAP DWR_ANL_RPT_MAP 2

3 Version Version Version includes these enhancements: The mapping for IBM s DB2 database system has been improved. It now supports view and view-column dependencies if the DB2 catalog data is provided by ASG-Rochade DB2SCAN. The mapping for the RDM model now supports the transformation of the RDM/ALLOWED_VALUE_LIST attributes of RDM/COLUMN items. The help text of the ScannerConfig templates has been improved. It now contains a warning that different types of data could become indistinguishable and existing items could be overwritten if the default names in a template are modified. Version includes these enhancements: The mapping for ASG-Rochade Scanner for Greenplum Database now also includes the mapping item DWR-SGP_SYSTEM_2_DAT_PACKAGE. The mapping for ASG-Rochade Scanner for SQL Server SQL now supports triggers and optional name suffixes for functions, procedures, and triggers. The ScannerConfig template has been extended accordingly and provides optional settings to support name suffixes for functions, procedures, and triggers. The mapping for Cloudera Navigator has been extended by HiveQL mapping. The ScannerConfig template provides this extension as metadata category BigData Lineage (Seed item by Operation Service). 3

4 Version The mapping from RDM to ERM data model now supports mapping from RDM/DIAGRAM to DWR_ERM_SUBJECT_AREA types. Version includes these enhancements: The DWRRIM10 information model includes these enhancements: It now supports parameter sets. It now supports transformations and transformation maps as a target of lookup relationships. A new mapping for Cloudera Navigator has been added. The Datastage mapping now supports parameter sets. The Hive mapping now supports links to RDB_TABLE and RBD_COLUMN in place, as well as to RDB_ENTITY-PROXY. The mapping for CDM RIM has been enhanced to support logical view dependencies on attribute level. For all big data scanner related mappings, the content of the attribute DWR_STAGE is set to the default value BigData in ScannerConfig (i.e., for each mapping schema that starts with the prefix DWR-RBD). Corrections and Fixes Version Version corrects these errors: The mapping for IBM s DB2 database system did not handle view column dependencies (i.e., DWR-DB2/COLUMN mapping) correctly. It caused this error when creating the DWR_RDB_COLUMN CTL_REC_TYP item in the root namespace: Error : E0021 Thread 1: Mapping DWR-DB2/COLUMN 4

5 Version The mapping for Microsoft SQL Server Integration Services did not handle the MSS_DESCRIPTION attribute correctly. It caused this warning message: Warning : W0020 Item type MSSIS_COLUMN: Not supported yet: DESCRIPTION. Version corrects these errors: A bug in ScannerConfig caused an error if the Rochade installation directory contained blanks and Kerberos authentication or an account file were used. The mapping for Teradata SPL ignored scripts if they were directly stored within the namespace of a component. System Requirements Supported Operating Systems Reconciliation Toolkit supports these operating systems: AIX 5L3, 6.1, and 7.1 HP-UX (IA64) 11iV2 and 11iV3 Linux (Intel): Red Hat Enterprise Linux Server 5 and 6 as well as SUSE Linux Enterprise Server 11 Microsoft Windows 7 and 8 as well as Microsoft Windows Server 2008, 2008 R2, 2012, and 2012 R2 Oracle Solaris (SPARC) 10 and 11 Hardware Requirements ASG recommends these minimum system resources for Reconciliation Toolkit: RAM: 4 GB Processor: 2 with CPU speed of 3 GHz Disk capacity: 150 MB 5

6 Software Requirements Reconciliation Toolkit requires these software components: DWR-RIM or later (a compatible version of the component is included in the Reconciliation Toolkit installation package). These additional components, which are included in the Rochade Client Package for Microsoft Windows (available on the ISP): ASG-Rochade AutoPilot or later ASG-Rochade Java API or later ASG-Rochade Account Utility or later ASG-Metability Scanner Configuration or later ScanLogging Utility or later Java Runtime Environment (JRE). Reconciliation Toolkit is certified for Oracle Java 7 and 8. ASG recommends using a 64-bit Java Runtime Environment with a minimum of 4 GB heap space. To check whether Java is installed and which version Run this command in a Windows command shell: java -version The result should look like this example: java version "1.7.0_51" Java(TM) SE Runtime Environment <build 1.7.0_511-b13> Java HotSpot(TM) Client VM (build b03, mixed mode) If this method does not work, complete these steps: 1 Use the search user interface to look for a file named java.exe: Under Windows, select Start Search programs and files. 2 Run the command java -version for the paths found in step 1. Installation Information ASG provides information about system recommendations and supported environments for all Rochade components in its ISP under Click the upper left hyperlink ASG-Rochade Product Groups to see the corresponding files (i.e., Rochade Ecosystem and Rochade Supported Environments). 6

7 Information Model Import The Reconciliation Toolkit installation includes the information model DWRRIM10. The model is imported into the Rochade database that you select for installing the product. If the model already exists in the database, a backup is created before the new version of the model is imported. The installation program displays a message informing you about the directory where you can find the information model backups. NOTE: Any ASG product that you install with Reconciliation Toolkit might cause additional model imports. For a list of information models that ship with a particular ASG product, see the release notes for that product. Installing Reconciliation Toolkit The installation of Reconciliation Toolkit follows the general installation procedure for Rochade applications, which is described in the readme_en.pdf file and consists of two phases: 1 ADM installation The ADM installation imports the DWRRIM10 information model, including its submodels, which play a central role for the Reconciliation Toolkit integration subject area. Further, in the course of the ADM installation, you must specify the control subject area for Reconciliation Toolkit. The control subject area consists of an ADMIN version, which holds the control data that are included in the installation, and a USER version (on top of the ADMIN version), which is meant to receive custom control data that are provided by you. The control data in the ADMIN version is visible to Reconciliation Toolkit through the USER version (according to the item visibility in subject area hierarchies). Logs automatically will be stored in the USER version. If you want to modify the control data provided by ASG, you must perform the modifications in the USER version. During update installations, the data in the ADMIN version are updated and existing control items are overwritten. The control data that are stored in the USER version will not be affected by the installation process. For all control items that are updated during the installation, you must make sure that you apply the updates also to the respective items that you modified in the USER version of the control subject area. You can run a Compare Subject Areas query in Metability to identify the affected control data. The control subject area is based on the information model DWRCTL10, and you have these options: In a first-time installation, the control subject area is created at this point and, by default, is named RRT-CONTROL ADMIN. 7

8 In an update installation, the existing control subject area is offered for reuse, but you have the option to specify a different control subject area. Subsequently, the control data that are included in the Reconciliation Toolkit installation are automatically imported into the specified control subject area. 2 Application installation The Application installation deposits the Java archive rrt.jar (required for running Reconciliation Toolkit) in the \bin subdirectory of the Rochade directory. Items containing descriptions of relevant models and a description of the transfer parameters for the JavaScripts called by Reconciliation Toolkit in the manner of a user exit are stored in the \TechDoc\recon-toolkit subdirectory. The XML schema RRT-MappingSchema.xsd (used for validating the Reconciliation Toolkit control data) is stored in the \XMLSchema subdirectory of the Rochade directory. Getting Started 8 For an overview of the supplied mappings, see the file Rochade\TechDoc\recon-toolkit\RRT-Type-mapping.xls, where Rochade is the target directory of your Rochade installation. Setting Up the Integration Subject Area Reconciliation Toolkit works with these subject areas: The source subject area, which holds the original metadata that provided by Rochade scanners. The integration subject area, which holds the transformed metadata in compliance with the information model DWRRIM10. It is usually the ASG-Rochade Browser (herein called Rochade Browser) subject area. If the Rochade Browser subject area does not yet exist, you must create an integration subject area based on the information model DWRRIM10 first before applying Reconciliation Toolkit. For details, see the ASG-Metability System Administrator s Guide. The Reconciliation Process Once all the required subject areas are in place, you can apply Reconciliation Toolkit. It functions as a two-step process: 1 The data of the source subject area are consolidated into the virtual intermediate subject area in a form that complies with the DWRRIM10 information model. 2 The data consolidated in the virtual intermediate subject area are integrated into the integration subject area.

9 Configuring the Control Data for Reconciliation Toolkit The control data included in the Reconciliation Toolkit package are stored in the control subject area specified during the ADM installation. By default, the subject area is named RRT-CONTROL (see step 1 on page 7). Understand the control data included in the package as templates. Extensions and modifications in the source metadata usually require adjustments in these data. NOTE: Future versions of Reconciliation Toolkit may extend the scope of the supported categories and the existing mappings. For this reason, ASG advises that you do not modify the ASG-supplied control data with extensions of your own, but instead, produce your own copies and modify those as needed. Reconciliation Toolkit requires consistency of the control data as an indispensable prerequisite for successful mappings. Therefore, be sure to review the detailed description of the control data and their structure in the ASG-Rochade Reconciliation Toolkit User s Guide. To ensure the consistency of the control data, you must validate the descriptions stored in XML format against the XML schema RRT-MappingSchema.xsd in the \XMLSchema subdirectory of your Rochade directory. When you access the control data in ASG-Rochade AutoPilot (herein called AutoPilot) to modify them, AutoPilot loads these data in the XML editor that is configured as the default XML editor on your machine. If you must adjust the NO-EDIT interface that AutoPilot uses to open the control data, adjust it by modifying the Rochade Procedure Language (RPL) procedure DWR-NOEDIT- DWR-MAPPINGSCHEMA (included in the Reconciliation Toolkit package). For details, see the ASG-Rochade AutoPilot System Administrator s Guide. NOTE: The templates provided by ASG were produced using Altova XMLSpy (see The ASG-Rochade Reconciliation Toolkit User s Guide provides a detailed description of the meaning, structure, and relationships of the control data. The explanations that follow present only a highly abbreviated summary of those explanations. 9

10 Configuring the Command File for Reconciliation Toolkit Running a Mapping ScannerConfig provides a graphical user interface that allows you to set up Reconciliation Toolkit s command and configuration files. For details, see the ASG-Rochade Reconciliation Toolkit User s Guide. Once you have set up the integration subject area and configured the control data and the command file for the intended mapping, you are ready to run the mapping. To run the mapping Run the command file. If you have created a command file for DB2, for example, you may have to run the file DWR-DB2_SYSTEM.cmd An item of type DWR_CTL_JOB_LOG is created in the control subject area DWR-CONTROL ADMIN. The name of the item is derived from the date, time, and AutoPilot user name relevant to the command file execution. The attribute DWR_CTL_JOB_LOG is filled with content analogous to the example log above. This concludes the successful mapping and transport of metadata to the integration subject area. Troubleshooting If you have questions or problems concerning Reconciliation Toolkit, contact ASG Customer Support. Be prepared to supply an export of the DWR_CTL_JOB_LOG item and detailed information about any modifications of control data and command files to enable ASG to analyze problems effectively and efficiently. 10