ASG-Rochade Reconciliation Toolkit Readme

Size: px

Start display at page:

Download "ASG-Rochade Reconciliation Toolkit Readme"

Jeffrey Pierce
5 years ago
Views:

ASG-Rochade Reconciliation Toolkit Readme Version 1.00.

1 ASG-Rochade Reconciliation Toolkit Readme Version November 23, 2005 This file contains installation instructions and information about getting started using the ASG-Rochade Reconciliation Toolkit (herein called Reconciliation Toolkit). Product Overview The Reconciliation Toolkit is a prerequisite for the ASG-Rochade Data Warehouse Application (herein called DW Application). It provides tools that allow you to transform and transport metadata in ASG-Rochade (herein called Rochade) from a source subject area to the central subject area of the DW Application in a consistent form. The metadata are organized and stored in compliance with the Data Warehouse Reconciliation Model DWRRIM10. The Reconciliation Toolkit is implemented as a Java application using the Rochade SRAP as the programming interface. You can understand it as a framework that realizes the required transformations on the basis of rules and JavaScript procedures that are stored in special control data within Rochade. In addition to the fixed rules that are defined in the framework, extensive options are available for the implementation of complex mappings by means of JavaScript via SRAP and the Rochade JavaAPI. For a detailed description of the control data and their structure, see the ASG-Rochade Reconciliation Toolkit Technical Guide. The Reconciliation Toolkit is invoked through a command file, for which you can find examples in the \Samples\rrt\cmd subdirectory of your Rochade installation directory after you have installed the toolkit. Note: The control data and command files included in the Reconciliation Toolkit package have template character, and ASG continues to extend and develop them further. Copyright 2005 ASG GmbH & Co. KG, a wholly owned subsidiary of Allen Systems Group, Inc. All rights reserved. All names and products contained herein are the trademarks or registered trademarks of their respective holders. 1

2 Hardware Requirements These are the minimum system requirements: 64 MB RAM (ASG recommends 2GB or more) 5 MB hard disk space 200 MHz Intel Pentium processor or equivalent (ASG recommends 2 GHz or more) Software Requirements At least these versions of these software modules are required: Rochade 6.30 Java API with SRAP JDK1.4 / Java 2 To check if Java is installed and which version of it is installed, complete this step: Execute this command in a Windows command shell: java -version The result should be something like this: java version "1.4.2" Java(TM) 2 Runtime Environment, Standard Edition (build b28) Java HotSpot(TM) Client VM (build b28, mixed mode) If this method does not work, complete these steps: 1 Use the search user interface to look for a file named java.exe: Under Windows 2000: Start Search For Files and Folders Windows XP: Start Search 2 Execute the command java -version again for the paths found in step 1. 2

3 Installation Information The installation of the Reconciliation Toolkit follows the general installation procedure for Rochade applications, which is described in the readme_en.pdf file and consists of two parts: 1 ADM installation The ADM installation imports the Data Warehouse Reconciliation Model DWRRIM10, including its submodels, which play a central role for the DW Application. Further, in the course of the ADM installation, you must complete a dialog in which you must specify the control area for the storage of control data for the Reconciliation Toolkit. The control area is based on the information model DWRCTL10, and you have these options: In the case of a first-time installation, it is created at this point, and by default, RRT-CONTROL ADMIN is suggested for its name. In the case of an update installation, the existing control area is offered for reuse, but you can specify a different control area. Subsequently, the control data supplied with the Reconciliation Toolkit package are automatically imported into the specified control area. 2 Application installation The Application installation deposits the Java archive rrt.jar, which is relevant for the execution of the Reconciliation Toolkit, in the \bin subdirectory of the Rochade directory. Further, it stores command files (*.cmd) to execute the toolkit under Windows in the \Samples\rrt\cmd subdirectory, and generates a file RRT-dataspace.xml in that directory to specify the communication parameters for the Rochade environment (e.g., server name, host name, and port number). Items containing descriptions regarding relevant models and a description of the transfer parameters for the JavaScripts that are called by the Reconciliation Toolkit in the manner of a user exit are stored in the \TechDoc\rrt subdirectory. The XML schema RRT-MappingSchema.xsd, which is used for validation of the Reconciliation Toolkit control data, is stored in the \XMLSchema subdirectory of the Rochade directory. 3

4 Getting Started General The Reconciliation Toolkit provides functionality to transform and transport metadata in Rochade from a source subject area to the DW Application subject area. The following summary lists the metadata sources by relevant areas of the data warehouse process along with the corresponding submodels of the Data Warehouse Reconciliation Model DWRRIM10. Also listed are the information models on which the source subject areas can be based and the corresponding metadata sources along with the interfaces available to populate the source subject areas with the metadata from those sources. OLAP (Online Analytical Processing) & Information Reporting Submodel of DWRRIM10: DWRRIM10_ANALYSIS Source subject area based on ASG-Rochade CGN-RIM Metadata source: Cognos Interface: ASG-Rochade Scanner for COGNOS ETL (Extract, Transform, and Load) Submodel of DWRRIM10: DWRRIM10_TRANSFORMATION Source subject area based on ASG-Rochade INFORMATICA RIM Metadata source: Informatica Interface: ASG-Rochade Scanner for Informatica Source subject area based on ASG-Rochade DSTAGE RIM Metadata source: DataStage Interface: ASG-Rochade Scanner for Ascential DataStage Database systems Submodel of DWRRIM10: DWRRIM10_RDBMS Source subject area based on ASG-Rochade I-RIM Metadata source: UDB/DB2 Interfaces: DB2BUS for ASG-Rochade and UDB2DDIC Solution Source subject area based on ASG-Rochade ORACLE RIM Metadata source: Oracle Interface: ASG-Rochade SCANORAC Source subject area based on ASG-Rochade ORACLE RIM Metadata source: MS SQL Server Interface: ASG-Rochade SCANSQLSRVR 4

5 Logical data models Submodel of DWRRIM10: DWRRIM10_ER_MODEL Source subject area based on ASG-Rochade CDM-RIM Metadata source: ER models Interfaces: ARTSBUS for ASG-Rochade, ASG-Rochade BUS for Casewise Corporate Modeler, ERWINBUS for ASG-Rochade, SABUS for ASG-Rochade, and PWRDESBUS for ASG-Rochade Physical database models Submodel of DWRRIM10: DWRRIM10_RDBMS Source subject area based on ASG-Rochade ERWBUS RIM Metadata source: ERwin RDBMS models Interface: ERWINBUS for ASG-Rochade Setting Up the DW Application Subject Area The Reconciliation Toolkit works with these subject areas: The source subject area: This is the subject area that holds the original metadata that are to be integrated into the DW Application. The DW Application subject area: This is the subject area that holds the metadata for the DW Application. Its name, including its version name, is left to you to decide; for this description, DWR INTEGRATION is assumed. If the DW Application subject area does not yet exist, your first preparatory step before applying the Reconciliation Toolkit must be to create it, based on the Data Warehouse Reconciliation Model DWRRIM10. Note: The DW Application subject area must support the extended Rochade naming concept, so you must create it as a subject area with tree-like structure. For details, see the ASG-Rochade AutoPilot System Administrator s Guide. 5

6 An intermediate subject area: This is an auxiliary subject area via which the Reconciliation Toolkit passes the metadata from the source subject area to the DW Application subject area. Its name, including its version name, is left to you to decide; for this description, DWR CONSOLIDATION is assumed. If the intermediate subject area does not yet exist, your second preparatory step before applying the Reconciliation Toolkit must be to create it, based on the Data Warehouse Reconciliation Model DWRRIM10. Note: Since the final destination of the data is the subject area DWR INTEGRATION, the intermediate subject area DWR CONSOLIDATION must be based on the same rules as the subject area DWR INTEGRATION. You can accomplish this, for instance, by creating a version called CONSOLIDATION for the DW Application subject area DWR INTEGRATION. For information on how to do this, see the ASG-Rochade AutoPilot System Administrator s Guide. Once all the required subject areas are in place, you can apply the Reconciliation Toolkit. The toolkit functions as a two-step process: 1 The data of the source subject area are consolidated into the intermediate subject area DWR CONSOLIDATION in a form that complies with the Data Warehouse Reconciliation Model DWRRIM10. 2 The data consolidated in the intermediate subject area DWR CONSOLIDATION are imported into the DW Application subject area DWR INTEGRATION. Afterwards, the data in the intermediate subject area are deleted. 6

7 Configuring the Control Data for the Reconciliation Toolkit The control data included in the Reconciliation Toolkit package are deposited in the control area specified during the ADM installation. By default, RRT-CONTROL ADMIN is suggested for the name of that subject area and its version name (see step 1 on page 3). This name is also used in this description. The control data included in the package are to be understood as templates. Extensions and modifications in the source metadata will usually require adjustments in these data. Note: Successive versions will extend the scope of the supported categories and complete various details of existing transformations. For this reason, ASG advises that you not modify the ASG-supplied control data templates with extensions of your own, but instead, produce your own copies and modify those as needed. The Reconciliation Toolkit requires consistency of the control data as an indispensable prerequisite for successful transformations. Therefore, be sure to see the detailed description of the control data and their structure in the ASG-Rochade Reconciliation Toolkit Technical Guide. To ensure the consistency of the control data, the descriptions stored in XML format must be validated against the XML schema RRT-MappingSchema.xsd in the \XMLSchema subdirectory of your Rochade directory. When you access the control data in ASG-AutoPilot (herein called AutoPilot) to modify them, AutoPilot will load these data in the XML editor that is configured as the default XML editor on your machine. If you need to adjust the NO-EDIT interface that AutoPilot uses to open the control data, do this by adjusting the RPL (Rochade Procedure Language) procedure DWR-NOEDIT- DWR-MAPPINGSCHEMA, which is included in the Reconciliation Toolkit package. For information on NO-EDIT interfaces, see the ASG-Rochade AutoPilot System Administrator s Guide. The templates provided by ASG were produced with Altova XMLSpy (see How to configure the control data is described in Control Data Configuration Example on page 8, using an example that involves the transformation of DB2 metadata for DB2BUS. Other transformations follow the same steps in analogous manner. The ASG-Rochade Reconciliation Toolkit Technical Guide contains a detailed description of the meaning, structure, and relationships of the Reconciliation Toolkit control data. The explanations that follow below present only a highly abbreviated summary of those explanations. 7

8 Control Data Configuration Example This example demonstrates how you would configure the control data for a DB2 metadata transformation. It starts with these assumptions: DB2BUS has created metadata in a Rochade subject area named DB2 and with version name CATALOGUE. The metadata in this subject area conform to the unmodified DB2 subset of the C-RIM included in ASG-Rochade I-RIM. The subject areas reside in a Rochade database named AP-DATA and are named as stipulated in Setting Up the DW Application Subject Area on page 5. Based on these assumptions, it is your task to configure for the subject area DB2 CATALOGUE an environment in which the transformation and transport of DB2 metadata to the DW Application subject area DWR INTEGRATION can be executed. Proceed as follows: 1 Select the appropriate job template. If DB2BUS was run in its standard configuration, open the item DWR-DB2 of type DWR_CTL_JOB in the control area RRT-CONTROL ADMIN, and create a copy of this job template item (e.g., with the name RUN-DB2). If a different DB2BUS name convention is used, select the template with the respective name suffix: DWR-DB2: Template for DB2BUS with creator links DWR-DB2-w/o_CreatorLinks: Template for DB2BUS without creator links DWR-DB2-w/o_CreatorLinks_butPrfx: Template for DB2BUS without creator links but name prefix DWR-DB2-w/o_CreatorLinks_butSffx: Template for DB2BUS without creator links but name suffix DWR-DB2-with_CreatorPrefix: Template for DB2BUS with creator links and name prefix DWR-DB2-with_CreatorSuffix: Template for DB2BUS with creator links and name suffix For information on the configuration of DB2BUS, see the DB2BUS for ASG-Rochade User s Guide. 8

9 2 In the next step, modify the attribute DWR_CTL_INSTRUCTIONS of the item copy DWR_CTL_JOB RUN-DB2 created in step 1 by replacing the placeholders as follows: <Transformation Database="AP-DATA"> <Source Database="AP-DATA" Type="S" Name="DB2" Version="CATALOGUE"/> <Consolidator Database="AP-DATA" Type="S" Name="DWR" Version="CONSOLIDATION"/> <Integrator Database="AP-DATA" Type="S" Name="DWR" Version="INTEGRATION"/> </Transformation> This code specifies the subject area DB2 CATALOGUE as the source of the transformation, the subject area DWR INTEGRATION as its target, and DWR CONSOLIDATION as the intermediate subject area. Configuring the Command File for the Reconciliation Toolkit The command files delivered with the Reconciliation Toolkit package are deposited in the \Samples\rrt\cmd subdirectory of your Rochade directory during the Application installation of the toolkit. The data space file RRT-dataspace.xml is stored in the same directory. In the data space file, check these specifications and adjust them as needed: Specification of the database in which the control area for the Reconciliation Toolkit resides: <DATABASE Name="AP-DATA"> Specification of the control area for the Reconciliation Toolkit: <APPLICATION appltype="rtc" Name="RRT-CONTROL" Version="ADMIN"/> Specification of the Rochade server: <SERVER Name="SERVER1" host="<host>" port="<port>"/> In the command file RRT-set-env.cmd for configuring the Reconciliation Toolkit environment, check these specifications and adjust them as needed: Specification of the Rochade root directory: set _ROCHADE_HOME=<Rochade> Specification of the Java Home directory: set _JAVA_HOME=<JAVA_HOME> In the next step, create a copy of the template command file DWR-DB2.cmd (or of the command file that corresponds to the DWR_CTL_JOB file that you selected in step 1 on page 8) under a name such as RUN-DB2.cmd. 9

10 Enter these specifications in the file RUN-DB2.cmd: Specification of the AutoPilot user who has READ rights for the subject areas RRT-CONTROL ADMIN and DB2 CATALOGUE, and has WRITE rights for the subject areas DWR CONSOLIDATION and DWR INTEGRATION: set _DWRT_USR=<AutopilotUser> Specification of the password for this AutoPilot User: set _DWRT_PWD=<PassWord> Specification of the transformation start item of type DB2/SYSTEM: set _DWRT_ITM=<SeedItemName> Running a Transformation Once you have set up the DW Application subject area and configured the control data and the command file for the intended transformation, you are ready to run the transformation. To run the transformation Execute the command file. For the example used in the above sections on setup and configuration, this is the file RUN-DB2.cmd. A log similar to this will be created and will appear on your computer screen: D:\Ro_6_30_000\DW>"C:\jdk142jb\bin\java" -Xmx512m de.rochade.dw.rt.reconciliationtoolset -ds="d:\ro_6_30_000\samples\rrt\rrt-dataspace.xml" -j=_dwr-db2 -i=prod -u=jr -p=_dwrt_pwd ASG-Rochade(TM) Reconciliation Toolkit - Version Copyright (c) 2005 ASG GmbH & Co. KG, a wholly owned subsidiary of Allen Systems Group, Inc. All rights reserved. I0005 Reconciliation Toolkit started at :36:22 I0034 Consolidator : identify items.. I0037 Consolidator : DB2/USER, mapping=dwr-db2/user, 1 items. I0037 Consolidator : DB2/DATABASE, mapping=dwr-db2/database, 1 items I0037 Consolidator : DB2/COLUMN, mapping=dwr-db2/column, 244 items.. I0037 Consolidator : DB2/SYSTEM, mapping=dwr-db2/system, 1 items I0037 Consolidator : DB2/INDEX, mapping=dwr-db2/index, 13 items I0037 Consolidator : DB2/REFERENTIAL-CONSTRAINT, mapping=dwr-db2/referential-constraint, 10 items I0037 Consolidator : DB2/TABLE, mapping=dwr-db2/table, 32 items I0040 Consolidator : 302 Rochade items I0021 Integrator : identify items... I0038 Integrator : DWR_RDB_DATABASE, mapping=dwr-db2/database, 1 items I0038 Integrator : DWR_RDB_SCHEMA, mapping=dwr-db2/user, 1 items I0038 Integrator : DWR_RDB_DB_SYSTEM, mapping=dwr-db2/system, 1 items. I0038 Integrator : DWR_RDB_COLUMN, mapping=dwr-db2/column, 244 items.. I0038 Integrator : DWR_RDB_INDEX, mapping=dwr-db2/index, 13 items 10

11 I0038 Integrator : DWR_RDB_FOREIGN_KEY, mapping=dwr-db2/referential-constraint, 10 items I0038 Integrator : DWR_RDB_TABLE_OR_VIEW, mapping=dwr-db2/table, 32 items I0041 Integrator : 302 Rochade items I0039 clear consolidation area... I0006 Reconciliation Toolkit finished at :37:15 In addition, an item of type DWR_CTL_JOB_LOG will be created in the control area DWR-CONTROL ADMIN. The name of the item will be derived from the date, time, and AutoPilot user name relevant to the command file execution. The attribute DWR_CTL_JOB_LOG will be filled with content analogous to this example: Text Attribute DEFINITION Job _DWR-DB2 Seed DB2/SYSTEM PROD Text Attribute DWR_CTL_JOB_LOG ASG-Rochade(TM) Reconciliation Toolkit - Version Copyright (c) 2005 ASG GmbH & Co. KG, a wholly owned subsidiary of Allen Systems Group, Inc. All rights reserved. I0005 Reconciliation Toolkit started at :36:22 I0002 java.version I0018 User : JR I0030 Rochade Server 630 patch level 0 I0031 Rochade Java API I0043 Source database : AP-DATA application : S DB2BUS 32CHAR I0044 Consolidator database : AP-DATA application : S DWRT TRANSFORMER I0045 Integrator database : AP-DATA application : S DWRT INTEGRATOR I0017 Seed item : DB2/SYSTEM PROD I0034 Consolidator : identify items I0037 Consolidator : DB2/USER, mapping=dwr-db2/user, 1 items I0037 Consolidator : DB2/DATABASE, mapping=dwr-db2/database, 1 items I0037 Consolidator : DB2/COLUMN, mapping=dwr-db2/column, 244 items I0037 Consolidator : DB2/SYSTEM, mapping=dwr-db2/system, 1 items I0037 Consolidator : DB2/INDEX, mapping=dwr-db2/index, 13 items I0037 Consolidator : DB2/REFERENTIAL-CONSTRAINT, mapping=dwr-db2/referential-constraint, 10 items I0037 Consolidator : DB2/TABLE, mapping=dwr-db2/table, 32 items I0040 Consolidator : 302 Rochade items I0021 Integrator : identify items I0038 Integrator : DWR_RDB_DATABASE, mapping=dwr-db2/database, 1 items I0038 Integrator : DWR_RDB_SCHEMA, mapping=dwr-db2/user, 1 items I0038 Integrator : DWR_RDB_DB_SYSTEM, mapping=dwr-db2/system, 1 items I0038 Integrator : DWR_RDB_COLUMN, mapping=dwr-db2/column, 244 items I0038 Integrator : DWR_RDB_INDEX, mapping=dwr-db2/index, 13 items I0038 Integrator : DWR_RDB_FOREIGN_KEY, mapping=dwr-db2/referential-constraint, 10 items I0038 Integrator : DWR_RDB_TABLE_OR_VIEW, mapping=dwr-db2/table, 32 items I0041 Integrator : 302 Rochade items I0039 clear consolidation area I0006 Reconciliation Toolkit finished at :37:15 Link Attribute DWR_CTL_JOB DWR_CTL_JOB _DWR-DB2 This concludes the successful transformation and transport of metadata to the DW Application subject area. 11

12 Troubleshooting If you have questions or problems concerning the Reconciliation Toolkit, contact ASG Customer Support. Be prepared to supply an export of the DWR_CTL_JOB_LOG item and detailed information about any modifications of control data and command files to enable ASG to analyze problems efficiently. 12

ASG-Rochade Reconciliation Toolkit Readme

ASG-Rochade Reconciliation Toolkit Readme Version 1.00.004 July 4, 2006 This file contains installation instructions, information about getting started using the ASG-Rochade Reconciliation Toolkit (herein