Quick-Start User Guide. Analytix Data Services Analytix: Mapping Manager. Quick-Start User Guide. Version 2.4

Transcription

1 Analytix Data Services Analytix: Mapping Manager Quick-Start User Guide Version 2.4 Analytix Data Services LLC Page 1 of 62

2 Copyright Notice This manual is Analytix Data Services LLC All rights reserved. No part of this manual may be reproduced, distributed or transmitted, in whole or in part, in paper, electronic or any other form or by any means other than as expressly permitted in the applicable Mapping Manager Software License Agreement or Software License and Maintenance Agreement, or as otherwise expressly permitted by Analytix data Services. Trademark Notice MAPPING MANAGER, ANALYTIX MAPPING MANAGER ANALYTIX: MAPPING MANAGER MAPPING MANAGER are trademarks of Analytix Data Services and may not be used without the express written permission of Analytix Data Services. This list of trademarks may not be complete; other trademarks or registered trademarks may be owned by Analytix Data Services from time to time and may be used in this manual. All other trademarks or service marks are the properties of their respective owners. Proprietary and Confidential Information Notice Analytix Data Services software products contain valuable trade secrets and proprietary information and are protected internationally, including without limitation, by Canadian, United States and international copyright, trademark, and other intellectual property laws and treaties. Unauthorized use of this manual or Analytix Data Services software products is strictly prohibited and may result in civil damages and criminal prosecution. See the applicable Analytix Data Services Software License Agreement or Software License and Maintenance Agreement for additional information. License Requirements for Third-Party Software included with Analytix: Mapping Manager Certain third-party software such as Eclipse Rich Client may be included with Analytix Data Services Software. Such software is provided pursuant to their respective licenses and not under Analytix Data Service s licenses. Please see their respective license documents for their terms and conditions. Disclaimers Analytix Data Services reserves the right to revise this manual and make periodic changes to its content without obligation on Analytix Data Services Corporation's part to notify any person of such revisions or changes. Analytix Data Services does not assume responsibility for the use of this manual. Analytix Data Services makes no representation or warranty as to the accuracy of the contents of this manual. All statements made and information provided in this manual is provided on an errors and omission accepted (E. & O.E.) basis only. Analytix Data Services LLC Page 2 of 62

3 Table of Contents Intended Use... 4 Introduction to Analytix: Mapping Manager... 4 About Analytix Mapping Manager... 4 Mapping Manager Modules... 4 Mapping Manager Access & Sign-In... 8 Getting Started Preparing the Analytix Mapping Manager Environment Verify Web Browser Settings... 9 Viewing Available Security Roles (Role-Based Security) Creating Users and Assigning Permissions Creating a Project Define Systems and Upload System Metadata Defining a System Importing Metadata for a System Environment Upload System Metadata using MS Excel metadata Upload template Import MS Excel Metadata Column Definitions Print System Reports Creating and Managing Mapping Specifications Define Transformations (Drive Enterprise Mapping Standards) Create and Manage Mapping Specifications Fully Metadata Driven Mapping Approach (Drag & Drop) Creating One-to-Many and Many-to-Many Mapping Specification Data Grid Deleting Rows from the Mapping Specification Data Grid Preview Data from Actual Databases Manually Enter (free form) Upload Mapping Specifications from ADS Template Mapping Specification Upload Template - Metadata Definitions Adding Mapping Details Using the User-Defined Flex Fields Printing the Mapping Specifications Performing Impact Analysis Running the Impact Analysis Report (Table and Column Impact) Running the Impact Analysis Report (Enterprise Transformation Impact) Exporting Mapping Specifications...43 Export an Individual Mapping Specification Export all Mapping Specifications at the Project Level Export Mapping Specification as an XML File Versioning, Base-Lining and Mapping Comparison...48 Create a New Version Create a new Base-Line Version Deleting a Mapping Specification & Rolling Back Changes to a Previous Version Mapping Comparison (Compare a Mapping Specification to Another Mapping Specification) Create and Maintain Data Dictionaries...57 Enter and Maintain Data Dictionaries Upload ETL Jobs as a Data Mapping Specification UpLoad ETL XML File as a Data Mapping Specification (Reverse Engineer an ETL Job) Appendix...60 Role-Based Security Definitions Global Security Roles: Grants & Controls Functional Security Roles: Grants & Controls Analytix Data Services LLC Page 3 of 62

4 Intended Use This document is a Quick-Start User Guide intended to train users on how to get quickly acclimated to the Analytix Mapping Manager application and using it to create dynamic metadata driven source to target mapping specifications. It is important to note this documents is not intended to be a fully detailed and in depth users guide. Thereby, this Quick-Start guide will cover only the basics in terms of the usability of the application. Many of the features and capabilities will not be covered in the Quick-Start User Guide. Analytix Data Services recommends clients procure onsite training or services for hands on training and mentoring of resources on usability and best practices regarding the use of Analytix Mapping Manager. Introduction to Analytix: Mapping Manager This section is intended for users of Analytix Mapping Manager who want to obtain an overview of the application. In this section, you will learn: About Analytix Mapping Manager Mapping Manager Modules Terminology About Analytix Mapping Manager Analytix Mapping Manager is a feature rich and intuitive graphical user interface (GUI) application which allows users to create, version and manage standards based enterprise source to target mapping specifications for integration projects. The application has built in features to better manage the analysis and requirements aspects of the project along with various levels of documentation approvals for Systems Development Methodology (SDM) compliance and project delivery standards. Analytix Mapping Manager enables users to automatically connect to, upload and import relational database metadata (technical and business), upload and share and collaborate on shared documents at many levels (the project, the system, the mapping specification) and dynamically build source to target mapping specifications by dragging and dropping data elements from the metadata browser. The Mapping Manager modules enables users to control versioning of mapping specifications, Impact Analysis at multiple levels (system, tables and columns), drive standards with shared user defined transformations and audit various levels of compliance (data model standards, systems development methodology SDM and Mapping Specification standards. Mapping Manager Modules The Mapping Manager is comprised of three modules. Each of the three modules represents a sequential workflow. Add Resource/Application User, Create Project (Resource Manager), Define Systems and upload metadata (System Manager), create Mapping Specifications (Mapping Manager). The modules are listed below along with some of the key features comprising the workflow. Analytix Data Services LLC Page 4 of 62

5 RESOURCE MANAGER / PROJECT MANAGER The resource manager contains the administration features of the application which are creating roles, users, projects and assigning permissions to application users. Create security roles Create application users and assign permissions and roles Create a project Assign application users to projects Enter Project Details Upload project files and manage documentation approvals SYSTEM MANAGER The System Manager is the modules where all the system analysis and documentation is stored. The modules allow users to create new systems, document the technical aspects of the system, specify connectivity details for multiple environments (dev, test, production etc), upload system documentation and files at the system level and automatically import the technical and business metadata at the environment level where it will later be used by the Mapping Manager module to drag and drop data elements when building the source to target mapping specifications. Add a system and enter systems analysis details Add environments (development, test production) Upload system metadata Upload system files and manage approvals MAPPING MANAGER The Mapping Manager is the module which will be used most frequently by most users. The Mapping Manager allows users to create shared user defined transformations that can be used to drive mapping standards across the enterprise. Once the shared transformations are defined users can create dynamic source to target mapping specifications by dragging and dropping items from the metadata browser to the integrated data grid. The mapping manager also has built in capabilities for versioning the mapping specifications, defining the source extract SQL, Definition of Update Strategies, integrated testing and ability to upload advanced technical specifications at the mapping specification level. Create shared user defined transformations View system business and technical metadata Create, Modify & Delete Mapping Specifications Version Mapping specifications Upload existing Mapping Specifications via.ms Excel upload template Enter advance mapping Specification details o Define Source Extract SQL o Define Update Strategy o Enter testing dispositions and notes at the mapping specification level o Upload advanced technical specifications at the mapping level and track documentation approvals Track status of mapping specification (completed vs work in-progress). Track changes of mapping specifications Terminology Analytix Mapping Manager Terminology Tree. Refers to objects/nodes which are expandable and collapsible in the tree menu. Analytix Data Services LLC Page 5 of 62

6 Roles. Roles are synonymous with group permissions. Permissions can be assigned to the user by adding roles. Right click (menu). The application supports additional actions by right clicking the mouse when giving focus to an item (left-click) such as an item on a tree. When right clicking an additional menu appears relevant to the item that you are selecting. Transformation. Transformation refers to the shared user defined transformations available in the project browser. Environment. Refers to system environments where connectivity instructions and metadata can be uploaded to the central repository. The System Manager can upload any number of development (n), test (n) and production (n) environments. Upload. Refers to the uploading of files or metadata to the central database repository for shared and collaborative access by other enterprise users. Mapping Manager Module Frame Terminology Project Browser. The left most frame where the transformations, projects and mappings are stored in the project tree. Metadata Browser. The right frame where the systems, tables and columns are stored. Columns support drag and drop from the Metadata Browser to the Data Grid. Metadata Properties. The metadata properties are a component of the metadata browser and is located directly below the Metadata Browser to allow visibility into the Technical Properties (datatype, precision, scale etc) and Business Properties (table and column level business terms). Data Grid. The center frame where the mapping specifications are defined. The Data Grid is integrated with both the Project Browser and the Metadata Browser. Mapping Details. The details are located at the bottom of the screen and contain the extended mapping specification details, testing dispositions and notes as well as uploaded technical specifications. Enterprise Transformations. Refers to any transformation that is entered into the list of transformations within the Mapping Manager Module (Transformations Node) Non-Enterprise Transformation. Refers to any transformation that is not in the list of transformations within the Mapping Manager Module and is thereby manually entered. Mapping Manager Module Screen Layout The Mapping Manager module is broken out into the following five sections using resizable frames. Tip: Each frame is resizable. Users can resize any of the frames to their preference by sliding the frame left, right, up or down. Analytix Data Services LLC Page 6 of 62

7 Metadata Browser Project Browser Data Grid Metadata Properties Mapping Details Analytix Data Services LLC Page 7 of 62

8 Mapping Manager Access & Sign-In See your Mapping Manager Administrator for the URL for system access and your user id and password for the system. (Log-in Screen is below) Getting Started Preparing the Analytix Mapping Manager Environment. Before getting started and creating mapping specifications, there are a couple steps that need to occur prior to working with the Mapping Manager Module. In this section, you will learn: 1. Verify Web Browser Settings 2. Viewing Available Security Roles (Role-Based Security) 3. Creating Users and Assigning Permissions 4. Creating a Project 5. Define Systems, Upload System Metadata Analytix Data Services LLC Page 8 of 62

9 6. Print System Reports Verify Web Browser Settings 1. Mozilla FireFox. If you are running Mozilla Firefox v 3.0 or greater you will need to download the IE Explorer add-on directly from Mozilla. This will enable FireFox to render IE Explorer Application/Websites seamlessly from the FireFox web browser. 2. Internet Explorer. If you are running IE Explorer, verify your web browser version is greater version 6 or greater. Note: if you are running Internet version 8 you will need to set the Compatibility View Settings from the tools menu. 3. Allow Pop-Ups from the mapping manager login url. From the web browser go to Tools\Pop-Up Blocker Setting and add url to allow pop-ups. Analytix Data Services LLC Page 9 of 62

10 Viewing Available Security Roles (Role-Based Security) 4. Click on the Resource Manager icon and Navigate to the Roles tab. 5. You can now see all available security Roles. Each of the Security roles can be assigned to users. Note: One User can have multiple security roles assigned. For example, one user can have Project Admin, System Admin and Transformation Admin roles assigned. When any non-public role is assigned to a user. The public role is also assigned by default granting the user to have Read-Only access to all modules. 3. For more information on available roles and specific grants available please see the Appendix: Role-Based Security Definitions. Analytix Data Services LLC Page 10 of 62

11 Note: The only difference between a Power User Role and an Administrator Role is the Administrator can add user to the Mapping Manager System. Tip 1. Locking Down Security We recommend granting roles based on functional use internal to the organization. For example, we recommend assignment of the following roles to lock down certain access in large organizations where data governance may want to restrict changes to Enterprise Transformations (Role: Transformation Admin), Updating and Refreshing of Metadata (Role: System Admin) Etc. Tip 2. Liberal Use Security Recommended for smaller development teams where various internal teams are not involved involved in the Mapping Specification, ETL Development, Governance and Testing process. For these organizations you may want to assign a power user role to all users granting read/write access to all modules, projects and mapping specifications. Creating Users and Assigning Permissions This section details the steps to create a user and assign role-based security settings. It is important to note that prior to creating a user which role-based security setting or settings should be assigned to the user. (the application support multiple roles assignments to a single user. For more information on available roles and specific grants available please see the Appendix: Role-Based Security Definitions. 1. Click on the Resource Manager icon and Navigate to the Users tab. Analytix Data Services LLC Page 11 of 62

12 2. Right click anywhere on the Users tree to delete or add a new user. Select New User 3. Enter the user information, specify all roles the user should be assigned. Click Save. User has been created and can login with their username and password. Analytix Data Services LLC Page 12 of 62

13 Creating a Project 1. Click on the Resource Manager icon and Navigate to the Projects tab. 2. Right click anywhere on the Tree to delete or add a New Project. Select New Project 3. Enter the descriptive Project information and Click Save. The New Project is now available for use in the Mapping Manager Module. 4. To Add project Documents or Users Navigate to the Project Documents or Project Users Tab respectively and click Add New. 5. Enter the descriptive information and save. The documents and users are now available to the enterprise for shared and collaborative access. Define Systems and Upload System Metadata Defining a System 1. Click on the Systems Manager icon and Right click anywhere on the Tree to delete or add a New System. Select New System 2. Enter the descriptive System Information and Click Save. The New System is now available in the Mapping Manager Module within the Metadata browser. Tip: The New System appears in the Metadata Browser; however it is not available until you upload the metadata for the system. See Next Steps. Analytix Data Services LLC Page 13 of 62

14 3. Go to System Environments tab and click add new Analytix Data Services LLC Page 14 of 62

15 4. Next enter the required fields below. Note: only System Environment Name and System Environment Type are required to save the record. The other fields are specific to your database system environment and are required when importing the system metadata. Tip: If you do not know your environment technical details consult your database administrator or your server administrator for the technical information specific to your environment. Although any user id with read permissions will work, as a best practice, Analytix recommends database administrators create a non-expiring read-only application id (user account) for the Mapping Manager on all systems. This user id should have read only access to the system database catalogs. Suggested Application Id: ADSMMRO (definition: ADS I Mapping Manager Read-Only) Analytix Data Services LLC Page 15 of 62

16 Importing Metadata for a System Environment Tip: The Mapping Manager supports the adding, storing and importing of metadata for multiple system environments (e.g. multiple test environments). As a best practice Analytix recommends users add all system environments and define each of their business purposes (e.g. Unit Testing, User Acceptance Testing, System Regression Testing etc). Analytix does NOT recommend users import metadata for each of these environments (unless, there is metadata (table/columns) that are required for source to target mapping purposes or metadata analysis purposes). Analytix recommends importing metadata for production environments to guarantee stable metadata while creating source to target mapping specifications. Of course the stability will be dependant on each of the systems and their state of ongoing development, support and maintenance releases. 5. Verify the correct technical information is correct for the specified environment Analytix Data Services LLC Page 16 of 62

17 6. Click test to ensure you have entered the correct technical information for your environment: 7. Next Click the import Metadata icon 8. Select the tables to import and click Save. Analytix Data Services LLC Page 17 of 62

18 9. Confirm the importing of the metadata tables. 10. The Imported metadata is now available in the Mapping Manager Module within the Metadata Explorer and ready for use (drag and dropping) to build mapping specifications. Tip: Remember to define an exhaustive list of enterprise transformation rules to drive source to target mapping standards across the enterprise. See next Step (Creating & Managing Mapping Specifications Section: Define Transformations). Upload System Metadata using MS Excel metadata Upload template The System Manager supports file based metadata to be uploaded via an import template that is available as a download from the system manager module. Analytix Data Services LLC Page 18 of 62

19 Tip: The MS Excel file based upload can be used for any type of metadata however should be used primarily for flat file definitions and format conversion standards for EDI type of transactions e.g (SWIFT & ISO standards for financial services exchange formats). Once the metadata is uploaded via MS Excel file it is then made available in the metadata browser to be used as drag and drop objects to define mapping specifications. To get started 1. Click on the Systems Manager icon then click the download metadata template icon 2. The following dialog box appears prompting you for the location to save the download template. 3. Fill in the metadata contents of the MS Excel file and Save. Note: All metadata columns labeled as a flag must contain the valid values: 1(true) or 0(false). For more information regarding the import column definitions see the section below Import MS EXCEL Metadata Column Definitions. 4. Click the System Environment tab of the System Manager and click add new environment Analytix Data Services LLC Page 19 of 62

20 5. Specify the Database Type as Ms Excel and click Save. The new environment then appears in the list of available environments. 6. click the MS Excel Reload System Icon and browse to the file you want to upload 7. When the metadata had finished uploading the follow dialog box will appear. Analytix Data Services LLC Page 20 of 62

21 8. The uploaded metadata now appears in the Metadata Browser of the Mapping Manager Module and is identified with a flat file imported icon to indicate the metadata was loaded and available as a MS Excel file. Analytix Data Services LLC Page 21 of 62

22 Import MS Excel Metadata Column Definitions The column definitions below contain the file format column definitions and requirements for importing MS Excel metadata into the system manager.! Note: Make note of the Red values below for identity and flag columns. Column Definition Data Type length Description TABLE_NAME varchar(50) Name of the source or target table name TABLE_DEF_COMMENTS varchar(max) Business definition of the table. COLUMN_NAME varchar(50) Name of the column from the table COL_DEF_COMMENTS varchar(max) Business definition of the column. COL_DATATYPE Identifies the Data type of the column varchar(20) Eg. varchar, nchar, number etc COL_LENGTH Identified the length of the column varchar(20) E.g. varchar(20), here 20 max char s are allowed. COL_PRECISION Required for numeric data types varchar(20) indicates the length a number can be to the left of the decimal point. COL_DB_DEFAULT_VALUE Column database default value E.g. default value for varchar is, null varchar(50) value etc. This is the default value the database assigns when inserts and updates contain nulls COL_ETL_DEFAULT_VALUE ETL default value for this column. Use varchar(50) this column to specify a default value for COL_IDENTITY_FLAG COL_NULLABLE_FLAG varchar(1) valid values: 1(yes) or 0(no) varchar(1) null values. Enter 1 Yes, if it is identity column 0 No, if it is not a identity column Enter 1 Yes, if nulls are allowed 0 No, if nulls are NOT allowed (valid values: 1 or 0) MIN_VALUE varchar(20) Minimum value for this column value MAX_VALUE varchar(20) Max value supported for this column PERCENT_NULL_VALUE varchar(20) Column percent null COL_SCALE PRIMARY_KEY_FLAG varchar(20) varchar(1) (valid values: 1 or 0) Required for numeric datatypes. Indicates the decimal point for rounding and how many decimals the column allows (e.g. a value of 4 indicates the are four decimals places to the right) Defines the column as a primary Key (unique identifier). Enter 1 Yes, if it is primary key 0 No, if it is not a primary key NATURAL_KEY_FLAG varchar(1) (valid values: 1 or 0) Defines the column as a Natural Primary key. Typically this is used only where natural Primary keys have been replaced by Surrogate Keys of an integer data Analytix Data Services LLC Page 22 of 62

23 type for performance purposes. Enter 1 Yes, if it is natural key 0 No, if it is not a natural key Print System Reports 1. Click on the Systems Manager icon and Right click anywhere on the Tree to delete, add a New System, or Print System Report 2. Select Report 3. Your System report is displayed below as follows. The system Details, environment details and any attached files and documents are shown for any specified system. Analytix Data Services LLC Page 23 of 62

24 Creating and Managing Mapping Specifications. Now that you have created users, created a project, defined systems and uploaded metadata you can begin creating Mapping specifications by dragging and dropping items from the Metadata Browser to the Data Grid (Mapping Specification). Analytix Mapping Manager is a tool designed to quickly build metadata driven source to target mapping specifications and enterprise standards around managing the mapping specifications process and tracking data lineage across the enterprise. Thereby the first place we start in the mapping manager module is to define an exhaustive list of transformations available at the enterprise level. Analytix Data Services LLC Page 24 of 62

25 Tip: Transformations are Shared user defined Transform Specs which are available for use in the Data Grid while building mapping specifications. These transformations are available to ALL projects and should be governed to control mapping standards at the transformation level. Analytix Recommends clients define an exhaustive list of Enterprise Transformations upon setup and routinely audit mapping specifications for compliance of these standards. These can be manually entered or imported/exported via MS Excel. In this section, you will learn: 1. Define Enterprise Transformations (Drive Enterprise Mapping Standards) a. Manually Enter (free form) b. Upload/Export Mapping Specifications from ADS Template c. Run Impact Analysis on Enterprise Transformations (determine usage). 2. Create and Manage Mapping Specifications a. Fully Metadata Driven Mapping Approach (Drag & Drop) b. Manually Enter (free form) c. Upload Mapping Specifications from ADS Template 3. Adding Mapping Details 4. Printing the Mapping Specifications 5. Performing Impact Analysis Define Transformations (Drive Enterprise Mapping Standards) Now that you have purchased Analytix Mapping Manager, you have already taken the first step toward enterprise integration standards around managing the source to target mapping specification process. The next step is a critical step in the process of maintaining a complete set of enterprise standards. While The Mapping Manager is flexible enough to allow users to enter their own free form transformations Analytix maintains this should be on an exception basis. Upon Setup it is highly recommended that clients define an exhaustive list of Enterprise Transformations that can be shared across the enterprise. This will not only drive standards but also, encourage use, streamline processes, efficiency and productivity of all constituents involved in the process now that they are using a set enterprise transformations and familiar with the transformation name, term and use. Define Enterprise Transformations 1. Click on the Mapping Manager icon and Navigate to the Transformations Node. Manually Enter Enterprise Transformations 1. Double-Click on the last row to add a new transformation and enter the following a. Transformation Name b. Pseudo Code c. Intended Use Description Note: you can tab to the next cell without clicking. Entering all data into the data grid automatically saves the record and takes you to the next record. Analytix Data Services LLC Page 25 of 62

26 Upload/Export Enterprise Transformations 1. Click on the download Transformation template to download the MS Excel file to enter the transformations. 2. Next, Right-Click on the transformation node and select Upload to browse for the file and load the transformations to the Grid. 3. Choosing Export Transformation will export all Transformations to MS Excel. The same file can then be edited and uploaded again to add new transformations. Deleting Enterprise Transformations 1. Select the row required for deletion. 2. Click the delete button and confirm the deletion Tip: If the transformation was used in Mapping Specifications it will still be there. The transformation will NOT be available as a selectable transformation in the mapping specification data grid within the Business Rule column. Analytix advocates defining Enterprise Transformations should be a collaborative effort across project and both Business Analyst (Mapping Designers) and ETL Developers (responsible for reading the mapping specifications and writing the procedural code). Analytix Data Services LLC Page 26 of 62

27 View Audit Trail and Change History for Transformations 1. Click the Icon and the audit trail and change history of the Transformation will be displayed. Run Impact Analysis for Enterprise Transformations (Determine Usage) 1. Select the transformation name in the data grid and Right-Click to choose default or advanced search. Default will pass the transformation name and immediately perform a search. Advanced search will prompt an editable dialog box for user to enter wildcard like searches (e.g. DataGo will display transformation match on all like base searches, DataGov and DataGovernance entries will all be returned). Create and Manage Mapping Specifications Now that you have created an exhaustive list of Enterprise Transformations (although you will want revisit and audit the use non-enterprise transformations routinely) we can now begin to create fully metadata driven source to target mapping specifications. The Mapping Manager supports three ways of creating a mapping specification. In this section, you will learn: Analytix Data Services LLC Page 27 of 62

28 1. Fully Metadata Driven Mapping Approach (Drag & Drop) 2. Preview Data from Actual Databases 3. Manually Enter (free form) 4. Upload Mapping Specifications from ADS Template Tip: Analytix advocates use of the fully metadata driven Mapping Approach to drive mapping standards, reduce data entry errors, increase quality of the mapping specifications and allow greater use of metadata reporting and impact analysis at the column metadata level. Manually Entered Mapping Specifications should be used on a case by case and exception basis only. Fully Metadata Driven Mapping Approach (Drag & Drop) 1. Click on the Mapping Manager icon and Navigate to the Project where you want to create the mapping specification. Select the Project Node. When you select the project the list of mapping specifications that are already created appear in the center frame. 2. Right click on the project node to create a new Mapping Specification. The following menu appears: 3. Select New and Enter the Mapping Specification High Level details. Note: the version is defaulted to From the Project Browser Click on the New Mapping Spec you created. The Mapping Manager Data Grid, Mapping Details and the Metadata Browser now ALL appear. Analytix Data Services LLC Page 28 of 62

29 5. From the Metadata Browser expand all nodes to see tables and columns from the systems in which you want to build your mapping specification from. 6. Click the columns you want as either a source or target and drag to any of the source (SRC) or target (TGT) column and the column and ALL of its technical metadata come with it. Note: Selecting multiple columns from the metadata browser. To select more than one column at a time HOLD the Shift or Control key. You can also select multiple columns from multiple systems (not just the table you are working in). Note: Organizing the mapping specification Data Grid: Column Sorting. All columns can be sorted by double clicking the column Column Reordering. All columns can be reordered by dragging and dropping columns on top of one another. Extended metadata viewing within the grid is possible by right clicking on any column to see the list of available grid items. Resizing of frames to increase viewing area of the grid. All frames can be resized by sliding the edge of the frame left, right, up or down. 7. Double Click the Business Rule column and the list of enterprise transformations are available. Select the one that is appropriate for the movement of that data item and enter any additional transformation rules Extended business rules Transformation Lookup Rule Analytix Data Services LLC Page 29 of 62

30 8. If the row is in the mapping specification has been reviewed and is considered complete, Mark the status checkbox with a check and click save. Tip: This is a way of tracking the status of every row in a mapping specification to ensure the entire specification has been reviewed and is considered complete. Note: you do not have to click save after every row. You can select save when you are ready to save multiple rows or the complete mapping specification. 9. Repeat as necessary to finish the mapping specification. Note: You can visibly tracking Changes within the data Grid prior to saving the mapping specification. New metadata driven changes (drag & drop from the metadata browser) are identified with Blue exclamation marks (!) to indicate new rows; Red exclamations marks (!) indicated manually overwritten or modified changes to existing rows. 10. Click Save to save your work. Analytix Data Services LLC Page 30 of 62

31 Creating One-to-Many and Many-to-Many Mapping Specification Data Grid The Integrated Data Grid supports the creation of one to many and many to many mapping relationships at the row level via the drag and drop metadata browser. Creating One-to-Many and Many-to-Many Mapping Specifications 1. Check the Append Check-Box 2. Next select multiple attributes from the metadata browser and drag to any source or target column in the data grid with a SRC or TGT. Deleting Rows from the Mapping Specification Data Grid The Integrated Data Grid supports the deletion of single or multi-row deletes from mapping specification. To delete rows of a mapping specification 3. select the row and click the delete icon Note: To select multiple rows press the CTRL Key while you select multiple rows with the mouse. Analytix Data Services LLC Page 31 of 62

32 4. Confirm the deletion by selecting OK. Preview Data from Actual Databases The Metadata Browser of the Mapping Manager Module allows users to preview the actual data in the system environment. This is useful when users need to preview the data at the time of defining the mapping specification to verify the data in the expected format without writing a line of code to do so. 1. Click on the Mapping Manager icon and Navigate to the Metadata Browser. 2. Right Click anywhere on the tree and choose Preview Data The top 100 records in the system environment are selected and displayed for user to analyze. Analytix Data Services LLC Page 32 of 62

33 Manually Enter (free form) Manual entering capabilities are present in the Mapping Manager, However it is recommended that manually entered specifications be used on a case by case and exception basis. 3. Follow steps 1-4 above to create a new mapping spec. 4. Double-Click on the last row to edit the data grid. 5. Enter mapping specification values for source and source, target metadata Note: Tab to the next cell. 6. Double Click the Business Rule column and the list of enterprise transformations are available. Select the one that is appropriate for the movement of that data item and enter any additional transformation rules Extended business rules Transformation Lookup Rule 7. Save. Upload Mapping Specifications from ADS Template The mapping manager has the capability to import pre-existing mapping specification that you may already be maintaining. These mapping specification can be imported into the Mapping Manager by using the Mapping Managers ADS Import template. First the respective columns on the pre-existing mapping specifications must be copied and pasted into the ADS import template and then they are ready for upload. 1. From the Module Menu Click the Download Mapping Spec template icon 2. Save the MS Excel template to disk. You will need this for import later. Analytix Data Services LLC Page 33 of 62

34 3. Open the MS Excel and copy and paste your existing mapping specifications into this template. You will need to save the template and save the file. Note: to see a list of the metadata column definitions see the section below: Mapping Specification Upload Template - Metadata Definitions. 4. Confirm your cut and paste have been aligned to the correct column in the ads template 5. From the Project browser Navigate to the New Mapping Specification you have created and want to import the MS Excel template to, Right Click and Choose Upload Excel. Note: Imports can only occur on new mapping specifications without any existing rows. New rows can be added after the import if complete. 6. Browse your file directory to the location you save the MS Excel template to and Click Upload. Analytix Data Services LLC Page 34 of 62

35 7. Confirm your Upload message 8. Click Save. Your mapping specification has been imported and can now be maintained using the Mapping Manager. Mapping Specification Upload Template - Metadata Definitions The table below contains the MS Excel template column definitions for importing existing mapping specifications via MS Excel. Note: there are 37 columns in the template. The upload capability uses a column header match while uploading (not file structure) so you can delete the columns which you do not need while uploading the mapping specification. Column Definition Data Type length Description EXTENDED_BUS_RULE varchar(max) Extended business rule definition for transformations between source column to target column. Should be used as and extension to the business rule column which is defined in the section: Enterprise Transformations. SRC_SYSTEM_NAME varchar(50) Source system name SRC_SYSTEM_ENVIRONMENT_NAM E varchar(50) Source environment name, source system can have multiple environments SRC_TABLE_NAME varchar(50) Source table name SRC_COLUMN_NAME varchar(50) Source table column name for which mapping needs to be done SRC_COL_DATATYPE varchar(20) Source column data type Analytix Data Services LLC Page 35 of 62

36 e.g.: varchar, number etc SRC_COL_LENGTH varchar(20) Source column length. SRC_COL_PRECISION Source column precision. Required for varchar(20) numeric data types. Defined the total number of digits in the number to the left of the decimal point. SRC_COL_SCALE Target column scale. Required for numeric datatypes. Indicates the number of values to varchar(20) the right of the decimal point for rounding and how many decimals the column allows (e.g. a value of 4 indicates the are four decimals places to the right) SRC_COL_DB_DEFAULT_VALUE varchar(50) Source column database default value SRC_COL_DEF_COMMENTS varchar(max) Source column description/comments SRC_COL_IDENTITY_FLAG SRC_COL_NULLABLE_FLAG char(1) (valid values: 1 or 0) char(1) (valid values: 1 or 0) 1 Yes if it is column identity 0 No if it is not a identity column 1 Yes if it is null 0 No if it is not a null SRC_PERCENT_NULL_VALUE varchar(20) Source column null able percent SRC_NATURAL_KEY_FLAG char(1) 1 - Yes if it is natural key (valid values: 1 or 0) 0 No, if it is not a natural key SRC_MIN_VALUE varchar(20) Source minimum value SRC_MAX_VALUE varchar(20) Source max value TGT_SYSTEM_NAME varchar(50) Target system name TGT_SYSTEM_ENVIRONMENT_NAM E varchar(50) Target system environment name TGT_TABLE_NAME varchar(50) Target table name TGT_COLUMN_NAME varchar(50) Target system table column name TGT_COL_DATATYPE varchar(20) Target column data type e.g.: varchar, number, string etc TGT_COL_LENGTH varchar(20) Target column length. TGT_COL_PRECISION varchar(20) Target column precision. Required for numeric data types. Defined the total number of digits in the number to the left of the decimal point. TGT_COL_SCALE varchar(20) Target column scale. Required for numeric data types. Indicates the number of values to the right of the decimal point for rounding and how many decimals the column allows (e.g. a Analytix Data Services LLC Page 36 of 62

37 value of 4 indicates the are four decimals places to the right) TGT_COL_ETL_DEFAULT_VALUE varchar(50) Target column ETL default value TGT_COL_DEF_COMMENTS varchar(max) Target column description/comments TGT_NATURAL_KEY_FLAG TGT_PRIMARY_KEY_FLAG char(1) (valid values: 1 or 0) char(1) (valid values: 1 or 0) 1 Yes, it is natural key 0-No, if not natural key 1-Yes, if it is primary key 0-No, if it is not a primary key TRNSF_LOOKUP_CONDITION varchar(max) transformation for mapping BUSINESS_RULE varchar(50) The Enterprise Transformation/Business rule for mapping the source to target column. COL_MAP_COMPLETE_FLAG char(1) (valid values: 1 or 0) 1-Yes, if it is complete flag 0-No if it is not complete flag USER_DEFINATION1 varchar(max) User description notes USER_DEFINATION2 varchar(max) User description notes USER_DEFINATION3 varchar(max) User description notes USER_DEFINATION4 varchar(max) User description notes USER_DEFINATION5 varchar(max) User description notes Adding Mapping Details The Mapping Details section allows the user to view additional mapping specification details, versions, define source system extract SQL, Define advanced target update strategies, workflow process tracking and integration with development and testing (testing notes and dispositions) and upload any set of attached files that are pertinent to the mapping specification such as Advanced Technical specifications that may be relevant for the individual mapping specification. 1. To Edit any Tab, Navigate to the respective Tab Analytix Data Services LLC Page 37 of 62

38 2. Click the EDIT button to edit the tab information 3. Next click either the Save or Cancel Button when Finished. Using the User-Defined Flex Fields The Mapping Manager has captures thirty-two metadata elements at the row level of the mapping specifications. If there is a data element that is not captured and labeled as per your specification we have also provided the use of 5 flexible user defined fields that can be used any way you deem appropriate at the row level of the mapping specification. To use the User defined fields 1. Click on the Mapping Manager icon and Navigate to the target mapping specification. 2. Next Right Click on the column headers of the Mapping Specification Data-Grid and select the user defined fields to be added to the grid layout Note: when you do this the grid will remember your last layout setting and it will be saved when you log back in later). Analytix Data Services LLC Page 38 of 62

39 3. The newly added columns are added to the grid-layout and are available for use. Printing the Mapping Specifications The Mapping Manager supports the printing of any Mapping Specification for a multitude of purposes. Primarily the Mapping Specification Report will be used as a handoff item (requirements) for ETL developers to create the ETL jobs. 1. From the Mapping Manager s Project Browser navigate to the Mapping Specification you would like to print. 2. Right Click to bring up the menu and select Print Report. 3. The Report Appears. Analytix Data Services LLC Page 39 of 62

40 Performing Impact Analysis. The Mapping Manager supports impact analysis at the table, column and enterprise transformation level. This is particularly useful in assessing the impact of any global transformation change, source or target level changes and their impact any project and the number of mappings or load process that may be affected resulting from the change to a system, table or column in a table. In this section, you will learn: 1. Running the Impact Analysis Report (table and Column Impact) 2. Running the Impact Analysis (Enterprise Transformation Impact) Running the Impact Analysis Report (Table and Column Impact) 1. From the Mapping Manager s Metadata Browser navigate to the System and Table or Column you would like to perform the Impact Analysis on. 2. Right Click and Select Impact Analysis Report Analytix Data Services LLC Page 40 of 62

41 Tip: Impact analysis can be performed on both tables and columns. The dialog box also allows the user to override any defaults from the metadata browser and search for any manually entered table or columns which may have been manually entered. 3. Choose General Search or Wildcard Search to over-ride any of the default values. Click Report or edit and override any defaults to search for any manually entered source or target tables or column in the mapping specifications. Then Click Report 4. Your Impact Analysis Report is displayed and indicates where the data item is used. Running the Impact Analysis Report (Enterprise Transformation Impact) Running impact analysis will display a report contain a list of project and mapping specifications where the transformation is used across all enterprise projects. 1. From the Mapping Manager s project browser select Transformations 2. Select the target enterprise transformation row and right click to run impact analysis. Analytix Data Services LLC Page 41 of 62

42 Analytix Data Services LLC Page 42 of 62

43 Exporting Mapping Specifications The mapping manager allows mappings to be exported as MS Excel files at the individual mapping specification level and at the project level. Tip: When you export all Mapping Specifications at the Project Level all CURRENT Mapping Specifications will be exported as a collection of MS Excel Files. None of the previously versioned mapping specifications located in the archive folders will be exported. To export previously versioned mapping specifications you must use the export individual mapping specification capabilities. In this section, you will learn: 1. Export an Individual Mapping Specification 2. Export all Mapping Specifications at the Project Level Export an Individual Mapping Specification 1. Click on the Mapping Manager icon and navigate to the Mapping Specification you want to export. Note: you can select a current mapping specification or a previously versioned mapping specification located in the archive folder of the project browser. Analytix Data Services LLC Page 43 of 62

44 2. Click the export as MS Excel icon and navigate to the directory where you would like to store the exported MS Excel file. Export all Mapping Specifications at the Project Level 1. From the Mapping Manager s Metadata Browser navigate to the System and Table or Column you would like to perform the Impact Analysis on. 2. Select the project in which you would like to export all Mapping Specifications and Right Click and choose Export All. Note: you must select the correct project. Analytix Data Services LLC Page 44 of 62

45 Note: When you select Export All ALL current mapping specifications are exported at the project level. The previously versioned mapping specifications located in the archive folder are not exported. 3. Next the following dialog box opens and will prompt you for the directory where you would like the.zip file to be saved. Analytix Data Services LLC Page 45 of 62

46 Export Mapping Specification as an XML File The mapping manager integrates with ETL Tool vendors using XML export capabilities. The mapping specifications can be exported as an XML file and then interpreted with ETL Tools such as Informatica, DataStage and Abinitio to generate a base-level data movement ETL Job. To export the mapping specification as an XML file 1. Click on the Mapping Manager icon and navigate to the Mapping Specification you want to export. Note: you can select a current mapping specification or a previously versioned mapping specification located in the archive folder of the project browser. Analytix Data Services LLC Page 46 of 62

47 2. Click on the Export XML Icon and navigate to the directory where you would like to store the exported XML file. 3. Next the following dialog box opens and will prompt you for the directory where you would like the.zip file to be saved. Analytix Data Services LLC Page 47 of 62

48 Versioning, Base-Lining and Mapping Comparison Analytix Mapping Manager offers robust change control and versioning features as well as a visual comparison of changes between mapping specifications. The advanced versioning capabilities of the Mapping Manager is what puts our application years ahead of our competition and facilitates a scalable, ETL agnostic solution where mapping specifications can survive the life cycle and lifespan of any system or integration project. Mapping Specifications can be versioned at the individual mapping specification (Versioning) or at the Project Level (Base-Lining). The robust yet flexible versioning capabilities provides the flexibility of managing the ever changing source to target mapping specifications at the Change/ Individual Mapping Specification Level or at the collection of mapping specifications at the Project Level. Just like popular code versioning systems (Change Control) prior to a mapping specification being versioned by a user the system forces the user to enter a change description prior to creating a new version. Advanced mapping comparison capabilities allows users to quickly and efficiently compare any two mapping specifications and visually view the changes between the two in a horizontal, vertical and row by row basis. The robust comparisons capabilities are not limited to a mapping specification and its previous versions (archive contents). The advanced comparison capabilities allow users to compare a mapping specification to any mapping specification across all projects (e.g. parallel project initiatives etc). In this section, you will learn: 1. Create a New Version 2. Create a new Base-Line Version 3. Deleting a Mapping Specification & Rolling Back Changes to a Previous Version 4. Mapping Comparison (Compare a Mapping Specification to Another Mapping Specification) Create a New Version 1. Click on the Mapping Manager icon and navigate to the Mapping Specification you want to version. 2. Right Click on the Mapping Specification and Select New Version. The following dialog box shows. Note: The mapping version number will increment automatically Version label is a label that can be applied by any user The system forces the user to enter a change description for audit control purposes Analytix Data Services LLC Page 48 of 62

49 3. Click Save. The archive folder is created in the project browser tree and the new versions are created in the archive folder and appear with the version suffix in the name (version 1.01 etc). Note: All previously versioned mapping specifications located in the archive folder are read-only and cannot be edited. Create a new Base-Line Version Base-Lining a project will bring ALL mapping specifications in the project to the same version (Max of all version numbers) number. 1. Click on the Mapping Manager icon and Navigate to the project where you would like to baseline (increment all mapping specification version numbers). 2. Select the Project, Right Click and Choose New Baseline Analytix Data Services LLC Page 49 of 62

50 Note: Note that prior to performing the baseline the current mapping specification for the collection of current mapping specifications are at version 1.02 and 3.00 in the screenshot above. After performing the New Base-Line the all current mapping specifications will be incremented (max) to a version number of (See screenshot is step 4 below.) 3. Confirm you want to proceed 4. Results of the New Base-Line in illustrate all mapping specifications have been incremented by the max version number (3 to 4). Analytix Data Services LLC Page 50 of 62

51 Deleting a Mapping Specification & Rolling Back Changes to a Previous Version The Mapping Manager allows deletes for the purposes of housecleaning (incomplete and irrelevant mapping specifications) and to rollback to a previously versioned state. Users have the ability to delete all versions at the mapping specification level (discard all versions) or to delete only the current version. Deleting the current version removes current version and makes the previously versioned mapping specification the current version. 1. Click on the Mapping Manager icon and Navigate to the mapping Specification you would like to delete. 2. Select the Mapping Specification, Right Click and Choose Delete Map. Analytix Data Services LLC Page 51 of 62

52 3. Next the following dialog box is displayed. Choose Delete Current Version to deleting the current version and revert to the previously versioned mapping specification (E.g. rollback to the previous version) Choose Delete ALL Version to remove the current mapping specifications and all contents of the Archive folder (previously versioned mapping specifications).! Note: Choosing ALL VERSIONS should only be used as a house-cleaning function as the mapping specification and all previous versions will be permanently deleted. Analytix Data Services LLC Page 52 of 62

53 4. Select Current Version, Click OK and confirm the deletion. The screenshot below illustrates a before and after screenshot showing the rollback to the previous version. Analytix Data Services LLC Page 53 of 62

54 Mapping Comparison (Compare a Mapping Specification to Another Mapping Specification) The advanced mapping comparison capabilities allows users to quickly and efficiently compare any two mapping specifications and visually view the changes between the two in a horizontal, vertical and row by row basis. The robust comparisons capabilities are not just limited to a mapping specification and its previous versions (archive contents). The advanced comparison capabilities allow users to compare a mapping specification to any mapping specification across all projects (e.g. parallel project initiatives etc). To compare mapping specifications, 1. Click on the Mapping Manager icon and Navigate to the mapping Specification you would like to compare. 2. Select one mapping then press CTRL and Select a Second Mapping Specification to use in the comparison. Note: Two Mapping Specifications Must be selected (highlighted in Blue) as illustrated below: 3. Right Click and Choose Compare To The following interactive comparison chart appears. Note: the following: The Red rows indicate the row has changed between versions or varies in comparison to other mapping specification The blank column in the middle of the horizontal comparison is the separator of the mapping specifications. The vs. - circled in the screen illustrates which mapping appears to the left versus the right. Analytix Data Services LLC Page 54 of 62

55 4. Select any red row by clicking on the red row to view to view the changes compared vertically side by side for that attribute. The following screen appears showing the red data elements that have changed. See below. 5. Noting the red columns appear in version 1 indicate the contact name was present in version 1.0 and was deleted in version 3 (hence it is null). Tip: If columns appear as null or deleted you may want to check the change descriptions on the each of the versions. This can be done by looking at the Mapping Spec Overview or by clicking the Mapping Details Tab of the Interactive Comparison Report. 6. To view the comparison of changes on the mapping specification details select the Mappings Tab. Note: The Mappings tab visually shows a comparison of the Mapping Specification Details which are represented as a cutout in Box #2 below. All the specification details represented in Analytix Data Services LLC Page 55 of 62

56 the tabular data items are available for comparison review. Analytix Data Services LLC Page 56 of 62

57 Create and Maintain Data Dictionaries The Mapping Manager supports the ability to create and maintain data dictionaries across all enterprise systems. Once the business definitions for tables and columns are updated in our Mapping Manager Module (updated directly from the metadata browser) they are then sharable across the organization to all associates having a user id. The metadata browser allows all business properties (business names for tables and columns) and technical properties (data type, precision, scale, null-rules, default values etc) to up maintained using the metadata browser. In this section, you will learn: 1. How to add and maintain data dictionary metadata values for tables, columns and technical metadata (data type, precision scale etc). Enter and Maintain Data Dictionaries 1. From the Mapping Manager s Metadata Browser navigate to the System and Table or Column you would like to create or change the data dictionary values (depicted in step 1 below). 2. In the Bus Prop (Business Properties)Right tab double click and enter the business value (depicted in step 2) in the graphic and click the save icon in (depicted in step 3) 3. To update Technical Properties repeat steps 1&2 above. Analytix Data Services LLC Page 57 of 62

58 Analytix Data Services LLC Page 58 of 62

59 Upload ETL Jobs as a Data Mapping Specification (Reverse engineer an ETL Job as a data mapping specification) The Mapping Manager supports the ability to dynamically upload/import an Informatica PowerCenter or IBM Information Server (DataStage) ETL Job from an xml file and Import the xml file using the mapping manager module to reverse engineer an ETL Job as a data mapping in our repository. Then maintain and version the data mapping using our tool as an accurate requirement specification for future releases or use the Mapping Comparison capability to audit the production ETL job against the requirement data mapping. (Audit/Test the Data Mapping: ETL Job against the requirement) In this section, you will learn: 1. How to Import an ETL Job from Informatica PowerCenter or IBM Information Server (DataStage) and reverse engineer as a data mapping specification UpLoad ETL XML File as a Data Mapping Specification (Reverse Engineer an ETL Job) 1. Export an ETL job from the ETL Tool (Informatica PowerCenter / IBM Information Server) as an XML File(May require support of an ETL Developer). Make note of the location of the XML file on the file system directory. 2. From the Mapping Manager s Project Browser navigate to the Project or Mapping you would like to import to and right-click and choose upload XML. 3. Next Choose which type of file to Import (Informatica or DataStage) and browse to the file system directory where the xml file is stored and click import. 4. The XML file will be converted to a data mapping specification and appear in the data grid as an editable data mapping specification. Tip: It is recommended to version a data mapping specification after uploading from XML. This allows the version to be preserved as a read-only copy after the initial import and before any changes have been applied. Analytix Data Services LLC Page 59 of 62

60 Appendix Role-Based Security Definitions This section defines all Role-Based Security settings available in the Mapping Manager and defines the specific security grants associated to each.. It is important to note that prior to creating a user which role-based security setting or settings should be assigned to the user. The application support multiple role assignments to a single user. The role-based security has been defined with the flexibility to address both larger and smaller organization needs. Larger organizations may want to lock down security within the application and distribute responsibility across multiple divisions of the organization (e.g. Business Analyst, ETL Developers, Testers, Data Governance etc) while smaller organizations may want to assign only read-vs write access and manage security within their existing teams. Please note our recommendations below regarding assigning role-based security. Recommendation 1: Locking Down Security We recommend granting roles based on functional use internal to the organization. For example, we recommend assignment of the following roles to lock down certain access in large organizations where data governance may want to restrict changes to Enterprise Transformations (Role: Transformation Admin), Updating and Refreshing of Metadata (Role: System Admin) Etc. Recommendation 2:. Liberal Use Security Recommended for smaller development teams where various internal teams are not involved in the Mapping Specification, ETL Development, Governance and Testing processes. For these organizations you may want to assign one of the global security (see below) roles to all users. For example a power user role can be assigned to all users granting read/write access to all modules, projects and mapping specifications and grant a public (Read-Only) role to only those who require read-only access. Note: The only difference between a Power User Role and an Administrator Role is the Administrator can add users and assign roles to users in Mapping Manager System. Analytix Data Services LLC Page 60 of 62