Note: Before using this information and the product it supports, read the information in Notices on page 88.

Transcription

1 o IBM Tivoli Storage Resource Manager, Version 1.3 Warehouse Enablement Pack, Version Implementation Guide for Tivoli Data Warehouse, Version 1.2 SC

2 Note: Before using this information and the product it supports, read the information in Notices on page 88. Second Edition (April 24) This edition applies to IBM Tivoli Storage Resource Manager, Version 1.3 (program number 5698-SRM) and to all subsequent releases and modifications until otherwise indicated in new editions. Order publications through your sales representative or the branch office serving your locality. Your feedback is important in helping to provide the most accurate and high-quality information. If you have comments about this book or any other Tivoli Storage Resource Manager documentation, please see Contacting customer support on page 4. Copyright International Business Machines Corporation 23, 24. All rights reserved. US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide ii

3 Contents 1 About this guide Who should read this guide Publications IBM Tivoli Storage Resource Manager library Tivoli Data Warehouse library Related publications Accessing publications online Ordering publications Accessibility Contacting software support Participating in newsgroups Typeface conventions Overview Overview of Tivoli Data Warehouse Overview of IBM Tivoli Storage Resource Manager Warehouse Pack Tivoli Storage Resource Manager Identifying unique computers across storage products Tracking unique servers and agents Identifying operating system (OS) type Time conversion Internationalization support for Tivoli Storage Resource Manager Reports How Has File System Utilization Changed Over Time? Sample report for How Has File System Utilization Changed Over Time? SQL queries for How Has File System Utilization Changed Over Time? Report navigation Which File Systems Have the Most Free Space? Sample report for Which File Systems Have the Most Free Space? SQL queries for Which File Systems Have the Most Free Space? Report navigation How Has Host System File System Usage Changed? Sample report for How Has Host System File System Usage Changed? SQL queries for How Has Host System File System Usage Changed? Report navigation Installing and configuring the warehouse pack Prerequisites Supported hardware and software Product notes and limitations Restarting process steps Pruning Data in the Tivoli Enterprise Data Warehouse Support for Multiple Tivoli Storage Resource Manager servers Tivoli Storage Resource Manager Enterprise Repository Database-sizing considerations Data sources and targets Pre-installation procedures Assigning ODBC Data Source Information Central Data Warehouse database backups...25 IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide iii

4 4.7 Installation of the warehouse pack Post-installation procedures Configuring file btm_my.translate Steps to reconfigure the Warehouse Source Steps for post installation Scheduling ETL processes Viewing Reports Uninstallation of the warehouse pack Multiple data centers Multiple customer environments Migrating from a previous release of the warehouse pack Maintenance and problem determination Backing up and restoring Pruning data Central data warehouse Data mart Extraction control (table Extract Control) Maintenance of the Tivoli Storage Resource Manager archive tables Problem determination The steps in the BTM_C1_ETL1_Process already have the mode set to production The BTM_c1_s2_Extract step failed The BTM_c1_s1_Pre_extract is not failing but the BTM.ERROR_STATUS shows an error The BTM_c1_s1_Pre_Extract failed The names of the hosts are being truncated in the graphs Why are my reports empty? Uninstall of warehouse pack never completes Debug Scenarios Crystal Reports still listed in web browser after warehouse pack uninstall BTM_m5_s4_Prune_Mart fails after reinstall of warehouse pack ETL processes BTM_c5_Initialize_Process BTM_c1_ETL1_Process BTM_c15_Expire_Process BTM_m5_ETL2_Process Central data warehouse information Component configuration Component type (table CompTyp) Component extension (table Comp_ext) Component (table Comp) Component relationship type (table RelnTyp) Component relationship rule (table RelnRul) Component relationship (table CompReln) Component type keyword (table CompTyp_Keyword) Attribute type (table AttrTyp) Attribute rule (table AttrRul) Attribute domain (table AttrDom) Component attribute (table CompAttr) Component type relationship (table CTypReln) Component attribute type relationship (table ATypReln) Component measurement Measurement group type (table MGrpTyp)...71 IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide iv

5 7.2.2 Measurement group (table MGrp) Measurement group member (table MGrpMbr) Measurement unit category (table MUnitCat) Measurement unit (table MUnit) Measurement alias names (table MTypReln) Time summary (table TmSum) Measurement source (table MSrc) Measurement source history (table MSrcHistory) Measurement type (table MsmtTyp) Component measurement rule (table MsmtRul) Measurement (table Msmt) Threshold measurement objective (table Mobj) Threshold measurement objective range (table MobjRng) Threshold severity level (table SevLvl) Measurement history (table Msrc_History) Component events Helper tables Exception and Expiration tables Incremental extraction Data mart schema information Star schemas Hourly File System Star Schema Daily File System Star Schema Weekly File System Star Schema Monthly File System Star Schema Quarterly File System Star Schema Yearly File System Star Schema Metric dimension tables BTM.D_FSYS_METRIC Dimension tables Dimension table BTM.D_SRM_SRVR Dimension table BTM.D_SRM_CLIENT Dimension table BTM.D_FILE_SYSTEM...88 IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide v

6 1 About this guide This document describes the warehouse pack, Version for IBM Tivoli Storage Resource Manager, Version This warehouse pack is designed to work with Tivoli Data Warehouse, Version 1.2. The Version Tivoli Storage Resource Manager Warehouse Pack is a replacement for the previous version of the warehouse pack. With this warehouse pack (hereafter referred to as warehouse pack), you can extract historical statistics about storage resource file system activity. Information on file system capacity, free space and used space is available for analysis through the central data warehouse. The first release, Version 1.1 of the warehouse pack, loads storage resource information into the central data warehouse; these statistics can be used by Tivoli products like IBM Tivoli Service Level Advisor. Warehouse pack Version for Tivoli Storage Resource Manager provides data mart ETL processing that populates a data mart and provides three reports. Tivoli Data Warehouse Version 1.2 introduces a new reporting interface utilizing Crystal Reports 9 that permits viewing of the warehouse reports from your web browser. This warehouse pack provides the following reports based on the enterprise monitored by Tivoli Storage Resource Manager: Which File Systems Have the Most Free Space? Quickly identify the file systems with available space on each host. View the individual file system usage for each host. Parameters are provided to limit your search to a particular server or host. How Has File System Utilization Changed Over Time? View the trend of your file system utilization. See information granularity by specifying a daily, weekly, monthly, quarterly or yearly time period. Choose to view file systems with a specified percent availability. How Has Host System File System Usage Changed? Track the percent of increase for used space by host and file system. See information in the granularity you specify daily, weekly, monthly, quarterly or yearly. Parameters are provided to specify a particular server, host or file systems with a specific percent of change. 1.1 Who should read this guide This guide is for administrators, installers, and report designers who plan for, install, configure, and maintain the warehouse packs for Tivoli Data Warehouse. Administrators, report designers, and installers should have the following knowledge or experience: Basic system administration and file management of the operating systems on which the components of Tivoli Data Warehouse are installed An understanding of the basic concepts of relational database management Experience administering IBM DB2 Universal Database 1.2 Publications This section lists publications in the Tivoli Data Warehouse library and other related documents. It also describes how to access Tivoli publications online and how to order Tivoli publications. The following sets of documentation are available to help you understand, install, and manage this warehouse pack: IBM Tivoli Storage Resource Manager (optional) Tivoli Data Warehouse IBM DB2, DB2 Data Warehouse Center, and DB2 Warehouse Manager IBM Redbooks IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 1

7 1.2.1 IBM Tivoli Storage Resource Manager library The following Tivoli Storage Resource Manager documents are available online at the Tivoli Software Information Center Web site: IBM Tivoli Storage Resource Manager Quick Reference, GC Provides basic information on how to use the main features and functions of Tivoli Storage Resource Manager. IBM Tivoli Storage Resource Manager Installation Guide, GC Describes how to install Tivoli Storage Resource Manager and its components on UNIX and Windows. IBM Tivoli Storage Resource Manager Configuration and Getting Started Guide, SC Describes how to configure Tivoli Storage Resource Manager for your environment. IBM Tivoli Storage Resource Manager User s Guide, SC Provides detailed information on how to use Tivoli Storage Resource Manager. IBM Tivoli Storage Resource Manager Messages, SC Provides Tivoli Storage Resource Manager messages to help in diagnosing problems Tivoli Data Warehouse library The following documents are available in the Tivoli Data Warehouse library. The library is available on the Tivoli Data Warehouse Documentation CD as well as online, as described in Accessing publications online on page 3. Tivoli Data Warehouse Release Notes, SC Provides late-breaking information about Tivoli Data Warehouse and lists hardware requirements and software prerequisites. Installing and Configuring Tivoli Data Warehouse, GC Describes how Tivoli Data Warehouse fits into your enterprise, explains how to plan for its deployment, and gives installation and configuration instructions. It contains maintenance procedures and troubleshooting information. Enabling an Application for Tivoli Data Warehouse, GC Provides information about connecting an application to Tivoli Data Warehouse. This book is for application programmers who use Tivoli Data Warehouse to store and report on their application data, data warehousing experts who import Tivoli Data Warehouse data into business intelligence applications, and customers who put their local data in Tivoli Data Warehouse. Tivoli Data Warehouse Messages, SC Lists the messages generated by Tivoli Data Warehouse, and describes the corrective actions you should take Related publications The following sections describe additional publications to help you understand and use Tivoli Data Warehouse IBM Redbooks IBM Redbooks are developed and published by the IBM International Technical Support Organization, the ITSO. They explore integration, implementation, and operation of realistic customer scenarios. The following Redbooks contain information about Tivoli Data Warehouse: Introduction to Tivoli Enterprise Data Warehouse, SG IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 2

8 Provides a broad understanding of Tivoli Data Warehouse. Some of the topics that are covered are concepts, architecture, writing your own ETLs, and best practices in creating data marts. Planning a Tivoli Enterprise Data Warehouse Project, SG Describes the necessary planning you must complete before you can deploy Tivoli Data Warehouse. The guide shows how to apply these planning steps in a real-life deployment of a warehouse pack using IBM Tivoli Monitoring. It also contains frequently used Tivoli and DB2 commands,1 and lists troubleshooting tips for Tivoli Data Warehouse IBM DB2, DB2 Data Warehouse Center, and DB2 Warehouse Manager library The DB2 library contains important information about the database and data warehousing technology provided by IBM DB2, DB2 Data Warehouse Center, and DB2 Warehouse Manager. Refer to the DB2 library for help in installing, configuring, administering, and troubleshooting DB2, which is available on the IBM Web site: After you install DB2, its library is also available on your system. The following DB2 documents are particularly relevant for people working with Tivoli Data Warehouse: IBM DB2 Universal Database for Windows Quick Beginnings, GC Guides you through the planning, installation, migration (if necessary), and setup of a partitioned database system using the IBM DB2 product on Microsoft Windows. IBM DB2 Universal Database for UNIX Quick Beginnings, GC9-297 Guides you through the planning, installation, migration (if necessary), and setup of a partitioned database system using the IBM DB2 product on UNIX. IBM DB2 Universal Database Administration Guide: Implementation, SC Covers the details of implementing your database design. Topics include creating and altering a database, database security, database recovery, and administration using the Control Center, which is a DB2 graphical user interface. IBM DB2 Universal Database Data Warehouse Center Administration Guide, SC Provides information on how to build and maintain a data warehouse using the DB2 Data Warehouse Center. IBM DB2 Warehouse Manager Installation Guide, GC Provides information on how to install the following Warehouse Manager components: Information Catalog Manager, warehouse agents, and warehouse transformers. IBM DB2 Universal Database and DB2 Connect Installation and Configuration Supplement, GC Provides advanced installation considerations, and guides you through the planning, installation, migration (if necessary), and set up of a platform-specific DB2 client. This supplement also contains information on binding, setting up communications on the server, the DB2 GUI tools, DRDA AS, distributed installation, the configuration of distributed requests, and accessing heterogeneous data sources. IBM DB2 Universal Database Message Reference Volume 1, GC and IBM DB2 Universal Database Message Reference Volume 2, GC Lists the messages and codes issued by DB2, the Information Catalog Manager, and the DB2 Data Warehouse Center, and describes the actions you should take Accessing publications online The publications CD or product CD contains the publications that are in the product library. The format of the publications is PDF, HTML, or both. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 3

9 IBM posts publications for this and all other Tivoli products, as they become available and whenever they are updated, to the Tivoli Software Information Center Web site. The Tivoli Software Information Center is located at the following Web address: Note: If you print PDF documents on other than letter-sized paper, select the Fit to page check box in the Adobe Acrobat Print dialog. This option is available when you click File Print. Fit to page ensures that the full dimensions of a letter-sized page print on the paper that you are using Ordering publications You can order many Tivoli publications online at the following Web site: You can also order by telephone by calling one of these numbers: In the United States: In Canada: In other countries, for a list of telephone numbers, see the following Web site: Accessibility Accessibility features help users with a physical disability, such as restricted mobility or limited vision, to use software products successfully. For the warehouse pack, you use the interfaces of IBM DB2 and the reporting tool. See those documentation sets for accessibility information. 1.4 Contacting software support If you have a problem with a Tivoli product, refer to the following IBM Software Support Web site: If you want to contact customer support, see the IBM Software Support Guide at the following Web site: The guide provides information about how to contact IBM Software Support, depending on the severity of your problem, and the following information: Registration and eligibility Telephone numbers and addresses, depending on the country in which you are located Information you must have before contacting IBM Software Support 1.5 Participating in newsgroups User groups provide software professionals with a forum for communicating ideas, technical expertise, and experiences related to the product. They are located on the Internet, and are available using standard news reader programs. These groups are primarily intended for user-to-user communication, and are not a replacement for formal support. You can use Web browsers like Netscape Navigator or Microsoft Explorer to view these newsgroups: Tivoli Data Warehouse news://news.software.ibm.com/ibm.software.tivoli.enterprise-data-warehouse 1.6 Typeface conventions This guide uses the following typeface conventions: IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 4

10 Bold Italic Lowercase commands and mixed case commands that are otherwise difficult to distinguish from surrounding text Interface controls (check boxes, push buttons, radio buttons, spin buttons, fields, folders, icons, list boxes, items inside list boxes, multicolumn lists, containers, menu choices, menu names, tabs, property sheets), labels (such as Tip and Operating system considerations) Column headings in a table Keywords and parameters in text Citations (titles of books, diskettes, and CDs) Words defined in text Emphasis of words (words as words) Letters as letters New terms in text (except in a definition list) Variables and values you must provide Monospace Examples and code examples File names, programming keywords, and other elements that are difficult to distinguish from surrounding text Message text and prompts addressed to the user Text that the user must type Values for arguments or command options IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 5

11 2 Overview The following sections provide an overview of Tivoli Data Warehouse and the warehouse pack for IBM Tivoli Storage Resource Manager. 2.1 Overview of Tivoli Data Warehouse Tivoli Data Warehouse provides the infrastructure for the following: Extract, transform, and load (ETL) processes through the IBM DB2 Data Warehouse Center tool Schema generation of the central data warehouse Historical reports As shown in Figure 1, Tivoli Data Warehouse consists of a centralized data store where historical data from many management applications can be stored, aggregated, and correlated. Figure 1. Tivoli Data Warehouse basic architecture IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 6

12 The central data warehouse uses a generic schema that is the same for all applications. As new components or new applications are added, more data is added to the database; however, no new tables or columns are added in the schema. A data mart is a subset of a data warehouse that contains data that is tailored and optimized for the specific reporting needs of a department or team. The central data warehouse ETL reads the data from the operational data stores of the application that collects it, verifies the data, makes the data conform to the schema, and places the data into the central data warehouse. The data mart ETL extracts a subset of data from the central data warehouse, transforms it, and loads it into one or more star schemas, which can be included in data marts to answer specific business questions. A program that provides these ETLs is called a warehouse pack. The ETLs are typically scheduled to run periodically, usually during non-peak hours. If an ETL encounters data that it cannot correctly transform, it creates an entry in an exception table. Exception tables are described on page Overview of IBM Tivoli Storage Resource Manager Warehouse Pack The warehouse pack for IBM Tivoli Storage Resource Manager, Version 1.2, introduces a data mart and reports about monitored file system capacity statistics over time. The warehouse pack provides the following reports: How Has File System Utilization Changed Over Time? Which File Systems Have the Most Free Space? How Has Host System Space Utilization Changed? Version 1.1 of the warehouse pack provided the steps to extract file system information from the Tivoli Storage Resource Manager Enterprise Repository database. The central data warehouse ETL transformed that data so it conformed to the central data warehouse format and then loaded it into the central data warehouse of Tivoli Enterprise Data Warehouse. Other products, like Tivoli Service Level Advisor, pull data into data marts they provide to use with service level agreement reports Tivoli Storage Resource Manager The product Tivoli Storage Resource Manager is based upon a three-tier architecture: a user interface (UI) component, a server component and an agent component. Agents collect statistics about storage and send that information to the centralized server. The agent must be installed on all of the machines where the user wants to monitor storage and storage activity. The server receives information from the agent(s) and saves that information in the Enterprise Repository. Collection of data from IBM Tivoli products into one central repository provides the user with the opportunity to see trends in operation, resource usage and cross product interoperability. Tivoli Storage Resource Manager historical data is available for use by Tivoli Service Level Advisor and Tivoli Storage Manager. Consult Tivoli Service Lever Advisor documentation for information about its installation, configuration, and use. Tivoli Enterprise Data Warehouse and IBM DB2 Data Warehouse Center ETL processes are designed to perform data collection at least once a day. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 7

13 Tivoli Storage Resource Manager is an intelligent console for the storage environment that provides a set of policydriven automated tools for managing storage capacity, availability, events, performance, and assets in your enterprise environment, including DAS, NAS, and SAN technologies. Tivoli Storage Resource Manager consists of the following components: Management Server A single server installed to manage all agents. This is the core of the product. Only the server has access to the database repository. Agents Installed on each computer whose storage is being managed. GUI Clients that access gathered data by connecting to the server Identifying unique computers across storage products Storage products has adopted the use of a globally unique identifier (GUID) that is permanently assigned to computer systems running the storage products. Tivoli Storage Resource Manager Version 1.2 implements this feature as a software identifier and GUID is installed on the server and agent platform(s). The GUID allows storage products to coordinate computer system data in the central data warehouse. The GUID is recorded as common data attribute SYSTEM_GUID in the central data warehouse for each Tivoli Storage Resource Manager server and monitored computer where the GUID is available for that system. The attribute type code is changing from STORAGE_GUID used in Tivoli Storage Resource Manager v1.2 Warehouse Pack v 1.1. During installation of the current warehouse pack v 1.2 any existing attributes with the type code STORAGE_GUID will be expired and renewed as SYSTEM_GUID for Tivoli Storage Resource Manager components. The format of the GUID also changes in Tivoli Storage Resource Manager Warehouse Pack version 1.2. Previously the STORAGE_GUID was represented as dotted hex, the SYSTEM_GUID attribute will use the new format of and will be updated when the first data extract is performed using the version 1.2 Tivoli Storage Resource Manager Warehouse Pack. Sample: ABCDEF IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 8

14 During installation of warehouse pack 1.2, the TWG.Attrtyp table will have the STORAGE_GUID attribute expired and the TWG.ATTRRUL table will have the components associated with Tivoli Storage Resource Manager updated to include SYSTEM_GUID instead of STORAGE_GUID. Affected components are IP_HOST, IP_INTERFACE, BTM_HOST, BTM_SERVER, and BTM_CLIENT. Until all agents are updated to version 1.2 or later of Tivoli Storage Resource Manager, a SYSTEM_GUID attribute will not be available in the data warehouse for all the monitored systems. Consult the Tivoli Storage Resource Manager Version 1.2 documentation for a list of platforms that supports GUID Tracking unique servers and agents Tracking data across multiple applications within Tivoli Enterprise Data Warehouse can cause problems with several products running on one system or redundant instances of one product running on the same system. Make sure systems can be individually identified when data is moved into the central data warehouse Server uniqueness While multiple instances of the Tivoli Storage Resource Manager may be set-up in an enterprise environment, each Tivoli Storage Resource Manager server must have a unique name if the central data warehouse is to manage data from each individual server installation. With Tivoli Data Warehouse v 1.2 that allows rollup of data from multiple servers into the central data warehouse, unique names for servers must be maintained Agent uniqueness Uniqueness for Tivoli Storage Resource Manager agents is controlled via the Tivoli Storage Resource Manager server. Within the central data warehouse data about each agent is guaranteed uniqueness by the relationship it has to the unique storage resource manger server and the GUID or hardware ID assigned to that system Identifying operating system (OS) type Tivoli Storage Resource Manager Version 1.2 manages additional types of network storage. Storage Resources OS_TYPE ID Description LINUX 1 platform AIX 2 platform HPUX 3 platform WIN_NT 4 platform WIN_2 5 platform SOLARIS 6 platform HPUX1 7 platform NETWARE 8 platform NAS 9 generic NAS NET_APP 1 Network appliance RESERVED 11 Not supported IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 9

15 ESS 12 IBM Shark Compaq HS 13 Compaq HS Hitachi Open 14 Hitachi Open IBM FASTT 15 IBM FASTT LSI MetaStor 16 LSI MetaStor SUN T3 17 SUN T3 HP Storage Works 18 HP Storage Works Storage Tek 19 Storage Tek MSCS Virtual Server 2 MSCS Virtual Server IBM SAN Volume Controller Cluster 21 IBM SAN Volume Controller Cluster Cluster 22 Cluster IBM SAN File System 23 IBM SAN File System Time conversion The times recorded in the Tivoli Storage Resource Manager Enterprise Repository are based on the server platform time zone. During the warehouse pack installation, a table is created that maps time zone identifiers to their GMT offset; daylight saving time is not supported. The time zone table is referenced during central data warehouse ETL processing as part of the transform to convert time timestamps from the Tivoli Storage Resource Manager Enterprise Repository to Coordinated Universal Time (UTC).. Times recorded in the attributes and measurements represent the simple time zone conversion for date and time. Any time zone identifier not listed in the table will be transferred without conversion under the assumption it is already GMT Internationalization support for Tivoli Storage Resource Manager Consult the documentation from Tivoli Storage Resource Manager Version 1.2 for a list of the languages supported for the product. The Tivoli Data Warehouse is a UNICODE repository and is compatible with the application.. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 1

16 3 Reports The warehouse pack provides the following reports: How Has File System Utilization Changed Over Time? Which File Systems Have the Most Free Space? How Has Host System File System Usage Changed? Based on the ITSRM_BTM Star Schema 3.1 How Has File System Utilization Changed Over Time? This report shows each IBM Tivoli Storage Resource Manager server and each host systems file system utilization. A summary of free space versus used space is presented over the time period you set for the Tivoli Resource Manager server followed by the break down for each host file system. Use this report to identify host systems that might be using excessive amounts of storage. Details are provided for all host systems (not just the top 1), so the report may also be used to identify how much storage any arbitrary host system is using. Parameters allow you to customize the report for a specific Tivoli Storage Resource Manager server if your enterprise has multiple server environments. Specify the time period granularity; daily granularity allows you to see data for last 7 days, month to date or year to date. A summarization over time for week, month, quarter and year can also be displayed. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 11

17 3.1.1 Sample report for How Has File System Utilization Changed Over Time? IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 12

18 3.1.2 SQL queries for How Has File System Utilization Changed Over Time? The following query will select the basic set of input data to this report. Presentation of the report relies on several features in Crystal Reports 9 such as formulas, record selection based on parameters and numerical summarization. select distinct s.srv_name, c.clt_name, f.fsys_name, m.met_name, v.meas_date, v.min_value, v.max_value, v.avg_value from btm.f_fsys v, btm.d_srm_srvr s, btm.d_srm_client c, btm.d_srm_fsys f, btm.d_fsys_metric m where v.granularity = 'DAY' and v.d_id_fsys = f.d_id_fsys and v.d_id_clt = c.d_id_clt and v.d_id_srvr = s.d_id_srv and v.metric_id = m.metric_id order by f.fsys_name; Report navigation Parameters Enter The Name Of The Tivoli Storage Resource Server (default is all servers) Enter The Name Of The Host System: (default is all hosts for the server) Select The Time Period To Report: ( default by day, [by week, by month, by quarter, by year]) Limit Number of days for Daily Table: (default is All [Last 7 days, Month to Date, Year to Date]) Enter a Percent for Free Space: (default is all file systems with zero or greater percent free space) When entering the name of the server or host system the value must match the name as it is appears in the central data warehouse for the BTM_SERVER and BTM_CLIENT components. Retention of data in the data mart tables is controlled by the prune data parameters you have set for your environment. If you only preserve data at the day granularity for 3 days, the default number of days to display is 3 days. Data is rolled up into the week, month, quarter and yearly totals after each data mart ETL process completes, the number of entries for each of these granularities is also controlled by the prune data parameters. (See the document for Enabling a Tivoli Data Warehouse for instructions on resetting these parameters.) Grouping The report is grouped by server, host system, file system and measurement day. At the server level a summarization is presented for free space and used space that includes all hosts. 3.2 Which File Systems Have the Most Free Space? This summary report presents an overview of how file system space is being occupied for each Tivoli Storage Resource Manager server the total free space for each host system is aggregated. A drill down on the individual server will present the free space and used space for each individual file system. The report will have parameters allowing the user to select information about a specific server or an individual host system. The time period for this report is as of the latest reported free space and used space measurements by day. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 13

19 3.2.1 Sample report for Which File Systems Have the Most Free Space? SQL queries for Which File Systems Have the Most Free Space? The following query will select the basic set of input data to this report. Presentation of the report relies on several features in Crystal Reports 9 such as formulas, record selection based on parameters and numerical summarization. select distinct s.srv_name, c.clt_name, f.fsys_name, m.met_name, v.meas_date, v.min_value, v.max_value, v.avg_value from btm.f_fsys v, btm.d_srm_srvr s, btm.d_srm_client c, btm.d_srm_fsys f, btm.d_fsys_metric m where v.granularity = 'DAY' and v.d_id_fsys = f.d_id_fsys and v.d_id_clt = c.d_id_clt and v.d_id_srvr = s.d_id_srv and v.metric_id = m.metric_id order by f.fsys_name; IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 14

20 3.2.3 Report navigation Parameters Enter The Name Of The Tivoli Storage Resource Server: (default is all servers) Enter The Name Of The Host System: (default is all hosts for server) Enter The Date To Report: (default is last date reported) When entering the name of the server or host system the value must match the name as it is appears in the central data warehouse for the BTM_SERVER and BTM_CLIENT components. Retention of data in the data mart tables is controlled by the prune data parameters you have set for your environment. If you only preserve data at the day granularity for 3 days, the default number of days to display is 3 days. Data is rolled up into the week, month, quarter and yearly totals after each data mart ETL process completes, the number of entries for each of these granularities is also controlled by the prune data parameters. (See the document for Enabling a Tivoli Data Warehouse for instructions on resetting these parameters.) Grouping The report should be grouped by server, host system, file system and measurement date and fact id (type of measurement free space and used space. 3.3 How Has Host System File System Usage Changed? This report highlights host systems that have increased their file system usage over time. Changes between time period will be analyzed to rate the percentage of change in file system capacity and used space (increase or decrease). The report will rely on the rollup data for week but the user can choose to look at information in a venue of weekly, monthly, quarterly or yearly. By default the report will display all Tivoli Storage Resource Manager servers and their host systems. Parameters will allow the user to target the report to specific servers or host systems, and select a daily, weekly, monthly, quarterly or yearly time period. Another parameter will let the user list the percentage of change in a host systems file system usage to use in the display. (i.e. only list changes greater than 2 percent or 5 percent). The report is organized by server and host. Information about host files systems is summarized by date in a graph if there are less than 1 files ystems, this is followed by a text presentation for the file system by date. When more than 1 file systems are available on one host only the text information is displayed. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 15

21 3.3.1 Sample report for How Has Host System File System Usage Changed? SQL queries for How Has Host System File System Usage Changed? The following query will select the basic set of input data to this report. Presentation of the report relies on several features in Crystal Reports 9 such as formulas, record selection based on parameters and numerical summarization. select distinct s.srv_name, c.clt_name, f.fsys_name, m.met_name, v.meas_date, v.min_value, v.max_value, v.avg_value from btm.f_fsys v, btm.d_srm_srvr s, btm.d_srm_client c, btm.d_srm_fsys f, btm.d_fsys_metric m where v.granularity = 'DAY' and v.d_id_fsys = f.d_id_fsys IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 16

22 order by f.fsys_name; and v.d_id_clt = c.d_id_clt and v.d_id_srvr = s.d_id_srv and v.metric_id = m.metric_id Report navigation Parameters Enter The Name Of The Tivoli Storage Resource Server: (default is all servers) Enter The Name Of The Host System: (default is all for server or servers) Select The Time Period To Report: ( default by day, [by week, by month, by quarter, by year]) Set number of days to view: (default is All [Last 7 days, Month to Date, Year to Date]) Enter a Percent for Used Space: (default is all file systems with zero or greater percent used space) When entering the name of the server or host system the value must match the name as it is appears in the central data warehouse for the BTM_SERVER and BTM_CLIENT components. Retention of data in the data mart tables is controlled by the prune data parameters you have set for your environment. If you only preserve data at the day granularity for 3 days, the default number of days to display is 3 days. Data is rolled up into the week, month, quarter and yearly totals after each data mart ETL process completes, the number of entries for each of these granularities is also controlled by the prune data parameters. (See the document for Enabling a Tivoli Data Warehouse for instructions on resetting these parameters.) Grouping The report should be grouped by server, host system, file system and measurement date. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 17

23 4 Installing and configuring the warehouse pack Use the Tivoli Enterprise Data Warehouse installation wizard (provided with the product) to install the warehouse pack. Review Installing and Configuring Tivoli Enterprise Data Warehouse for specific installation steps. Installation of this warehouse pack requires the following actions: Make sure all the prerequisite products are installed (see section 4.1). Read sections 2.2 through 4.6 to learn about important information that may impact your Tivoli Enterprise Data Warehouse environment. Make sure you have the database connection and login information for the Tivoli Storage Resource Manager Enterprise Repository. Do not use the login created by the Tivoli Storage Resource Manager installation as your data warehouse login. It is recommended you create a login for data warehouse access. The data warehouse login defined to access the repository for Tivoli Storage Resource Manager must be configured with the following general permissions: Process: Step: Process: Step: BTM_c5_Initialize_Process BTM_c5_s1_CREATE_ARCHIVE Set default database to the Tivoli Storage Resource Manager repository; User permission to create tables, create index, create sequence and create trigger. BTM_c1_ETL1_Process BTM_c1_s1_EXTRACT Set default database to the Tivoli Storage Resource Manager repository; User permission to select from tables and delete records from tables. Consult the documentation for the database platform regarding configuration. Use the Microsoft Control Panel to verify or configure the ODBC data source information about Tivoli Storage Resource Manager Enterprise Repository and Tivoli Enterprise Data Warehouse. Check with the database vendor regarding ODBC installation and configuration. Use the installation wizard for Tivoli Enterprise Data Warehouse to install the warehouse pack. Follow these post installation instructions as described further in section 4.6: 1. Verify the control database is set for the DB2 Data Warehouse Center if this is the initial warehouse installation. 2. Configure the btm_my.translate file for your enterprise installation source database ODBC connections. 3. In btm_my.translate configure the name of the Tivoli Storage Resource Manager Enterprise Repository table owner if you are not using the default value of TIVOLISRM. 4. Run the initialize step to create a set of triggers and archive tables in the Tivoli Storage Resource Manager database if this is a new install. For an upgrade to the 1.2 warehouse pack, you can continue to use the existing archive tables and triggers created in the Tivoli Storage Resource Manager repository. 5. Schedule the central data warehouse ETL process to run. 6. Adjust the prune parameters for the central data warehouse and data mart tables. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 18

24 4.1 Prerequisites Before installing the warehouse pack for IBM Tivoli Storage Resource Manager, you must have installed the following software: IBM Tivoli Storage Resource Manager, Version 1.3 IBM DB2 Universal Database Enterprise Edition, Version 7.2 IBM DB2 Universal Database Enterprise Edition, Version 7.2 Fix Pak 1 A IBM DB2 Universal Database for z/os and OS/39, Version 7.1 Tivoli Data Warehouse, Version 1.2 Tivoli Data Warehouse, Version 1.2 Fix Pack 1 I M P O R T A N T If you are doing a migration from Tivoli Storage Resource Manager Warehouse Enablement Pack version 1.1, uninstall this warehouse pack after upgrading to Tivoli Data Warehouse version 1.2, then install Fix-Pack 1. You must install the Tivoli Data Warehouse Fix Pack 1 before installing version 1.2 of the warehouse pack for Tivoli Storage Resource Manager. 4.2 Supported hardware and software The warehouse pack for Tivoli Storage Resource Manager requires Tivoli Storage Resource Manager, Version 1.3 or higher. Data extraction from Tivoli Storage Resource Manager Enterprise Repository is supported for the database platforms documented in the Tivoli Storage Resource Manager Installation Guide with the exception of Cloudscape. For information about the hardware and software requirements of Tivoli Data Warehouse, see the Tivoli Data Warehouse Release Notes. 4.3 Product notes and limitations Restarting process steps The Tivoli Storage Resource Manager Warehouse Pack is designed to run with one or more data sources when adding data to the central data warehouse. Each step is designed to be restarted if a failure occurs. If a step fails while running the BTM_c1_ETL1_Process or the BTM_m5_ETL2_Process steps restart the step that failed after the problem is resolved. Do not restart the at step BTM_c1_s1_Pre_Extract if the failure occurred before running the central data warehouse ETL load steps. Section 5 of this document identifies problem solving scenarios Pruning Data in the Tivoli Enterprise Data Warehouse The Tivoli Data Warehouse version 1.2 is designed to prune data in the central data warehouse and data marts based on parameters set by the user. For instructions on manual pruning of measurements contact the IBM Tivoli Support Center. The warehouse pack is configured with default values for pruning in the Central Data Warehouse and Data Marts. For more information about the database tables used in the pruning process, see Enabling an Application for Tivoli Data Warehouse. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 19

25 For the central data warehouse the prune control is TWG.PRUNE_MSMT_CONTROL and the default is to keep measurements for one year. For the Storage Resource Manager data marts the prune control is table BTM.PRUNE_MART_CONTROL in TWH_MART and defaults are set as follows: Aggregation Fact Table Interval Prune_Mart_Control Value Hourly F_FSYS_HOUR 3 months 3 Daily F_FSYS_DAY 6 months 6 Weekly F_FSYS_WEEK 1 year 1 Monthly F_FSYS_MONTH 1 year 1 Quarterly F_FSYS_QUARTER 1 Year 1 Year F_FSYS_YEAR 1 Year Support for Multiple Tivoli Storage Resource Manager servers Tivoli Storage Resource Manager v 1.3 Warehouse pack v 1.2 is designed to work with Tivoli Data Warehouse v1.2 and allow scheduling of the source data extract from multiple Tivoli Storage Resource Manager repositories in the enterprise. The Tivoli Data Warehouse v 1.2 Installation and Configuration document presents the procedure and the steps to follow in order to configure your environment. To collect data from multiple servers you will need to define a unique ODBC connection to each separate Tivoli Storage Resource Manager Enterprise Repository. Identify each source repository in the warehouse pack installation procedure and after completing installation, update the btm_my.translate file to identify each data source ODBC name and schema owner for the individual Tivoli Storage Resource Manager repository. Contact the IBM Tivoli Support Center for more information on setting this up in your environment. Some limitations may apply for your enterprise configuration Tivoli Storage Resource Manager Enterprise Repository The Tivoli Storage Resource Manager Enterprise repository is the data source for the warehouse pack ETL processing. One or more repositories for Tivoli Storage Resource Manager servers can deposit data into the warehouse. The Tivoli Data Warehouse v 1.2 provides the means to automatically schedule the extraction processing from multiple data sources for one application. To maintain accurate warehouse data related to Tivoli Storage Resource Manager, the warehouse processing requires during warehouse pack installation that three archive tables and four triggers be created in the source database repository. When a record is deleted in a storage resource manager repository table, a copy of that record will be retained in the archive table. In the data source extraction process, the ETL collects records from the archive tables about deleted computer hosts and file systems so the central data warehouse information can be synchronized. The archive tables and triggers require very little additional space for the Tivoli Storage Resource Manger Enterprise Repository. For a system with 1 agents and an average of 5 file systems each, you should not need more than 3 MB of additional space. Take into consideration your daily deletion of computers and file systems; 3MB should accommodate deletion of all 1 computers and the 5 file systems at one time. 4.4 Database-sizing considerations Ensure that you have sufficient space in the central data warehouse database for the historical data collected by the warehouse pack. To estimate how much space is required for the warehouse pack, complete the following worksheets. Three types of central data warehouse information is addressed in estimating how much space the Tivoli Storage Resource Manager Warehouse pack will require in processing. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 2

26 Staging tables are used to store the extracted data during central data warehouse ETL processing. These tables are dropped and recreated with each run of the BTM_c1_ETL1_Process steps, allowing the user to review the data that was processed in case of errors. Temporary or session tables are transitory tables created in the processing of central data warehouse ETL steps. However, the table is deleted after it is used. Static Tivoli Storage Resource Manger information is recorded in the components, attributes and relationships of the data warehouse. This data is recorded once but is not updated unless a delete component is processed or the value of a component attribute changes. The measurements are the data that is collected and stored when each extract process from Tivoli Storage Resource Manager is run The following example assumes a Tivoli Storage Resource Manager server with 1 agents averaging 5 file systems per agent is being used. Proposed Daily Extract: Average Bytes Per Record Number of Records Total Bytes Megabytes Agent data 884 x 1 = 884, File System data: 44 x 5 = 2,2, Server data: 263 x 1 = 263 2,94,263 3MB Staging and Temporary Tables in the CDW: Staging Tables 7 x 3 MB = 21 MB TempSpace 29 x 3 MB = 87 MB Static Data: Component Bytes Per Component Number of Records Total Bytes Megabytes BTM_SERVER 2276 x 1 = 2,276 BTM_CLIENT 8374 x 1 = 4,678, FILE_SYSTEM 2289 x 5 = 11,445, 16,125, MB Measurements: Component Bytes for Mearurements Number of Records Total Bytes Megabytes FILE_SYSTEM 21 x 5 = 1,5, 1MB Estimate of Additional Central Data Warehouse Megabytes for 9 Days Staging Tables 21. Static Data 15.4 (Measurements x Number of Days) 1 x MB Estimate of Additional Central Data Warehouse Megabytes for 18 Days Staging Tables 21. Static Data 15.4 IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 21

27 (Measurements x Number of Days) 1 x MB Estimate of Additional Central Data Warehouse Megabytes for 1 Year Staging Tables 21. Static Data 15.4 (Measurements x Number of Days) 1 x MB 4.5 Data sources and targets The source database for this warehouse pack is the IBM Tivoli Storage Resource Manager server repository this is named by default TivoliSRM; contact your local database administrator about the name used in your environment for the repository. This warehouse pack can be configured to extract data from any of the supported database vendor platforms used by the Tivoli Storage Resource Manager server. Note: No extract support is provided for the cloudscape database used by Tivoli Storage Resource Manager demos. The Warehouse Sources folder of the DB2 Data Warehouse Center interface configures both the external ODBC source for the Tivoli Storage Resource Manager repository and the internal connections for the TWH_CDW and TWH_MART when it is referenced by the ETL as a source for the DB2 Data Warehouse Center steps. See section 4.6 for instructions on configuring the external ODBC source. The warehouse pack central data warehouse ETL target is the central data warehouse named TWH_CDW. The configuration named BTM_TWH_CDW_Target is in the Warehouse Targets folder of the DB2 Data Warehouse Center interface. The data mart for Tivoli Storage Resource Manager v1.3 Warehouse Pack v1.2 uses the TWH_MART database as the target. The configuration named BTM_TWH_MART_Target is in the Warehouse Targets folder of the DB2 Data Warehouse Center interface. The source for the data mart information is the TWH_CDW configuration named BTM_TWH_CDW_Source in the Warehouse Sources folder. DB2 Data Warehouse Center Warehouse Source definitions: BTM_<ITSRM1>_Source BTM_TWH_CDW_Source BTM_TWH_MART_Source One or more definitions identifying each Tivoli Storage Resource Manager server. These entries will be configured automatically by the warehouse pack installation wizard. <ITSRM1> is the name you gave during installation for each Tivoli Storage Resource Manager data source. Manual definition is also supported if you add additional Tivoli Storage Resource Manager data sources to your environment at a later time. Identifies the TWH_CDW as the source for ETL steps in all the processes defined under BTM_Tivoli_Storage_Resource_Manager_v1.2_Subject_Area. Configured during warehouse pack installation. Identifies the TWH_MART as the source for ETL steps in all the processes defined under BTM_Tivoli_Storage_Resource_Manager_v1.2_Subject_Area. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 22

28 Configured during warehouse pack installation. CDW_TWH_CDW_Source CDW_MART_CDW_Source CDW_MD_CDW_Source Created by the Tivoli Data Warehouse at installation DB2 Data Warehouse Center Warehouse Targets definitions: BTM_TWH_CDW_Target BTM_TWH_MART_Target CDW_TWH_CDW_Target CDW_MART_CDW_Target Identifies the TWH_CDW as the target for ETL steps in all the processes defined under BTM_Tivoli_Storage_Resource_Manager_v1.2_Subject_Area. Configured during warehouse pack installation. Identifies the TWH_MART as the target for ETL steps in all the processes defined under BTM_Tivoli_Storage_Resource_Manager_v1.2_Subject_Area. Configured during warehouse pack installation. Created by the Tivoli Data Warehouse at installation See the Installing and Configuring Tivoli Enterprise Data Warehouse for instructions on configuring multiple TWH_CDW and TWH_MART databases for your environment. 4.6 Pre-installation procedures Before installing the Tivoli Storage Resource Manager Warehouse Pack the Tivoli Storage Resource Manager Version 1.3 must be installed. Follow the Tivoli Storage Resource Manager v 1.3 installation instructions when installing the product and its database repository. Your database administrator can provide connection and login information for the repository created during the Tivoli Storage Resource Manager installation. You will need the name and version number of the database vendor product used for the Tivoli Storage Resource Manager repository, along with the database name or schema name. This is required for the ODBC configuration. You need to have your database administrator identify the database login information you will use to access the data source. During warehouse pack installation you will set up the warehouse source names to use based on the ODBC configurations defined in the pre-installation. Your database administrator needs to provide source database login access with permission to do the following during the installation for the BTM_c5_s1_Create_Archive step: Create table Create index Create trigger Create sequence For daily central data warehouse ETL operation the Tivoli Storage Resource Manager login permissions require: Select IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 23

29 Delete Configure the ODBC connection for your source repositories. You must have the client for your Tivoli Storage Resource Manager database vendor platform installed on the system where you install the Warehouse pack. Set up the ODBC connection using the ODBC Data Source Administrator found for Windows 23 and 2 under Administrative Tools of the Control Panel or for Windows NT on the Control Panel. Consult the Installing and Configuring Tivoli Data Warehouse documentation for the list of ODBC drivers to use for your source database vendor platform. Each source database ODBC must be configured under the System DSN Tab of the ODBC Data Source Administrator. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 24

30 4.6.1 Assigning ODBC Data Source Information DB2 Oracle Sybase MS SQL Server Data Source Name Name you assign Name you assign Name you assign Name you assign Description optional optional optional optional Server Name IP Address or fully qualified hostname Use the name defined via Oracle TNS configuration IP Address or fully qualified hostname IP Address or fully qualified hostname For DB2 you can also use the DB2 Configuration Assistant and it will create the ODBC Data Source Administrator entry. Consult the database vendor documentation about the database client configuration steps Central Data Warehouse database backups Before starting the installation of the Tivoli Storage Resource Manager Version 1.3 Warehouse Pack Version 1.2 you should make a back up the databases used by Tivoli Data Warehouse: TWH_CDW, TWH_MD and TWH_MART. These backups can be used to restore the Tivoli Data Warehouse environment if there is a problem installing the warehouse pack. 4.7 Installation of the warehouse pack Follow the installation steps for the warehouse pack as described in Installing and Configuring Tivoli Data Warehouse. The following notes are provided to clarify information about installing the Tivoli Storage Resource Manager Warehouse Pack. You will need Tivoli Data Warehouse Version 1.2 to install the warehouse pack. The warehouse pack can be installed as part of the Tivoli Data Warehouse Version 1.2 installation or any time later using the Tivoli Data Warehouse installation wizard from Start -> Programs -> Tivoli Data Warehouse -> Install a Warehouse Pack. The installation media for the Tivoli Storage Resource Manager Version 1.3 Warehouse Pack Version. 1.2 is an electronic distribution via the IBM Tivoli Support web site: ftp://ftp.software.ibm.com/software/tivoli/products/itsrm/fixes/warehouse/wep During the installation the properties file for the warehouse pack (twh_install_props.cfg) is required. This file is located in the warehouse pack image under the directory \TDW_WEPS\btm. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 25

31 During the Warehouse Pack installation you will be prompted to create the Warehouse Source to the Tivoli Storage Resource Manager repository. If your environment will be extracting data from more than one storage resource manager repository, configure a connection for each source during the installation process. Note: The extract process automatically executes against each Warehouse Source you define. If a Warehouse Source is not available during the daily central data warehouse ETL processing, the extract will fail when contacting that source. Restart the extract when the connection is restored. For each source you identify, you will create a unique Data Source Name, fill in the User ID and Password for that source reporitory, identify the Database Type based on the database vendor and associate this entry with the ODBC connection using the Database Alias drop down box. The list in the Database Alias will contain the ODBC Data Source you created in the pre-installation step. Connections to TWH_CDW or IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 26

32 TWH_MART are set up by the installation wizard so you are only defining Tivoli Storage Resource Manager repository connections in this step. This name cannot be the same as the ODBC connection name, we recommend using ITSRM<n> where <n> is an integer identifier, one through however many sources your environment will support for multisource extract. This name will be used in the btm_my.translate file configuration described in section Post-installation procedures After you install the warehouse pack, follow instructions in section when you perform the following configuration tasks: Configure the btm_my.translate file. Described in section In the translate file you must have one line for each data source you will use and the name of your Tivoli Storage Resource Manager Enterprise Repository table owner or schema. If this is a first time installation or addition of an new data source, you will need to run the BTM_c5_s1_Create_Archive step in the BTM_c5_Initialize process from the DB2 Data Warehouse Center. Perform the create archive step to install the archive tables and triggers for each Tivoli Storage Resource Manager Enterprise repository you use as a data source. If you did not schedule your central data warehouse ETL and data mart ETL steps to run during the installation procedure, schedule your central data warehouse ETL process to run daily. Information to schedule two dependent ETL processes is documented under Scheduling Warehouse Pack ETL Processes in Installing and Configuring Tivoli Data Warehouse Make sure you have set your pruning parameters for the central data warehouse and data marts. For more information, see section 3.3.1, Restarting process steps The Tivoli Storage Resource Manager Warehouse Pack is designed to run with one or more data sources when adding data to the central data warehouse. Each step is designed to be restarted if a failure occurs. If a step fails while running the BTM_c1_ETL1_Process or the BTM_m5_ETL2_Process steps restart the step that failed after the problem is resolved. Do not restart the at step BTM_c1_s1_Pre_Extract if the failure occurred before running the central data warehouse ETL load steps. Section 5 of this document identifies problem solving scenarios. Pruning Data in the Tivoli Enterprise Data Warehouse Configuring file btm_my.translate The file btm_my.translate is loaded with the warehouse pack. This file identifies the Data Source Name you use with the unique schema name for each Tivoli Storage Resource Manager repository. The Tivoli Storage Resource Manager application has a unique feature permitting the database repository to be named by the user when the product is installed. If the user does not use a proprietary name the default TIVOLISRM is used for all platforms except Sybase where the default is DBO. To manage this dynamic environment the warehouse pack uses the file btm_my.translate delivered in the directory: <x>:\program Files\TWH\apps\btm\v12\translate On warehouse pack install the Tivoli Data Warehouse wizard copies this file to the directory: IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 27

33 <x>:\program Files\TWH\apps\translate this is the copy of the file you will update and that will be accessed when the central data warehouse ETL extract process is run. In the translate file, you must have one line for each data source you use. If you only have one data source and it uses the default schema TIVOLISRM, you can use the default setting included with the warehouse pack in the btm_my.translater file but you need to create your ODBC data source with the name ITSRM1. Default entry in btm_my.translate: ITSRM1_TIVOLISRM TIVOLISRM Name definition: Double underscore is required ITSRM1 This is the same as the Data Source Name you configured during the Warehouse pack installation. _TIVOLISRM This is a required element of the name TIVOLISRM The second TIVOLISRM following the space is the name of the Schema used by the Tivoli Storage Resource Manager repostiroy. If your data source schema does not use TIVOLISRM or you need to create data sources for multiple ITSRM repositories, the following example will help explain how to do it. EXAMPLE: Lets say you have three data sources from which you wish to extract data. Each data source has a different schema name and you have set up three Warehouse Sources during the warehouse pack install: Warehouse Sources BTM_ITSRM1_Source BTM_ITSRM2_Source BTM_ITSRM3_Source Data Source Name ITSRM1 ITSRM2 ITSRM3 Note: Data Source Name is the name you use for your ODBC connection, it is not the same as either the database name or the schema name, however the same name can be assigned to each. During the warehouse pack installation the name created is inserted into the Warehouse Source name. And schema names of: TIVOLISRM (for ITSRM1, vendor DB2, database ITSRM) TIVOLISRM (for ITSRM2, vendor Oracle, tablespace TIVOLISRM) ITSRM_SANBOX11 (for ITSRM3, vendor MSSQL, database ITSRM) The database vendor can be the same for all data sources or you can use different vendors for each separate data source.. To define these 3 connections edit the custom file in the directory: D:\Program Files\TWH\apps\translate\btm_my.translate The following three entries must be included in the btm_my.translate file to define each configuration: IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 28

34 ITSRM1_TIVOLISRM TIVOLISRM ITSRM2_TIVOLISRM TIVOLISRM ITSRM3_TIVOLISRM ITSRM_SANBOX11 Why does this work? In the Warehouse Pack step BTM_C1_S2_Extract.generic a header is declared for the custom translate file: --#USE_TRANSLATE btm_my.translate Within the select statements, a macro is used to access ITSRM tables on the data source. When the extract step is scheduled to run it knows whether you are extracting from one or multiple sources. The btm_my.translate file ties the name of the data source you defined as the Warehouse Source in the DB2 Data Warehouse Center to the table name on the data source Steps to reconfigure the Warehouse Source Re-configuration of the Warehouse Source for the Tivoli Storage Resource Manager repositories may be required if you are using two unique User Ids with different assigned privileges or at some point in the future you want to point to another source repository. The daily ETL processing requires select and delete permissions in the source repository while the one time only step to create the archive tables and triggers requires additional privileges To make this change is not difficult and continues to use the existing Warehouse Source names your created during the installation processing. Note: if you need to use a different user login for the source database to execute the BTM_c5_Initialize_Process that creates the archive tables and triggers versus the login required for daily extract operation, you do not need to define a separate Warehouse Sources entry. Use the existing entry, just modify the user id and password for each source using the properties menu data source tab. In the following example we will change the User ID and Password for ITSRM1: Set Data source name to the name of the ODBC connection for the Tivoli Storage Resource Manager database used by the server. If this Warehouse Source is to a Tivoli Storage Resource Manager repository and it was configured during the warehouse pack installation the Data Source Name will be the name you defined. Note: an ODBC connection for the item in the example was created during the warehouse pack installation with the Data Source Name you selected: ITSRM1. Set the User ID field. Set the Password field to the password used to access the source repository. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 29

35 4.8.4 Steps for post installation 1. Make sure the DB2 Data Warehouse Center control database is set to TWH_MD. 2. Configure the btm_my.translate file. An entry must exist in this file for each Tivoli Storage Resource Manager data source you are using. Additionally, the proper schema name must be set for each one of these data sources in this file. 3. Verify the properties in the Warehouse Source folder for each Tivoli Storage Resource Manager data source. The information required is for the Tivoli Storage Resource Manager database ODBC_data source. You can verify the settings in the properties. 4. Run the BTM_c5_Initialize_Process under the subject areas folder on the DB2 Data Warehouse Center to install the triggers on the Tivoli Storage Resource Manager database and create the archive tables. Each BTM_<ITSRM1>_Source in the Warehouse Source folder must use a login that has permission to create tables, indices, sequences and triggers. See section for instruction on temporarily changing the User ID and Password to one with the required privledges. BTM_C5_Initialize is run only once as part of the installation of the warehouse pack. You may need to re-run this step if a pre-warehouse pack version of the database is restored or you add another Tivoli Storage Resource Manager repository as a data source. Remember to reset your user id and password if this step required using a database login with additional administrator authority. 5. Set up the daily schedule in the DB2 Data Warehouse Center to run the BTM_c1_ETL1_.Process and BTM_m5_ETL2_Process under the BMT_Tivoli_Storage_Resource_Manager_v1.2_Subject_Area. Follow the instructions in the document Installing and Configuring Tivoli Data Warehouse. Specify dependencies between processes and schedule processes that are to run automatically. The processes should be run in the following order: BTM_C1_ETLProcess runs as the scheduled process to populate the central data warehouse. The recommended frequency for running this process is daily. BTM_M5_ETL2_Process runs as the scheduled process to populate the data mart. The recommended frequency for running this process is daily. 6. BTM_C15_Expire_Server process should not be scheduled. This process is run only if you want to remove all information and statistics about the Tivoli Storage Resource Manager server from the central data warehouse. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 3

36 7. Verify the prune data parameters are configured properly for your environment. See section Scheduling ETL processes Only you can determine when to schedule ETLs to run in your environment. Consider any dependencies carefully before scheduling your ETLs. Read the section on completing the install of the warehouse pack in the document Installing and Configuring Tivoli Data Warehouse. The ETL scheduling process specifies when and how often the ETL processes run. The best time to schedule ETL processes is during times of minimal system activity. The time required for an ETL process to complete is dependent upon the amount of data generated by the application. Do not schedule the ETL processes to run during regular business hours, if possible. If you linked ETL processes together with shortcuts, you specify scheduling parameters for the first step in the first ETL process. The remaining processes are run automatically upon the successful completion of the previous step. If any step in an ETL process fails, none of the remaining processes are run Configuring a process dependency The Tivoli Storage Resource Manager warehouse pack can be configured so that the central data warehouse ETL process must run successfully before automatically starting the data mart ETL process. You create shortcuts for application ETL processes in order to run them in a specified order and ensure that they do not run if the previous ETL process did not complete successfully. Alternatively, if you know when you want your ETL processes to run and know how long they will take, you can schedule ETL processes to run at specified times. Note: Before creating shortcuts, make a backup copy of the TWH_MD database in case you need to recover to the initial state of the installed warehouse pack. Also, write down which processes you link and the steps that you take to link these processes and save this information. If you install a fix to warehouse pack, it is possible that you will have to reschedule all ETL processes and the information you write down now will help you do this. Perform the following steps to create a shortcut: 1. In the DB2 Data Warehouse Center window in the Processes folder for a warehouse pack, double-click the name of the first warehouse pack ETL process to be run, BTM_c1_ETL1_Process: IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 31

37 IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 32

38 The Process Model window is displayed, which illustrates how data is transformed by that ETL process. The ETL process flow is defined in steps, and the data tables used in the process are distinguished from the actual steps by different icons. Scroll down in the Process Model window to find the name of the last step run in the process. 2. Change the mode of the last step in the process in which you are creating a shortcut as follows: a. From the Process Model window, right-click the step name of the last step in the Process Model window.: BTM_c1_s5_Filesystem_Load b. Select Mode then Development. Steps must be in Development mode to create or modify shortcuts. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 33

39 3. Click the create shortcut button at the bottom in the left toolbar of the Process Model window and move the mouse to an empty location below the last step in the Process Model window and click again. 4. In the Create Shortcut window, locate the appropriate Processes folder and select the process that contains the next step to be run. The process that you select is either the next process listed in the Processes folder for the same warehouse pack, or it is the first process listed in the Processes folder for the next warehouse pack. The next step to run is the first step in this new process. 5. Select the next step to be run, move the step from Available steps to Selected steps by clicking the right arrow button (>), and then click OK. The shortcut is added to the Process Model window. Shortcuts are identified by a curved arrow attached at the lower left of the step icon and the name of the shortcut reflects the next ETL step to be run. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 34

40 8. Right-click the newly added shortcut in the Process Model window, and then click the link tools button, the multicolored, downward pointing arrow icon on the left side of the window pane. 9. Select On Success and connect the BTM_c1_s5_Filesystem_Load to the BTM_m5_s1_Mart_Pre_Extract step. 1. Click on BTM_c1_s5_Filesystem_Load move the cursor to BTM_m5_s1_Mart_Pre_Extract - a green arrow will display. 11. In the Process Model window, click Process -> Save. 12. Close the Process Model window. When the ETL processes run, the On success parameter specifies that the step called by the shortcut only runs if the previous step completed successfully. If an error occurs when the step runs, that step exits and subsequent steps are not run Schedule set up The same procedure is followed for each ETL process that you are scheduling, however, you specify different parameters in the Schedule window depending on whether you are scheduling ETL processes to run only once, at specified times, or sequentially, as shown in the following steps: 1. After a warehouse pack installation, the mode of the steps in the ETL processes should be set to Development. However, if you need to change the mode for the ETL steps, perform the following steps: a. In the left navigational panel of the DB2 Data Warehouse Center window, click the Processes folder for a warehouse pack. b. Click the name of a warehouse pack ETL process that is scheduled to run. The process steps are then listed in the right navigational panel. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 35

41 c. Select all the step names that you need to change. d. Right-click the set of steps, and then select Mode. e. Select Development or Test. A step must be in Development mode or Test mode to schedule it. f. Repeat these steps for each process that contains ETL steps whose mode needs to change. 2. If you have DB2 remote warehouse agents defined for your deployment, for each step in the ETL processes for a warehouse pack, complete these instructions to specify the DB2 warehouse agent that is to run each step: a. Right-click the step name and select Properties. b. Click the Processing Options tab. c. Select the DB2 remote warehouse agent site to run the step from the agent site drop-down list. Click OK to close the Properties window. 3. From the Processes folder, select the process that contains the first step to run. The individual steps contained in the process are listed in the right pane of the DB2 Data Warehouse Center window. BTM_c1_ETL1_Process 4. In the right pane of the DB2 Data Warehouse Center window, select the step that you want to schedule and select Schedule. BTM_c1_s1_Pre_Extract IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 36

42 5. To schedule an ETL step to run automatically, perform the following steps: a. In the Schedule tab of the Schedule window, select an interval other than One time only and enter a frequency, day, and start and end date. This specifies when and how often the ETL step runs. b. Click Add. 6. Click the Notification tab in the Schedule window. 7. In the Notify on box, select Completion to specify that you want to be notified when ETL step has completed. 8. Select the type of notification, the users to receive notification, and other details, and then click Add. See the IBM DB2 Universal Database Data Warehouse Center Administration Guide or the online help for the DB2 Data Warehouse Center for more information about setting notification. Note: It is strongly recommended that you select Notify on: Completion for each step in your ETL processes. This information is helpful for monitoring your ETL processes. Selecting the Notification Type of helps ensure prompt action to resolve errors. 9. Click OK. 1. Change all steps to production mode, as follows: a. In the right pane of the DB2 Data Warehouse Center window, select all the steps except for steps that are called by a shortcut, and right-click. b. Select all the step names that you need to change. c. Right-click the set of steps, and then select Mode. d. Select Production. A step must be in Production mode for it to run automatically. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 37

43 11. In the Process Model window, click Process then Save. 12. Close the Process Model window Viewing Reports Tivoli Storage Resource Manager Warehouse Pack reports can be viewed using the web browser on your computer. The Crystal Reports server was configured as part of the Tivoli Data Warehouse v 1.2 installation, you can utilize the Crystal Launchpad to access eportfolio by entering the URL for the system where the Crystal Server is installed: or IP Address>/crystal/enterprise9 The Crystal LaunchPad provides the following selections: eportfolio eportfolio is a Web-based desktop that is an interface for users to view, schedule, and monitor published reports. You can also use eportfolio to help troubleshoot problems that users encounter when viewing and scheduling reports. Crystal Configuration Manager (CCM) This server administration tool is provided in two forms. In a Windows environment, you can use CCM to manage local and remote servers through its GUI or from a command line. In a UNIX environment, you can use the CCM shell script (ccm.sh) to manage servers from a command line. Crystal Management Console (CMC) This Web application provides a single interface to use to perform almost every task related to user management, content management, and server management. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 38

44 Using report parameters Each report provides a set of parameters to customize the view of the report by specifying an individual Tivoli Storage Resource Manager server or host, or permit you to view all historical information on the environment. You must fill in all parameter selections then scroll down the page and click the OK button at the bottom of the page. Some fields have preset values others require you to make a parameter selection. Percent for Free Space - leaving the value as zero displays free space for all file systems; modifying the value will display file systems with a percent of free space equal to or greater than the value you entered Set number of days to view - permits you to limit the number of days yof daya you will see in the report when the Granularity is Daily Granularity - will display data in the time period you select Click OK to complete your report selection You must click on Select to set ALL to view all data or enter your choice and then click on Select IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 39

45 4.9 Uninstallation of the warehouse pack To uninstall a Warehouse Pack follow the uninstall procedures outlined in the Installing and Configuring Tivoli Data Warehouse document. If you are upgrading a Tivoli Storage Resource Manager Warehouse Pack v 1.1 to the version 1.2 level, you will need to install the Tivoli Data Warehouse v 1.2. As part of the data warehouse version 1.2 upgrade any currently installed warehouse pack will be migrated to the data warehouse version 1.2 environment. Allowing the migration to complete for a currently installed Warehouse Pack insures existing central data warehouse data gets moved to the new table formats for processing. After the Tivoli Data Warehouse 1.2 upgrade is complete, you will need to uninstall the Tivoli Storage Resource Manager Warehouse Pack v1.1 and then install Warehouse Pack version 1.2. The uninstall of the old version of the warehouse pack and installation of the new version does not affect application data already in the central data warehouse and the new Warehouse Pack will continue to add data to the existing information. Since Tivoli Storage Resource Manager v 1.1 did not have a data mart ETL nothing in the TWH_MART database will be affected. When the v 1.2 warehouse pack is uninstalled, the following tables and views are removed, but the data in the central data warehouse database remains and is still useable by other applications. Tables BTM.BKUP_FILESYSTEM BTM.CUST_LOOKUP BTM.CENTR_LOOKUP BTM.ERROR_STATUS BTM.EXPIRE_STATUS BTM.EXCEPT_MSMT BTM.EXCEPT_SRVR BTM.STAGE_EXP_COMP BTM.STAGE_EXP_FILESYS BTM.EXPIRE_COMPUTER BTM.EXPIRE_FILESYSTEM BTM.EXPIRE_STATUS BTM.TMZONE_LOOKUP BTM.STAGE_A_SERVER BTM.STAGE_A_COMPUTER BTM.STAGE_A_STAT_COMP BTM.STAGE_A_FILESYSTEM BTM.STAGE_R_SERVER BTM.STAGE_R_COMPUTER BTM.STAGE_R_FILESYSTEM BTM.STAGE_SERVER BTM.STAGE_COMPUTER BTM.STAGE_FILESYSTEM BTM.STAGE_S_COMPUTER BTM.STAGE_FILESHOUR BTM.STAGE_FILESHOURAVG BTM.STAGE_FILESMSMT Views BTM.VE_SRM_SRVBYHNAME BTM.VE_SRM_CLIENT BTM.VE_SRM_FSYS BTM.VE_FSYS_METRIC BTM.VE_STG_F_FSYS_HOUR BTM.VD_SRM_SRVBYHNAME BTM.VD_SRM_CLIENT BTM.VD_SRM_FSYS BTM.VD_FSYS_METRIC BTM.VF_STG_F_FSYS_HOUR IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 4

46 4.1 Multiple data centers After you install the warehouse pack, you can configure Tivoli Data Warehouse to separate data for multiple data centers. To set this up, you must create SQL scripts with the following values: Information for scripts Field in source data Name of lookup table Name of center list Value or location Fully qualified host name or IP address BTM.Centr_Lookup TWG.Centr For the procedural instructions and example SQL statements, see the information about warehouse pack installation in the Installing and Configuring Tivoli Data Warehouse guide. After the initial configuration for multiple data centers, you must modify the tables when data centers are added and removed Multiple customer environments After you install the warehouse pack, you can configure Tivoli Data Warehouse to separate data for the multiple customer environment. To set this up, you must create SQL scripts with the following values: Information for scripts Field in source data Name of lookup table Column to use for lookup Name of customer list Value or location Fully qualified host name or IP address BTM.Cust_lookup table <select either Cust_ID or Cust_Acct_Cd> TWG.Cust For the procedural instructions and example SQL statements, see the information about warehouse pack installation in the Installing and Configuring Tivoli Data Warehouse guide. After your initial configuration of the multiple customer environment, you must modify these tables when customers are added and removed Migrating from a previous release of the warehouse pack Since the previous release of this warehouse pack, the following are the changes to the data model: Installation of Tivoli Storage Resource Manager v 1.3 Warehouse Pack v 1.2 is a new install of the data warehouse application. Where an enterprise is currently using Tivoli Storage Resource Manager v 1.2 Warehouse Enablement Pack v 1.1, follow the procedure to uninstall the v 1.1 warehouse pack and install the 1.2 version. Data in the Central Data Warehouse will remain and not be affected by uninstalling the previous version of the warehouse pack. Added objects: BTM_c1_s1_Pre_Extract, new to support multiple data source extract, see page 57 BTM_m1_ETL2_Process, five new steps for data mart processing, see page 61 BTM.Prune_Mart_Control, data mart pruning, see page 47 Btm_my.translation file, aligns data source schema or owner with the data source name, see page 32 IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 41

47 Changed objects: BTM_c1_s2_Extract, combines transform step, see page 57 The central data warehouse attribute STORAGE_GUID is replaced by the common attribute name SYSTEM_GUID, see page 13. The format of the SYSTEM_GUID will now be , see page 13 Removed objects: BTM_c1_s2_Transform, combined with BTM_c1_s2_Extract In order for the warehouse pack to function correctly, you must perform the following procedure: 1. Successfully install and configure the warehouse pack, run the step to create archive tables on each data source, configure the btm_my.translate file. 2. On a regular basis run the central data warehouse ETL and data mart ETL processes. 3. Re-specify the dependencies between processes and re-schedule the processes that are to run automatically. See Error! Reference source not found. on page Error! Bookmark not defined.. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 42

48 5 Maintenance and problem determination Normal maintenance processing includes pruning of data from the central data warehouse and data mart as well as the rollup of data mart fact table information into the periodic tables for week, month, quarter and year. These procedures are automatically performed after setting up the proper configuration parameters. No other maintenance procedures should be required. See section 5.2 on deleting data for further information. 5.1 Backing up and restoring This section provides information about backing up and restoring Tivoli Storage Resource Manager and Tivoli Data Warehouse. Due to the creation of the archive files and triggers in the Tivoli Storage Resource Manager Enterprise Repository, you may need to rerun the BTM_c5_Initialize_Process against the repository if the database backup was done before these tables and triggers were created. Any archive records in a restore that had been previously processed are ignored. Performing the step BTM_c5_Initialize_Process again does not negatively impact the restored database and tables however, if there is recent data in the archive tables and you rerun this step the new data will be lost. A restore of the Tivoli Storage Resource Manager Enterprise Repository may result in duplicate measurements for the current days run if an update probe or scan is not performed in order to record current system status. Do not attempt to delete measurement data from the central data warehouse. Consult the Tivoli Enterprise Data Warehouse documentation for procedures on backing up the central data warehouse and data mart. 5.2 Pruning data To manage the high volume of measurement data, use the Prune_Msmt_Ctrl table to delete that data. The Prune_Msmt_Log table keeps a history of data deletion. The default length of the deletion interval is 3 months. The CDW_c5_Prune_and_Mark_Active process within the CDW_Tivoli_Data_Warehouse_v1.2._Subject_Area controls deletion. By default, this process runs daily at 6: a.m. Measurements are automatically pruned from the Tivoli Enterprise Data Warehouse via the parameter set in the TWG.Prune_Msmt table in the central data warehouse. The warehouse pack sets the prune measurement at 3 days. The measurement set for the Tivoli Storage Resource Manager warehouse pack will not affect measurements for other application warehouse packs Central data warehouse The Prune_Msmt_Control governs which data is deleted. By default the data older than 3 months is deleted when the CDW_c5_Prune_and_Mark_Active process runs Deleting measurement data (table Prune_Msmt_Control) Measurement data is deleted from the Msmt table based on the age specified in the PMsmtC_Age_In_Days column. MSrc_Cd CHAR (6) TmSum_Cd CHAR PMsmtC_Age_In_Days BTM H Data mart Deleting data from the fact tables is implemented in the BTM_m5_s5_Mart_Prune. The prune mart control table BTM.Prune_Mart_Control governs which data is deleted and contains a date duration value for the D_FSYS_HOUR, D_FSYS_DAY, D_FSYS_WEEK, D_FSYS_MONTH, D_FSYS_QUARTER, and D_FSYS_YEAR fact tables. By default, all hourly and daily fact data older than 9 days is deleted when the process IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 43

49 step runs. Additionally, all weekly and monthly fact data older than 12 months is deleted. The BTM.Prune_Mart_Log table keeps a history of data deletion. 5.3 Extraction control (table Extract Control) The extraction control table assists you in incrementally extracting data from a source database. For an example of incremental extraction, see the Enabling an Application for Tivoli Data Warehouse guide. Extract Control Source Extract Control Target Purpose Name Name 1.ITSRM BTM.Stage_R_SERVER Updates ExtCtl_From_DtTm and ExtCtl_To_DtTm with timestamp of current extract 1.ITSRM BTM.Stage_R_COMPUTER Updates ExtCtl_From_DtTm and ExtCtl_To_DtTm with timestamp of current extract 1.ITSRM BTM.Stage_R_FILESYSTEM Updates ExtCtl_From_DtTm and ExtCtl_To_DtTm with maximum timestamp from target table name. This controls no duplicate entries are entered into the TWG.MSMT table BTM.MSMT.CONTROL BTM File System Msmt to Process Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq with either a or 1. This insures there is no duplication of measurement data if an central data warehouse ETL step needs to be restarted. BTM.VE_FSYS_METRIC BTM.T_FSYS_METRIC Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq values with the value in the extract view table. This controls extract of records from the transform views in TWH_CDW to TWH_MART. ExtCtl_To_IntSeq is set to the record number of the last record extracted. BTM.VE_SRM_CLIENT BTM.T_SRM_CLIENT Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq values with the value in the extract view table. This controls extract of records from the transform views in TWH_CDW to TWH_MART.. ExtCtl_To_IntSeq is set to the record number of the last record extracted. BTM.VE_SRM_FSYS BTM.T_SRM_FSYS Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq values with the value in the extract view table. This controls extract of records from the transform views in TWH_CDW to TWH_MART.. ExtCtl_To_IntSeq is set to the record number of the last record extracted. BTM.VE_SRM_SRVBYHNAME BTM.T_SRM_SRVR Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq values with the value in the extract view table. This controls extract of records from the transform views in TWH_CDW to TWH_MART.. ExtCtl_To_IntSeq is set to the record number of the last record extracted. BTM.VE_STG_F_FSYS_HOUR BTM.STG_F_FSYS_HOUR Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq values with the value in the extract view table. This controls extract of records from the transform views in TWH_CDW to TWH_MART.. ExtCtl_To_IntSeq is set to the record number of the last record extracted. BTM.VE_SRM_FSYS BTM.T_SRM_FSYS Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq values with the value in the extract view table. This controls extract of records from the transform views in TWH_CDW to TWH_MART.. ExtCtl_To_IntSeq is set to the record number of the last record extracted. BTM.VE_SRM_SRVBYHNAME BTM.T_SRM_SRVR Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq values with the value in the extract view table. This controls extract of records from the transform views in TWH_CDW to TWH_MART.. ExtCtl_To_IntSeq is set to the record number of the last record extracted. BTM.VE_STG_F_FSYS_HOUR BTM.STG_F_FSYS_HOUR Updates ExtCtl_From_IntSeq and ExtCtl_To_IntSeq values with the value in the extract view table. This controls extract of records from the transform views in TWH_CDW to TWH_MART.. ExtCtl_To_IntSeq is set to the record number of the last record extracted. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 44

50 Sample of table contents: ExtCtl_Sou rce VARCHA R (12) ExtCtl_Tar get VARCHA R (12) ExtCtl_Fro m_rawseq CHAR (1) ExtCtl_to_ RawSeq CHAR (1) ExtCtl_Fro m_intseq BIGINT ExtCtl_To_ IntSeq BIGINT ExtCtl_Fro m_dttm TIMESTA MP ExtCtl_To_ DtTm TIMESTA MP ExtCtl_MS rc_corr_c d CHAR (6) 1.ITSRM 1.ITSRM 1.ITSRM BTM.MSMT.CONTROL BTM.VE_F SYS_METR IC BTM.VE_S RM_CLIEN T BTM.VE_S RM_FSYS BTM.VE_S RM_SRVBY HNAME BTM.VE_S TG_F_FSY S_HOUR BTM.VE_S RM_FSYS BTM.VE_S RM_SRVBY HNAME BTM.VE_S TG_F_FSY S_HOUR BTM.Stag e_r_serv ER BTM.Stag e_r_comp UTER BTM.Stag e_r_file SYSTEM BTM File System Msmt to Process BTM.T_FS YS_METRI C BTM.T_SR M_CLIENT BTM.T_SR M_FSYS BTM.T_SR M_SRVR BTM.STG_ F_FSYS_H OUR BTM.T_SR M_FSYS BTM.T_SR M_SRVR BTM.STG_ F_FSYS_H OUR NULL NULL :5:24.73 NULL NULL :5:24.73 NULL NULL :5:24.73 NULL NULL :5:24.73 NULL NULL 1-1 ::. NULL NULL 1-1 ::. NULL NULL 1-1 ::. NULL NULL 1-1 ::. NULL NULL 1-1 ::. NULL NULL 1-1 ::. NULL NULL 1-1 ::. NULL NULL 1-1 :: :5: :5: :5: :5: ::. 1-1 ::. 1-1 ::. 1-1 ::. 1-1 ::. 1-1 ::. 1-1 ::. 1-1 ::. BTM BTM BTM BTM BTM BTM BTM BTM BTM BTM BTM BTM 5.4 Maintenance of the Tivoli Storage Resource Manager archive tables The archive tables created in the Tivoli Storage Resource Manager Enterprise Repository are cleared of existing records after the Warehouse Pack extract process is run. When Tivoli Storage Resource Manager deletes a computer or file system, the appropriate records are removed from their internal tables and the record is written to its corresponding archive table. A set of three archive tables is created to coordinate deleted system information with the central data warehouse. A trigger is installed on the Tivoli Storage Resource Manager Enterprise Repository base tables copies deleted records into the archive tables. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 45

51 When the BTM_C1_ETL1_Process is run, the extract step copies records from the Tivoli Storage Resource Manager Enterprise Repository archive tables to the central data warehouse so that the appropriate information can be expired. The extract step deletes the table when it collects records from the archive tables. 5.5 Problem determination The steps in the BTM_C1_ETL1_Process already have the mode set to production. When you installed the warehouse pack you did not click the selection to not automatically schedule the warehouse pack to run each day at a specified time. To schedule your warehouse pack just set the mode for each step in the process back to mode development The BTM_c1_s2_Extract step failed Check the BTM.ERROR_STATUS table in the TWH_CDW for entries describing the failure. If no errors are listed proceed to the logging directory and look for the BTM_c1_s2_Extract.log file, the last entry in the log will show the sql statement that failed. This step is programmed to fail for the following reasons: A. More than one server is listed in the source database table T_RES_SERVER. Look for a table in TWH_CDW named BTM.EXCEPT_SERVER containing the entry: 'BTM: can only have one ITSRM server name'. To resolve the problem remove all records in the table not for the current ITSRM server; rerun the extract step. B. The server name cannot be formatted as either a fully qualified hostname or IP address. Look for a table in TWH_CDW named BTM.EXCEPT_SERVER containing the entry: 'BTM: ITSRM server name failed inspection - check BTM_c1_s2_Extract log for mismatch on IP Address.' To resolve the problem examine the entry in the T_RES_COMPUTER and the T_RES_SERVER tables fix the name and rerun the extract step The BTM_c1_s1_Pre_extract is not failing but the BTM.ERROR_STATUS shows an error BTM_c1_s1_Pre_Extract: Stage_Filesystem copied to BKUP_Filesystem. Rerun the Load steps. This can occur if there are no new records in the BTM.Stage_Filesystem table. Execution will continue to the other steps no records were copied to BTM.BKUP_Filesystem. Ignore the error The BTM_c1_s1_Pre_Extract failed Look for a table in TWH_CDW named BTM.EXCEPT_MSMT containing the entry: 'BTM: BTM.Stage_Filesystem reset flag for TWG.MSMT table has not been cleared. Re-setting BTM.MSMT.CONTROL flag will result in records being deleted from BTM.Stage_Filesystem.' To resolve this problem rerun the BTM_c1_s2_Extract step and proceed through each of the load steps. To insure measurements from the data source do not get added to the central data warehouse more than once, a flag is set in the TWG.EXTRACT_CONTROL table after valid extracted records are added to BTM.Stage_Filesystem. The flag is then reset after the records are added to the TWG.MSMT table in the TWH_CDW database. The BTM_c1_s1_Pre_Extract step always checks that the record to load flag is set to zero so source records are not lost. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 46

52 BTM.MSMT.CONTROL BTM File System Msmt to Process NULL NULL ::. 1-1 ::. BTM When extracting data from more than one source database for ITSRM, a failure can occur while extracting data from the different sources. If the BTM_C1_S1_pre_extract step is re-run to restart the central data warehouse ETL instead of just restarting the BTM_c1_s2_Extract step, the data collected from the previous successful extracts will be lost since TDW extracts data from each defined source before continuing to the load steps. In order to protect against this happening the pre_extract step examines the record to load flag and also copies any records currently in BTM.Stage_Filesystem to a backup table BTM.BKUP_Filesystem The names of the hosts are being truncated in the graphs The names are longer than the space allotted by Crystal in creating the report. This is controlled by the crystal graph processor - we only pass them the column name for the axis Why are my reports empty? In general the reports are empty because no data was available to populate them. This can occur for two reasons: A) the parameters set when the report request was submitted excluded valid data or B) no data exists in the data mart. IBM Tivoli Storage Resource Manager Warehouse Pack Implementation Guide 47