Install and configure Tivoli Common Reporting Cognos-based reports within IBM Tivoli Monitoring v6

Transcription

1 IBM Tivoli Monitoring v6 Cognos Reporting Installation Best Practices September 2010 Install and configure Tivoli Common Reporting Cognos-based reports within IBM Tivoli Monitoring v6 Document version 1.0 Antonio Sgro, Services Availability Performances Management Best Practice Team Stefano Borghetti, IBM Tivoli Monitoring Development Payal Chakravarty, IBM Tivoli Common Reporting Development Copyright International Business Machines Corporation All rights reserved. US Government Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

2 CONTENTS 1 Overview Installation Prerequisites TDW Configuration Prepare TDW with Shared Dimensions Adding Tivoli Reporting and Analytics Model (TRAM) Dimensions Creating and Populating the Managed System Dimension Connecting to the TDW via Database Client over ODBC Installation using the java tool Manual Installation Configuration Setup multiple TDW connection Setting up Database connection via ODBC using Database Client Creating new database connection in TCR Main Features OS Agents Data Model Virtual Servers (VMWare VI) Agents Data Model System P Agents Data Model IBM Tivoli Monitoring Agents TCR Cognos-based Reports Known issues Appendix...39 Useful links...39 Acronyms...40 ii

3 Trademarks...41 iii

4 1 Overview Reporting capability is one of the most important features inside an IT organization in order to have full visibility of the resources usage over the time. Tivoli Common Reporting is the Tivoli standard infrastructure for creating, viewing, and managing Tivoli product reports. Throughout the rest of this document, we will refer to Tivoli Common Reporting as TCR. More than 40 Tivoli products ship TCR reports providing historical views of availability, utilization, performance, and many other key metrics that helps users to take value of the collected data in terms of capacity plan, data analysis. TCR 1.1 and 1.2 versions were built on a BIRT reporting technology that required too much programming skill, demanded too much knowledge of the database in order to create reporting. In order to address the above user s panic and with the help of Cognos acquisition, TCR 1.3 was built on top of Cognos reporting technology and several benefits were provided. Simplified report editing (Launch web-based Report Studio in context from a report) Ad-Hoc Reporting (Create ad-hoc, on the fly reports) Other Misc. capabilities o o o Automatic ing of reports Additional report formats (XML. Excel 2000 & 2000 Single Sheet, 2002, and 2007, CSV, html,pdf) Greater granular data security (row-level authorization) In addition, the newest TCR based on Cognos technology brings values in terms of Consistency in reporting with other Tivoli products Allowing report data correlation with other Tivoli product data and business data Allowing customers access (via upgrade) to IBM Cognos advanced analytics modules for advanced reporting capabilities that were not possible with TCR 1.2. BIRT reporting continues to be supported by TCR, and some of the adopting products in the Tivoli portfolio may ship BIRT based reports as well as Cognos based reports. 2 Installation In order to install TCR Cognos-based reports, there are some prerequisites that need to be satisfied and that depends from the specific product that is releasing the set of reports. Cognos relies on a data model that helps normalize the data and make it easier to configure reports. The data model is unique to each product. The following section outlines how to configure the prerequisites.

5 2.1 Prerequisites These are the prerequisite components for installing and running Tivoli Common Reporting packages in Tivoli Monitoring products. To use the reports, you need at least the following components: IBM Tivoli Monitoring Version Fix Pack 2 or later for OS Agents reports. (OS Agents can be at level or later) ITM or later for ITCAM for Transactions reports ITM FP02 or later for System P and VMWare agents reports Tivoli Common Reporting Version 1.3 Report packages: your product might have a separate reports package that must be extracted For additional requirements, the specific product documentation needs to be consulted. Notes: With Tivoli Common Reporting 1.3, both BIRT and Cognos report engines can co-exist TDW Configuration In order to make TCR Cognos-based reports working, some pre-configuration steps need to be executed on the TDW database. The following sections describe all of the necessary steps Prepare TDW with Shared Dimensions Preparing the Tivoli Data Warehouse for Tivoli Common Reporting includes creating the shared dimensions which are required for running the TCR Cognos-based reports and using the data models. There are two types of dimensions you will create: 1. IBM Tivoli Reporting and Analytics Model (TRAM) Dimensions: This includes the Time Dimension and Computer System dimension which are used by products across Tivoli providing support for cross product reporting. 2. ITM s Managed System Dimension or Resource Dimension: This dimension consists of a normalized table of all the Managed Systems in the environment Adding Tivoli Reporting and Analytics Model (TRAM) Dimensions The following dimensions tables are created by this procedure: Schema IBM_TRAM: TRAM stands for Tivoli Reporting and Analytics Model which is the common data model used by Tivoli products. Table TIME_DIMENSION with years of time dimensional data and granularity to a specified number of minutes. Each row of this table is a unique minute key with various dimensions related to it, such as hour, weekday, day of month, quarter etc. Table MONTH_LOOKUP globalizes the month names for Time Dimension. Table WEEKDAY_LOOKUP globalizes the weekday names for Time Dimension. 5

6 Other dimensions conforming to the Tivoli Common Data Model, such as ComputerSystem, BusinessService, and SiteInfo. You will need the database scripts included in the extracted reports package under the db_scripts directory. Procedure IBM DB2 1. Copy the database scripts (.db2 files) from the reports package to a location where they can be run against the Tivoli Data Warehouse. The scripts are in the db_scripts directory. 2. Log in as a user with database administrative rights such as db2admin or db2inst1. Your user ID must have administrator access to create the IBM_TRAM schema. 3. Connect to the database that you want to create the dimension tables for. This is your Tivoli Data Warehouse. db2 connect to WAREHOUS; 4. If you have an older version of the database scripts already installed clean up the database: start clean.db2 5. Create the schema and tables: db2 -tf create_schema_ibm_tram.db2 After the command completes successfully, several new tables are created in the IBM_TRAM schema: TIME_DIMENSION, MONTH_LOOKUP, WEEKDAY_LOOKUP, ComputerSystem, BusinessService, SiteInfo, and so on. 6. Create the stored procedure for generating the time dimension: db2 -td@ -vf gen_time_dim_granularity_min.db2 7. To populate TIME_DIMENSION table, call the time dimension stored procedure with dates and granularity to generate the timestamps. You can generate up to five years at a time or have the data regenerated every day. 8. db2 call IBM_TRAM.CREATE_TIME_DIMENSION('start_date', 'end_date', granularity_of_data); where start date and end date are in this format YYYY-MM-DD-HH.MM.SS.MILSEC and granularity of data is the frequency in minutes. For example, the following command generates from 12/31/2009 to 12/31/2010 with 60-minute granularity. db2 call IBM_TRAM.CREATE_TIME_DIMENSION(' ', ' ', 60); Microsoft SQL Server 1. Copy the database scripts (.sql files) from the reports package to a location where they can be run against the Tivoli Data Warehouse. The scripts are in the db_scripts branch of the directory where the reports package was extracted to. 2. Customize the provided scripts by changing the default database name in the use statement (replace USE IBM_TRAM) to match the name of your Tivoli Data Warehouse database. If the name of your Tivoli Data Warehouse is "warehouse" the statement is USE warehouse : 6

7 a. createschema.sql b. createprocedure.sql c. clean.sql d. populatetimedimension.sql Also, modify the boundary parameters for the time dimension and granularity. For example, = ' :00:00', = ' = 60, If Monday must be the first day of the week, add the fourth parameter equal to 1; otherwise, release three = 7 3. If you have an older version of the database scripts already installed, clean up the database. sqlcmd -i clean.sql [-U username -P password] [-S hostname] 4. Run the scripts at the MS SQL command line in this order: sqlcmd -i createschema.sql [-U username -P password] [-S host] sqlcmd -i createprocedure.sql [-U username -P password] [-S host] sqlcmd -i populatetimedimension.sql [-U username -P password] [-S host] Oracle manual installation 1. Copy the database scripts (.sql files) from the reports package to a location where they can be run against the Tivoli Data Warehouse. The scripts are in the db_scripts branch of the directory where the reports package was extracted to. 2. Start an SQL *Plus session if it is not already running. 3. Check that you can access remotely as sys user. 4. If you have an older version of the database scripts already installed clean up the database (the procedure must be called by the sys user): clean.sql 5. Take one of the following steps: Oracle batch installation If you can access remotely as sys user, run this command and provide all the information that the script If you cannot access remotely as sys user, run this command locally at the Oracle server and provide all the information that the script 1. Copy the database scripts (.sql files) from the reports package to a location where they can be run against the Tivoli Data Warehouse. The scripts are in the db_scripts branch of the directory where the reports package was extracted to. 2. Start an SQL *Plus session if it is not already running. 7

8 3. If you have an older version of the database scripts already installed clean up the database (the procedure must be called by the sys user): clean.sql 4. Create user IBM_TRAM (the script must be called by a user with system rights, such as SYS/SYSTEM): TCR_PASS USER_TBSPC TEMPORARY_TBSPC where TCR_PASS is the password for the new user, USER_TBSPC is the default user tablespaces name (must exist), and TEMPORARY_TBSPC is the default temporary tablespaces name (must exist) 6. Create the IBM_TRAM tables (the script must be called by the IBM_TRAM user created in the previous step): USER_TBSPC where USER_TBSPC is the default user tablespaces name (must exist) 8. Grant privileges to the user, such as ITMUSER (the script must be called by the IBM_TRAM user): USER where USER is the name of the user to grant privileges to. 10. Create the procedure (the script must be called by the IBM_TRAM 11. Load the lookup data (the script must be called by the IBM_TRAM 12. Generate the time dimension (the procedure must be called by the IBM_TRAM user): StartDate EndDate Granularity where StartDate is the start date in the format 'yyyy-mm-dd HH:MM', EndDate is the end date in the format 'yyyy-mm-dd HH:MM', and Granularity is the number of minutes. Example: ' :00' ' :00' '60' The shared dimensions and time dimensions tables are complete Creating and Populating the Managed System Dimension NOTE: This step is only required for OS Agents. If you are deploying reports for other agents you do not need to implement this step and can proceed to the next step. Preparing the Tivoli Data Warehouse for Tivoli Common Reporting also includes creating and populating the resource dimension table "ManagedSystem", which is required for running the Cognos-based reports and using the data models. This dimension consists of a normalized table of all the Managed Systems in the environment Before you begin 8

9 Before starting the procedure to build the resource dimension table, you must first configure historical data collection for one or more of the following attribute groups, depending on the operating system you are getting reports for: Type Attribute group Table Summarize Linux Linux IP Address Linux_IP_Address Daily UNIX UNIX IP Address UNIX_IP_Address Daily Windows Computer Information NT_Computer_Information Daily You can configure historical data collection in the Tivoli Enterprise Portal or in the Command Line Interface. The following example shows how a local historical collection for NT Computer Information was created and distributed from the CLI: tacmd login -s MyComputer -u MyUser -p MyPassword tacmd tepslogin -s localhost -u sysadmin tacmd histconfiguregroups -t knt -o "NT Computer Information" -m -d YQMQDH -p Y=2y,Q=2y,M=1y,W=1y,D=6m,H=14d,R=7d tacmd histcreatecollection -t knt -o "NT Computer Information" -a "ComputerInformation" -c 15m -i 15m -l TEMA -e "Needed for resource dimension table for TCR." tacmd histstartcollection -t "knt" -o "NT Computer Information" -u sysadmin See "Historical collection configuration" or tacmd histconfiguregroups, tacmd histcreatecollection, and tacmd histstartcollection in the IBM Tivoli Monitoring Command Reference for details. Procedure IBM DB2 1. Log in as db2admin. Your user ID must have administrator access to create the resource dimension. 2. Connect to the database that you want to create the resource dimension table for. This is your Tivoli Data Warehouse. db2 connect to WAREHOUS; 3. Create the tables: db2 -tf gen_resources.db2 After the command completes successfully, a new table is shown under the ITMUSER schema: ManagedSystem. 4. Create the stored procedure to populate the ManagedSystem table: db2 -td@ -vf populate_resources.db2 5. To populate ManagedSystem table, call the stored procedure: 9

10 db2 call ITMUSER.POPULATE_OSAGENTS(); Microsoft SQL Server 1. Customize the provided scripts: a. In create_table.sql, change the default database name in the use statement (replace USE WAREHOUS) if it is different from the default. b. In create_procedure.sql, change the default database name in the use statement (replace USE WAREHOUS) if it is different from the default. c. In populate_agents.sql, change the default database name in the use statement (replace USE WAREHOUS) if it is different from the default. 2. Run the scripts at the MS SQL command line in this order: sqlcmd -i create_table.sql [-U myusername -P mypassword] [-H myhost]) sqlcmd -i create_procedure.sql [-U myusername -P mypassword] [-H myhost]) sqlcmd -i populate_agents.sql [-U myusername -P mypassword] [-H my_host]) Oracle manual installation 1. Start a SQL *Plus session if it is not already running. 2. Run this command (path with no spaces) and provide all the information that the script Oracle batch installation 1. Start a SQL *Plus session if it is not already running. 2. Create the ITMUSER.ManagedSystem table. The script must be called by the Tivoli Data Warehouse user, which is ITMUSER by default. If you used a different user name, modify the script for the correct 3. Create the procedure to populate the 4. Start the procedure to populate the ManagedSystem table: 5. begin 6. POPULATE_OSAGENTS('ITMUSER'); 7. end; Results The ManagedSystem table is created and populated with managed systems of your environment. Automating the Population of Managed System Table Regularly The Managed System table is dynamic i.e., the contents will change based on the changes in your environment. As and when new managed systems are added the table needs to be updated. Hence it is necessary to automate the task of populating this table regularly. In order to do this the stored procedure to populate the ManagedSystem table (POPULATE_OSAGENTS) needs to be called by a daily scheduled automated job. 10

11 2.2 Connecting to the TDW via Database Client over ODBC Cognos uses ODBC to connect to the database. Hence it is important to first install a database client on the Tivoli Common Reporting server and have it connect to the Tivoli Data Warehouse. DB2 Make sure you have deployed a DB2 database client on the machine where Cognos-based Tivoli Common Reporting engine is installed. The client should be of the same version as the database that Tivoli Data Warehouse is using. Connect the DB2 database client to the database server by running the Configuration Assistant, configuring the local net service name configuration, and restarting your system. Oracle Make sure you have deployed the Oracle database client on the machine where Cognosbased Tivoli Common Reporting engine is installed. Connect the Oracle database client to the database server by running the Oracle Net Configuration Assistant, configuring the local net service name configuration, and restarting your system. MS SQL Server Make sure you have deployed the MS SQL database client on the machine where Cognosbased Tivoli Common Reporting engine is installed. Connect the MS SQL client to the database server by running the MS SQL Management Studio Express, configuring the local net service name configuration, and restarting your system. Important: Note the name of the connection you have created as it is used when you install the reports in Section Installation using the java tool In order to install IBM Tivoli Monitoring Cognos-based Reports, it is needed to use the reports installer packaged with each set of Cognos reports. Here is the procedure to install the reports (as example, the OS Agents reports are installed) Procedure 1. From the Cognos reports disk or the directory where the report package was extracted, launch the setup script: setup.bat tcr_install_dir. (If Tivoli Common Reporting is installed in the default directory, you do not need to specify an installation directory.)./setup.sh tcr_install_dir 2. Click Next on the following panel 11

12 3. Select the Reports on the left side and click on the right arrow (circled in red in the next panel) 4. Click on the Next button 12

13 5. In the following panel complete the following: Installation Type = on TCR 1.3 Enter the directory where TCR is installed Enter the Cognos user name and password And click on the Next button 13

14 6. If you want to skip the TDW data source, check the devoted flag and click next but then it will be needed to be created manually 7. In the next panel you will be prompted for the TDW data source connection parameters; after filling them, click on the Next button 8. In the following screenshot, you will see a summary of all the installation steps and settings; if everything is ok, click on the Install button 14

15 9. If the installation was successfully executed, you will see the following panel: 15

16 2.4 Manual Installation The Cognos-based reports can be installed not only using the Java installation tool but using a manual procedure hereafter described (as example Virtual Server Reports are used to describe the manual installation procedure): 1. In the ITM reports installation media, identify the reports zip file (ex. IBM Tivoli Monitoring for Virtual Servers Reports.zip) 2. Copy the report zip file in the TCR deployment directory: <TCR_Installation_Directory>/products/tcr/Cognos/c8/deployment 3. In the ITM Reports installation media, identify the images directory (ex. C:\IBM_Tivoli_Monitoring_Virtual_Servers_Reports\packages\VMwareVIAgents\im ages) 4. Copy the content of the images directory in the TCR images directory <TCR_Installation_Directory>/profiles/TIPProfile/installedApps/TIPCell/IBM Cognos 8.ear/p2pd.war/tivoli/ITM/images 5. Launch the IBM Cognos Administration from the TCR portal as described in the following figure: 16

17 6. From the Configuration tab create a TDW data source 7. From the Content Administration tab choice New Import Button and the report package that needs to be imported 17

18 8. Click Next, choose a name for your report package, and then click Next again. 9. Select the report package and click on the Next button 18

19 10. Click Next twice, and on this panel choose the Save and run once option and click Finish. 11. Finally, click Run and the report package will appear. 19

20 3 Configuration General configuration information 3.1 Setup multiple TDW connections Cognos supports the creation of multiple TDW connections. In order to do this you will need to do two things: 1. Use the database client to set up an ODBC connection to your database as explained in Section 2. Add the new connection in Cognos Administration Setting up Database connection via ODBC using Database Client DB2: Connect the DB2 database client to the database server by running the Configuration Assistant, configuring the local net service name configuration, and restarting your system. MS SQL Server: Connect the MS SQL client to the database server by running the MS SQL Management Studio Express, configuring the local net service name configuration, and restarting your system. Oracle: Connect the Oracle database client to the database server by running the Oracle Net Configuration Assistant, configuring the local net service name configuration, and restarting your system. Important: Note the name of the connection you have created as it is used in one of the following steps. 20

21 3.1.2 Creating new database connection in TCR Once you have completed step 1 and are ready to set up the data source connection in TCR, browse to the TCR panel and launch IBM Cognos Administration. In the Admin Panel, go to Configuration -> Data Sources. Select TDW that was created by the installation wizard. This is the data source name that is understood by Cognos for the ITM data model. DO NOT change this name. When you click on TDW it will take you to the database connections that the data source TDW points to. Here you will see a default TDW defined. Click on the New Connection icon to create a new database connection. 21

22 As you can see in the figure above, the Data Source TDW contains six different database connections. Once configured, when a report is run the user is given the ability to choose which database to go against. Once you click the New Connection, the New Connection wizard will launch. Enter the Name for this connection. It s a good practice to give this the same name as the database alias you have created in the client. Click Next. 22

23 In the next panel choose the database type from the drop down list i.e., either DB2, Oracle or Microsoft SQL Server (ODBC) and click Next. DB2 1. On the second panel choose an DB2 database as Type. 2. On the third panel specify the name of the connection you have noted before as the DB2 connect string, and in the Signon section specify a new User ID and Password. MS SQL 1. On the second panel choose an Microsoft SQL Server database as Type. 2. On the third panel specify the name of the connection you have noted before as the Server name, and in the Signon section specify a new User ID and Password. 23

24 Oracle 1. On the second panel choose an Oracle database as Type. 2. On the third panel specify the name of the connection you have noted before as the SQL*Net connect string, and in the Signon section specify a new User ID and Password. On the third panel as explained above enter the database name and the user ID and password. 24

25 Click the Test Connection link. In the next panel, click the Test Connection button If the connection is successful it will return saying the Connection is successful. You can then hit Close on this panel and the subsequent panel and then a final Finish to save this connection. Now when you look under the TDW data source you will find a new connection called Production TDW. Video Demo Link to set up data source connections: 25

26 4 Main Features 4.1 OS Agents Data Model The data model for the Monitoring Agents for Windows OS, Linux OS, and UNIX OS reports will have the following features: The model covers data from OS Agents only. The model contains a subset of the attribute groups collected by OS Agents in aggregated form for time dimension: AVG, MIN, MAX, SUM, LAT, TOT, HI, LOW. The model includes a Managed System dimension with the Agent Type attribute (on Windows, Linux, and UNIX systems). It is placed in the IBM Tivoli Monitoring Shared Dimensions namespace. The model includes all the aggregations handled by Summarization and Pruning from daily to yearly including the raw data. The model contains forecasting based on the linear trend for the following metrics for each time dimension: For Linux: KLZ_CPU_FCAST_XX.AVG_Idle_CPU KLZ_Disk_FCAST_XX.AVG_Disk_Used_Percent KLZ_VM_Stats_FCAST_XX.AVG_Used_Virtual_Storage_Pct KLZ_Network_FCAST_XX.AVG_Bytes_Transmitted_per_sec KLZ_Network_FCAST_XX.AVG_Bytes_Received_per_sec For UNIX: System_FCAST_XX.AVG_Idle_CPU Disk_FCAST_XX.AVG_Space_Used_Percent Unix_Memory_FCAST_XX.AVG_Virtual_Storage_Pct_Used Network_FCAST_XX.AVG_Transmitted_MB_Total Network_FCAST_XX.AVG_Received_MB_Total For Windows: NT_System_FCAST_XX.AVG_%_Total_Processor_Time NT_Logical_Disk_FCAST_XX.AVG_%_Used NT_Memory_64_FCAST_XX.AVG_Available_Usage_Percentage NT_Server_FCAST_XX.AVG_Bytes_Transmitted/sec NT_Server_FCAST_XX.AVG_Bytes_Received/sec The metrics data items names reflect the catalog attributes names with the following prefixes: SUM_ into Sum LAT_ into Latest MIN_ into Minimum MAX_ into Maximum TOT_ into Total AVG_ into Average HI_ into Higher LOW_ into Lower 26

27 Linux Agent Attribute Group Linux Linux CPU Averages KLZ_CPU_Averages UNIX Windows Linux CPU Linux Disk Linux Network Linux Process Linux VM Stats Linux Disk IO Linux Disk Usage Trends Linux IO Ext Linux NFS Statistics Linux Process User Info Linux RPC Statistics Linux Sockets Detail Linux Sockets Status Linux Swap Rate Linux System Statistics Linux User Login UNIX Disk Network Process Unix Memory System Disk Performance NFS and RPC Statistics SMP CPU Solaris Zones User Windows Logical Disk Hourly Memory Hourly Network Interface Hourly Process Hourly Server Hourly System Hourly ICMP Statistics Hourly IP Statistics Hourly Cache Hourly Device Dependencies Hourly Devices Hourly Event Log Hourly Monitored Logs Report Hourly Network Port Hourly Objects Hourly Paging File Hourly Physical Disk Hourly KLZ_CPU KLZ_Disk KLZ_Network KLZ_Process KLZ_VM_Stats KLZ_Disk_IO KLZ_Disk_Usage_Trends KLZ_IO_Ext KLZ_NFS_Statistics KLZ_Process_User_Info KLZ_RPC_Statistics KLZ_Sockets_Detail KLZ_Sockets_Status KLZ_Swap_Rate KLZ_System_Statistics KLZ_User_Login Disk Network Process Unix_Memory System Disk_Performance N_F_S_and_R_P_C_Statistics SMP_CPU Solaris_Zones User NT_Logical_Disk NT_Memory_64 NT_Network_Interface NT_Process_64 NT_Server NT_Process_64 ICMP_Statistics IP_Statistics NT_Cache NT_Device_Dependencies NT_Devices NT_Event_Log NT_Monitored_Logs_Report NT_Network_Port NT_Objects NT_Paging_File NT_Physical_Disk 27

28 Printer Hourly Processor Hourly Processor Summary Hourly Redirector Hourly Server Work Queues Hourly Service Dependencies Hourly Services Hourly Thread Hourly Print Queue Hourly Process IO Hourly TCP Statistics Hourly UDP Statistics Hourly NT_Printer NT_Processor NT_Processor_Summary NT_Redirector NT_Server_Work_Queues_64 NT_Service_Dependencies NT_Services NT_Thread Print_Queue Process_IO TCP_Statistics UDP_Statistics Table 1. Attributes groups supported by the data model The following figure shows the data model and some of its available attributes: 28

29 Forecasting The OS Agents reports provide a new functionality to forecast potential future values for a set of metrics. This forecast is calculated using a linear trending based on historical data in a most recent time window. There are limits in the future for forecasting depending on the aggregation type: Hourly: 60 hours in the future Daily: 60 days in the future Weekly: 1 year in the future Monthly: 5 years in the future Quarterly: no limit Yearly: no limit For reference, these are the formulas used to compute the forecasted values: Input data: INPUT = {( x, y ): i = 1,2, Κ n} i i, where: x i = series of timestamps expressed as number of seconds from : 00 : 00 y i = series of measurements ( x i, yi ) = metric value yi measured in the given time xi n = number of samples Output data 1. Linear approximation of historical data expressed as: y = ax + b R2 [ 0,1) 2. to show correlation between historical data and computed linear approximation. Solution 29

30 S S S S S xx yy xy x y R = = = = = = i= 1 i= 1 i= 1 n i= 1 n i= 1 d = ns a = b = n n n xx ( ns xy S xs y )/ d ( S xxs y S xs xy )/ d ( ns S S )/ ( ns S S )( ns S S ) R2 = R 2 x x x y x xy y y i i i i i i y i S i x S x x y xx x x yy y y Note: Related to forecasting, there is a subset of tables that are visible in the model, but cannot be used in custom queries and reports. These tables are contained in the following folders: Forecast Hourly Forecast Daily Forecast Weekly Forecast Monthly Forecast Quarterly Forecast Yearly Refer to the following link to see video demos of drag and drop report creation using the ITM OS Agents data model: Virtual Servers (VMWare VI) Agents Data Model The VMware VI Agent Data Model is what you will see in Data Navigation tree of the Query and Report Studio. The data model is a layer built on top of the Tivoli Data Warehouse to make the data more consumable to the end user. The data model contains pre-defined relationships so that when you drag and drop elements across different tables and views they join and you do not have to manually write any SQL. 30

31 The structure of the VMware VI Agents data model is shown in the figure below. Cognos data models are virtual star schema models separated into facts and dimensions. Facts are measurable quantities that can be aggregated like CPU Utilization, Number of Processors. Dimensions are the main identifiers by which facts can be grouped, aggregated and sliced and diced. For example, time and server are dimensions by which the fact CPU Utilization can be grouped. The data model contains two types of dimensions: the TCR Shared Dimension which are shared across Tivoli by products like Time and ITM for Virtual Servers Shared Dimension which are dimensions shared across the ITM for Virtual Servers agents like Datacenter, Clusters, Servers, Virtual Machines, Datastores etc. The facts in the data model are organized under folders by their summarization type like Daily and Hourly. On expanding Daily and Hourly you will be able to see the attribute groups (see figure below). Each attribute group corresponds to a table/view in the data warehouse. As you can see each attribute group contains a bunch of facts or measures like MIN_CPU_Utilization, AVG_CPU_Utilization and some identifiers like CPU_Number and Shift and Vacation periods. 31

32 To build a report you need to follow this order for dragging and dropping elements: 1. First drag one of the ITM for Virtual Servers Shared Dimensions like Datacenter/Cluster/Server/VMs.Server_Hostname 2. Drag any metric from a related attribute group. For example if you want to look at Server related metrics, drag anything under the attribute groups starting with Server in their name. For VM, drag data from the attribute groups whose names start with VM. 3. Finally drag and drop the Standard Timestamp from TCR Shared Dimensions.Time Dimension. For daily data you can use Date instead of Standard Timestamp. You can use any of the other time metrics to slice and dice the data. 32

33 Refer to the following link to see video demos of drag and drop report creation using the VMware VI Agent data model System P Agents Data Model The System P Agents Data Model is what you will see in Data Navigation tree of the Query and Report Studio. The data model is a layer built on top of the Tivoli Data Warehouse to make the data more consumable to the end user. The data model contains pre-defined relationships so that when you drag and drop elements across different tables and views they join and you do not have to manually write any SQL. The structure of the System P Agents data model is shown in the figure below. Cognos data models are virtual star schema models separated into facts and dimensions. Facts are measurable quantities that can be aggregated like CPU Utilization, Number of LPARs. Dimensions are the main identifiers by which facts can be grouped, aggregated and sliced and diced. For example, time and server are dimensions by which the fact CPU Utilization can be grouped. The System P Agents Data Model contains data for three agents: CEC Base Agent, AIX Premium Agent and VIOS Agent. When you expand the data model you will see TCR Shared Dimensions, CEC Base Agent, VIOS Agent and AIX Premium Agent. 33

34 The data model contains two types of dimensions: the TCR Shared Dimension which are shared across Tivoli by products like Time and agent specific dimensions like CEC Base Agent Dimensions, VIOS Agent Dimensions and AIX Premium Agent Dimensions which are dimensions shared across the System P agents like ManagedSystem, LPAR etc. The facts in the data model are organized under folders by their summarization type like Daily and Hourly. On expanding Daily and Hourly you will be able to see the attribute groups (see figure below). Each attribute group corresponds to a table/view in the data warehouse. Each attribute group contains a bunch of facts or measures like AVG_CPU_Allocated and some identifiers like CPU_Number and Shift and Vacation periods. 34

35 To build a report you need to follow this order for dragging and dropping elements: 1. First drag one of the dimensions from a specific agent like LPAR from CEC Base Agent Dimensions or SystemName from AIX Premium Agent Dimensions. 2. Drag any metric from a related attribute group. Note that dimensions and metrics should be dragged from the same agent. 3. Finally drag and drop the Standard Timestamp from TCR Shared Dimensions.Time Dimensions. For daily data you can use Date instead of Standard Timestamp. You can use any of the other time metrics to slice and dice the data. An example of a report for the AIX Premium Agent is shown below. 35

36 5 IBM Tivoli Monitoring Agents TCR Cognos-based Reports Here is the list of ITM agents that already provided a set of TCR Cognos based reports: ITCAM for Transactions v ITM for Virtual Servers: VMware VI Agent v6.2.2 System P Agents (CEC, VIOS and AIX Premium) v ITM OS Agents v6.2.2 FP02 Each of the above reports needs some Warehouse Data collection to be enabled and Daily and hourly summarization must be enabled for. The following section document the needed attributes groups, however all the details are documented in the specific product documentation. OS Agents Reports attributes groups: Windows agent: o Logical_Disk o Memory o Process o Server o System Linux agent: 36

37 o CPU o Disk o Network o Process o VM_Stats UNIX agent: o Disk o Network o Process o System o Memory System P Agents Reports attributes groups: AIX Premium Agent o KPX_AMS_POOL o KPX_CPU_DETAIL o KPX_CPU_SUMMARY o KPX_ACTIVE_MEMORY_EXPANSION o KPX_DISKS o KPX_FILE_SYSTEMS o KPX_LOGICAL_PARTITION o KPX_LOGICAL_VOLUMES o KPX_NETWORK_ADAPTERS_RATES o KPX_NETWORK_INTERFACES o KPX_PAGING_SPACE o KPX_PHYSICAL_VOLUMES o KPX_VOLUME_GROUPS o KPX_PHYSICAL_MEMORY o KPX_SYSTEM_CALL o KPX_SYSTEM_IO o KPX_PROCESSES_SUMMARY o KPX_WPAR_INFORMATION o KPX_WPAR_NETWORK o KPX_VIRTUAL_MEMORY_MANAGEMENT CEC Base Agent o KPK_AMS_POOLS o KPK_CPU_POOLS o KPK_GLOBAL_CEC o KPK_MON_LPARS o KPK_MON_UNMON_ALLOC o KPK_PER_LPAR HMC Base Agent o No reports VIOS Premium Agent o KVA_AMS_POOL o KVA_CPU_DETAIL o KVA_CPU_SUMMARY o KVA_DISKS KVA_FILE_SYSTEMS o KVA_INTERNET_PROTOCOL_DETAIL o KVA_INTERNET_PROTOCOL_SUMMARY o KVA_LOGICAL_PARTITION o KVA_LOGICAL_VOLUMES o KVA_NETWORK_ADAPTERS_RATES o KVA_NETWORK_MAPPINGS o KVA_PAGING_SPACE 37

38 o o o o o o o KVA_VOLUME_GROUPS KVA_PHYSICAL_MEMORY KVA_SYSTEM_CALL KVA_SYSTEM_IO KVA_STORAGE_MAPPINGS KVA_PROCESSES_SUMMARY KVA_PROCESSES_DETAIL KVA_TCP VMWare Agent Reports attributes groups Windows agent: o Logical_Disk o Memory o Process o Server o System Linux agent: o CPU o Disk o Network o Process o VM_Stats UNIX agent: o Disk o Network o Process o System o Memory ITCAM for Transactions Agent Reports attributes groups For the Application Management Console agent (raw data): o AMC_Application o AMC_Client o AMC_Transaction o AMC_Server o AMC_Internet_Service o AMC_Internet_Service_Agent o AMC_Internet_Service_Element For the Client Response Time agent (raw data, hourly, daily): o CRT_Application_Status o CRT_Transaction_Status For the Web Response Time agent (raw data, hourly, daily): o WRT_Application_Status o WRT_Transaction_Status o WRT_User_Sessions For the Robotic Response Time agent (raw data, hourly, daily): o RRT_Application_Status o RRT_Transaction_Status o RRT_SubTransaction_Status o RRT_Robotic_Playback_Events (raw data only) For the Internet Service Monitoring agent (raw data, hourly, daily): o KIS_HOST_STATISTICS o KIS_MONITOR_STATUS 38

39 o KIS_SERVICE_INSTANCE_STATISTICS o KIS_HTTP o KIS_ICMP For the Transaction Tracking agent (raw data, hourly, daily): o Aggregates o Interactions 6 Known issues There is a known issue when the TDW schema is different from ITMUser where the Cognosbased reports cannot be executed. VMWare agent workaround is documented at the following link: _user129.htm#wq154 System P Agent workaround is documented at the following link: er177.htm#import The OS Agent workaround is to install the following Interim Fix that can be obtained through L2 support opening a pmr: TIV_ITM_OS_AGENTS_REPORTS-LA0001 The APAR number is IZ78074 and will be officially released within ITM FP03 7 Appendix Useful links Tivoli Common Reporting Web Site ew?communityuuid=9caf63c9-15a1-4a03-96b3-8fc700f3a364 Tivoli Common Reporting Support Site ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Common_Reporting IBM Tivoli Monitoring Administration Guide 22fp2_admin.htm 39

40 IBM Tivoli Monitoring Operating System Agents User Guides IBM Tivoli Monitoring Operating System Agents TCR Cognos-based Reports _unix170.htm#wq319 IBM Tivoli Monitoring for Virtual Server (VI VMWare Agent) TCR Cognos-based Reports e622_user127.htm#wq149 IBM Tivoli Monitoring System P Agent TCR Cognos-based Reports 1_user174.htm#wq168 IBM Tivoli Composite Application Manager for Transactions User Guide IBM Tivoli Monitoring support site Acronyms ITM TCR TDW TEMS TEP TEPS WPA SPA IBM Tivoli Monitoring Tivoli Common Reporting Tivoli Data Warehouse Tivoli Enterprise Management Server Tivoli enterprise Portal Tivoli Enterprise Portal Server Warehouse Proxy Agent Summarization Pruning Agent Table 1 - Acronyms 40

41 Trademarks Copyright IBM Corporation 2010 IBM United States of America Produced in the United States of America All Rights Reserved The e-business logo, the eserver logo, IBM, the IBM logo, OS/390, zseries, SecureWay, S/390, Tivoli, DB2, Lotus and WebSphere are trademarks of International Business Machines Corporation in the United States, other countries or both. Lotus, Lotus Discovery Server, Lotus QuickPlace, Lotus Notes, Domino, and Sametime are trademarks of Lotus Development Corporation and/or IBM Corporation. Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries or both. Other company, product and service names may be trademarks or service marks of others. INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PAPER AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. Information in this paper as to the availability of products (including portlets) was believed accurate as of the time of publication. IBM cannot guarantee that identified products (including portlets) will continue to be made available by their suppliers. This information could include technical inaccuracies or typographical errors. Changes may be made periodically to the information herein; these changes may be incorporated in subsequent versions of the paper. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this paper at any time without notice. Any references in this document to non-ibm Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: 41

42 IBM Director of Licensing IBM Corporation 4205 South Miami Boulevard Research Triangle Park, NC U.S.A. 42