Options for Sending z/os Events to Netcool/OMNIbus and TBSM

Transcription

1 Options for Sending z/os Events to Netcool/OMNIbus and TBSM This document can be found on the web at Search for author s name under the category of White Papers. Version 2.0 Date: May 2011 IBM Advanced Technical Skills Mike Bonett Executive I/T Specialist bonett@us.ibm.com

2 Special Notices This document reflects the IBM Advanced Technical Skills organizations understanding of sending z/os information to Netcool /OMNIbus and IBM Tivoli Business Service Manager. It was produced and reviewed by the members of the IBM Advanced Technical Skills organization. This document is presented As-Is and IBM does not assume responsibility for the statements expressed herein. It reflects the opinions of the IBM Advanced Technical Skills. This information could include technical inaccuracies or typographical errors. Changes may be made periodically to the information herein; these changes may be incorporated in subsequent versions of the paper. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this paper at any time without notice. These opinions are based on the authors experiences. If you have questions about the contents of this document, please contact the author at Trademarks The following are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. IBM, the IBM logo, Candle, DB2, developerworks, iseries, Passport Advantage, pseries, Redbooks, Tivoli Enterprise Console, WebSphere, z/os, xseries, zseries. A full list of U.S. trademarks owned by IBM may be found at NetView, Tivoli and TME are registered trademarks and TME Enterprise is a trademark of Tivoli Systems, Inc. in the United States and/or other countries. Microsoft, Windows, Windows NT, Internet Explorer, and the Windows logo are registered trademarks of Microsoft Corporation in the United States and/or other countries. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. UNIX is a registered trademark in the United States and other countries licensed exclusively through The Open Group. Intel and Pentium are registered trademarks and MMX, Pentium II Xeon and Pentium III Xeon are trademarks of Intel Corporation in the United States and/or other countries. Other company, product and service names may be trademarks or service marks of others.

3 Table of Contents Introduction and Acknowledgements... 4 Overview... 5 Overview... 5 How does Netcool/OMNIbus receive s?... 5 How does TBSM receive s?... 6 Which probe integration option(s) should be used?... 7 Tivoli Event Integration Facility (EIF) overview EIF options for z/os Using the Event Pump for z/os to send EIF s Using Tivoli NetView for z/os to send EIF s Using IBM Tivoli Monitoring (ITM) to send EIF s Using the EIF Java classes to send EIF s Using the EIF command line interface to send EIF s EIF probe and ObjectServer customization requirements SNMP traps overview Using Tivoli NetView on z/os to send SNMP traps Using IBM Tivoli Monitoring to send SNMP traps SNMP Probe customization requirements Sending s to the Socket probe Sending s to the probe Sending s to the Generic Log File probe Additional considerations Summary References ( 3

4 Introduction and Acknowledgements This document describes several methods for sending z/os information to Netcool /OMNIbus and Tivoli Business Service Manager 4.x (which requires the Netcool/OMNIbus platform). It gives examples of various ways of capturing z/os information and structuring the information into formats usable by Netcool/OMNIbus and TBSM. The paper assumes some familiarity with: IBM Tivoli Netcool/OMNIbus Tivoli Business Service Manager 4.x IBM Tivoli NetView for z/os IBM Tivoli Monitoring Special thanks to the following people who contributed information to this effort: Becky Anderson, IBM Software Group, Worldwide Sales Bill Davis, IBM Software Group, Tivoli System z Art Eisenhour, IBM Advanced Technical Skills John Irwin, IBM Software Group, Tivoli Beta Programs Blaine Meyer, IBM Software Group, Tivoli Development Tod Thorpe, IBM Software Group, Tivoli Development ( 4

5 Overview IBM Tivoli Netcool/OMNIbus provides enterprise wide management. It can receive s from a variety of sources across hardware, operating system, and network platforms, and apply automated functions to aid in determining root cause problems and to invoke automated procedures. Netcool/OMNIbus is also the real time source used by Tivoli Business Service Manager (TBSM) version 4 and later.. Events that map against TBSM resources which are created via Discovery Library Adapters provided by TBSM - to determine their state, and the resulting impact on business services, are obtained from Netcool/OMNIbus. Various options exist to send z/os information as s to s to Netcool/OMNIbus, and if TBSM is also installed allow state change to flow to TBSM. This paper provides and overview of the technical options available for sending z/os information to Netcool/OMNIbus. It is meant to provide enough information so that an environment can start evaluating which options are best for their requirements. Each method has software licensing requirements that are beyond the scope of this paper; an environment should examine their current software licensing to determine the licensing impact of using any of these options. All references to Netcool/OMNIbus, unless otherwise noted, refer to versions 7.2 and later. All references to TBSM, unless otherwise noted, refer to versions 4.2 and later. How does Netcool/OMNIbus receive s? Netcool/OMNIbus runs on a variety of operating system platforms, including Linux on System z. The following picture provides an overview of the Netcool/OMNIbus infrastructure: ( 5

6 The Netcool OMNIbus ObjectServer is the core component that receives s. It receives s from probes, and exchanges s with other ObjectServers, relational databases, and applications via gateways. A probe connects to an source to detect, acquire, and forward s. Each probe has a set of properties that control the probe configuration, and a set of rules that create ObjectServer s by mapping the source information to fields in the ObjectServer tables. The s received from the probe by Netcool/OMNIbus are viewable through the Web GUI or the provided Event List application. There are many probes available for sending s to Netcool/OMNIbus. The best consolidate source of information on the probes is the IBM Tivoli Netcool/OMNIbus Information Center website. The probes are both specialized (e.g. for a particular vendor resource) and generalized (e.g. for a particular protocol). How does TBSM receive s? TBSM 4.x uses the OMNIbus ObjectServer as its store. It scans incoming s to determine which ones impact components that are part of defined business services. The following picture provides an overview of the TBSM 4.2 architecture: ( 6

7 Details of the architectural components can be found in the TBSM product documentation. The section of the picture surrounded by the dotted line highlights the key interface for TBSM - the IBM Tivoli EIF probe. TBSM provides custom rules for the EIF probe to allow mapping Event Integration Facility (EIF) s to ObjectServer fields and to TBSM resources. TBSM also provides a schema update to the ObjectServer that adds specific tables and fields to support this mapping. TBSM 4.x builds on the existing IBM Tivoli Enterprise Console (TEC) and Netcool integration which is documented in the Tivoli & Netcool Event Flow Integration white paper. The paper, along with the related integration code, is available on the IBM Integration Service Management Library (formerly the Open Process Automation Library website). Which probe integration option(s) should be used? To determine how to send z/os information as s to Netcool/OMNIbus, the following questions must be answered: 1. What probes are most appropriate from receiving information from z/os? Many probes are available to integrate s into the ObjectServer. Since neither the ObjectServer nor the probes run on z/os, the probes must be evaluated to determine which ones provide a means to receive data via a remote z/os connection. A review of the probes reveals the best options to be: Socket z/os sending data via TCP/IP to the socket probe. Log file z/os logging data to a file being monitored by the Generic Log File probe. This is limited to files residing in Unix System Services (USS). EIF z/os sending an EIF to the EIF probe, via TCP/IP. SNMP z//os sending a Simple Network Management Protocol (SNMP) trap to the SNMP probe. z/os sending an to a recipient address being monitored by the probe. Sending data from z/os applications and system functions to any of these probes can be performed using custom programming. However, some are much easier to implement and use than others, especially if existing z/os functions or products already provide and interface to connect to a probe. The answers to the next two questions help determine the best options for a particular environment. 2. What are the possible sources on z/os? z/os components normally surface status and status change information in several structured ways: ( 7

8 SYSLOG messages. These are messages that appear on the operating system console. JOBLOG messages. These are messages that appear on the address space JES2 JOBLOG datasets. Performance monitors. These are normally incidents created from thresholds defined in a system or subsystem performance monitoring product, which can be surfaced (usually to the SYSLOG or an automation product) when the threshold is reached, or an exceed threshold has returned to normal. SNA Alerts. These are notification s in the SNA protocol that would normally be received by a SNA Management program such as NetView for z/os. TCP/IP SNMP traps. These are notification s in the TCP/IP protocol that would normally be received by a SNMP Manager, such as NetView for z/os, or IBM Tivoli Monitoring for Network Performance (ITMNP). Log file. These are s written to a dataset or USS file. SYSLOG and JOBLOG messages are the most likely sources on z/os; however, the desired components may also use one or more of the other options. It is important to understand, for a component to be monitored, the sources it uses. This will better determine the answer to the next question. 3. Which probes can z/os most efficiently surface s to? For each source, both how an is detected, and how that can be sent to the probe must be considered. For z/os there are three implementation options, from most to least desirable: 1. A product or function with the ability to both detect s from the source and interface to one of the probes, with no programming required. Having a product perform this function minimizes or eliminates having to write and maintain custom code. Enabling the product to perform this function is more likely a policy level customization, instead of programming customization. Some programming may be required, but in all cases this should be minimal. The type of z/os products most likely to offer this level of integration include: o Automation products o Performance products 2. A product or function that can surface the so that a custom program can access the information and send it to a probe. The custom program might execute as an extension of the detecting product (such as an automation product with programming capabilities) or it might be a ( 8

9 separate program that can be invoked by the detecting product. The policy level customization is done to detect the, and the custom program takes the, extracts the desired information, and formats it to meet the requirements of the destination probe. 3. Writing a custom program to both surface the and send it to the probe. This can be the least desirable integration option, based on the amount of programming required to capture, format, and send the, as well as maintaining the program as requirements and the environment changes. Capturing the will likely require constant polling of the source. There is more custom code to maintain than the previous options, and thus more effort required to keep things synchronized as the environment changes. One helpful method to narrow the options and identify the most likely sources and probes is to look at the existing management products (primarily automation and performance) and evaluate their functions against the above options. For example, the following matrix uses existing IBM Tivoli z/os management product functions and rates them on the above three options, in the context of the identified sources and target probes. The rows are the z/os source, the columns are the probes, and the numbers correspond to the implementation options (for implementation options 1 and 2 the product which can provide the functions in whole or in part is identified): ( 9

10 SYSLOG JOBLOG 1 (Event Pump for z/os) Performance 1 (IBM Tivoli Monitoring Services) SNA alert SNMP trap EIF Log File SNMP Socket 1 (Event 2 (NetView 2 (NetView 1 (NetView Pump for for z/os for z/os for z/os) z/os, NetView for z/os) 1 (NetView for z/os) 2 (NetView for z/os (IBM Tivoli Monitoring Services 2 (NetView for z/os 2 (NetView for z/os 2 (IBM Tivoli Monitoring Services 2 NetView for z/os 2 (NetView for z/os 1 (IBM Tivoli Monitoring Services) 1 (NetView for z/os) 1 (NetView for z/os Log file (NetView for z/os 2 (IBM Tivoli Monitoring Services 2 (NetView for z/os 2 (NetView for z/os From this quick analysis, several things become clear: Automation products, such as NetView for z/os, usually offer the widest option for surfacing information from the sources. Automation products normally provide the most interfaces into and integration with the sources. In the case of NetView, it has many different ways to both surface information and then connect to multiple probes. Where the automation product does not have a defined interface, it likely includes a programming language where the custom code to interface to a source can be implemented. Performance monitoring products such as IBM Tivoli Monitoring may provide direct generation of s to map to a probe. They may also interface with an automation product via an source (e.g. generating a message when a performance expectation occurs, for an automation product to detect). For performance monitors that can create SYSLOG or JOBLOG messages, the options for those sources apply and should be investigated. The EIF and SNMP probes are the most likely options, as several products have built in support for creating and sending EIF and SNMP s. These probes also provide an initial set of rules to support the sources, which can then be tailored as needed. ( 10

11 The rest of this paper provides basic requirements and usage information for each of the identified integration options. Tivoli Event Integration Facility (EIF) overview The Tivoli Event Integration Facility (EIF) is an interface that applications can use to send or receive notification s. These s are referred to as EIF s or Tivoli Enterprise Console (TEC) s, as that is where the format originated. The EIF is provided with Netcool OMNIbus and the Tivoli Enterprise Console, and is documented in the IBM Tivoli Netcool/OMNIbus Event Integration Facility Reference (SC ). An EIF consists of an Event Class and a list of slots or tokens, which is a set of name-value pairs. For example: EVENT: EIFTBSM41; source=ussjava; sub_source=eif2tbsm41; hostname=hasl125; origin=hasl125; sub_origin=hcb$; status=closed; severity=harmless; msg= IEF403I TCPIP Started ; The Event Class is EIFTBSM41, and the slots/tokens are source, sub_source, origin, sub_origin, status, severity, and msg. Each slot/token has a value associated with it. EIF options for z/os There are several ways to send EIF s from z/os: The Event Pump for z/os product generates EIF s from various sources, including SYSLOG, address space JOBLOGs, subsystems such as CICS, IMS, and DB2, and from data received by a provided API. The NetView for z/os Event/Automation Service provides a Message Adapter. This adapter receives information via the NetView for z/os Program-to-Program interface (PPI) and can use the received information to create and send an EIF. This allows NetView to send s to TEC; since the EIF probe receives s in the same format, it can also be used to send s to the EIF Probe. IBM Tivoli Monitoring can generate EIF s from triggered situations. The EIF is sent from the Hub Tivoli Enterprise Monitoring Server (TEMS), which can be on z/os, Linux on System z, or a supported distributed platform. EIF provides a set of Java classes that can be installed in the z/os Unix Systems Services (USS) environment. These classes allow Java programs that execute in USS to create and send (or receive) EIF s. Any z/os program that can call a Java program can potentially send information to be transformed into an ( 11

12 EIF. If the z/os application cannot directly invoke a Java program, it may still be able to send data if it has a socket interface. The Java classes can be used within a TCP/IP Socket server program to receive information, format the data into an EIF, and send it to the EIF probe. EIF provides the posteifmsg command (in earlier releases called postzmsg), which can be installed in the USS environment. Just as with the EIF Java classes, any program that can call a USS command (or USS shell script) can invoke the posteifmsg command. Also, as described for the Java classes, the posteifmsg can be used within a USS TCP/IP Socket server program. Given all of the available options to generate EIF s from z/os, deciding which one to use may seem like a challenge. However, positioning the options is fairly easy based upon the overall objectives for sending s, and the skills and resources available to implement and maintain these functions. The Event Pump for z/os is almost always the best option to use. It provides the following advantage over the other options: Defining the resources to monitor and the s to capture are done via policy specifications, not by writing program code. The major task is identifying the address spaces and z/os messages to be monitored. NetView for z/os definitions and code are provided to more easily capture information not available via SYSLOG messages from specific subsystems and products. Currently information from IMS, DB2 Universal Database for z/os, DB2 Performance Monitor, DB2 Performance Expert, and System Automation for z/os is captured by the Event Pump through NetView. An External Data Interface (EDI) is provided which can be used to map information to a standard format that can be filtered by the Event Pump and sent to OMNIbus without requiring custom mapping rules for OMNIbus and the Tivoli EIF Probe. When using TBSM the Event Pump is especially attractive, since it provides the EIF probe rules to map z/os information to the TBSM fields in the ObjectServer database. In fact, the TBSM on z/os product consists of the Event Pump and the Discovery Library Adapter for z/os. The Event Pump can register the address spaces that have been discovered by the z/os Discovery Library adapter and are used in TBSM services. This registration automatically generates the EIF s that will map to the objects in TBSM to correctly reflect the address space status. These advantages can make the Event Pump a better candidate for initial use than the other options. IBM Tivoli Monitoring is the best option if s based on performance and availability information from any ITM agents on z/os (including OMEGAMON, ITCAM, or product provided agents) is desired. If TBSM is installed, s from both the Event Pump and ITM that are from the same z/os resource will map to the single TBSM object that represents that resource. ( 12

13 The NetView for z/os Event/Automation Service can also generate EIF s from policy definitions without requiring any programming. However, some programming will be required if there are specific mapping requirements for example, if the information captured by NetView must be mapped against a TBSM resource. Both the Java classes and posteifmsg methods require custom coding on z/os and corresponding custom rules and mappings in OMNIbus, which then have to be maintained once developed. In summary, if existing z/os integration with OMNIbus or TBSM is not in place, the z/os Event Pump should be the first option considered. If another option is being used, the functions of the z/os Event Pump should be compared to the current option implementation, to determine if the Event Pump can be used to achieve the same or better results in a more efficient manner. Using the Event Pump for z/os to send EIF s The IBM Tivoli Event Pump for z/os is specifically designed to capture z/os s and create EIF s. It is the best option when using EIF s since no programming is required, and extensive functions are provided to capture information and send it to Netcool/OMNIbus. The white paper Sending z/os Events to IBM Tivoli Netcool/OMNIbus using the IBM Tivoli Event Pump for z/os, available on the IBM TechDocs website, provides a detailed walkthrough of setting up the Event Pump to send z/os s to the ObjectServer. Using Tivoli NetView for z/os to send EIF s The Tivoli NetView Event/Automation Service (NetView EAS) allows NetView to create and send EIF s to an EIF recipient (TEC, the EIF Probe, another NetView Event/Automation Service, or a custom written EIF receiver). The NetView EAS contains many functions: Create and send TEC/EIF s Receive EIF s and convert to SNA Alerts Receive SNMP traps and convert to SNA Alerts Send SNA alerts as SNMP traps Details on enabling the NetView EAS components can be found in IBM Tivoli NetView for z/os Installation: Configuring Additional Components (SC ). Using IBM Tivoli Monitoring (ITM) to send EIF s ITM Tivoli Monitoring can send EIF s in three ways: ( 13

14 1. A situation can be customized to send an EIF when it becomes true. ITM will also send a clearing EIF for sampled situations when the situation is no longer true. The ITM/OMNIbus integration also allows an that is cleared on OMNIbus (such as an EIF from a pure situation) to then be cleared on ITM. 2. An agent (in ITM fix pack 1 and later) can be customized to directly send EIF s to an EIF receiver, without defining a situation. These are called private situations, and can be useful at times when a notification must be sent and the agent is running but not connected to a TEMS. Files on the agent have to be customized to enable this integration. 3. A situation can, in its reflex automation, invoke a program that implements the EIF API (such as a Java program), or the EIF posteifmsg command line interface. This will require custom programming to create and send the. Details on the use of these options can be found in the IBM Tivoli Monitoring Administration Guide. Using the EIF Java classes to send EIF s The EIF provides an application programming interface (API) for creating, sending, and receiving s. Details of the API are provided in the IBM Tivoli Netcool/OMNIbus Event Integration Facility Reference. The language support includes Java; a set of Java classes is provided, which can be used in the USS environment: The main Java objects provided are: TECAgent object class that defines a sender or receiver for s. When a TECAgent object is created, one of the parameters is a file of configuration information. This file, just like the NetView EAS Message Adapter configuration file, contains the ServerPort and ServerLocation parameters the agent will use to connect to the EIF Probe. TECEvent object class for creating EIF s The following Java code fragment provides an example of how these objects are used: ( 14

15 //Define a TECAgent sender object. The connectivity parameters //are read from a file named eif2tbsm41.conf File conf = new File("eif2tbsm41.conf"); Reader cfgin = new FileReader(conf); TECAgent agent = new TECAgent(cfgin, TECAgent.SENDER_MODE, false); //Create a TECEvent object and set its class TECEvent = new TECEvent();.setClassName(class); //Set the tokens for the.setslot("source", "USSJAVA");.setSlot("msg",msg);.setSlot("origin",origin);.setSlot("sub_source","eif2tbsm41");.setSlot("sub_origin",smf_id);.setSlot("severity",severity);.setSlot("hostname",hostname); //Send the TECEvent via the TECAgent int rc = agent.sendevent(.tostring(true)); Any z/os application that can invoke a USS program can invoke a Java program that uses the EIF classes, and can pass information to the program to build the EIF. Another option is to run a Java program that implements the EIF functions as a standalone server (batch job or started task), and have other z/os applications send information to it via TCP/IP sockets, which the Java server can receive and convert to EIF s. Using the EIF command line interface to send EIF s The process for using the EIF command line interface is similar to that for using the EIF Java classes. The main difference is that instead of writing a Java application, the EIF posteifmsg command is used. This command is provided by the EIF in Netcool/OMNIbus 7.3 and later for the USS environment, and can be used directly, within a shell script, or called from another USS program to create and send the EIF. The parameters passed to the posteifmsg command are the name of a configuration file with connectivity information, the class name, and the token names and their values. The posteifmsg syntax is documented in the IBM Tivoli Netcool/OMNIbus Event Integration Facility Reference. For Netcool/OMNIbus versions prior to 7.3 the postzmsg command, part of the EIF toolkit shipped with Tivoli Enterprise Console but available for use by Netcool /OMNIbus users, can be used. The command syntax is the same as posteifmsg. ( 15

16 EIF probe and ObjectServer customization requirements The EIF probe rules file implements a default mapping of the EIF contents to the fields in the ObjectServer alerts.status table. ITM, TBSM, and the Event Pump for z/os provide additional rules and ObjectServer schema updates for use with s specific to their environments. TBSM adds several fields to the ObjectServer schema. Among the updates is the BSM_Identity field, which is added to the alerts.status table. TBSM uses this field to identify which s (and associated status) map against specific TBSM objects. Rules for the EIF probe are provided to map z/os information to BSM_Identity field. The TBSM discovery mechanism defines one or more BSM_Identity values for each discovered resource. For an status to be properly processed for a TBSM resource, the ObjectServer BSM_Identity field value must match an existing TBSM resource BSM_Identity value. Events coming from other probes can also be mapped to TBSM resources, but this requires custom probe rules to be created for the respective probe to map the probe contents to the fields required by TBSM. With TBSM, the EIF probe, and products or functions that can create EIF s, are the best option to use. The Identifier field determines the uniqueness of an. It essentially acts as the unique key field for the alerts.status table. Events with the same Identifier value go through a deduplication process where they are stored as a single row in the alerts the status table, and processing is applied to ensure alert information is not duplicated. Details of deduplication can be found in the Netcool/OMNIbus Administrators Guide. SNMP traps overview Simple Network Management Protocol (SNMP) traps are the method for TCP/IP resources to send notifications of significant s. There are multiple versions of trap types (v1, v2c, v3), generally speaking traps consist of specific field to identify the trap type (general type 0-6 and, for trap type 6, specific trap code set by the source), and a variable binding list a list of name-value pairs with data specific to the trap. Using Tivoli NetView for z/os to send SNMP traps Tivoli NetView for z/os provides two options for sending SNMP traps: 1. The NetView Event/Automation Service provides an Alert-to-Trap adapter. Any SNA alert can be converted to an SNMP trap (via the NetView NPDA TRAPROUT filter) and forwarded. Since NetView provides a GENALERT command, any data captured from or created by NetView automation can be transformed into an alert using GENALERT, and then forwarded to the Alert-to-Trap adapter. The adapter ( 16

17 formats the alert data (based on a default or customized mapping file) to a SNMP trap and sends it to the z/os Communications Server OSNMPD SNMP agent address space, to be forwarded to the target destination(s). 2. The NetView SNMP TRAP command can directly create and forward traps. This method has fewer NetView components involved than the Event/Automation Service, but requires more custom programming (primarily defining automation table entries for desired s and/or REXX programming to extract the desired data and issue the command). The full syntax of the command is documented in the NetView Command Reference manual. The command allows the following parameters (among others) be defined: The trap version (and require parameters based on the trap version) The destination host (and port) to receive the trap The trap type (usually generic type 6 with a specific type) The oid, type and value of the variables sent with the trap Using IBM Tivoli Monitoring to send SNMP traps ITM Tivoli Monitoring can send SNMP traps in three ways: 1. A workflow policy situation can be customized to send an SNMP. ITM provides a SNMP emitter than can be invoked through policy workflows. The IBM technote Create a workflow policy to emit an SNMP trap after a situation fires, available on the IBM Support Website, provides a detailed walk through of how to use this capability. 2. An agent (in ITM and later) can be customized to directly send SNMP traps, without defining a situation. These are called private situations, and can be useful at times when a notification must be sent and the agent is running but not connected to a TEMS. A set of files must be configured on the agent platform to enable this capability; further documentation can be found in 3. A situation can, in its reflex automation, invoke a program that can create and send a SNMP trap from the execution platform command line interface. The invoked custom program can create a SNMP trap based on information passed from the situation. However, this will require ongoing maintenance based on the number of different situation for which this is desired and the potential number of different platforms where the custom program(s) will have to be maintained. SNMP Probe customization requirements The SNMP probe is installed on the same system with the Netcool/OMNIbus ObjectServer. It runs as a process on UNIX/Linux and can run as a service on Windows. Details on the installation and configuration of the SNMP Probe are documented in the IBM Tivoli Netcool/OMNIbus SNMP Probe manual (SC ). ( 17

18 The default SNMP probe rules create a number of variables from the trap contents, but on a small subset of them by default are mapped to ObjectServer fields. Custom probe rules have to be created to examine these variables and take actions or set ObjectServer field mappings based on their content - for example, the Severity (or clearing) could be set based on the type of trap being received. Sending s to the Socket probe Sending s from z/os to Netcool OMNIbus via the Socket probe requires custom programming on z/os to send the information, and customizing the Socket probe properties and rules to receive and parse the information for mapping into ObjectServer fields. This method is most applicable where there is already an application sending information via TCP/IP sockets; the main requirement is then understanding the data format, and customizing the Socket probe to properly receive, parse, and map the desired information into the ObjectServer. Sending s to the probe Sending s from z/os to Netcool OMNIbus via the probe requires custom programming on z/os (and possibly enabling the z/os Communications Server mail facilities) to send the information, and customizing the Socket probe properties and rules to receive and parse the information for mapping into ObjectServer fields. This method is most applicable where there is already an application sending s. The main requirement is then understanding the data format, and customizing the probe to properly receive, parse, and map the desired information into the ObjectServer. Sending s to the Generic Log File probe Sending s from z/os to Netcool OMNIbus via the Generic Log File probe is possible under the following conditions: The z/os function must write information to a file on Unix System Services. The Generic Log File probe will only work with these types of files on z/os. The file being monitored must be accessible by the probe. The probe cannot be installed directly on z/os, so the monitored file must be shared with the system where the probe runs. For this to occur one the following configurations is required: 1. The monitored file resides on a z/os USS file system that is shared (via NFS or SMB sharing) with the probe system. 2. The monitored file resides on the probe system, in a file system that is accessible from z/os (via NFS or SMB sharing). Using the Generic Log File probe requires custom programming on z/os to write the information to the monitored file, and customizing the probe properties and rules to receive ( 18

19 and parse the information for mapping into ObjectServer fields. This method is most applicable for applications that logging information to USS files. For example, the IBM HTTP Server on z/os log files could be monitored in this way. The main requirement is then understanding the log file data format, and customizing the Generic Log File probe to properly receive, parse, and map the desired information into the ObjectServer. Additional considerations The above information illustrates that it is very possible, and not difficult, to send z/os information for use in Netcool/OMNIbus and TBSM. Additional factors come into account when considering the use of any of these methods: The best method to use, in addition to the sources and product capabilities outlined above, also on the products in use and the skills available for those products. The subset of s from the desired sources (messages, SNA alerts, etc.) must be clearly identified. z/os generates many s, and the vast majority do not need to be forwarded to Netcool/OMNIbus. Only those s that indicate state changes impacting key resources should be used. This will have to be analyzed for each type of z/os subsystem that produces these s. For the subset of s that will be forwarded, the rate at which they might occur has to be understood to determine any potential performance impact to the Netcool/OMNIbus and/or TBSM environment. Any chosen to be sent should also have an associated clearing that can be sent an that indicates the problem is resolved. Otherwise additional manual work is introduced to determine the current status and manually clear the s. If the source does not directly generate a clearing, automation products may be able to, when the occurs, invoke monitoring to check the situation status, and create a clearing when the status has returned to normal. Existing automation products being used on z/os are useful in identifying the critical z/os s to send to Netcool/OMNIbus. Automation may already be in place to detect or control startup/shutdown/problem situations for z/os address spaces based on certain messages. If so, then these messages are also the ones to consider for forwarding to Netcool/OMNIbus for use by TBSM, to properly reflect the address space status. Summary With forethought and planning, it is very feasible to integrate s from z/os into Netcool/OMNIbus and TBSM. Testing of this capability can be started using any method desired. Production usage will depend on specific identification of a subset of s that are deemed critical enough for usage in Netcool OMNIbus, or TBSM 4.x. Varying levels of customization is required, across multiple components, but the number of available options provides flexibility for designing an flow that efficiently meets specific requirements. ( 19

20 References The following product manuals are all available online at the IBM Tivoli Software Information Center ( ), under the respective software product: Exploring Tivoli Business Service Manager (GI ) IBM Tivoli Netcool/OMNIbus Event Integration Facility Reference (SC ) IBM Tivoli Netcool/OMNIbus Administration Guide (SC ) IBM Tivoli Discovery Library Adapter for z/os (SC ) IBM Tivoli NetView for z/os Installation: Configuration Additional Components (SC ) IBM Tivoli NetView for z/os Customization Guide (SC ) IBM Tivoli Monitoring Installation and Setup Guide (GC ) IBM Tivoli Netcool/OMNIbus Probe for Tivoli EIF (SC ) IBM Tivoli Netcool/OMNIbus SNMP Probe (SC ) The following document is available from the IBM Integrated Service Management Library at Tivoli and Netcool Event Flow Integration The following document is available from the IBM TechDocs website: Sending z/os Events to IBM Tivoli Netcool/OMNIbus using the IBM Tivoli Event Pump for z/os The following document is available from the IBM Support website: Create a workflow policy to emit an SNMP trap after a situation fires ( 20