IBM. Installing, configuring, using, and troubleshooting. IBM Operations Analytics for z Systems. Version 3 Release 1

Transcription

1 IBM Operations Analytics for z Systems IBM Installing, configuring, using, and troubleshooting Version 3 Release 1

2

3 IBM Operations Analytics for z Systems IBM Installing, configuring, using, and troubleshooting Version 3 Release 1

4 ii Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

5 Contents What's new in Version Conentions used in this documentation 3 Operations Analytics for z Systems oeriew z/os Insight Packs IBM Operations Analytics - Log Analysis V Log Analysis extensions for z/os Problem Insights and client-side Expert Adice IBM zaware V3.1 feature IBM zaware data gatherer IBM Common Data Proider for z Systems V Logstash and the ioaz Logstash output plugin.. 12 Installation and configuration checklists Planning for installation and configuration of Operations Analytics for z Systems System requirements Planning for installation of Log Analysis Planning for Log Analysis to support LDAP interaction with RACF for sign-on authentication.. 17 Planning for configuration of IBM Common Data Proider for z Systems Correlation of the data to be analyzed with the associated data stream types, data source types, and dashboards Determination of time zone information for z/os log records Requirements for enabling the generation of SMF data at the source Requirements for configuring SMF data streams 26 Requirements for gathering WebSphere Application Serer for z/os log data Planning for configuration of Logstash Creation of data sources Data source types Installing Operations Analytics for z Systems Installing Log Analysis Installing the z/os Insight Packs, extensions, and IBM zaware data gatherer Installing Logstash and the ioaz Logstash output plugin Uninstalling the Insight Packs Remoing data from Log Analysis Uninstalling Log Analysis Upgrading Operations Analytics for z Systems Upgrading to Log Analysis V Upgrading from Log Analysis V1.3.2 to V Upgrading from Log Analysis V to V Upgrading the z/os Insight Packs and extensions 47 Replacing the scala Logstash output plugin with the ioaz Logstash output plugin Migrating to IBM Common Data Proider for z Systems for streaming z/os operational data Upgrading to the new z/os Log Forwarder and the System Data Engine Migrating to use IBM Common Data Proider for z Systems for gathering data Migrating to route SMF data directly from the System Data Engine to the Data Streamer Configuring Operations Analytics for z Systems Configuring the IBM zaware data gatherer Configuration reference for the IBM zaware data gatherer Configuring Logstash Verifying the identity of the Log Analysis serer 61 Updating the Logstash configuration with the user password for the Log Analysis serer Securing communication between IBM Common Data Proider for z Systems and Logstash Preparing to analyze z/os log data.. 65 Logging in to Log Analysis Use of cookies in the Log Analysis UI Grouping data sources to optimize troubleshooting in your IT enironment Extending troubleshooting capability with Custom Search Dashboard applications Customizing the dashboard applications Running the dashboard applications Getting started with the IBM zaware data gatherer 72 Getting started with Problem Insights for z/os.. 74 Getting started with client-side Expert Adice Troubleshooting Operations Analytics for z Systems Log files Enabling tracing for the ioaz Logstash output plugin APPLID alues for CICS Transaction Serer might not be correct in Log Analysis user interface DB2 or MQ command prefix alues might not be correct in Log Analysis user interface Log record skipped and not aailable in Log Analysis z/os log data is missing in Log Analysis search results After upgrade, interal anomaly data is not isible in Log Analysis user interface Copyright IBM Corp. 2014, 2016 iii

6 Reference Configuration options for the ioaz Logstash output plugin WebSphere Application Serer for z/os data source types zos-was-sysout data source type zos-was-sysprint data source type zos-was-hpel data source type z/os Network data source types zos-netview data source type z/os SMF data source types zos-smf30-annotate annotations zos-smf80-annotate annotations zos-smf110_e-annotate annotations zos-smf110_s_10-annotate annotations zos-smf120-annotate annotations z/os SYSLOG data source types zos-syslog-console data source type zos-syslog-sdsf data source type zos-syslogd data source type zos-cics-msgusr data source type: three ariations zos-cics-eyulog data source type: three ariations zos-cics-annotate annotations zos-anomaly-interal data source type Dashboards WebSphere Application Serer for z/os Custom Search Dashboard applications z/os Network Custom Search Dashboard applications z/os SMF Custom Search Dashboard applications z/os SYSLOG Custom Search Dashboard applications Sample searches WebSphere Application Serer for z/os sample searches z/os Network sample searches z/os SMF sample searches z/os SYSLOG sample searches SMF type 80-related records that the System Data Engine creates SMF80_COMMAND record type SMF80_LOGON record type SMF80_OMVS_RES record types SMF80_OMVS_SEC record types SMF80_OPERATION record type SMF80_RESOURCE record type Notices Terms and conditions for product documentation 152 Trademarks i Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

7 What's new in Version Reiew the new features that are aailable in Version of IBM Operations Analytics for z Systems to understand the potential benefits to your z/os-based IT operations enironment. V3.1.0 features Table 1 lists the new features and indicates where you can find more information. Table 1. New features in IBM Operations Analytics for z Systems V3.1.0 Feature description IBM Common Data Proider for z Systems replaces the z/os Log Forwarder. It can automatically monitor and forward z/os data to Logstash. IBM Operations Analytics for z Systems proides a copy of Logstash and an ioaz Logstash output plugin that forwards data to the IBM Operations Analytics - Log Analysis serer for processing by the z/os Insight Packs. IBM Operations Analytics for z Systems includes the separately installable IBM z Adanced Workload Analysis Reporter (IBM zaware) feature. Updates to the Problem Insights UI: The Problem Insights and Suggested Actions table has an Interal Score column that shows the zaware anomaly score for the interal in which the problem occurred. You can click a score to naigate in context to IBM zaware for more information about the interal. If you hoer oer a number in the Count column of the Problem Insights and Suggested Actions table, a pop-up window shows more information about each occurrence of the problem, including the interal score. You can click the score to naigate in context to IBM zaware for more information about that interal. A bar chart for each system within the selected sysplex shows the interal scores for the last 24 hours. If you hoer oer an indiidual bar in a bar chart, a pop-up window shows the start and stop time for the interal, the total number of unique message IDs for the interal, and the interal score. You can click an indiidual bar to naigate in context to IBM zaware for more information about the interal. More information Operations Analytics for z Systems oeriew on page 5 Planning for installation and configuration of Operations Analytics for z Systems on page 15 Installing Operations Analytics for z Systems on page 33 Upgrading Operations Analytics for z Systems on page 43 Configuring Operations Analytics for z Systems on page 57 Configuration options for the ioaz Logstash output plugin on page 85 Log files on page 77 Enabling tracing for the ioaz Logstash output plugin on page 78 Operations Analytics for z Systems oeriew on page 5 IBM zaware V3.1 feature on page 11 Log Analysis extensions for z/os Problem Insights and client-side Expert Adice on page 9 Installing the z/os Insight Packs, extensions, and IBM zaware data gatherer on page 35 Getting started with Problem Insights for z/os on page 74 Copyright IBM Corp. 2014,

8 Table 1. New features in IBM Operations Analytics for z Systems V3.1.0 (continued) Feature description More information Support for gathering interal anomaly data from IBM zaware Operations Analytics for z Systems oeriew on page 5 IBM zaware V3.1 feature on page 11 IBM zaware data gatherer on page 12 Installing the z/os Insight Packs, extensions, and IBM zaware data gatherer on page 35 Configuring the IBM zaware data gatherer on page 57 Configuration reference for the IBM zaware data gatherer on page 58 Getting started with the IBM zaware data gatherer on page 72 After upgrade, interal anomaly data is not isible in Log Analysis user interface on page 82 z/os SYSLOG data source types on page 111 zos-anomaly-interal data source type on page 124 zos-anomaly-interal-annotate annotations on page Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

9 Conentions used in this documentation This information describes conentions that are used in the IBM Operations Analytics for z Systems documentation. LA_INSTALL_DIR LA_INSTALL_DIR is the directory where IBM Operations Analytics - Log Analysis is installed. The default location for installing Log Analysis is the /home/user/ibm/ LogAnalysis directory, where user represents the user ID of the non-root user who installs Log Analysis. LOGSTASH_INSTALL_DIR LOGSTASH_INSTALL_DIR is the directory where Logstash is installed. Copyright IBM Corp. 2014,

10 4 Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

11 Operations Analytics for z Systems oeriew IBM Operations Analytics for z Systems extends the capabilities of IBM Operations Analytics - Log Analysis to help you more quickly identify, isolate, and resole problems in a z/os-based IT operations enironment. IBM Operations Analytics for z Systems proides the following capabilities: Capability to gather z/os logs across the System z enterprise and to forward them to the IBM Operations Analytics - Log Analysis serer for analysis Capability to index, search, and analyze application, middleware, and infrastructure log data across the System z enterprise Capability to quickly search and isualize errors across thousands of log records Expert adice that is based on linking search results to aailable troubleshooting information, such as best practices and preiously documented solutions Continuous streaming of z/os logs IBM products that are closely related to IBM Operations Analytics for z Systems IBM Operations Analytics for z Systems is closely related to the following products: IBM Operations Analytics - Log Analysis IBM Operations Analytics - Log Analysis is a cross-platform solution for identifying and resoling problems in an IT operations enironment. In IBM Operations Analytics - Log Analysis, an Insight Pack is software that extends the capabilities of IBM Operations Analytics - Log Analysis to proide support for loading and analyzing data from sources that share common characteristics. Examples include log sources for a specific operating system or for a specific application, such as IBM WebSphere Application Serer. IBM Operations Analytics for z Systems includes z/os Insight Packs that extend IBM Operations Analytics - Log Analysis function to analyze the arious types of z/os log data. IBM Operations Analytics for z Systems also includes a ersion of IBM Operations Analytics - Log Analysis. IBM Common Data Proider for z Systems IBM Common Data Proider for z Systems is a single data proider for sources of both structured and unstructured data. It proides a near real-time data feed of z/os log data and System Management Facilities (SMF) data that can be used by IBM Operations Analytics for z Systems and other software. IBM Operations Analytics for z Systems includes a ersion of IBM Common Data Proider for z Systems. IBM z Adanced Workload Analysis Reporter (IBM zaware) IBM zaware monitors software and models normal system behaior to detect and diagnose anomalies in z/os and Linux on z Systems enironments. It creates a model of normal system behaior that is based on prior system data, and it uses pattern recognition techniques to identify Copyright IBM Corp. 2014,

12 unexpected messages from the systems that it is monitoring. This analysis of eents proides near real-time detection of anomalies that you can easily iew through a GUI, which you can also use to determine the cause of past or current anomalies. This early detection can help IT personnel correct problems before the problems affect system processing. IBM Operations Analytics for z Systems includes the separately installable IBM zaware feature. Flow of source data Figure 1 illustrates the flow of data among the following primary components of IBM Operations Analytics for z Systems: z/os Insight Packs on page 7 IBM Operations Analytics - Log Analysis V1.3.3 on page 8 IBM Common Data Proider for z Systems V1.1.0 on page 12 z/os LPAR 1 Log Log Common Data Proider for z Systems Log Analysis serer 3 User Interface Log 1 z/os LPAR 2 Generic Receier 2 z/os Insight Packs Log Log Common Data Proider for z Systems Log Figure 1. Flow of source data among IBM Operations Analytics for z Systems components The following steps describe the data flow among components, which is indicated by arrows in the illustration: 1. In each z/os logical partition (LPAR), the IBM Common Data Proider for z Systems retriees the data from the respectie source (log or SMF record) and forwards it to the IBM Operations Analytics - Log Analysis serer. The IBM Common Data Proider for z Systems sends data to the IBM Operations Analytics - Log Analysis serer. 2. The source data is processed by the respectie z/os Insight Pack. Insight is proided for data from the following source types: z/os system log (SYSLOG) CICS Transaction Serer for z/os EYULOG or MSGUSR log data Network data, such as data from UNIX System Serices system log (syslogd) or z/os Communications Serer 6 Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

13 z/os Insight Packs NetView for z/os message data SMF data WebSphere Application Serer for z/os logs that include SYSOUT, SYSPRINT, or HPEL log data 3. Users can see isualizations of the analyzed data in the Log Analysis user interface. Insight is also proided for IBM zaware interal anomaly data. The flow of interal anomaly data includes the following steps: 1. The IBM zaware data gatherer of IBM Operations Analytics for z Systems resides on the same distributed system as the Log Analysis serer. It retriees interal anomaly data from the IBM zaware Representational State Transfer (REST) API and proides it to the Log Analysis serer. 2. The source data is processed by the z/os SYSLOG Insight Pack. 3. Users can see isualizations of the analyzed data in the Log Analysis user interface. Each z/os Insight Pack in IBM Operations Analytics for z Systems extends IBM Operations Analytics - Log Analysis function to analyze a unique type of z/os log data. The following z/os Insight Packs are aailable: z/os SYSLOG Insight Pack This Insight Pack enables Log Analysis to ingest, and perform searches against, data that is retrieed from the following sources: z/os SYSLOG console, which includes data from the following software: IBM CICS Transaction Serer for z/os IBM DB2 for z/os IBM IMS for z/os IBM MQ for z/os IBM Resource Access Control Facility (RACF ) z/os Communications Serer CICS Transaction Serer for z/os EYULOG and MSGUSR log UNIX System Serices system log (syslogd) IBM z Adanced Workload Analysis Reporter (IBM zaware) Summary: Install this Insight Pack if you want to analyze any of the following data: z/os SYSLOG console data CICS Transaction Serer for z/os EYULOG or MSGUSR log data UNIX System Serices system log (syslogd) IBM zaware interal anomaly data SMF data z/os Network Insight Pack This Insight Pack enables Log Analysis to perform searches against z/os network data. It also enables Log Analysis to ingest, and perform searches against, IBM Tioli NetView for z/os message data. Operations Analytics for z Systems oeriew 7

14 To analyze z/os network data, you must use both the z/os Network Insight Pack and the z/os SYSLOG Insight Pack. The z/os SYSLOG Insight Pack enables Log Analysis to ingest z/os network data. Summary: Install this Insight Pack if you want to analyze any of the following data: Network data, such as data from UNIX System Serices system log (syslogd) or z/os Communications Serer NetView for z/os message data z/os SMF Insight Pack This Insight Pack enables Log Analysis to ingest, and perform searches against, data that is retrieed from IBM z/os System Management Facilities (SMF). Summary: Install this Insight Pack if you want to analyze SMF data. WebSphere Application Serer for z/os Insight Pack This Insight Pack enables Log Analysis to ingest, and perform searches against, logging data that is retrieed from IBM WebSphere Application Serer for z/os. Summary: Install this Insight Pack if you want to analyze WebSphere Application Serer for z/os log data. Table 2 lists each z/os Insight Pack with its associated installation package file. Table 2. Installation package files for z/os Insight Packs Insight Pack WebSphere Application Serer for z/os Insight Pack z/os Network Insight Pack Important: To analyze network data, install both the z/os SYSLOG Insight Pack and this insight pack. z/os SMF Insight Pack Important: To analyze SMF data, install both the z/os SYSLOG Insight Pack and this insight pack. z/os SYSLOG Insight Pack Installation package WASforzOSInsightPack_ zip zosnetworkinsightpack_ zip SMFforzOSInsightPack_ zip SYSLOGforzOSInsightPack_ zip IBM Operations Analytics - Log Analysis V1.3.3 IBM Operations Analytics - Log Analysis is used with the z/os Insight Packs to proide support for analyzing z/os log data. IBM Operations Analytics - Log Analysis includes the following functions that you can configure in the Log Analysis UI: Role-based access control You can create and modify users and roles to assign role-based access control to indiidual users. For more information, see Users and roles in the Log Analysis documentation. Alerting You can create alerts that are based on eents, and you can define the 8 Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

15 actions for the system to take when an alert is triggered. For example, you can define an action to send an notification to one or more people when an alert is triggered. For more information, see Managing alerts in the Log Analysis documentation. Some components that are preinstalled with Log Analysis Although many components are preinstalled with IBM Operations Analytics - Log Analysis, the following components might be especially useful with your z/os Insight Packs: Expert adice custom search dashboard The Expert Adice proides links to contextually releant information to help you resole problems quickly. Using this application, you can select any column or cells in the Grid iew and can launch a search of the IBM Support Portal. The application searches for matches to unique terms that are contained in the column that you select. You can start the Expert Adice application by clicking Search Dashboards in the naigation pane of the Search workspace. For more information about this component, see Custom Search Dashboards in the Log Analysis documentation. Tip: To use this Expert Adice, the Log Analysis serer must hae access to the Internet. With the client-side Expert Adice extension that is proided by IBM Operations Analytics for z Systems, you can access expert adice een if the Log Analysis serer does not hae access to the Internet. For more information about the client-side Expert Adice, see Log Analysis extensions for z/os Problem Insights and client-side Expert Adice. WebSphere Application Serer Insight Pack This Insight Pack is different from the WebSphere Application Serer for z/os Insight Pack. It is intended for use with WebSphere Application Serer on distributed platforms. It does not support natie logging formats for WebSphere Application Serer for z/os. Howeer, if you configure your WebSphere Application Serer for z/os enironment to use a distributed logging format, this Insight Pack proides you with the required annotation and indexing capabilities. For more information about this component, see WebSphere Application Serer Insight Pack in the Log Analysis documentation. Log Analysis extensions for z/os Problem Insights and client-side Expert Adice IBM Operations Analytics for z Systems proides extensions to IBM Operations Analytics - Log Analysis for z/os Problem Insights and client-side Expert Adice. To use these extensions, you must hae Log Analysis Version Fix Pack 1 installed. Problem Insights The Problem Insights extension proides near real-time insight into problems in your IT enironment, with suggested actions to help resole the problems. Operations Analytics for z Systems oeriew 9

16 If the Problem Insights component of IBM Operations Analytics for z Systems is installed, the Log Analysis UI includes a new tab that is titled Problem Insights. For each sysplex from which data is being forwarded to the Log Analysis serer, the Problem Insights page includes insight about certain problems that are identified in the ingested data. You can select the time range for which you want to see insights. For example, if you want to know about certain problems that were identified in the last hour, you can select the last hour as the time range. The default time range is the last 15 minutes. You can also reload the Problem Insights page and associated data by clicking the Refresh button that is located in the area of the UI where you select the time range. Each sysplex in the monitored enironment is represented by a button that indicates the number of problems that are found in that sysplex. You can toggle the showing or hiding of data for a sysplex by clicking the button for the respectie sysplex. The Problem Insights and Suggested Actions table includes the following information about each problem that is discoered: Seerity The indication of the seerity of the problem. Sysplex The sysplex where the problem occurred. System The system where the problem occurred. Interal score Indicates unusual patterns of message IDs within an analysis interal, in comparison to the model of normal system behaior for the respectie system. This score is extracted from the IBM zaware interal anomaly score. A score of 99.5 or higher indicates an anomaly. You can click a score to naigate in context to IBM zaware for more information about the interal. Subsystem The subsystem or system resources manager where the problem occurred. Time The last time that the problem occurred in the selected time range. For example, if you select the last 15 minutes as the time range, this column shows the last time that the problem occurred in the last 15 minutes. Problem Summary A summary of the problem that proides insight into the cause. Count The total number of occurrences of the problem in the selected time range. If you hoer oer this number, a pop-up window shows the following information about each occurrence: Time Seerity Sysplex System 10 Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

17 Interal score. You can click a score to naigate in context to IBM zaware for more information about that interal. Suggested Actions Click a question mark (?) to go to more insight about the problem, including actions that you can take to resole the problem. The Suggested Actions information also includes links to other sources of information that might help you resole the problem, such as releant topics in the IBM Knowledge Center. Eidence The message number of the message that identifies the problem, which you can click to iew more information. After the Problem Insights and Suggested Actions table, a bar chart for each system within the selected sysplex shows the interal scores for the last 24 hours. If you hoer oer an indiidual bar in a bar chart, a pop-up window shows the following information: The start and stop time for the interal The total number of unique message IDs for the interal Interal score IBM zaware V3.1 feature You can click an indiidual bar to naigate in context to IBM zaware for more information about the interal. Client-side Expert Adice If the IBM Operations Analytics - Log Analysis serer is behind a firewall, and therefore does not hae access to the Internet, you can use the Client-side Expert Adice (IBMSupportPortal-ExpertAdice on Client) rather than the default Expert Adice (IBMSupportPortal-ExpertAdice) in the Log Analysis UI. After you install the IBM Operations Analytics for z Systems extensions, the following applications are shown under Expert Adice in the Custom Search Dashboards panel of the left naigation pane of the Search workspace: IBMSupportPortal-ExpertAdice, which is the default Expert Adice in Log Analysis. When this Expert Adice is launched, the Log Analysis serer sends search requests to the IBM Support Portal and displays the query search results in the Log Analysis UI. IBMSupportPortal-ExpertAdice on Client, which is the extension for client-side Expert Adice that is proided by IBM Operations Analytics for z Systems. When this Expert Adice is launched, the client browser sends search requests directly to the IBM Support Portal and opens a new browser tab to display the query search results. IBM Operations Analytics for z Systems includes the separately installable IBM z Adanced Workload Analysis Reporter (IBM zaware) feature. For more information about this feature, see the IBM zaware Guide. This feature must be installed before you can use the IBM zaware data gatherer. Operations Analytics for z Systems oeriew 11

18 IBM zaware data gatherer IBM Operations Analytics for z Systems includes the IBM zaware data gatherer, which is a client that gathers interal anomaly data from an IBM zaware serer. The data is ingested into IBM Operations Analytics - Log Analysis and is shown in the Log Analysis user interface under the data source that is named zos Anomaly Interal. Visualizations of this interal anomaly data are also shown on the new Problem Insights page, if the Problem Insights extension is installed. Before you can use the IBM zaware data gatherer, both the IBM zaware V3.1 feature and the data gatherer must be installed, and the data gatherer must be configured with an IBM zaware serer definition. The data gatherer is installed as part of the installation of the z/os Insight Packs and extensions. You can also configure the data gatherer during this installation by responding to prompts for the alues of the configuration parameters. IBM Common Data Proider for z Systems V1.1.0 The IBM Common Data Proider for z Systems automatically monitors z/os log data and SMF data and forwards it to the configured destination. In each logical partition (LPAR) from which you want to analyze z/os log data or SMF data, a unique instance of IBM Common Data Proider for z Systems must be installed and configured to specify the type of data to be gathered and the destination for that data. For more information, see the IBM Common Data Proider for z Systems V1.1.0 documentation. Logstash and the ioaz Logstash output plugin Logstash is an open source data collection engine that in near real-time, can dynamically unify data from disparate sources and normalize the data into the destinations of your choice for analysis and isualization. IBM Operations Analytics for z Systems proides a copy of Logstash and an ioaz Logstash output plugin that forwards data to the IBM Operations Analytics - Log Analysis serer for processing by the z/os Insight Packs. Because IBM Common Data Proider for z Systems cannot forward data directly to the Log Analysis serer, the z/os log data and SMF data that is gathered by IBM Common Data Proider for z Systems must be forwarded to a Logstash instance. The Logstash instance then forwards the data to the Log Analysis serer. Through Logstash, you can also forward data to other destinations, or route data through a message broker such as Apache Kafka. For more information about Logstash, see the Logstash documentation. Installation and configuration checklists These checklists summarize the important steps for installing and configuring the Operations Analytics for z Systems components, including IBM Operations Analytics - Log Analysis, the z/os Insight Packs, the Log Analysis extensions, the IBM zaware data gatherer, and Logstash and the ioaz Logstash output plugin. The following steps outline the installation and configuration process: 12 Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

19 1. Install IBM Operations Analytics - Log Analysis V1.3.3 Fix Pack 1. See Checklist for IBM Operations Analytics - Log Analysis. 2. On the Log Analysis serer, install the z/os Insight Packs, the Log Analysis extensions, and the IBM zaware data gatherer. See Checklist for z/os Insight Packs, Log Analysis extensions, and the IBM zaware data gatherer. 3. Install Logstash and the ioaz Logstash output plugin. See Checklist for Logstash and the ioaz Logstash output plugin on page Determine the sources from which you want to gather log data. For more information, see Planning for configuration of IBM Common Data Proider for z Systems on page Install the IBM Common Data Proider for z Systems in each z/os logical partition (LPAR) for which you want to process z/os log messages. Then, in each LPAR where the IBM Common Data Proider for z Systems is installed, configure it to gather and forward z/os log data to Logstash. For more information, see the IBM Common Data Proider for z Systems V1.1.0 documentation. If you plan to gather System Management Facilities (SMF) data, also complete the configuration steps that are described in Requirements for configuring SMF data streams on page Secure communication between IBM Common Data Proider for z Systems and Logstash. For more information, see Securing communication between IBM Common Data Proider for z Systems and Logstash on page Ensure that z/os data sources are configured and grouped appropriately in Log Analysis. For more information, see Grouping data sources to optimize troubleshooting in your IT enironment on page 66. Checklist for IBM Operations Analytics - Log Analysis 1. Verify that the system requirements for Log Analysis, including the installation of the prerequisite software, are met. For more information, see Hardware and software requirements. 2. Using the ulimit command, tune the Linux operating system so that the number of concurrent files is 4096, and the irtual memory is unlimited. For more information, see Hardware and software requirements. 3. To install and run Log Analysis, you must be logged in to the Linux computer system with a non-root user ID. Either create this non-root user ID, or use an existing non-root user ID. For more information, see Planning for installation of Log Analysis on page Reiew the network port assignments for Log Analysis. If the default ports are not aailable, determine alternatie port assignments. For more information, see Default ports in the Log Analysis documentation. 5. The IBM Tioli Monitoring Log File Agent is an optional component that is proided with Log Analysis and other IBM products. If you plan to gather logs from Linux for z Systems, install this agent as part of the Log Analysis installation. For more information, see Installing and configuring the IBM Tioli Monitoring Log File Agent in the Log Analysis documentation. 6. Verify that you are logged in to the Linux computer system with the non-root user ID, and complete the steps in Installing Log Analysis on page 33. Checklist for z/os Insight Packs, Log Analysis extensions, and the IBM zaware data gatherer 1. Verify that Log Analysis is running. Operations Analytics for z Systems oeriew 13

20 2. Verify that you are logged in to the Linux computer system with the non-root user ID that was used to install Log Analysis. 3. Install the z/os Insight Packs with sample searches, the extensions for Problem Insights and client-side Expert Adice, and the IBM zaware data gatherer. For more information, see Installing the z/os Insight Packs, extensions, and IBM zaware data gatherer on page 35. Tip: If the IBM zaware data gatherer is not configured, you cannot see isualizations of the interal anomaly data in the Log Analysis user interface. Therefore, if you choose not to configure the IBM zaware data gatherer at installation time, you must configure it before you use the Problem Insights for z/os. For more information about configuring and getting started with the data gatherer, see the following topics: Configuring the IBM zaware data gatherer on page 57 Getting started with the IBM zaware data gatherer on page 72 Checklist for Logstash and the ioaz Logstash output plugin 1. Verify that the Logstash system requirements are met. For more information, see Logstash requirements on page Install Logstash and the ioaz Logstash output plugin. For more information, see Installing Logstash and the ioaz Logstash output plugin on page Configure Logstash and the ioaz Logstash output plugin. For more information, see Configuring Logstash on page Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

21 Planning for installation and configuration of Operations Analytics for z Systems System requirements In planning for the installation of IBM Operations Analytics for z Systems, you must also plan for the installation of IBM Operations Analytics - Log Analysis and the installation and configuration of both Logstash and the IBM Common Data Proider for z Systems. Ensure that your enironment meets the system requirements for IBM Operations Analytics for z Systems. Log Analysis requirements For information about the system requirements for Log Analysis, see Hardware and software requirements. Restriction: If you deploy Log Analysis on a Linux on System z system, you can use the IBM InfoSphere BigInsights Hadoop Version 3.0 serice, but at this time, the Hadoop Distributed File System (HDFS) must be installed on an x86 Linux system. The support for integrating Log Analysis on a Linux on System z system with other Hadoop distributions is dependent on the platform support that is proided by the respectie Hadoop distribution. z/os Insight Pack requirements You must install the z/os Insight Packs on a Log Analysis serer with Log Analysis Version The IBM Operations Analytics for z Systems Version package includes IBM Operations Analytics - Log Analysis Version Standard Edition. To use the extensions for z/os Problem Insights and client-side Expert Adice and to use the IBM zaware data gatherer, you must also hae Log Analysis Version Fix Pack 1 (also known as V ) installed. Tip: To get Log Analysis Version Fix Pack 1 ( ), complete the following steps: 1. Go to IBM Fix Central. 2. In the Product selector field, start typing IBM Operations Analytics - Log Analysis, and when the correct product name is shown in the resulting list, select it. More fields are then shown. 3. In the Installed Version field, select In the Platform field, select All. 5. Click Continue. 6. In the resulting Identify fixes window, select Browse for fixes, and click Continue. 7. In the Select fixes window, you should see the fix pack TIV-IOALA- FP001, which you can select and download. For installation instructions, see the readme file. Table 3 on page 16 indicates the software ersions that are supported by each z/os Insight Pack. Copyright IBM Corp. 2014,

22 Table 3. Software ersions that are supported by each z/os Insight Pack Insight Pack Supports data ingestion from this software z/os Network Insight Pack z/os 2.1 and 2.2 NetView for z/os 6.1, 6.2, and z/os SMF Insight Pack z/os 2.1 and 2.2 z/os SYSLOG Insight Pack z/os 2.1 and 2.2 WebSphere Application Serer for z/os Insight Pack DB2 for z/os 10.1 and 11.1 CICS Transaction Serer for z/os 4.2, 5.1.1, 5.2, and 5.3 IMS for z/os 12.1, 13.1, and 14.1 MQ for z/os 7.1, 8.0, and 9.0 WebSphere Application Serer for z/os 7.0, 8.0, 8.5.5, and 9.0 Logstash requirements You must install Logstash on a Linux on System p, System x, or System z system with the following software: Red Hat Enterprise Linux 6 or 7 or SUSE Linux Enterprise Serer 11 or 12 Jaa Runtime Enironment (JRE) 7 or 8 Logstash requires 180 MB of disk space, plus more disk space for logs and the cache. The cache is used for storing data that cannot be sent if the IBM Operations Analytics - Log Analysis serer is down. IBM Common Data Proider for z Systems requirements You must run the IBM Common Data Proider for z Systems in each z/os logical partition (LPAR) for which you want to process z/os log messages. For information about the system requirements for this proider, see the IBM Common Data Proider for z Systems V1.1.0 documentation. Planning for installation of Log Analysis Before you install IBM Operations Analytics - Log Analysis, reiew the requirements for the installation user ID, domain name resolution, and network connectiity. Also, decide whether you need to use the optional IBM Tioli Monitoring Log File Agent component. Before you begin Verify that your enironment meets the system requirements that are described in System requirements on page 15. About this task Installation user ID To install and run Log Analysis, you must be logged in to the Linux computer system with a non-root user ID. 16 Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

23 Domain name resolution On the system where you plan to install Log Analysis, erify that the details for the Log Analysis serer are maintained correctly in the /etc/hosts file. Network connectiity between the Logstash serer and the IBM Common Data Proider for z Systems LPARs The LPARs with IBM Common Data Proider for z Systems must communicate with the Logstash serer on the port that is defined during Logstash pipeline configuration. The default alue for this port is Ensure that communication on this port is not blocked by a firewall or by other aspects of your network configuration. Network connectiity between the Logstash serer and the Log Analysis serer The Logstash serer must communicate with the Log Analysis serer on the port that is defined during Log Analysis installation. The default alue for this port is Ensure that communication on this port is not blocked by a firewall or by other aspects of your network configuration. Optional Log File Agent component The IBM Tioli Monitoring Log File Agent is an optional component that is proided with Log Analysis and other IBM products. You can use this agent to collect logs from Linux for z Systems and from other Linux and UNIX operating systems. For more information, see Installing and configuring the IBM Tioli Monitoring Log File Agent in the Log Analysis documentation. Planning for Log Analysis to support LDAP interaction with RACF for sign-on authentication You can configure IBM Operations Analytics - Log Analysis to support LDAP interaction with RACF for sign-on authentication. Procedure You must plan for the following configuration steps: 1. In Log Analysis, erify that no data sources are created and that no permissions are assigned to data sources. 2. In the Log Analysis UI, create at least one user ID for use in doing administratie tasks in Log Analysis. 3. Define other user IDs for use in doing non-administratie tasks in Log Analysis. 4. Because LDAP and RACF authenticate these user IDs during sign-on, ensure that these user IDs are also defined to LDAP and RACF. Tip: In this configuration with LDAP and RACF, Log Analysis cannot use the default user IDs unityadmin and unityuser due to RACF restrictions on the length of a user ID (it must be 7 characters or less). 5. Configure Log Analysis to use LDAP for authentication. For instructions, see LDAP configuration in the Log Analysis documentation. For z Systems LDAP support, choose the Tioli Directory Serer as the type of LDAP serer. Planning for installation and configuration 17

24 6. Sae a backup copy of the following Log Analysis configuration files so that you can update these files. LA_INSTALL_DIR/utilities/datacollector-client/ jaadatacollector.properties LA_INSTALL_DIR/remote_install_tool/config/rest-api.properties LA_INSTALL_DIR/UnityEIFReceier/config/unity.conf LA_INSTALL_DIR/solr_install_tool/scripts/register_solr_instance.sh 7. Update the passwords in the four preiously listed Log Analysis configuration files. For more information, see Changing a user password in the Log Analysis documentation. 8. Map your LDAP groups to the Log Analysis security role so that the users can access Log Analysis. Indiidual users can also be granted user or administratie priileges. For more information, see LDAP configuration in the Log Analysis documentation. 9. If you use the delete utility to prune data sources, proide a user ID and password that are known to RACF to this utility. You can encrypt the password if you choose to store it in the Log Analysis serer. Planning for configuration of IBM Common Data Proider for z Systems You must determine the sources from which you want IBM Common Data Proider for z Systems to gather data. About this task You can gather data from the following sources: Any job log Any z/os UNIX log file or entry-sequenced Virtual Storage Access Method (VSAM) cluster that meets the following criteria: Is encoded in Extended Binary Coded Decimal Interchange Code (EBCDIC) Is record-based Has a time stamp in each record z/os SYSLOG console System Management Facilities (SMF) Tip: Table 11 on page 22 indicates the SMF record types that are supported by the z/os SMF Insight Pack with their associated data stream types. These data stream types are shown in the configuration tool for the IBM Common Data Proider for z Systems under the title SMF Data > IOAz. NetView for z/os messages WebSphere Application Serer for z/os High Performance Extensible Logging (HPEL) logs If you plan to gather data from the following software, you must gather z/os SYSLOG data: IBM CICS Transaction Serer for z/os 18 Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

25 IBM DB2 for z/os IBM IMS for z/os IBM MQ for z/os IBM Resource Access Control Facility (RACF) z/os Communications Serer Procedure Determine the sources from which you want to gather data so that you can configure the IBM Common Data Proider for z Systems to stream the operational data to IBM Operations Analytics for z Systems and can obtain actionable insights into the health of your IT operations enironment. Correlation of the data to be analyzed with the associated data stream types, data source types, and dashboards For the data that you want to analyze, you must know which data stream types to define during the configuration of the IBM Common Data Proider for z Systems. The data stream types are associated with data source types that are defined in the IBM Operations Analytics - Log Analysis serer and UI. The data source types are also associated with IBM-proided Custom Search Dashboard applications. You can customize these dashboard applications, which are aailable in the Log Analysis UI, to help you analyze the data and troubleshoot problems. The following sections correlate each type of data that you can analyze with the following information: The Insight Packs that must be installed to analyze the data The associated data stream types that you must define during the configuration of IBM Common Data Proider for z Systems The associated data source types that are defined in the Log Analysis serer and UI The associated Custom Search Dashboard applications from the Insight Pack that you can customize and use in the Log Analysis UI Important: All data that is ingested into Log Analysis must be encoded in Unicode Transformation Format, 8-bit encoding form (UTF-8). All data streams that are destined for Log Analysis must be transformed to UTF-8 encoding. Planning for installation and configuration 19

26 CICS Transaction Serer for z/os data Table 4. Analyzing CICS Transaction Serer for z/os data: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Dashboard applications z/os SYSLOG Insight Pack z/os SYSLOG zos-syslog-console zos-syslog-sdsf CICS EYULOG zos-cics-eyulog None CICS EYULOG DMY zos-cics-eyulogdmy None CICS EYULOG YMD zos-cics-eyulogymd None CICS User Messages zos-cics-msgusr None CICS User Messages DMY zos-cics-msgusrdmy None CICS User Messages YMD zos-cics-msgusrymd None CICS Transaction Serer for z/os Dashboard on page 134 z/os SMF Insight Pack SMF110_E zos-smf110_e z/os SMF Custom Search Dashboard applications on page 128 SMF110_S_10 zos-smf110_s_10 z/os SMF Custom Search Dashboard applications on page 128 DB2 for z/os data Table 5. Analyzing DB2 for z/os data: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Dashboard applications z/os SYSLOG Insight Pack z/os SYSLOG zos-syslog-console zos-syslog-sdsf DB2 for z/os Dashboard on page 134 IMS for z/os data Table 6. Analyzing IMS for z/os data: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Dashboard applications z/os SYSLOG Insight Pack z/os SYSLOG zos-syslog-console zos-syslog-sdsf IMS for z/os Dashboard on page 135 MQ for z/os data Table 7. Analyzing MQ for z/os data: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Dashboard applications z/os SYSLOG Insight Pack z/os SYSLOG zos-syslog-console zos-syslog-sdsf MQ for z/os Dashboard on page Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting

27 NetView for z/os messages Table 8. Analyzing NetView for z/os messages: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Dashboard applications z/os Network Insight Pack NetView Netlog zos-netview NetView for z/os Dashboard on page 127 Network data Network data can include, for example, data from UNIX System Serices system log (syslogd) or z/os Communications Serer. Table 9. Analyzing network data: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Custom Search Dashboard application z/os Network Insight Pack z/os SYSLOG Insight Pack None None z/os Networking Dashboard on page 127 z/os SYSLOG zos-syslog-console USS Syslogd Admin USS Syslogd Debug USS Syslogd Error zos-syslog-sdsf zos-syslogd None SYSLOG for z/os Time Comparison Dashboard on page 133 SYSLOG for z/os Dashboard on page 132 Security data Table 10. Analyzing security data: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Custom Search Dashboard application z/os SYSLOG Insight Pack z/os SMF Insight Pack SMF80_COMMAND z/os SYSLOG zos-syslog-console SMF80_LOGON SMF80_OMVS_RES_1 SMF80_OMVS_RES_2 SMF80_OMVS_SEC_1 SMF80_OMVS_SEC_2 SMF80_OPERATION SMF80_RESOURCE zos-syslog-sdsf Security for z/os Dashboard on page 136 zos-smf80 (RACF) z/os SMF Custom Search Dashboard applications on page 128 Planning for installation and configuration 21

28 SMF data Table 11. Analyzing SMF data: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Dashboard applications z/os SMF Insight Pack SMF30 zos-smf30 (z/os job performance) z/os SYSLOG Insight Pack SMF80_COMMAND SMF80_LOGON SMF80_OMVS_RES_1 SMF80_OMVS_RES_2 SMF80_OMVS_SEC_1 SMF80_OMVS_SEC_2 SMF80_OPERATION SMF80_RESOURCE z/os SMF Custom Search Dashboard applications on page 128 zos-smf80 (RACF) z/os SMF Custom Search Dashboard applications on page 128 SMF110_E zos-smf110_e (CICS Transaction Serer for z/os) SMF110_S_10 zos-smf110_s_10 ( CICS Transaction Serer for z/os) WASACT_REQAPPL WASACT_REQCONT zos-smf120 (WebSphere Application Serer for z/os) z/os SYSLOG zos-syslog-console zos-syslog-sdsf z/os SMF Custom Search Dashboard applications on page 128 z/os SMF Custom Search Dashboard applications on page 128 z/os SMF Custom Search Dashboard applications on page 128 None WebSphere Application Serer for z/os data Table 12. Analyzing WebSphere Application Serer for z/os data: required Insight Packs and associated data stream types, data source types, and dashboards Required Insight Packs Data stream types Data source types Dashboard applications WebSphere Application Serer for z/os Insight Pack WebSphere SYSOUT zos-was-sysout WebSphere Application Serer for z/os Custom Search Dashboard applications on page 126 WebSphere SYSPRINT WebSphere USS Sysprint z/os SMF Insight Pack WASACT_REQAPPL zos-was-sysprint WebSphere Application Serer for z/os Custom Search Dashboard applications on page 126 WebSphere HPEL zos-was-hpel WebSphere Application Serer for z/os Custom Search Dashboard applications on page 126 WASACT_REQCONT zos-smf120 z/os SMF Custom Search Dashboard applications on page Operations Analytics for z Systems: Installing, configuring, using, and troubleshooting