IBM Performance Management Agent installation and configuration

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "IBM Performance Management Agent installation and configuration"

Transcription

1 IBM Performance Management Agent installation and configuration Getting Started Guide January, 2017

2 Table of content Table of content... 2 About this document... 3 General agent installation steps... 4 Running agents as a non-root user... 5 Installing and configuring the agents in silent mode... 6 Response Time Monitoring agent... 7 Installing the HTTP Server and Response Time Monitoring agents... 7 Configuring the Response Time agent... 7 Response Time Monitoring and load balancers... 9 Setting custom geolocations CSV file examples Monitoring HTTPS Traffic using packet sniffing WebSphere MQ and IIB agents Installing the agents WebSphere MQ agent permissions Enabling tracing in WebSphere MQ Configuring WebSphere MQ to collect metrics Configuring the WebSphere MQ agent Configuring IIB to collect metrics Configuring the IIB agent WebSphere Applications agent Installing the agent Configuring the data collector Enabling development mode DB2 agent Installing the agent Preconfiguration steps Configuring the DB2 agent DataPower and OS agents Installing the agent Configuring the DataPower agent Configuring DataPower Enabling transaction tracking Configuring Web Service Proxy for transaction tracking Configuring Multi-Protocol Gateway for transaction tracking Configure transforms to enable instance-level transaction tracking Configure the Web Service Proxy for tracking instance topologies Configure the Multi-protocol Gateway for tracking instance topologies WebSphere Infrastructure Manager Installing the agent Log file monitoring FMT files Tips for building a REGEX expression... 25

3 About this document Use this document as a guide for getting started with the Performance Management agents in a trial or proof of concept (POC) environment. The installation and configuration steps represent one flow through the installation and configuration process. For the complete and up-to-date steps for installing and configuring the agents, see the IBM Performance Management V8.1.3 on premises knowledge center or the IBM Performance Management on Cloud knowledge center.

4 General agent installation steps Complete the following general steps to install the agents: 1. Download the agent installation package. In an IBM Performance Management on premises environment, download an agent package from Passport Advantage. In an IBM Performance Management SaaS environment, go to the Product and Services page on IBM Marketplace, select your Performance Management subscription, and download an agent package. 2. Configure the agents in the installation package to communicate with the Performance Management server. You can configure the media file during the installation of the Performance Management server. During the installation, the media file is placed in /opt/ibm/ccm/depot by default. Alternatively, you can manually configure the agent media file. Run these commands: /opt/ibm/ccm/make_configuration_packages.sh /opt/ibm/ccm/configure_agent_images.sh The media file is placed in /opt/ibm/ccm/depot by default. Use the media file in /opt/ibm/ccm/depot to install the agents. In an IBM Performance Management SaaS environment, this step is skipped. The media file is preconfigured to connect to the Performance Management server. 3. Extract the tar or zip media file. 4. Install one or more agents using the./installapmagents.sh or installapmagents.bat file. 5. Specify the agents to install by entering the corresponding numbers separated by commas. For example, 22, Follow the onscreen prompts to complete the installation. For some of the agents, you must also configure the agent for it to function. 7. To start the agents, complete these steps: On Windows systems, click Start > All Programs > IBM Monitoring Agents > IBM Performance Management. Right-click an agent and click Start. On Linux and AIX systems, you can start the agent from the command line:./name-agent.bat start Many of the agents are configured and started automatically, and report data to the Performance Management server as soon as you install them. Some agents require manual configuration but start automatically. Other agents must be configured and started manually. Single instance agents and multi-instance agents are configured as follows: For single instance agents, enter the <agent_name>.sh config command to configure the agent. Restart the agent after you configure it. For multi-instance agents, enter the <agent_name>.sh config <instance_name> to configure the first instance of the agent. Restart the agent instance using the <agent_name>.sh start <instance_name> command. Tip: Set the instance name to something that represents the instance that you are monitoring. For more information about which agents you must configure and start manually, see the Agent post installation checklist in the on cloud or on premises knowledge centers. Tip: If the installation fails and displays a prerequisite checker error, verify that the agent meets the system requirements. If installing the agent is valid, try to bypass the prerequisites check using the command export SKIP_PRECHECK=1 and rerun the installer.

5 Running agents as a non-root user You might run an agent as a non-root user for the following two reasons: The application or middleware that you plan to monitor requires a user other than user root. In your environment, your policy is not to run applications or scripts as user root. For more information about installing the agents as a non-root user, see Installing agents as a nonroot user in the on cloud or on premises knowledge centers. If you install all agents as a non-root user, you must have a root password to configure the /etc/init.d or chkconfig startup files so that the agents start at boot time. If you install the OS agent as a non-root user, the agent cannot collect all of the metrics. The most common installation scenario is to install the agents as the root user and to configure one of the middleware agents as a non-root account. Remember: If all agents are installed and run as the same user, the installation and configuration process is similar to an installation as a root user. If the agents are run as different users, you must use the secure.sh script. Complete these steps to install the agents as user root and then to configure one of the middleware agents as a non-root account: 1. Install the agents that you want to run. 2. Stop all of the agents that are running on the computer system. 3. Run the /ibm/apm/agent/bin/secure.sh command to create a group for the agents. The command changes the group ownership of directories and files. For example, if you want to configure the mqm group to have the required permissions, run the command: /opt/ibm/apm/agent/bin/secure.sh g mqm 4. Configure the agent as a user that is a member of the mqm group.

6 Installing and configuring the agents in silent mode To perform a silent installation, use the file <media>/apmadv_silent_install.txt. This file includes a list of agents that are available in the media. Uncomment the agents that you plan to install and then run the installation command:./installapmagents.sh p./apmadv_silent_install.txt When you install an agent, a sample silent configuration file is placed in /opt/ibm/apm/agent/samples. For example, ynv_silent_config_agent.txt and datapower_silent_config.txt. Edit the silent configuration file and then run one of these commands: Single Instance agent: <product_code-agent.sh> config <path to response file> Multi-instance agent: <product_code-agent.sh> config <instance name> <path to response file> For example:./db2-agent.sh config db2inst1 /opt/ibm/apm/agent/samples/db2_silent_config.txt Note: Some agents, such as the WebSphere Applications agent, have multiple silent configuration files for performing configuration tasks such as configuring the data collector. The rest of this document focuses on the configuration steps for some of the most common monitoring agents.

7 Response Time Monitoring agent The Response Time Monitoring agent supports two modes: Network packet analyzer mode, where the agent analyzes the network traffic. HTTP Server Response Time module (plug-in) mode. This is only supported on IHS and Apache in IBM Performance Management V The installer automatically detects IHS or Apache and sets up the plug-in. It s possible to modify the *t5.cfg file and configure the network packet analyzer, but it is not recommended. When using the network packet analyzer, you can add JavaScript Instrumentation by adding a line of JavaScript to the web pages. Figure 1 shows the capabilities of each monitoring approach. Figure 1 Capabilities available with each monitoring approach Installing the HTTP Server and Response Time Monitoring agents Note: If you are using the network packet analyzer, note that Performance Management only supports the HTTPS ciphers that are documented in the Monitoring HTTPS transactions topic in the on cloud and on premises knowledge centers. Complete these steps to install the HTTP Server and Response Time Monitoring agents. 1. Change the directory to /<media>/apmadv_agent_install_ Run the installer. Enter:./installAPMAgents.sh Note: You must install the HTTP Server agent with the Response Time agent. The plugin for IHS is part of the HTTP Server agent. 3. When prompted, specify the corresponding numbers to install the Response Time Monitoring and HTTP Server agents. 4. Accept the defaults for the installation. The HTTP Server agent does not require any extra configuration. Configuring the Response Time agent If you are using IHS or Apache, you do not have to configure the Response Time Monitoring agent. However, you must configure the plug-in on the IHS or Apache server. Note: In the following section, the examples shown are for IHS.

8 When the HTTP Server agent and Response Time Monitoring agents are installed, the following file is set up in the agent directory: /opt/ibm/apm/agent/tmp/khu/khu.opt.ibm.httpserver.conf.httpd.conf This file must be set up in an include statement in the httpd.conf file. Complete these steps to configure the Response Time Monitoring agent to edit the /opt/ibm/httpserver/conf/httpd.conf file: 1. Go to the end of the file and add this line: Include /opt/ibm/apm/agent/tmp/khu/khu.opt.ibm.httpserver.conf.httpd.conf 2. Recycle the IHS server. Stop and Start the IHS server using these commands: /opt/ibm/httpserver/bin/apachectl k stop /opt/ibm/httpserver/bin/apachectl k start If you have high transaction rates and you want to throttle the consumption of the Response Time Monitoring agent, you have two options. Edit the kfcenv file and set one of the following variables: KFC_MAX_PROTOCOL_PACKETRATE The maximum number of packages processed. For example, if you set the value to 2000, the agent processes 2000 packets per second. KFC_CPUTHROTTLE_TARGET Set this to the % CPU utilization at which you want to cap the agent. The parameter represents a percentage of each CPU. If you want the agent to consume 10% of the system resources on a 4 core system, set the value to 10. It will consume an average of 10% of all 4 cores.

9 Response Time Monitoring and load balancers If you are monitoring response times and you have load balancers in your environment, some additional customization is required (see Figure 2 for an example). Figure 2 Environment with load balancers To handle load balancers correctly, turn off URL rewrite on the load balancer. URL rewrite is normally off on most load balancers, but not on an Apache load balancer. The following sample configuration is from the Apache httpd.conf file: Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=balancer_route_changed ProxyPass /trade balancer://mycluster stickysession=jsessionid jsessionid scolonpathdelim=on <Proxy balancer://mycluster> </Proxy> BalancerMember route=1 BalancerMember route=2 BalancerMember route=3 ProxySet stickysession=routeid ProxyPass / balancer://mycluster/ ProxyPassReverse / balancer://mycluster/ ProxyPreserveHost On Response time traffic is configured to flow from the load balancer. When using Response Time monitoring through a load balancer, if you turn off URL rewrite, it will appear that all traffic is coming through the load balancer. Then, when you log in to the Performance Management console and you click the "Read" button to specify a hostname:port, you can specify the hostname and port number of the load balancer. All traffic through the load balancer will be monitored. Next, install and configure the Response Time agent on each of the HTTP servers.

10 Setting custom geolocations The agent automatically detects the location of the requests based on the IP address and it specifies a country, state, and city for the requests. However, some addresses, such as NAT ed addresses and private IP addresses, do not correctly resolve to a location. To set a location for an IP address or address range, complete these steps: 1. On the Performance Management server, go to /opt/ibm/geolocation. 2. To import a single IP address or range and location, run the following command:./import-ip.sh <IP address> <City> <State> <Country> For example./import-ip.sh /24 Munich Bavaria Germany 3. To import values from a CSV file, run the following command:./import-csv.sh <path and filename of CSV file> 4. To clear previously resolved IP addresses and locations, run the following command:./clear-resolved-ips.sh CSV file examples The CSV file must have the following values as a header. The values can be in any order, and the entries must match that order: IP_ADDRESS,CITY,REGION,COUNTRY For example: IP_ADDRESS,CITY,REGION,COUNTRY 2001::0001:0:0/96,Perth,WA,Australia /24, Alice Springs, NT, Australia Monitoring HTTPS Traffic using packet sniffing Packet sniffing of HTTPS traffic requires an SSL certificate to decrypt the traffic. 1. Set up a keystore. To set up a keystore, following the instructions in the Setting up the keystore topic in the on cloud or on premises knowledge centers. 2. Configure the agent. Run /opt/ibm/apm/agent/bin/rt-agent.sh config 3. When prompted, specify 1 to indicate that you want to monitor HTTPS traffic. 4. Specify the HTTPS keystore, for example: /tmp/keys.kdb. 5. Map the certificate to the HTTP servers/ports. For example: label1, ,9443;label1, , Specify the NIC card that you want to monitor. If no NIC is specified, all NICs are monitored. 7. After configuring the agent, start the agent: /opt/ibm/apm/agent/bin/rt-agent.sh start

11 8. Log in to the Performance Management console and click System Configuration > Agent Configuration. Click the Response Time tab (see Figure 3). Select the Response Time agent and specify the ports to monitor. Figure 3 HTTP ports to monitor

12 WebSphere MQ and IIB agents Installing the agents Complete these steps to install the WebSphere MQ and IIB agents. 1. Log in to the WebSphere MQ server. 2. Go to the agent installation directory. Enter: cd /<agent_media>/offering_agent_install_ Run the installation script. Enter:./installAPMAgents.sh 4. Enter 20,22 to select the WebSphere MQ and IIB agents. 5. Confirm your selection. 6. Accept the default installation directory of /opt/ibm/apm/agent. 7. Enter 1 to accept the license agreement. The WebSphere MQ and IIB agents are installed. You must configure the WebSphere MQ and IIB agents. WebSphere MQ agent permissions For the WebSphere MQ agent to work correctly, the user account that runs the agent must have the appropriate WebSphere MQ permissions. You can either: Configure and run the agent as mqm. Grant the required permissions to another user such as the root user. Enabling tracing in WebSphere MQ To collect transaction tracking data from WebSphere MQ, you must enable tracing in WebSphere MQ. Complete these steps: 1. Enter /opt/mqm/bin/runmqsc to open the MQSC command prompt. 2. Enter: ALTER QMGR ACTVTRC(ON). 3. Quit the MQSC command prompt. Configuring WebSphere MQ to collect metrics Before you configure the agent, you must configure WebSphere MQ to expose metrics. Complete these steps: 1. Enter: setmqaut -m <queue_manager> -t qmgr -p <user_id> +inq +connect +dsp +setid For example: setmqaut -m TRADEQM -t qmgr -p root +inq +connect +dsp +setid 2. As the mqm user, enter: runmqsc <queue_manager> 3. To enable the queue manager to emit specific enter types, complete these steps: a. For authority events, enter ALTER QMGR AUTHOREV(ENABLED) b. For channel events, enter ALTER QMGR CHLEV(ENABLED)

13 c. For command events, enter ALTER QMGR CMDEV(ENABLED) d. For configuration events, enter ALTER QMGR CONFIGEV(ENABLED) e. For IMS Bridge events, enter ALTER QMGR BRIDGEEV(ENABLED) f. For inhibit events, enter ALTER QMGR INHIBTEV(ENABLED) g. For local events, enter ALTER QMGR LOCALEV(ENABLED) h. For logger events, enter ALTER QMGR LOGGEREV(ENABLED) i. For performance events, enter ALTER QMGR PERFMEV(ENABLED) j. For remote events, enter ALTER QMGR REMOTEEV(ENABLED) k. For SSL events, enter ALTER QMGR SSLEV(ENABLED) l. For start and stop events, enter ALTER QMGR STRSTPEV(ENABLED) 2. If you do not see queue and channel data, enter the MQ command: ALTER QMGR MONQ(MEDIUM) MONCHL(MEDIUM) 3. To enable Transaction Tracking information, enter: ALTER QMGR ACTVTRC(ON) 4. Recycle the queue manager. Enter: endmqm <queue_manager> strmqm <queue_manager> Configuring the WebSphere MQ agent To configure the WebSphere agent, complete these steps: 1. Go to the installation directory. Enter: cd /opt/ibm/apm/agent/bin 2. Start the configuration script. Enter:./mq-agent.sh config <instance_name> 3. Enter 1 to edit the settings. 4. Specify the agent ID: <unique_name> 5. Specify the Queue Manager name: <queue manager name> 6. Enter the middle qualifier for the managed system name. Use a meaningful name, for example, use the host name mq. The name is displayed on the UI. 7. Specify the WebSphere MQ 64-bit library path: /opt/mqm/lib64 8. Start the agent. Enter:./ mq-agent.sh start <instance_name> 9. To enable transaction tracking, log in to the Performance Management console and click System Configuration > Agent Configuration. Click the WebSphere MQ tab (see Figure 4). Click Set Transaction Tracking > Enabled. Figure 4 - WebSphere MQ tab on the Agent Configuration page Configuring IIB to collect metrics Before you configure the agent, configure IIB to collect the metrics that are required for monitoring.

14 Complete these steps: 1. Set up the environment variables. Enter:../mqsiprofile 2. Enable flow statistics. Enter: mqsichangeflowstats <broker_name> -a -g -j -c active -n advanced -t basic -o xml 3. To change the roll-up interval to 1 minute, enter: mqsichangebroker <broker_name> -v 1 Tip: You can specify any number of minutes. 4. Enable resource statistics for integration servers that belong to the integration broker. Enter: mqsichangeresourcestats <broker_name> -c active 5. Recycle IIB. Enter: mqsistop <broker_name> mqsistart <broker_name> 6. To enable transaction tracking data, set the user exit to active. Enter: mqsistop <broker_name> mqsichangebroker <broker_name> -e "KQIUserExit mqsistart <broker_name> 7. To verify that the user exit is active, enter: mqsireportbroker <broker_name> Verify that you see Active user exits = 'KQIUserExit in the results. Configuring the IIB agent To configure the IIB agent, complete these steps: 1. Go to the installation directory. Enter: cd /opt/ibm/apm/agent/bin 2. Start the configuration script. Enter:./iib-agent.sh config <broker_name> 3. Enter 1 to edit the settings. 4. Specify the agent ID: <unique_name> 5. Specify the IIB installation directory: /opt/ibm/mqsi/ To monitor all brokers, enter 5. Otherwise, to monitor a specific broker, enter 1 and complete these steps to add the broker. a. Specify the broker name: <broker_name> b. Enter 1 if you want to collect node definition data. c. Type 5 to exit. 7. When prompted, enter 1 to indicate that MQ is installed on this system. 8. Specify the WebSphere MQ 64-bit library path: /opt/mqm/lib64 9. To enable transaction tracking, log in to the Performance Management console and click System Configuration > Agent Configuration. Click the IBM Integration Bus tab (see Figure 4). Click Set Transaction Tracking > Enabled.

15 WebSphere Applications agent Installing the agent Complete these steps: 1. Enter: cd /<media>/apmadv_agent_install_ Enter:./installAPMAgents.sh 3. Enter 19 to select the Monitoring Agent for WebSphere Applications. 4. Confirm your selection. 5. Accept the default installation directory of /opt/ibm/apm/agent. 6. Enter 1 to accept the license agreement The WebSphere Applications agent is installed. The WebSphere Applications agent does not require any configuration. You can optionally change the default listening port from to another port. This port is used to communicate with the data collector. Configuring the data collector To configure the data collector, complete these steps: 1. Enter: cd /opt/ibm/apm/agent/yndchome/ /bin The 7.3.x directory name depends on the version of the data collector you installed. 2. Enter:./simpleconfig.sh Note: For most environments, simpleconfig.sh is sufficient. For more complex environments, you can run config.sh from the same directory. When using simpleconfig.sh, the script automatically discovers the application server home directories. 3. Enter the number corresponding to the discovered home directory. 4. You can specify to retrieve security settings from a client properties file. It is usually not required. In this case, choose the default option. 5. Enter the wsadmin username. For example: wasadmin 6. Enter the wsadmin password. Note: If simpleconfig.sh does not work, you ll need to run config.sh instead. 7. Restart the application server: a. Go to /opt/ibm/websphere/appserver/profiles/appsrv01/bin b. Enter:./stopServer.sh <server name> c. When prompted, type the wasadmin username and password. d. Enter:./startServer.sh <server name> The agent is running and the data collector is configured and is collecting data. It can take a few minutes for the data to display in the Performance Management console. If the data is not showing, wait 10 minutes, and complete the next step. 8. Optionally, configure the transaction tracking and deep dive diagnostics settings from the Performance Management console.

16 a. Log in to the Performance Management console and click System Configuration > Agent Configuration. b. Click the WebSphere Applications tab. c. Select the check box next to the WebSphere Applications agent and click Actions > Enable Transaction Tracking. d. Select the check box next to the WebSphere Applications agent and click Actions > Enable Diagnostic Mode and Method Trace Enabling development mode By default, the deep dive diagnostics only capability of the WebSphere Applications agent gathers method trace data for very slow or failed transactions. This helps lower the overhead of the agent. When developing code or during POCs, it can be useful to enable diagnostics for all transactions. Complete these steps: 1. Edit <agent home>/yndchome/../<was..>/custom/gdc/gdc_custom.properties 2. Uncomment the following line by removing the pound sign and set it its value to true: dfe.enable.methoddata=true 3. To capture method trace on every transaction by disabling the sampler, set the following properties: dc.sampling.methsampler.enabled=false dc.sampling.enable=false 4. You can modify the threshold for in flight transactions. The default is to capture transactions that last longer than 60 seconds. In the gdc.properties file, modify the line: com.ibm.itcam.gdc.inflight.request.seconds=60 5. Restart the application server.

17 DB2 agent Installing the agent Complete these steps to install the DB2 agent. 1. Go to /<agent media>/apmadv_agent_install_ Enter:./installAPMAgents.sh 3. When prompted, enter 18 to install the Monitoring Agent for DB2. 4. Enter 1 to confirm your selection. 5. Specify the agent home directory or type enter to accept the default: /opt/ibm/apm/agent 6. Type 1 to accept the license agreement. After the installation completes, you must configure and start the agent. Preconfiguration steps The user that configures and runs the agent must have access to DB2. You can either add user root to the db2iadm1 group, or you can run the agent as a different user. For the DB2 agent to run correctly, the agent must have permissions within the DB2 database. The user who runs the agent must have DB2 SYSADM authority for the monitored instance. The agent requires the SYSADM authority to turn on all monitor switches. In the Performance Management on cloud and on premises knowledge centers, the Privileges for viewing DB2 metric data topic lists the GRANT statement commands that you must run to give access to specific metrics. However, if your user is a member of the db2iadm1 group, you are not required to run these GRANT statements. Configuring the DB2 agent To configure the DB2 agent, complete these steps: 1. As user root, enter: cd /opt/ibm/apm/agent/bin 2. Enter:./db2-agent.sh config <db2_instance> Where db2apm is the DB2 instance name. 3. Enter 1 to edit the settings. 4. Optionally, specify the path and file name of a customized SQL definition file. 5. Specify the path to the DB2 diagnostics log (db2diag.log): /home/<instance owner>/sqllib/db2dump 6. Optionally, specify a MSGID to filter the DB2 diagnostics log. This is a regular expression. Alternatively, press enter to accept the default value. 7. Choose the default option to enable monitoring of partitions on remote hosts. 8. Choose the default option to enable monitoring of all databases. 9. Start the agent. Enter:./db2-agent.sh start db2apm

18 Note: With the customized SQL definition, you can define custom SQL statements that will populate a set of user defined fields within Performance Management. A sample file is provided in /opt/ibm/apm/agent/config/kudcussql.properties. MSGID is a combination of the message type, message number, and severity level. You can use a regular expression in the MSGID filter, for example, ADM1\d*1E ADM222\d2W.

19 DataPower and OS agents Installing the agent The DataPower agent gathers data remotely from the DataPower appliance. To install the DataPower agent, complete these steps: 1. Using an AIX or Linux system, install the DataPower agent. Note: The DataPower Agent may not be installed inside a Docker container. 2. From the base OS, go to /opt/images/apmadv_agent_install_ Run the installer. Enter./installAPMAgents.sh 4. Follow the prompts and select the agents to install. Type 1,11 to install the OS agent and DataPower agent. 5. Accept the remaining prompts using the default values. The OS Agent does not require any configuration and immediately sends data to the Performance Management server. The DataPower agent must be configured. Configuring the DataPower agent Complete these steps to configure the agent: 1. Go to: /opt/ibm/apm/agent/bin 2. Enter:./datapower-agent.sh config <instance name> Tip: Choose a meaningful instance name, for example, use the host name of the DataPower appliance. 3. Type 1 to add a DataPower appliance to monitor. 4. For Managed System Name, enter any name. Remember: the name displays on the user interface. 5. For device host, enter <IP address or host name> 6. For XML Management Interface port, enter: For User ID, enter admin 8. Enter your password: <password> 9. Reenter the password: <password> 10. For System Log, enter logstore://system.log 11. For SSL profile enabled, enter Select 5 to exit. Your agent is now configured. 13. Start the agent. Enter:./datapower-agent.sh start <instance name> Configuring DataPower For information about the additional steps that are required to enable resource monitoring, see the Configuring DataPower Appliances topic in the on cloud or on premises knowledge centers.

20 Enabling transaction tracking Complete these steps on each DataPower appliance where you plan to collect tracking data: 1. Log in to the WebGUI for the DataPower Appliance that you want to monitor. 2. Select the default domain. 3. Search for XML Management Interface. Set the following values and click Apply. On the Main tab, in the Enabled services section, enable WS-Management endpoint 4. Search for Web Services Management Agent. Set the following values and click Apply. a. Set the Administrative state to enabled. b. Set the Capture Mode to None. c. Set the Buffering Mode (deprecated) to Discard. Next, configure the Web Service Proxy or Multi-Protocol Gateway for transaction tracking. Configuring Web Service Proxy for transaction tracking Complete these steps on each Web Service Proxy where you plan to collect tracking data. 1. Select the domain that the Web Service Proxy belongs to. 2. On the Proxy Settings tab, set the following values and click Apply. Set the Monitor via Web Services Management Agent to on. 3. To report SOAP faults, disable error processing and enable error reporting in IBM Performance Management. On the Advanced Proxy Settings tab, set Process HTTP Errors to off, and click Apply. Configuring Multi-Protocol Gateway for transaction tracking Complete these steps on each Multi-Protocol Gateway where you plan to collect tracking data. 1. Select the domain of the Multi-Protocol Gateway. 2. On the Advanced tab for the Multi-Protocol Gateway, set the following values and click Apply: a. Set the Monitor via Web Services Management Agent to on. b. If the web server uses redirects, set Follow Redirects to off. Set the Rewrite Location URL to on. 3. If you are monitoring a Multi-Protocol Gateway with a response type or request type of Non- XML, you must define a Multi-Protocol Gateway Policy with rules that cover both client to server and server to client directions. If a non-xml Multi-Protocol Gateway does not have rules in its policy, no traffic is captured by either the Web Services Management agent or a DataPower debug probe, if enabled. 4. To propagate the HTTP response code from the back-end server and to report SOAP faults, on the Advanced Settings tab, set Process Backend Errors to off, and click Apply. Configure transforms to enable instance-level transaction tracking 1. To track REST traffic, complete the following steps: 2. Download the files from the following location: For Linux systems, /opt/ibm/apm/agent/lx8266/bn/bin For AIX systems, /opt/ibm/apm/agent/aix536/bn/bin

21 3. Upload the XSL files to each DataPower appliance that you want to monitor as part of the IBM integration stack. 4. Configure the Web Service Proxy or Multi-Protocol Gateway. 5. For each domain that you plan to monitor, complete the following steps: a. Select the Domain from the drop-down list in the DataPower Gateway header. b. In the Control Panel navigator, select Objects > Device Management > Web Services Management Agent. c. Set the Buffering Mode (deprecated) to Discard. d. Click Apply. Configure the Web Service Proxy for tracking instance topologies Complete these steps: 1. In the Configure Web Service Proxy page, select the name of the Web Service Proxy to configure. 2. On the Policy tab, expand proxy : domain and click Processing Rules. 3. In the Policy Configuration section, select an existing Client to Server rule, or click New Rule to create one. a. Drag a transform to the timeline. b. Double-click the transform to edit it. c. In the Configure Transform with XSLT style sheet Action window, next to Transform File, select apm_req.xsl from the data store that you uploaded it to. For example, local:///. If the file does not exist, click Upload to retrieve the file from the installed location. d. Click Done. 4. In the Policy Configuration section, repeat step 3 to configure a Server to Client rule, or click New Rule to create one. a. Drag a transform to the timeline. b. Double-click the transform to edit it. c. In the Configure transform with XSLT style sheet Action window, next to Transform File, select apm_rsp.xsl from the data store that you uploaded it to. For example, local:/// If the file does not exist, click Upload to retrieve the file from the installed location. d. Click Done. 5. In the Policy Configuration section, repeat step 3 to configure an Error rule, or click New Rule to create one. a. Drag a transform to the timeline. b. Double-click the transform rule to edit it. c. In the Configure transform with XSLT style sheet Action window, next to Transform File, select apm_error.xsl from the data store that you uploaded it to. For example, local:/// If the file does not exist, click Upload to retrieve the file from the installed location. d. Click Done. 6. In the Configure Web Service Proxy page, click Apply. Configure the Multi-protocol Gateway for tracking instance topologies Complete these steps: 1. On the Configure Multi-Protocol Gateway page, click the name of the Multi-Protocol Gateway that you plan to configure.

22 2. On the Multi-Protocol Gateway Policy page, configure the policy. Click. 3. On the Configure Multi-Protocol Gateway Style Policy page, select an existing Client to Server rule, or click New Rule to create one. a. Drag a Transform to the timeline. b. Double-click the Transform rule to edit it. c. In the Configure Transform with XSLT style sheet Action window, next to Transform File, select apm_req.xsl from the data store that you uploaded it to. For example, local:///. If the file does not exist, click Upload to get it from the installed location. d. Click Done. 4. On the Configure Multi-Protocol Gateway Style Policy page, select an existing Server to Client rule, or click New Rule to create one. a. Drag a Transform to the timeline. b. Double-click the Transform rule to edit it. c. In the Configure Transform with XSLT style sheet Action window, next to Transform File, select apm_rsp.xsl from the data store that you uploaded it to. For example, local:///. If the file does not exist, click Upload to get it from the installed location. d. Click Done. 5. On the Configure Multi-Protocol Gateway Style Policy page, select an existing Error rule, or click New Rule to create one. a. Drag a Transform to the timeline. b. Double-click the Transform rule to edit it. c. In the Configure Transform with XSLT style sheet Action window, next to Transform File, select apm_error.xsl from the data store that you uploaded it to. For example, local:/// If the file does not exist, click Upload to get it from the installed location. d. Click Done. 6. On the Configure Multi-Protocol Gateway Style Policy page, on the Advanced tab, set the Monitor via Web Services Management Agent to on, and click Apply. 7. Click Apply.

23 WebSphere Infrastructure Manager Installing the agent The WebSphere Infrastructure Manager agent is installed on the WebSphere Deployment Manager (dmgr) server. To configure the agent, complete these steps: 1. Enter:./wim-agent.sh config <instance_name> 2. Set the Java home directory. The default is /opt/ibm/apm/agent/jre/lx8266/jre. 3. Set the deployment manager profile home directory. The default is /opt/ibm/websphere/appserver/profiles/dmgr Enter your JMX user ID. The default is wasadmin. 5. Enter your JMX password.

24 Log file monitoring The log file monitoring capabilities of the OS agents are extremely flexible. The Config file (*.conf) controls which log files to process and how to process them. The Format file (*.fmt) controls the regex parsing of each row and how to represent the result set on the user interface. The capabilities of log file monitoring are: Handles rolling log file names. Handles log file paths and file names that are not static. Supports event throttling. Handles event deduplication. Handles log files that are normally tailed and log files that are rewritten at specific intervals. Monitors local or remote logs via ssh. Optionally configured to reread a portion or all of a log if the agent is restarted. Designed for ASCII logs, but can handle the AIX Error Report log. Allows for summarization of event data at specified intervals. For example, log entry XYZ occurred 15 times in the last 5 minutes. Supports the transmission of events directly from the agent to OMNIbus. Note: We recommend this approach due to the high-volume nature of log events. If you have a high volume of logs or if your REGEX expressions are poorly written, log file monitoring can consume much of your resources. The following are some tips to avoid performance problems. More details can be found in the following technote here. Focus on the logs that are writing the most data. Check the list of monitored logs on the user interface (see Figure 5). Figure 5 Monitored logs Avoid multi-line regex patterns (one containing \n or TEC style %n ) Order the regex expressions by the likelihood that they match an entry in the log. When a match is found, other regex expressions are not processed. The file lists the regex expressions in reverse order. The expressions at the bottom of the file are processed first.

25 FMT files The following are the characteristics of an FMT file. Matching is done from the bottom up. Once a row is matched, it stops processing. Include your most common expressions at the bottom. Items in parentheses () are mapped to $1, $2, $3 etc. For example: // Matches a syslog message indicating that the 'su' command succeeded for a user, like: REGEX RESuSucceeded FOLLOWS REGenericSyslog ^([A-Z][a-z]{2}) ([ 0-9][0-9]) ([0-9]{2}:[0-9]{2}:[0-9]{2}) (.*?) su: 'su (\S+)' succeeded for (\S+) on (\S+) Service "Su" target $5 CustomSlot1 CustomSlot2 user $6 CustomSlot3 tty $7 CustomSlot5 msg END PRINTF("SU: %s -> %s", user, target) // Matches made up disk error like: REGEX REThreeLineDiskError ^Error on disk device:\n((?:(?!cdrom).)*)\nerror Level: (.*)$ DiskDevice $1 CustomSlot1 ErrorLevel $2 CustomSlot10 msg END PRINTF("%s error on %s", ErrorLevel, DiskDevice) Tips for building a REGEX expression Here is an example entry from a WebSphere Log: [4/8/16 13:50:35:818 EDT] d1 webapp E com.ibm.ws.webcontainer.webapp.webapp logservleterror SRVE0293E: [Servlet Error]-[TradeAppServlet]: java.lang.nullpointerexception Typically, you parse a log into different elements so that each element displays in different slots. You might decide to parse the timestamp as a separate slot. You might also decide to parse the event severity. Elements are parsed using (). The parsing might result in a complex regular expression. Your REGEX might look like the following example: ^\[([0-9]{1,2}/[0-9]{1,2}/[0-9]{1,2} [0-9]{1,2}[0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2}:[0-9]{3} EST)\] (\S+) (.*)$ You should start small and build up your REGEX expression. Complete these steps: 1. Start with the first characters: ^\[([0-9]{1,2}/[0-9]{1,2}/[0-9]{1,2}.*$

26 2. If that parses properly, add a few more characters: ^\[([0-9]{1,2}/[0-9]{1,2}/[0-9]{1,2} [0-9]{1,2}[0-9]{1,2}.* 3. Continue until you have successfully parsed the string. Tip: Several websites are available where you can test your REGEX without having to use the agent to test it. For common log records that are not needed, expressly filter out these records with the *DISCARD* stanzas. Put this stanza near the bottom of the FMT file. To help you identify unmatched log records so that you can apply the *DISCARD* option, use UnmatchLog=/tmp/unmatch.log. The following is an example of a DISCARD statement: REGEX *DISCARD* ^.*Some common uninteresting message.*$ END Remember that the UnmatchLog file will continue to grow indefinitely. Once you finish developing your config and format files, remove the UnmatchLog to avoid large files. Use as much literal data as possible. Avoid.* wildcards because they force more processing. Here is an example: Log entries: Disk error on device: /dev/sda1 Disk error on device: /dev/sdb2 Option 1: ^Disk.*:.*$ (Poor) Option 2: ^Disk error on device: /dev/sd[a-b][0-9]$ (Good) Avoid using unnecessary sub-expressions: Sub-expressions (in parentheses) tell the regex engine that the value should be captured for future reference or future use. Example of a log entry: write failure in writing to client Error Broken pipe Example of an inefficient regular expression stanza in the FMT file: REGEX WriteFailure ^write failure in writing to client (.*)\. Error (.*)$ ClientAddr $1 CustomSlot1 END The log file agent captures the text after "Error" in the log message, but it is not used in any attribute value since only one assignment is made using $1. Capturing unused portions of the log can negatively impact performance. If you must use parentheses for grouping, but not for capture, use the?: operator that instructs the regex engine not to capture the value. Begin your regular expressions with ^ and end with $. For Windows systems event log monitoring, PreFilter and PreFilterMode are effective at decreasing the utilization of the agent.

27 Do not use the OR ( ) operator. The OR ( ) operator can cause the regex engine to revert back and to match entries that did not initially match. This is often dramatically less efficient than simply having two separate expressions. Example of using inefficient syntax REGEX DiskError ^.*disk error.*$ ^.*disk failure.*$ END Example of using more efficient syntax REGEX DiskError ^.*disk error.*$ END REGEX DiskError ^.*disk failure.*$ END The log file agent uses an implementation of the ICU regex processor. For more tips, see the generic tips here. These tips can be applied to the regular expressions of the log file agents. For regular expression performance tips, see the "Performance Tips" section here. Some sample config and format files in /root and on the Linux desktop. A sample of WAS SystemOut log, Linux syslog, and Windows Event log are available for download from IBM DeveloperWorks here.

28 Notices This information was developed for products and services offered in the US. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-ibm product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive, MD-NC119 Armonk, NY US For license inquiries regarding double-byte character set (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to: Intellectual Property Licensing Legal and Intellectual Property Law IBM Japan Ltd , Nihonbashi-Hakozakicho, Chuo-ku Tokyo , Japan INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. IBM may use or distribute any of the information you provide in any way it believes appropriate without incurring any obligation to you.

29 Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact: IBM Director of Licensing IBM Corporation North Castle Drive, MD-NC119 Armonk, NY US Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee. The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to actual people or business enterprises is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs are provided "AS IS", without warranty of any kind. IBM shall not be liable for any damages arising out of your use of the sample programs.