Hadoop Integration User Guide. Functional Area: Hadoop Integration. Geneos Release: v4.9. Document Version: v1.0.0

Transcription

1 Hadoop Integration User Guide Functional Area: Hadoop Integration Geneos Release: v4.9 Document Version: v1.0.0 Date Published: 25 October 2018

2 Copyright ITRS Group Ltd. All rights reserved. Information in this document is subject to change without notice. The software described in this document is furnished under a license agreement or nondisclosure agreement. The software may be used or copied only in accordance with the terms of those agreements. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or any means electronic or mechanical, including photocopying and recording for any purpose other than the purchaser's personal use without the written permission of ITRS Group Ltd. ITRS Group Ltd 6th Floor, The Bonhill Building, 15 Bonhill Street, London, EC2A 4DN, UK t: +44 (0)

3 Table of Contents Hadoop Monitoring User Guide 4 General Information 4 Overview 4 User Requirements 4 System Requirements 5 Audience 5 Install And Set Up 6 Set Up The Samplers 8 Set Up The Variables 9 Set Up The Rules 10

4 Hadoop Monitoring User Guide General information Overview Hadoop monitoring is a Gateway configuration file that enables monitoring of the Hadoop cluster, nodes, and daemons through the JMX and Toolkit plug-ins. This Hadoop integration template consists of the following components: Hadoop Distributed File System (HDFS) Yet Another Resource Negotiator (YARN) The Hadoop Distributed File System or HDFS provides scalable data storage that can be deployed on hardware and optimised operations for large datasets. The other component Yet Another Resource Negotiator or YARN assigns the computation resources for executing the application: YARN ResourceManager - takes inventory of available and allocate resources to running applications. YARN NodeManagers - monitors resource usage and communicates with the ResourceManager. This guide discusses the steps to set up the Hadoop integration on a Gateway. Once the integration is set up, the samplers providing the dataviews become available to that Gateway. To view the sample metrics and dataviews, see Hadoop Monitoring Technical Reference. User requirements This monitoring template is a Gateway configuration setup that can be included in the Gateway Setup Editor. To use this template, your configuration must meet the following requirements: Set up the HDFS with at least one DataNode. A machine running the Netprobe must have access to the Hadoop published HTTP address and port. An Active Console 2 that is connected to the Gateway. Basic connectivity setup for JMX or Toolkit plug-in. Hadoop configuration files extracted to their Gateway set-up. Page 4 of 12

5 System requirements The following requirements must be met prior to the installation and setup of the template: Template package: geneos-integration-hadoop-<version>.zip. Managed entities utilising the samplers defined in the include/hadoopmonitoring.xml. Netprobe version 4.5. Apache Hadoop or higher. Python 2.7 or higher. Audience This document is a reference guide to introduce templates and scripts built using the Geneos development toolkit plug-ins. The template allows you to integrate specific applications and services to collect metrics without having to create a new plug-in. Administrator - oversees the administration and installation of the server. User - uses the Hadoop metrics via JMX plug-in to monitor the data and reports. Application Team - provides the connection settings for the JMX plug-in and queries for the Toolkit plug-in. Page 5 of 12

6 Install and set up Ensure that you have read and can follow the system requirements prior to installation and setup of this integration template. 1. Download the config file (geneos-integration-hadoop-<version>.zip) from the ITRS Downloads site. 2. Open Active Console Extract the file into the Gateway Setup directory. 4. In the Navigation panel, click Includes to create a new file. 5. Enter the location of the file to include in the Location field. In this example, use the include/hadoopmonitoring.xml: 6. Expand the file location in the Includes section. 7. Select Click to load Click Yes to load the new Hadoop include file. 9. Click Managed entities in the Navigation panel. 10. Add the Hadoop-Cluster and Hadoop-Node types to the Managed Entity section that you will use to monitor Hadoop. Page 6 of 12

7 11. Click the Validate button to check your configuration. The Validate button allows you to check if there are any errors or warnings in your configuration setup. Once the Gateway configuration appears in the Includes section, you can add the samplers and other variables. Page 7 of 12

8 Set up the samplers These are the pre-configured samplers available to use in HadoopMonitoring.xml. Configure the required fields by referring to the table below: Samplers Hadoop-HDFS-NamenodeInfo Hadoop-HDFS-NamenodeCluster Hadoop-HDFS-SecondaryNamenodeInfo Hadoop-HDFS-DatanodesSummary Hadoop-HDFS-DatanodeVolumeInfo Hadoop-YARN-ResourceManager Hadoop-YARN-NodeManagersSummary Page 8 of 12

9 Set up the variables The HadoopMonitoring.xml template provides the variables that are set in the Environments section. Samplers HADOOP_HOST_NAMENODE HADOOP_HOST_SECONDARYNAMENODE HADOOP_HOST_DATANODE HADOOP_HOST_RESOURCEMANAGER HADOOP_PORT_JMX_NAMENODE HADOOP_PORT_JMX_ SECONDARYNAMENODE HADOOP_PORT_WEBJMX_DATANODE Description IP/Hostname where Namenode daemon is running. IP/Hostname where Secondarynamenode daemon is running. IP/Hostname where the specific Datanode daemon is running. IP/Hostname where ResourceManager is running. Namenode JMX port. Secondarynamenode JMX port. Datanode web UI port. Default: 9864 HADOOP_PORT_JMX_ RESOURCEMANAGER ResourceManager JMX port HADOOP_PORT_WEBJMX_NAMENODE Namenode UI port. Default: 9870 HADOOP_PORT_WEBJMX_ RESOURCEMANAGER ResourceManager web UI port. Default: 8088 PYTHON_EXECUTABLE_PATH Script that runs the Python program. After checking and saving the changes, the samplers you have set in the Gateway configuration display in Active Console 2. Page 9 of 12

10 Set up the rules The HadoopMonitoring-SampleRules.xml template also provides a separate sample rules that you can use to configure the Gateway Setup Editor. Your configuration rules must be set in the Includes section. 1. Enter the location of the file to include in the Location field. In this example, use the include/hadoopmonitoring-samplerules.xml: The priority controls the importance of a file when merging. Sections in a higher priority file will take precedence over sections in a lower priority file. This priority setting affects the priority of configuration in the main setup file. 2. Expand the file location in the Includes section. 3. Select Click to load Click Yes to load the new Hadoop include rules file. Page 10 of 12

11 5. Click Rules in the Navigation panel to create new rules. Sample Rules Hadoop-NameNodeCluster-Disk- Remaining Hadoop-DataNode-Disk-Remaining Hadoop-Datanodes-In-Errors Hadoop-Blocks-In-Error Hadoop-Nodemanager-In-Error: Hadoop-Applications-In-Error: Hadoop-SecondaryNamenode- Status : Hadoop-NodeManager-State: Description Checks the remaining disk ratio of the entire Hadoop cluster. Checks the remaining disk ration of a single datanode HADOOP_RULE_DISK_REMAINING_THRESHOLD: Possible values Checks the number of nodes with errors HADOOP_RULE_DATANODES_ERROR_THRESHOLD: Integer values. Checks the number of blocks with error HADOOP_RULE_BLOCKS_ERROR_THRESHOLD: Integer values. Checks the number of nodemanagers with error HADOOP_RULE_NODEMANAGER_ERROR_THRESHOLD: Integer value. Checks the number of application with error HADOOP_RULE_APPLICATION_ERROR_THRESHOLD: Integer values. Checks the connection status of JMX plugin to Secondarynamenode service. Checks the state of nodemanager HADOOP_RULE_NODEMANAGER_UNHEALTHY: Default: UNHEALTY Page 11 of 12

12 Note: The sample configuration file is verified to be working with Apache Hadoop 3.0.0, Python 2.7, and Netprobe 4.5 version. Once the Gateway for rules configuration appears in the SampleRules Includes section, you can set the rules and alerts. Page 12 of 12