Greenplum Command Center 1.2 Administrator Guide. P/N: Rev: A02

Transcription

1 Greenplum Command Center 1.2 Administrator Guide P/N: Rev: A02

2 Copyright 2012 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com All other trademarks used herein are the property of their respective owners.

3 Greenplum Command Center Administrator Guide 1.2 Contents Greenplum Command Center Administrator Guide 1.2 Contents Preface... 1 About This Guide... 1 Document Conventions... 2 Text Conventions... 2 Command Syntax Conventions... 3 Getting Support... 3 Product information... 3 Technical support... 3 Chapter 1: About Greenplum Command Center... 4 Supported Greenplum Platforms... 4 Greenplum Data Collection Agents... 5 Greenplum Command Center Database... 6 Greenplum Command Center Console... 6 Greenplum Command Center Web Service... 7 Chapter 2: Setting Up Greenplum Command Center... 8 Data Computing Applicance (DCA) Environments... 8 Greenplum Database Software Environments... 8 Enabling the Data Collection Agents... 8 Installing the Greenplum Command Center Console...10 Setting up the Command Center Console...11 Connecting to the Greenplum Command Center Console...13 Configuring Authentication for the Command Center Console...13 About the Command Center Installation...15 Uninstalling Greenplum Command Center...16 Chapter 3: Administering Command Center...17 Starting and Stopping Greenplum Command Center...17 Starting and Stopping Command Center Agents...17 Starting and Stopping Command Center Console...17 Configuring Greenplum Command Center...17 Command Center Agent Administration...18 Adding and Removing Hosts...18 Viewing and Maintaining Master Agent Log Files...18 Command Center Database Administration...19 Connecting to the Command Center Database...19 Backing Up and Restoring the Command Center Database...19 Maintaining the Historical Data Tables...19 Web Server Administration...20 Configuring the Web Server...20 Viewing and Maintaining Web Server Log Files...20 Chapter 4: Accessing Command Center Data...21 Accessing Command Center Data using SQL...21 Viewing Historical System s using SQL...21 Viewing Historical Query s using SQL...22 Appendix A: Integration with Enterprise Monitoring Tools...24 Table of Contents iii

4 Greenplum Command Center Administrator Guide 1.2 Contents Accessing Command Center Data using SNMP...24 SNMP on the Greenplum Data Computing Appliance...24 SNMP on Greenplum Database...25 Appendix B: Command Center Utility Reference...26 gpperfmon_install...26 gpcmdr...28 Appendix C: Command Center Configuration File Reference...30 Command Center Agent Parameters...30 Command Center Console Parameters...32 Web Application Parameters (gpperfmonui.conf)...32 Web Server Parameters (lighttpd.conf)...32 Greenplum Database Server Configuration Parameters...33 Appendix D: Command Center Database Reference...34 database_*...36 emcconnect_history...37 filerep_*...39 health_*...43 interface_stats_*...44 iterators_*...46 Iterator s...50 Append...51 Append-only Scan...51 Append-only Columnar Scan...51 Aggregate...52 BitmapAnd...52 BitmapOr...52 Bitmap Append-Only Scan Bitmap Append-Only Scan...53 Bitmap Heap Scan...53 Bitmap Index Scan...53 Broadcast Motion...53 Explicit Redistribute Motion...54 External Scan...56 Function Scan...56 Gather Motion...56 GroupAggregate...57 Hash Join...58 HashAggregate...58 Index Scan...59 Limit...59 Materialize...60 Merge Join...60 Nested Loop...61 Redistribute Motion...61 Result...62 Repeat...63 Seq Scan...63 Table of Contents iv

5 Greenplum Command Center Administrator Guide 1.2 Contents SetOp...63 Shared Scan...64 Sort...64 Subquery Scan...65 Tid Scan...65 Unique...65 Values Scan...66 Window...66 queries_*...67 segment_*...69 socket_stats_*...70 system_*...71 tcp_stats_*...73 udp_stats_*...74 iterators_*_rollup...75 dynamic_memory_info...78 memory_info...79 Table of Contents v

6 Greenplum Command Center Administrator Guide 1.2 Preface Preface This guide provides information about Greenplum Command Center, a management tool for the Greenplum Big Data Platform. About This Guide Document Conventions Getting Support About This Guide This guide provides information and instructions for installing and using Greenplum Command Center. It also provides installation instructions for Greenplum Command Center Console (the graphical web application for viewing Command Center data). For information about using the Console and details about the system and query metrics you can view and administrative tasks you can perform with this optional web-based application, click the Help link in the Console user interface. This guide assumes knowledge of Linux/Unix system administration, database management systems, database administration, and structured query language (SQL). Chapter 1, About Greenplum Command Center explains the general architecture and functionality of Greenplum Command Center. Chapter 2, Setting Up Greenplum Command Center provides instructions for setting up the data collection agents within the Greenplum environment, and for installing the web application (Command Center Console). Chapter 3, Administering Command Center provides information on system, server, and database administration of a Greenplum Command Center installation. Appendix A, Integration with Enterprise Monitoring Tools is a brief overview of accessing Greenplum Command Center data using SNMP. Appendix B, Command Center Utility Reference is a reference for the setup and management utilities of Greenplum Command Center (data collection agents and web application). Appendix C, Command Center Configuration File Reference is a reference of the configuration parameters for Greenplum Command Center. Appendix D, Command Center Database Reference is a reference of the gpperfmon database schema. About This Guide 1

7 Greenplum Command Center Administrator Guide 1.2 Preface Document Conventions The following conventions are used throughout the Greenplum Database documentation to help you identify certain types of information. Text Conventions Command Syntax Conventions Table 1 Text Conventions Text Conventions Text Convention Usage Examples bold italics monospace monospace italics monospace bold UPPERCASE Button, menu, tab, page, and field names in GUI applications New terms where they are defined Database objects, such as schema, table, or columns names File names and path names Programs and executables Command names and syntax Parameter names Variable information within file paths and file names Variable information within command syntax Used to call attention to a particular part of a command, parameter, or code snippet. Environment variables SQL commands Keyboard keys Click Cancel to exit the page without saving your changes. The master instance is the postgres process that accepts client connections. Catalog information for Greenplum Database resides in the pg_catalog schema. Edit the postgresql.conf file. Use gpstart to start Greenplum Database. /home/gpadmin/config_file COPY tablename FROM 'filename' Change the host name, port, and database name in the JDBC connection URL: jdbc:postgresql://host:5432/m ydb Make sure that the Java /bin directory is in your $PATH. SELECT * FROM my_table; Press CTRL+C to escape. Document Conventions 2

8 Greenplum Command Center Administrator Guide 1.2 Preface Command Syntax Conventions Table 2 Command Syntax Conventions Text Convention Usage Examples { } Within command syntax, curly braces group related command options. Do not type the curly braces. [ ] Within command syntax, square brackets denote optional arguments. Do not type the brackets.... Within command syntax, an ellipsis denotes repetition of a command, variable, or option. Do not type the ellipsis. Within command syntax, the pipe symbol denotes an OR relationship. Do not type the pipe symbol. FROM { 'filename' STDIN } TRUNCATE [ TABLE ] name DROP TABLE name [,...] VACUUM [ FULL FREEZE ] $ system_command # root_system_command => gpdb_command =# su_gpdb_command Denotes a command prompt - do not type the prompt symbol. $ and # denote terminal command prompts. => and =# denote Greenplum Database interactive program command prompts (psql or gpssh, for example). $ createdb mydatabase # chown gpadmin -R /datadir => SELECT * FROM mytable; =# SELECT * FROM pg_database; Getting Support EMC support, product, and licensing information can be obtained as follows. Product information All Greenplum documentation is available for download from Powerlink. Technical support For technical support, go to Powerlink and choose Support. On the Support page, you will see several options, including one for making a service request. Note that to open a service request, you must have a valid support agreement. Please contact your EMC sales representative for details about obtaining a valid support agreement or with questions about your account. Getting Support 3

9 Greenplum Command Center Administrator Guide 1.2 Chapter 1: About Greenplum Command Center 1 About Greenplum Command Center Greenplum Command Center is a management tool for the Greenplum Big Data Platform. Greenplum Command Center monitors system performance metrics, system health, and also provides administrators the ability to perform management tasks such as start, stop, and recovery of systems in a Greenplum environment. The Greenplum Command Center Console is an interactive graphical web application that can optionally be installed on a web server on the master host, and used to view and interact with the collected Greenplum system data. Greenplum Command Center is comprised of data collection agents that run on the master host and each segment host (either Greenplum Hadoop or Greenplum Database hosts). The agents collect data about queries and system utilization and send it to the Greenplum master host at regular intervals. Greenplum Command Center stores its data and metrics in a dedicated Greenplum database (the Command Center database) whose information is distributed among the master host and segment hosts, like any other Greenplum Database. The data stored in the Greenplum Command Center database can be accessed through the Greenplum Command Center Console or through SQL queries. Note: Greenplum Database is a requirement for operating Command Center (Command Center stores its information in a Greenplum database). Supported Greenplum Platforms Greenplum Command Center is currently certified for the Greenplum Data Computing Appliance (DCA) and Greenplum Database software-only environments. Command Center monitors the following for each environment: Greenplum Data Computing Appliance: Greenplum Database Module (version 4.0 and higher supported, or higher recommended) Greenplum Hadoop (HD) Module (version 1.1 and above) Greenplum Data Integration Accelerator (DIA) Module Greenplum Data Computing Appliance Hardware Greenplum Database (software-only environments): Greenplum Database (4.0 and higher supported, or higher recommended) 4

10 Greenplum Command Center Administrator Guide 1.2 Chapter 1: About Greenplum Command Center Figure 1 Greenplum Command Center Architecture Note: The Hadoop nodes in the above diagram are only applicable to Data Computing Appliance environments. Greenplum Data Collection Agents Command Center can be enabled or disabled using the gp_enable_gpperfmon server configuration parameter. Once enabled, data collection agents run on all Greenplum hosts (master and segments), and are started and stopped along with the Greenplum Database server processes. The master agent polls all segment agents for system metrics and other data at a configurable interval (known as the quantum). The master agent pulls together the data from all segments, stores it in flat files, and periodically commits the data in the files to the Greenplum Command Center database. Greenplum Data Collection Agents 5

11 Greenplum Command Center Administrator Guide 1.2 Chapter 1: About Greenplum Command Center Greenplum Command Center Database The Greenplum Command Center database (gpperfmon) is a database within your Greenplum system dedicated to storing and serving system data. Your Greenplum installation includes setup scripts to install the Command Center database (gpperfmon). Note: It should be noted that when this document refers to the Command Center database, it is referring to the database named gpperfmon. Greenplum administrators can connect to the Command Center database using client programs such as psql or application programming interfaces (APIs) such as JDBC (Java Database Connectivity) and ODBC (Open Database Connectivity). Administrators can also use the Greenplum Command Center Console to view reports on current and historical performance and perform other managment tasks. The Command Center database consists of three sets of tables; now tables store data on current system metrics such as active queries, history tables store data on historical metrics, and tail tables are for data in transition. Tail tables are for internal use only and should not be queried by users. The now and tail data are stored as text files on the master host file system, and accessed by the Command Center database via external tables. The history tables are regular database tables stored within the Command Center (gpperfmon) database. See Appendix D, Command Center Database Reference for the schema definitions of these tables. Greenplum Command Center Console Greenplum Command Center provides a graphical console for viewing Greenplum System metrics and performing certain database administrative tasks. This browser-based application provides the following functionality: Database administrative Controls Ability to stop/start the database Ability to recover / rebalance segments An interactive View of System s Realtime Historic (configurable by time) An Interactive View of System Hardware Health This functionality is only available for Greenplum Data Computing Appliance environment. Database Query Monitoring Ability to view, search, prioritize, or cancel any query in the system. Ability to view the internals of any query, including the query plan, query plan iterator-level details, and real-time information about individual scans and joins. Database Workload Management Greenplum Command Center Database 6

12 Greenplum Command Center Administrator Guide 1.2 Chapter 1: About Greenplum Command Center Ability to configure resource queues Ability to prioritize users Figure 2 Greenplum Command Center Dashboard If you have multiple Greenplum environments, you can create separate Command Center instances for them. Each separate console instance operates on a unique port and has its own unique configuration options. For more information, see Installing the Greenplum Command Center Console on page 10. Greenplum Command Center Web Service The Greenplum Command Center Console queries the Command Center database through a web service framework composed of a lightweight lighttpd web server and Python-based middleware. The lighttpd server is an open-source web server with a low memory footprint and light CPU load. For more information, see The console setup utility sets up the lighttpd web server and web service, prompting you for basic configuration information on the desired port and SSL options. Under normal conditions, the web server and web service API require minimal maintenance and administration, as described in Web Server Administration on page 20. Greenplum Command Center Console 7

13 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center 2 Setting Up Greenplum Command Center This chapter describes how to install and set up your Greenplum Command Center system. Data Computing Applicance (DCA) Environments The Greenplum Data Computing Appliance (DCA) version of the Greenplum Command Center is already installed on the appliance (versions 1.2 and above). EMC Services will work with you to further configure Command Center for your environment. For more information about setting up and configuring Greenplum Command Center on a Greenplum DCA, refer to the Greenplum Data Computing Appliance Software Upgrade Guide x > , and the Greenplum Data Computing Appliance Installation and Configuration Guide, Greenplum Database Software Environments If you are using Greenplum Command Center in a software-only environment, installing and enabling Greenplum Command Center is a two-step process. First you must create the Greenplum Command Center database (gpperfmon) and enable the data collection agents within your Greenplum system. After data collection is enabled, the next (optional) step is to install and configure the Greenplum Command Center Console (the Web application used to view the Command Center data stored in Greenplum Database). Greenplum Command Center software for Greenplum Database software-only environments is available as a separate download. Download the installer file from the EMC Download Center. Installer files are available for RedHat 64-bit or SuSE Linux 64 bit platforms. This section contains instructions to perform the following tasks: Enabling the Data Collection Agents Installing the Greenplum Command Center Console Uninstalling Greenplum Command Center Enabling the Data Collection Agents This section describes how to create the Command Center database and enable the Command Center data collection agents. When the data collection agents are enabled, their processes are started and stopped along with the Greenplum Database server processes (using gpstart and gpstop). Greenplum provides a gpperfmon_install utility that performs the following tasks: Creates the Command Center database (gpperfmon). Data Computing Applicance (DCA) Environments 8

14 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center Creates the Command Center superuser role (gpmon). Configures Greenplum Database server to accept connections from the Command Center superuser role (edits the pg_hba.conf and.pgpass files). Sets the Command Center server configuration parameters in the Greenplum Database server postgresql.conf files. Enable Command Center Agents 1. Log in to the Greenplum master host as the gpadmin user. $ su - gpadmin 2. Source the path file from your master host s Greenplum Database installation directory: # source /usr/local/greenplum-db/greenplum_path.sh 3. Run the gpperfmon_install utility with the --enable option. You must supply the connection port of the Greenplum Database master server process, and set the password for the gpmon superuser that will be created. For example: $ gpperfmon_install --enable --password p@$$word --port When the utility completes, restart Greenplum Database server. The data collection agents will not start until the database is restarted: $ gpstop -r 5. Using the ps command, verify that the data collection process is running on the Greenplum master. For example: $ ps -ef grep gpmmon 6. Run the following command to verify that the data collection processes are writing to the Command Center database. If all of the segment data collection agents are running, you should see one row per segment host. $ psql gpperfmon -c 'SELECT * FROM system_now;' The data collection agents are now running, and your Greenplum system now has a gpperfmon database installed. This is the database where Command Center data is stored. You can connect to it as follows: $ psql gpperfmon To configure a standby master host (if enabled) 7. Copy the $MASTER_DATA_DIRECTORY/pg_hba.conf file from your primary master host to your standby master host. This ensures that the required connection options are also set on the standby master. 8. Copy your ~/.pgpass file from your primary master host to your standby master host. This file usually resides in the gpadmin user s home directory. Note that the permissions on.pgpass must be set to 600 (for example: chmod 0600 ~/.pgpass). Greenplum Database Software Environments 9

15 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center Installing the Greenplum Command Center Console The Command Center Console provides a graphical interface for viewing performance data and for administering certain aspects of your Greenplum system. Normally installed on the Greenplum master host, the console is a web-based user interface accessed through a supported browser. The Command Center Console is typically installed on the Greenplum Database master host. However, you have the option to install the console on a host different from the master host. Note that this setup incurs a performance penalty due to the numerous database connections the console must open over the network. If you have multiple Greenplum Database instances, you can create separate Command Center Console instances for each of them. Each separate console instance operates on a unique port and has its own unique configuration options. The Command Center Console supports for any browsers that have Adobe Flash 9.0 or higher enabled. For example, the following browsers are supported: Internet Explorer for Windows XP and Vista Mozilla Firefox for Windows and Linux Apple s Safari browser for Macintosh Google s Chrome The Command Center Console runs on a lighttpd web server. The default web server port is For more information about the web server, see Web Server Administration on page 20. Installing the Command Center Console involves the following high-level tasks: Install the Command Center Console Create the software installation directory. Set up the Greenplum Command Center Console Set up the environment variables and create and configure a Command Center Console instance and its supporting web services. Install the Command Center Console If you are installing the Command Center Console on a remote system (that is, not the same system on which you installed Greenplum Database), you must also install Greenplum Database installation binary files on the remote system. After installing the binary files, source greenplum_path.sh, then perform the Console installation steps described below. Note that you do not need to initiate the database. See the Greenplum Database Installation Guide for more information. 1. Download the installer file from the EMC Download Center. Installer files are available for RedHat 64-bit or SuSE Linux 64 bit platforms. You do not need to download the installer file if you are installing the console on an EMC Data Computing Appliance; the installer file is already loaded on DCAs. 2. Unzip the installer file where PLATFORM is either RHEL5-x86_64 (RedHat 64-bit), SOL-x86_64 or SuSE10-x86_64 (SuSe Linux 64-bit). For example: # unzip greenplum-cc-web-versionx.x-platform.zip Greenplum Database Software Environments 10

16 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center 3. Login in as gpadmin. 4. Launch the installer using bash. For example: # /bin/bash greenplum-cc-web-versionx.x-platform.bin 5. Type yes to accept the license agreement. 6. The installer will prompt you to provide an installation path. Press ENTER to accept the default install path: (/usr/local/greenplum-cc-web-versionx.x) or enter an absolute path to an install location. You must have write permissions to the location you specify. 7. Once the installation has completed successfully, create a host file listing all remaining hostnames, including the Standby Master host. Note that hostnames must be DNS resolveable. 8. The installation directory contains a gpcc_path.sh file with path and environment settings for the Console. Source this and the Greenplum path, as follows: $ source /usr/local/greenplum-db/greenplum_path.sh $ source /usr/local/greenplum-cc-web-versionx.x/gpcc_path.sh 9. Run the gpccinstall utility to install Command Center on all hosts listed in the host file you created. $ gpccinstall -f hostfilename 10. Configure the Console as described in Setting up the Command Center Console. Setting up the Command Center Console The gpcmdr utility sets up the Command Center Console on the current host. On hosts other than the Greenplum master host, the console experiences slower performance due to frequent connections to the gpperfmon database. Set up the Command Center Environment Variable During the setup process, the utility prompts you for values to configure Console connections to a single Greenplum Database instance. To configure connections to multiple Greenplum Database instances, run the setup routine multiple times. To accept the displayed default values for any of these parameters at configuration time, hit the ENTER key. Set up the Greenplum Command Center Console 1. Log in as the Greenplum administrator (gpadmin). 2. Stop the Greenplum Database. 3. Add the following environmental variable in your user s startup shell profile (such as ~/.bashrc) GPPERFMONHOME the Command Center home directory Greenplum Database Software Environments 11

17 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center For example: GPPERFMONHOME=/usr/local/greenplum-cc-web-versionx.x source $GPPERFMONHOME/gpcc_path.sh Ensure that this file has entries for greenplum_path.sh file and MASTER_DATA_DIRECTORY. See the Greenplum Database Administrator Guide for details. 4. Save and source the.bashrc file. 5. Start the Database. 6. With the Greenplum Database instance running, launch the setup utility. For example: $ gpcmdr --setup 7. Provide an instance name for the Greenplum Database instance monitored by this Console. To monitor multiple instances, you must run the setup utility separately for each instance. 8. Select y or n to specify if the Greenplum Database master for this instance is on a remote host. Note that Console performance is better when the Console and Greenplum Database master are on the same host. If the master host is remote, enter y and enter the hostname of the master at the prompt. 9. Provide a display name for the instance. This name is shown in the Console user interface. This prompt does not appear if the master host is remote. 10. Provide the port for the Greenplum Database master instance. 11. Provide a port number for the Command Center Console web server. The default is (Optional) Enter y or n to set up SSL connections for the Command Center Console. If you enter Y, you are prompted for the following distinguished name (DN) information used to create an unsigned certificate for the Command Center Console. For example: Country Name (2 letter code) [GB]:US State or Province Name (full name) [Berkshire]:California Locality Name (eg, city) [Newbury]:San Mateo Organization Name (eg, company) [My Company Ltd]:Greenplum Organizational Unit Name (eg, section) []:Engineering Common Name (eg, your name or your server's hostname) []:mdw1 Address []:gpadmin@greenplum.com Note: Because database login information is sent over the network, Greenplum recommends using SSL to encrypt these communications. Greenplum Database Software Environments 12

18 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center 13. Enter y to enable IPv6 support. IPv6 support is disabled by default. 14. Enter y or n to specify whether you want this installation copied to a Standby Master. If you enter Y, you are subsequently prompted for the Standby Master host name. 15. Start and log into the Console. See Connecting to the Greenplum Command Center Console. You can also configure authentication so that other Greenplum users can log into the Console, as described in Configuring Authentication for the Command Center Console on page 13. Connecting to the Greenplum Command Center Console Start the Greenplum Command Center Console by entering: gpcmdr --start If you do not specify an instance name, all Command Center Console instances are started. To start a particular instance, you can specify the name of the instance. For example: gpcmdr --start "instance_name" See Starting and Stopping Greenplum Command Center on page 17 for a complete list of commands. After the instance is running, you can open the Command Center Console in a supported browser using the correct hostname and port. For example, to open a Command Center instance running on port on the local host with SSL, use the following web address: At the login prompt, enter the user name and password of a Greenplum role that has been properly configured to allow authentication to Greenplum Command Center (for example, gpmon), then click Login. This opens the Dashboard page of the Command Center Console, which provides a graphical system snapshot and a summary view of active queries. See the Command Center Console online help for more information. You must be a Greenplum administrator to fully use the Greenplum Command Center Console. Administrators can view information for all queries as well as system metrics, while regular database users can only monitor their own queries. Configuring Authentication for the Command Center Console When you installed Greenplum Command Center database and enabled the data collection agents, a gpmon superuser was created by the installation utility. This is the Greenplum role used to manage the Command Center components and data within the Greenplum environment. The gpmon role is configured to use md5-encrypted password authentication to connect to the Greenplum Database instance. Greenplum Database Software Environments 13

19 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center Typically, you will not be connecting to the Command Center Console as gpmon, and instead connect as another Greenplum user. The Command Center Console is configured by default to require md5-encrypted password authentication, so make sure the Greenplum role has an md5-encrypted password set. Note: If you are using Greenplum Database version or higher, you have the option of using SHA-256-encrypted password authentication. You can specify SHA-256 authentication by changing the password_hash_algorithm server parameter. This parameter can be set either system-wide or on a session level. The Command Center administrative user, gpmon, is created automatically by the dca_setup utility on the Data Computing Appliance, and by the gpperfmon_install utility in a Greenplum Database software-only environment. Any other Greenplum Database users with appropriate privileges can access Command Center. To create a new Command Center user, first you have to create a Greenplum Database user, then edit the pg_hba.conf file to give that user access to Command Center. The following is an example of the steps necessary to create a new user for Command Center. This example uses the psql command line to create a user with read-only privileges. See the Greenplum Database Administrator Guide for more detailed information about creating database users and roles. 1. On the Master host, login as gpadmin. 2. Start psql: $psql 3. Enter the create role command to create a read-only user: # create role CC_user with login password new_password ; To create a role with superuser privileges: # create role CC_user with login password new_password ; superuser createdb 4. Verify that the user was created successfully using the following command: # \du 5. The new user you just created should be returned, along with the attributes you specified. 6. Exit psql. 7. Edit the pg_hba.conf file to give the new user access to Command Center: $ vi $MASTER_DATA_DIRECTORY/pg_hba.conf Scroll to the bottom of the file and insert the following text to give the new user, CC_user, access from any IP address using password authentication: host all CC_user /28 md5 8. Save the file and exit the editor. 9. Enter the following command to reload Greenplum Database processes. Greenplum Database Software Environments 14

20 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center Using SSL Encryption If you enable SSL at setup time, the installer creates a self-signed certificate and uses OpenSSL encryption for all connections to the Command Center web server. Because this is a self-signed certificate, supported browsers may need to be configured to accept or allow it as a security exception. This does not detract from the added security gained from encrypting communications between the browser client and the web server. Optionally, you can obtain and use a certificate signed by a trusted certificate authority such as Verisign. If you use a trusted certificate, edit the lighttpd.conf file (located in $GPPERFMONHOME/instances/instance_name/conf), and set the ssl.pemfile to point to the location of the certificate. For example: ssl.pemfile = "$GPPERFMONHOME/instances/instance_name/conf/cert.pem" For more information on the lighttpd.conf file, see Web Server Administration on page 20. About the Command Center Installation The installation and setup procedures create a software installation directory and a directory containing files and folders to support each Greenplum Command Center Console instance. Software Installation Directory The following files and first-level subdirectories are copied into the installation folder that you specified when you installed Greenplum Command Center Console. This location is referred to as $GPPERFMONHOME. gpcc_path.sh file containing environment variables for Command Center bin program files for Greenplum Command Center etc contains openssl.conf file ext Python directory and files instances contains a subdirectory of resources for each Greenplum Database instance monitored by the console lib library files for Greenplum Command Center www web service and user interface files Instances Directory The $GPPERFMONHOME/instances directory contains subdirectories named for each instance created during console setup. The conf subdirectory contains configuration files that you can edit. Other files and folders are used by the web services for the instance, and should not be modified or deleted. Each subdirectory contains the following files and first-level subdirectories: lighttpd.pid -- file containing an updated process ID for the web server process for the instance conf console and web server configuration files, gpperfmonui.conf and lighttpd.conf Greenplum Database Software Environments 15

21 Greenplum Command Center Administrator Guide 1.2 Chapter 2: Setting Up Greenplum Command Center logs logs for the web server for this instance perfmon.fastcgi.socket-0 dynamically updated socket information, which cannot be viewed or updated sessions files storing information about session state tmp temporary files used by the web server web symbolic links to web server files in the installation directory Uninstalling Greenplum Command Center To uninstall, you must stop both the Command Center Console and disable the data collection agents. Optionally, you may also want to remove any data associated with Greenplum Command Center by removing your Command Center Console installation and the Command Center database. To uninstall the Command Center Console 1. Stop Command Center Console if it is currently running. For example: $ gpcmdr --stop 2. Remove the Command Center installation directory from all hosts. For example: $ rm -rf /usr/local/greenplum-cc-web-version To disable the Data Collection Agents 1. Log in to the master host as the Greenplum administrative user (gpadmin): su - gpadmin 2. Edit $MASTER_DATA_DIRECTORY/postgresql.conf and disable the data collection agents: gp_enable_gpperfmon = off 3. Remove or comment out the gpmon entries in pg_hba.conf. For example: #local gpperfmon gpmon md5 #host gpperfmon gpmon /0 md5 4. Drop the Command Center superuser role from the database. For example: $ psql template1 -c 'DROP ROLE gpmon;' 5. Restart the Greenplum Database instance: $ gpstop -r 6. Clean up any uncommitted Command Center data and log files that reside on the master file system: $ rm -rf $MASTER_DATA_DIRECTORY/gpperfmon/data/* $ rm -rf $MASTER_DATA_DIRECTORY/gpperfmon/logs/* 7. If you do not want to keep your historical Command Center data, drop the gpperfmon database: $ dropdb gpperfmon Greenplum Database Software Environments 16

22 Greenplum Command Center Administrator Guide 1.2 Chapter 3: Administering Command Center 3 Administering Command Center After installation and startup, Greenplum Command Center requires minimal maintenance by administrators. Tasks for administration of the web server or the Command Center database are performed with native tools or Command Center utilities, as described in this chapter. Starting and Stopping Greenplum Command Center Starting and Stopping Command Center Agents Starting and Stopping Command Center Console Starting and Stopping Command Center Agents Whenever the Greenplum Database server configuration parameter gp_enable_gpperfmon is enabled in the master postgresql.conf file, the Command Center agents will run and collect data. These agents are automatically stopped and started together along with the Greenplum Database instance. If you wish to disable the Command Center data collection agents, you must disable the gp_enable_gpperfmon parameter, and restart the Greenplum Database instance. Starting and Stopping Command Center Console You can start, stop and restart Greenplum Command Center Console instances with the following commands: $ gpcmdr --start ["instance name"] $ gpcmdr --stop ["instance name"] $ gpcmdr --restart ["instance name"] If you do not specify an instance name, all instances are started, stopped or restarted at once. You can check the status of instances using: $ gpcmdr --status ["instance name"] Configuring Greenplum Command Center Configuration parameters for Greenplum Command Center are stored in the following configuration files: Agent Configuration Changes to these files require a restart of the Greenplum Database instance (gpstop -r). $MASTER_DATA_DIRECTORY/gpperfmon/conf/gpperfmon.conf $MASTER_DATA_DIRECTORY/postgresql.conf Starting and Stopping Greenplum Command Center 17

23 Greenplum Command Center Administrator Guide 1.2 Chapter 3: Administering Command Center Console Configuration Changes to these files require a restart of Command Center Console (gpcmdr --restart). $GPPERFMONHOME/instances/instance_name/conf/gpperfmonui.conf $GPPERFMONHOME/instances/instance_name/conf/lighttpd.conf See Appendix C, Command Center Configuration File Reference for a description of the configuration parameters in these files. You should not need to manually edit any of the files. Running the Command Center setup utility will make all the necessary modifications to these configuration files. Command Center Agent Administration This section describes basic agent administration tasks, including adding hosts and viewing agent log files. Adding and Removing Hosts Segment agents on new hosts are detected automatically by the master agent. Whenever gp_enable_gpperfmon is enabled on the master, the master monitor agent automatically detects, starts, and begins harvesting data from new segment agents. To verify the addition of a new monitored host, you can check for the new hostname in the Greenplum Command Center Console System s view described in the Greenplum Command Center Console online help. Alternately, you can query the system_now table for the row containing current metrics for each host. For example: =# SELECT * FROM system_now WHERE hostname='new_hostname'; Viewing and Maintaining Master Agent Log Files Log messages for the master agent are written to $MASTER_DATA_DIRECTORY/gpperfmon/logs/gpmmon.log by default. To change the log file location, edit the log_location parameter in gpperfmon.conf. On the segment hosts, agent log messages are written to a gpsmon.log file in the segment instance s data directory. For a host with multiple segments, the agent log file is located in the data directory of the first segment, as listed in the gp_configuration table by dbid. If the segment agent is unable to log into this directory, it will log messages to the home directory of the user running Command Center (typically gpadmin). Configuring Log File Rollover At higher logging levels, the size of the log files may grow dramatically. To prevent the log files from growing to excessive size, you can add an optional log rollover parameter to gpperfmon.conf. The value of this parameter is measured in bytes. For example: max_log_size = Command Center Agent Administration 18

24 Greenplum Command Center Administrator Guide 1.2 Chapter 3: Administering Command Center With this setting, the log files will grow to 10MB before the system rolls over the log file. The timestamp is added to the log file name when it is rolled over. Administrators must periodically clean out old log files that are no longer needed. Command Center Database Administration Data collected by Command Center agents is stored in a dedicated database called gpperfmon within the Greenplum Database instance. This database requires the typical database maintenance tasks such as clean up of old historical data and periodic ANALYZE. See Command Center Database Reference on page 34 for a reference of the tables and views in the gpperfmon database. Connecting to the Command Center Database Database administrators can connect directly to the Command Center database (gpperfmon) using any Greenplum Database-compatible client program (such as psql). For example: $ psql -d gpperfmon -h master_host -p U gpadmin Backing Up and Restoring the Command Center Database The history tables of the Command Center database (gpperfmon) can be backed up and restored using the Greenplum Database parallel backup and restore utilities (gp_dump, gp_restore, gpcrondump, gpdbrestore). See the Greenplum Database Administrator Guide for more information. Because the Command Center database has a low number of tables, you may prefer to devise a backup plan using the table-level backup features of gp_dump. For example, you can create scripts to run gp_dump to back up the monthly partitions of the historical data tables on a monthly schedule. Alternately, you can back up your Command Center database at the database level. Maintaining the Historical Data Tables All of the *_history tables stored in the Command Center database (gpperfmon) are partitioned into monthly partitions. A January 2010 partition is created at installation time as a template partition (it can be deleted once some current partitions are created). The Command Center agents automatically create new partitions in two month increments as needed. Administrators must periodically drop partitions for the months that are no longer needed in order to maintain the size of the Command Center database. See the Greenplum Database Administrator Guide for more information on dropping partitions of a partitioned table. Command Center Database Administration 19

25 Greenplum Command Center Administrator Guide 1.2 Chapter 3: Administering Command Center Web Server Administration The Lighttpd web server and web service middleware are installed in the www directory of your Greenplum Command Center installation. For detailed information on Lighttpd administration, see Configuring the Web Server The Lighttpd web server configuration file is stored in $GPPERFMONHOME/instances/instance_name/conf/lighttpd.conf. Some of the parameters in this configuration file are set by the gpcmdr setup utility, including the web server port and SSL options. See Web Server Parameters (lighttpd.conf) on page 32 for a description of the parameters in this file. You should not need to manually edit this file, if you do, you may break some functionality. Contact EMC Support if you want to make custom modifications to this file. Viewing and Maintaining Web Server Log Files Web server access and error logs are written to $GPPERFMONHOME/instances/instance_name/logs. These two logs are: lighttpd-access.log lighttpd-error.log If you experience errors viewing the Greenplum Command Center Console, refer to these logs for more information. To prevent web server logs from growing to excessive size, you can set up log file rotation using logrotate or cronolog, both of which are widely used with Lighttpd. Web Server Administration 20

26 Greenplum Command Center Administrator Guide 1.2 Chapter 4: Accessing Command Center Data 4 Accessing Command Center Data Greenplum Command Center database stores system, query, health, disk space, and dial-home data collected by data collection agents that run on the master host and each segment (either Hadoop or database) host. This information provides immediate visibility into key areas affecting your Greenplum system performance, and is typically accessed through the Greenplum Command Center Console, however you can also access access this data by other means as described below. Accessing Command Center Data using SQL You can also use direct SQL queries and views on Command Center data. Before performing any of the tasks in this section, make sure the Greenplum Command Center is completely installed and configured, as described in Installing the Greenplum Command Center Console on page 10, and Setting up the Command Center Console on page 11. Viewing Historical System s using SQL Historical tables in the Command Center database store all system data collected by Command Center. You can analyze this data to identify trends in the performance of the Greenplum system over time. With a default quantum setting of 15 seconds, Command Center collects very large amounts of historical system data. To view meaningful groupings of system data records, you can view the data by selected time intervals with a SQL view (as described below). The following provides an example SQL view for viewing aggregated system metrics. In this example view named system_metrics_aggregate_1min, rows of system metrics from system_history are averaged over one-minute intervals and aggregated across all segments in the array: DROP VIEW IF EXISTS system_metrics_aggregate_1min; CREATE VIEW system_metrics_aggregate_1min AS ( SELECT date_trunc('minute', ctime) AS sample_time, hostname, avg(mem_total) AS mem_total, avg(mem_used) AS mem_used, avg(mem_actual_used) AS mem_actual_used, avg(mem_actual_free) AS mem_actual_free, avg(swap_total) AS swap_total, avg(swap_used) AS swap_used, avg(swap_page_in) AS swap_page_in, avg(swap_page_out) AS swap_page_out, avg(cpu_user) AS cpu_user, avg(cpu_sys) AS cpu_sys, avg(cpu_idle) AS cpu_idle, avg(load0) AS load0, avg(load1) AS load1, avg(load2) AS load2, avg(disk_ro_rate) AS disk_ro_rate, avg(disk_wo_rate) AS disk_wo_rate, Accessing Command Center Data using SQL 21

27 Greenplum Command Center Administrator Guide 1.2 Chapter 4: Accessing Command Center Data avg(disk_rb_rate) AS disk_rb_rate, avg(disk_wb_rate) AS disk_wb_rate, avg(net_rp_rate) AS net_rp_rate, avg(net_wp_rate) AS net_wp_rate, avg(net_rb_rate) AS net_rb_rate, avg(net_wb_rate) AS net_wb_rate FROM system_history GROUP BY hostname, sample_time ); To change the interval for averaging results, adjust the value specified for date_trunc. Valid values are microseconds, milliseconds, second, minute, hour, day, week, month, quarter, year, decade, century and millennium. Note: You use can also the Command Center Console to view historical system metrics. See the Command Center Console online help for more information. Viewing Historical Query s using SQL Historical tables in the Command Center database store all query and iterator data collected by Command Center. You can analyze this data to identify trends in the performance of Greenplum system over time. Calculating the average, minimum and maximum values for system-wide query statistics aggregated over time can yield a useful view of database utilization. The following information provides an example SQL view for viewing query statistics in this way. In this example view named database_metrics_1hour, rows of statistics from the table database_history are averaged over hour-long intervals: DROP VIEW if exists database_metrics_1hour; CREATE VIEW database_metrics_1hour AS ( SELECT date_trunc('hour', ctime) AS sample_time, avg(queries_total) AS queries_total, min(queries_total) AS queries_total_min, max(queries_total) AS queries_total_max, avg(queries_running) AS queries_running, min(queries_running) AS queries_running_min, max(queries_running) AS queries_running_max, avg(queries_queued) AS queries_queued, min(queries_queued) AS queries_queued_min, max(queries_queued) AS queries_queued_max FROM database_history GROUP BY sample_time ); Accessing Command Center Data using SQL 22

28 Greenplum Command Center Administrator Guide 1.2 Chapter 4: Accessing Command Center Data To change the interval for averaging results, adjust the value specified for date_trunc. Valid values are microseconds, milliseconds, second, minute, hour, day, week, month, quarter, year, decade, century and millennium. Note: You use can also the Command Center Console to view historical query metrics. See the Command Center Console online help for more information. Accessing Command Center Data using SQL 23

29 Greenplum Command Center Administrator Guide 1.2 Appendix A: Integration with Enterprise Monitoring Tools A. Integration with Enterprise Monitoring Tools Accessing Command Center Data using SNMP You can configure a Greenplum system to trigger SNMP alerts or send notifications to system administrators whenever certain database events occur. Greenplum s gpsnmpd agent is an SNMP (Simple Network Management Protocol) daemon that supports SNMP requests on the health and state of a Greenplum system by using a set of MIBs (Management Information Bases). MIBs are a collection of objects that describe an SNMP-manageable entity in this case, a Greenplum system. MIBs can be used by enterprise monitoring systems such as IBM Tivoli, HP Openview, and Nagios to monitor our Greenplum system. SNMP on the Greenplum Data Computing Appliance For more information about SNMP on the DCA, see the Greenplum Data Computing Appliance Getting Started Guide. Starting with Greenplum Data Computing Appliance version 1.2, we have a new MIB definition that will help you monitor software and hardware as follows: Software Version information Database Hadoop Hardware Master Node Segment Nodes Interconnect switches DIA/ETL loads Hadoop Master Nodes Hadoop Worker Nodes Admin switches Integration with Enterprise Monitoring Tools 24