Data Server Manager Setup Planner, Best Practices, and Troubleshooting

Transcription

1 Marichu Scanlon Last Updated: Nov 17, 2016 Data Server Manager Setup Planner, Best Practices, and Troubleshooting

2 IBM Data Server Manager Architecture Optional Repository Database DB2 for LUW 3 1 Web console 2 IBM Data Server Manager HTTP / HTTPS 4 Data Tools Runtime Client 5 DB2 Database DB2 Database CLI Driver JCC Driver Monitored Databases 2 2

3 1 Before you invoke the setup script, check: Supported platforms Windows, Linux, Unix Please see the System Requirements link in the Resources page for the latest version numbers of the different platforms Download and product storage requirements Downloading Data Server Manager: Base Edition (from dw) or Enterprise Edition (from PPA) Download Base Edition now, apply LAK from PPA later 500 MB for download image (.zip or.tgz); Download technote: Product install directory: default directory name is ibm-datasrvrmgr Size: 1 GB Updated Capacity Planning document for DSM v

4 User IDs and Privileges Default setup user id is admin. This user id does not even need to exist in the OS where you setup DSM You can use a DB2 user id that will have privileges on your database repository. Minimum privilege is SYSCTRL, Additonal details: Minimum Privileges for Data Server Manager Privileges Required for DSM on DB2 on z/os: 4

5 1 Userid and Privileges considerations Web Console access Default setup user id is admin. This user id does not even need to exist in the OS where you setup DSM Other userids to be granted VIEWER or ADMINISTRATOR access to DSM console Trusted certificates, see topic: Importing trusted SSL certificates for DSM: Database access For LUW, id to collect monitoring data is called the collection id You can use a DB2 user id that will have privileges on your database repository. Minimum privilege is SYSCTRL. Privileges required for the ID used by Data Server Manager: 5

6 1 Ready, Set, Go - Up and running quickly Up and running in minutes No installer code After download, run setup script found in ibm-datasvrmgr folder Setup.sh in Linux; setup.bat in Windows Base Edition uses Derby, no DB2 repository used for persistence Repository database only required for historical data Enterprise Edition allows for DB2 repository User can upgrade from Base Enterprise at any time (more on this later) No fat client installation required web browser only Silent setup option for System Administrators on Unix UI-based setup for Windows Users 6

7 1 3-step setup 7

8 1 For silent setup setup.conf Setup script requires and uses the setup.conf file The DSM package includes both a default setup.conf and setup.conf.sample User can use setup UI to accept license agreement, specify ports, web console userid/password If a user wants to configure a repository at setup time, he must Have Enterprise LAK applied Update setup.conf and specify the repositorydb.* property values (eg: repositorydb.host=emme.svl.ibm.com) 8

9 2 Data Server Manager Login Page Before you launch Data Server Manager, check if your browser is supported. Mozilla Firefox, Internet Explorer, Google Chrome Please see the System Requirements page for the latest versions Check if Adobe Flash is needed. Needed for Mozilla Firefox and Google Chrome Check if you have a firewall between the DSM server machine and where you will be launching the DSM web console Launch Data Server Manager using the URL reported in the setup: server hostname or IP address>: server hostname or IP address>:11081 (secure) 9

10 2 Welcome page 10

11 3 Configure Optional Repository Database Required to take advantage of Enterprise features Historical performance monitoring Smart alerts SQL and workload tuning Configuration management Client monitoring 11

12 3 Optional Step: Configure Repository DB for Historical Persistence Can be done at a later time! Create a DB2 for LUW database and specify the hostname, database name, port and user credentials here. 12

13 3 Optional Repository setup Creating the repository can take some time lots of SQL, not uncommon to take upwards of 10 minutes depending on system s CPU. Latest Capacity Planning spreadsheet: connections.ibm.com/files/app#/file/9c5f f-4f82-bbcb-1f086c2c2177 If it appears to hang user should let to continue to run until it completes. If user interrupts process by killing browser, the repo scripts will continue to run in background As part of DB2 repo creation, if they were using Base Enterprise, key data will be migrated from Derby DB2 repo (connections, profiles, jobs,...) If you go back to the Setup >Repository Database Configuration, you should specify a valid and existing database. If the repository is in a DB2 for LUW instance, consider adding the repository as a monitored database. 13

14 4 Add Managed Connections Go to Setup >Databases, to create managed connections to your monitored databases For DB2 for LUW managed connections, enter DATABASE name, HOSTNAME or HOST IP ADDR, and PORT. For DB2 for z/os managed connections, obtain the LOCATION, HOSTNAME or HOST IP ADDR, and PORT from DBA, or DISPLAY DDF output. (DSM v2.1) Always run the Validate Features for each managed connection If the Feature Validation run reports errors, resolve these first before using the product. How to enable SSL connectivity from Data Server Manager to managed connections: How to setup trusted certificates for accessing Data Server Manager: 14

15 4 Discover Databases Import connections from Data Studio is also supported 15

16 5 Data Tools Runtime Client Download the Bulk Deploy zip file for your client OS from Passport Advantage. If you are using WAS, download the zip file for the OS where the WAS server is installed. Follow the instructions in this link to setup DTRC: ols.dsweb.ots.installconfig.doc/topics/tparentclients.html 16

17 Maintenance

18 Standard Maintenance The DSM repository database is a DB2 database that must be managed In DB2 for LUW, default db cfg is AUTO_MAINT. If the default is not used, make sure you have processes in place to do maintenance on the repository. If the repository is in DB2 for z/os, you will need to manage the database like you would any other database in your subsystem. Assign a userid that will be responsible for monitoring the DB2 for LUW repository Monitors repository database health Gets notified for issues regarding the repository s performance Gives new users access to DSM console Understands the various logs generated Talks to IBM support as needed Handles maintenance of the DSM server (e.g. requires more storage, CPU, memory, etc) 18

19 Standard Maintenance Automate runstats and reorganization for the DB2 for LUW repository database Repository database is setup with automatic table maintenance For automatic reorganization, make sure that you have defined an offline maintenance window html Manage database configuration of the repository database DSM accepts the current database and database management configuration settings of the instance where the repository database is created. Before install, user has the option to create and configure the repository outside of the installation. Specify to use an already existing database Script to modify is found in <DSM install directory>/samples/repository/create.sql 19

20 Additional maintenance tips Review the values for the configuration parameters or zparms of subsystem where the repository database is created Configure the maintenance windows for automatic maintenance of the repository database. Ensure that your database backup is set up to run automatically, and at regular intervals. Monitor the disk space usage of table spaces of the repository database, and add more space as needed. Ensure that the values of the logprimary and logsecond parameters are high enough to allocate the storage space that is required for the recovery log files. Investigate creating a configuration tracking job for the repository database or subsystem where the repository database is created, so you can periodically check changes to the database or subsystem. 20

21 Checking the status of the repository Using the web console: change password at the OS level. logon to the web console immediately. Go to the Product Setup >Configuration Repository > Select Repository Server and change the password. Using command line In a command window, navigate to the DSM install directory. CD to the bin directory Type:./status.sh (Linux) or status.bat (Windows) 21

22 Changing the Password for the Repository Database Using the web console: change password at the OS level. logon to the web console immediately. Go to the Product Setup >Configuration Repository > Select Repository Server and change the password. Alternatively, use utility script login as SYSADM of DB2 instance Go to the DSM install directory In Unix/Linux, issue: <install_dir>/bin/updatemetadb.sh -user <userid> -password <new password> In Windows, issue: <install_dir>\bin\updatemetadb.bat user <userid> -password <new password> 22

23 Collecting information When a problem is found Collect log files and configuration information Open a PMR in order to upload the collected data to IBM since in most cases data is to big to send via . For Windows: Open a DB2 command line window Change to the <DSM install dir>\support directory Enter the command: collector.bat For Linux: Log on as DB2 instance owner of the DB2 instance that Data Server Manager uses Change to the <DSM install dir>\support directory Enter the command: collector.sh The collector script creates a zip file containing all needed files. Select the option to collect all diagnostics Upload the created zip file to the PMR. Inform the IBM contact person about the PMR number and uploaded data Inquire from IBM Support about current interim fixes, technotes affecting DSM. Link to Fix Central: 23

24 Troubleshooting DSM

25 Setup.bat (Windows) or Setup.sh (Linux) fails To restart the setup process, do the following: 1. Verify that the backup directory 1 contains a backup copy of the Config, dscomponents, dscomponent_with_rep and java directories. 2. Remove the Config, dscomponents, dscomponent_with_rep and java directories from the installation directory, typically ibm_datasrvrmgr 3. Copy or move the backed-up directories from the backup directory to the installation directory. 4. Remove or rename backup directory. 5. Unpack the IBM Data Server Manager install package again 2 and restart the setup process. 1 The dsm_bak directory is created during setup; and does not exist when the setup script completes successfully. 2 This recreates the deleted folders in the ibm-datasrvrmgr directory. 25

26 Uninstall, Install or Upgrade DSM DSM has an aggressive release schedule. We usually have a new release or fix pack every 3 months. When a new version of DSM becomes available and you already have a previous versions of DSM installed, you have 2 options: becomes ava Option 1: Uninstall and re-install the new version Option 2: Upgrade to the new version Check out the following links for specific steps for either options How to uninstall DSM: Installing and upgrading IBM Data Server Manager: 01.ibm.com/support/docview.wss?uid=swg Installing and upgrading IBM Data Server Manager v2.1: 01.ibm.com/support/docview.wss?uid=swg

27 Is DSM Active? Data Server Manager executes as a java process Check the status On Unix /Linux: <DSM_install_dir>/bin/status.sh On Windows <DSM_install_dir>/bin/status.bat Data Server Manager server process is started using the instance owner On Unix/Linux ps ef grep liberty db2inst :33? 00:03:23 /home/db2inst1/desktop/ibmdatasrvrmgr/java/jre/bin/java -Xms128M -Dhttps.protocols=TLSv1.2 com.ibm.datatools.embedded.liberty.adapter.main -start On Windows Check the services for IBM Data Server Manager are running (see next slides) 27

28 Starting Data Server Manager Ensure the server is completely started Check logs under <DSM_install_dir>/logs 1. In dsserver.log, look for: CWWKT0016I: Web application available (default_host): CWWKZ0001I: Application dsweb started in seconds. CWWKF0011I: The server dsweb is ready to run a smarter planet. : stdout (270) <timestamp> /ots/index.jsp ACTIVE stdout (1) <timestamp> SERVER STATUS: ACTIVE 2. In iocm.log, look for: PruneInstallerRunnable: run() Prune Preferences Tracker registered with Preference Service 3. In DS_System.0.log, look for: com.ibm.datatools.dsweb.ots.connections.thread.testrepothread.run: {"isactive":true} 28

29 Stopping Data Server Manager on Windows Data Server Manager on Windows runs as a service Sometimes, after stopping DSM, the service is not completely stopped. To verify Go to Start >Administrative Tools >Services Search for IBM Data Server Manager in the Services list If the service is NOT stopped, right-click on the service then issue the Stop. 29

30 Troubleshooting First steps Google the error message If the problem has been reported before by either a customer or documented in an APAR, technote, article, blog,etc. the Google search engine will direct you to this document. The search engine also searches our DSM Forums. Check out the FAQ and Troubleshooting forum posts in the Data Server Manager external community Data Server Manager Common Questions Data Server Manager Best Practices 30

31 Troubleshooting Typical Problems Problem: No data seen in the dashboard Questions to ask Did it work before? When did it stop? Which screen? Advice Check if DSM completely shut down or started up (see slides later) Check to see if the repository database in the Setup >Repository database is Active. If using Enterprise Edition, check if Repository Persistence is enabled Check if the required collection setting is enabled in the Monitoring profile 31

32 Troubleshooting Typical Problems Problem: User can't get in to the DSM URL (User not authorized ) Questions to ask Is DSM completely started? Is the repository database reachable? Advice Execute the status script Check if DB2 is started Check the dsserver.log and DS_System.* for any error messages Check to see that no service has terminated in the above log files. Check the firewall for the monitored database ports (50000, 50001, etc.), DSM EI ports (60000, 60001) and whether the web console server ports are listening (default port number) netstat a grep port# 32

33 Troubleshooting Typical Problems Problem: User encounters an error message on screen Questions to ask Is it during configuration of a database? On what view / dashboard did it show up? Advice Use Show Details button to see the error Take a screenshot Check the dsserver.log and DS_System.* for any error messages issued 3 3 Copyright IBM Corporation

34 Troubleshooting Typical Problems Problem: DSM browser is not responsive, or web console process crashed or hung Questions to ask: Is this occurring often? Is the browser left unattended for hours or days when you see this issue? Are there core dumps or javacore* files from the directory where the DSM start was issued? Is the login shell to the UNIX/Linux system timed out ex: ssh or putty to the system when it timed out brings down other child processes Advice Clear the browser cache Check the dsserver.log and DS_System.* for any error messages issued Issue : ps ef grep liberty db2admin Feb10 pts/2 00:05:00 java com.ibm.datatools.zero.servlet.adapter.main -start The PPID (parent process ID) is "1" - meaning init - which means that the "zero" process will not die when the login shell that started is exited (or timed out). If its not "1", then this process could get hung-up when the login shell is exited (in this case their X11 login session?) Stop DSM (if its not already stopped) Start DSM Note: the startup information will not be seen in the console, will get logged to the dsserver.log file and nohup.out log. 34

35 Troubleshooting Typical Problems Problem: Out-of-memory errors Questions to ask How much memory does my machine have Are you monitoring a large number of databases or partitions or database objects? Advice Gather information about the machine the DSM server is on Verify against the capacity planning recommendations Check the Java heap size for the repository service (Windows) or task (Linux) In Windows In Linux 3 5 Copyright IBM Corporation

36 Troubleshooting Typical Problems Problem: Dashboards take a long time to paint Questions to ask Is there enough memory? Do I see a slowdown on DSM server metrics display Check you have enough file open descriptor handles in Linux/Unix. 3 6 Advice Check the maximum memory allocated to the DSM Server Java process. Check if you are doing memory swapping on the DSM server to see if you are memory constrained. vmstat output for si and so might help If you are noticing a slow down in the DSM server, check the I/O wait on the server. Use ps command. Check and increase the number of file open descriptors from default of 1024 to a higher number by talking to your system administrator. ulimit -a Copyright IBM Corporation

37 What are the main logs to examine Location of files <install_dir>/logs on unix <install_dir>\logs on windows Main log files of interest dsserver.log Start/Stop/Status of DSM services Re-created after each recycling of DSM DS_System.* Start/Stop/Status of DSM services; Entries appended iocm.log.* Configuration Manager log Shows errors, sqlcodes, and exceptions for CM Note: Some log file names use a rolling log setting. I.E. when DSM is recycled, the old log file is copied into xxx.n or xxx.n.log where N is a number 37

38 Additional Logs (1 of 2) RtMonWeb.* The file name depends on the rolling log settings. Contains errors and traces for real time monitoring (similar to OPM s old datatools.log) OTSMonLite.* The file name depends on the rolling log settings. Contains errors and traces for historical monitoring and SQL Capture (similar to OPM s old datatools.log) AdminLogger.* The file name depends on the rolling log settings. Contains log entries for database explorer administration functions oqwtjobmgrlogger.* The file name depends on the rolling log settings. Contains log entries for query tuning functions 38

39 Additional Logs (2 of 2) HealthSnapshotLogger.* 39 The file name depends on the rolling log settings. Contains errors and traces for health monitoring AlertNotificationLogger.* The file name depends on the rolling log settings. Contains errors and traces for alert notification through or Simple Network Management Protocol (SNMP) BlackoutLogger.* The file name depends on the rolling log settings. Contains log entries that are associated with the execution of the blackout events JobManagerLogger.* The file name depends on the rolling log settings. Contains log entries that are associated with the schedule and execution of scripts that are connected with user-defined alert types

40 How to configure logs DSM Log files, by default, are located in: <DSM install path>/logs/ User can use Set up->logging and Tracing to view and configure DSM logs online, no need to restart DSM. 40

41 How to configure logs (con t) For admin function problem: AdminLogger and DS_System log files can greatly help L2 and L3 to locate and investigate customer problems. The info of browser is also important and needed to be collected from customer, like IE, Firefox, Chrome type and version. 41

42 How to configure logs (cont ) User can reconfigure Admin log path and log level, such as Debug, Trace. And these operations are online, no need to restart DSM. 42

43 How to capture logs to send to IBM Support Stop DSM Create backup copies of your dsserver.log, iocm.log and DS_System.log files Delete the above log files Start DSM Configure the logs as described in previous slides; set level to Trace Recreate the problem Collect the logs Go to <DSM_Install_dir>/support directory Run collector.sh (Linux / Unix) or collector.bat When asked for a PMR, enter a PMR if available, or 99999,999,999 Send the generated zip file to IBM Support Reconfigure the log files, set level back to Information 43

44 Contacting IBM Support 4 4 Contact IBM support to open service requests (PMR) ibm.com/support/entry/portal/Downloads/Software/Information_Manageme nt/db2_for_linux,_unix_and_windows?lnk=mhsd Provide as much detail as possible on the context of the issue The error output, including the message ID The name and version of your operating system The version of Data Server Manager and Data Tools Runtime Client that you are running The name and version of your monitored DB2 database The name and version of your application server (if applicable) The type (e.g. IBM, Sun, etc) and version of JDBC driver Run collector.bat or collector.sh from the <DSM Install directory>/support Collects log files, trace files, database tables, and configuration files that relates to your problem. Provide all traces as requested by the service team Copyright IBM Corporation