MIMIX Availability. Version 7.1 MIMIX Operations 5250

Transcription

1 MIMIX Availability Version 7.1 MIMIX Operations 5250

2 Notices MIMIX Operations User Guide April 2014 Version: Copyright 1999, 2014 Vision Solutions, Inc. All rights reserved. The information in this document is subject to change without notice and is furnished under a license agreement. This document is proprietary to Vision Solutions, Inc., and may be used only as authorized in our license agreement. No portion of this manual may be copied or otherwise reproduced without the express written consent of Vision Solutions, Inc. Vision Solutions provides no expressed or implied warranty with this manual. The following are trademarks or registered trademarks of their respective organizations or companies: MIMIX and Vision Solutions are registered trademarks and AutoGuard, Data Manager, Director, Dynamic Apply, ECS/400, GeoCluster, IntelliStart, Integrator, ioptimize, itera, itera Availability, MIMIX AutoNotify, MIMIX Availability, MIMIX Availability Manager, MIMIX DB2 Replicator, MIMIX Director, MIMIX dr1, MIMIX Enterprise, MIMIX Global, MIMIX Monitor, MIMIX Object Replicator, MIMIX Professional, MIMIX Promoter, OMS/ODS, RecoverNow, Replicate1, RJ Link, SAM/400, Switch Assistant, Vision AutoValidate, and Vision Suite are trademarks of Vision Solutions, Inc. Double-Take Share, Double-Take Availability, and Double-Take RecoverNow DoubleTake Inc. AIX, AIX 5L, AS/400, DB2, eserver, IBM, Informix, i5/os, iseries, OS/400, Power, System i, System i5, System p, System x, System z, and WebSphere International Business Machines Corporation. Adobe and Acrobat Reader Adobe Systems, Inc. HP-UX Hewlett-Packard Company. Teradata Teradata Corporation. Intel Intel Corporation. Java, all Java-based trademarks, and Solaris Sun Microsystems, Inc. Linux Linus Torvalds. Internet Explorer, Microsoft, Windows, and Windows Server Microsoft Corporation. Mozilla and Firefox Mozilla Foundation. Netscape Netscape Communications Corporation. Oracle Oracle Corporation. Red Hat Red Hat, Inc. Sybase Sybase, Inc. Symantec and NetBackup Symantec Corporation. UNIX and UNIXWare the Open Group. All other brands and product names are trademarks or registered trademarks of their respective owners. If you need assistance, contact Vision Solutions CustomerCare team at: CustomerCare Vision Solutions, Inc. Telephone: or support@visionsolutions.com Web Site:

3 Contents Who this book is for What is in this book The MIMIX documentation set Sources for additional information How to contact us Chapter 1 MIMIX overview 15 MIMIX concepts Product concepts System role concepts Journaling concepts Configuration concepts Process concepts Additional switching concepts Best practices for maintaining your MIMIX environment Authority to products and commands Accessing the MIMIX Main Menu Chapter 2 MIMIX policies 26 Environment considerations for policies Policies in environments with more than two nodes or bi-directional replication. 27 When to disable automatic recovery for replication and auditing Disabling audits and recovery when using the MIMIX CDP feature Setting policies - general Changing policies for an installation Changing policies for a data group Resetting a data group-level policy to use the installation level value Policies which affect an installation Changing retention criteria for procedure history Policies which affect replication Errors handled by automatic database recovery Errors handled by automatic object recovery Policies which affect auditing Policies for auditing runtime behavior Policies for submitting audits automatically When automatically submitted audits run Changing auditing policies Changing when automatic audits are allowed to run Changing scheduling criteria for automatic audits Changing the selection frequency of priority auditing categories Changing the audit level policy when switching Changing the system where audits are performed Changing retention criteria for audit history Restricting auditing based on the state of the data group Preventing audits from running Disabling all auditing for an installation Disabling all auditing for a data group Disabling automatically submitted audits Policies for switching with model switch framework Specifying a default switch framework in policies

4 Setting polices for MIMIX Switch Assistant Setting policies when MIMIX Model Switch Framework is not used Policy descriptions Chapter 3 Checking status in environments with application groups 60 Checking application group status Resolving problems reported in the Monitors field Resolving problems reported in the Notifications field Resolving problems reported in Status columns Resolving a procedure status problem Resolving an *ATTN status for an application group Resolving other common status values for an application group Status for Work with Node Entries Status for Work with Data Resource Group Entries Verifying the sequence of the recovery domain Changing the sequence of backup nodes Examples of changing the backup sequence Chapter 4 Working with status of procedures and steps 77 Displaying status of procedures Displaying status of the last run of all procedures Displaying available status history of procedure runs Resolving problems with procedure status Responding to a procedure in *MSGW status Resolving a *FAILED or *CANCELED procedure status Displaying status of steps within a procedure run Resolving problems with step status Responding to a step with a *MSGW status Resolving *CANCEL or *FAILED step statuses Acknowledging a procedure Running a procedure Resuming a procedure Overriding the attributes of a step Canceling a procedure Chapter 5 Monitoring status with MIMIX Availability Status 93 Checking replication status from the MIMIX Availability Status display Checking audit and notification status from the MIMIX Availability Status display Checking status of supporting services from the MIMIX Availability Status display.. 96 Chapter 6 Working with data group status 98 The Work with Data Groups display Problems reflected in the Audits/Recov./Notif. field Problems reflected in the Data Group column Resolving problems highlighted in the Data Group column Manager problems reflected in the Source and Target columns Replication problems reflected in the Source and Target columns Setting the automatic refresh interval Working with the detailed status of data groups Displaying data group detailed status Merged view

5 Object detailed status views Database detailed status views Identifying replication processes with backlogs Data group status in environments with journal cache or journal state Resolving a problem with journal cache or journal state Chapter 7 Working with audits 121 Auditing overview Components of an audit Phases of audit processing Object selection methods for automatic audits How priority auditing determines what objects to select How audits are submitted automatically Audit status and results Audit compliance Guidelines and considerations for auditing Auditing best practices Considerations for specific audits Recommendations when checking audit results Displaying audit runtime status Running an audit immediately Resolving audit problems Checking the job log of an audit Ending audits Displaying audit history Audits with no selected objects Working with audited objects Displaying audited objects from a specific audit run Displaying a customized list of audited objects Working with audited object history Displaying the audit history for a specific object Displaying audit compliance Determining whether auditing is within compliance Displaying scheduling information for automatic audits Chapter 8 Working with system-level processes 149 Displaying status of system-level processes Resolving *ACTREQ status for a system manager Checking for a system manager backlog Starting a system manager or a journal manager Ending a system manager or a journal manager Starting collector services Ending collector services Starting target journal inspection processes Ending target journal inspection processes Displaying status of target journal inspection Displaying results of target journal inspection Displaying details associated with target journal inspection notifications Displaying messages for TGTJRNINSP notifications Identifying the last entry inspected on the target system

6 Chapter 9 Working with notifications and recoveries 159 What are notifications and recoveries Displaying notifications What information is available for notifications Detailed information Options for working with notifications Notifications for newly created objects Displaying recoveries What information is available for recoveries Detailed information Options for working with recoveries Orphaned recoveries Determining whether a recovery is orphaned Removing an orphaned recovery Chapter 10 Starting and ending replication 169 Before starting replication Commands for starting replication What is started with the STRMMX command STRMMX and ENDMMX messages What is started by the default START procedure for an application group Choices when starting or ending an application group What occurs when a data group is started Journal starting point identified on the STRDG request Journal starting point when the object send process is shared Clear pending and clear error processing Starting MIMIX Starting an application group Starting selected data group processes Starting replication when open commit cycles exist Checking for open commit cycles Resolving open commit cycles Before ending replication Commands for ending replication Command choice by reason for ending replication Additional considerations when ending replication Ending immediately or controlled Controlling how long to wait for a controlled end to complete Ending all or selected processes When to end the RJ link What is ended by the ENDMMX command What is ended by the default END procedure for an application group What occurs when a data group is ended Ending MIMIX Ending with default values Ending by prompting the ENDMMX command After you end MIMIX products Ending an application group Ending a data group in a controlled manner Preparing for a controlled end of a data group

7 Performing the controlled end Confirming the end request completed without problems Ending selected data group processes What replication processes are started by the STRDG command What replication processes are ended by the ENDDG command Chapter 11 Resolving common replication problems 207 Working with message queues Working with the message log Working with user journal replication errors Working with files needing attention (replication and access path errors) Working with journal transactions for files in error Placing a file on hold Ignoring a held file Releasing a held file at a synchronization point Releasing a held file Releasing a held file and clearing entries Correcting file-level errors Correcting record-level errors Record written in error Working with tracking entries Accessing the appropriate tracking entry display Holding journal entries associated with a tracking entry Ignoring journal entries associated with a tracking entry Waiting to synchronize and release held journal entries for a tracking entry Releasing held journal entries for a tracking entry Releasing and clearing held journal entries for a tracking entry Removing a tracking entry Working with objects in error Using the Work with DG Activity Entries display Retrying data group activity entries Retrying a failed data group activity entry Determining whether an activity entry is in a delay/retry cycle Removing data group activity history entries Chapter 12 Starting, ending, and verifying journaling 230 What objects need to be journaled Authority requirements for starting journaling MIMIX commands for starting journaling Journaling for physical files Displaying journaling status for physical files Starting journaling for physical files Ending journaling for physical files Verifying journaling for physical files Journaling for IFS objects Displaying journaling status for IFS objects Starting journaling for IFS objects Ending journaling for IFS objects Verifying journaling for IFS objects Journaling for data areas and data queues

8 Displaying journaling status for data areas and data queues Starting journaling for data areas and data queues Ending journaling for data areas and data queues Verifying journaling for data areas and data queues Chapter 13 Switching 244 About switching Planned switch Unplanned switch Switching application group environments with procedures Switching data group environments with MIMIX Model Switch Framework Switching an application group Switching a data group-only environment Switching to the backup system Synchronizing data and starting MIMIX on the original production system Switching to the production system Determining when the last switch was performed Checking the last switch date Problems checking switch compliance Performing a data group switch Switch Data Group (SWTDG) command Chapter 14 Less common operations 259 Starting the TCP/IP server Ending the TCP/IP server Working with objects Displaying long object names Considerations for working with long IFS path names Displaying data group spooled file information Viewing status for active file operations Displaying a remote journal link Displaying status of a remote journal link Identifying data groups that use an RJ link Identifying journal definitions used with RJ Disabling and enabling data groups Procedures for disabling and enabling data groups Determining if non-file objects are configured for user journal replication Determining how IFS objects are configured Determining how data areas or data queues are configured Using file identifiers (FIDs) for IFS objects Operating a remote journal link independently Starting a remote journal link independently Ending a remote journal link independently Chapter 15 Troubleshooting - where to start 276 Gathering information before reporting a problem Obtaining MIMIX and IBM i information from your system Reducing contention between MIMIX and user applications Data groups cannot be ended Verifying a communications link for system definitions Verifying the communications link for a data group

9 Verifying all communications links Checking file entry configuration manually Data groups cannot be started Cannot start or end an RJ link Removing unconfirmed entries to free an RJ link RJ link active but data not transferring Errors using target journal defined by RJ link Verifying data group file entries Verifying data group data area entries Verifying key attributes Working with data group timestamps Automatically creating timestamps Creating additional timestamps Creating timestamps for remote journaling processing Deleting timestamps Displaying or printing timestamps Removing journaled changes Performing journal analysis Removing journal analysis entries for a selected file Appendix A Interpreting audit results - supporting information 299 Interpreting results for configuration data - #DGFE audit When the difference is not found Interpreting results of audits for record counts and file data What differences were detected by #FILDTA What differences were detected by #MBRRCDCNT Interpreting results of audits that compare attributes What attribute differences were detected Where was the difference detected What attributes were compared Appendix B IBM Power Systems operations that affect MIMIX 310 MIMIX procedures when performing an initial program load (IPL) MIMIX procedures when performing an operating system upgrade Prerequisites for performing an OS upgrade on either system MIMIX-specific steps for an OS upgrade on a backup system MIMIX-specific steps for an OS upgrade on a production system with switching MIMIX-specific steps for an OS upgrade on the production system without switching MIMIX procedures when upgrading hardware without a disk image change Considerations for performing a hardware system upgrade without a disk image change MIMIX-specific steps for a hardware upgrade without a disk image change Hardware upgrade without a disk image change - preliminary steps Hardware upgrade without a disk image change - subsequent steps MIMIX procedures when performing a hardware upgrade with a disk image change Considerations for performing a hardware system upgrade with a disk image change

10 MIMIX-specific steps for a hardware upgrade with a disk image change Hardware upgrade with a disk image change - preliminary steps Hardware upgrade with a disk image change - subsequent steps Handling MIMIX during a system restore Prerequisites for performing a restore of MIMIX Index

11 Who this book is for Who this book is for What is in this book The MIMIX Operations book describes how to perform routine operational tasks and basic troubleshooting for MIMIX Enterprise and MIMIX Professional from a 5250 emulator. The MIMIX Operations book provides these distinct types of information: A summary of concepts within MIMIX Application group and data group status and troubleshooting Audit status, troubleshooting, scheduling, and history Procedures for starting, ending, and switching replication Procedures for starting, ending, and verifying journaling Procedures for handling MIMIX when performing operations such as IPLs or hardware and operating system upgrades. The MIMIX documentation set The following documents about MIMIX Availability products are available: Using License Manager License Manager currently supports MIMIX Availability, itera Availability, and ioptimize. This book describes software requirements, system security, and other planning considerations for installing software and software fixes for Vision Solutions products that are supported through License Manager. The preferred way to obtain license keys and install software is by using Vision AutoValidate and the product s Installation Wizard. However, if you cannot use the wizard or AutoValidate, this book provides instructions for obtaining licenses and installing software from a 5250 emulator. This book also describes how to use the additional security functions from Vision Solutions which are available for License Manager and MIMIX and implemented through License Manager. MIMIX Administrator Reference This book provides detailed conceptual, configuration, and programming information for MIMIX Enterprise and MIMIX Professional. It includes checklists for setting up several common configurations, information for planning what to replicate, and detailed advanced configuration topics for custom needs. It also identifies what information can be returned in outfiles if used in automation. MIMIX Operations with IBM i Clustering This book is for administrators and operators in an IBM i clustering environment who either use the basic support for IBM i clustering provided within MIMIX or who use MIMIX Global to integrate cluster management with MIMIX logical replication or supported hardware-based replication techniques. This book 11

12 The MIMIX documentation set focuses on addressing problems reported in MIMIX status and basic operational procedures such as starting, ending, and switching. MIMIX Operations This book provides high level concepts and operational procedures for managing your high availability environment using MIMIX Enterprise or MIMIX Professional from a 5250 emulator. This book focuses on tasks typically performed by an operator, such as checking status, starting or stopping replication, performing audits, and basic problem resolution. Using MIMIX Monitor This book describes how to use the MIMIX Monitor user and programming interfaces available with MIMIX Enterprise or MIMIX Professional. This book also includes programming information about MIMIX Model Switch Framework and support for hardware switching. Using MIMIX Promoter This book describes how to use MIMIX commands for copying and reorganizing active files. MIMIX Promoter is available with MIMIX Enterprise and as nocharge feature for MIMIX Professional. MIMIX for IBM WebSphere MQ This book identifies requirements for the MIMIX for MQ feature which supports replication in IBM WebSphere MQ environments. This book describes how to configure MIMIX for this environment and how to perform the initial synchronization and initial startup. Once configured and started, all other operations are performed as described in the MIMIX Operations book. 12

13 Sources for additional information Sources for additional information This book refers to other published information. The following information, plus additional technical information, can be located in the IBM System i and i5/os Information Center. From the Information center you can access these IBM Power Systems topics, books, and redbooks: Backup and Recovery Journal management DB2 Universal Database for IBM Power Systems Database Programming Integrated File System Introduction Independent disk pools OptiConnect for OS/400 TCP/IP Setup IBM redbook Striving for Optimal Journal Performance on DB2 Universal Database for iseries, SG IBM redbook AS/400 Remote Journal Function for High Availability and Data Replication, SG IBM redbook Power Systems iasps: A Guide to Moving Applications to Independent ASPs, SG The following information may also be helpful if you replicate journaled data areas, data queues, or IFS objects: DB2 UDB for iseries SQL Programming Concepts DB2 Universal Database for iseries SQL Reference IBM redbook AS/400 Remote Journal Function for High Availability and Data Replication, SG

14 How to contact us How to contact us For contact information, visit our Contact CustomerCare web page. If you are current on maintenance, support for MIMIX products is also available when you log in to Support Central. It is important to include product and version information whenever you report problems. 14

15 CHAPTER 1 MIMIX overview This book provides operational information and procedures for using MIMIX Enterprise and MIMIX Professional through its 5250 emulator user interface. For simplicity, this book uses the term MIMIX to refer to the functionality provided by either product unless a more specific name is necessary. MIMIX Availability version 7.1 provides high availability for your critical data in a production environment on IBM Power Systems through real-time replication of changes and the ability to quickly switch your production environment to a ready backup system. These capabilities allow your business operations to continue when you have planned or unplanned outages in your System i environment. MIMIX also provides advanced capabilities that can help ensure the integrity of your MIMIX environment. Replication: MIMIX continuously captures changes to critical database files and objects on a production system, sends the changes to a backup system, and applies the changes to the appropriate database file or object on the backup system. The backup system stores exact duplicates of the critical database files and objects from the production system. MIMIX uses two replication paths to address different pieces of your replication needs. These paths operate with configurable levels of cooperation or can operate independently. The user journal replication path captures changes to critical files and objects configured for replication through a user journal. When configuring this path, shipped defaults use the remote journaling function of the operating system to simplify sending data to the remote system. In previous versions, MIMIX DB2 Replicator provided this function. The system journal replication path handles replication of critical system objects (such as user profiles, program objects, or spooled files), integrated file system (IFS) objects, and document library object (DLOs) using the system journal. In previous versions MIMIX Object Replicator provided this function. Configuration choices determine the degree of cooperative processing used between the system journal and user journal replication paths when replicating database files, IFS objects, data areas, and data queues. Switching: One common use of MIMIX is to support a hot backup system to which operations can be switched in the event of a planned or unplanned outage. If a production system becomes unavailable, its backup is already prepared for users. In the event of an outage, you can quickly switch users to the backup system where they can continue using their applications. MIMIX captures changes on the backup system for later synchronization with the original production system. When the original production system is brought back online, MIMIX assists you with analysis and synchronization of the database files and other objects. 15

16 Automatic verification and correction: MIMIX enables earlier and easier detection of problems known to adversely affect maintaining availability and switch-readiness of your replication environment. MIMIX automatically detects and corrects potential problems during replication and auditing. MIMIX also helps to ensure the integrity of your MIMIX configuration by automatically verifying that the files and objects being replicated are what is defined to your configuration. MIMIX is shipped with these capabilities enabled. Incorporated best practices for maintaining availability and switch-readiness are key to ensuring that your MIMIX environment is in tip-top shape for protecting your data. User interfaces allow you to fine-tune to the needs of your environment. Analysis: MIMIX also provides advanced analysis capabilities through the MIMIX portal application for Vision Solutions Portal (VSP). When using the VSP user interface, you can see what objects are configured for replication as well as what replicated objects on the target system have been changed by people or programs other than MIMIX. (Objects changed on the target system affect your data integrity.) You can also check historical arrival and backlog rates for replication to help you identify trends in your operations that may affect MIMIX performance. Uses: MIMIX is typically used among systems in a network to support a hot backup system. Simple environments have one production system and one backup system. More complex environments have multiple production systems or backup systems. MIMIX can also be used on a single system. You can view the replicated data on the backup system at any time without affecting productivity. This allows you to generate reports, submit (read-only) batch jobs, or perform backups to tape from the backup system. In addition to real-time backup capability, replicated databases and objects can be used for distributed processing, allowing you to off-load applications to a backup system. The topics in this chapter include: MIMIX concepts on page 17 summarizes key concepts that you need to know about MIMIX. Best practices for maintaining your MIMIX environment on page 23 summarizes recommendations from Vision Solutions. Authority to products and commands on page 23 identifies authority levels to MIMIX functions when additional security features provided by Vision Solutions are used. Accessing the MIMIX Main Menu on page 24 describes the MIMIX Basic Main menu and the MIMIX Intermediate Main Menu. The MIMIX Basic Main menu is used to access the MIMIX Availability Status (WRKMMXSTS) display. 16

17 MIMIX concepts MIMIX concepts Product concepts The following subtopics organize the basic concepts associated with MIMIX into related groups. More detailed information is available in the MIMIX Administrator Reference book. MIMIX installation - The network of IBM Power Systems systems that transfer data and objects among each other using functions of a common MIMIX product. A MIMIX installation is defined by the way in which you configure the MIMIX product for each of the participating systems. A system can participate in multiple independent MIMIX installations. Replication - The activity that MIMIX performs to continuously capture changes to critical database files and objects on a production system as they occur, send the changes to a backup system, and apply the changes to the appropriate database file or object on the backup system. Switch - The process by which a production environment is moved from one system to another system and the production environment is made available there. A switch may be performed as part of a planned event such as for system maintenance, or an unplanned event such as a power or equipment failure. MIMIX provides customizable functions for switching. Audits - Audits are predetermined programs that are used to check for differences in replicated objects and other conditions between systems. Audits run and can correct detected problems automatically. Policies control when audits run and many other aspects of how audits are performed. Additional auditing concepts and recommendations are described in the auditing chapter of this book. Automatic recovery - MIMIX provides a set of functions that provide the ability to automatically correct problems detected in a MIMIX installation during database replication, object replication, and auditing. During these activities, when MIMIX detects any of a set of scenarios known to interfere with maintaining your MIMIX environment, it will automatically start recovery actions to correct them. Through policies, you have the ability to disable automatic recovery in any of these areas at the installation or data group level. Application group - A MIMIX construct used to group and control resources from a single point in a way that maintains relationships between them. The use of application groups is best practice for MIMIX Professional and MIMIX Enterprise and required for MIMIX Global. Data group - A MIMIX construct that is used to control replication activities. A data group is a logical grouping of database files, data areas, objects, IFS objects, DLOs, or a combination thereof that defines a unit of work by which MIMIX replication activity is controlled. A data group may represent an application, a set of one or more libraries, or all of the critical data on a given system. Application environments may define a data group as a specific set of files and objects. 17

18 MIMIX concepts Prioritized status - MIMIX assigns a priority to status values to ensure that problems with the highest priorities, those for detected problems or situations that require immediate attention or intervention, are reflected on the highest level of the user interface. Additional detail and lower priority items can be viewed by drilling down to the next level within the interfaces. Those interfaces are the Work with Systems display and depending on your configuration, either the Work with Application Groups display or the Work with Data Groups display. Policies - A policy is a mechanism used to enable, disable, or provide input to a function such as replication, auditing, or MIMIX Model Switch Framework. For most policies, the initially shipped values apply to an installation. However, policies can be changed and most can also be overridden for individual data groups. Policies that control when audits are automatically performed can be set only for each specific combination of audit rule and data group. Notifications - A notification is the resulting automatic report associated with an event that has already occurred. The severity of a notification is reflected in the overall status of the installation. Notifications can be generated by a process, program, command, or monitor. Because the originator of notifications varies, it is important to note that notifications can represent both real-time events as well as events that occurred in the past but, due to scheduling, are being reported in the present. Recoveries - This term recovery is used in two ways. The most common use refers to the recovery action taken by a replication process or an audit to correct a detected difference when automatic recovery polices are enabled. The second use refers to a temporary report that provides details about a recovery action in progress that is created when the recovery action starts and is removed when it completes. System role concepts MIMIX uses several pairs of terms to refer to the role of a system within a particular context. These terms are not interchangeable. Production system and backup system - These terms describe the role of a system relative to the way applications are used on that system. A production system is the system currently running the production workload for the applications. In normal operations, the production system is the system on which the principal copy of the data and objects associated with the application exist. A backup system is the system that is not currently running the production workload for the applications. In normal operations, the backup system is the system on which you maintain a copy of the data and objects associated with the application. These roles are not always associated with a specific system. For example, if you switch application processing to the backup system, the backup system temporarily becomes the production system. Typically, for normal operations in basic two-system environment, replicated data flows from the system running the production workload to the backup system. Source system and target system - These terms identify the direction in which an activity occurs between two participating systems. 18

19 MIMIX concepts A source system is the system from which MIMIX replication activity between two systems originates. In replication, the source system contains the journal entries. Information from the journal entries is either replicated to the target system or used to identify objects to be replicated to the target system. A target system is the system on which MIMIX replication activity between two systems completes. Management system and network system - These terms define the role of a system relative to how the products interact within a MIMIX installation. These roles remain associated with the system within the MIMIX installation to which they are defined. One system in the MIMIX installation is designated as the management system and the remaining one or more systems are designated as network systems. A management system is the system in a MIMIX installation that is designated as the control point for all installations of the product within the MIMIX installation. The management system is the location from which work to be performed by the product is defined and maintained. Often the system defined as the management system also serves as the backup system during normal operations. A network system is any system in a MIMIX installation that is not designated as the management system (control point) of that MIMIX installation. Work definitions are automatically distributed from the management system to a network system. Often a system defined as a network system also serves as the production system during normal operations. Journaling concepts MIMIX uses journaling to perform replication and to support newer analysis functionality. Journaling and object auditing - Journaling and object auditing are techniques that allow object activity to be logged to a journal. Journaling logs activity for selected objects of specific object types to a user journal. Object auditing logs activity for all objects to the security audit journal (QAUDJRN, the system journal), including those defined to a user journal. MIMIX relies on these techniques and the entries placed in the journal receivers for replicating logged activity. Journal - An IBM i system object that identifies the objects being journaled and the journal receivers associated with the journal. The system journal is a specialized journal on the system which MIMIX uses. Journal receiver - An IBM i system object that is associated with a journal and contains the log of all activity for objects defined to the journal. Journal entry - A record added to a journal receiver that identifies an event that occurred on a journaled object. MIMIX uses file and record level journal entries to recreate the object on a designated system. Remote journaling - A function of IBM i that allows you to establish journals and journal receivers on one system and associate them with specific journals and journal receivers on another system. Once the association is established, the operating system can use the pair of journals to replicate journal entries in one direction, from the local journal to the remote journal on the other system. In some configurations, 19

20 MIMIX concepts MIMIX uses remote journaling for transferring data to be replicated from the source system to the target system. Configuration concepts MIMIX configuration provides considerable flexibility to enable supporting a wide variety of customer environments. Configuration is implemented through sets of related commands. The following terms describe configuration concepts. Definitions - MIMIX uses several types of named definitions to identify related configuration choices. System definitions identify systems that participate in a MIMIX installation. Each system definition identifies one system. Transfer definitions identify the communications path and protocol to be used between systems. Journal definitions identify journaling environments that MIMIX uses for replication Each journal definition identifies a system and characteristics of the journaling environment on that system. Data group definitions identify the characteristics of how replication occurs between two systems. Each data group definition determines the direction in which replication occurs between the systems, whether that direction can be switched, and the default processing characteristics for replication processes. Application group definitions identify whether the replication environment does or does not use IBM i clustering. When clustering is used, the application group also defines information about an application or proprietary programs necessary for controlling operations in the clustering environment. Data group entries - A data group entry is a configuration construct that identifies a source of information to be replicated by or excluded from replication by a data group. Each entry identifies at least one object and its location on the source system. Classes of data group entries are based on object type. MIMIX uses data group entries to determine whether a journal entry should be replicated. Data groups that replicate from both the system journal and a user journal can have any combination of data group entries. Remote journal link (RJ link) - An RJ link is a MIMIX configuration element that identifies an IBM i remote journaling environment used by user journal replication processes. An RJ link identifies the journal definitions that define the source and target journals, primary and secondary transfer definitions for the communications path used by MIMIX, and whether the IBM i remote journal function sends journal entries asynchronously or synchronously. Cooperative processing - Cooperative processing refers to MIMIX techniques that efficiently replicate certain object types by using a coordinated effort between the system journal and user journal replication paths. Configuration choices in data group definitions and data group entries determine the degree of cooperative processing used between the system journal and user journal replication paths when replicating database files, IFS objects, data areas, and data queues. 20

21 MIMIX concepts Process concepts Tracking entries - Tracking entries identify objects that can be replicated using advanced journaling techniques and assist with tracking the status of their replication. A unique tracking entry is associated with each IFS object, data area, and data queue that is eligible for replication using advanced journaling. IFS tracking entries identify eligible, existing IFS objects while object tracking entries identify eligible, existing data areas and data queues. The following terms identify MIMIX processes. Some, like the system manager, are required to allow MIMIX to function. Others, like procedures, are used only when invoked by users. Replication path - A replication path is a series of processes used for replication that represent the critical path on which data to be replicated moves from its origin to its destination. MIMIX uses two replication paths to accommodate differences in how replication occurs for user journal and system journal entries. These paths operate with configurable levels of cooperation or can operate independently. The user journal replication path captures changes to critical files and objects configured for replication through a user journal. When configuring this path, shipped defaults use the remote journaling function of the operating system to simplify sending data to the remote system. The changes are applied to the target system. The system journal replication path handles replication of critical system objects (such as user profiles, program objects, or spooled files), integrated file system (IFS) objects, and document library object (DLOs) using the system journal. Information about the changes are sent to the target system where it is applied. System manager - The system manager is a pair of communications jobs between the management system and a network system which must be active to enable replication. The system manager monitors for configuration changes and automatically moves any configuration changes to the network system. Dynamic status changes are also collected and returned to the management system. The system manager also gathers messages and timestamp information from the network system and places them in a message log and timestamp file on the management system. In addition, the system manager performs periodic maintenance tasks, including cleanup of the system and data group history files. Journal manager - The journal manager is a job on each system that MIMIX uses to maintain the journaling environment on that system. By default, MIMIX performs both change management and delete management for journal receivers used by the replication process. Collector services - A group of jobs that are necessary for MIMIX to track historical data and to support using the MIMIX portal application within the Vision Solutions Portal. One or more collector service jobs collect and combine MIMIX status from all systems. Cluster services - When MIMIX Global is configured for IBM i clustering, MIMIX uses the cluster services function provided by IBM i to integrate the system management functions needed for clustering. Cluster services must be active in order for a cluster 21

22 MIMIX concepts node to be recognized by the other nodes in the cluster. MIMIX integrates starting and stopping cluster services into status and commands for controlling processes that run at the system level. Target journal inspection - A MIMIX process that reads a journal on a system being used as the target system for replication. The process identifies people or processes other than MIMIX that accessed replicated objects on the target system. Users can access the resulting information from the Replicated Objects portlet within the MIMIX portal application in Vision Solutions Portal. Procedures and steps - Procedures and steps are a highly customizable means of performing operations for application groups. A set of default procedures for each application group provide the ability to start, end, perform pre-check activity for switching, and switch the application group. Each operation is performed by a procedure that consists of a sequence of steps and multiple jobs. Each step calls a predetermined step program to perform a specific sub-task of the larger operation. Steps also identify runtime attributes for handling before and after the program call within the context of the procedure. Log space - A MIMIX object that provides an efficient storage and manipulation mechanism for replicated data that is temporarily stored on the target system during the receive and apply processes. Additional switching concepts The following concepts are specific to switching. Environments configured with application groups perform switching through procedures. Planned switch - An intentional change to the direction of replication for any of a variety of reasons. You may need to take the system offline to perform maintenance on its hardware or software, or you may be testing your disaster recovery plan. In a planned switch, the production system (the source of replication) is available. When you perform a planned switch, replication is ended on both the source and target systems. The next time you start replication, it will be set to replicate in the opposite direction. Unplanned switch - A change the direction of replication as a response to a problem. Most likely the production system is no longer available. When you perform an unplanned switch, you must initiate the switch from the target system. Replication is ended on the target system. The next time you start replication, it will be set to replicate in the opposite direction. MIMIX Model Switch Framework - A set of programs and commands that provide a consistent framework to be used when performing planned or unplanned switches in environments that do not use application groups. Typically, a model switch framework is customized to your environment through its exit programs. MIMIX Switch Assistant - A guided user interface that guides you through switching using your default MIMIX Model Switch Framework. MIMIX Switch Assistant is accessed from the MIMIX Basic Main Menu and does not support application groups. 22

23 Best practices for maintaining your MIMIX environment Best practices for maintaining your MIMIX environment MIMIX is shipped with default settings that incorporate many best practices for maintaining your environment. Others may require changing policies and adopting best practices within your organization. Best practices include: Allow MIMIX to automatically correct differences detected during database and object replication processes that would otherwise result in errors. If MIMIX is unable to perform the recovery, the problem is reported as a replication error (a file is placed in held error or an object is in error). Allow MIMIX to automatically perform audits and to automatically recover any differences detected by audits. Best practice is to allow regularly scheduled audits of all objects configured for replication and daily audits of prioritized categories of replicated objects. User interfaces summarize audit results and indicate whether MIMIX is unable to recover an object. Perform all audits with the audit level set at level 30 immediately prior to a planned switch to the backup system and before switching back to the production system. Perform switches on a regular basis. Best practice is to switch every three to six months. You need to set aside time for performing planned switches. Environments that continue to use MIMIX Switch Assistant can use policies so that compliance with regular switching is automatically reported in the user interface. Authority to products and commands If your MIMIX environment takes advantage of the additional security available in the product and command authority functions which Vision Solutions provides through License Manager, you may need a higher authority level in order to perform MIMIX daily operations. A MIMIX administrator can change your authorization level to commands and displays. Authorization levels typically fall into these categories: Viewing information requires display (*DSP) authority. Controlling operations requires operator (*OPR) authority. Creating or changing configuration requires management (*MGT) authority. For example, consider audits. You can view an audit if you have display authority, perform audits if you have operator authority, and change policies that affect how auditing is performed if you have management authority. For more information about these provided security functions, see the Using License Manager book. 23

24 Accessing the MIMIX Main Menu Accessing the MIMIX Main Menu The MIMIX command accesses the main menu for a MIMIX installation. The MIMIX Main Menu has two assistance levels, basic and intermediate. The command defaults to the basic assistance level, shown in Figure 1, with its options designed to simplify day-to-day interaction with MIMIX. Figure 2 shows the intermediate assistance level. The options on the menu vary with the assistance level. In either assistance level, the available options also depend on the MIMIX products installed in the installation library and their licensing. The products installed and the licensing also affect subsequent menus and displays. Accessing the menu - If you know the name of the MIMIX installation you want, you can use the name to library-qualify the command, as follows: Type the command library-name/mimix and press Enter. The default name of the installation library is MIMIX. If you do not know the name of the library, do the following: 1. Type the command LAKEVIEW/WRKPRD and press Enter. 2. Type a 9 (Display product menu) next to the product in the library you want on the Vision Solutions Installed Products display and press Enter. Changing the assistance level - The F21 key (Assistance level) on the main menu toggles between basic and intermediate levels of the menu. You can also specify the the Assistance Level (ASTLVL) parameter on the MIMIX command. Figure 1. MIMIX Basic Main Menu MIMIX MIMIX Basic Main Menu System: SYSTEM1 Select one of the following: 1. Work with application groups WRKAG 2. Start MIMIX 3. End MIMIX 4. Switch all application groups 5. Start or complete switch using Switch Asst. 6. Work with data groups WRKDG 10. Availability status WRKMMXSTS 11. Configuration menu 12. Work with monitors WRKMON 13. Work with messages WRKMSGLOG 14. Cluster menu More... Selection or command ===> F3=Exit F4=Prompt F9=Retrieve F21=Assistance level F12=Cancel (C) Copyright Vision Solutions, Inc., 1990,

25 Accessing the MIMIX Main Menu Note: On the MIMIX Basic Main Menu, options 5 (Start or complete switch using Switch Asst.) and 10 (Availability Status) are not recommended for installations that use application groups. Figure 2. MIMIX Intermediate Main Menu MIMIX Select one of the following: MIMIX Intermediate Main Menu System: SYSTEM1 1. Work with data groups WRKDG 2. Work with systems WRKSYS 3. Work with messages WRKMSGLOG 4. Work with monitors WRKMON 5. Work with application groups WRKAG 6. Work with audits WRKAUD 7. Work with procedures WRKPROC 11. Configuration menu 12. Compare, verify, and synchronize menu 13. Utilities menu 14. Cluster menu More... Selection or command ===> F3=Exit F4=Prompt F9=Retrieve F21=Assistance level F12=Cancel (C) Copyright Vision Solutions, Inc., 1990,

26 CHAPTER 2 MIMIX policies Each MIMIX policy is a mechanism used to enable, disable, or provide input to a function such as replication, auditing, or MIMIX Model Switch Framework. A policy may also determine how you are notified about certain problems that may occur. For most policies, the initially shipped values apply to an installation. However, policies can be changed and most can also be overridden for individual data groups. When a policy is set for a data group, it takes precedence over the installation policy. Some policies, such as ones that control when audits are automatically submitted, apply to individual audit rules for specific data groups. Policies must be changed from the management system. Changing policies requires that you have management-level authority to the Set MIMIX Policy (SETMMXPCY) command. You can set policies from a command line or from the Work with Audits, the MIMIX Availability Status, and the Work with DG Definitions displays. The topics in this chapter include: Environment considerations for policies on page 27 describes additional considerations for setting policies for environments with more than two nodes or bi-directional replication. Also, applications and features can conflict with policycontrolled automatic recovery functions. Setting policies - general on page 29 provides basic procedures for changing policies. Other topics in this chapter include more in-depth procedures for specific policy-controlled functionality. Policies which affect an installation on page 31 identifies the policies that are set for an installation and which cannot be overridden by a data group-level setting. Also, this includes procedures for changing retention criteria for procedure history. Policies which affect replication on page 32 identifies the policies associated with automatic error detection and correction during replication and identifies the common object and file error situations that can be automatically recovered. Policies which affect auditing on page 36 identifies policies that influence audit runtime behavior and control scheduling for automatically submitted audits. Shipped audits and their descriptions and default scheduling details are included. Changing auditing policies on page 41 provides additional information and procedures for changing policies associated with auditing. This includes changing the auditing level before switching, changing automatic audit scheduling, changing audit history retention, restricting auditing based on the state of data groups, and disabling auditing. Policies for switching with model switch framework on page 48 identify the policies associated with model switch framework and includes instructions for changing these policies. Policy descriptions on page 50 describes polices used by MIMIX. 26

27 Environment considerations for policies Environment considerations for policies Default settings for policies are chosen to address the needs of a broad set of customer environments. However, in more complex environments, you need to consider the effect of policies. Also, applications and other MIMIX features in some environments can conflict with automatic recovery actions during replication and with auditing. Policies in environments with more than two nodes or bi-directional replication Policy values may affect data throughout your entire environment, not just a single installation or data group. This is of particular concern in environments that have more than two systems (nodes) or which have replication occurring simultaneously in more than one direction (bi-directional). Specifically, be aware of the following: In these environments, the value *DISABLED for the Objects only on target policy is recommended. When the policy is disabled, audits will detect that objects exist only on the target system but will not attempt to correct them. The commands used by an audit are aware of all objects on the target system, not just those which originate from the source system of the data group associated with the audit. In these environments, the values *DELETE and *SYNC must be used with care. When the policy value is Delete, audits will delete objects which may have originated from systems not associated with the data group being audited. When the policy value is Synchronize, audits will synchronize the objects to the source system of the data group being audited, which may not be the source system from which they originated. Synchronization of user profiles and authorization lists associated with an object will occur unless the user profiles and authorization lists are explicitly excluded from the data group configuration. In the environments mentioned, this may result in user profiles and authorization lists being synchronized to other systems in your configuration. This behavior occurs whenever any of the automatic recovery policies are enabled (database, object, audit). To prevent this from occurring, you must explicitly exclude the user profiles and authorization lists from replication for any data group for which you do not want them synchronized. In a simultaneously bi-directional environment, determine which system wins in the event of a data conflict, that is, which system will be considered as having the correct data. Choose one direction of replication that will be audited and allow auditing for those data groups. Disable audits for data groups that replicate in the opposite direction. For example, data groups AB and BA are configured for bidirectional replication between system A and system B. Data group AB replicates from system A to system B and data group BA replicates the opposite direction. System B is also the management system for this installation. You chose system A as the winning system and want to permit auditing in the direction from A to B. The Audit level policy for data group AB must be set to a level that permits audits to run (level 10 or higher). The Audit level policy for data group BA must be set to disable audits. The results of audits of data group AB will be available on system B, because system B is the management system and default policy values cause 27

28 Environment considerations for policies rules to be run from the management system. In environments with three or more systems in the same installation, you need to evaluate each pair of systems. For each pair of systems, evaluate the directions in which replication is permitted. If any pair of systems supports simultaneous bidirectional replication, determine the winning system in each pair and determine the direction to be audited. Set the audit level policy to permit auditing for the data group that replicates in the chosen direction. Disable auditing for the data group which replicates in the other direction. You may also want to consider changing the values of the Run rule on system policy for the installation or the audited data groups to balance processing loads associated with auditing. In environments that permit multiple management systems in the same installation, in addition to evaluating the direction of replication permitted within each pair of systems, you must also consider whether the systems defined by each data group are both management systems. If any pair of systems supports simultaneous bi-directional replication, choose the winning system and change the Audit level policies for each data group so that only one direction is audited. You may need to change the Run rule on system policy to prevent certain data groups from being audited from specific management systems. When to disable automatic recovery for replication and auditing At times, you may need to disable automatic recoveries during replication and auditing for certain data groups because a feature in use or an application being replicated may interact with auditing in an undesirable way. Features - Do not use automatic recoveries during auditing and replication in any data group that is using functions provided by the MIMIX CDP feature. This feature, which requires an additional license key, permits you to perform operations associated with maintaining continuous data protection. By configuring a recovery window for a data group, you introduce an automatic delay into when the apply processes complete replication. By setting a recovery point for a data group, you identify a point that, when reached, will cause the apply processes to be suspended. In both cases, source system changes have been transferred to the target system but have not been applied. In such an environment, comparisons will report differences and automatic recoveries will attempt recovery for items that have not completed replication. To prevent this from occurring, disable comparisons and automatic recoveries for any data group which uses the MIMIX CDP feature. For details, see Disabling audits and recovery when using the MIMIX CDP feature on page 29. Applications - At times, data groups for some applications will encounter problems if the application cannot acquire locks on objects that are defined to MIMIX. These data groups may need to be excluded from auditing. MIMIX acquires locks occasionally to save and restore objects within the replication environment. Some applications may fail when they cannot acquire a lock on an object. Refer to our Support Central for FAQs that list specific applications whose data groups should be excluded from auditing. For those excluded data groups, you can still run compares to determine if objects are not synchronized between source and target systems. Care must be taken to recover from these unsynchronized conditions.the applications may need to be ended prior to manually synchronizing the objects. 28

29 Setting policies - general To exclude a data group from audits, use the instructions in Preventing audits from running on page 45. Disabling audits and recovery when using the MIMIX CDP feature The functions provided by the MIMIX CDP feature 1 create an environment in which source system changes have been transferred to the target system but have not been applied. Any data group which uses this feature must disable automatic comparisons and automatic recovery actions for the data group. Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. For the Data group definition, specify the full three-part name of the data group that uses the MIMIX CDP feature. 3. Press Enter to see all the policies and their current values. 4. For Automatic object recovery, specify *DISABLED. 5. For Automatic database recovery, specify *DISABLED. 6. For Automatic audit recovery, specify *DISABLED. 7. For Audit level, select *DISABLED. 8. To accept the changes, press Enter. Setting policies - general Policies must be changed from the management system. Changing policies requires that you have management-level authority to the Set MIMIX Policy (SETMMXPCY) command. The following procedures describe the basic procedures for setting policies. Changing policies for an installation This procedure changes a policy value at the installation level. The installation level value will overridden if a data group level policy has been specified with a value other than *INST. Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. Verify that the value specified for Data group definition is *INST. 3. Press Enter to see all the policies and their current values. 4. Specify a value for the policy you want. Use F1 (Help) to view descriptions of possible values. 1. The MIMIX CDP feature requires an additional license key. 29

30 Setting policies - general 5. To accept the changes, press Enter. Changing policies for a data group Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. For the Data group definition, specify the full three-part name. 3. Press Enter to see all the policies and their current values. 4. Specify a value for the policy you want defined for the data group. Use F1 (Help) to view descriptions of possible values. 5. To accept the changes, press Enter. Resetting a data group-level policy to use the installation level value Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. For the Data group definition, specify the full three-part name. 3. Press Enter to see all the policies and their current values. 4. For the policy you want to reset, specify *INST. 5. To accept the changes, press Enter. 30

31 Policies which affect an installation Policies which affect an installation While many policies can be set for an installation, the policies in Table 1 cannot be overridden for an individual data group. At the data group level, these policies always have a value of *INST. Table 1. Policy Policies that can be set only at the installation level and shipped default values. Shipped Values Installation Independent ASP library ratio 5 Procedure history retention Minimum days Minimum runs per procedure Min. runs per switch procedure Changing retention criteria for procedure history The procedure history retention policy determines how long to retain historical information about procedure runs that completed, completed with errors, or that failed or were canceled and have been acknowledged. Environments configured with application groups use procedures to control operations such as starting, ending, or switching. History information for a procedure includes timestamps indicating when the procedure was run and detailed information about each step within the procedure. The policy specifies how many days to keep history information and the minimum number of runs to keep. You can specify a different number of runs to keep for switch procedure runs than what is kept for other types of procedures. Each procedure run is evaluated individually against the policy and its history information is retained until the specified minimum days and minimum runs are both met. When a procedure run exceeds these criteria, system manager cleanup jobs will remove the historical information for that procedure run from all systems. The values specified at the time the cleanup jobs run are used for evaluation. To change the procedure history retention policy for the installation, do the following: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. Verify that the value *INST is specified for the Data group definition prompt: 3. Press Enter to see all the policies and their current values. 4. Locate the Procedure history retention policy. The current values are displayed. Specify values for the elements you want to change. 5. To accept the changes, press Enter. 31

32 Policies which affect replication Policies which affect replication Table 2 identifies the policies which can affect replication and their shipped default values. Table 2. Policies associated with replication and shipped default values. Policy Shipped Values Replication Processes Installation Data Groups Data group definition *INST Name 1 Automatic system journal recovery Automatic user journal recovery System Journal Yes *ENABLED *INST 1 Yes 2 User Journal Yes *ENABLED *INST Yes 2 System journal recovery notify on success User journal recovery notify on success *YES *INST Yes *YES *INST Yes DB apply cache *DISABLED *INST Yes Access path maintenance 3 Optimize for DB apply Maximum number of jobs *DISABLED 99 *INST *INST Yes Synchronize threshold size 9,999,999 *INST Yes Yes Number of third delay retry attempts 100 *INST Yes Third delay retry interval 15 *INST Yes 1. A data group definition value of *INST indicates the policy is installation-wide. A name indicates the policies are in effect only for the specified data group. 2. When this policy is enabled, the other policies in the same column are in effect unless otherwise noted. 3. This policy is available only on systems running service pack or higher. When running on earlier levels, the Parallel AP maintenance provides similar functionality. For more information about both access path maintenance functions, see the MIMIX Administrator Reference book. MIMIX can automatically attempt to correct problems it encounters during replication when the policies for Automatic system journal recovery and Automatic user journal recovery are enabled. The following topics identify what errors can be recovered in this way: Errors handled by automatic database recovery on page 33 Errors handled by automatic object recovery on page 34 32

33 Policies which affect replication Errors handled by automatic database recovery MIMIX can detect and correct the most common file error situations that occur during database replication. When the Automatic database recovery policy is enabled, database replication processes detect the types of errors listed in Table 3. When an error is detected, MIMIX automatically attempts to correct the error by starting a job to perform an appropriate recovery action. The recovery action also sends a report of a recovery in progress to the user interface. The reports are on the Work with Recoveries display (WRKRCY command). When the recovery action completes, the report is removed. The DB rcy. notify on success policy determines whether a successful recovery generates an informational notification. Only when all recovery options are exhausted without success is a file placed in hold error (*HLDERR) status. Recovery actions that end in an error do not generate a separate error notification because the error is already reflected in MIMIX status. Table 3. Error Errors detected and corrected during database replication when automatic database recovery is enabled. Description File level errors - and - Unique-key record level error Record level errors Errors on IFS objects configured for user journal replication Errors on data area and data queue objects configured for user journal replication Errors when DBAPY cannot open the file or apply transactions to the file Typically invoked when there is a missing library, file, or member. Also invoked when an attempt to write a record to a file results in a unique key violation. Without database autonomics, these conditions result in the file being placed in *HLDERR status. Invoked when the database apply process detects a data-level issue while processing record-level transactions. Without database autonomics, any configured collision resolution methods may attempt to correct the error. Otherwise, these conditions result in the file being placed in *HLDERR status. Invoked during the priming of IFS tracking entries when replicated IFS objects are determined to be missing from the target system. Priming of tracking entries occurs when a data group is started after a configuration change or when Deploy Data Grp. Configuration (DPYDGCFG) is invoked. Invoked during the priming of object tracking entries when replicated data area and data queue objects are determined to be missing from the target system. Priming of tracking entries occurs when a data group is started after a configuration change or when the Deploy Data Grp. Configuration (DPYDGCFG) is invoked. Invoked when a temporary lock condition or an operating system condition exists that prevents the database apply process (DBAPY) from opening the file or applying transactions to the file. Without database autonomics, users typically have to release the file so the database apply process (DBAPY) can continue without error. 33

34 Policies which affect replication Errors handled by automatic object recovery MIMIX can detect and correct the most common object error situations that occur during replication. When the Automatic object recovery policy is enabled, object replication processes detect the types of errors listed in Table 4. When an error is detected, MIMIX automatically attempts to correct the error by starting a job to perform an appropriate recovery action. Unless the object is explicitly excluded from replication for a data group, the autonomic recovery action will synchronize the object to ensure that it is on the target system. Note: Object automatic recovery does not detect or correct the following problems: Missing spooled files on the target system. Files and objects that are cooperatively processed. Although the files and objects are not addressed, problems with authorities for cooperatively processed files and objects are addressed. Activity entries that are stuck in a perpetual pending status (PR, PS, PA, or PB). The recovery action also sends a report of a recovery in progress to the user interface. In a 5250 emulator, the reports are on the Work with Recoveries display (WRKRCY command). When the recovery action completes, the report is removed. The Obj. rcy. notify on success policy determines whether a successful recovery generates an informational notification. Only when all recovery options are exhausted without success is an activity entry placed in error status. Recovery actions that end in an error do not generate a separate error notification because the error is already reflected in MIMIX status. Table 4. Error Errors detected and recoveries attempted by object autonomics during object replication Description Missing objects on target system 1 Missing parent objects on target system 1 Missing *USRPRF objects on target system 1 An object (library-based, IFS, or DLO) exists on the source system and is within the name space for replication, but MIMIX detects that the object does not exist on the target system. Without object automatic recovery, this results in a failed activity entry. Notes: Missing spooled files are not addressed. Missing objects that are configured for cooperative processing are not synchronized. However, any problems with authorities (*AUTL or *USRPRF) for the missing objects are addressed. Any operation against an object whose parent object is missing on the target system. Without object autonomics, this condition results in a failed activity entry due to the missing parent object. Any operation that requires a user profile object (*USRPRF) that does not exist on the target system. Without object autonomics, this results in authority or object owner issues that cause replication errors. 34

35 Policies which affect replication Table 4. Error Errors detected and recoveries attempted by object autonomics during object replication Description Missing *AUTL objects on target system 1 In-use condition Any operation that requires a authority list (*AUTL) that does not exist on the target system.without object autonomics, this results in authority issues that cause replication errors. Applications which hold persistent locks on objects can result in object replication errors if the configured values for delay/retry intervals are exceeded. Default values in the data group definition provide approximately 15 minutes during which MIMIX attempts to access the object for replication. If the object cannot be accessed during this time, the result is activity entries with errors of Failed Retrieve (for locked objects on the source system) and Failed Apply (for locked objects on the target system) and a reason code of *INUSE. Notes: 1. The Number of third delay/retries policy and the Third retry interval policy determine whether automatic recovery is attempted for this error. 2. Automatic recovery for this error is not attempted when the objects are configured for cooperative processing. 1. The synchronize command used to automatically recover this problem during replication will correct this error any time the command is used. 35

36 Policies which affect auditing Policies which affect auditing Policies for auditing are divided into these subsets: Policies that affect the behavior of all audits in an installation. These policies can be overridden at the data group level. When set for a specific data group, these policies affect all audits for the data group. Policies that affect when audits automatically run and how those audits select objects. These policies are set for each unique combination of audit and data group. Policies for auditing runtime behavior The policies identified in Table 5 affect all audit runs regardless of whether the audit was automatically submitted or manually invoked. These policies can be set for the installation as well as overridden for an individual data group. The shipped default values for both levels are indicated. When the Set MIMIX Policies (SETMMXPCY) command specifies a data group definition value of *INST, the policies being changed are effective for all data groups in the installation, unless a data group-level override exists. When the data group definition specifies a name, policies which specify the value *INST inherit their value from the installation-level policy value and polices which specify other values are in effect for only the specified data group. Table 5. Policy Shipped default values of policies associated with auditing runtime behavior. Shipped Values Installation Data Groups Data group definition *INST Name Automatic audit recovery *ENABLED *INST Audit notify on success *RULE *INST Notification severity *RULE *INST Object only on target action *DISABLED *INST Journal attribute differences action MIMIX configured higher MIMIX configured lower *CHGOBJ *NOCHG *INST *INST User journal apply threshold action *END *INST Maximum rule runtime 1440 *INST Audit warning threshold 1 7 *INST Audit action threshold 1 14 *INST Audit level *LEVEL30 *INST Run rule on system *MGT *INST 36

37 Policies which affect auditing Table 5. Policy Shipped default values of policies associated with auditing runtime behavior. Shipped Values Action for running audits Inactive data group Repl. process in threshold Audit history retention Minimum days Minimum runs per audit Object details DLO and IFS details Installation *NOTRUN 2 *NOTRUN 7 1 *YES *YES Data Groups *INST *INST *INST *INST *INST *INST Synchronize threshold size 9,999,999 *INST CMPRCDCNT commit threshold *NOMAX *INST 1. These policies are not limited to recovery actions. 2. This is the default shipped value on systems running MIMIX service pack or higher. For earlier software levels, the shipped default value is *NONE. Policies for submitting audits automatically The Audit rule, Audit schedule, and Priority audit policies control when audits are automatically submitted. These policies do not have a shipped value for the installation level. The shipped values for the data group level are listed in Table 6. If the Audit level policy is disabled, all auditing is disabled, regardless of the values specified for Audit schedule and Priority audit policies. This includes manually submitted audits. Each shipped audit rule has default values for submitting priority audits as well as scheduled audits. The shipped values for a rule are used for all new data groups. When you specify names for Data group definition and Audit rule on the SETMMXPCY command, you can adjust the values for a specific audit of a single data group. Table 6. Policy Shipped default values of policies for automatically submitting audits. Shipped Values Installation Data Groups Data group definition *INST Name Audit rule Varies by rule 37

38 Policies which affect auditing Table 6. Policy Shipped default values of policies for automatically submitting audits. Shipped Values Audit schedule State Frequency Scheduled date Scheduled day Scheduled time Relative day of month Priority audit State Start after Start until New objects selected Changed objects selected Unchanged objects selected Audited with no differences Installation Data Groups *ENABLED 1 *WEEKLY 1 *SUN 2 Varies by rule, see Table 7. *ENABLED *DAILY *DAILY *WEEKLY *MONTHLY 1. The State element in the Audit schedule policy is available in MIMIX version and higher. For data groups that existed before upgrading to version , if the Frequency specified was a value other than *NONE, that value is preserved by the upgrade process and the State is set to *ENABLED. If the Frequency value was *NONE, it is changed to *WEEKLY and the State set to *DISABLED. 2. The shipped default for Scheduled day changed in MIMIX version 7.1. For data groups created after installing version 7.1, the shipped default is *SUN (previously, it was *ALL). For data groups that existed before upgrading to version 7.1, the previous value for Scheduled day remains unchanged. 3. The Priority audit policy is new in MIMIX version 7.1. The State element for the Priority audit policy is available in MIMIX version and higher. For data groups that existed before upgrading from any version 7.0 level to version or higher, State is set to *DISABLED and Start after is set to For data groups that existed before upgrading from versions through to version or higher, if the Start after value specified was a value other than *NONE, that value is preserved by the upgrade process and the State is set to *ENABLED. However if the Start after value was *NONE, it is changed to and State is set to *DISABLED. When automatically submitted audits run For each audit rule, its shipped values enable both prioritized audits and scheduled audits to run automatically. A prioritized audit starts one or more times an hour every day during the time range specified in the Priority audit policy. A scheduled audit runs once at its specified time on the days or dates for its frequency as specified in the Audit schedule policy. For scheduled audits, the shipped value for start time of each 38

39 Policies which affect auditing audit rule is staggered, beginning at 2 a.m. Table 7 shows the default times for priority audits versus scheduled audits. Table 7. MIMIX rules and their shipped default times for Audit schedule (SCHEDULE) policy. Shipped Priority Start Range Shipped Scheduled Time Rule Name Description Job Name n/a 1 2:00 a.m. #DGFE Checks configuration for files using cooperative processing. Uses the Check Data Group File Entries (CHKDGFE) command. sdn_dgfe All other audits: 3 a.m. to 8 a.m. 2:05 a.m. #OBJATR Compares all attributes for all object types supported for replication. Uses the Compare Object Attributes (CMPOBJA) command 2:10 a.m. #FILATR Compares all file attributes. Uses the Compare File Attributes (CMPFILA) command. 2:15 a.m. #IFSATR Compares IFS attributes. Uses the Compare IFS Attributes (CMPIFSA) command. 2:20 a.m. #FILATRMBR Compares basic file attributes at the member level. Uses the Compare File Attributes (CMPFILA) command. 2:25 a.m. #DLOATR Compares all DLO attributes. Uses the Compare DLO Attributes (CMPDLOA) command. 2:30 a.m. #MBRRCDCNT Compares the number of current records (*CURRDS) and the number of deleted records (*NBRDLTRCDS) for physical files that are defined to an active data group. Uses the Compare Record Counts (CMPRCDCNT) command. Note: Equal record counts suggest but do not guarantee that files are synchronized. This audit does not have a recovery phase. Differences detected by this audit appear as not recovered in the Audit Summary. 2:35 a.m. #FILDTA 2 Compares file contents. Uses the Compare File Data (CMPFILDTA) command. sdn_objatr sdn_filatr sdn_ifsatr sdn_mbratr sdn_dloatr sdn_rcdcnt sdn_fildta 1. The #DGFE audit is not eligible for prioritized auditing because it checks configuration data, not objects. 39

40 Policies which affect auditing 2. The #FILDTA audit and the Compare File Data (CMPFILDTA) command require TCP/IP communications as their communications protocol. 40

41 Changing auditing policies Changing auditing policies This topic describes how to change specific policies that affect auditing behavior and when automatic audits will run. MIMIX service providers are specifically trained to provide a robust audit solution that meets your needs. Changing when automatic audits are allowed to run Policies control aspects of when both prioritized auditing and scheduled auditing are automatically submitted. To effectively audit your replication environment you may need to fine-tune when one or both types of audits are submitted. For both types of auditing, consider: How much time or system resource can you dedicate to audit processing each day, week, or month? How often should all data within the database be audited? Business requirements as well as time and system resources need to be considered. Does automatic scheduling conflict with regularly scheduled backups? Are there jobs running at the same time as audits that could lock files needing to be accessed during recovery? For scheduled auditing (which select all objects), also consider: Are there are a large number of objects to be compared? Are there a large number of objects for which a rule is expected to attempt recovery? Specific audits may have additional needs. See Considerations for specific audits on page 127. While you may decide to vary the scheduled times, it is recommended that you maintain the same relative order indicated in When automatically submitted audits run on page 38. Changing scheduling criteria for automatic audits Both scheduled audits and priority audits have scheduling information. A change to an audit s scheduling information is effective immediately. If an audit is in progress at the time its scheduling information is changed, the change is effective on the next automatic run of the audit. Do the following from the management system: 1. Do one of the following to access the Schedule view of the Work with Audits display: From the MIMIX Intermediate Main Menu, select option 6 (Work with audits) and press Enter. Then use F10 as needed to access the Schedule view. Enter the command: installation-library/wrkaud VIEW(*SCHEDULE) 2. Type 37 (Change audit schedule) next to the audit you want to change and press 41

42 Changing auditing policies Enter. 3. The Set MIMIX Policies (SETMMXPCY) command appears, showing the selected audit rule and data group. The current values for the Audit schedule and Priority audit policies are displayed. Do one of the following: To change when MIMIX is scheduled to run the audit to check all configured objects, specify the values you want for elements of the Audit schedule policy. To change when MIMIX is allowed to submit priority-based runs of the audit every day, specify values for the Start after and Start until elements of the Priority audit policy. 4. To make the changes effective, press Enter. Changing the selection frequency of priority auditing categories When priority auditing is used, you can control how often objects within priorities are eligible for selection. Objects which had differences in their previous audit are always selected. For other priority classes, you can change how often objects within the class are eligible for selection by a prioritized audit. For descriptions of the priority classes with changeable frequencies, see the Priority audit policy description. If an audit is in progress at the time its category frequency information is changed, the change is effective on the next automatic run of the audit. Do the following from the management system: 1. Do one of the following to access the Work with Audits display: From the MIMIX Intermediate Main Menu, select option 6 (Work with audits) and press Enter. Then use F10 as needed to access the Schedule view. Enter the command: installation-library/wrkaud 2. Type 37 (Change audit schedule) next to the audit you want to change and press Enter. 3. The Set MIMIX Policies (SETMMXPCY) command appears, showing the selected audit rule and data group. Page Down to see the current values of the Priority audit policy. 4. Specify values in the following prompts that indicate how often objects in each category are eligible for selection by a priority audit. New objects selected Changed objects selected Unchanged objects selected Audited with no diff. 5. To make the changes effective, press Enter. 42

43 Changing auditing policies Changing the audit level policy when switching Regardless of the level you use for daily operations, Vision Solutions strongly recommends that you perform audits at audit level 30 before the following events to ensure that 100 percent of the data is valid on the target system: Before performing a planned switch to the backup system. Before switching back to the production system. For more information about the risks associated with lower audit levels, see Considerations for user-defined rules on page 660. From a 5250 emulator, do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. Verify that the value specified for Data group definition is *INST. 3. Press Enter to see all the policies and their current values. 4. For Audit level, specify *LEVEL30. Then press Enter. Changing the system where audits are performed The Run rule on system policy determines the system on which audits run. The shipped default is to run all audits for the installation from the management system. When changing the value of this policy, also consider your switching needs. Click this link to see additional information about the Run rule on system policy. Note: This procedure changes a policy value at the installation level. The installation level value can be overridden by a data group level policy value. Therefore, if a data group has value other than *INST for this policy, that value remains in effect. To change the policy for the installation, do the following 1. On the management system type the following command and press F4 (Prompt) installation-library/setmmxpcy 2. Verify that the value *INST appears for the Data group definition. 3. Locate the Run rule on system policy. Specify the value you want. 4. Press Enter. Changing retention criteria for audit history The Audit history retention policy determines whether to retain information about the results of completed audits and the objects that were audited. The policy specifies how many days to keep history information and how many audit runs to keep, as well as whether details about audited library-based objects and audited DLO and IFS 43

44 Changing auditing policies objects are to be kept with the history information. Each audit is evaluated individually against the policy values. The policy is checked when an audit runs to determine whether to keep details about the objects audited by that run. The policy is also checked when system manager cleanup jobs run to determine if any audit has history information which exceeds both specified retention criteria. The policy value in effect at the time each check occurs determines the result. To change the audit history retention policy, do the following: 1. From the MIMIX Intermediate Main Menu, select option 6 (Work with Audits) and press Enter. 2. Determine whether to change the policy for the installation or at the data group level. From the Work with Audits display, do one of the following: To change the policy for all audits in the installation, press F16 (Inst. policies). Then, press Enter when the Set MIMIX Policies (SETMMXPCY) command appears. To change the policy for all audits for a specific data group, type 36 (Change DG policies) next to any audit for the data group you want and press Enter. 3. Locate the Audit history retention policy. The current values for the level you chose in Step 2 are displayed. Specify values for the elements you want to change. Note: When large quantities of objects are eligible for replication, specifying *YES to retain either Object details or DLO and IFS details may use a significant amount of disk storage. Consider the combined effect of the quantity of replicated objects for each data group, the number of days to retain history, the number of audits to retain, and the frequency in which audits are performed. 4. To accept the changes, press Enter. Restricting auditing based on the state of the data group You may want to control when audits are allowed to run based on the state of the data group at the time of the audit request. For example, if you end MIMIX so that a batch process can run, you may want to prevent audits from running while data groups are inactive. If a data group process has a backlog during peak activity, you may want to prevent audits from running while the backlog exists. Or, you may want to prevent only automatic recovery from occurring during a backlog or when the data group is inactive. The Action for running audits policy provides the ability to define what audit activity will be permitted based on the state of the data group at the time of audit request. This policy can be set for an installation or for a specific data group. Note: For installations running service pack and higher, most audits check for threshold conditions in all database and object replication processes, including the RJ link. #FILDTA audits only check for threshold warning conditions in the RJ link and database replication processes. #DLOATR audits only check for threshold warning conditions in object replication processes. 44

45 Changing auditing policies For installations running earlier service packs, only database and object apply processes are checked for thresholds. Restricting audit activity in an installation based on data group state: Do the following from the management system: Note: This procedure changes a policy value at the installation level. The installation level value can be overridden by a data group level policy value. Therefore, if a data group has value other than *INST for this policy, that value remains in effect. 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. Verify that the value specified for Data group definition is *INST. 3. Press Enter to see all the policies and their current values. 4. For Action for running audits, do the following: a. Specify the value you want for Inactive data group that indicates the audit actions to permit when the data group is inactive b. Specify the value you want for Repl. process in threshold that indicates the audit actions to permit when any replication process checked by an audit has reached its configured threshold. 5. To accept the changes, press Enter. Restricting audit activity for a specific data group based on its state: Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. For the Data group definition, specify the full three-part name. 3. Press Enter to see all the policies and their current values. 4. For Action for running audits, do the following: a. Specify the value you want for Inactive data group that indicates the audit actions to permit when the data group is inactive b. Specify the value you want for Repl. process in threshold that indicates the audit actions to permit when any replication process checked by an audit has reached its configured threshold. 5. To accept the changes, press Enter. Preventing audits from running There may be scenarios when you need to disable auditing completely for either an installation or a specific data group. Auditing may not be desirable on a test data group or during system or network maintenance. The Audit level policy can be used to disable all auditing, including manually invoked audits.the Audit level can be set for an installation or for specific data groups. Note that an explicitly set value for a data group will override the installation value and may still allow an audit to run. 45

46 Changing auditing policies You can also prevent audits for a data group from being submitted automatically but still allow them to be invoked manually. Automatic submission can be prevented for a specific audit of a data group by values specified for its priority audit and audit schedule policies. In addition to auditing, automatic recovery during replication may need to be prevented from running due to issues with applications or MIMIX features, For more information, see When to disable automatic recovery for replication and auditing on page 28. Disabling all auditing for an installation Note: This procedure changes a policy value at the installation level. The installation level value can be overridden by a data group level policy value. Therefore, if a data group has value other than *INST for this policy, that value remains in effect. Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. Verify that the value specified for Data group definition is *INST. 3. Press Enter to see all the policies and their current values. 4. Specify *DISABLED for the Audit level policy. 5. To accept the changes, press Enter. Disabling all auditing for a data group Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. For the Data group definition, specify the full three-part name. 3. Press Enter to see all the policies and their current values. 4. Specify *DISABLED for the Audit level policy. 5. To accept the changes, press Enter. Disabling automatically submitted audits You can control whether each audit for a data group can be submitted automatically by priority or by schedule. The Priority audit and Audit schedule policies act independently so that you can have both, one, or neither type of automatic auditing. Disabling a scheduled audit: Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. For the Data group definition, specify the full three-part name. 3. For Audit rule, specify the name of the MIMIX rule. 4. Press Enter to see the current values for the Audit schedule policy. 5. Do one of the following: 46

47 Changing auditing policies a. For installations running version or higher, specify *DISABLED for the State prompt. b. For installations running earlier software levels, specify *NONE for the Frequency prompt. 6. To accept the changes, press Enter. Disabling a prioritized audit: Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. For the Data group definition, specify the full three-part name. 3. For Audit rule, specify the name of the MIMIX rule. 4. Press Enter to see the current values for the Priority audit policy. 5. Do one of the following: a. For installations running version or higher, specify *DISABLED for the State prompt. b. For installations running earlier software levels, specify *NONE for the Start after prompt. 6. To accept the changes, press Enter. 47

48 Policies for switching with model switch framework Policies for switching with model switch framework In environments that do not use application groups, MIMIX Switch Assistant (which implements MIMIX Model Switch Framework) is usually used for switching. MIMIX Model Switch Framework cannot be used to switch application groups. Table 8 identifies the policies associated with switching using MIMIX Model Switch Framework and the shipped default values of those policies. For these policies, MIMIX Switch Assistant uses only the policy values specified for the installation. If MIMIX cannot determine whether a MIMIX Model Switch Framework is defined, the switch framework policy is *DISABLED. If the SETMMXPCY command specifies a data group name, the switch framework is required to be *INST. The switch thresholds are *DISABLED by default but can be changed. The policies in Table 8 have no effect on application group switching. Table 8. Policy Shipped values of policies used by MIMIX Switch Assistant. Shipped Values Installation Data Groups Data group definition *INST Name 1 Switch warning threshold 90 *DISABLED Switch action threshold 180 *DISABLED Default model switch framework MXMSFDFT *INST 1. A data group definition value of *INST indicates the policy is installation-wide. A name indicates the policies are in effect only for the specified data group. Specifying a default switch framework in policies MIMIX Switch Assistant requires that you have a configured MIMIX Model Switch Framework and that you specify it in the default model switch framework policy for the installation. You may also want to adjust policies for thresholds associated with MIMIX Switch Assistant. If you do not have a configured MIMIX Model Switch Framework, contact your Certified MIMIX Consultant. From a 5250 emulator, do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. Verify that the value specified for Data group definition is *INST. 3. Press Enter to see all the policies and their current values. 4. At the Default model switch framework prompt, specify the name of the switch framework to use for switching this installation. 5. To accept the changes, press Enter. 48

49 Policies for switching with model switch framework Setting polices for MIMIX Switch Assistant If the value of the installation-level policy is disabled, you must change the policy in order to use MIMIX Switch Assistant. Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. Verify that the value specified for Data group definition is *INST. 3. Press Enter to see all the policies and their current values. 4. Specify values for the following fields: a. For Switch warning threshold, the value 90 is recommended. b. For Switch action threshold, the value 180 is recommended. c. For Default model switch framework, specify the name of your MIMIX Model Switch Framework. 5. To accept the changes, press Enter. Setting policies when MIMIX Model Switch Framework is not used If you do not use MIMIX Model Switch Framework for switching, you disable the default model switch framework policy at the installation level. Do the following from the management system: 1. From the command line type SETMMXPCY and press F4 (Prompt). 2. Verify that the value specified for Data group definition is *INST. 3. Press Enter to see all the policies and their current values. 4. At the Default model switch framework prompt, specify *DISABLED. 5. To accept the change, press Enter. 49

50 Policy descriptions Policy descriptions There are minor differences in the names of policies between user interfaces for a 5250 emulator and Vision Solutions Portal. The names shown here are those used in the 5250 emulator. For a complete description of all policy values, see online help for the command. Data group definition - Select the scope of the policies to be set. When the value *INST is specified, the policies being set by the command apply to all systems and data groups in the installation, with the exception of any policy for which a data grouplevel override exists. When a three-part qualified name of a data group is specified, the policies being set by the command apply to only that data group and override the installation-level policy values. Audit rule - Select the MIMIX rule for which an audit schedule will be set for the specified data group definition. The Audit schedule policy determines when this rule will audit the data group. The audit rule must specify the value *NONE when changing any policy except the audit schedule. Automatic object recovery Determines whether to enable functions that automatically start recovery actions to correct detected common object errors that occur during replication from the system journal. Automatic database recovery Determines whether to enable functions that automatically start recovery actions to correct detected common file errors that occur during replication from the user journal. Automatic audit recovery Determines whether to enable audits to start automatic recovery actions to correct differences detected during their compare phase. Object recovery notify on success Determines whether automatic object recovery actions send an informational (*INFO) notification upon successful completion. This policy is only valid when the Automatic object recovery policy is enabled. Database recovery notify on success Determines whether automatic database recovery actions send an informational (*INFO) notification upon successful completion. This policy is only valid when the Automatic database recovery policy is enabled. Audit notify on success Determines whether activity initiated by audits, including recovery actions, should automatically send an informational (*INFO) notification upon successful completion. If an audit is run when the Automatic audit recovery policy is disabled, successful notifications are sent only for the compare phase of the audit. Notification severity Determines the severity level of the notifications sent when a rule ends in error. This policy determines the severity of the notification that is sent, not the severity of the error itself. The policy is in effect whether the rule is invoked manually or automatically. This policy is useful for setting up an order precedence for notifications at the data group level. For example, if you set this policy for data group CRITICAL to be 50

51 Policy descriptions *ERROR when the value for the installation-level policy is *WARNING, any error notifications sent from data group CRITICAL will have a higher severity than those from other data groups. Object only on target action Determines how the recovery action for specific audits should handle objects that are configured for replication but exist only on the target system. The following rules check for the only-on-target error: #OBJATR, #IFSATR, #DLOATR, #FILATR, and #FILATRMBR. When the Automatic audit recovery (AUDRCY) policy is enabled, these rules use the value from this policy to attempt recovery for this error. See Policies in environments with more than two nodes or bi-directional replication on page 27 for additional information. Journaling attribute difference action Determines the recovery action to take for scenarios in which audits have detected differences between the actual and configured values of journaling attributes for objects journaled to a user journal. This type of difference can occur for the Journal Images attribute and the Journal Omit Open/Close attribute. Differences found on either the source or target object are affected by this policy. MIMIX configured higher Determines the recovery for correcting a difference in which the MIMIX configuration specifies an attribute value that results in a higher number of journal transactions than the object's journaling attribute. MIMIX configured lower Determines the recovery action for correcting a difference in which the MIMIX configuration specifies an attribute value that results in a lower number of journal transactions than the object's journaling attribute. DB apply threshold action Determines what action to pass to the Compare File Data (CMPFILDTA) command or the Compare Record Count (CMPRCDCNT) command when it is invoked with *DFT specified for its DB apply threshold (DBAPYTHLD) parameter. The command s parameter determines what to do if the database apply session backlog exceeds the threshold warning value configured for the database apply process. This policy applies whenever these commands are used and the backlog exceeds the threshold. The shipped default for this policy causes the requested command to end and may cause the loss of repairs in progress or inaccurate counts for members. You can also set this policy to allow the request to continue despite the exceeded threshold. DB apply cache Determines whether to use database (DB) apply cache to improve performance for database apply processes. 1 When this policy is enabled, MIMIX uses buffering technology within database apply processes in data groups that specify *YES for journal on target (JRNTGT). This policy is not used by data groups which specify JRNTGT(*NO) or by data groups whose target journals use journal caching or journal standby functionality provided by the IBM feature for High Availability Journal Performance (IBM i option 42). 1. This policy is not available in MIMIX Availability Manager. 51

52 Policy descriptions Note: When DB apply cache is used, before and after journal images are sent to the local journal on the target system.this will increase the amount of storage needed for journal receivers on the target system if before images were not previously being sent to the journal. Access path maintenance Determines whether MIMIX can optimize access path maintenance during database apply processing as well as the maximum number of jobs allowed per data group when performing delayed maintenance. Enabling optimized access path maintenance improves performance for the database apply process. To make any change to this policy effective, end and restart the database apply processes for the affected data groups. This policy and the access path maintenance function it controls are available on systems running or higher and replace the parallel AP maintenance (PRLAPMNT) policy and its related function offered in earlier software levels. For more information about either method of optimizing access path maintenance, see the MIMIX Administrator Reference book. Optimize for DB apply Specify whether to enable optimized access path maintenance. When enabled, the database apply processes are allowed to temporarily change the value of the access path maintenance attribute for eligible replicated files on the target system. Eligible files include physical files, logical files, and join logical files with keyed access paths that are not unique and that specify *IMMED for their access path maintenance. Maximum number of jobs Specify the maximum number of access path maintenance jobs allowed for a data group when optimized access path maintenance is enabled. The actual number of jobs varies as needed between a minimum of one job and the specified value. The default value is 99. Maximum rule runtime Determines the maximum number of minutes an audit can run when the Automatic audit recovery policy is enabled. The compare phase of the audit is always allowed to complete regardless of this policy s value. The elapsed time of the audit is checked when the recovery phase starts and periodically during the recovery phase. When the time elapsed since the rule started exceeds the value specified, any recovery actions in progress will end. This policy has no effect on the #MBRRCDCNT audit because it has no recovery phase. The shipped default for this policy of 1440 minutes (24 hours) prevents running multiple instances of the same audit within the same day. Valid values are 60 minutes through minutes (1 week). Audit warning threshold Determines how many days can elapse after an audit was last performed before an indicator is set. When the number of days that have elapsed exceeds the threshold, the indicator is set to inform you that auditing needs your attention. The shipped default value of 7 days is at the limit of best practices for auditing. Note: It is recommended that you set this value to match the frequency with which you perform audits. It is possible for an audit to be prevented from running for several days due to environmental conditions or the Action for running audit policy. You may not notice that the audit did not run when expected until the 52

53 Policy descriptions Audit warning threshold is exceeded, potentially several days later. If you run all audits daily, specify 1 for the Audit warning threshold policy. If you do not run audits daily, set the value to what makes sense in your MIMIX environment. For example, if you run the #FILDTA audit once a week and run all other audits daily, the default value of 7 would cause all audits except #FILDTA to have exposure indicated. The value 1 would be appropriate for the daily audits but the #FILDTA audit would be identified as approaching out of compliance much of the time. Audit action threshold Determines how many days can elapse after an audit was last performed before an indicator is set. When the number of days that have elapsed exceeds the threshold, the indicator is set to inform you that action is required because the audit is out of compliance. The shipped default of 14 days is the suggested value for this threshold, which is 7 days beyond the limit of best practices for auditing. Note: It is recommended that you set this value to match the frequency with which you perform audits. It is possible for an audit to be prevented from running for several days due to environmental conditions or the Action for running audit policy. You may not notice that the audit did not run when expected until the Audit action threshold is exceeded, potentially several days later. If you run all audits daily, specify 1 for the Audit action threshold policy. If you do not run audits daily, set the value to what makes sense in your MIMIX environment. For example, if you run the #FILDTA audit once a week and run all other audits daily, the default value of 14 would cause all audits except #FILDTA to have exposure indicated. The value 2 would be appropriate for the daily audits but the #FILDTA audit would be identified as approaching out of compliance much of the time. Audit level Determines the level of comparison that an audit will perform when a MIMIX rule which supports multiple levels is invoked against a data group. The policy is in effect regardless of how the rule is invoked. The amount of checking performed increases with the level number. This policy makes it easy to change the level of audit performed without changing the audit scheduling or rules. No auditing is performed if this policy is set to *DISABLED. The audit level you choose for audits depends on your environment, and especially on the data compared by the #FILDTA, #DLOATR, and #IFSATR audits. When choosing a value, consider how much data there is to compare, how frequently it changes, how long the audit runs, how often you run the audit, and how often you need to be certain that data is synchronized between source and target systems. Note: Best practice is to use level 30 to perform the most extensive audit. If you use a lower level, consider its effect on how often you need to guarantee data integrity between source and target systems. Regardless of the level you use for daily operations, Vision Solutions strongly recommends that you perform audits at audit level 30 before the following events to ensure that 100 percent of the data is valid on the target system: Before performing a planned switch to the backup system. Before switching back to the production system. 53

54 Policy descriptions For additional information, see Guidelines and considerations for auditing on page 126 and Changing auditing policies on page 41. Run rule on system Determines the system on which to run audits. This policy is used when audits are invoked with *YES specified for the value of the Use run rule on system policy (USERULESYS) parameter on the Run Rule (RUNRULE) or Run Rule Group (RUNRULEGRP) command. When *YES is specified in these commands, this policy determines the system on which to run audits. While this policy is intended for audits, any rule that meets the same criteria will use this policy. The policy s shipped default value, *MGT, runs audits from the management system. In multi-management environments where both systems defined to a data group are management systems, the value *MGT will run audits only on the target system. You can also set the policy to run audits from the network system, the source or target system, or from a list of system definitions. When both systems of a data group are in the specified list, the target system is used. When choosing the value for the Run rule on system policy, also consider your switching needs. Action for running audits Determines the type of audit actions permitted when certain conditions exist in the data group. If a condition exists at the time of an audit request, audit activity is restricted to the specified action. If multiple conditions exist and the values specified are different, only the most restrictive of the specified actions is allowed. If none of the conditions are present, the audit requests are performed according to other policy values in effect. Inactive data group Specify the type of auditing actions allowed when any replication process required by the data group is inactive. For example, a data group of TYPE(*ALL) is considered inactive if any of its database or object replication processes is in a state other than active. This element has no effect on the #FILDTA and #MBRRCDCNT audits because these audits can run only when the data group is active. Repl. process in threshold Specify the type of auditing actions allowed when a threshold warning condition exists for any process used in replicating the class of objects checked by an audit 1. If a checked process has reached its configured warning value, auditing is restricted to the specified actions. Most audits check for threshold conditions in all database and object replication processes, including the RJ link. #FILDTA audits only check for threshold warning conditions in the RJ link and database replication processes. #DLOATR audits only check for threshold warning conditions in object replication processes. Audit history retention Determines criteria for retaining historical information about audit results and the objects that were audited. History information for an audit includes timestamps indicating when the audit was performed, the list of objects that were audited, and result statistics. Each audit, a unique combination of audit rule and 1. This behavior applies to instances running service pack or higher. Instances running earlier services packs check for thresholds on only the database apply and object apply processes. 54

55 Policy descriptions data group, is evaluated separately and its history information is retained until the specified minimum days and minimum runs are both met. When an audit exceeds these criteria, system manager cleanup jobs will remove the historical information for that audit from all systems and will remove the audited object details from the system on which the audit request originated. The values specified at the time the cleanup jobs run are used for evaluation. Minimum days Specify the minimum number of days to retain audit history for each completed audit. Valid values range from 0 through 365 days.the shipped default is 7 days. Minimum runs per audit Specify the minimum number of completed audits for which history is to retained. Valid values range from 1 through 365 runs. The shipped default is 1 completed audit. Object details Specify whether to retain the list of audited objects and their audit status for each completed audit of library-based objects. The specified value in effect at the time an audit runs determines whether object details for that run are retained. The specified value has no effect on cleanup of details for previously completed audit runs. Cleanup of retained details occurs at the time of audit history cleanup. The shipped default is *YES. DLO and IFS details Specify whether to retain the list of audited objects and their audit status for each completed audit of DLO and IFS objects. The specified value in effect at the time an audit runs determines whether object details for that run are retained. The specified value has no effect on cleanup of details for previously completed audit runs. Cleanup of retained details occurs at the time of audit history cleanup. The shipped default is *YES. Note: When large quantities of objects are eligible for replication, specifying *YES to retain either Object details or DLO and IFS details may use a significant amount of disk storage. Consider the combined effect of the quantity of replicated objects for all data groups, the number of days to retain history, the number of audits to retain, and the frequency in which audits are performed. Synchronize threshold size Determines the threshold, in megabytes (MB), to use for preventing the synchronization of large objects during recovery actions. When any of the Automatic system journal recovery, Automatic user journal recovery, or Automatic audit recovery policies are enabled, all initiated recovery actions use this policy value for the corresponding synchronize command's Maximum sending size (MB) parameter. This policy is useful for preventing performance issues when synchronizing large objects. Number of third delay retry attempts Determines the number of times to retry a process during the third delay/retry interval. This policy is used when the Automatic system journal recovery policy is enabled. Object replication processes use this policy value when attempting recovery of an in-use condition that persists after the data group s configured values for the first and second delay/retry intervals are exhausted. The shipped default is 100 attempts. 55

56 Policy descriptions This policy and its related policy, Third delay retry interval, can be disabled so that object replication does not attempt the third delay/retry interval but still allow recoveries for other errors. Third delay retry interval Determines the delay time (in minutes) before retrying a process in the third delay/retry interval. This policy is used when the Automatic system journal recovery policy is enabled. Object replication processes use this policy value when attempting recovery of an in-use condition that persists after the data group s configured values for the first and second delay/retry intervals are exhausted. The shipped default is 15 minutes. Switch warning threshold Determines how many days can elapse after the last switch was performed before an indicator is set for the installation. When the number of days that have elapsed exceeds this threshold, the indicator is set to inform you that switching may need your attention. The shipped default is 90 days, which is considered at the limit of best practices for switching. The indicator is associated with the Last switch field. The Last switch field identifies when the last completed switch was performed using the default model switch framework (DFTMSF) policy. Switch action threshold Determines how many days can elapse after the last switch was performed before an indicator is set for the installation. When the number of days that have elapsed exceeds this threshold, the indicator is set to inform you that action is required. The shipped default of 180 days is the suggested value for this threshold, which beyond the limit of best practices for switching. The indicator is associated with the Last switch field. The Last switch field identifies when the last completed switch was performed using the default model switch framework (DFTMSF) policy. Default model switch framework Determines the default MIMIX Model Switch Framework to use for switching. This value is used by configurations which switch via model switch framework. The shipped default value is MXMSFDFT, which is the default model switch framework name for the installation. If the default name is not being used, this value should be changed to the name of the MIMIX Model Switch Framework used to switch the installation. Independent ASP library ratio Determines the number for n in a ratio (n:1) of independent ASP libraries (n) on the production system to SYSBAS libraries on the backup system 1. For each switchable independent ASP defined to MIMIX by a device resource group, a monitor with the same name as the resource group checks this ratio. When the number of independent ASP libraries falls to a level that is below the specified ratio, the monitor sends a notification to inform you that action may be required. This signals that your recovery time objective could be in jeopardy because of a prolonged independent ASP switch time. CMPRCDCNT commit threshold Determines the threshold at which a request to compare record counts (CMPRCDCNT command or #MBRRCDCNT audit) will not perform the comparison due to commit cycle activity on the source system. The value specified is the maximum number of uncommitted record operations that can exist for 1. The library ratio monitor and the policy it uses require a license key for MIMIX Global. 56

57 Policy descriptions files waiting to be applied at the time the compare request is invoked. Each database apply session is evaluated against the threshold independently. As a result, it is possible that record counts will be compared for files in one apply session but will not be compared for files in another apply session. For additional information see the MIMIX Administrator Reference book. Procedure history retention Specifies criteria for retaining historical information about procedure runs that completed or completed with errors. History information for a procedure includes timestamps indicating when the procedure was run and detailed information about each step within the procedure. Each procedure run, a unique combination of procedure name and application group, is evaluated separately and its history information is retained until the specified minimum days and minimum runs are both met. When a procedure run exceeds these criteria, system manager cleanup jobs will remove the historical information for that procedure run from all systems. The values specified at the time the cleanup jobs run are used for evaluation. Minimum days Specifies the minimum number of days to retain procedure run history. The default value is 7. Minimum runs per procedure Specifies the minimum number of completed procedure runs for which history is to retained. This value applies to procedures of all other types except *SWTPLAN and *SWTUNPLAN. The default value is 1. Min. runs per switch procedure Specifies the minimum number of completed switch procedure runs for which history is to retained. This value applies to procedures of type *SWTPLAN and *SWTUNPLAN that are used to switch an application group. The default value is 12. Audit schedule Determines the scheduling information that MIMIX uses to automatically submit audit requests for the specified data group and rule that will check all objects selected by data group configuration entries. Only configuration entries associated with the specified type of rule are used. To allow an audit to be automatically submitted, *ENABLED must be specified for State 1. Changes to this policy are effective immediately. If an audit is in progress at the time of the change, the change will be reflected in the next scheduled run of the audit. Scheduled dates are entered and displayed in job date format. When the job date format is Julian, the equivalent month and day are used to determine when to schedule audit requests. State 1 Specify whether scheduled auditing is enabled or disabled for this data group and audit rule. 1. The State element is available in installations running MIMIX version or higher. In installations running earlier software levels, scheduled auditing requires specifying a value other than *NONE for Frequency and specifying values for Scheduled time and either Scheduled date or Scheduled day. Frequency is qualified by the values specified in the other elements 57

58 Policy descriptions Frequency Specify how often the audit request is submitted. The values specified for other elements further qualify the specified frequency. Scheduled date Select a value or specify a date, in job date format, on which the audit request is submitted. Scheduled day Select the day or days of the week on which the audit request is submitted. If today is the day of the week that is specified and the scheduled time has not passed, the audit request is submitted today. Otherwise, the job is submitted on the next occurrence of the specified day. For example, if it is 11:00 a.m. on a Friday when you set the audit schedule to specify Friday for Scheduled day and 12:00:00 for Scheduled time, the audit request is submitted today. If you are setting the policy at 4:00 p.m. on a Friday or at 11:00 a.m. on a Monday, the audit request is submitted the following Friday. Scheduled time Select a value or specify a time in 24-hour format at which the audit request is submitted on the scheduled date or day. Although the time can be specified to the second, the activity involved in submitting a job and the load on the system may affect the exact time at which the job is submitted. Time can be specified with or without a time separator. Without a time separator, specify a string of 4 or 6 digits (hhmm or hhmmss) where hh = hours, mm = minutes, and ss = seconds. Valid values for hh range from 00 to 23. Valid values for mm and ss range from 00 to 59. With a time separator, specify a string of 5 or 8 digits where the time separator specified for your job is used to separate the hours, minutes, and seconds. If this command is entered from the command line, the string must be enclosed in apostrophes. If a time separator other than the separator specified for your job is used, this command will fail. Relative day of month Select a value or specify one or more numbers with which to qualify what day a monthly audit request is submitted, relative to its occurrence in the month. A relative day is only valid when the schedule Frequency is Monthly and Scheduled day is a value other than None. For example, if Frequency is Monthly, Scheduled day is Tuesday and Thursday, and Relative day of month is 1, the audit request is submitted on the first Tuesday and first Thursday of every month. If both 1 and 4 are specified for relative day, the audit request is submitted on the first Tuesday, first Thursday, fourth Tuesday, and fourth Thursday of the month. Priority audit Determines when priority-based audit requests for the specified data group and rule are allowed to automatically start and how often replicated objects are eligible for auditing based on their priority classification. The #DGFE rule does not support priority auditing. To allow priority-based auditing to be performed, *ENABLED must be specified for State. 1 Changes to this policy are effective immediately. If an audit is in progress at 58

59 Policy descriptions the time of the change, the change will be reflected in the next priority-based run of the audit. State 1 Specify whether priority auditing is enabled or disabled for this data group and audit rule. Start after Select a value or specify a time after which priority-based audits are allowed to start. This is the beginning of a range of time during which priority-based audits can start each day. The value *ANY allows priority-based audits to run repeatedly throughout the day. Note: Times specified for Start after and Start until elements is in 24-hour format and can be specified with or without a time separator. Without a time separator, specify a string of 4 or 6 digits (hhmm or hhmmss) where hh = hours, mm = minutes, and ss = seconds. Valid values for hh range from 00 to 23. Valid values for mm and ss range from 00 to 59. With a time separator, specify a string of 5 or 8 digits where the time separator specified for your job is used to separate the hours, minutes, and seconds. If this command is entered from the command line, the string must be enclosed in apostrophes. If a time separator other than the separator specified for your job is used, this command will fail. Start until Specify the end of the time range during which priority-based audits are allowed to start. Priority-based audits can start until this time. This value is ignored when Start after is *ANY. New objects selected Select the frequency at which new objects are considered for auditing. A new object is one that has not been audited since it was created. Changed objects selected Select the frequency at which changed objects are considered for auditing. A changed object is one that has been modified since the last time it was audited. Unchanged objects selected Select the frequency at which unchanged objects are considered for auditing. An unchanged object is one that has not been modified since the last time it was audited. Audited with no diff. Select the frequency at which objects with no differences are considered for auditing. An object with no differences is one that has not been modified since the last time it was audited and has been successfully audited on at least three consecutive audit runs. 1. The State element is available in installations running MIMIX or higher. In installations running earlier software levels, priority auditing requires a value other than *NONE for Start after. 59

60 Checking application group status CHAPTER 3 Checking status in environments with application groups Monitoring status of environments that use application groups begins at the level of the application group and may include investigation into additional displays for more detailed information. The following displays are typically used: Work with Procedure Status (WRKPROCSTS command) Work with Application Groups (WRKAG command) Work with Node Entries (WRKNODE command) Work with Data Rsc. Grp. Ent. (WRKDTARGE command) Work with Data Groups (WRKDG command) Note: This chapter does not include status for application groups that are configured for an IBM i clustering environment. If you are using clustering or have MIMIX Global configured, see the MIMIX Operations with IBM i Clustering book for status information within a clustering environment. Checking application group status The status view of the Work with Application Groups display provides a summary of all status associated with an environment configured with application groups. 1. Do one of the following to access the Work with Application Groups display: Select option 1 (Work with application groups) from the MIMIX Basic Main Menu. Select option 5 (Work with application groups) from the MIMIX Intermediate Main Menu. Enter the command: WRKAG 2. If necessary, use F10 to access the status view. Figure 3. Status view of Work with Application Groups display 60

61 Checking application group status Work with Application Groups System: SYSA Monitors..... : *ACTIVE Notifications.. : *NONE Type options, press Enter. 1=Create 2=Change 4=Delete 5=Display 6=Print 9=Start 10=End 12=Node entries 13=Data resource groups 15=Switch App App App Node Data Rsc Data Node Repl. Proc. Opt Group Status Status Grp Status Status Status Status SAMPLEAG *ACTIVE *ACTIVE *COMP Bottom Parameters or command ===> F3=Exit F4=Prompt F5=Refresh F6=Create F9=Retrieve F10=View config F12=Cancel F13=Repeat F18=Subset F23=More options F24=More keys All status columns except the App Status column are summations of multiple processes. Investigation into lower-level displays may be necessary to determine the cause of a problem. Ideal status conditions exist when the fields and columns have the following values: The Monitors field is *ACTIVE. The Notifications field is *NONE. The Proc. Status column is *COMP. For a non-cluster application group, the App Node Status and Repl. Status fields are *ACTIVE. The App Status, Data Rsc Grp Status, and Data Node Status columns will always be blank. For any other status values, see the following: Resolving problems reported in the Monitors field on page 61 Resolving problems reported in the Notifications field on page 63 Resolving problems reported in Status columns on page 64 Resolving problems reported in the Monitors field The Monitors field located in the upper right corner of the Work with Application Groups display summarizes the status of the MIMIX monitors on the local system. Each node or system in the product configuration has MIMIX monitors which run on that system to check for specific potential problems. A status of *ACTIVE indicates that all enabled monitors on the local system are active. 61

62 Checking application group status Table 9 shows possible status values for the Monitors field that require user action. For a complete list of possible values, press F1 (Help). Table 9. Monitor field status values that may require user action Monitor Status *ATTN *INACTIVE Description Either one or more monitors on the local system failed or there are both active and inactive monitors on the local system. All enabled monitors on the local system are inactive. Do the following: 1. Press F14 (Monitors) to display the list of monitors on the local system on the Work with Monitors display. 2. Check the Status column for status values of FAILED, FAILED/ACT, and INACTIVE. 3. If the monitor is needed on the local system as indicated in Table 10, use option 9 (Start) to start the monitor. Table 10. Monitor Possible monitors and the nodes on which they should be active When and Where Needed journal-name - remote journal link monitor Checks the journal message queue for indications of problems with the remote journal link. A monitor exists for both the local and remote system of the RJ link. MMIASPMON - independent ASP threshold monitor Checks the QSYSOPR message queue for indications that the independent ASP threshold has been exceeded. This monitor improves the ability to detect overflow conditions that put your high availability solution at risk due to insufficient storage. MMNFYNEWE - monitor for new object notification entries Monitors the source system for the newly created libraries, folders, or directories that are not already included or excluded for replication by a data group configuration. Primary node and the current Backup node of application groups which perform logical replication. On all nodes which control an independent ASP. Primary node when the application group is configured for logical replication. 62

63 Checking application group status Table 10. Monitor Possible monitors and the nodes on which they should be active When and Where Needed short-data-group-name_papm - Parallel access path maintenance group monitor. When this monitor exists, there are always associated monitors of one of the following types: short-data-group-namepapmnnn - Parallel access path maint monitor nnn short-data-group-namejobname - Parallel access path maint monitor job-name Target node of data group replication processes when Parallel access path maintenance policy has been enabled. Note: These monitors and the policy which enables them are only available on systems running software levels earlier than The replacement for this function on systems running or higher does not use monitors. For more information about optimizing access path maintenance, see the MIMIX Administrator Reference book. Resolving problems reported in the Notifications field The Notifications field located in the upper right corner of the Work with Application Groups display summarizes the status of notifications that exist for the MIMIX installation. Notifications are sent by MIMIX processes, such as monitors or audits, to inform you of potential problems. A value of *NONE indicates that no new notifications exist. Table 11 shows possible status values for the Notifications field that require user action. Table 11. Notification field status values that may require user action Notification Status *ERROR *WARNING *INFO Description Action is required. At least one new notification exists with a severity of *ERROR. At least one new notification exists with a severity of *WARNING, which indicates that the operation may be successful but an error exists. There are no new notifications with a severity of *ERROR. At least one new notification exists with a severity of *INFO. There are no new notifications with severity of *ERROR or *WARNING. Do the following: 1. Press F15 (Notifications) to display the list of notifications for the installation on the Work with Notifications display. 2. Use option 5 (Display) to view any notifications with a status of *NEW. 3. Take any further action indicated to resolve the problem. 4. When the problem is resolved, use either option 46 (Acknowledge) or option 4 (Remove) to address the notification itself. Notifications can only be removed from the system on which they originated. 63

64 Checking application group status Resolving problems reported in Status columns Except for the App Status column, all other columns on the Work with Application Groups display represent summations of status for multiple nodes or multiple data resource groups associated with the application groups. Investigation into lower-level displays may be necessary to determine the cause of the problem. Troubleshooting Tip: When investigating problems, begin with the Proc. Status column. A problem with procedure status can affect values in other columns. When any procedure status problems are resolved, refresh the display. Then check the other columns beginning the left-most column that is reporting a problem. Resolve the most severe problem in that column first, then refresh the display. Investigate problems in the remaining columns from left to right. To address the most common problems with status for application groups, do the following: 1. Resolve any problems reported in the Proc. Status column using Table Resolve any *ATTN status problems first, using Table Then address less severe problems, using Table 14. For a complete list of status values for each column, press F1 (Help). Resolving a procedure status problem The Proc. Status column represents a summary of the most recent run of all procedures defined for the application group. Table 12. Procedure Status values that require attention Column Value *ACTIVE *ATTN Description and Action One or more of the last started runs of the procedures to run are still active or queued. Wait for the procedure to complete. Do not attempt to correct other status problems reported on the display until the procedure completes. Use option 21 (Procedure status) to view the status of the last started runs of procedures for the application group. One or more of the last started runs of the procedures for the application group have a status that requires attention. Use option 21 (Procedure status) to view the status of the last started runs of the procedures for the application group. The resulting procedures shown on the Work with Procedure Status display. which have status values of *ATTN, *CANCELED, *FAILED, *MSGW, or *PENDCNL require user action. Also, it may be necessary to check status of the steps within the procedure to resolve a step problem before the procedure can continue. Do not attempt to correct other status problems reported on the Work with Application Groups display until the procedure problems have been resolved. For detailed information, see Working with status of procedures and steps on page 77. Note: The status *COMP indicates that the most recently started run of each procedure for the application group has completed as directed. This includes procedures that completed with errors and cancelled or failed procedures 64

65 Checking application group status whose status have been acknowledged by user action. For any individual procedure that completed with errors, user action is recommended to investigate the cause of the error and assess its implications. Resolving an *ATTN status for an application group The value *ATTN can appear in each column of the Work with Application Groups display to indicate that user action is required to correct a problem. Important! Check the status of the Proc. Status column and address any problem indicated by *ATTN or *ACTIVE status before attempting to resolve any problem reported in other columns. Use Resolving a procedure status problem on page 64. If there are no procedure problems, each of the other columns with an *ATTN status must be addressed individually, starting from the left-most column. Table 13. Resolving *ATTN status for columns (except Proc. Status) on the Work with Application Groups display *ATTN Status in Column App Node Status Replication Status Description and Actions for *ATTN Status The App Node Status column is a summary of the status of the nodes associated with the application group. The status includes the MIMIX system manager, journal manager, target journal inspection, and collector services jobs for the nodes in the application group. *ATTN indicates that the node status and the MIMIX manager status values do not match. Investigate the status of the associated nodes and MIMIX managers using option 12 (Node entries). For additional information see Status for Work with Node Entries on page 66. The Replication Status column is a summary status of data replication activity for the data resource groups associated with an application group. *ATTN indicates that data replication for at least one data group for the data resource groups has a status that does not match the status of the appropriate data resource group, has a failed state, an error condition, is active with an incorrect source system, has audit errors, or has pending recoveries. To determine the cause, use option 13 (Data resource groups) to identify the data resource group where the problem exists. For more information, see Status for Work with Data Resource Group Entries on page

66 Status for Work with Node Entries Resolving other common status values for an application group Table 14 lists other common problems with application group status and identifies how to begin to their resolution. Table 14. Column Value *ATTN Other problem statuses which may appear in multiple columns on the Work with Application Groups display Description and Action Each column has a unique recovery. See Resolving an *ATTN status for an application group on page 65. *INACTIVE *UNKNOWN The current status of the resource group or node is inactive. This status is possible in the Repl. Status column. If all columns with a status value are *INACTIVE, the application group may have been ended intentionally. Use option 9 (Start) to start the application group. If this value appears only in the App Node Status column, the application resource group nodes are all inactive and all MIMIX manager jobs are also inactive. Use option 12 (Node entries) to investigate further. For more information see Status for Work with Node Entries on page 66. If this value appears only in the Repl. Status column, logical replication is not active. Use option 13 (Data resource groups) to investigate. For more information see Status for Work with Data Resource Group Entries on page 68. The current status is unknown. The local node is a network node in a noncluster application group and does not participate in the recovery domain. Its status cannot be determined. When this status appears in all columns, do one of the following: Enter the command WRKSYS. On the Work with Systems display, check the status of Cluster Services for the local system definition. If necessary, use option 9 (Start) to start cluster services. Sign on to a node that is active and use the WRKAG command to check the application group status. If the status is still *UNKNOWN, use option 12 (Node entries) to check the status of Cluster Services on the node. Status for Work with Node Entries The Work with Node Entries displays a list of the nodes associated with an application group or a data resource group. The Resource group and Type fields at the top of the display indicate what the nodes are associated with. Figure 4. Status view of Work with Node Entries display for an application group which does 66

67 Status for Work with Node Entries not participate in a cluster Work with Node Entries Application group..... : SAMPLEAG System: SYSA Type options, press Enter. 1=Add 2=Change 4=Remove 5=Display 6=Print 9=Start 10=End Current Manager Opt Node Role Sequence Data Provider Status SYSB *PRIMARY *PRIMARY *ACTIVE SYSA *BACKUP 1 *PRIMARY *ACTIVE Bottom Parameters or command ===> F3=Exit F4=Prompt F5=Refresh F6=Add F7=Systems F9=Retrieve F10=View config F11=Sort by node F12=Cancel F18=Subset F24=More keys For each node listed, check the Manager Status column for status values that require attention. For a complete list of status values for each field and column, press F1 (Help). Manager Status - This column indicates the status of all of the MIMIX system manager, journal manager, target journal inspection, and collector services jobs for the specified node. Table 15. Status Value *ATTN Manager Status values that require user action. Description and Action At least one of the system manager, journal manager, target journal inspection, or collector services jobs for the node has failed. When all the nodes listed do not have the same value, use F7 (Systems) to access the Work with Systems display. Check the status of the system and journal managers, target journal inspection, and collector services. Use option 9 (Start) to start the managers and services that are not active on the node. *INACTIVE All system manager, journal manager, target journal inspection, and collector services jobs for the specified node are inactive. This may be intentional when MIMIX is ended to perform certain activities. Use F7 (Systems) to access the Work with Systems display. 67

68 Status for Work with Data Resource Group Entries Status for Work with Data Resource Group Entries The Work with Data Resource Group Entries display lists the data resource groups associated with an application group. Each entry identifies a data resource group and the summary of the replication status from its associated data groups. Figure 5. The Work with Data Resource Group Entries display for an application group that does not participate in a cluster Application group..... : Work with Data Rsc. Grp. Ent. SAMPLEAG System: SYSA Type options, press Enter. 1=Add 2=Change 4=Remove 5=Display 6=Print 8=Data groups 9=Start 10=End 12=Node entries 14=Build environment 15=Switch Resource Resource Group Node Replication Opt Group Type Status Status Status AGRSGRP *DTA *ACTIVE Bottom Parameters or command ===> F3=Exit F4=Prompt F5=Refresh F6=Add F9=Retrieve F12=Cancel F13=Repeat F18=Subset F19=Load F21=Print list Resource Group Status - This column identifies the status of the data resource group. In environments that do not include IBM i clustering, this column is always blank. Node Status - This column identifies the status of the nodes for the data resource group. In environments that do not include IBM i clustering, this column is always blank Replication Status - The value in this column is a summary status of data replication activity for the data resource group. The status includes the status of all data group processes, replication direction, replicated object and file entries, audits, and recoveries. 68

69 Status for Work with Data Resource Group Entries Figure 16 identifies status values for replication that require user action. Table 16. Replication Status values that require user action. Status Value *ATTN Description and Action One or more of the following problems exist for data groups within the data resource group. The source system of an active data group is not the primary node of of its application group. A data group has a failed state, an error condition, audit errors, or pending recoveries. To prevent damage to data in your environment, it is important that you begin by determining which system should be the source for the data groups. Do the following: 1. From this display, use option 8 (Data groups) to check which system is the current source for the data group. 2. Determine which node has the role of current primary for the application group. From the Work with Application Groups display, use option 12 (Node entries), then check the current node role. If the current primary node is correct and a data group with an incorrect source system is active, end the data group and contact CustomerCare. If the data groups in question have the correct source system but the primary node for the application group is not correct, you need to change the recovery domain for the application group to make the correct node become primary. Use Changing the sequence of backup nodes on page 71. Once you have ensured that the data groups have the correct source system, resolve any error conditions reported on the Work with Data Groups display. Note: Not all data groups should necessarily be active. Only the data groups currently being used for data replication should be active. You will need to look at the current node roles and data providers for the node entries to determine which data groups should be active. *INACTIVE All replication in the data resource group is inactive. This may be normal if replication was ended to perform certain activities. Use option 8 (Data groups) to access the Work with Data Groups display. 69

70 Verifying the sequence of the recovery domain Verifying the sequence of the recovery domain Ensuring that sequence of the current backup nodes is set properly is critical to a successful and predictable switch process. The current sequence of backup nodes should match your recovery guidelines. Do the following to confirm the sequence of the current backup nodes before performing a switch and before removing or restoring a backup node from the cluster. 1. From the MIMIX Intermediate Main Menu, type 5 (Work with application groups) and press Enter. 2. From the Work with Application Groups display, type 12 (Node entries) next to the application group you want and press Enter. 3. The Work with Node Entries display appears, showing current information for the nodes. Confirm that the current backup nodes have the sequence order that you expect. Note: It is important that you are viewing current information on the status view of the display. Figure 6 shows an example of how the resulting Work with Node Entries display appears with current status information. If you see configured information instead, press F10 (View status). 4. If you need to change the sequence of current backup nodes, use Changing the sequence of backup nodes on page 71. Figure 6. Example of displaying the current sequence information for backup nodes Application group..... : Work with Node Entries APP1 System: NODED Type options, press Enter. 1=Add 2=Change 4=Remove 5=Display 6=Print 9=Start 10=End Current Manager Opt Node Role Sequence Data Provider Status NODEA *PRIMARY *NONE *ACTIVE NODEB *BACKUP 1 NODEA *ACTIVE NODEC *BACKUP 2 NODEA *ACTIVE NODED *BACKUP 3 NODEA *ACTIVE Bottom Parameters or command ===> F3=Exit F4=Prompt F5=Refresh F6=Add F7=Systems F9=Retrieve F10=View config F11=Sort by node F12=Cancel F18=Subset F24=More keys 70

71 Changing the sequence of backup nodes Changing the sequence of backup nodes Use this procedure if you need to change the sequence of the current backup nodes. This procedure may change the configured sequence for multiple nodes so that you can achieve the desired sequence for backup nodes. The changes are not effective until Step 5 is performed. Do the following from an active application group: 1. From the Work with Application Groups display, type 12 (Node entries) next to the application group you want and press Enter. 2. The Work with Node Entries display appears. Using F10 to toggle between configuration view and status views, confirm that the node with the configured role of *PRIMARY is the same node that is shown as the current *PRIMARY role. If the same node is identified as *PRIMARY for the current role and the configured role, skip to Step 4. If the configured *PRIMARY node does not match the current *PRIMARY node, perform Step 3 to correct this situation.before making any changes to the configured sequence of backup nodes. Figure 7 is an example of how configuration information appears on the Work with Node Entries display. 3. Perform this step only if you need to correct the configured primary node to match the current primary node. This step will demote the configured primary node to a backup, then promote the correct node to become the configured primary node. Do the following: a. From the configuration view of the Work with Node Entries display, type 2 (Change) next to the configured primary node and press Enter. b. On the Change Node Entry (CHGNODE) display, specify *BACKUP for Role and press Enter. Then specify *FIRST for List position and press Enter. c. On the Work with Node Entries display, press F5 (Refresh) to view changes. All nodes in the configured view should have *BACKUP roles. d. If necessary toggle to the status view to confirm which node is the current primary node. Type 2 (Change) next to the current primary node and press Enter. e. On the Change Node Entry (CHGNODE) display, specify *PRIMARY for Role and press Enter. Then press Enter two more times. f. On the Work with Node Entries display, press F5 (Refresh) to view changes. You should see the correct node as the configured primary node. Note: The numbering for the backup sequence may not update; however, the relative order for the configured backup sequence remains unchanged. Gaps in configured sequence numbers are ignored when switching to a backup. As long as the relative order is correct, it is not necessary to change the configured sequence of backup nodes just to remove gaps in numbering. 71

72 Changing the sequence of backup nodes g. If the configured backup sequence is what you expect, skip to Step 5 to make the change effective. 4. To change the sequence of backup nodes, do the following: a. From the configured view of the Work with Node Entries display, type 2 (Change) next to the backup node whose sequence you want to change. b. On the Change Node Entry (CHGNODE) display, specify *BACKUP for Role and press Enter. Then specify either *FIRST or a number for List position and press Enter. Note: If you specify a number, it cannot already be used in the configured sequence list. c. On the Work with Node Entries display, press F5 (Refresh) to view changes. d. Repeat Step 4 until the correct sequence is shown on the configuration view. Note: Gaps in configured sequence numbers are ignored when switching to a backup. For example, in a configuration with two backup nodes, there is no operational difference between a backup sequence of 1, 2 and a backup sequence of 2, 5 as long as the same nodes are specified in the same relative order. 5. To make the changes to the backup order effective, do the following: a. Press F12 (Cancel) to return to the Work with Application Groups display. b. Type 9 (Start) next to the application group you want and press F4 (Prompt). c. On the Start Application Group (STRAG) display, specify *CONFIG for Current node roles and press Enter. d. The Procedure prompt appears. If needed, specify a different value and then press Enter. 6. Confirm that the node entries have changed. Type 12 (Node entries) next to the application group and press Enter. If necessary, use F10 to access the status view. The current backup nodes should be in the new order. 72

73 Changing the sequence of backup nodes Figure 7. Example of displaying the configured sequence information for backup nodes: Application group..... : Work with Node Entries APP1 System: NODED Type options, press Enter. 1=Add 2=Change 4=Remove 5=Display 6=Print 9=Start 10=End Configured Opt Node Role Sequence Data Provider NODEA *PRIMARY *PRIMARY NODEB *BACKUP 1 *PRIMARY NODEC *BACKUP 4 *PRIMARY NODED *BACKUP 5 *PRIMARY Bottom Parameters or command ===> F3=Exit F4=Prompt F5=Refresh F6=Add F7=Systems F9=Retrieve F10=View status F11=Sort by node F12=Cancel F18=Subset F24=More keys Examples of changing the backup sequence The following examples illustrate problems with the current backup sequence and how to correct them. Example 1 - Changing the backup sequence when primary node is ok Table 17 shows a four-node environment where the current backup sequence does not reflect the desired behavior in the event of a switch. Also, the relative order of the 73

74 Changing the sequence of backup nodes configured backup sequence does not match the relative order of either the current sequence or the desired sequence. Table 17. Example 1, showing discrepancies in backup sequences Desired Order Initial Values, Example 1 Work with Node Entries Status View Current Opt Node Role Sequence Data Provider NODEA *PRIMARY *PRIMARY NODEB *BACKUP 1 *PRIMARY NODED *BACKUP 2 *PRIMARY NODEC *BACKUP 3 *PRIMARY Configured View Configured Opt Node Role Sequence Data Provider NODEA *PRIMARY *PRIMARY NODEC *BACKUP 1 *PRIMARY NODEB *BACKUP 2 *PRIMARY NODED *BACKUP 3 *PRIMARY Each row in Table 18 shows a change to be made to the nodes on the configured view of the Work with Node Entries display. The rows are in the order that the changes need to occur to correct this example configuration to the desired order. Table 18. Order in which to change nodes to achieve the desired configuration for example 1 Node to Change Change To Effect on Configured Order, Example 1 Notes NODEB Role = *BACKUP Position = *FIRST Configured View Configured Opt Node Role Sequence Data Provider NODEA *PRIMARY *PRIMARY NODEB *BACKUP 1 *PRIMARY NODEC *BACKUP 2 *PRIMARY NODED *BACKUP 3 *PRIMARY Intermediate step 74

75 Changing the sequence of backup nodes Table 18. Order in which to change nodes to achieve the desired configuration for example 1 Node to Change Change To Effect on Configured Order, Example 1 Notes NODED Role = *BACKUP Position = *FIRST Configured View Configured Opt Node Role Sequence Data Provider NODEA *PRIMARY *PRIMARY NODED *BACKUP 1 *PRIMARY NODEB *BACKUP 2 *PRIMARY NODEC *BACKUP 3 *PRIMARY Desired configuration but it is not effective until STRAG ROLE (*CONFIG) is performed. Example 2 - Correcting the configured primary node and changing the backup sequence Table 19 shows a four-node environment where the current backup sequence does not reflect the desired behavior in the event of a switch. Also, the current and configured primary node do not match.the configured primary node must be corrected first, before attempting to correct any backup node sequence problems. Table 19. Example 2, showing discrepancies in primary node and backup sequences Desired Order Initial Values, Example 2 Work with Node Entries Status View Current Opt Node Role Sequence Data Provider NODEA *PRIMARY *PRIMARY NODEB *BACKUP 1 *PRIMARY NODED *BACKUP 2 *PRIMARY NODEC *BACKUP 3 *PRIMARY Configured View Configured Opt Node Role Sequence Data Provider NODEB *PRIMARY *PRIMARY NODEC *BACKUP 1 *PRIMARY NODEA *BACKUP 2 *PRIMARY NODED *BACKUP 3 *PRIMARY 75

76 Changing the sequence of backup nodes Each row in Table 20 shows a change to be made to the nodes on the configured view of the Work with Node Entries display. The rows are in the order that the changes need to occur to correct this example configuration to the desired order. Table 20. Order in which to change nodes to achieve the desired configuration for example 2. Node to Change Change To Effect on Configured Order, Example 2 Notes NODEB Role = *BACKUP Position = *FIRST Configured View Configured Opt Node Role Sequence Data Provider NODEB *BACKUP 1 *PRIMARY NODEC *BACKUP 2 *PRIMARY NODEA *BACKUP 3 *PRIMARY NODED *BACKUP 4 *PRIMARY Intermediate step NODEA Role = *PRIMARY Configured View Configured Opt Node Role Sequence Data Provider NODEA *PRIMARY *PRIMARY NODEB *BACKUP 1 *PRIMARY NODEC *BACKUP 2 *PRIMARY NODED *BACKUP 3 *PRIMARY Intermediate step, corrects configured *PRIMARY. The sequence number for Backup 3 may appear as 4. The relative order is equivalent. NODED Role = *BACKUP Position = *FIRST Configured View Configured Opt Node Role Sequence Data Provider NODEA *PRIMARY *PRIMARY NODED *BACKUP 1 *PRIMARY NODEB *BACKUP 2 *PRIMARY NODEC *BACKUP 3 *PRIMARY Desired configuration but it is not effective until STRAG ROLE (*CONFIG) is performed. 76

77 CHAPTER 4 Working with status of procedures and steps This chapter describes how to work with procedures and steps. Procedures are used to perform operations for application groups. All procedures are associated with an application group. This chapter does not apply to configurations that do not use application groups. When working with status of procedures and steps, it is important to understand how multiple jobs are used to process the steps in a procedure. A procedure uses multiple asynchronous jobs to run the programs identified within its steps. Starting a procedure starts one job for the application group and an additional job for each of its data resource groups. These jobs operate independently and persist until the procedure ends. Each persistent job evaluates each step in sequence for work to be performed within its domain. When a job for a data resource group encounters a step that acts on data groups, it spawns an additional job for each subordinate data group. Each spawned data group job performs the work for that step and then ends. This chapter contains the following topics: Displaying status of procedures on page 78 describes how to display the status of procedure runs, including the most recent run as well as runs kept for their status history. Resolving problems with procedure status on page 80 describes the conditions which cause each procedure status value and the actions required to resolve problem statuses. This includes how to resolve procedure inquiry messages and failed or canceled procedures. Displaying status of steps within a procedure run on page 83 describes how to display status of steps within a procedure as well as the differences between the collapsed and expanded views of the Work with Step Status display. Resolving problems with step status on page 85 describes the conditions which cause each step status value and the actions required to resolve problem statuses. This includes how to resolve step inquiry messages and failed or canceled steps. Acknowledging a procedure on page 89 describes how to manually change a procedure with a status of *CANCELED, *FAILED, or *COMPERR to an acknowledged status. Running a procedure on page 90 describes how to start a user procedure and the parameter that controls the step at which the procedure begins. Canceling a procedure on page 92 describes how to cancel an active procedure. 77

78 Displaying status of procedures Displaying status of procedures You can view the status of runs of procedures from the Work with Procedure Status display. The term the last run of a procedure refers to the most recently started run of a procedure, which may be in progress or may have completed. Also, the status of other previously performed runs of procedures may be available, subject to the current settings of the Procedure history retention policy. The Work with Procedure Status display lists procedures in reverse chronological order so that the most recently started procedures are at the top of the list. Procedures that have never been requested to run do not appear on this display. Figure 8 shows an example of the Work with Procedure Status display subsetted to show only runs of a specific procedure and application group. F11 toggles between views that show the Start time column and columns for the Duration of the procedure and the Node on which the procedure was started. Timestamps are in the local job time. If you have not already ensured that the systems in your installation use coordinated universal time, see Setting the system time zone and time on page 313. Figure 8. A subsetted view of the Work with Procedure Status display. Work with Procedure Status System: Type options, press Enter. 5=Display 6=Print 8=Step status 9=Run 11=Display message 12=Cancel 13=Change status 14=Resume SYSTEMA Opt Procedure App Group Type Status ---Start Time---- SWTPLAN SAMPLEAG *SWTPLAN *COMPLETED 03/01/10 11:25:05 SWTPLAN SAMPLEAG *SWTPLAN *COMPLETED 03/01/10 11:04:58 Bottom Parameters or command ===> F3=Exit F4=Prompt F5=Refresh F9=Retrieve F11=Duration F12=Cancel F13=Repeat F18=Subset F21=Print list Displaying status of the last run of all procedures To display the status of the last run of all procedures for an application group, do the following: 1. From the MIMIX Basic Main Menu, select option 1 (Work with application groups). 2. The Work with Application Groups display appears. Type 21 (Procedure status) 78

79 Displaying status of procedures next to the application group you want and press Enter. The last run of all procedures for the application group are listed on the Work with Procedure Status display. 3. Locate the procedure you want and check the value of the Status column. Displaying available status history of procedure runs To display status of all available runs of a selected procedure, do the following: 1. From the MIMIX Basic Main Menu, select option 1 (Work with application groups). 2. The Work with Application Groups display appears. Type 20 (Procedures) next to the application group you want and press Enter. 3. The Work with Procedures display appears, listing all procedures for the selected application group. Type 14 (Procedure status) next to the procedure you want and press Enter. All available runs for the selected procedure are listed on the Work with Procedure Status display. The most recently started procedure runs are at the top of the list, and may still be active. 4. Locate the run of the procedure you want and check the value of the Status column. Note: To view status of all runs of all procedures for all application groups, you can either press F20 (Procedure status) from the Work with Application Groups display, press F14 (Procedure status) from the Work with Procedures display, or enter the command: WRKPROCSTS. 79

80 Resolving problems with procedure status Resolving problems with procedure status Table 21 identifies the possible status values that can appear on the Work with Procedure Status display and identifies the action to take to resolve reported problems. Table 21. Procedure status values with action required Category Status Value Description and Action Required Active *ACTIVE The procedure is currently running. No steps require attention. *ATTN *MSGW *PENDCNL *QUEUED The procedure requires attention. Either there is a step with a status of *MSGW, or there is an active step and one or more steps with step status values of *ATTN, *CANCEL, *FAILED, or *IGNERR. Action Required: Determine the status of each step and the action required to correct that status. See Resolving problems with step status on page 85. A step within the procedure is waiting for a response to an inquiry message. The procedure cannot process the step or any subsequent steps without a reply to the message. Action Required: Display and respond to the inquiry message using Responding to a procedure in *MSGW status on page 81. A request to cancel the procedure is in progress. When the activity for the steps in progress at the time of the cancel request ends, the procedure status changes to *CANCELED. A request to run the procedure is currently waiting on the job queue. When the procedure becomes an active job, the procedure status changes to *ACTIVE. Resumable *CANCELED Either the procedure was canceled and did not complete, or steps within the procedure were canceled as a response to inquiry messages from the steps. The procedure was partially performed. Action Required: Use Resolving a *FAILED or *CANCELED procedure status on page 82 to determine the state of your environment and whether to resume the procedure or to acknowledge its status. *FAILED The procedure failed. Jobs for one or more steps had errors. Those steps were configured to end if they failed. The procedure was partially performed. Action Required: Use Resolving a *FAILED or *CANCELED procedure status on page 82 to determine the state of your environment and whether to resume the procedure or to acknowledge its status 80

81 Resolving problems with procedure status Table 21. Procedure status values with action required Category Status Value Description and Action Required Acknowledged *ACKCANCEL The procedure was canceled and a user action acknowledged the cancellation so that the procedure can no longer be resumed. *ACKFAILED *ACKERR The procedure failed and a user action acknowledged the failure so that the procedure can no longer be resumed. The procedure completed with errors and a user action acknowledged the procedure. It is assumed that the user reviewed the steps with errors. A status of completed with errors is only possible when the steps with errors had been configured (within the procedure) to ignore errors or a user s response to a step in message wait status was to ignore the error and continue running the procedure. After the step is acknowledged, the procedure status changes to *ACKERR. Completed *COMPERR The procedure completed with errors. One or more steps had errors and were configured to continue processing after an error. Action Recommended: Investigate the cause of the error and assess its implications. *COMPLETED The procedure completed successfully. Responding to a procedure in *MSGW status A procedure in *MSGW status is effectively paused at a known point in its processing as a result of a runtime attribute on one of its steps. The procedure sent an inquiry message because a step specified *MSGW for its Action before step (BEFOREACT) attribute. All jobs for the procedure have completed processing all previous steps and are waiting to run the step s program. An operator response is required. To respond to a procedure in *MSGW status, do the following from the Work with Procedure Status display: 1. To see which step is waiting, type 8 (Step status) next to the procedure and press Enter. 2. The Work with Step Status display appears. The information on this display can be used to determine which step is waiting to start. You will see steps with values of *COMP, *IGNERR, or *DSBLD followed by no status for all remaining steps. The first step with no status is the step that is waiting to start. Based on that step, determine how to respond to the message and whether you are ready to respond. 3. You cannot display or respond to the procedure message from the Work with Step Status display. Press F12 to return to the Work with Procedure Status display. 4. Type 11 (Display message) next to the procedure in *MSGW status and press Enter. 5. You will see the message Procedure name for application group name requires response. (G C). Do one of the following: 81

82 Resolving problems with procedure status A response of G (Go) is required to start processing the step. Type G and press Enter. A response of C (Cancel) will cancel the procedure. Type C and press Enter. Resolving a *FAILED or *CANCELED procedure status When a procedure fails or is canceled, subsequent attempts to run the same procedure will fail until user action is taken. You need to determine the best course of action for your environment based on the implications of the partially performed procedure. This topic will assist you in evaluating the cause of the failure or cancellation, as well as the state of other steps within the procedure. Important! Steps with failed or canceled jobs need to be resolved. Other asynchronous jobs may have successfully processed the same step and continued on to process other subsequent steps before the procedure ended. The actions taken by those steps as well as by completed steps which preceded the problem are not reversed. Some steps may not have been processed at all. Do the following from the Work with Procedure Status display: 1. Type 8 (Step status) next to the *FAILED or *CANCELED run of the procedure and press Enter. 2. The Work with Step Status display appears. Look for steps with a status of *CANCEL, *FAILED, or *ATTN. Also use F7 (Expand) to see status for the jobs which processed the steps. A procedure with *FAILED status did not complete due to errors. In the collapsed status view, one or more steps will have a status *ATTN or *FAILED. Other jobs may have processed subsequent steps before the procedure ended. In the expanded view, look for one or more jobs with a status of *FAILED. For detailed information use Resolving problems with step status on page 85. A procedure with *CANCELED status did not complete due to user action. Any of the following may have occurred: A user cancelled an inquiry message sent by the procedure because a step was configured to wait for a reply before starting. This scenario is identified by the absence of steps with status values of *FAILED, *CANCEL, or *ATTN. Instead, you will see steps with values of *COMP, *IGNERR, or *DSBLD followed by no status for all remaining steps. The first step with no status is the step that waited to start. Continue with step Step 3. A user cancelled an inquiry message sent by a step which had a job that ended in error. At least one step in the collapsed view will have a status of *ATTN or *CANCEL. One or more steps will have job with a status of *CANCEL in the expanded view. Other jobs may have processed subsequent steps before the procedure ended. For detailed information use Resolving problems with step status on page 85. A user canceled the procedure by using option 12 (Cancel) from the Work with Procedure Status display or by using the Cancel Procedure (CNLPROC) command. Steps in the collapsed view could have any status except *ACTIVE or *MSGW. Determine if there are any jobs with status values of *FAILED or 82

83 Displaying status of steps within a procedure run *CANCEL in the expanded view. Other jobs may have processed subsequent steps before the procedure ended. For detailed information use Resolving problems with step status on page After you have completed your evaluation and have taken any needed corrective action to resolve why jobs failed or were canceled, determine how to best complete the procedure. Choices are: Resume the procedure. If you resume a failed procedure, processing will begin with the step that failed. If you resume a canceled procedure, processing will begin with steps following the cancelled step. Optionally, if you were unable to resolve a problem for a step in error, you can override the attributes of that step for when the procedure is resumed. See Resuming a procedure on page 91. Acknowledge the procedure status. Procedures with a status of *CANCELED or *FAILED can be acknowledged (set to *ACKCANCEL or *ACKFAILED, respectively) to indicated you have investigated the problem steps and want to run the procedure again starting at its first step. This option should only be used after you have evaluated the effect of activity performed by the procedure. See Acknowledging a procedure on page 89. Displaying status of steps within a procedure run The Work with Step Status display provides access to detailed information about status of steps for a specific run of a procedure for an application group. Timestamps are in the local job time. If you have not already ensured that the systems in your installation use coordinated universal time, see the MIMIX Administrator Reference book for the setting system time topic. To display step status for a procedure run, do the following: 1. Use one of the following to access the run of the procedure you want: Displaying status of the last run of all procedures on page 78 Displaying available status history of procedure runs on page From the Work with Procedure Status display, type 8 (Step status) next to the run of the procedure you want and press Enter. 3. Press F7 (Expand) to view status of the individual jobs used to process each step. The steps listed on the Work with Step Status display appear in sequence number order as defined by steps in the procedure. If the procedure is in progress, the display shows status for the steps that have run, the start time and status of the step that is in progress, and blank status and start time for steps that have not yet run. 83

84 Displaying status of steps within a procedure run Collapsed view - Figure 9 shows the initial collapsed view of the Work with Step Status display. In this view, each step of the procedure is shown as a single row and step status represents the summary of all jobs used by the step. Figure 9. Collapsed view of the Work with Step Status display. Work with Step Status System: SYSTEMA Procedure: SWTPLAN App. group: SAMPLEAG Type: *SWTPLAN Procedure status: *COMPLETED Start time: 03/01/10 11:04:58 Type options, press Enter. 5=Display 6=Print 8=Work with job 11=Display message Step Node Start Jobs Opt Program Type Type Time Duration Status Pend MXCHKCOM *AGDFN *LOCAL 11:05:00 00:00:01 *COMP *NO MXCHKCFG *DGDFN *NEWPRIM 11:05:00 00:00:01 *COMP *NO ENDUSRAPP *AGDFN *PRIMARY 11:05:00 00:00:03 *COMP *NO MXENDDG *DGDFN *NEWPRIM 11:05:01 00:00:05 *COMP *NO MXENDRJLNK *DGDFN *NEWPRIM 11:05:16 00:00:01 *COMP *NO MXAUDACT *DGDFN *NEWPRIM 11:05:18 00:00:01 *COMP *NO MXAUDCMPLY *DGDFN *NEWPRIM 11:05:19 00:00:01 *COMP *NO MXAUDDIFF *DGDFN *NEWPRIM 11:06:10 00:00:54 *COMP *NO More... Parameters or command ===> F3=Exit F4=Prompt F5=Refresh F7=Expand F9=Retrieve F12=Cancel F13=Repeat F15=Cancel proc. F18=Subset F21=Print list Expanded view - Figure 10 shows an example of an expanded view. In the expanded view, step programs of type *AGDFN will have one row for each node on which the step runs. Steps which run step programs at the level of the data resource group or data group are expanded to have multiple rows so that the status of the step for each data resource group or data group is visible. For step programs of type *DTARSCGRP, there will be a summary row for the application group followed by a row for each data resource group within the application group. For step programs of type *DGDFN, there will be a summary row for the application group, then for each data resource group, there is a summary row for the data resource group followed by a row for each of its data groups. Summary rows are identified by a dash (-) in the columns that are being summarized. 84

85 Resolving problems with step status Also, for step programs of type *AGDFN, the Data Rsc. Grp. column and the Data Group column will always be blank. For step programs of type *DTARSCGRP, the Data Group column will always be blank. Figure 10. Expanded view of the Work with Step Status display. Work with Step Status System: SYSTEMA Procedure: SWTPLAN App. group: SAMPLEAG Type: *SWTPLAN Procedure status: *COMPLETED Start time: 03/01/10 11:04:58 Type options, press Enter. 5=Display 6=Print 8=Work with job 11=Display message Step Data Data Start Opt Program Rsc. Grp. Group Node Time Duration Status MXCHKCOM LTIAS01 11:05:00 00:00:01 *COMP MXCHKCFG - - LTIAS02 11:05:00 00:00:01 *COMP MXCHKCFG DRG1 - LTIAS02 11:05:00 00:00:01 *COMP MXCHKCFG DRG1 DG1A LTIAS02 11:05:00 00:00:01 *COMP MXCHKCFG DRG1 DG1B LTIAS02 11:05:00 00:00:01 *COMP MXCHKCFG DRG1 DG1C LTIAS02 11:05:00 00:00:01 *COMP MXCHKCFG DRG2 - LTIAS02 11:05:00 00:00:01 *COMP MXCHKCFG DRG2 DG2A LTIAS02 11:05:00 00:00:01 *COMP More... Parameters or command ===> F3=Exit F4=Prompt F5=Refresh F7=Expand F9=Retrieve F12=Cancel F13=Repeat F15=Cancel proc. F18=Subset F21=Print list Resolving problems with step status When working with step status, it is important that you understand how multiple jobs are used to process the steps in a procedure. At any given time, job activity may be in progress for multiple steps. Or, one job may have failed processing a step while other jobs may have already processed that step and continued beyond it. Important! Before you take action to resolve a problem with status for a step, be sure you understand the current state of your environment as a result of completed steps and steps in progress, as well as the effect of any action you take. Table 22 identifies the possible status values that can appear on the Work with Step Status display and the action to take to resolve reported problems. Table 22. Status Value blank Step status values with action required Description and Action Required The procedure has started but processing has not yet started for the step. 85

86 Resolving problems with step status Table 22. Status Value *ATTN *ACTIVE *COMP *DSBLD *CANCEL or *FAILED Step status values with action required Description and Action Required The step requires attention. The value *ATTN can only appear in the collapsed view or on a summary row in the expanded view. If the procedure status is considered active, at least one job submitted by this step has a status of *FAILED, *CANCEL or *MSGW. If the procedure status is *FAILED or *CANCELED, this step has at least one job that has not started or has a status of *CANCEL or *FAILED. Action Required: Use F7 to see the expanded view. Determine the specific data resource group or data group for which the problem status exists. Then address the status indicated for that job. The step is currently running. The step has successfully completed. The step has been disabled and did not run. One or more jobs used by the step ended in error. In the expanded view of status, the job is identified as *CANCEL or *FAILED. The status is due to the error action specified for the step. For *CANCEL status, user action canceled the step. The step ran, ended in error, and issued an inquiry message. The user s response to the message was Cancel. For *FAILED status, the step ran, one or more jobs ended in error. The Action on error attribute specified to quit the job. The type of step program used by the step determines what happens to other jobs for the step and whether subsequent steps are prevented from starting, as follows: If the step program is of type *DGDFN, jobs that are processing other data groups within the same data resource group continue. When they complete, the data resource group job ends. Subsequent steps that apply to that data resource group or its data groups will not be started. However, subsequent steps will still be processed for other data resource groups and their data groups. If the step program is of type *DTARSCGRP, subsequent steps that apply to that data resource group or its data groups will not be started. Jobs for other data resource groups may still be running and will process subsequent steps that apply to their data resource groups and data groups. If the step program is of type *AGDFN, subsequent steps that apply to the application group will not be started. Jobs for data resource group or data group steps may still be running and will process subsequent steps that apply to their data resource groups and data groups. When all asynchronous jobs for the procedure finish, the procedure status is set to *CANCELED or * FAILED, accordingly. If both canceled and failed steps exist when the procedure ends, the procedure status will be *FAILED. Action Required: Determine the cause of the problem using Resolving *CANCEL or *FAILED step statuses on page

87 Resolving problems with step status Table 22. Status Value *IGNERR *MSGW Step status values with action required Description and Action Required The step ran and an error occurred, but processing ignored the error and continued. Action Recommended: Use option 8 (Work with job) to determine the cause of the failure. Consider whether any changes are needed to your procedure or step or to your operating environment to prevent this error from occurring again. The step ran and issued a message that is waiting to be answered. One or more jobs for the step ended in error. Step attributes require that an operator respond to the message. Action Required: Determine which job issued the message, investigate the problem, and then respond to the inquiry message using Responding to a step with a *MSGW status on page 87. Responding to a step with a *MSGW status When a step or a job for step has a status of *MSGW, it is the result of an error condition. An inquiry message was sent because the step specified *MSGW for its Action on error attribute. An operator response is required before any additional processing for the job can occur. To respond to a step in *MSGW status, do the following from the Work with Step Status display: 1. To see which job is waiting, use F7 to view the Expanded view. 2. To view information about what caused the job to end in error, type 8 (Work with job) next to job with *MSGW status and press Enter. 3. On the Work with Job display, type 10 (Display job log, if active, on job queue, or pending) and press Enter. 4. The job log is displayed. Use F1 to view details of any of the messages. Find the error that caused the job to end. You will see the inquiry message in the job log; however you cannot respond to it from here. 5. Press F12 twice to return to the Work with Step Status display. 6. Type 11 (Display message) next to the step job in *MSGW status and press Enter. 7. You will see the message Error in step at sequence number number in procedure name. (R C I). Do one of the following: A response of R (Retry) will retry processing the step program within the same job. Type R and press Enter. A response of C (Cancel) will set the job status to *CANCEL as indicated in the expanded view of step status. Subsequent steps are handled in the same manner as if the Action on error has specified the value *QUIT. Type C and press Enter. 87

88 Resolving problems with step status A response of I (Ignore) will set the job status to *IGNERR as indicated in the expanded view of step status, and processing continues as if the job had not ended in error. Type I and press Enter. Resolving *CANCEL or *FAILED step statuses Evaluate the cause of the failure or cancellation, as well as the state of other steps within the procedure. All steps with failed or canceled jobs need to be resolved. Important! For any step which ended in error, other asynchronous jobs may have successfully processed the same step and continued on to process other subsequent steps. The actions taken by those steps as well as by completed steps which preceded the problem cannot be reversed. Do the following from the Work with Step Status display: 1. Use F7 to view the Expanded view. 2. All steps which have a job that has a step status of *CANCEL or *FAILED must be evaluated and the cause of the problem must be resolved. To view information about why a job had an error processing a step, do the following: a. Type 8 (Work with job) next to the job you want and press Enter. b. On the Work with Job display, type 4 (Work with spooled file) and press Enter. c. Display the spooled file for the job and check for the cause of the error. d. Evaluate whether any immediate action is needed due to the condition which caused the error. Consider the nature and severity of the error. 3. If the procedure is still active and you need to take corrective action or perform additional investigation, cancel the procedure using F15 (Cancel proc.). Any steps that are currently running will complete, then the procedure status is set to *CANCELED. 4. Check which steps have completed, failed, were canceled, or have not yet started. Then evaluate the current state of your environment as a result. If needed, take corrective action that is appropriate for the extent of the errors and the extent to which steps completed. Note: It is strongly recommended that you cancel the procedure, if it is active, before attempting any corrective action. 5. Determine how to best complete the procedure in the current state of your environment. When the procedure is *FAILED or *CANCELED, your choices are: Resume the procedure from the point where the procedure ended. If you resume a failed procedure, processing will begin with the step that failed. If you resume a canceled procedure, processing will begin with steps following the cancelled step. Optionally, if you were unable to resolve a problem for a step in error, you can override the attributes of that step for when the procedure is resumed. See Resuming a procedure on page 91. Acknowledge the procedure status allowing the procedure for *CANCELED or *FAILED to be resumed starting with the first step. This choice indicates you have investigated the problem steps and want to run the procedure again 88

89 Acknowledging a procedure starting at its first step. This option should only be used after you have evaluated the effect of activity performed by the procedure. See Acknowledging a procedure on page 89. Acknowledging a procedure Acknowledging a procedure allows you to manually change the status of procedures that either failed or have errors in order to control where the next attempt to run the procedure will start. Procedures with a status of *CANCELED, *FAILED, or *COMPERR can be acknowledged (set to *ACKCANCEL, *ACKFAILED, or *ACKERR, respectively) to indicated you have investigated the problem steps. A procedure of *CANCELED or *FAILED allows you to rerun the procedure from its first step. Once acknowledged, a procedure with either of these statuses cannot be resumed from the point where the procedure ended. This is appropriate when you have determined that your environment will not be harmed if the next attempt to run starts at the first step. A *COMPERR procedure that is acknowledged (*ACKERR) can never be resumed because the procedure completed. By acknowledging a procedure with this status, you are confirming the problems have been reviewed. The last run of a procedure with a status of *ACKCANCEL or *ACKFAILED and the last run of the set of start/end/switch procedures can be returned to their previous status (*CANCELED or *FAILED, respectively). The next attempt to run the procedure will resume at the failed or canceled step or at the first step that has not been started. Note: Acknowledging the last run of a failed or canceled procedure will acknowledge all previous failed or canceled runs of the procedure. Important! Before changing status of a procedure, it is important that you evaluate and understand the effect of the partially performed procedure on your environment. Changing procedure status does not reverse the actions taken by preceding steps that completed or the actions performed by other asynchronous jobs which did complete the same step and then processed subsequent steps. It may not be appropriate for the next run of the procedure to begin with the first step, for example, if the failure occurred in a step which synchronizes data or changes states of MIMIX processes. Likewise, it may not be appropriate to return to the previous status to resume a procedure run was not recently run. To change the status of a procedure, do the following: 1. From the Work with Procedure Status display type 13 (Change status) next to the failed or canceled procedure you want and press Enter. 2. The Change Procedure Status (CHGPROCSTS) display appears. Specify the value you want for the Status prompt and press Enter. 3. If you specified *ACK in Step 2, the Start time prompt appears, displaying the timestamp of the selected procedure run. Do one of the following: To acknowledge only the selected failed or canceled run, press Enter. To acknowledge all previously failed or canceled runs of the selected procedure, specify *ALL for Start time and press Enter. 89

90 Running a procedure Running a procedure The procedure type determines what command to use to run the procedure. For an application group, multiple procedures of type *USER can run at the same time if they have unique names. Only one run of a uniquely named procedure of type *USER can occur at a time. All other procedure types must be invoked by the application group command associated with the procedure type. For example a procedure of type *START can only be invoked by the Start Application Group (STRAG) command. Where should the procedure begin? The value specified for the Begin at step (STEP) parameter on the request to run the procedure determines the step at which the procedure will start. The status of the last run of the procedure determines which values are valid. The default value, *FIRST, will start the specified procedure at its first step. This value can be used when the procedure has never been run, when its previous run completed (*COMPLETED or *COMPERR), or when a user acknowledged the status of its previous run which failed, was canceled, or completed with errors (*ACKFAILED, *ACKCANCEL, or *ACKERR respectively). Other values are for resolving problems with a failed or canceled procedure. When a procedure fails or is canceled, subsequent attempts to run the same procedure will fail until user action is taken. You will need to determine the best course of action for your environment based on the implications of the canceled or failed steps and any steps which completed. The value *RESUME will start the last run of the procedure beginning with the step at which it failed, the step that was canceled in response to an error, or the step following where the procedure was canceled. The value *RESUME may be appropriate after you have investigated and resolved the problem which caused the procedure to end. Optionally, if the problem cannot be resolved and you want to resume the procedure anyway, you can override the attributes of a step before resuming the procedure. The value *OVERRIDE will override the status of all runs of the specified procedure that did not complete. The *FAILED or *CANCELED status of these procedures are changed to acknowledged (*ACKFAILED or *ACKCANCEL) and a new run of the procedure begins at the first step.. To run a procedure of type *USER, do the following: 1. From the Work with Procedures or Work with Procedure Status display type 9 (Run) next to the user procedure you want and press F4 (Prompt). 2. Specify the value you want for Begin at step and press Enter. To run a procedure type other than *USER, do the following: From a command line, enter the application group command associated with the procedure type. For example a procedure of type *START can only be invoked by the Start Application Group (STRAG) command. 90

91 Running a procedure To resume a procedure with a status of *CANCELED or *FAILED, see Resuming a procedure on page 91. Resuming a procedure To resume a procedure of with a status of *CANCELED or *FAILED, do the following: 1. Investigate and resolve problems for steps with errors. See Resolving problems with step status on page Optional: If the problem cannot be resolved, and you want to resume the procedure anyway, use the Override Step (OVRSTEP) command to change the configured value of the step for when the procedure is resumed. See Overriding the attributes of a step on page For a procedure of type *USER, from the Work with Step Status display use F14 (Resume proc.). For all other procedure types, from a command line, enter the appropriate application group command and specify *RESUME as the value for Begin at step (STEP). Overriding the attributes of a step The attributes of a step can be overridden to change the configured value of the step for the current run of the procedure by using the Override Step (OVRSTEP) command. The attributes determine whether the step is run or actions if the step errors for the current run of the procedure when it is resumed. The OVRSTEP command can be used for a procedure that has a status of active (*ACTIVE, *ATTN, *MSGW, *PENDCNL, or *QUEUED), *CANCELED or *FAILED and steps that have a status of *CANCEL or *FAILED. The overridden values apply only for the current run of the procedure when it is resumed. Note: Regardless of procedure status, attributes cannot be overridden for a required MIMIX step or any step with a step status of *COMP or *IGNERR. A procedure with a status of *CANCELED or *FAILED requires user action to resolve a problem. If the problem cannot be resolved and you want to resume the procedure anyway, you can use the OVRSTEP command to disable the step in error or specify the error action to occur when the step is retried. Important! Overriding the attributes of a step should only be done after you have considered how rerunning the step impacts your environment. It is important that you understand the implications for steps which preceded the cancellation or failure in the last run of the procedure. Processing for steps that completed is not reversed. The changes made when using the OVRSTEP command will only apply to the current run of the procedure. The attributes that can be changed will vary depending on the statuses of the specified procedure and step. Consider the following: When the specified procedure has a status of *ACKCANCEL, *ACKFAILED, *ACKERR, *COMPLETED, or *COMPERR, no attributes can be overridden on any step in the procedure. 91

92 Canceling a procedure When the specified procedure has a status that is considered active (*ACTIVE, *ATTN, *MSGW, *PENDCNL, or *QUEUED), only the Action on error (ERRACT) can be overridden. When the specified procedure has a status that can be resumed (*CANCELED or *FAILED), the Action before step (BEFOREACT), Action on error (ERRACT), or State (STATE) can be overridden only on steps that have not yet run, that failed, or that were canceled. Do the following from the Work with Step Status display: 1. Press F7 (Expand) to view status of the individual jobs used to process each step. 2. Type 13 (Override step) next to the step you want and press Enter. 3. On the Override Step (OVRSTEP) display, specify the values you want and press Enter. From the Work with Step Status display, use F14 (Resume proc.) to resume the procedure. See Resuming a procedure on page 91. Canceling a procedure Use this procedure to cancel a procedure with a status that is considered active. This includes procedure statuses of: *ACTIVE, *ATTN, *MSGW, *PENDCNL, and *QUEUED. Important! Use this command with caution. Processing ends without reversing any actions performed by completed steps, which may leave your environment in an undesirable state. For example, ending a switch procedure could result in partially switched data. The status of the procedure will be changed to *PENDCNL. If there are any inquiry messages waiting for an operator response, they are processed as if the response was Cancel. When all activity for currently running steps end, the status of the procedure will be automatically changed to *CANCELED. To cancel an active procedure, do one of following: From the Work with Procedure Status display, type 12 (Cancel) next to the procedure you want and press Enter. From the Work with Step Status display, press F15 (Cancel proc.). A procedure that has been canceled can be resumed later, as long as its status has not been changed to *ACKCANCEL. When a canceled procedure is resumed, processing begins immediately after the point where it was ended. 92

93 CHAPTER 5 Monitoring status with MIMIX Availability Status The MIMIX Availability Status display is useful in environments that do not use application groups. Note: The MIMIX Availability Status should not be used in environments that use application groups. The MIMIX Availability Status display, shown in Figure 11, provides one location for quickly assessing the overall state of an entire MIMIX installation, reflecting both source and target systems. The status values are prioritized and are a composite view reflecting both source and target systems. In addition to determining status, unique features of this display enable its use as the starting point for performing routine actions and resolving problems. To access this display, do one of the following: Select option 1 on the MIMIX Basic Main Menu Enter the command WRKMMXSTS and press Enter. Figure 11 shows the MIMIX Availability Status display. Figure 11. MIMIX Availability Status window. This example shows that MIMIX is active but the installation is not complying with best practices for switching (red) and audits (yellow). Additional fields - In the upper right corner of the display, additional fields report information that is relevant to maintaining the installation. Recoveries - Identifies the total number of recoveries in progress for the 93