Avaya Aura System Manager 5.2 HA and CLI Restore

Transcription

1 Avaya Aura System Manager 5.2 HA and CLI Restore Version: 1.0 June 22nd, 2010

2 Table of Contents 1. Introduction HA Overview How to setup Failure cluster CLI Restore Overview Assumptions CLI Utility Properties Using CLI Utility...15

3 1. Introduction The document describes failover mechanism provided by Avaya Aura TM performing System Manager 5.2 database restore using the CLI utility. System Platform and 2. HA 2.1 Overview Avaya Aura TM System Manager 5.2 GA follows the failover mechanism provided by Avaya Aura TM System Platform System Platform HA (High Availability) implements Active/Standby mode of failover - resources (virtual machines - VMs) are running on one node only. All disk data is continuously mirrored from the active to the standby, and in case of failure of the active the resources are automatically started (VMs are booted) on the standby. System Platform HA considers virtual machines running on active cluster node only - configuration also known as Active/Standby mode''. All block devices that are part of DRBD synchronisation propagates all changes from the active node to the standby node. We use the most reliable protocol C - synchronized replication This protocol ensures that all block changes are committed and acknowledged on the secondary node before continuing. Thus all changes that occur on the active node are immediately replicated to the secondary node. Following actions are executed in the ''failover'' scenario: Heartbeat detects problems of the active (primary) node by missing heartbeat checks for defined period of time. Secondary node is assigned as a new primary. DRBD devices will be set as primary on a new active node. Virtual machines are booted on the new active node. Following actions are executed in the ''manual switchover'' scenario: Virtual machines will be shutdown gracefully on the active node. DRBD devices will be set as secondary on the active node. Secondary node is assigned as a new primary node and vice versa. DRBD devices will be set as primary on new active node. Virtual machines are booted on a new active node. System Platform uses the simplest and basic HA scenario: Use 2 exactly equal nodes with 1 public NIC and 1 HA dedicated NIC (used for HA pings and DRBD propagation between the two nodes). Use both NICs as ping paths between cluster nodes, use network switch (gateway IP address) as a public ping point - so each node has 3 ping points. Heartbeat is able to detect which node has more communication paths available and migrates resources into such node. Use default ports for pings (694). We use Active/Standby configuration where all resources runs on one node only and all resources are migrated. The IP addresses for all virtual machines (e.g. CDom and System Manager) remain the same on the active and standby. Heartbeat starts DRBD service on both nodes. For each VM on selected cluster node, heartbeat starts all VMs DRBD resources and after that it starts VM itself. There is a Domain0 LVM that is used to synchronize data by DRBD between cluster nodes - for this resource we define also mounting procedure on active node.

4 2.2 How to setup Failure cluster This section briefly explains how to install and setup System Platform failover cluster: You need to have 2 exactly the same boxes (configuration procedure checks HW and SW components). You should use System Platform supported HWs. You must setup crossover connection between nodes so you have to have access to both boxes and they must be close to each other. You must install System Platform on both nodes, for this you will need: o Two IP addresses for primary Dom0 and cdom. o Two IP addresses and secondary Dom0 and cdom, although the secondary cdom will be removed later. o There is no need to install System Manager template VMs on both machines. Install System Manager only the preferred node. After both the systems are installed, you can login as administrator into the "preferred" node webconsole - the one that you want to make primary.

5 Those two boxes must be connected with crossover cable at the time failover cluster setup, it '''must''' be eth2 (usually you do not need real crossover cable as new NICs are able to emmulate it) On the webconsole, click `Server Management - Failover`. You can also click the link `Failover status:` in the page header.

6 On the failover web page, you will see the configuration details. The first step is Configure Failover (cluster). Click the corresponding button at the bottom of the page.

7 You will be redirected to the Configure cluster detail page. You must specify secondary `cdom` IP address and password of its `admin` user. Click the create cluster if you are ready, or you can cancel the operation by hitting `Cancel` button. When Create cluster method has finished successfully, you can then `Start Failover Mode` - you will find additional button on the failover detail status page. Click on this button. This method restarts cdom so you have to confirm the warning message if you want really to continue.

8 You will be disconnected from webconsole for a while - until cdom is rebooted (aprox. 3 minutes). After the console domain is up again, you have to log in again.

9 You can go to the HA detail status webpage.

10 On the detail HA status page, you will find additional table at the bottom of the page. This enlists DRBD devices that are set up on the system and you can see the synchronisation progress. By default the synchronisation speed is throttled to 30MB/s rate as it provides enough throughput for propagation of online changes. You have an option to increase this rate temporally if you want to speed up the initial synchronisation. However, maximum rate available is highly dependant on the hardware setup. Note: Do not change the default sync speed if system is busy or if your hardware is less powerful! If the system is able to handle increased sync speed, you should see the increased number of actual sync speed column.

11 Until the initial full sync process is not finished yet, you cannot execute `manual switchover` - the data are not yet fully transferred into secondary node and VMs would not be able to boot up. During this period the system is not covered - failover would most likely fail. Note: Try to prevent any network or power outages during initial sync process. After initial sync process is completed, a new button (Switch Over) will appear on the webpage. This button enables to migrate all resources from actual active node into actual standby node. It means, the system is now covered and any hardware failure would automatically trigger failover to standby node. The methods does the following steps: o stops local VMs o unmounts DRBD devices that are mounted o stops local DRBD devices o starts remote DRBD devices o mounts remote DRBD devices o start remote VMs If you have to execute the manual switchover (e.g. for maintenance of the active machine) you can trigger it from webconsole by clicking on the Switch Over button.

12 You also have to confirm the warning message. After that, the cdom is shutdown on the active node and booted on the standby node (that basically will become new active one). You can see the difference on the list of Virtual Machines by checking the Domain-0 IP address or on the detail failover status page.

13

14 3. CLI Restore 3.1 Overview The CLI Restore utility will be used by Admin users while performing the System Manager 5.2 database restore using the CLI utility. 3.2 Assumptions The backup that is planned to be restores is created using the backup functionality provided by System Manager [Home / Settings / Backup and Restore /Backup]. The restore should be performed on the version of System Manager same as that of System Manager on which backup was done. The restore should be performed on System Manager setup with same IP address and hostname as that of System Manager on which backup was done. 3.3 CLI Utility Properties While performing a CLI restore user may need to modify few of the properties related to restore as per the current setup. The file containing the properties related to CLI restore is: $MGMT_HOME/pem/fileRestoreCLIUtility/fileRestoreCLIUtility.properties The complete set of properties related to CLI restore are listed in the table below: No. Property Name Description 1 version Version of the current System Manager setup where restore need to be performed. The value can be determined from both UI and CLI as: 1. UI Login to System Manager Access screen [Home / Settings / Service Profile Management / System Manager 5.2] Value specified by property [System Manager Properties >> Build Version] specify the System manager version. 2. CLI Content of the below file specify the System Manager version string: $MGMT_HOME/installer_relno.txt 2 db_type Database type. Default set to postgres and should not be modified. 3 db_directory Location of database utility installation. Default set to /usr/bin and should not be modified. 4 db_host IP/hostname of the database machine (in this case System Manager machine). Default set to localhost and should not be modified. 5 db_port Database server port. Default set to 5432 and should not be modified. 6 db_name Database name that need to be connected for restore. Default set to avmgmt and should not be modified. 7 db_scpport SSH port to be used to connect database machine. Default set to 22 and does not need changes unless user has modified configuration for SSH port.

15 8 backup_destination Full path of the directory to be used as temporary directory for extracting and processing backup archives. Default set to /var/lib/pgsql/backup and does not need modification. 9 backup_name Full path to the backup archive including the archive name, e.g. if the archive name s backup.zip and the path where the archive is present is directory /var/lib/pgsql/backup/manual/ then the value of the backup_name property should be /var/lib/pgsql/backup/manual/backup.zip 10 scp Specifies if backup archive is stored on local System Manager machine or a remote machine. Value false means archive is on local System Manager machine and the value true means the archive is on some remote machine. Default set to false. 11 scp_ip IP/hostname of the remote server having backup archive. The property is used when the value of property scp is set to true. 12 scp_port SSH port to be used to connect to remote server having backup archive (default set to 22). The property is used when the value of property scp is set to true. 13 user User performing restore operation. Any user name can be specified. 14 remote_utility_directory Full path to the directory having System Manager utilities required for restore. Default is set to /var/lib/pgsql and should not be modified. 15 cont_restore_imsm_ver_mismatch Flag to specify if to continue with the restore if the System Manager version in backup archive is different than current version of System Manager (i.e., backup is taken on different version and restore is being performed on different version of restore). The value true mean to continue with restore process in such case and the value of false means to abort the restore process in such case. Default set to false and should not be modified. 3.4 Using CLI Utility 1. Login to System Manager machine. 2. Update file $MGMT_HOME/pem/fileRestoreCLIUtility/fileRestoreCLIUtility.properties with required details. When the backup archive is present on the local System Manager machine and the default values related to restore (as specified in section 3) are not modified then user needs to update below properties: version backup_name scp (set to false) user If the backup is on a remote machine and the default values related to restore (as specified in section 3) are not modified then user needs to update below properties: version backup_name scp (set to true) scp_ip scp_port user

16 3. The utility for CLI restore is present in the directory $MGMT_HOME/pem/fileRestoreCLIUtility. Run the below mentioned command to move to the directory having CLI scripts: cd $MGMT_HOME/pem/fileRestoreCLIUtility 4. Run the below mentioned command from current directory:./file_restore.sh $MGMT_HOME/pem/fileRestoreCLIUtility 1 5. It will prompt for entering the full path of backup archive. If the value is not specified in the filerestorecliutility.properties as mentioned in point 2 above then specify the full path to the backup archive. If the value is already specified in the filerestorecliutility.properties then do not specify any value here and press Enter. 6. If you are performing restore with the backup archive present on a remote computer (scp set to true), then the system prompts you to enter the scp user name. Specify the user name that can be used for performing SSH on the remote computer and obtain permission to access the backup archive. Press Enter. 7. If you are performing restore with the backup archive present on a remote computer (scp set to true), then the system prompts you to enter the password for scp user. Specify the password of the scp user mentioned in Step 6 and press Enter. 8. It will prompt for entering the database super user name. Specify postgres as the value and press Enter. 9. It will prompt for entering the password for database super user. Specify the password of database super user postgres and press Enter. 10. It will prompt for entering the choice for overwriting the current database with the one present in the backup archive. Specify y as the value and press Enter. 11. It will prompt for entering the database application user name. Specify postgres as the value and press Enter. 12. It will prompt for entering the password for database application user. Specify the password of database application user postgres and press Enter.