Abstract. Avaya Solution & Interoperability Test Lab

Transcription

1 Avaya Solution & Interoperability Test Lab Application Notes on placing an Avaya Aura Contact Center R6.2 (Mission Critical High Availability) installation out-of-service and back in-service - Issue 1.0 Abstract These Application Notes provide guidelines and recommended practices to be followed by personnel involved in Avaya Aura Contact Center R6.2 maintenance who must place the installation in an out-of-service condition. It is important when taking an Avaya Aura Contact Center out-of-service that components are power down in a predetermined order to ensure all services are terminated in the correct manner and that customer and configuration data is preserved. Failure to follow such procedures may result in loss of data and/or a malfunctioning or inoperative Avaya Aura Contact Center when service is resumed. Placing the Avaya Aura Contact Center back inservice also requires careful consideration of the order services are brought back on-line to prevent lockout and confirm all service are operating normally. Information in these Application Notes has been obtained through Solution and Interoperability Lab full stack testing and additional technical discussions. 1 of 52

2 1. Introduction Avaya Aura Contact Center (AACC) is a standards-based, robust, software only solution that facilitates agents and knowledge workers to efficiently and effectively collaborate with customers and partners across multiple media types to manage customer experience. A typical Avaya Aura Contact Center installation (see Figure 1) consists of multiple software components installed and configured on industry standard servers. Due to the complex interactions between the various Avaya Aura Contact Center elements, it is important to execute correct procedures when bringing an installation out-of-service. Failure to follow correct actions may result in loss of data or an inoperable installation when service is resumed. These Application Notes will document the procedures necessary to place an Avaya Aura Contact Center R6.2 (High Availability) installation in an out-of-service condition. Such actions are often necessary as a precursor to Data Center outages or LAN topology changes and personnel should ensure these actions are taken prior to any loss of data connectivity or power. Note that these Application Notes do not cover the additional equipment installed to support Avaya Aura Contact Center installation (e.g., Active Directory\Domain Name Servers, Telecommunications systems, Servers and Agent Desktops). When considering bringing the Avaya Aura Contact Center installation back into service it is vital to ensure Telephony and Data Center services are operational to an acceptable level prior to commencing this action as failure or non availability of key ancillary services may result in misoperation or poor service. 2. Interoperability Testing While there are many different supported AACC configurations, these Application Notes will discuss a Mission Critical High Availability (HA) installation. AACC R6.2 applications are installed on servers running Windows Server 2008 R2 64-bit operating system. It is permitted to install more than one AACC application per server to save hardware costs and rack space, however, a Mission Critical High Availability (HA) installation requires some core components to be the sole application installed on a server. The following is a list of the core applications and the install specification. Avaya Aura Contact Center Applications Contact Center Manager Server (CCMS) Communication Control Toolkit (CCT) Contact Center Manager Administration (CCMA) Contact Center Multimedia (CCMM) Avaya Media Server Co-residency Application Status Can be co-resident with CCT and CCMA Can be co-resident with CCMS and CCMA Can be co-resident with CCT and CCMS Standalone application only Standalone application only As well as core AACC applications; a typical installation may also contain an Avaya Aura Experience Portal (AAEP) to support Interactive Voice Response (IVR) for callers and to detect Dual Tone Multi-Frequency (DTMF) responses to IVR prompts. Data from the AAEP may be sent to an Automatic Speech Recognition\Text-to-Speech (ASR\TTS) server which will render the submitted data into speech and play this back to the caller to confirm their input. 2 of 52

3 2.1. Test Description and Coverage The following diagram (Figure 1) shows a HA AACC installation in a typical AACC environment with associated telecommunications equipment and contact center agent equipment. Avaya Aura Communication Manager Avaya Aura Contact Server Avaya Aura Application Enablement Services - HA Avaya Aura Session Manager 1 Avaya Aura Session Manager 2 Avaya Aura Communication Manager HA Contact Center Manager Server (CCMS) Contact Communication Toolkit (CCT) Contact Center Manager Administration (CCMA) Contact Center Multimedia (CCMM) Avaya Aura Agent Desktop (AAAD) Figure 1: Typical Avaya Aura Contact Center Installation The procedures discussed in the following sections only apply to servers and applications which are core elements of the AACC (as shown in the diagram) or supply critical non call associated services to the contact center core Test Results and Observations The following observations and outcomes are brought to the attention of users to document unexpected consequences or actions which might be observed when following these Application Notes. In Section 6.2, the MPP application on the AAEP server had to be restarted manually after the server was restarted. This is design intent if the MPP application is manually stopped before server shutdown. In Section 6.4, the standby CCMS/CCMA/CCT server database had to be restored before it could re-enter service. This is an unexpected step, given that the standby server was removed from service before the active server and restored to service after the active server. Restoring the database is normally required after a server outage or swap over from active to standby servers. 3 of 52

4 In Section 6.5, the active and standby Avaya Media Servers were powered up and process locks had to be removed before the servers would re-enter service. Restarting the servers does not clear pending locks automatically. This is design intent. 3. Reference Configuration The following diagram shows the reference contact center equipment list. AACC servers and applications will be discussed in later sections. Figure 2: Mission Critical Avaya Aura Contact Center 4 of 52

5 4. Equipment and Software Validated The following equipment and software were used for the reference configuration provided. Equipment Dell PowerEdge R610 Dell PowerEdge 1950 Dell PowerEdge 1950 HP Proliant DL360 G7 Server Software Avaya Aura Contact Centre CCMS / CCT / CCMA R6.2 SP5 + RU01, RU02, RU03 (roll-up) GA patches. Avaya Aura Contact Centre CCMM R6.2 SP5 + RU01, RU02, RU03 (roll-up) GA patches. Avaya Media Server R QFE Patches 1 11 (GA patches). Avaya Aura Experience Portal R6.0.1 Dell PowerEdge 1950 Nuance Speech server R5.0 5 of 52

6 5. Shutdown Avaya Aura Contact Center Components Perform the following procedures in sequence to correctly shutdown all elements of AACC. Shutdown the Active and Standby Avaya Media Server Shutdown the ARS\TTS application Shutdown the Avaya Aura Experience Portal server (AAEP) Shutdown the Avaya Aura Contact Center Multimedia server Shutdown the Active and Standby Contact Center servers 5.1. Shutdown the Active and Standby Avaya Media Server Log on to the active Avaya Media Server using a web browser (Internet Explorer recommended) by browsing to In the log in screen, enter root as the User ID, the Password is the root password. Click on the Log In button. 6 of 52

7 In the AMS element manager sidebar, click on Element Status; in the resulting page click on the More Actions drop down menu and select the Pending Lock list entry. The page will refresh, click on the Confirm button to apply pending lock. Repeat these three steps for the standby AMS by browsing to URL 7 of 52

8 Click on the active AMS element manager sidebar entry Monitoring to expand a submenu. Click on the Active Sessions entry and in the resulting page; ensure the Sessions and Resources counts are zero for all entries. If there are active calls present, this may take several minutes. Note the CPU Load (%) value may fluctuate and have a non zero value, this is normal. Repeat this procedure for the standby AMS. Monitor both active and standby AMS. When all Session and Resource counts have reached zero, click on the active AMS element manager Element Status sidebar entry. In the resulting screen, click on the Stop button. 8 of 52

9 The page refreshes and presents an Element Status Confirmation screen. Click on the Confirm button. Repeat this step for the standby AMS. Both active and standby AMS s should display a screen similar to the following. Note the Stop and Restart buttons should be grayed out and Service Status: should read Stopped. 9 of 52

10 Launch a SSH client (Putty or similar) and log into the active Media Servers by entering the IP address of each server, using port 22 and with root credentials. Type shutdown h now at the command line and press return. The AMS will respond with: - The system is going down for system halt NOW!; the SSH session will end. Repeat this procedure for the standby AMS. This completes the AMS shutdown procedure. 10 of 52

11 5.2. Shutdown the ARS\TTS application The ARS\TTS server runs on a Windows Server 2008 platform. From a client machine launch the Windows utility Remote Desktop Connection and remotely logon to the ARS\TTS server using the Administrator logon ID and Password. The Nuance Speech Server and Nuance Licensing Service are both installed as Windows local services and automatically start if the ARS\TTS server is restarted. 11 of 52

12 To shutdown the ARS/TTS server, click on the arrow next to the Windows Log off button and select the Shut down option from the list. When the Shut Down Windows dialog box appears, click the Planned checkbox, select the Other (Planned) option and type a reason in the comment box. Press the OK button when completed. This completes the ARS/TTS server shutdown procedure. 12 of 52

13 5.3. Shutdown the Avaya Aura Experience Portal server (AAEP) Experience Portal is designed to automate caller interactions across all media and modes, improve customer satisfaction and reduce staffing costs in Contact Centers. The installation comprises four sub components; Experience Portal media server(s), Experience Portal Manager, Orchestration Designer, and the Application Execution Environment. Individual components may reside on separate servers or for smaller scale deployments can be loaded on a single server. In this installation, all AAEP components reside on the same server platform. Use a web browser (Internet Explorer recommended) and browse to URL For the User Name:, enter administrator user ID, Password is the administrator password. Click on the Logon button when ready. 13 of 52

14 In the main element manager page, click on the Application Server side menu entry. The page will refresh and present the Application Server status. Ensure the checkbox next to aaep is checked, and then click the Stop button. A confirmation dialog box will appear, click on the OK button to continue. Stopping the Application Server may take several minutes to complete. 14 of 52

15 When the Application is stopped, the control page will show the aaep service status as Stopped. Click on the MPP Manager side menu entry. The page will refresh and present the MPP Manager status. Ensure the checkbox next to MPP is checked, and then click the Stop button. No confirmation dialog is presented, the MPP service commences shutdown. To monitor the activity, regularly press the Refresh button or else wait for the screen to update as shown below. 15 of 52

16 When AAEP and MPP services have been successfully shutdown, it is recommended to perform a server backup prior to shutdown. Navigate to the System Backup side menu. A new page appears. Confirm the backup parameters are acceptable and press the Backup Now button when ready. Press the Refresh button to confirm the backup task has started. 16 of 52

17 Wait for the backup task to complete or press the Refresh button until the following is displayed. Launch a SSH client (Putty or similar) and log into the AAEP server using the IP address, with port 22 and with craft logon credentials. Type su sroot at the command line and press return. 17 of 52

18 Enter the sroot password. Type shutdown h now and press return. The server will respond with: - The system is going down for system halt NOW!; the SSH session will end. This completes the AAEP shutdown procedure. 18 of 52

19 5.4. Shutdown the Avaya Aura Contact Center Multimedia Server Contact Center Multimedia server expands the Contact Center capability to allow agents to view, respond to and track requests from customers sent over the Internet. , Web, SMS text, Faxed documents and Voice mail contacts are supported. A High Availability CCMM installation setup requires a pair of CCMM servers (Active and Standby). CCMM runs on Windows Server From a client machine launch the Windows utility Remote Desktop Connection to remotely logon to the Active CCMM server using the Administrator logon ID and Password, as in Section 5.2. Click on the Start button on the Windows taskbar and select Database Maintenance. 19 of 52

20 The Contact Center Database Maintenance application launches. Select Backup Locations from the Main menu and confirm the settings are valid and sufficient disk space exists. Click on Immediate Backup. Confirm Network Location is checked, the Backup Location value matches the previous screen and CCMM and ADMIN are checked. Click on the Backup button. 20 of 52

21 A dialog will ask for confirmation, click on the Yes button. Database backup will commence and may take several minutes. The elapsed time is indicated on screen. When the operation is completed, a dialog box appears. Click on the OK button. Click on the Exit button to dismiss the Contact Center Database Maintenance application. Click on the Start button on the Windows taskbar and select System Control and Monitor Utility. 21 of 52

22 In the resulting window, select the CCMM tab and then click on the Shut down CCMM button. A dialog box appear, press the Yes button. Monitor the progress on screen. 22 of 52

23 When shutdown is complete (as indicated under the Progress bar), click on the Close button. To shutdown the CCMM server, click on the arrow next to the Windows Log off button and select the Shut down option from the list. 23 of 52

24 When the Shut Down Windows dialog box appears, click the Planned checkbox, select Other (Planned) under Option: and type a reason in the comment box. Press the OK button when completed. This completes the CCMM active server shutdown procedure. Repeat this procedure (excluding the Database Backup steps) for the Standby CCMM server. 24 of 52

25 5.5. Shutdown the Active and Standby Contact Center servers A set of Contact Center applications (CCMS, CCT, and CCMA) co-reside on the same server platform. These components are the core of the Contact Center and process scripts and contacts. In a High Availability installation, there are two core server installations, one set of applications is called the active set, and the other set the standby. The standby applications track the state of active calls but do not process calls. The standby CCMS monitors the active CCMS, the standby CCT monitors the active CCT. In the event of the active server going offline, the standby assumes the active role; consequently the standby server must be powered off first. Logon to the standby CCMS/CCT/CCMA server by using a client machine to run the Windows utility Remote Desktop Connection, and remotely logon to the server using the Administrator logon ID and Password as per Section 5.2. On the Windows taskbar, press the Start button and select the System Control and Monitor Utility entry. 25 of 52

26 The Contact Center System Control and Monitor Utility application launch. Click on the Shut down HA System button. A confirmation dialog appears, click the Yes button. A System Update notification appears above the Windows taskbar indicating the standby server is shutting down. 26 of 52

27 Progress bars in the Contact Center System Control and Monitor Utility confirm the shutdown process in real time. Wait until all progress bars have reached 100%, then click the Close button. 27 of 52

28 To shutdown the standby CCMS/CCT/CCMA server, click on the arrow next to the Windows Log off button and select the Shut down option from the list. When the Shut Down Windows dialog box appears, click the Planned checkbox, select the Other (Planned) Option: and type a reason in the comment box. Press the OK button when completed. 28 of 52

29 This completes the standby CCMS/CCT/CCMA server shutdown procedure. 29 of 52

30 Log on to the active CCMS/CCT/CCMA server using a client machine to run the Windows utility Remote Desktop connection and remotely logon to the server using the Administrator logon. On the Windows taskbar, press the Start button and select the Database Maintenance entry. The Contact Center Database Maintenance application launches. Select Backup Locations from the Main menu and confirm the settings are valid and sufficient disk space exists. 30 of 52

31 Click on the Immediate Backup menu entry. Confirm the Network Location is correct, the Backup Location value matches the previous screen and Applications CCT, CCMS and ADMIN are checked. Click on the Backup button. A dialog will ask for confirmation, click on the Yes button. Database backup will commence and make take several minutes. The elapsed time is indicated on screen. When the operation is completed, a dialog box appears. Click on the OK button. 31 of 52

32 Click on the Exit button to dismiss the Contact Center Database Maintenance application. Click on the Start button on the Windows taskbar and select System Control and Monitor Utility. Click on the Shut down HA System button. A confirmation dialog appears, click the Yes button. 32 of 52

33 The Contact Center System Control and Monitor Utility display the shutdown in real time. When the shutdown is complete, all services are shutdown. Click the Close button. 33 of 52

34 To shutdown the active CCMS/CCT/CCMA server, click on the arrow next to the Windows Log off button and select the Shut down option from the list. When the Shut Down Windows dialog box appears, tick the Planned checkbox, select the Other (Planned) Option: and type a reason in the comment box. Press the OK button when completed. This completes the active CCMS/CCT/CCMA server shutdown procedure. Power off all servers and equipment (if required). 34 of 52

35 6. Restart Avaya Aura Contact Centre Perform the following procedures in sequence to correctly restart all elements of AACC Restart the ARS\TTS application Restart the Avaya Aura Experience Portal server Restart the Active and Standby Contact Center Multimedia servers Restart the Active and Standby Contact Center Manager Server, Contact Communication Toolkit, Contact Center Manager Administration servers Restart the Active and Standby Avaya Media Servers 6.1. Restart the ARS\TTS application Apply power to the ARS\TTS server and press the power button (if required). Confirm the server is operational by logging in using the administrator credentials and check the following services are running by clicking on the taskbar Start button, then selecting Run and entering services.msc. Press Return. This completes the ARS\TTS server restart procedure Restart the Avaya Aura Experience Portal server Apply power to the ARS\TTS server and press the power button (if required). Use a web browser (Internet Explorer recommended) and browse to URL Enter the administrator User Name and Password in the logon page. Click on the Logon button when ready. 35 of 52

36 In the main element manager page, click on the MPP Manager side menu entry. The page will refresh and present the MPP Manager status. Ensure the checkbox next to MPP is checked, and then click the Start button. In the new page, press the Refresh button or wait until the page refreshes automatically. Ensure the MPP status is as the following screenshot. Mode should be Online State should be Running Config should be OK 36 of 52

37 Click on the Application Server side menu entry. The page will refresh and present the Application Server status. Ensure the State value says Running. This completes the AAEP restart procedure Restart the Active and Standby Contact Center Multimedia servers Apply power to the active CCMM server and press the power button (if required). Confirm the server is operational by logging in using the administrator credentials and clicking on the Start button in the Windows taskbar and select System Control and Monitor Utility. 37 of 52

38 In the resulting window, select the CCMM tab and check services are operational as in the following screenshot. Click the Close button when ready. Repeat the procedure for the Standby CCMM server, and it will have fewer services operating (see following screenshot). Click the Close button. This completes the CCMM server restart procedures. 38 of 52

39 6.4. Restart the Active and Standby Contact Center Manager Server, Contact Communication Toolkit, Contact Center Manager Administration servers Apply power to the active CCMS/CCT/CCMA server and press the power button (if required). Confirm the server is operational by logging in using the administrator credentials and clicking on the Start button in the Windows taskbar and select System Control and Monitor Utility. 39 of 52

40 In the resulting window, select the LM tab and check the service is operational. 40 of 52

41 Click on the CCMS tab and ensure all services are running. Click on the CCMA tab and ensure all services are running. 41 of 52

42 Click on the CCT tab, and ensure all services are running. Click on the Close button when ready. This completes the active CCMS/CCT/CCMA restart procedure. 42 of 52

43 The Standby CCMS/CCT/CCMA server database needs to be restored before bringing the server back into service. On the Windows taskbar, press the Start button and select the Database Maintenance entry. The Contact Center Database Maintenance application launches. Select Restore from Main Menu and check the radio button beside CCT. Click on the Restore button when ready to begin. 43 of 52

44 A dialog box appears asking for confirmation, click on the Yes button. Restore progress is indicated by a timer. When completed, the following is displayed. Click on the OK button. Repeat the above steps for the CCMS application. Repeat the above steps for the ADMIN application. 44 of 52

45 The database restore procedure installs a snapshot of the active CCMS/CCT/CCMA server database on the standby CCMS/CCT/CCMA server. This snapshot will contain the IP address of the active CCMS/CCT/CCMA server, which will be an incorrect setting in the standby server configuration. Before restarting standby contact center applications and services, the correct IP address needs to be restored to the standby Server configuration. On the Windows taskbar, press the Start button and select the Server Configuration entry. 45 of 52

46 The Server Configuration application launches. Change the IP Address to the correct value for the standby CCMS/CCT/CCMA server. Click on the Apply All button, then the OK button An information dialog box will appear, press the OK button. 46 of 52

47 Click on the Start button in the Windows taskbar and select System Control and Monitor Utility. The System Control and Monitor Utility launches. Click on Start HA System. The services will commence starting. Progress will be reported in the System Control and Monitor Utility and should result in the screenshot below if successful. 47 of 52

48 To confirm all services are restarted, click the LM tab. Ensure the License Service is running. Click the CCMS tab, and ensure all services except CCMS_MasterService are running. 48 of 52

49 Click the CCMA tab, and ensure all services except IceRTDService are running. Finally, click the CCT tab, ensure all services are running. This completes the CCMS/CCT/CCMA server restarts. 49 of 52

50 6.5. Restart the Active and Standby Avaya Media Servers Apply power to the active and standby AMS servers and press the power button (if required). Login to the active AMS server (as in Section 5.1). In the element manager sidebar, click on Element Status; in the resulting page click on the More Actions heading and select the Unlock list entry. The screen refreshes and presents a confirmation page. Click on the Confirm button. The screen refreshes automatically, wait until Operational State: Unlocked appears. 50 of 52

51 Repeat this procedure for the standby AMS server. On the element sidebar, click on the Cluster Status entry. In the Cluster Status page, confirm both elements have Normal Element Status and there are No Alarms reported. 7. Verification Steps Use the ICMP utility ping to confirm servers which were shut down are no longer reachable. Observe the front panel displays, indicators are present which will show the server hardware status. When powering up servers, use ping to determine when to remotely login. Place calls to the contact center phone number and confirm the caller gets IVR treatment. Confirm the caller can use DTMF tones to enter account numbers and the AAEP collects these, passes them on to the ARS/TTS server, which repeats the numbers to the caller. Ensure the call is transferred to a contact center agent. 8. Conclusion These Application Notes document the procedures to be used when powering down an Avaya Aura Contact Center and also to be used when returning the installation to full service. By following these procedures, the chance of data corruption will be kept to a minimum and the contact center can be returned to service callers in the most expedient manner. Some unexpected minor issues were discovered during the testing, see Section 2.2 for details. 51 of 52

52 Avaya and the Avaya Logo are trademarks of Avaya Inc. All trademarks identified by and are registered trademarks or trademarks, respectively, of Avaya Inc. All other trademarks are the property of their respective owners. The information provided in these Application Notes is subject to change without notice. The configurations, technical data, and recommendations provided in these Application Notes are believed to be accurate and dependable, but are presented without express or implied warranty. Users are responsible for their application of any products specified in these Application Notes. Please any questions or comments pertaining to these Application Notes along with the full title name and filename, located in the lower right corner, directly to the Avaya Solution & Interoperability Test Lab at 52 of 52