Avaya IQ 5.1 Database Server Configuration Recommendations
Avaya IQ Database Server Page 2 of 11 Issue 4.0 1. INTRODUCTION... 3 1.1 Purpose...3 1.2 BACKGROUND...3 1.3 Terminology...3 2. CONFIGURING IQ DATABASE SERVER... 3 2.1 Physical Disk Requirements...4 2.1.1 Data Storage Disk Requirements...4 2.1.2 System and Oracle Managed Files...5 3. ORACLE PROCEDURES FOR AVAYA IQ DATABASE... 6 4. KNOWN IQ ISSUES RELATED TO DATABASE SERVER... 7 5. ORACLE DOCUMENT REFERENCES... 7 6. APPENDIX A MULTI-HOST DATABASE SERVER... 8 7. APPENDIX B DUAL HOST DATABASE SERVER... 9 8. APPENDIX C REPORT MIX... 10
Avaya IQ Database Server Page 3 of 11 Issue 4.0 1 1. INTRODUCTION 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 1.1 PURPOSE This document provides Avaya IQ database configuration recommendations for software-only deployments. Guidelines are provided for Oracle configuration details. The information provided is based on benchmarks gathered from specific usage scenarios. The recommendations are provided to assist in configuring the IQ database server for a given workload. This document does not provide database setup procedures (see Avaya IQ Preparing for Implementation). Database server sizing recommendations are provided by the Avaya IQ Implementation Planning Tool (IPT) Customer Report output. The recommendations in this document apply to Avaya IQ release 5.1 or later. All throughputs, processor utilization, memory, disk, and other performance data below are estimates based on certain assumptions and conditions regarding typical customer usage patterns. No representation is made that these estimates are accurate. Any configuration recommendations should be tested and verified. 1.2 BACKGROUND Avaya IQ has characteristics of both online transactional processing database (OLTP) and online analytical processing database (OLAP). Processing of contact center administration and traffic data generates bursts of small database transactions (inserts/updates). This is typical OLTP operations. The Avaya IQ reports, aggregations and data export applications execute complex queries against the multidimensional views of data in the real-time and historical database. This is typical database operation in an OLAP (i.e. decision support system, DSS; Data Warehouse) database. The impact of these operations on the performance of the database server depends on the traffic volume and call complexity. The database must provide sufficient performance and throughput to keep up with the database transactions. If the database cannot keep up with your contact center traffic rate, real-time and historical data may be delayed or lost. 1.3 TERMINOLOGY CM Busy Hour BHC IPT Avaya Communication Manager The busy hour or hours in a 24 hour period during which a contact center experiences the greatest number of contact completions. It is expected that the system will experience peak real-time reporting usage during this time period. Number of contacts (includes CM, Proactive Contact, Voice Portal Contacts) during Busy Hour. Implementation Planning Tool Avaya utility to provide recommendations for Avaya IQ and database server hardware requirements. 29 30 31 32 2. CONFIGURING IQ DATABASE SERVER Oracle 10g and 11g are the database management systems for Avaya IQ. Oracle is installed on a separate server. Avaya IQ 5.1 currently supports Oracle version 10.2.0.4 and 11.1.0.7.0. The Avaya IQ Interoperability document is a living document and stays current with 3 rd party product support. This
Avaya IQ Database Server Page 4 of 11 Issue 4.0 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 document should be consulted at support.avaya.com to assure proper Oracle version support and associated patch set. It is the customer s responsibility to work with Oracle or the vendor they are purchasing the Oracle software/licenses from for the correct quantity and recommendation of type of licenses. The Oracle website www.oracle.com should be referenced to determine number and type of Oracle licenses required for the Avaya IQ database server. This section provides guidelines for configuring the IQ database server. A copy of the IPT Customer Report output form should be available. It is required for determining Busy Hour Contacts (BHC), number of Report Users, and the database server number of hard drives and number of Disks (Database tablespaces). Oracle 32-bit and 64-bit databases are supported. For Oracle 11 release 1, the OS and Oracle database architecture must be the same (32-bit Oracle on 32-bit OS or 64-bit Oracle on 64-bit OS). 64-bit OS and 64-bit Oracle 64-bit are recommended, especially if your number of Report Users is greater than 100. The Oracle 64-bit database provides scalability to support more users and process larger data volume. Large data and transaction processing takes place in memory resulting in efficient query processing. 2.1 PHYSICAL DISK REQUIREMENTS I/O contention must be avoided for efficient database performance. System (Operating System and Oracle software) and Oracle managed files (undo, temp, redo, archive log, etc) must be separated from Oracle data storage files (datafiles). This section provides recommendations for this distribution. Oracle has the Automatic Storage Option (Automatic Storage Option) option which simplifies disk management and optimizes data layout for I/O performance. Avaya has not tested this configuration. Consult your DBA if this option is a consideration. 2.1.1 Data Storage Disk Requirements This section addresses disk requirements for IQ real-time and historical data (datafiles). Disk requirements are determined by the performance and storage needs of the database server. The IQ IPT tool provides total database space requirements and recommendations for the number database disks for IQ data storage. The number of physical disks is critical for database I/O performance. RAID striped configurations (5, 10) are proven I/O performers. Oracle recommends RAID 10 for data storage. Avaya agrees with this recommendation, especially for large configurations. In a striped RAID disk configuration (5 or 10), the more disk spindles in the storage configuration, the faster the I/O access. This efficiency results from concurrent access across the stripe set. The IQ database I/O rate will suffer in the absence of enough disk spindles. As a result, IQ report access and traffic processing performance will be impacted. The primary IQ impact is: 1) buffering of event data resulting in latency of data availability to reports; and 2) delayed report refresh rates. Be certain to provide the number of database tablespace disks as recommended in the Customer Implementation Report (Disks (Database tablespaces)). Note, Oracle managed files (control files, redo and archive logs, Oracle SYSTEM, SYSAUX, TEMP, USERS, UNDO tablespaces) should not reside on the RAID storage disk.
Avaya IQ Database Server Page 5 of 11 Issue 4.0 76 77 78 79 80 81 82 83 84 2.1.2 System and Oracle Managed Files This section is a supplement to the IPT Customer Report database server recommendation. The location of the Operating System, Oracle software, and Oracle managed files depends on the size of the configuration. The following system and file locations are based on the number of data storage disks (as determined in Customer Report, Database Server hardware and software requirements: Disks (Database tablespaces)). 3 options are provided for the large disk configuration. # of Data Storage System and Oracle Managed File configuration Disks Disk Configuration Files to Locate Less than 8 RAID1: 1 mirrored (2 physical disks 146 GB) OS, Oracle 10g, Oracle SYSTEM, SYSAUX, USERS, TEMP, UNDO tablespaces; control files, redo 8 or greater Configuration (option 1) RAID 1: at least 3 mirrored (6 physical disks 146 GB) and archive logs OS, Oracle 10g, Oracle SYSTEM, SYSAUX, USERS, TEMP, UNDO tablespaces; control files, redo and archive logs 8 or greater Configuration (option 2) 8 or greater Configuration (option 3) RAID1: at least 1 mirrored (2 physical disks 146 GB) RAID10: 1 mirrored (4 physical disks 73 GB) RAID1: at least 1 mirrored (2 physical disks 146 GB) OS, Oracle 10g, Oracle SYSTEM, SYSAUX, USERS tablespaces, Oracle control files UNDO, TEMP tablespaces, redo logs (1, 2, and 3), archive logs OS, Oracle 10g, Oracle SYSTEM, SYSAUX, USERS Tablespaces, Oracle control files 85 4 separate disks with the files distributed across the disks as designated by d2,d3,d4,d5 UNDO, TEMP tablespaces -d2 redo logs A (1, 2, and 3) - d3 redo logs B (1, 2 and 3) d4 archive logs d5 86 87 88 89 2.1.2.1 Oracle Redo Log File Sizing and Associated Archive Log Destination Redo log file management is important to overall databases throughput. Redo log files are typically part of a larger database backup and restore strategy. Redo log sizing impacts the efficiency of the Oracle LGWR and ARCH background processes. Efficient redo log management has not been observed in our
Avaya IQ Database Server Page 6 of 11 Issue 4.0 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 lab configurations with properly sized and multiplexed redo log files (3 logfile groups each with a single file sized as recommended). A guideline is provided for redo logfile size based on total BHC 1. Redo log file size guideline total BHC < 50K total BHC 50K-150K total BHC 150K-300K 1GB 2GB 3GB The size can be adjusted to accommodate backup media sizing efficiencies (for backup of archive log files). Redo log size modeling does not account for extraneous operations such as data imports. If regular operations of this type take place, additional redo log sizing considerations may be necessary (consult Oracle Database Administrator's Guide). The archive log destination (arch_log_dest_1) must have sufficient disk space to hold the archive log files between backups. Archive log backup frequency can be increased to facilitate archive log destination space management. With the recommended sizing, typically 2 3 archive log files are generated during busy hour. If historical data purge is active (and there is data to delete), 8 12 archive log files are generated an hour for the duration of the purge. 2.1.2.2 Oracle Undo and TEMP Tablespace Sizing The following are recommendations for Undo and TEMP tablespaces. 1. use Oracle Automatic Undo Management; 2. enable undo tablespace AUTOEXTEND option. If you have disk capacity concerns and choose to set MAXSIZE, periodically monitor undo size to assure maximum is not exceeded or approached. 3. The following recommendations exist for Oracle undo and TEMP tablespace sizing. Periodic index rebuilds are recommended so a likely configuration is: undo 25 GB, TEMP 15/20 GB. IQ Condition Recommended undo tablespace size Recommended TEMP tablespace size 5 GB 5 GB Data purge active 15 GB 15 GB Index Rebuild active 25 GB 15 GB Upgrade to 5.1 2 30 GB 15 GB Administered Agents > 5000 One of the above 20 GB 3. ORACLE PROCEDURES FOR AVAYA IQ DATABASE A database backup and restore strategy should be formulated and implemented based on the Oracle Database Backup and Recovery document. The customer is responsible for Oracle backup and restore operations. All IQ tablespaces should be configured to auto-extend mode. Periodically verify the disks on which the tablespaces reside have sufficient capacity for tablespace growth. 1 Sum of BHC for all data sources. 2 Undo tablespace size can be reduced after upgrade.
Avaya IQ Database Server Page 7 of 11 Issue 4.0 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 Avaya IQ performs an hourly rebuild of IQ Real-time indexes to assure efficient data access. Weekly index rebuilds are recommended for all IQ Historical indexes. The index rebuilds can be implemented as preferred by the DBA. Avaya provides an Oracle procedure (hist_index_rebuild) to rebuild indexes. The index rebuilds can be distributed by executing hist_index_rebuild for specific tables. In Oracle Enterprise Manager, by default, Manage Optimizer Statistics schedules a job to update statistics nightly. Do not disable this job; updates statistics are critical for efficient report queries. A qualified DBA can perform database tuning activities in accordance with the Oracle Performance Tuning Guide. 4. KNOWN IQ ISSUES RELATED TO DATABASE SERVER A security audit of the IQ database after IQ installation results in policy violations for PUBLIC EXECUTE privilege on the following packages: DBMS_RANDOM, UTL_FILE, UTL_HTTP, UTL_SMTP, and UTL_TCP. A development MR currently exists for this item. 5. ORACLE DOCUMENT REFERENCES 140 Oracle document Oracle Database 2 Day DBA 11g Release 1 (11.1) Part Number B28301-03 Oracle Enterprise Manager Concepts 11g Release 11.1.0.1 Part Number E11982-02 Oracle Database PL/SQL Packages and Types Reference 11g Release 1 (11.1) Part Number B28419-03 Oracle Database Reference 11g Release 1 (11.1) Part Number B28320-03 Oracle Database Performance Tuning Guide 11g Release 1 (11.1) Part Number B28274-02 Oracle Database Administrator's Guide 11g Release 1 (11.1) Part Number B28310-04 Oracle Database Backup and Recovery Reference 11g Release 1 (11.1) Part Number B28273-03 Oracle Database Backup and Recovery User s Guide 11g Release 1 (11.1) Part Number B28270-03 Oracle Database Administrator s Reference 11g Release 1 (11.1) for Linux and Unix-Based Operating Systems Part Number B32009-07 Oracle Database Installation Guide 11g Release 1 (11.1) for Linux Part Number B32002-09 What information it contains Manual for information on monitoring, diagnosing, and tuning the database Manual for information about monitoring and diagnostic tools available with Oracle Enterprise Manager Manual for information about monitoring and diagnostic PL/SQL packages provided by Oracle database software Manual for information about database performance related initialization parameters. Manual for general database performance tuning guidelines. Manual for information on administration Oracle database. Manual for database administrators who perform backup and recovery of an Oracle database server using Recovery Manager (RMAN). Manual for database administrators on applying backup and recovery strategies. Manual for installing and configuring Oracle 11.1. Manual for installing and configuring Oracle 11.1 on Linux x86-64.
Avaya IQ Database Server Page 8 of 11 Issue 4.0 141 142 143 144 145 146 147 6. APPENDIX A MULTI-HOST DATABASE SERVER Sample Oracle database server configuration for Avaya IQ 5.1 multi-host. This is an actual test configuration. This configuration supports up to 900 report users and 300K BHC traffic (up to 16 CMs/PCs with concurrent VP traffic). System Component Configuration Server Dell PowerEdge R900 OS 64 Bit Red Hat ES 5.2 (kernel: 2.6.18-92.e15) CPU 6 Intel (R) Xeon(R) E7450 2.40 GHz (quad core) Memory 64 GB Disks 6 x 73 GB 15K SAS disks (internal); 24 x 146 GB 15K SAS disks on two external host bus adapters, two 3G SAS and two 4G fiber. RAID controller PERC 6/E Database Oracle Enterprise Edition Database 10g R2 (10.2.0.4) 64 bit DB initialization parameters General Purpose Starter Database Template is used to create the DB Connection Mode: Dedicated Server Mode db_block_size 8K db_file_multiblock_read_count 16 filesystemio_options: none disk_asynch_io true processes 10000 sga_target: 2G sga_max_size 8G undo_management auto log_mode archivelog Disk Contents RAID1: 1 mirrored 73 GB disks Operating System Oracle Software Oracle Control, Trace, Logs Oracle SYSAUX tablespace Oracle SYSTEM tablespace Oracle USERS tablespace Disk2: 73 GB Oracle TEMP tablespace Oracle UNDO tablespace Disk3: 73 GB Oracle multiplexed (A) redo logs (3) Disk4: 73 GB Oracle multiplexed (B) redo logs (3) Disk5: 73 GB Archive Log files RAID10: 12 mirrored 146 GB disks All IQ tablespaces Configuation: Two hardware RAID10 LUNs. 6 mirrored 146 GB disks each Stripe-unit size: 256K Linux Volume Manager is configured to create one RAID0 volume to distribute I/O to the two LUNs OS parameters fs.aio-max-size = 1048576
Avaya IQ Database Server Page 9 of 11 Issue 4.0 148 149 150 151 152 153 154 7. APPENDIX B DUAL HOST DATABASE SERVER fs.file-max = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 1024 65000 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 262144 Sample Oracle database server for Avaya IQ 5.1 dual host. This is an actual test configuration. This configuration supports 400 report user with 130K BHC traffic (up to 8-CMs/PCs with concurrent VP traffic). System Component Configuration Server Avaya S8800 OS 64 Bit Red Hat ES 5.4 (kernel: 2.6.18-164.e15) CPU 4 x Intel (R) Xeon(R) X5570 2.93 GHz (quad core) Memory 32 GB (8 x 4 GB of 1333 Mhz, fully buffered DDR-3 RDIMMs) Disks 2 x 146.8 GB IBM 10K RPM SAS EXP3000 Disk Array: RAID10: 12 x 600 GB 15K SAS (3.4 TB storage capacity) RAID controller ServeRAID-MR10m Database Oracle Enterprise Edition Database 11g R1 (11.1.0.7.0) 64bit DB initialization parameters General Purpose Starter Database Template is used to create the DB Connection Mode: Dedicated Server Mode db_block_size 8K db_file_multiblock_read_count 16 filesystemio_options none disk_asynch_io true sga_target 2208M sga_max_size 2208M shared_pool_size 0 processes 10000 undo management auto OS parameters fs.file-max = 6553600 kernel.shmmax = 4000000000 kernel.shmall = 4294967296 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 1024 65000 net.core.rmem_default = 4194304 net.core.rmem_max = 4194304
Avaya IQ Database Server Page 10 of 11 Issue 4.0 net.core.wmem_default = 262144 net.core.wmem_max = 262144 155 156 The disk layout mapping 157 Disk RAID1: 1 mirrored 146 GB disks RAID10: EXP3000 Disk Array: 6 mirrored 600GB disks Contents Operating System Oracle Software Oracle Control, Trace, Logs Oracle TEMP tablespace Oracle SYSAUX tablespace Oracle SYSTEM tablespace Oracle UNDO tablespace Oracle USERS tablespace Oracle Redo logs (3x2048 MB) All IQ tablespaces 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 8. APPENDIX C REPORT MIX Avaya IQ system engineers profiled customer usage scenarios. A typical customer report usage model resulted from this investigation and documented a report mix to represent this model. The following is a summary of the report mix. Benchmarks (I/O rate, CPU occupancy, memory utilization, etc) were captured with various number of report users, executing the below report mixes with 130,000 contact traffic on the dual host configuration and 300,000 BHC contact traffic on the multi-host. Report Mix Ratio The profile report mix used is a combination of Real Time Dashboard (RTD), and Cognos based Real Time (RT) and Historical (Hist) reports. Report Profile Mix Table (RTD and Cognos Based RT/Hist Reports): User Type % of Number Report Reports Users Calc for 52 Rpt Users Type Supervisor 75% 35 RTD B, D, G, H (2 Activities) 25% 12 RTD A, B, C, D, G, H (3 Activities) 50% 5 Hist I, J, K 50% 5 Hist I, L, M Directors 75% 3 RTD F, G, H (2 Activities) 25% 1 RTD G, H (2 Activities) RT O, P Operation Managers 100% 1 Hist RTD RT RTD Reports: A. Agent Behavior ab B. Agent Contact (Agent Status) ac C. Agent Performance ap D. Agent Utilization au E. Recent Trend rt (not utilized) F. Queue Performance qp G. Queue Status qs H. Stream Status ss N D, G, H (2 Activities) O, P
Avaya IQ Database Server Page 11 of 11 Issue 4.0 179 180 181 182 183 184 185 186 187 188 189 190 191 192 Historical Reports: I. Work Handling Agent Behaviors Summary (totals) J. Short Contacts Details K. Short Contacts Behaviour Trend L. Short Wrap-up Details M. Short Wrap-up Trend N. Queue Service Level Trend Old Style RT Reports: O. Queue Performance P. Stream Status