VERITAS Database Edition for Oracle on HP-UX. Performance Report

Size: px

Start display at page:

Download "VERITAS Database Edition for Oracle on HP-UX. Performance Report"

Lawrence Heath
6 years ago
Views:

1 VERITAS Database Edition for Oracle on HP-UX Performance Report A P R I L

2 Executive Summary Until recently, Oracle administrators had a difficult choice to make between performance and manageability for their database files. They either sacrificed manageability for the performance advantages of raw devices or settled for the degraded performance that came along with the ease of use of file systems. The introduction of VERITAS Database Edition for Oracle Quick I/O technology provides a simple solution to this problem, providing raw device performance with file system manageability. The purpose of this study was to compare the performance of raw devices and Quick I/O as well as examine the tuning options best suited for Online Transaction Processing (OLTP) workloads. VERITAS Database Edition for Oracle is an integrated suite of industry-leading VERITAS online storage management technologies, engineered specifically to improve performance, availability, and manageability of Oracle database servers running on HP-UX. The Database Edition is the industry s only integrated database management solution that combines the power of a highly scalable, high performance file system that delivers raw device performance through an enhanced technology with breakthrough backup and recovery technologies. The introduction of Block-Level Incremental Backup decreases the time to back up changed data significantly. The performance of Block-Level Incremental Backup is examined in another white paper: VERITAS NetBackup Technical White Paper - BLI Backup compared with Oracle RMAN Backup. Introduction This document describes the performance of VERITAS Database Edition 2.1 for Oracle and the impact of its Quick I/O technology, as measured by an OLTP workload. Typical OLTP systems involve processing simple to moderately complex transactions with multiple updates to the database. The benchmark used for this performance comparison was derived from the commonly known TPC-C benchmark. This document studies the impact of different memory and other system configurations. This release of the Database Edition supports HP-UX 11.0 (both 32-bit and 64-bit) and Oracle7, Oracle8, and Oracle8i. The Database Edition comprises the following components: VERITAS File System TM (VxFS ) with Quick I/O and Storage Checkpoint technology VERITAS NetBackup Block-Level Incremental Backup Extension for Oracle VERITAS VxDBA utility VERITAS Volume Manager TM (VxVM ) VERITAS Database Edition for Oracle performance is also compared to the buffered and direct mount options of VERITAS File System (VxFS), which function similar to the journaling file system (JFS) shipped with HP-UX and the OnlineJFS add-on available from HP as HP OEMs this technology from VERITAS. Performance is also compared to a raw striped volume (RAID-0) configuration. VERITAS Quick I/O provides a raw device interface to regular files created under VxFS. The advantages of using Quick I/O are as follows: Quick I/O bypasses the file system s buffering for reads and writes, thereby eliminating the double-buffering and doublecopying performance penalties. Quick I/O also results in less use of system memory, so more memory can be allocated to the database buffers. Quick I/O bypasses the file system s locking layer, eliminating the additional locking overhead associated with traditional file systems. Normally, the file system s locking mechanism maintains a single-writer-lock on files, which ensures that only one process writes to a file at a time to guarantee data integrity. Since databases typically maintain their own locks at either row or page level, this is an unnecessary overhead that serializes write operations, slowing database performance. This overhead is eliminated when databases are built on VERITAS File Systems using Quick I/O. Quick I/O supports kernel asynchronous I/O, which is inherently faster and more efficient than synchronous I/O. Other than with Quick I/O, async I/O support is only available with raw devices. 1

3 About the Benchmark The OLTP workload used for performance comparison was based on the industry standard TPC-C benchmark from the Transaction Processing Council. The TPC-C benchmark is an established metric used to compare hardware and software platforms. This benchmark comprises a mixture of read-only and update intensive transactions that simulate a warehouse supplier environment. (Details on this benchmark can be obtained from the Transaction Processing Council s web page at However, the goal for this report was to compare the impact of the underlying storage I/O configuration and not to achieve the best possible numbers. Accordingly, these numbers are only shown as a reference. The size of the database used for the test was 25 GB with a total of 59 Oracle datafiles, including redo logs, indices, rollback segments, and temporary and user tablespaces. The size of the database was that of a fully scaled TPC-C database with a scale factor of 100 warehouses. Test Configuration The tests were run on an N Class 4360 HP9000 computer system with MHZ CPUs and 4GB of physical memory. The N- Class machine had a total of 42, 9GB disk drives. Of the 42 disk drives, two were internal SCSI LVD drives the remainder being fibre channel devices housed in 4 HP 1010D Enclosures. Each 1010D was connected to the N Class through a HP Fibre Channel Mass Storage adapter, 4 fibre channel adapters total. The two internal drives were devoted to operating system specific files systems such as root, /stand, /opt, and swap space. Additional swap space was configured on a fibre channel drive. Total swap space was approximately 3GB. Oracle along with the TPC-C binaries, executable scripts and configuration files were also placed on the internal LVD drives. A total of 13 fibre channel drives were used to build the TPC-C database. Twelve drives split between two HP Fibre Channel Mass Storage adapters housed the datafiles for the warehouse. The Oracle log files were placed on a separate fibre channel disk and controller. Both raw and file system tests were implemented on 12-way striped volumes. Asynchronous I/O was employed for the raw and QIO benchmark runs. For all Direct I/O and Buffered I/O tests the following Oracle parameters were utilized: disk_asynch_io dbwr_io_slaves For the Direct I/O and Buffered I/O runs disk_asynch_io was set to false. Setting this value to false bypasses any asynchronous I/O facilities enabled within the operating system. Setting disk_asynch_io to false and an appropriate value for dbwr_io_slaves, asynchronous I/O can be emulated by having multiple dbwr_io_slaves. The dbwr_io_slaves parameter was set to a value of 20 for all Buffered and Direct I/O configurations. The following software releases were used in testing: Oracle (64-bit) HP-UX 11.0 (64-bit) VERITAS Database Edition 2.1 for Oracle consisting of the following components: VxFS with Quick I/O and Storage Checkpoint technology NetBackup BLI Backup Extension 3.2 for Oracle VxDBA Utility VxVM 3.0 The following I/O configurations were tested: Raw I/O uses the VxVM raw volumes directly VxFS Quick I/O uses Quick I/O feature of VxFS Buffered I/O and Direct I/O used VxFS (VxFS with Buffered I/O is comparable to JFS, while VxFS with Direct I/O is comparable to OnlineJFS.) Raw I/O traditionally delivers the best performance because no OS level locking or buffering is performed in this mode. Additionally, raw I/O takes advantage of asynchronous I/O, which boosts performance by enabling the system to handle multiple I/O requests simultaneously. However, this configuration has a major shortcoming: managing databases built directly on raw volumes involves a significant load on system or database administrators. The vast array of tools that are available for monitoring and managing file systems cannot be used in this configuration, which may result in human errors from which it may be difficult to recover. 2

4 Quick I/O allows regular files built on VERITAS File System to be accessed as raw devices, bypassing the double-buffering and double-locking associated with databases built on top of regular file systems. Although the double-buffering is avoided when traditional file systems are mounted in the Direct I/O mode, this mode still includes the file system locking to ensure that only one process/thread issues writes to a file at any point in time. Since databases maintain their own locks to ensure data integrity, this extra locking imposes an unnecessary serialization of I/Os and impacts performance. The double-locking also imposes additional CPU overhead. Quick I/O bypasses file system locking and lets the database server control data access. The focus of this brief is to demonstrate and quantify OLTP throughput of the Database Edition using VxVM, in a HP-UX environment. The primary variable as shown in the following charts is the Oracle SGA size, which ranges from 50MB to 700MB Overview and Analysis of Results The performance metric used for this comparison is the throughput rate measured by the number of transactions completed per minute (TPM). The transaction mix in this benchmark represents the processing of an order as it is entered, paid for, checked, and delivered and follows the model of a complete business activity. Therefore, the TPM metric is considered a measure of business throughput. The TPM achieved for each of the I/O configurations is shown. During the test, sar data (system activity reporter data), Oracle statistics, and Quick I/O statistics (using qiostat) were gathered in addition to the throughput numbers. 3

5 The first chart and table show OLTP throughput for the four I/O configurations. As can be seen, raw and Quick I/O throughput remain essentially equal across all SGA sizes with less than four percent variance when compared. At smaller SGA sizes the Buffered I/O option appears to perform well but remains flat when compared across the SGA sizes. The Direct I/O option, although under-performing at smaller SGA sizes when compared to Buffered I/O, shows a linear throughput increase as more memory is allocated to the Oracle SGA. This result suggests Direct I/O scales as the Oracle SGA size increases, similar to raw and Quick I/O. Chart 1 OLTP Throughput 64-Bit HP-UX/64-Bit Oracle tpmc SGA Size (MB) Raw QIO Buffered Direct Throughput 64-bit Oracle 64-bit HP-UX I/O Configuration Oracle SGA Size 50MB 100MB 200MB 300MB 400MB 500MB 600MB 700MB Raw QIO Buffered Direct

6 The following chart and table depicts the CPU utilization for the four I/O configurations. The Buffered I/O CPU utilization curve shows there is very little increase in efficiency as the SGA size increases. As echoed in the throughput graph, on the following chart buffered I/O enjoys an efficiency edge for smaller SGA sizes, but is not maintained as SGA sizes increase when contrasted to Direct I/O. Quick I/O is capable of more efficient I/O operations, which is most apparent for the larger SGA sizes where raw CPU utilization is 13% higher for the 600 MB SGA configuration. The more efficient I/O operations performed by Quick I/O can be attributed to the fact it is capable of utilizing multiple worker threads while raw I/O cannot. Chart 2 CPU Utilization 64-Bit HP-UX/64-bit Oracle Utilization (%) SGA Size (MB) Raw QIO Buffered Direct CPU Utilization (%) 64-bit Oracle 64-bit HP-UX I/O Configuration Oracle SGA Size 50MB 100MB 200MB 300MB 400MB 500MB 600MB 700MB Raw QIO Buffered Direct

7 Useful Tuning Tips The following are some performance tuning guidelines that may be helpful when using VERITAS Database Edition to configure an Oracle database for OLTP workloads: Tip No. 1: Use VxFS Quick I/O for Databases VxFS in its default mode supports buffered I/O as does JFS, in which all I/O incurs the double copying and double buffering penalties. In addition, the UNIX semantics require that only one writer have the exclusive right to update a file, which forces other read and write operations on the same file to wait. Therefore, databases on file systems often perform slower than on raw devices. Quick I/O has I/O characteristics similar to raw I/O (a higher degree of read/write parallelism with less CPU overhead). Therefore, using Quick I/O provides a significant advantage in OLTP scenarios. Tip No. 2: Increase Oracle Tunable Parameter DB_BLOCK_BUFFERS The size of the Oracle buffer cache is determined by the setting of the INIT.ORA variable DB_BLOCK_BUFFERS. If the Oracle database files were built on top of regular file systems, this parameter would have to be set after taking into consideration the amount of system memory consumed by the file system buffering. Since Quick I/O bypasses the buffering at the file system level, this tunable could potentially be set higher, thereby increasing the size of the Oracle cache. Increasing the size of the Oracle cache typically results in improved performance. Tip No. 3: Use Striped Volumes for Performance In general, the best way to achieve the optimal throughput is by balancing the I/O across as many disks as possible. This balancing can be achieved by creating the database on a file system on a striped (RAID-0) volume using all the available disks. However, since redo logs tend to be accessed sequentially, one may see better performance by putting the redo logs on a separate file system, using a simple volume on a dedicated disk. If database reliability is important, mirror (using RAID-1) at least the volume used by the redo logs. Tip No. 4: Use async I/O for Quick I/O and sync I/O for buffered and direct I/O Async I/O provides increased throughput and significant decrease in CPU utilization for Quick I/O. When using Quick I/O, in the init.ora file ensure disk_asynch_io = true. Ensure that the async I/O parameter in Oracle is not turned on if using buffered I/O or direct I/O. In the init.ora file ensure disk_asynch_io = false. Tip No. 5: Use 2KB Oracle block size The use of larger block sizes appears to have a significant effect on degrading database throughput. Summary The OLTP performance measurements illustrate that the Quick I/O feature enables the VERITAS Database Edition for Oracle to have equal performance compared to raw partition configurations and significant performance improvements over traditional file systems. It achieves this performance while delivering the manageability only attainable by running Oracle on file systems. Using the OLTP benchmark, VERITAS Database Edition for Oracle outperforms traditional file system Buffered I/O and Direct I/O configurations by a large margin. In addition, Quick I/O files can be managed as regular files and are backed by the reliability of VERITAS File System. With its log-based implementation, VxFS allows fast recovery after a system crash, which is not available in the HFS environment. VxFS also offers online administration features such as online resizing and defragmentation for higher availability and performance. VERITAS Database Edition for Oracle integrates the above trusted storage products with Oracle-specific agents and extensions to create a software solution that enhances database performance, manageability, and availability. VERITAS Database Edition provides immediate and significant performance improvements for Oracle databases, while it simplifies the DBA s job and reduces administrative down time. 6

8 Appendix A The first two pages of Appendix A show the system file for building the HP-UX kernel used in the TPC-C benchmark. *************************************** * $Source: /usr/local/kcs/sys.rose_800/filesets.info/core-krn/rcs/generic,v $ * $Revision: $ $Author: kcs $ * $State: Exp $ $Locker: CRT $ * $Date: 97/07/12 21:51:58 $ * *************************************** * Additional drivers required in every machine-type to create a complete * system file during cold install. This list is every driver that the * master.d/ files do not force on the system or is not identifiable by * ioscan. * Other CPU-type specific files can exist for their special cases. * see create_sysfile (1m). *************************************** * * Drivers/Subsystems CentIf CharDrv asyncdsk asyncdsk_included sba lba btlan3 c720 ccio sctl sdisk asio0 fct1 fct1_cntl fcp PCItoPCI cdfs nfs_core nfs_client nfs_server dlpi inet uipc tun telm tels netdiag1 nms btlan6 fcgsc_lan fc_arp fct1_fcp fcpmux lvm lv hpstreams clone strlog sad echo sc timod tirdwr pipedev pipemod ffs ldterm 7

9 ptem pts ptm pckt diag0 diag2 dmem dev_config btlan sapic diag1 nfsm token_arp maclan autofsc stape tape2 target vxfsvrts vxportalvrts fdd vxvm vxdmp vol vols STRMSGSZ num_tachyon_adapters 5 max_fcp_reqs 512 dump lvol nstrpty 60 eqmemsize 512 maxswapchunks max_thread_proc maxuprc 1024 nproc 2512 nflocks 4500 nfile 5000 ninode 5000 maxssiz 0X maxdsiz 0X maxfiles 512 maxfiles_lim 512 max_async_ports 1024 semmni 2048 semmns 2048 shmmax 0X shmmni 800 shmseg 120 swapmem_on 0 8

10 The following shows the Oracle parameters for the TPC-C tests. # # $Header: p_run.ora /05/02 19:22:28 plai Generic<base> $ Copyr (c) 1993 Oracle # #==================================================================+ # Copyright (c) 1996 Oracle Corp, Redwood Shores, CA # OPEN SYSTEMS PERFORMANCE GROUP # All Rights Reserved #==================================================================+ # FILENAME # p_run.ora # DESCRIPTION # Oracle parameter file for running TPC-C. #================================================================== control_files #disk_asynch_io #tape_asynch_io = (/tpcc_disks/cntrl1) = TRUE = TRUE disk_asynch_io = FALSE dbwr_io_slaves = 20 parallel_max_servers = 30 recovery_parallelism = 10 db_name = tpcc db_files = 200 db_file_multiblock_read_count = 32 db_block_buffers = # Set to 700MB dml_locks = 500 hash_join_enabled = FALSE log_archive_start = FALSE log_checkpoint_interval = log_checkpoints_to_alert = TRUE log_buffer = gc_releasable_locks = 0 max_rollback_segments = 220 open_cursors = 200 processes = 100 sessions = 400 transactions = 400 distributed_transactions = 0 transactions_per_rollback_segment = 1 rollback_segments = (t1,t2,t3,t4,t5,t6,t7,t8,t9,t10,t11,t12,t13,t14,t15,t16,t17,t18,t19,t20,t21,t22,t23,t24,t25,t26,t 27,t28,t29,t30, t31,t32,t33,t34,t35,t36,t37,t38,t39,t40,t41,t42,t43,t44,t45,t46,t47,t48,t49,t50) shared_pool_size = 30M cursor_space_for_time = TRUE audit_trail = FALSE 9

11 Appendix B The results of the database performance measurements are summarized in Appendix B. Most of the columns below are selfexplanatory, but some of the comparisons are not. The fom is a figure of merit derived by dividing the transaction rate TPM by the CPU utilization CPU %. A larger number represents a more efficient option. The last three columns compare the two sets of results by comparing thruput (ratios greater than one represent better throughput), CPU % ratios (where less than one is a better result), and finally a comparison of the figures of merit for the two compared results. 64-bit HP-UX 64-bit Oracle file system blocksize=1 KB Veritas Volume Manager Oracle blocksize=2 KB compared type clone SGA size* #run TPM cpu % to raw fom** Raw 50 MB Raw 100 MB Raw 200 MB Raw 300 MB Raw 400 MB Raw 500 MB Raw 600 MB Raw 700 MB QIO 50 MB QIO 100 MB QIO 200 MB QIO 300 MB QIO 400 MB QIO 500 MB QIO 600 MB QIO 700 MB BIO 50 MB BIO 100 MB BIO 200 MB BIO 300 MB BIO 400 MB BIO 500 MB BIO 600 MB BIO 700 MB DIO 50 MB DIO 100 MB DIO 200 MB DIO 300 MB DIO 400 MB DIO 500 MB DIO 600 MB DIO 700 MB

12 Corporate Headquarters 1600 Plymouth Street Mountain View, CA North American Sales Headquarters 400 International Parkway Heathrow, FL or Fax Global Locations United Kingdom or 44-(0) (0) Fax France Fax Germany 49-(0) (0) Fax South Africa Fax Australia BACKUP Fax Hong Kong Fax Japan Fax Malaysia Fax Singapore Fax China Fax Electronic communication World Wide Web: VERITAS Software Corporation. All rights reserved. VERITAS is a registered trademark of VERITAS Software Corporation in the US and other countries. The VERITAS logo, Business Without Interruption, VERITAS Database Edition, VERITAS NetBackup, VERITAS File System, VERITAS File System and VERITAS Volume Manager are trademarks of VERITAS Software Corporation in the US and other countries. Other product names mentioned herein may be trademarks and/or registered trademarks of their respective companies. Specifications and product offerings subject to change without notice. Printed in USA. April xxx

Storage Edition 2.0. for Oracle. Performance Report

Storage Edition 2.0. for Oracle. Performance Report Storage Edition 2.0 for Oracle Performance Report O C T O B E R 1 9 9 9 Table of Contents Executive Summary...1 Introduction...1 About the Benchmark...2 Test Configuration...2 Overview and Analysis of