Storage Edition 2.0. for Oracle. Performance Report

Size: px

Start display at page:

Download "Storage Edition 2.0. for Oracle. Performance Report"

Stephen Scott
6 years ago
Views:

1 Storage Edition 2.0 for Oracle Performance Report O C T O B E R

2 Table of Contents Executive Summary...1 Introduction...1 About the Benchmark...2 Test Configuration...2 Overview and Analysis of Results...3 Useful Tuning Tips...6 Summary...6 Appendix A...7 Appendix B...9

3 Executive Summary Until recently, Oracle administrators had a difficult choice to make between performance and manageability for their database files. They either sacrificed manageability for the performance advantages of raw devices or settled for the degraded performance that came along with the ease of use of file systems. The introduction of VERITAS Storage Edition f or Oracle Quick I/O technology provides a simple solution to this problem, providing raw device performance with file system manageability. The purpose of this study was to compare the performance of raw devices and Quick I/O as well as examine the tuning options best suited for OLTP workloads. VERITAS Storage Edition for Oracle is an integrated suite of industry-leading VERITAS storage management technologies, engineered specifically to improve performance, availability, and manageability of Oracle database servers running on HP-UX. The Storage Edition is the industry s only integrated storage management solution that combines the power of a highly scalable, high performance file system that delivers raw device performance with breakthrough backup and recovery technologies. The introduction of Block-level Incremental Backup decreases the time to back up changed data significantly. The performance of Block-level Incremental Backup is examined in another white paper: VERITAS NetBackup Technical White Paper - BLI Backup compared with Oracle RMAN Backup. Introduction This document describes the performance of VERITAS Storage Edition 2.0 for Oracle and the impact of its Quick I/O technology, as measured by an Online Transaction Processing (OLTP) workload. Typical OLTP systems involve processing simple to moderately complex transactions with multiple updates to the database. The benchmark used for this performance comparison was derived from the commonly known TPC-C benchmark. This document studies the impact of different memory and other system configurations. This release of the Storage Edition supports HP-UX 11.0 (both 32-bit and 64-bit) and Oracle7 and Oracle8. The Storage Edition comprises the following components: VERITAS File System TM (VxFS ) with Quick I/O VERITAS NetBackup Block-Level Incremental Backup Extension for Oracle VERITAS VxDBA utility VERITAS Storage Edition for Oracle performance is also compared to the buffered and direct mount options of VERITAS File System (VxFS), which function similar to the journaling file system (JFS) shipped with HP-UX and the OnlineJFS add-on available from HP, as well as a raw striped volume (RAID-0) configuration. VERITAS Quick I/O provides a raw device interface to regular files created under VxFS. The advantages of using Quick I/O are as follows: Quick I/O bypasses the file system s buffering for reads and writes, thereby eliminating the double-buffering and doublecopying performance penalties. Quick I/O also results in less use of system memory, so more memory can be allocated to the database buffers. Quick I/O bypasses the file system s locking layer, eliminating the additional locking overhead associated with traditional file systems. Normally, the file system s locking mechanism maintains a single-writer-lock on files, which ensures that only one process writes to a file at a time to guarantee data integrity. Since databases typically maintain their own locks at either row or page level, this is an unnecessary overhead that serializes write operations, slowing database performance. This overhead is eliminated when databases are built on VERITAS File Systems using Quick I/O. Quick I/O supports kernel asynchronous I/O, which is inherently faster and more efficient than synchronous I/O. Other than with Quick I/O, async I/O support is only available with raw devices. 1

4 About the Benchmark The OLTP workload used for performance comparison was based on the industry standard TPC-C benchmark from the Transaction Processing Council. The TPC-C benchmark is an established metric used to compare hardware and software platforms. This benchmark comprises a mixture of read-only and update intensive transactions that simulate a warehouse supplier environment. (Details on this benchmark can be obtained from the Transaction Processing Council s web page at However, the goal for this report was to compare the impact of the underlying storage I/O configuration and not to achieve the best possible numbers. Accordingly, these numbers are only shown as a reference. The size of the database used for the test was 25 GB with a total of 59 Oracle datafiles, including redo logs, indices, rollback segments, and temporary and user tablespaces. The size of the database was that of a fully scaled TPC-C database with a scale factor of 100 warehouses. Test Configuration Tests were run on two K-class HP9000 computer systems. Each had 1GB of system memory and 14 disk drives: twelve of which were used for the databases, one for root/boot/swap and benchmark work spaces, and the last was used for the log. The log disk and root disk were internal to the systems and the data disks were housed in a rack with two fast/wide/differential controllers. The computer systems were a 4-way K420 used for the 32-bit tests and a 4-way K570 used for the 64-bit tests. The K420 CPUs run at 120MHz and the K570 CPUs run at 200 MHz, consequently comparisons of 32-bit and 64-bit operating systems and database systems are not possible. The following software releases were used in testing: Oracle (both 32-bit and 64-bit) HP-UX 11.0 (both 32-bit and 64-bit) VERITAS Storage Edition 2.0 for Oracle consisting of the following components: VxFS with Quick I/O NetBackup BLI Backup Extension 3.2 for Oracle VxDBA utility HP-UX logical volume manager (LVM) was used to stripe the twelve data drives The following I/O configurations were tested: Raw I/O uses the LVM raw volumes directly VxFS Quick I/O uses Quick I/O feature of VxFS Buffered I/O and direct I/O used VxFS (VxFS with buffered I/O is comparable to JFS, while VxFS with direct I/O is comparable to OnlineJFS) Raw I/O traditionally delivers the best performance because no locking or buffering is performed in this mode. Additionally, raw I/O takes advantage of asynchronous I/O, which boosts performance by enabling the system to handle multiple I/O requests simultaneously. However, this configuration has a major shortcoming: managing databases built directly on raw volumes involves a significant load on system or database administrators. The vast array of tools that are available for monitoring and managing file systems cannot be used in this configuration, which may result in human errors from which it may be difficult to recover. Quick I/O allows regular files built on VERITAS File System to be accessed as raw devices, bypassing the double-buffering and double-locking associated with databases built on top of regular file systems. Although the double-buffering is avoided when traditional file systems are mounted in the direct I/O mode, this mode still includes the file system locking to ensure that only one process/thread issues writes to a file at any point in time. Since databases maintain their own locks to ensure data integrity, this extra locking imposes an unnecessary serialization of I/Os and impacts performance. The doublelocking also imposes additional CPU overhead. Quick I/O bypasses file system locking and lets the database server control data access. The variables measured in this study were the following: File system block size Oracle block size 2

5 Async I/O versus sync I/O 32-bit and 64-bit Oracle 32-bit and 64-bit HP-UX The five tables in the appendix summarize these results. The tests chosen from the many that are possible were meant to make the following comparisons: Comparisons of raw, Quick I/O, buffered I/O, and direct I/O. Effect on throughput of file system block size Effect on throughput of Oracle block size Effect on throughput and CPU utilization of the async I/O option versus the sync option Overview and Analysis of Results In order to synopsize the plethora of data in the tables in the appendix, we have plotted four charts that point out the most interesting performance observations for the Storage Edition. The performance metric used for this comparison is the throughput rate measured by the number of transactions completed per minute (TPM). The transaction mix in this benchmark represents the processing of an order as it is entered, paid for, checked, and delivered and follows the model of a complete business activity. Therefore, the TPM metric is considered a measure of business throughput. The TPM achieved for each of the I/O configurations is shown. Tests were run on six database configurations, each with a different Oracle buffer cache size as shown. During the test, sar data (system activity reporter data), Oracle statistics, and Quick I/O statistics (using qiostat) were gathered in addition to the throughput numbers. The first chart compares a 32-bit HP-UX/Oracle combination using a 1KB file system block size and a 2KB Oracle block size with async I/O used for raw and Quick I/O configurations. Two conclusions can be made here: Raw and Quick I/O throughput are essentially equal for all SGA sizes. Using Quick I/O delivers performance up to 168% faster than traditional file systems using buffered I/O and up to 88% faster performance than those using direct I/O. OLTP Throughput for 32-bit HP-UX and 32-bit Oracle raw QIO BIO DIO Transactions per minute SGA Size-MB Chart 1 3

6 Chart 2 shows the results for 64-bit HP-UX and 64-bit Oracle. As in the previous graph, Quick I/O performance is virtually equivalent to that of raw devices. In this scenario, using Quick I/O delivers performance up to 188% faster than traditional file systems using buffered I/O and up to 101% faster performance than file systems using direct I/O. Chart 2 OLTP Throughput 64-bit HP-UX/64-bit Oracle Transactions per Minute raw QIO BIO DIO SGA Size-MB Chart 3 illustrates the differences in throughput experienced when turning on async I/O parameter in the init.ora file. Normally, async I/O cannot be used with traditional file systems; however, setting the async flag in Oracle while running the database on either buffered I/O or direct I/O file systems causes significant performance degradation. Chart OLTP Throughput for Traditional File Systems using Async I/O vs. Sync I/O with 64-bit HP-UX/64-bit Oracle BIO async BIO sync DIO async DIO sync Transactions per Minute SGA Size-MB 4

7 Chart 4 shows that 2KB is a much better choice of database block size for OLTP workloads than 8KB Oracle block size as it significantly speeds performance. Chart 4 Transactions per Minute Oracle Block Size Comparison 64-bit HP-UX/64-bit Oracle raw-2kb Oracle blocksize raw-8kb Oracle blocsize QIO-2KB Oracle blocksize QIO-8KB Oracle blocksize SGA Size-MB Chart 5 illustrates that Quick I/O is on average 14% more efficient than raw with the async option turned on and that Quick I/O in async mode is 26% more efficient than Quick I/O running in synchronous mode. CPU utilization when using Quick I/O tends to be less than when using raw devices due to Quick I/O being able to take advantage of multiple worker threads, which raw cannot. Chart CPU Utilization Comparison 64bit HP-UX/64-bit Oracle raw-async I/O raw-sync I/O QIO-async I/O QIO-sync I/O CPU Utilization-% SGA Size-MB 5

8 The tables in the appendix show that there is no discernable trend in 32-bit versus 64-bit Oracle when they are both running on 64-bit HP-UX. Additionally, the tables indicate that the file system block size does not have much of an effect. Useful Tuning Tips The following are some performance tuning guidelines that may be helpful when using VERITAS Storage Edition to configure an Oracle database: Tip No. 1: Use VxFS Quick I/O for Databases VxFS in its default mode supports buffered I/O as does JFS, in which all I/O incurs the double copying and double buffering penalties. In addition, the UNIX semantics require that only one writer have the exclusive right to update a file, which forces other read and write operations on the same file to wait. Therefore, databases on file systems often perform slower than on raw devices. Quick I/O has I/O characteristics similar to raw I/O (a higher degree of read/write parallelism with less CPU overhead). Therefore, using Quick I/O provides a significant advantage in OLTP scenarios. Tip No. 2: Increase Oracle Tunable Parameter DB_BLOCK_BUFFERS The size of the Oracle buffer cache is determined by the setting of the INIT.ORA variable DB_BLOCK_BUFFERS. If the Oracle database files were built on top of regular file systems, this parameter would have to be set after taking into consideration the amount of system memory consumed by the file system buffering. Since Quick I/O bypasses the buffering at the file system level, this tunable could potentially be set higher, thereby increasing the size of the Oracle cache. Increasing the size of the Oracle cache typically results in improved performance. Tip No. 3: Use Striped Volumes for Performance In general, the best way to achieve the optimal throughput is by balancing the I/O across as many disks as possible. This balancing can be achieved by creating the database on a file system on a striped (RAID-0) volume using all the available disks. However, since redo logs tend to be accessed sequentially, one may see better performance by putting the redo logs on a separate file system, using a simple volume on a dedicated disk. If database reliability is important, mirror (using RAID-1) at least the volume used by the redo logs. Tip No. 4: Use async I/O for Quick I/O and sync I/O for buffered and direct I/O Async I/O provides increased throughput and significant decrease in CPU utilization for Quick I/O. Ensure that the async I/O parameter in Oracle is not turned on if using buffered I/O or direct I/O. In the init.ora file ensure disk_asynch_io = false. Tip No. 5: Use 2KB Oracle block sizes for OLTP workloads The use of larger block sizes appears to have a significant effect on degrading database throughput. Summary The OLTP performance measurements illustrate that the Quick I/O feature enables the VERITAS Storage Edition for Oracle to have equal performance compared to raw partition configurations and up to 188% performance improvement over traditional buffered I/O configurations. It achieves this performance while delivering the manageability only attainable by running Oracle on file systems. Using the OLTP benchmark, VERITAS Storage Edition for Oracle outperforms traditional file system buffered I/O and direct I/O configurations by a large margin. In addition, Quick I/O files can be managed as regular files and are backed by the reliability of VERITAS File System. With its log-based implementation, VxFS allows fast recovery after a system crash, which is not available in the HFS environment. VxFS also offers online administration features such as online resizing and defragmentation for higher availability and performance. VERITAS Storage Edition for Oracle integrates the above trusted storage products with Oracle-specific agents and extensions to create a software solution that enhances database performance, manageability, and availability. VERITAS Storage Edition provides immediate and significant performance improvements for Oracle databases, while it simplifies the DBA s job and reduces administrative down time. 6

9 Appendix A The following system file was used to tune HP-UX for the Storage Edition: *************************************** * Additional drivers required in every machine-type to create a complete * system file during cold install. This list is every driver that the * master.d/ files do not force on the system or is not identifiable by * ioscan. * Other CPU-type specific files can exist for their special cases. * see create_sysfile (1m). *************************************** * * Drivers/Subsystems ccio asyncdsk1 asyncdsk_included2 GSCtoPCI c720 sctl sdisk mux2 CentIf stape ps2 cdfs nfs_core nfs_client nfs_server lan2 dlpi inet uipc tun telm tels netdiag1 nms fcgsc_lan fc_arp lvm lv hpstreams clone strlog sad echo sc timod tirdwr pipedev pipemod ffs ldterm ptem pts ptm pckt diag1 nfsm 1 Used for async I/O 2 Used for async I/O 7

10 autofsc btlan6 token_arp btlan3 maclan diag0 diag2 dmem dev_config STRMSGSZ dump lvol nstrpty 60 vxfsvrts vxportalvrts fdd *************************************** * Tunable parameters *************************************** eqmemsize 512 max_async_ports maxdsiz 0x unlockable_mem 1 swapmem_on 0 maxssiz maxfiles 512 maxfiles_lim 512 max_thread_proc 5100 maxuprc 1024 nfile 5000 nflocks 4500 nproc 2512 ninode 5000 semmni 2048 semmns 2048 shmmax 0X shmmni 800 shmseg 50 maxswapchunks *************************************** * End changes *************************************** 3 Used with async I/O 8

11 Appendix B The results of the database performance measurements are summarized here. Most of the columns in these spreadsheets are selfexplanatory, but some of the comparisons are not. The fom is a figure of merit derived by dividing the transaction rate TPM by the CPU utilization CPU %. A larger number represents a more efficient option. The last three columns compare the two sets of results by comparing thruput (ratios greater than one represent better throughput), CPU % ratios (where less than one is a better result), and finally a comparison of the figures of merit for the two compared results. Each spreadsheet represents a variation in one of the possible parameters used in this study. For example, the first spreadsheet on the next page compares variations in file system block sizes. 9

12 Asynchronous I/O Oracle blocksize=2kb HP-UX 32bit/Oracle 32bit 1 KB VxFS block size HP-UX 32bit/Oracle 32bit 8 KB VxFS block size Comparisons Oracl compare compare 1KB / 8KB type clone e SGA #run TPM cpu % d to raw FOM* #run TPM cpu % d to raw FOM* thruput: CPU % FOM Raw 50 MB Raw 100 MB Raw 200 MB Raw 300 MB Raw 400 MB Raw 500 MB QIO 50 MB QIO 100 MB QIO 200 MB QIO 300 MB QIO 400 MB QIO 500 MB BIO 50 MB BIO 100 MB BIO 200 MB BIO 300 MB BIO 400 MB BIO 500 MB DIO 50 MB DIO 100 MB DIO 200 MB DIO 300 MB DIO 400 MB DIO 500 MB

13 file system blocksize=1 KB Oracle blocksize=2kb HP-UX 32bit/Oracle 32bit Asynchronous I/O HP-UX 32bit/Oracle 32bit Synchronous I/O Comparisons compared compared async / sync type clone SGA #run TPM cpu % to raw FOM* #run TPM cpu % to raw FOM* thruput: CPU % FOM Raw 50 MB Raw 100 MB Raw 200 MB Raw 300 MB Raw 400 MB Raw 500 MB QIO 50 MB QIO 100 MB QIO 200 MB QIO 300 MB QIO 400 MB QIO 500 MB BIO 50 MB BIO 100 MB BIO 200 MB BIO 300 MB BIO 400 MB BIO 500 MB DIO 50 MB DIO 100 MB DIO 200 MB DIO 300 MB DIO 400 MB DIO 500 MB

14 64-bit HP-UX and 64-bit Oracle file system blocksize=1 KB Async I/O Sync I/O Comparisons async/sync Oracle blocksize=2 KB compared compared type clone SGA size* #run TPM cpu % to raw fom** #run TPM cpu % to raw fom** thruput: CPU % fom Raw 50 MB Raw 100 MB Raw 200 MB Raw 300 MB Raw 400 MB Raw 500 MB QIO 50 MB QIO 100 MB QIO 200 MB QIO 300 MB QIO 400 MB QIO 500 MB BIO 50 MB BIO 100 MB BIO 200 MB BIO 300 MB BIO 400 MB BIO 500 MB DIO 50 MB DIO 100 MB DIO 200 MB DIO 300 MB DIO 400 MB DIO 500 MB

15 64-bit HP-UX file system blocksize=1 KB Oracle blocksize=2 KB 64-bit Oracle compared 32-bit Oracle compared Comparisons 64-bit/32-bit type clone SGA size* #run TPM cpu % to raw fom** #run TPM cpu % to raw fom** thruput: CPU % fom Raw 50 MB Raw 100 MB Raw 200 MB Raw 300 MB Raw 400 MB Raw 500 MB QIO 50 MB QIO 100 MB QIO 200 MB QIO 300 MB QIO 400 MB QIO 500 MB BIO 50 MB BIO 100 MB BIO 200 MB BIO 300 MB BIO 400 MB BIO 500 MB DIO 50 MB DIO 100 MB DIO 200 MB DIO 300 MB DIO 400 MB DIO 500 MB

16 OS/DB version: 64-bitHP-UX/64-bit Oracle Oracle blocksize 1 KB Oracle blocksize 8 KB Comparisons file system blocksize=1 KB Async I/O 1KB/8KB Oracle blocksize=varied compared compared type clone SGA size* #run TPM cpu % to raw fom** #run TPM cpu % to raw fom** thruput: CPU % fom Raw 50 MB Raw 100 MB Raw 200 MB Raw 300 MB Raw 400 MB Raw 500 MB QIO 50 MB QIO 100 MB QIO 200 MB QIO 300 MB QIO 400 MB QIO 500 MB BIO 50 MB BIO 100 MB BIO 200 MB BIO 300 MB BIO 400 MB BIO 500 MB DIO 50 MB DIO 100 MB DIO 200 MB DIO 300 MB DIO 400 MB DIO 500 MB

17 Corporate Headquarters 1600 Plymouth Street Mountain View, CA North American Sales Headquarters 400 International Parkway Heathrow, FL or Fax Global Locations United Kingdom or 44-(0) (0) Fax France Fax Germany 49-(0) (0) Fax South Africa Fax Australia 61-(0) (0) Fax Hong Kong Fax Japan Fax Malaysia Fax Singapore Fax China Fax Electronic communication World Wide Web: VERITAS Software Corporation. All rights reserved. VERITAS is a registered trademark of VERITAS Software Corporation in the US and other countries. The VERITAS logo, Business Without Interruption and VERITAS Storage Edition are trademarks of VERITAS Software Corporation in the US and other countries. Other product names mentioned herein may be trademarks and/or registered trademarks of their respective companies. Specifications and product offerings subject to change without notice. Printed in USA. October i UNIX-UNX10-SE2ORWPR-9 15

VERITAS Database Edition for Oracle on HP-UX. Performance Report

VERITAS Database Edition for Oracle on HP-UX Performance Report A P R I L 2 0 0 0 Executive Summary Until recently, Oracle administrators had a difficult choice to make between performance and manageability