ORACLE RMAN DESIGN BEST PRACTICES WITH PANZURA QUICKSILVER CLOUD STORAGE CONTROLLERS

Transcription

1 WHITE PAPER ORACLE RMAN DESIGN BEST PRACTICES WITH PANZURA QUICKSILVER CLOUD STORAGE CONTROLLERS Oracle is the de facto standard in the enterprise world when it comes to mission critical databases. Panzura Quicksilver Cloud Storage Controllers provide a highperformance and cost-effective storage target that seamlessly integrates with Oracle RMAN and efficient, massively-scalable cloud storage. This document describes configuration best practices for optimizing RMAN-Panzura-cloud deployments.

2 Background Traditional Oracle database environments consist of one or more servers running Oracle databases that typically use tier-1 SAN storage. The high transaction nature of databases drives the usage of very expensive and reliable SAN storage. For the purposes of this document, the act of replicating the database for data protection and BC/DR purposes will be referred to as a DB dump. Figure 1 illustrates a traditional DB dump environment. The goal is to minimize the impact from a CPU, memory and storage standpoint. This translates to maximizing bandwidth throughput to dump the database in order to minimize the impact on the storage containing the active database. WAN SRDF SRDF SAN Tier 1 Storage Offsite SAN Tier 1 Backup Server Staging Disk Tape Library Transportation Vault Storage In this situation, a very expensive 2nd disk-based staging or replication area is required. That staging area was traditionally 5-10 times larger than the original tier-1 storage. Over the years, as databases have gotten larger and larger while IT budgets have shrunk, the staging area has in many cases shrunk dramatically and in some cases only hold 1-2 copies of the database itself. Thus, finding high throughput and inexpensive capacity to offload such storage has become more and more critical to maintain acceptable levels of data protection. Oracle DB Dumps with Panzura Controller A typical topology of an Oracle dump when using Panzura Controllers is to directly use the NFS mounts as DB dump destinations. 2

3 Below is a figure that illustrates this: SAN Tier 1 Either the Oracle Server or another server mounting from the SAN tier-1 storage can mount the Panzura Controller over NFS and dump the DB directly to it.. Benefits of Using Panzura Controllers for DB Dumps There are many benefits in using Panzura Controllers as the destination target for Oracle DB dumps. The two main benefits are: High Throughput: Panzura Controllers provide a scale out NAS destination for Oracle DB dumps. Each controller, when equipped with the dual 10GE option, can provide SAN-like throughput. Customers frequently achieve 3-4Gbps per controller under steady state (note: 1st run seed performance is limited by the WAN performance). Panzura Controllers can be scaled out to increase performance as needed and can grow dynamically over time as the database increases in size. Panzura essentially decouples storage throughput from storage capacity. Unlimited Scale-out Deduplication: Panzura Controllers provide global deduplication across all controllers when configured together, allowing the Oracle server to dump to any controller without risk of generating redundant data segments. Any redundancy is detected across all controllers globally, with duplicate data sent to multiple controllers resulting in only one copy being stored. In addition, unlike traditional deduplication systems, there is no artificial limit to a Panzura Controller s deduplication capacity range (PBs of storage are supported). Panzura Controllers transmit data to the cloud after real-time deduplication, compression and encryption. The storage in the cloud provides an instantly available offsite DR and business continuity solution. 3

4 Transmission to the cloud starts immediately as data is received by the controller from the Oracle server. There is no staging on the controller itself. When coupled with Panzura s data/metadata separate and patent pending global snapshot technology, each Oracle DB dump creates a consistent file view across all Panzura controllers in the same Panzura Global File System, whether or not payload data has completely arrived into the cloud yet. Any remote system equipped with Panzura Controllers and security certificates can recover a full set of any DB dumps from any location via the cloud without any active connectivity to the originating Oracle server that initiated the dump. This recovery at one location of any location s DB dump results in no performance degradation for the originating site. As illustrated in the figure below, the recovery of any DB dump is done completely out of band: SITE 2 SAN Tier 1 SAN Tier 1 Oracle RMAN Terminology SITE 3 This section describes a number of important Oracle RMAN concepts and terminologies that directly relate to the use of Panzura Controllers. Types of Oracle DB Dumps Full DB Dump This is a dump of the entire database along with the log dumps that make up the ability to completely recover the original database itself. Partial Dump As its name implies, this results in a partial dump of the database itself. These are rarely used in conjunction with Panzura and typically satisfy a very specific use and workflow for a particular application and are not typical of a DB dump as a whole. 4

5 Incremental Dump This is used to store incremental changes from the last full DB dump. It is frequently used when there is not enough bandwidth or are not enough resources to supply a full dump. Given Panzura Controllers provide scale out global deduplication, it is generally not necessary to do incremental dumps since Panzura will only store between full DB dumps. Image Dump These provide an exact replica of the database including unused whitespace. Panzura will deduplicate and compress the whitespace so image dumps are not capacity inefficient. These dumps, however, do present a higher LAN requirement and in turn increase the DB dump window. Full DB dumps provides the same benefit and are much more frequently used by DB administrators. What Not to Do During a Pilot A practice to avoid is to schedule SAN-based DB dumps in parallel with a DB dump to Panzura. This will result in very poor performance because all server systems, including Solaris and Linux, prioritize SAN over Ethernet connectivity. It is critical to plan and schedule DB dump pilots for Oracle servers separately from SAN-based DB dumps. Oracle RMAN Parameters Below are some of the Oracle RMAN parameters that affect the operation of the system when a DB dump occurs onto a Panzura Controller. Managing these parameters can raise the overall throughput while minimizing the data sent to and stored in the cloud. Note that Oracle uses the the term backup in its literature when coupled with a DB dump and the resulting output is called a backupset. MAXOPENFILES [=] integer Controls the maximum number of input files that a BACKUP command can have open at any given time (the default is 8). Set this parameter to whatever setting prevents Too many open files error messages from your server when backing up a large number of files into a single backupset. The setting does not impact Panzura. FILESPERSET [=] integer When used with commands that create backupsets, this specifies the maximum number of files to include in each created backupset. By default, RMAN divides files among backupsets in order to make optimal use of channel resources. The number of files to be backed up is divided by the number of channels. If the result is less than 64, then it is the number of files placed in each backupset. Otherwise, 64 files will be placed in each backupset. Panzura requires this setting to be kept at 1. In theory, when Panzura data locality is appropriately applied, multiplexing different DB dumps into the same backupset would deduplicate well. However, this can add complexity that might lead to significant debug challenges so should be avoided when possible. 5

6 MAXPIECESIZE [=] integer Specifies the maximum size of each backup piece created on the defined channel. The size is specified in bytes, kilobytes(k), megabytes(m), or gigabytes(g). For example, if you set MAXPIECESIZE=5000, RMAN sets the maximum piece size to 4 kilobytes, which is the lower four kilobyte boundary of 5000 bytes. Given that Panzura starts moving data into the cloud independent of how the data arrives (whether one massive 100TB file or millions of small files), this setting can be kept at its default or the existing setting in the DB dump environment. Compression RMAN provides multiple types of compression depending on the Oracle versions. When used in conjunction with Panzura Controllers, all Oracle RMAN compression must be turned off. This is because Panzura Controllers must receive uncompressed data to detect redundancy. The controllers will deduplicate, compress and encrypt all received data before storing it in the cloud. Panzura s compression after deduplication will always be more optimal than only compressing it. Panzura compression is based on an algorithm that results in comparable performance to that used in Oracle RMAN (bzip2). Encryption Oracle RMAN has the capability of encrypting data. Encryption in RMAN must be turned off, for the same reason that compression should be disabled. Encrypted data cannot be deduplicated and optimized. Note that Panzura Controllers have received FIPS validation and the HW versions of the controller include an optional FIPS HSM for maximum security. All data is protected through key management of RSA Data at rest is encrypted using AES- CBC-256 and validated using HMAC-SHA256. Data in transit is in turn doubly encrypted using TLS and dependent on the cloud provider used. Typically in-transit encryption is also RSA & AES based, providing superior encryption to that from Oracle RMAN. Parallel Processes (aka Channels) Oracle RMAN dumps data using a combination of separate process called Channels. The Channels are meant to run in parallel to optimize the amount of CPU, memory and IOPs in the storage. One must take care not to increase the parallel processing to a point where the primary tier-1 SAN storage runs out of IOPs. A typical configuration is to run with 4 parallel channels (as configured in the Backup Settings of RMAN). Backup Settings -> Parallelism: 4 -> Disk Backup Location: /cloudfs/cc1/oracledb1 -> Disk backup type: Backup Set Scheduled Customized Backup: Review: Edit RMAN Script Run { Allocate channel oem_backup_disk1 type disk format /cloudfs/cc1/oracledb1/%u ; Allocate channel oem_backup_disk2 type disk format /cloudfs/cc1/oracledb1/%u ; Allocate channel oem_backup_disk3 type disk format /cloudfs/cc1/oracledb1/%u ; Allocate channel oem_backup_disk4 type disk format /cloudfs/cc1/oracledb1/%u ; backup filesperset = 1 as BACKUPSET tag %TAG database include current controlfile; backup filesperset = 1 as BACKUPSET tag %TAG archivelog all not backed up; release channel oem_backup_disk1; release channel oem_backup_disk2; release channel oem_backup_disk3; release channel oem_backup_disk4; } 6

7 The administrator is encouraged to run as many channels across all Panzura Controllers as possible to maximize throughput and load balance across them. However, care must be taken to make sure not to oversubscribe storage IOPs. The optimal setting is dependent on the IOPs performance of the source device. Panzura Controllers present one single global namespace across all appliances. When used with NFS under Oracle DB dump, each controller has a destination path to where it writes and all data can be read across all the other destination paths. This allows for a very flexible and intuitive management of the write destination while providing full recovery capabilities at remote sites independent of how many controllers are available at the remote site. For example, a financial customer has 12 controllers in NJ doing DB dumps and 6 controllers in NY available to recover the DB dumps in case of a natural disaster in NJ. Oracle OS Settings Linux Mount Options: LINUX mount -t nfs -o nolock,nfsvers=3,tcp,rsize= ,wsize= ,bg <cc-name>:/cloudfs /<ccname>/ Kernel Params: echo > /proc/sys/net/ipv4/tcp_rmem echo > /proc/sys/net/ipv4/tcp_wmem echo > /proc/sys/net/core/rmem_max echo > /proc/sys/net/core/wmem_max echo > /proc/sys/net/core/rmem_default echo > /proc/sys/net/core/wmem_default echo 5000 > /proc/sys/net/core/netdev_max_backlog echo 0 > /proc/sys/net/ipv4/tcp_timestamps echo > /proc/sys/kernel/msgmnb echo 300 > /proc/sys/kernel/msgmni echo 8192 > /proc/sys/kernel/msgmax echo > /proc/sys/kernel/sem echo 4096 > /proc/sys/kernel/shmmni echo > /proc/sys/kernel/shmall echo > /proc/sys/kernel/shmmax NFS Client Configuration: sysctl sunrpc.tcp_slot_table_entries=128 echo sunrpc.tcp_slot_table_entres=128 >> /etc/sysctl.conf Configured before mounts are created 7

8 MTU: subject to the intervening network supporting Jumbo frames. Linux Server Version: Note: Redhat 6 has significant performance advantages over Redhat 5 when it comes to NFS. Solaris Mount Options: mount -o rsize= ,wsize= ,llock <cc-name>:/cloudfs /<cc-name>/ Note: Solaris 10 or above NFS implementation is comparable in performance to Redhat 6. Common Implementation Most Panzura customers implement a full DB dump every day; a few also complete incremental dumps. The load is spread across multiple Panzura Controllers to maximize throughput. The archivelog destination can be placed in the same directory structure as the database dump but care must be taken in making sure that the Panzura data locality policy is configured appropriately based on the record size. It is recommended that non-database file extensions do not have deduplication enabled. The Panzura Controller GUI image below shows how to enable 8KB record size deduplication on Oracle DB dumps. The same type of policy settings could be coupled to deduplicate VMDK, SQL dumps, etc., at different record size. 8

9 Initial Seeding The initial seeding (1st dump) may sometimes take the longest, since no deduplication record exists and the Panzura Controller must send almost all the data it has received to the cloud. So for example, if a DB dump is 200TB, resulting in 50% compression, 100TB of data must move into the cloud. In a situation where the LAN speed is 20GE and WAN is 1GE, this would take 2+ days to transmit. It is recommended that the initial seeding DB dump be transferred into the Panzura system from an existing dump (either tape or staged) prior to scheduling an actual DB dump directly from the Oracle server. Subsequent dumps can be completed and transferred normally. In this example, assuming 20GE across 10 Panzura Controllers and a WAN link of 1GE and 10x deduplication, the total amount of data would be transferred in 1-2 hours, depending on the load of the Oracle server, and the resulting 10TB would be moved into the cloud within 1 day. Please use the Panzura Oracle RMAN Backup Sizing Calculator to properly size your environment. Conclusion Panzura Controllers, when used as an NFS target for Oracle RMAN DB dumps, provide unparalleled throughput and global deduplication across policy-based configurations for data protection and BC/DR of Oracle databases. Configuration is straightforward and implementation is relatively simple. By following the guidelines outlined in this paper, customers can achieve compelling results; however, it is imperative that the sizing and configuration of the policies and operating system associated with the DB dump be carefully configured. Typical customer results for deduplication are 10-30x, on average, along with throughputs at multiple 10GE LAN rates. 695 Campbell Technology Parkway #225 Campbell, CA (408) For more information: info@panzura.com For sales: sales@panzura.com 9