Disaster Recovery Strategies for RAC on Oracle SE Arjen Visser Dbvisit Software Limited

Transcription

1 Disaster Recovery Strategies for RAC on Oracle SE Arjen Visser Dbvisit Software Limited Keywords Oracle RAC on Standard Edition Standby Database with RAC on Standard Edition Different standby database architectures with Oracle RAC Index: RAC and Standard Edition (SE) What is different with RAC and standby databases on SE Different standby database architectures for RAC How to keep the standby database up to date How to create the single instance standby database from RAC primary database Third party products available Dbvisit Standby 1. RAC on Standard Edition When Oracle is implemented using the Real Application Clusters (RAC) option, it provides aspects of high availability and scalability, but does not in itself deliver DR; DR requires a remote copy of the database to be kept in synch. This remote copy of the database is referred to as a standby database. The underlying physical standby database is the same whether using Oracle Enterprise Edition with Data Guard or whether using a manual standby database using a third party tools such as Dbvisit Standby with Oracle Standard Edition. The process involves transferring the redo (archive or online) from the primary to the standby database and applying the redo to the standby database in a recovery mode. The basic overview of the process can be summarized as follows:

2 Oracle Enterprise Edition is delivered with Data Guard, which can deliver DR for Oracle RAC, but Data Guard does not come with Oracle Standard Edition or Standard Edition One. This paper looks at how to address DR on Oracle Standard Edition. DR is implemented with a standby database both with Data Guard under Enterprise Edition and Standard Edition. RAC adds additional complexities to the single-instance database and this includes complexities to the standby database. Oracle RAC on Standard Edition is a free option and can be used on servers with up to 4 sockets (cores not counted as with EE). The following configurations using RAC on Standard Edition are valid: 2 RAC nodes with 2 sockets each 3 RAC nodes with 2 nodes having 1 socket and 1 node having 2 sockets 4 RAC nodes with 1 socket each Oracle Standard Edition on RAC must use the ASM file system. All files with in the database (including redo and archives) must be in ASM. The diagram below illustrates the various infrastructure layers in a modern web application environment, and the method for providing HA and scalability at each layer. Oracle RAC provides HA and scalability at the database server layer. Redundant servers and additional products such as load balancers can be used to provide HA and scalability at the application and web server layers. 2. What is different with RAC and standby databases on SE A normal single instance database has only one redo thread. A RAC database has multiple threads, each instance having its own thread.

3 A standby database is a 100% binary copy of the primary database so this means that the standby database will mirror the primary database. If the primary database is a two node RAC database the primary database will have two threads and the standby database will have two threads. 3. Different standby database architectures for RAC For a RAC primary database, there are two possible standby database architectures. They are: 1. Single instance standby database 2. RAC standby database 3.1 Single instance standby database The simplest architecture is using a single instance standby database with a RAC primary database. Even though this will be a single instance standby database, there will still be the same number of threads at the standby database and the archivelogs from all threads will be needed to keep the standby database up to date. See Figure 1 - Single instance standby database Figure 1 - Single instance standby database Key settings of a single instance standby database: The value of DB_NAME should be the same in both the standby and primary initialization parameter files. INSTANCE_NAME is same as DB_NAME (by default) Example: Primary RAC database SQL> select INSTANCE_NUMBER, INSTANCE_NAME, STATUS from gv$instance; INSTANCE_NUMBER INSTANCE_NAME STATUS ORCL1 OPEN 2 ORCL2 OPEN

4 SQL> show parameter cluster_database NAME TYPE VALUE cluster_database boolean TRUE cluster_database_instances integer 2 Standby single instance SQL> select INSTANCE_NUMBER, INSTANCE_NAME, STATUS from v$instance; INSTANCE_NUMBER INSTANCE_NAME STATUS ORCL MOUNTED SQL> show parameter cluster_database NAME TYPE VALUE cluster_database boolean FALSE cluster_database_instances integer RAC standby database A RAC standby database is also possible. The RAC standby database does not need to have the same number of nodes as the primary RAC database but there will be the same number of threads at the standby database and the archivelogs from all threads will be needed to keep the standby database up to date. See Error! Reference source not found.. Figure 2 - RAC standby database Key settings of a RAC instance standby database: The value of DB_NAME should be the same in both the standby and primary initialization parameter files. Start with single instance standby database (already have multiple threads) Have RAC infrastructure ready on standby nodes Convert to RAC standby database using standard documented procedures

5 Example: Primary RAC database SQL> select INSTANCE_NUMBER, INSTANCE_NAME, STATUS from gv$instance; INSTANCE_NUMBER INSTANCE_NAME STATUS ORCL1 OPEN 2 ORCL2 OPEN SQL> show parameter cluster_database NAME TYPE VALUE cluster_database boolean TRUE cluster_database_instances integer 2 Standby RAC database NAME TYPE VALUE cluster_database boolean TRUE cluster_database_instances integer 2 4. How to keep the standby database up to date Standby databases are held up to date by applying archive (redo) log files from all threads from the primary database: The Standby database is in constant recovery mode Uses standby controlfile Archive log files from all threads are needed Standby redo logs are not available in SE The command to apply (manually recover) the archive log files on the standby database is: sqlplus> recover standby database Archives are applied in order of SCN across the threads. This means that the same archive can be used multiple times. An example of applying archives to the standby database is: Log seq 2715 thread 2 applied to standby database ORCL. Log seq 2326 thread 1 applied to standby database ORCL. Log seq 2327 thread 1 applied to standby database ORCL. Log seq 2715 thread 2 applied to standby database ORCL. Log seq 2716 thread 2 applied to standby database ORCL. Log seq 2327 thread 1 applied to standby database ORCL. Log seq 2328 thread 1 applied to standby database ORCL. Log seq 2716 thread 2 applied to standby database ORCL. Log seq 2717 thread 2 applied to standby database ORCL. Log seq 2328 thread 1 applied to standby database ORCL. Once there are no more logs to be applied, the cancel command is issued: sqlplus> recover standby database cancel

6 Media recovery cancelled. When one node is down, then archives are only needed from the active node as Oracle has closed this thread. This is true for Oracle 11. In Oracle 10g, if one node is down then the archives for both nodes are still needed. 5. How to create the single instance standby database from RAC primary database The best way to create a standby database from a RAC primary database is to use RMAN. There are different commands a standby database can be created with RMAN. They are: 1. Using the duplicate command 2. Using backupsets. A backup set contains one or more binary files in an RMAN-specific format. Each of these files is known as a backup piece. A backup set can contain multiple datafiles. For example, you can back up 10 datafiles into a single backup set consisting of a single backup piece. In this case, RMAN creates one backup piece as output. The backup set contains only this backup piece. 3. Using imagecopy. An image copy is an exact copy of a single data file, archived redo log file, or control file. Image copies are not stored in an RMAN-specific format. They are identical to the results of copying a file with operating system commands. RMAN can use image copies during RMAN restore and recover operations, and you can also use image copies with non-rman restore and recovery techniques. Using one of the above RMAN commands, a single instance standby database is created with the same number of threads as the primary. RMAN does not backup redo logs and these are not required in a standby database. With Oracle Enterprise Edition, standby redo logs are possible, but Oracle Standard Edition does not have standby redo logs. The simplest RMAN method to create a standby database is to use duplicate command, but this has disadvantages: You need TNS connection between primary and standby The operation is not resumable. With Oracle 11.2 and above, this is no longer the case and the operation is resumable. When creating a standby database, having a resumable operation is important because the standby database can be very large and take a lot of time. When something goes wrong, it is hard to debug. I prefer to use backupsets. The reasons are: Each step is controllable It works for all (most) versions of Oracle There are no TNS configuration needed The operation is resumable The main high level steps in creating a standby database using RMAN backupsets are: 1. BACKUP Backup each primary datafile one by one as a compressed backupset using RMAN this will result in generating one or more backup pieces Create standby controlfile and standby spfile with:

7 cluster_database = false cluster_database_instances = 1 2. TRANSFER Transfer all backup pieces, including standby controlfile and spfile to the standby server using OS copy (not TNS) 3. RESTORE SQL>startup nomount SQL>alter database mount standby database; Restore each datafile one by one using RMAN Rename datafiles and tempfiles if needed The steps required are: 1. Create standby pfile with cluster_database = false cluster_database_instances = 1 2. Create RMAN backup backup datafile AS COMPRESSED BACKUPSET $file_id format '$bck_format'; 3. Create standby controlfile backup current controlfile for standby format 'xxx'; 4. Copy all files (backup pieces, standby controlfile and spfile/pfile) to standby server 5. Restore standby controlfile backup current controlfile for standby format 'xxx'; 6. Startup standby database as mount: SQL>startup nomount SQL>alter database mount standby database; 7. Restore datafiles on standby server restore datafile $file_id; 8. Rename file if necessary set newname for datafile $file_id to '$newname ; restore datafile $file_id; switch datafile $file_id; 6. Third party products available Dbvisit Standby

8 As Oracle Data Guard is not available on Oracle Standard Edition, it is necessary to create an alternate solution to provide DR for RAC on Oracle Standard Edition. On the surface, creating a manual scripted solution may seem simple: extract the archive log(s), transfer the log, apply them to the standby database. Beyond these simple command however, lies a myriad of considerations and risks related to the operation, maintenance, reliability and support of the solution: Is it robust/secure enough? Can it recover from all outages and glitches? Does it have a good locking and transport mechanism? Does the transportation of data between primary and standby sites secure the critical business data? Has it been tested under all scenarios? Has it been proven to work in a disaster recovery situation? Can the primary site be reinstated after the outage? Does it have solid notifications? Does it cover all the DBMS errors and exceptions? Will it support multiple databases? Does it scale as your data volumes increase? What will happen when you change, upgrade or patch the DBMS? What happens when you rebuild or refresh the standby database? Does it support DBMS feature such as RAC, OMF and ASM? Does it include support and documentation? Will other DBA's be comfortable using and maintaining it if the current DBA leaves? Delivering on all of these critical features can add significant overhead to the total cost of the homegrown manual scripted solution. Once developed these systems need to be supported, placing a critical dependency on the staff member or members who originally developed them. These systems also need to be maintained and tested with each change to the underlying environments, including updated operating systems and database management systems, as well as infrastructural changes. It is also critical to assess the reliability of the solution in the areas outlined above. Without extensive testing of exception cases it is difficult to be confident that the home-grown solution will operate as required. This level of testing can be difficult to perform in a customer environment with minimal test environments and infrastructure. The sole purpose of these data recovery solutions is to provide certainty in the event of a site failure, so if this cannot be assured then the approach needs to be questioned. The risks of an in-house solution can be summarized as follows: 1) Development cost unknown may or may not be cheaper to build than licenses for a third party product. 2) Reliability of maintaining synchronization, failover management and recovery is all unproven 3) The need to maintain staff skills in order to provide the ongoing support, maintenance, enhancement and operation of the solution

9 The option of third party tools provides an ideal balance between the in-house development and Data Guard approaches, delivering a proven, reliable and supported product in a cost effective manner. The question then becomes selecting the right third-party tool. Dbvisit Standby is a proven, reliable and stable product used by customers in over 80 countries round the globe. Dbvisit Standby delivers a proven and low risk solution at a relatively low total cost, including licensing, implementation and ongoing operation. Dbvisit Standby is supported on RAC and the combination: Oracle Standard Edition, RAC and Dbvisit Standby makes a compelling, robust and scalable configuration that not only achieves High Availability (HA), Disaster Recovery (DR) but also cost-effectiveness. For more information on Dbvisit Standby, please visit Contact address: Arjen Visser Founder and CEO Dbvisit Software Limited Auckland, New Zealand Tel arjen.visser@dbvisit.com