An Oracle White Paper September Replicating between Cloud and On-Premises using Oracle GoldenGate

Transcription

1 An Oracle White Paper September 2013 Replicating between Cloud and On-Premises using Oracle GoldenGate

2 Executive Overview... 2 Requirements for Ensuring Data Consistency... 3 Availability... 3 Reduced latency... 3 Heterogeneity... 3 Flexibility... 3 Reliability... 3 Topology... 3 Low impact and minimally intrusive... 4 Efficient Network usage... 4 Globalization... 4 Customization... 4 Ease of use... 4 Oracle GoldenGate... 4 Architecture... 5 Setup and Configuration... 6 Acquiring and Configuring Cloud Instance... 7 Database Installation... 9 Configuring networking between the cloud and on-premises instances 10 Oracle GoldenGate Installation Oracle GoldenGate Configuration Conclusion... 22

3 Executive Overview Enterprises today have greater flexibility in determining whether investing in applications, platforms and infrastructure should be a capital expenditure or operational expenditure or both. As such enterprises are increasingly using a mix of public cloud, private cloud and onpremises strategy to gain sustainable competitive advantage. However, enterprises have to ensure that business processes run effectively and reliably irrespective of whether the applications and its associated data are on the on-premises instances or in the cloud. For example, an enterprise which has adopted a CRM strategy could be relying on an onpremises based marketing application used for developing and nurturing leads and could be using a SaaS based Sales application to create opportunities and quotes. The sales and the marketing teams which use these systems need to be able to access and share the data in a reliable and cohesive way. This example can be extended to other applications areas such as HR, Supply Chain, and Finance and the demands the users place on getting a consistent view of the data. Another example, an enterprise may have established on-premises based Business Intelligence and Reporting platform which is used by employees in various roles to retrieve their respective reports and perform analysis. Typically, a Business Intelligence platform requires data from various sources to be aggregated so it can provide rich capabilities to slice and dice the data. Enterprises which have a mix of public cloud, private cloud and onpremises will need to ensure that the relevant data from these sources are made available to the Business Intelligence system. Most enterprises have spent years avoiding the data silos that inhibit productivity. IT has had its fill of new integration paradigms, from CORBA to Client/Server to Web services, EAI, SOA and replicating databases. After decades of locking down critical issues such as interface definitions, governance, reliability, transaction management, exception handling, and transaction monitoring, it is imperative to extend these solutions into environments which has a mix of cloud and on-premises applications and its associated data. The objective of this white paper is to discuss the requirements, architecture and the options for replicating between databases hosted on-premises and in the cloud. In this paper we will illustrate replication between two Oracle database Linux 64 bit instances one hosted in the Amazon EC2 cloud and the other running within the Oracle on-premises environment. It is 2

4 assumed that reader of this paper has working knowledge of Oracle GoldenGate replication product, Amazon EC2 cloud, Oracle database, working knowledge of Linux and networking. The option and the related procedure presented in this paper is one among the many options an IT organization can pursue when considering replicating databases between the onpremises and the cloud environments. The option and the procedure discussed should not be construed as a recommendation and hence should not be considered for production without performing due-diligence. Requirements for Ensuring Data Consistency The benefits of using a hybrid environments consisting of on-premises and cloud solutions become apparent only when the requirements listed below are addressed. Availability Irrespective of where the data is captured either in the cloud or on-premises it must be accessible anywhere at or near 24/7/365 without service interruption, performance degradation or data inconsistency. Reduced latency Data must remain fresh. As it ages, data becomes less relevant and less valuable day-old data is often insufficient in today s competitive landscape. Heterogeneity Integration and replication solutions should be able to transform and distribute data across diverse IT systems. Flexibility Integration and replication solutions must have the flexibility to work with IT infrastructure and policies which are put in place for security and compliance. Reliability Reliability must be guaranteed despite system or connectivity issues that can occur between the cloud and on-premises instances. Topology The solution should support a wide variety of topologies. These include one-to-one, one-to-many, many-to-one, and many-to-many for both unidirectional and bidirectional configurations. The ability to handle and resolve conflicts in a bi-directional active-active scenario should also be supported. 3

5 Low impact and minimally intrusive The solution should be minimally intrusive and should have a low impact on the hardware, software systems (e.g. databases), and network resources. Efficient Network usage The solution should be able to route transactions over WANs and LANs as well as the internet, handle large data volumes with minimal bandwidth requirements and latency. Globalization The solution should be able to handle the globalization aspects of heterogeneous database environments when replicating data between the cloud and on-premises. Customization The solution should provide capabilities for customization for transforming and delivery of data. Ease of use The solution should be easy to use, understand, configure, deploy, and diagnose Oracle GoldenGate Oracle GoldenGate is used for major Fortune 500 companies and other industry leaders worldwide to support mission-critical systems for data availability and integration. Oracle GoldenGate 11g empowers organizations to capture, route, transform, and deliver transactional data between heterogeneous databases in real time with minimal overhead. Using unique real-time, logbased CDC and replication technology, Oracle provides continuous availability, disaster tolerance, and real-time data integration solutions that enable the management and movement of transactional data across the enterprise. Oracle GoldenGate is designed for low-impact implementation, operation, modification, and extension to support the evolving needs of enterprise information. Oracle GoldenGate addresses the requirements for ensuring data consistency in a hybrid environment involving the cloud and on-premises instances thus facilitating the business process to run effectively and reliably. 4

6 Architecture Oracle GoldenGate processes running on the cloud and on-premises need to communicate with one another on facilitate a bi-directional replication. There are several options to consider which enable bidirectional replication between the cloud and on-premises. In this section we will illustrate the high level architecture of some of those options and in the subsequent sections discuss the options in detail. Option 1 Secured network established between the cloud and on-premises The following diagram illustrates a high level architecture and the components involved in replication between two databases one deployed in the cloud and the other in the on-premises environment with a secured network established between them. A secured network is established between the cloud and on-premises which enables the applications running on the cloud and on-premises to have seamless connectivity to other applications irrespective of where they are running. Option 2 Restricted network established between the cloud and on-premises The following diagram illustrates a high level architecture and the components involved in replication between two databases one deployed in the cloud and the other in the on-premises environment with a restricted network established between them. A restricted network is established between the cloud and on-premises which requires the ports required by Oracle GoldenGate explicitly opened on both the cloud and on the on-premises instances and the IP addresses of the cloud and on-premises instances white listed. 5

7 Option 3 Restricted network access from on-premises and cloud through HTTP proxy The following diagram illustrates a high level architecture and the components involved in replication between two databases one deployed in the cloud and the other in the on-premises environment with a restricted HTTP proxy between them. In scenarios where the on-premises machine connects to the internet through a restricted proxy and the ports required by Oracle GoldenGate are not open and the cloud instance is not white listed on the on-premises instance, then tunneling through HTTP proxy may be considered provided proper security exceptions are obtained. Setup and Configuration Broadly, the setup and configuration required for replicating databases between a cloud and onpremises environment involves the following steps: Acquiring and configuring cloud instance. Installing and configuring database in the cloud and on-premises instances. Configuring networking between the cloud and on-premises instances. Installing and configuring Oracle GoldenGate on the cloud instance and on-premises instance. 6

8 Acquiring and Configuring Cloud Instance Depending upon the cloud provider selected, acquiring and configuring the cloud instance will vary. In this paper we will focus on the Amazon EC2 cloud and examine the options and the steps required to acquire and configure the Amazon EC2 cloud instance. Acquiring Amazon EC2 Instance The following provides a list of options for acquiring an EC2 instance from Amazon Renting the Linux instance Amazon provides the requested Linux environment and access to this environment though SSH. Oracle Database and Oracle GoldenGate will need to be downloaded and installed. Renting the Oracle Database - Amazon will create and provide access to the requested Oracle database. The database can be accessed via SQL NET from a remote machine. Amazon does not provide access to the machine or'sys' access to the database. Thus, Oracle GoldenGate has to be installed on a remote machine or on a separate instance in EC2. Renting the Linux instance with Oracle database installed Linux machine with Oracle database is available as public AMI with Amazon. This instance is accessible and provides full control of the database. The AMI ID selected was ami-3d739c54. This image has Oracle Database bit Enterprise Edition installed on a 64 bit RedHat Enterprise Linux Server version 5.4. All the discussions in this paper are with reference to this AMI. This option was selected for the purpose of this paper. Configuring the EC2 instance The EC2 user guides provide detailed information about using and configuring the EC2 instance. However, for the purpose of this paper, some of the important configuration will be covered. Navigate to your account in Amazon using a web browser and perform the following configuring the key pair, configuring security groups, and starting the EC2 instance. Configuring Key Pair The EC2 image can be accessed only through SSH by default, as such associating it with the security key is important. A key pair can be created before or when launching the instance. The key pair will need be associated with the instance. Download and keep the private key generated in a secure location. The private key is required when accessing the EC2 instance. Please note, if appropriate ports are opened and correct packages are installed, EC2 instance can also be accessed by GNOME desktop via VNC or RDP. However, for the purposes of this discussion we will focus on accessing by SSH. Configuring Security Groups The security group provides the rules for allowing external connection into the EC2 instance. Using the security group add rules which will specify the protocol, port and the IP address from which the connection is desired. Create a security group and associate the security group with the EC2 instance. 7

9 It is mandatory to create at least one rule which specifies SSH protocol to enable logging in to the instance. In the Inbound tab, create a new rule for SSH and specify the on-premises IP address. If connection from additional machines, such as laptop, is required then add the IP addresses for these machines or the proxy IP address the machine uses. Using the link from the machine the connection is desired will provide the proxy's IP address. Additional security rules can be created as required we will discuss this in the appropriate sections. Starting the EC2 instance Launch the EC2 instance associated with the above mentioned AMI through the browser and monitor to ensure the successful starting of the instance. The default root id is 'root'. Use this when logging in to EC2 instance. Logging on to the EC2 Instance The EC2 instance will need to be accessed using SSH to complete the configuration and perform other activities. EC2 instance can either be accessed from utility such as Putty from a windows laptop machine and by using the ssh utility from the on-premises machine. Please note that the proxy server used by either laptop or on-premises machine should allow SSH connectivity. Using Putty Create a new session and specify the connection to the EC2 instance. Convert the private key provided by EC2 instance into Putty format using PuttyGen and store this into a file with the ppk extension. Specify this file in Putty configuration under Connection->SSH->Auth->Private Key file. If the laptop, is using VPN, for example, then in the Proxy settings (Connection->Proxy) configure the Proxy hostname (set to the VPN proxy server) and set the proxy type = HTTP. If additional software on the laptop such as Oracle SQL Developer needs to connect to the database on the EC2 instance then do the following in addition to the above. In the Putty window, navigate to Connection->SSH->Tunnel and specify the Source port and Destination. For the Source port specify any free port that is available on the laptop and for the Destination specify the value in the format :<Destination Port>. For example, if connecting to Oracle instance the Destination Port = Connect to the EC2 instance and keep the connection window alive. Modify the properties in the Oracle SQL Developer and specify 'localhost' for the Host name and the value used for Source port for the Port and connect to the Oracle Instance running on the EC2 instance. Using SSH from the on-premises machine From a command prompt window on the on-premises machine, login or sudo to root. Modify the 'config' file under ~/.ssh directory and add the following. Substitute appropriate values for the EC2_Instance_Address and for the proxy server. 8

10 Typically, the proxy will not allow SSH connection from the on-premises machine to a remote machine outside the firewall. Utilities such as 'corkscrew' or 'netcat' facilitate tunneling through the HTTP proxy. We will use 'corkscrew' utility for this paper. It is assumed the utility 'corkscrew' is installed on the on-premises machine. Host 'EC2_Instance_Address' ProxyCommand corkscrew 'proxy server' 80 %h %p KeepAlive yes ServerAliveInterval 30 Place the private key file obtained from EC2 instance in an secured directory. Enter the following command to SSH into EC2 instance. ssh i <private_key_directory_name>/<private_key_file>.pem root@ EC2_Instance_Address Executing the above commend will login the terminal into the EC2 instance with root access. Database Installation Depending upon the cloud provider selected, acquiring and configuring the database instance will vary. In this paper we will focus on the Amazon EC2 cloud and examine steps required to configure the database. As we had mentioned previously, there are 3 options when working with the Amazon EC2 cloud and we had chosen to discuss the option 'Renting the Linux instance with Oracle database installed' as this option provided the convenience and flexibility when working with the database. EC2 Instance When logging into the EC2 instance created based on the AMI ami-3d739c54 for the first time, it will go through a questionnaire to configure the Oracle database to set the SYS password, Oracle SID, and other properties required to bring up the Oracle database. This is done only once. Note down the passwords and SID and other information. Subsequently, when connecting from SSH, change user to oracle - 'su oracle' which ensures that the required environment is setup. Test the connection to the database from Sql Developer from the laptop using Sqlplus from the ssh terminal. After ensuring the database is up and running do the following. Create two users An Oracle GoldenGate user named 'gguser' and another user 'ogg_amz'. We will use these users throughout this paper. Create a table in ogg_amz for replication. 9

11 For example - CREATE TABLE "ogg_amz"."cust_info" ( "CUST_ID" NUMBER NOT NULL ENABLE, "NAME" VARCHAR2(20 BYTE) NOT NULL ENABLE, CONSTRAINT "CUST_INFO_PK" PRIMARY KEY ("CUST_ID") USING INDEX ) On-premises Instance It is assumed that Oracle Database bit Enterprise Edition is installed on a 64 bit RedHat Enterprise Linux Server version 5.4. After ensuring the database is up and running do the following. Create two users An Oracle GoldenGate user named 'gguser' and another user and another user 'ogg_ora'. We will use these users throughout this paper. Create a table in ogg_ora, for replication. For example - CREATE TABLE "ogg_ora"."cust_info" ( "CUST_ID" NUMBER NOT NULL ENABLE, "NAME" VARCHAR2(20 BYTE) NOT NULL ENABLE, CONSTRAINT "CUST_INFO_PK" PRIMARY KEY ("CUST_ID") USING INDEX ) Configuring networking between the cloud and on-premises instances As we had discussed earlier, broadly there are three options to consider when configuring the networking between the cloud and the on-premises instance. In this paper we will focus on the Amazon EC2 cloud and examine options. Oracle GoldenGate processes running on the EC2 instance and on-premises need to communicate with one another on facilitate a bi-directional replication. There is no specific requirement for what ports Oracle GoldenGate can use; any unused port can be used. The following sections discuss the options available for the user to setup replication between EC2 and on-premises instance. Option 1 - Secured network established between the cloud and on-premises A secured network is established between the cloud and on-premises which enables the applications running on the cloud and on-premises to have seamless connectivity to other applications irrespective of where they are running. In this scenario configuring Oracle GoldenGate for replication between 10

12 the databases residing on the cloud with the databases residing on the on-premises requires no additional or special configuration. Oracle GoldenGate security features such as password, trail and transport encryption can be considered as required. Option 2 - Restricted network established between the cloud and on-premises A restricted network is established between the cloud and on-premises which requires the ports required by Oracle GoldenGate opened on both the EC2 and on the on-premises and the IP addresses of the EC2 instance and on-premises white listed. On the EC2, use the EC2 Security Group and add rules to white list on-premises IP addresses and the open the TCP ports configured to be used by Oracle GoldenGate manager running on EC2 and the dynamic ports configured in the manager parameter file. On the on-premises white list the EC2 IP address and open the TCP ports configured to be used configured to be used by Oracle GoldenGate manager running on on-premises and the dynamic ports configured in the manager parameter file. Oracle GoldenGate security features such as password, trail and transport encryption can be considered as required. Additionally, the Oracle GoldenGate passive Extract feature can be considered when replicating from EC2 to on-premises. When a target system resides inside a trusted intranet zone, initiating connections from the source system (the standard Oracle GoldenGate method) may violate security policies if the source system is in a less trusted zone. It also may violate security policies if a system in a less trusted zone contains information about the ports or IP address of a system in the trusted zone, such as that normally found in an Oracle GoldenGate Extract parameter file. In this kind of intranet configuration, a passive-alias Extract configuration can be considered. Connections are initiated from the target system inside the trusted zone by an alias Extract group, which acts as an alias for a regular Extract group on the source system, known in this case as the passive Extract. Once a connection between the two systems is established, data is processed and transferred across the network by the passive Extract group in the usual way. In the next section we will discuss the configuration of using passive Extract on EC2 Option 3 - Restricted network access from on-premises and cloud through HTTP proxy In scenarios where the on-premises machine connects to the internet through a restricted proxy and the ports required by Oracle GoldenGate are not open and the EC2 instance is not white listed on the on-premises instance, then the following option may be considered. If the proxy allows SSH tunneling over HTTP, then it can be used facilitate replication between the on-premises machine and the EC2 instance. Please note some environments do not allow HTTP to be used as a transport for SSH. Unless proper exceptions from IT security are obtained, this option is not recommended. In this section we will discuss the configuration of SSH tunneling through HTTP. In 11

13 the next section we will discuss the configuration of Oracle GoldenGate to use the SSH tunnel for replication. SSH Tunneling though HTTP The following steps illustrate the procedure to set up SSH tunneling though HTTP to facilitate replication between an on-premises machine and an EC2 instance. On the EC2 instance set up the Security Group to allow SSH connection from the on-premises instance proxy server. Determine the Oracle GoldenGate ports and the dynamic port you plan to use in the manager parameter file on the EC2 instance and on-premises instance. For the purpose of this paper the following ports are defined on the EC2 and on-premises instance. o o Ports on the EC2 instance Manager port = Dynamic ports = Ports on the on-premises machine Manager port = Dynamic ports = On the on-premises machine setup the configuration for SSH tunneling through HTTP proxy, local and remote port forwarding. As discussed earlier, utilities such as 'corkscrew' or 'netcat' facilitate tunneling through the HTTP proxy. It is assumed the utility 'corkscrew' is installed on the on-premises machine. The port forwarding feature of SSH makes available remote ports as local ports for the applications to use. SSH encrypts the messages sent by the application and tunnels it though the HTTP proxy decrypts the message on the target and delivers the message to the port the target application is listening. Configuring SSH to tunnel though HTTP and facilitate port forwarding involves the following steps. On the on-premises instance login in as root or sudo to 'root' to the on-premises machine. Navigate to the ~/.ssh directory and edit the SSH 'config' file and add the parameters listed in the following table. The ProxyCommand instructs the SSH to use 'corkscrew' to tunnel through the specified HTTP proxy server. The LocalForward parameter instructs SSH to forward ports used by Oracle GoldenGate on the EC2 instance as corresponding local ports on the on-premises instance. The RemoteForward parameter instructs SSH to forward ports used by Oracle GoldenGate on the on-premises instance as corresponding local ports on the EC2 instance. 12

14 Refer to the Oracle GoldenGate Configuration section for details about configuring Oracle GoldenGate on utilizing the forwarded ports. Host 'EC2 Instance Address' ProxyCommand corkscrew 'proxy server address' 80 %h %p GatewayPorts yes KeepAlive yes ServerAliveInterval 30 LocalForward localhost:15000 LocalForward localhost:15010 LocalForward localhost: enter the remaining dynamic ports specified in the manager parameter file on the EC2 instance.. RemoteForward localhost:17000 RemoteForward localhost:17010 RemoteForward localhost: enter the remaining dynamic ports specified in the manager parameter file on the on-premises box.. Place the private key file obtained from EC2 instance in a secured directory. Enter the following command to establish SSH tunnel though the proxy. Keep this terminal alive. ssh i <private_key_directory_name>/<private_key_file>.pem root@ EC2_Instance_Address Alternatively, the following command can be issued to establish an SSH tunnel though the proxy without having to have a separate terminal. nohup ssh -N i <private_key_directory_name>/<private_key_file>.pem root@ EC2_Instance_Address & Oracle GoldenGate Installation EC2 and On-premises Instance Download the Oracle GoldenGate version which is suitable for the version of the Linux operating system and the version of the Oracle database. Please refer to the Oracle GoldenGate user guide for additional details on installation and configuration. Ensure that you are logged in as 'oracle' before installing Oracle GoldenGate. 13

15 Oracle GoldenGate Configuration We will discuss the Oracle GoldenGate configuration for options 1, 2 and 3 discussed above. We will first discuss the common configurations required irrespective of the options selected then we will discuss option specific configuration that will be required. Common database configuration for required for all the options In this section we will discuss the common configuration required to be performed irrespective of the options selected. Oracle GoldenGate User As we had discussed in the database installation section, ensure that the following database users are created in the EC2 and on-premises. EC2 Instance gguser and ogg_amz On-premises gguser and ogg_ora Checkpoint table On both the EC2 and on-premises instances, Create a 'GLOBALS' file in the Oracle GoldenGate install directory and specify the checkpoint table name, this is used when adding the replicat. GLOBALS files on EC2 instance checkpointtable amz_checkpointtable GLOBALS file on on-premises instance checkpointtable ora_checkpointtable From ggsci command shell enter the following command on both the EC2 and on-premise instances. ggsci>dblogin userid gguser, password xyz ggsci>add checkpointtable 14

16 Enable supplemental logging Minimal supplemental logging must be enabled at the database level to allow GoldenGate to properly capture updates to primary keys. To enable supplemental logging at the database level on both EC2 and on-premises instances do the following. Login to SQL*Plus as a user with ALTER SYSTEM privileges Enable minimal supplemental logging by executing the following command: sqlplus>alter DATABASE ADD SUPPLEMENTAL LOG DATA; Verify that supplemental logging is enabled at the database level via the command: sqlplus>select SUPPLEMENTAL_LOG_DATA_MIN FROM V$DATABASE; The output of the query must be YES On both EC2 and on-premises instances database must be set to log the table key values whenever it logs a row change, so they are available to Oracle GoldenGate. This is accomplished by issuing the following ggsci command. EC2 Instance ggsci>dblogin userid gguser, password xyz ggsci>add trandata ogg_amz.* On-Premises Instance ggsci>dblogin userid gguser, password xyz ggsci>add trandata ogg_ora.* Common Oracle GoldenGate Configuration for all the options In this section we will discuss the common Oracle GoldenGate Configuration for all the options in the EC2 and the On-premises instance Configuration on the EC2 Instance The configuration for Oracle GoldenGate Manager, Capture, and Apply process on the EC2 instance is illustrated below. Ensure you are logged in as 'oracle' (su oracle). Oracle GoldenGate Manager The manager parameter file is shown below. 15

17 port dynamicportlist purgeoldextracts./dirdat/*, usecheckpoints Start the manager and ensure it starts successfully. ggsci>start mgr ggsci>info mgr Oracle GoldenGate Capture Ensure, supplemental logging is enabled for the table. Supplemental logging is enabled by issuing the command 'add tranlog ogg_ora.cust_info' from the 'ggsci' command shell. The capture parameter file is shown below. extract eamz tranlogoptions excludeuser gguser exttrail./dirdat/ez userid gguser, password abc table ogg_amz.*; Add and start the extract using the following command from 'ggsci' command shell and ensure it starts successfully. ggsci>add extract eamz, tranlog, begin now ggsci>add exttrail./dirdat/ez, extract eamz, megabytes 50 ggsci>start extract eamz ggsci>info eamz Oracle GoldenGate Delivery The replicat parameter file is shown below. The replicate applies the changes from the on-premises to the database on the cloud. replicat rora assumetargetdefs discardfile./dirrpt/rora.dsc, purge userid gguser, password abc MAP ogg_ora.*, target ogg_amz.*; Add and start the replicat and ensure it starts successfully. 16

18 ggsci>dblogin userid gguser, password xyz ggsci>add replicat rora, exttrail./dirdat/po ggsci>start rora ggsci>info rora Configuration on the on-premises instance The configuration for Oracle GoldenGate Manager, Capture the Apply process on the on-premises instance is illustrated below. Oracle GoldenGate Manager The manager parameter file is shown below. port dynamicportlist purgeoldextracts./dirdat/*, usecheckpoints Start the manager and ensure it starts successfully. ggsci>start mgr ggsci>info mgr Oracle GoldenGate Capture Ensure, supplemental logging is enabled for the table. Supplemental logging is enabled by issuing the command 'add tranlog ogg_ora.cust_info' from the 'ggsci' command shell. The capture parameter file is shown below. extract eora tranlogoptions excludeuser gguser exttrail./dirdat/eo userid gguser, password abc table ogg_ora.*; Add and start the extract using the following command from 'ggsci' command shell and ensure it starts successfully. ` ggsci>add extract eora, tranlog, begin now ggsci>add exttrail./dirdat/eo, extract eora, megabytes 50 ggsci>start extract eora ggsci>info eora 17

19 Oracle GoldenGate Delivery The replicat parameter file is shown below. replicat ramz assumetargetdefs discardfile./dirrpt/ramz.dsc, purge userid gguser, password abc MAP ogg_amz.*, target ogg_ora.*; Add and start the replicat and ensure it starts successfully. ggsci>dblogin userid gguser, password xyz ggsci>add replicat ramz, exttrail./dirdat/pz ggsci>start ramz ggsci>info ramz Specific configuration for Oracle GoldenGate based on the options This section discusses the specific configuration based on the option selected. In this section we will discuss the specific Oracle GoldenGate Configuration on the EC2 and the On-premises instances based on the option selected. Option 1 - Secured network established between the cloud and on-premises Configuration on the EC2 instance The configuration for Oracle GoldenGate Pump process on the EC2 instance is illustrated below. Ensure you are logged in as 'oracle' (su oracle). The configuration for Oracle GoldenGate Pump process on the on-premises is illustrated below. extract pamz rmthost 'host name or IP address of on-premises instance', mgrport 17000, compress rmttrail./dirdat/po passthru table ogg_amz.*; Add and start the Pump and ensure it starts successfully 18 ggsci>add extract pamz, exttrailsource./dirdat/ez ggsci>add rmttrail./dirdat/pz, extract pamz, megabytes 50 ggsci>start pamz ggsci>info pamz

20 Configuration on the on-premises instance The configuration for Oracle GoldenGate Pump process on the on-premises is illustrated below. extract pora rmthost 'host name or IP address of EC2 instance', mgrport 15000, compress rmttrail./dirdat/po passthru table ogg_ora.*; Add and start the Pump and ensure it starts successfully ggsci>add extract pora, exttrailsource./dirdat/eo ggsci>add rmttrail./dirdat/po, extract pora, megabytes 50 ggsci>start pora ggsci>info pora Option 2 Restricted network established between the cloud and on-premises Configuration on the EC2 instance Oracle GoldenGate Passive Pump The pump parameter file is shown below. Observe in the parameter file the parameter 'rmthost' is not used. The alias extract (discussed below) started on the on-premises will dynamically configure the port which the passive Pump will use to communicate with the Oracle GoldenGate collector on the on-premises instance. extract pamz rmttrail./dirdat/pz passthru table ogg_amz.*; Add the Passive pump using the following command ggsci>add extract pamz, passive, exttrailsource./dirdat/ez ggsci>add rmttrail./dirdat/pz, extract pamz, megabytes 50 19

21 Starting and stopping of the passive pump is controlled by alias pump which is configured and running on the on the on-premises instance. Configuration on the on-premises instance Configuration of the alias-pump To create an Extract group in alias mode, use the ADD EXTRACT (Pump) command as follows from the ggsci command prompt. ggsci>add extract pamz, rmthost 'host name or IP address of EC2 instance', mgrport To start or stop Oracle GoldenGate extraction in the passive-alias Extract configuration, start or stop the alias Extract (pump) group from GGSCI as follows. ggsci>start pamz ggsci>stop pamz The status of the passive-alias extract configuration can be determined by running the following ggsci command. This will return the status of both the alias and the passive pump process. ggsci>info pamz ggsci>info pamz EXTRACT PAMZ Last Started :29 Status RUNNING ALIAS PAMZ ec2_instance:15000 Forwarding command to EXTRACT PAMZ Last Started :29 Status RUNNING Checkpoint Lag 00:00:00 (updated 00:00:09 ago) Log Read Checkpoint File./dirdat/ez :32: RBA 1245 Configure Oracle GoldenGate Pump to publish changes from on-premise to EC2 instance The pump parameter file is shown below. 20

22 extract pora rmthost 'host name or IP address of EC2 instance', mgrport 15000, compress rmttrail./dirdat/po passthru table ogg_ora.*; Add and start the pump and ensure it starts successfully ggsci>add extract pora, exttrail source./dirdat/eo ggsci>add rmttrail./dirdat/po, extract pora, megabytes 50 ggsci>start pora ggsci>info pora Option 3 - Restricted network access from on-premises and cloud through HTTP proxy Configuration on the EC2 Instance Configure Oracle GoldenGate Pump The pump parameter file is shown below. Observe in the parameter file 'localhost' is used as the value for parameter 'rmthost' this is because we have configured SSH to do port forwarding of the OGG ports. extract pamz rmthost localhost, mgrport 17000, compress rmttrail./dirdat/pz passthru table ogg_amz.*; Add and start the pump and ensure it starts successfully. ggsci>add extract pamz, exttrailsource./dirdat/ez ggsci>add rmttrail./dirdat/pz, extract pamz, megabytes 50 ggsci>start pamz ggsci>info pamz Configuration on the on-premises Instance 21

23 Configure Oracle GoldenGate Pump The pump parameter file is shown below. Observe in the parameter file 'localhost' is used as the value for parameter 'rmthost' this is because we have configured SSH to do port forwarding of the OGG ports. extract pora rmthost localhost, mgrport 15000, compress rmttrail./dirdat/po passthru table ogg_ora.*; Add and start the pump and ensure it starts successfully.. ggsci>add extract pora, exttrail source./dirdat/eo ggsci>add rmttrail./dirdat/po, extract pora, megabytes 50 ggsci>start pora ggsci>info pora Execute a Test Irrespective of the option selection, insert a record into the table on EC2 instance and On-premises instance and validate the records are replicated between the instances. Conclusion Oracle GoldenGate addresses the requirements for ensuring data consistency in a hybrid environment involving the cloud and on-premises instances thus facilitating the business process to run effectively and reliably. This paper discussed in detail the database replication between the EC2 instance and onpremises instance using Oracle GoldenGate and presented multiple options to achieve a desired objective. 22

24 23 Replicating between Cloud and On-premises using Oracle GoldenGate

25 Ensuring Data Consistency with Oracle GoldenGate Veridata July 2013 Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA U.S.A. Worldwide Inquiries: Phone: Fax: oracle.com Copyright 2013, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0113