Configuring Oracle GoldenGate OGG 11gR2 local integrated capture and using OGG for mapping and transformations

Similar documents
1. Some OGG fundamental concepts : PK/UK, KEYCOLS, ADD TRANDATA, Before and After images and they how work together

CSE 781 Data Base Management Systems, Summer 09 ORACLE PROJECT

DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT. [Docket No. FR-6090-N-01]

Panelists. Patrick Michael. Darryl M. Bloodworth. Michael J. Zylstra. James C. Green

Figure 1 Map of US Coast Guard Districts... 2 Figure 2 CGD Zip File Size... 3 Figure 3 NOAA Zip File Size By State...

Ocean Express Procedure: Quote and Bind Renewal Cargo

Telecommunications and Internet Access By Schools & School Districts

2018 NSP Student Leader Contact Form

A New Method of Using Polytomous Independent Variables with Many Levels for the Binary Outcome of Big Data Analysis

The Lincoln National Life Insurance Company Universal Life Portfolio

CostQuest Associates, Inc.

Upon completion of this lesson, you will be able to configure GoldenGate to provide Oracle to TimesTen synchronization.

Oracle GoldenGate for Oracle to Oracle

MAKING MONEY FROM YOUR UN-USED CALLS. Connecting People Already on the Phone with Political Polls and Research Surveys. Scott Richards CEO

Fall 2007, Final Exam, Data Structures and Algorithms

Distracted Driving- A Review of Relevant Research and Latest Findings

GoldenGate for Oracle to DB2 on zos

NSA s Centers of Academic Excellence in Cyber Security

2018 Supply Cheat Sheet MA/PDP/MAPD

Accommodating Broadband Infrastructure on Highway Rights-of-Way. Broadband Technology Opportunities Program (BTOP)

Team Members. When viewing this job aid electronically, click within the Contents to advance to desired page. Introduction... 2

GoldenGate for Oracle to MS SQL Server

Charter EZPort User Guide

Tina Ladabouche. GenCyber Program Manager

MERGING DATAFRAMES WITH PANDAS. Appending & concatenating Series

Global Forum 2007 Venice

Silicosis Prevalence Among Medicare Beneficiaries,

Department of Business and Information Technology College of Applied Science and Technology The University of Akron

B.2 Measures of Central Tendency and Dispersion

Presented on July 24, 2018

State IT in Tough Times: Strategies and Trends for Cost Control and Efficiency

Post Graduation Survey Results 2015 College of Engineering Information Networking Institute INFORMATION NETWORKING Master of Science

Expert Oracle GoldenGate

GoldenGate for DB2 to Oracle

Getting started. Amardeep Sidhu

IT Modernization in State Government Drivers, Challenges and Successes. Bo Reese State Chief Information Officer, Oklahoma NASCIO President

Name: Business Name: Business Address: Street Address. Business Address: City ST Zip Code. Home Address: Street Address

AASHTO s National Transportation Product Evaluation Program

Moonv6 Update NANOG 34

Best Practices in Rapid Deployment of PI Infrastructure and Integration with OEM Supplied SCADA Systems

Contact Center Compliance Webinar Bringing you the ANSWERS you need about compliance in your call center.

THE LINEAR PROBABILITY MODEL: USING LEAST SQUARES TO ESTIMATE A REGRESSION EQUATION WITH A DICHOTOMOUS DEPENDENT VARIABLE

2015 DISTRACTED DRIVING ENFORCEMENT APRIL 10-15, 2015

Prizm. manufactured by. White s Electronics, Inc Pleasant Valley Road Sweet Home, OR USA. Visit our site on the World Wide Web

Oracle GoldenGate for DB2 to Oracle

Jurisdictional Guidelines for Accepting a UCC Record Presented for Filing 2010 Amendments & the 2011 IACA Forms

A Capabilities Presentation

On All Forms. Financing Statement (Form UCC1) Statutory, MARS or Other Regulatory Authority to Deviate

The Outlook for U.S. Manufacturing

PulseNet Updates: Transitioning to WGS for Reference Testing and Surveillance

Presentation Outline. Effective Survey Sampling of Rare Subgroups Probability-Based Sampling Using Split-Frames with Listed Households

ACCESS PROCESS FOR CENTRAL OFFICE ACCESS

GoldenGate: An Introduction

Sideseadmed (IRT0040) loeng 4/2012. Avo

GOLDENGATE A LIVE INTRODUCTION PATRICK HURLEY

Oracle GoldenGate for Sybase to Sybase on UNIX

HP NonStop SQL/MX to Oracle

State HIE Strategic and Operational Plan Emerging Models. February 16, 2011

Homework Assignment #5

Amy Schick NHTSA, Occupant Protection Division April 7, 2011

enterprise professional expertise distilled Implementer's guide Oracle GoldenGate 11g

Oracle GoldenGate Performance Best Practices O R A C L E W H I T E P A P E R M A Y

CMPE 180A Data Structures and Algorithms in C++ Spring 2018

DSC 201: Data Analysis & Visualization

Defining Multiple Replicat Processes to Increase GoldenGate Performance

Touch Input. CSE 510 Christian Holz Microsoft Research February 11, 2016

Oracle GoldenGate Director

Oracle GoldenGate 11g Release 2 ( ) Release Notes E

How to Make an Impressive Map of the United States with SAS/Graph for Beginners Sharon Avrunin-Becker, Westat, Rockville, MD

Using a Probabilistic Model to Assist Merging of Large-scale Administrative Records

GoldenGate for DB2 on zos to DB2 on UNIX

Oracle GoldenGate for MS SQL Server 2005

Presentation to NANC. January 22, 2003

DTFH61-13-C Addressing Challenges for Automation in Highway Construction

Expanding Transmission Capacity: Options and Implications. What role does renewable energy play in driving transmission expansion?

Steve Stark Sales Executive Newcastle

2013 Product Catalog. Quality, affordable tax preparation solutions for professionals Preparer s 1040 Bundle... $579

CIS 467/602-01: Data Visualization

Strengthening connections today, while building for tomorrow. Wireless broadband, small cells and 5G

Geographic Accuracy of Cell Phone RDD Sample Selected by Area Code versus Wire Center

Oracle 1Z0-539 Exam Questions & Answers

GoldenGate for NonStop SQL/MP to Oracle

ORCHESTRATING ORACLE GOLDENGATE MICROSERVICES USING PL/SQL

Oracle GoldenGate 11g Fundamentals for Oracle

Selling Compellent Hardware: Controllers, Drives, Switches and HBAs Chad Thibodeau

Deploy Oracle GoldenGate Veridata Version 3.0 for comparison between Oracle RDBMS databases replicated using OGG 11gR2

Oracle GoldenGate 12c Tips and Tricks. Infrastructure at your Service.

Medium voltage Marketing contacts

MIS2502: Review for Exam 2. Jing Gong

Oracle Fusion Middleware Upgrading Oracle GoldenGate for Windows and UNIX. 18c (18.1.0)

Data Replication With Oracle GoldenGate Looking Behind The Scenes Robert Bialek Principal Consultant Partner

What Did You Learn? Key Terms. Key Concepts. 68 Chapter P Prerequisites

STATE DATA BREACH NOTIFICATION LAWS OVERVIEW OF REQUIREMENTS FOR RESPONDING TO A DATA BREACH UPDATED JUNE 2017

ASR Contact and Escalation Lists

Oracle GoldenGate 11g: Advanced Configuration for Oracle Student Guide

Teradata to Teradata: Max Performance

Wechsel von Oracle Streams. nach Oracle GoldenGate 12c. Joachim Jaensch Principal Sales Consultant

GoldenGate for NonStop SQL/MP to SQL Server

Real World Algorithms: A Beginners Guide Errata to the First Printing

ARE WE HEADED INTO A RECESSION?

Transcription:

Configuring Oracle GoldenGate OGG 11gR2 local integrated capture and using OGG for mapping and transformations In the article you will have a look at an OGG configuration example for local integrated capture and discover how to deploy some of the powerful OGG features related to data mapping and transformation. Starting from OGG 11gR2 Oracle implemented an integrated capture, in addition to the already existing transaction log capture now a day called classic capture, only from Oracle RDBMS data sources by integrating the OGG extract with the database logmining server. OGG mapping and transformation is a feature that works with both classic and integrated capture enabling complex transactions to be replicated across heterogeneous environments. I will cover each topic separately in the following sub-sections. Example of local integrated capture using OGG 11gR2 Example of mapping and transformation using OGG 11gR2 The article is based on OGG 11gR2 environment configuration previously described here. Example of local integrated capture using OGG The integrated capture mode introduced in OGG 11gR2 benefits from the integration with the existing Oracle Streams API. In integrated capture mode, the OGG extracts interact with the Oracle logmining server to capture data changes as a logical change records (LCR). The database where the Oracle logmining server is running must be 11.2.0.3 with patch as per MOS Doc ID 1411356.1 for integrated capture mode to handle almost all objects and data types otherwise there are restrictions. My source test database RACD and my target database RACDB are both 11.2.0.3 and patched as per the requirements. The integrated capture can be as follows depending on where the logminig server is running. Local integrated capture logminig server is running on the database where the transactions are executed and captured Downstream integrated capture - logminig server is running on a different database from the database where the transactions are executed. The database running the transactions is configured to ship the redo and archivelogs to the mining database. Useful Oracle sources are Oracle Installation and Setup Guide from the official Oracle OGG documentation accessible here and MOS notes (Best Practice - Oracle GoldenGate 11gr2 integrated extract and Oracle Data Guard - Switchover/Fail-over Operations Version 1.1 Document ID: 1436913.1 and Oracle GoldenGate Best Practices: Configuring Downstream Integrated Extract Document ID: 1485620.1). Integrated capture mode in OGG 11gR2 enables you to process Oracle database objects and data types that cannot be processed in the classing capture mode. For a detailed list of Oracle database objects and data types that can be

processed in integrated/classic capture mode look at Oracle Installation and Setup Guide. In this section I will show an example of configuring a local integrated capture. In a separate blog post I will show an example of a downstream integrated capture. The following tables will be created on both source and target databases for illustration of OGG mapping and transformation while replicating data from the source database to the target database. The source objects are in test3 schema in RACD database. The target objects are in the test3_t schema in the RACDB database. I will configure extract extint on the source and replicat repint on the target. The tables listed below will be created on both source and target. create table customer ( NID varchar2(10) primary key, Name varchar2(400), Address_street varchar2(400), Adress_city Tel state_code zip varchar2(50), varchar2(4), char(2), char(9) ); create table customer_detail ( customer_id number(10) primary key, First_name varchar2(40), Last_name NID Tel_country Tel_area Tel_number Country varchar2(40), varchar2(10), varchar2(3), varchar2(3), varchar2(8), varchar2(30), Address_street varchar2(400), address_city Tel_c_code State_name varchar2(50), varchar2(4), varchar2(50),

zip1 varchar2(5), zip2 varchar2(5) ); create table customer_detail1 ( customer_id number(10) primary key, First_name varchar2(40), Last_name NID Tel_country Tel_area Tel_number Country varchar2(40), varchar2(10), varchar2(3), varchar2(3), varchar2(8), varchar2(30), Address_street varchar2(400), address_city Tel_c_code State_name zip1 varchar2(50), varchar2(4), varchar2(50), varchar2(5), zip2 varchar2(5) ); This section will provide an overview of the steps to configure a local integrated extract. The steps are executed in order as follows: 1. Grant permissions to the user (for local integrated capture one user will suffice although you can use two dedicated users). 2. While logged in with dblogin add trandata for the source tables. 3. Register extract for integrated capture while logged in with dblogin 4. Create an extract for integrated capture 5. Create a source and target definition file using DEFGEN. 6. Create a parameter file for the extract using local integrated capture 7. Create a trail for the extract 8. Create a replicat 9. Start OGG extract and replicat 10. Troubleshooting tips/tricks for extract 1. Grant permissions to the extract user I am using the same OGG_EXTRACT user for extraction as defined here for OGG to interact to both source Oracle database and logmining server running on the source database. Grant privileges that required for the user to interact

with the Oracle logmining server by executing the following SQL statement while logged in to the source database using as sysdba. SQL> exec DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE (grantee=>'ogg_extract', privilege_type=>'capture',grant_select_privileges=>true, do_grants=>true); SQL> PL/SQL procedure successfully completed. SQL> 2. Add trandata on the source database tables that will be replicated. GGSCI (raclinux1.gj.com) 6> dblogin userid ogg_extract, password ogg_extract Successfully logged into database. GGSCI (raclinux1.gj.com) 8> add trandata test3.state Logging of supplemental redo data enabled for table TEST3.STATE. GGSCI (raclinux1.gj.com) 9> add trandata test3.customer Logging of supplemental redo data enabled for table TEST3.CUSTOMER. GGSCI (raclinux1.gj.com) 10> add trandata test3.customer_detail Logging of supplemental redo data enabled for table TEST3.CUSTOMER_DETAIL. GGSCI (raclinux1.gj.com) 11> GGSCI (raclinux1.gj.com) 28> add schematrandata test3 2012-10-25 13:29:20 INFO OGG-01788 SCHEMATRANDATA has been added on schema test3. GGSCI (raclinux1.gj.com) 29> 3. Register the extract for integrated capture GGSCI (raclinux1.gj.com) 29> dblogin userid ogg_extract, password ogg_extract Successfully logged into database. GGSCI (raclinux1.gj.com) 30> register extract extint database 2012-10-25 13:31:09 WARNING OGG-02064 Oracle compatibility version 11.2.0.0.0 has limited datatype support for integrated capture. Version 11.2.0.3 required for full support.

2012-10-25 13:32:26 INFO OGG-02003 Extract EXTINT successfully registered with database at SCN 4629827. GGSCI (raclinux1.gj.com) 31> 4. Create a source and a target definition files using DEFGEN utility Create a parameter file for generating the source definition file on the source system. GGSCI (raclinux1.gj.com) 4> view params defgensource defsfile./dirdef/source_int.def, purge userid ddl_ogg, password ddl_ogg table test3.state; table test3.customer; table test3.customer_detail; GGSCI (raclinux1.gj.com) 5> Run defgen utility from OGG installation directory on the source system./defgen paramfile./dirprm/defgensource.prm Create parameter file for generating the target definition file on the target system. GGSCI (raclinux1.gj.com) 5> view params defgentarget defsfile./dirdef/target_int.def, purge userid ddl_ogg, password ddl_ogg table test3_t.state; table test3_t.customer; table test3_t.customer_detail; table test3_t.customer_detail1; GGSCI (raclinux1.gj.com) 6> Run defgen utility from the OGG installation directory on the target system../defgen paramfile./dirprm/defgentarget.prm Copy the source definition file to the target. Copy the target definition file to the source. 5. Create the extract

From the Reference Guide you will see that in case or RAC database with local integrated capture you should not specify threads. In practice you will not be allowed to do so. GGSCI (raclinux1.gj.com) 33> add extract extint, integrated tranlog, begin now, threads 2 ERROR: THREADS is not allowed for INTEGRATED extract. GGSCI (raclinux1.gj.com) 34> add extract extint, integrated tranlog, begin now, threads 1 ERROR: THREADS is not allowed for INTEGRATED extract. GGSCI (raclinux1.gj.com) 35> add extract extint, integrated tranlog, begin now EXTRACT added. GGSCI (raclinux1.gj.com) 36> 6. Create the parameter file for extract and replicat The extract and replicat parameter files are in the appendix. Here I will list the parameter required to set local integrated capture extract. TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 700, parallelism 3) userid ogg_extract, password ogg_extrac 7. Create a trail Create a remote trail from GGSCI. GGSCI (raclinux1.gj.com) 36> add rmttrail./dirdat/2x, extract extint, megabytes 20 RMTTRAIL added. GGSCI (raclinux1.gj.com) 37> 8. Create a replicat Look at the appendix for the replicat parameter file. Create a replicat from GGSCI. GGSCI (raclinux1.gj.com) 8> add replicat repint, exttrail./dirdat/2x REPLICAT added. GGSCI (raclinux1.gj.com) 9> 9. Start OGG extract and replicat Start extract and replicat as usual using the OGG start extract and start replicat commands. GGSCI (raclinux1.gj.com) 6> start extract extint

Sending START request to MANAGER... EXTRACT EXTINT starting GGSCI (raclinux1.gj.com) 7> GGSCI (raclinux1.gj.com) 7> start replicat repint Sending START request to MANAGER... REPLICAT REPINT starting GGSCI (raclinux1.gj.com) 8> Note the difference between integrated extract extint compared to a classic extract extt1. GGSCI (raclinux1.gj.com) 11> info extract extint EXTRACT EXTINT Last Started 2012-10-25 15:37 Status RUNNING Checkpoint Lag 00:00:00 (updated 01:45:42 ago) Log Read Checkpoint Oracle Integrated Redo Logs 2012-10-25 13:55:43 SCN 0.0 (0) GGSCI (raclinux1.gj.com) 12> GGSCI (raclinux1.gj.com) 12> info extract extt1 EXTRACT EXTT1 Last Started 2012-10-25 15:37 Status RUNNING Checkpoint Lag 00:00:00 (updated 110:37:09 ago) Log Read Checkpoint Oracle Redo Logs 2012-10-21 01:05:14 Thread 1, Seqno 276, RBA 9411088 SCN 0.4572497 (4572497) Log Read Checkpoint Oracle Redo Logs 2012-10-13 15:48:49 Thread 2, Seqno 0, RBA 0 SCN 0.0 (0) GGSCI (raclinux1.gj.com) 13> 10. Troubleshooting tips and tricks Initially, without proper configuration I faced the errors specified in the appendix. The following helped to work around the problems Commented --tranlogoptions asmuser sys@asm, asmpassword sys1 in order to resolve OGG-02000 Ignoring option ASMUSER because it is incompatible with an integrated capture configuration and OGG-02000 Ignoring option ASMPASSWORD because it is incompatible with an integrated capture configuration. Configured the database with the following parameters and added OGG parameter TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 700, parallelism 3). Note that the source database is 11.2.0.3.

alter system set memory_max_target=3g scope=spfile sid='*'; alter system set memory_target=3g scope=spfile sid='*'; alter system set streams_pool_size=1g scope=spfile sid='*'; alter system set compatible='11.2.0.3.0' scope=spfile sid='*'; This helped avoiding the following errors 2012-10-25 16:29:57 WARNING OGG-00869 ORA-00600: internal error code, arguments: [KGHFRE3], [0x08AB26040], [], [], [], [], [], [], [], [], [], [] ORA-26914: Unable to communicate with GoldenGate capture process "OGG$CAP_EXTINT" from outbound server "OGG$EXTINT". 2012-10-25 16:29:04 WARNING OGG-02045 Database does not have streams_pool_size initialization parameter configured. 2012-10-25 16:29:02 WARNING OGG-02064 Oracle compatibility version 11.2.0.0.0 has limited datatype support for integrated capture. Version 11.2.0.3 required for full support. 2012-10-25 17:30:27 WARNING OGG-02051 Not enough database memory to service Extract in integrated capture mode. 2012-10-25 17:32:52 WARNING OGG-01223 TCP/IP error 111 (Connection refused). Example of mapping and transformation using OGG 11gR2 OGG mapping and transformation features enable replication across dissimilar heterogeneous table structures. OGG SQLEXEC allows execution of SQL on both extract and replicat sites. In this case SQLEXEC will be used for table lookups. OGG allows mapping to be done on the source system using the extract TABLE parameter or on the target system using the replicat MAP parameter. For all mappings on the target using COLMAP and transformations use a source definition file specified with the SOURCEDEFS parameter. For all mappings on the source using COLMAP and transformations use a target definition file specified with the TARGETDEFS parameter. Source and target tables are identical if: Tables have same column names Columns of the names are of the same type Column order is the same in source and target columns If source and target tables are not identical use the SOURCEDEFS/TARGETDEFS parameters in extract/replicat parameter file and specify data definition files generated with defgen see the 1.4.

I will illustrate it using the following tables. SQL> desc customer Name Null? Type ----------------------------------------- -------- ---------------------------- NID NAME ADDRESS_STREET ADRESS_CITY TEL STATE_CODE ZIP NOT NULL VARCHAR2(10) VARCHAR2(400) VARCHAR2(400) VARCHAR2(50) VARCHAR2(40) VARCHAR2(3) VARCHAR2(12) SQL> SQL> desc customer_detail Name Null? Type ----------------------------------------- -------- ---------------------------- CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL_COUNTRY TEL_AREA TEL_NUMBER COUNTRY ADDRESS_STREET ADDRESS_CITY NOT NULL NUMBER(10) VARCHAR2(40) VARCHAR2(40) VARCHAR2(10) VARCHAR2(3) VARCHAR2(3) VARCHAR2(8) VARCHAR2(30) VARCHAR2(400) VARCHAR2(50)

TEL_C_CODE STATE_NAME ZIP1 ZIP2 VARCHAR2(4) VARCHAR2(50) VARCHAR2(5) VARCHAR2(5) SQL> SQL> desc state Name Null? Type ----------------------------------------- -------- ---------------------------- STATE_ID STATE_DESC STATE_CAPITAL NOT NULL CHAR(2) VARCHAR2(50) VARCHAR2(50) SQL> In the example state table is replicated as it is. Customer table from the source is mapped to the customer_detail and customer_detail1 on the target. The mappings and the transformations from customer to customer_detail[1] are as follows: Source table column Target Table column Transformation CUSTOMER_ID NID Make it a number FIRST_NAME NAME Extracts the first name LAST_NAME NAME Extracts the surname NID NID None. NID=NID TEL_COUNTRY TEL +1 if state_code is in US TEL_AREA TEL Extracts first 3 characters from TEL TEL_NUMBER TEL Extracts from 4 th position to the 10 th position from TEL COUNTRY STATE_CODE USA if state_code is in USA ADDRESS_STREET ADDRESS_STREET NONE ADDRESS_CITY ADRESS_CITY NONE TEL_C_CODE STATE_CODE +1 if state_code is in US STATE_NAME STATE_CODE Gets STATE_DESK using SQLEXEC lookup using state table ZIP1 ZIP Extracts first 5 characters from ZIP ZIP2 ZIP Extracts from 6 th position to the 10 th position from ZIP

The logical mapping above is implemented at OGG level using the underlined parameters from the extract extint and replicat repint parameter files shown bellow with abbreviations for the sake of clarity. Note that there are two mappings: customer -> customer_detail : mapping is done in the replicat customer -> customer_detail1 : mapping is done in the extract and replicat. This is an illustration how to do mapping and transformation on the extract in case the target is Tandem/NSK /HP SQL/MX based system. The complete extract and replicat parameter files are in the appendix. GGSCI (raclinux1.gj.com) 1> view params extint extract extint SETENV (ORACLE_SID = "RACD1") --tranlogoptions asmuser sys@asm, asmpassword sys1 TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 700, parallelism 3) targetdefs./dirdef/target_int.def userid ogg_extract, password ogg_extract rmthost raclinux1, mgrport 7809 rmttrail./dirdat/2x table test3.state; table test3.customer; table test3.customer_detail; table test3.customer target test3_t.customer_detail1, sqlexec( id statelookup, query " select state_desc from test3.state where state_id = :state_code", PARAMS(state_code=state_code)), colmap( usedefaults, customer_id = @NUMSTR(NID), First_name = @STREXT(Name,1,@STRFIND(Name," ")), Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)), NID = NID, Tel_country = @IF (@VALONEOF(state_code,"AL","AK",... ), "+1", "+44"),

Tel_area = @STREXT (Tel, 1, 3), Tel_number = @STREXT (Tel, 4, 10), country = @IF (@VALONEOF(state_code,"AL","AK",... ), "USA", "UK"), Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", "...), "+1", "+44"), State_name = @GETVAL(statelookup.state_desc), zip1 = @strext(zip,1,5), zip2 = @strext(zip,6,10));; GGSCI (raclinux1.gj.com) 2> view params repint replicat repint --reperror(default,ignore) SETENV (ORACLE_SID = "RACDB1") userid ogg_replicat, password ogg_replicat sourcedefs./dirdef/source_int.def discardfile./dirrpt/repint.dsc, purge map test3.state, target test3_3.state; map test3.customer, target test3.customer; map test3.customer, target test3_t.customer_detail1, sqlexec( id statelookup1, query " select state_desc from test3_t.state where state_id = :state_code", PARAMS(state_code=state_code)), colmap( usedefaults, ADDRESS_CITY = ADRESS_CITY, customer_id = @NUMSTR(NID), First_name = @STREXT(Name,1,@STRFIND(Name," ")), Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)), NID = NID, Tel_country = @IF (@VALONEOF(state_code,"AL","AK",... ), "+1", "+44"), Tel_area = @STREXT (Tel, 1, 3), Tel_number = @STREXT (Tel, 4, 10),

country = @IF (@VALONEOF(state_code,"AL","AK",...), "USA", "UK"), Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK",..."), "+1", "+44"), State_name = @GETVAL(statelookup1.state_desc), zip1 = @strext(zip,1,5), zip2 = @strext(zip,6,10)); map test3.customer, target test3_t.customer_detail, sqlexec( id statelookup, query " select state_desc from test3_t.state where state_id = :state_code", PARAMS(state_code=state_code)), colmap( usedefaults, ADDRESS_CITY = ADRESS_CITY, customer_id = @NUMSTR(NID), First_name = @STREXT(Name,1,@STRFIND(Name," ")), Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)), NID = NID, Tel_country = @IF (@VALONEOF(state_code,"AL","AK",...), "+1", "+44"), Tel_area = @STREXT (Tel, 1, 3), Tel_number = @STREXT (Tel, 4, 10), country = @IF (@VALONEOF(state_code,"AL","AK",...), "USA", "UK"), Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK",...), "+1", "+44"), State_name = @GETVAL(statelookup.state_desc), zip1 = @strext(zip,1,5), zip2 = @strext(zip,6,10)); Let s test it. Insert into test3.customer and observe the transformations on the target. insert into test3.customer values('222222222','john Smith','93 Some street','dallas','5555555555','tx','75209-5555');

insert into test3.customer values('333333333','alejandro García', '46 Some street','wayne','6666666666','nj','07470-5555'); insert into test3.customer values('444444444','frank Meyer','56 Some street','los Angeles','7777777777','CA','90018-5555'); commit; on the target we have the following result: SQL> select * from customer_detail; CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2 ----------- -------------------- -------------------- ---------- --- -- - -------- ------ ----------------- ---------- ---- ---------- ----- -- --- 222222222 John Smith 222222222 +1 555 5555555 USA 93 Some street Dallas +1 Texas 75209-5555 333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA 46 Some street Wayne +1 New Jersey 07470-5555 444444444 Frank Meyer 444444444 +1 777 7777777 USA 56 Some street Los Angele +1 California 90018-5555 s SQL> select * from customer_detail1;

CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2 ----------- -------------------- -------------------- ---------- --- -- - -------- ------ ----------------- ---------- ---- ---------- ----- -- --- 222222222 John Smith 222222222 +1 555 5555555 USA 93 Some street Dallas +1 Texas 75209-5555 333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA 46 Some street Wayne +1 New Jersey 07470-5555 444444444 Frank Meyer 444444444 +1 777 7777777 USA 56 Some street Los Angele +1 California 90018-5555 s SQL> Perform an update on the source update test3.customer set TEL='8888888888' where NID='222222222'; Verify the result on the target SQL> select * from customer_detail; CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2 ----------- -------------------- -------------------- ---------- --- --- -------- ------ ----------------- ---------- ---- ---------- ----- ----- 222222222 John Smith 222222222 +1 888 8888888 USA 93 Some street Dallas +1 Texas 75209-5555 333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA 46 Some street Wayne +1 New Jersey 07470-5555

444444444 Frank Meyer 444444444 +1 777 7777777 USA 56 Some street Los Angele +1 California 90018-5555 s SQL> select * from customer_detail1; CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2 ----------- -------------------- -------------------- ---------- --- --- -------- ------ ----------------- ---------- ---- ---------- ----- ----- 222222222 John Smith 222222222 +1 888 8888888 USA 93 Some street Dallas +1 Texas 75209-5555 333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA 46 Some street Wayne +1 New Jersey 07470-5555 444444444 Frank Meyer 444444444 +1 777 7777777 USA 56 Some street Los Angele +1 California 90018-5555 s SQL> Delete a record from the source delete from test3.customer where NID='222222222'; Observe the result on the target SQL> select * from customer_detail; CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2 ----------- -------------------- -------------------- ---------- --- --- -------- ------ ----------------- ---------- ---- ---------- ----- ----- 333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA 46 Some street Wayne +1 New Jersey 07470-5555 444444444 Frank Meyer 444444444 +1 777 7777777 USA 56 Some street Los Angele +1 California 90018-5555 s SQL> select * from customer_detail;

CUSTOMER_ID FIRST_NAME LAST_NAME NID TEL TEL TEL_NUMB COUNTR ADDRESS_STREET ADDRESS_CI TEL_ STATE_NAME ZIP1 ZIP2 ----------- -------------------- -------------------- ---------- --- --- -------- ------ ----------------- ---------- ---- ---------- ----- ----- 333333333 Alejandro Garc??a 333333333 +1 666 6666666 USA 46 Some street Wayne +1 New Jersey 07470-5555 444444444 Frank Meyer 444444444 +1 777 7777777 USA 56 Some street Los Angele +1 California 90018-5555 s SQL> The test concluded that the replication is successful from the source to the target and that OGG performs the mapping and transformation as expected. Note that on the target you have Garc??a instead of 'Alejandro García' due to character conversion. Summary In the article you had a look at the steps to configure and troubleshoot OGG using local integrated capture extract. You had a glimpse at an example utilizing OGG SQLEXEC and OGG functions for mapping and data transformation for replication across dissimilar table structures. Appendix GGSCI (raclinux1.gj.com) 1> view params extint extract extint SETENV (ORACLE_SID = "RACD1") --tranlogoptions asmuser sys@asm, asmpassword sys1 TRANLOGOPTIONS INTEGRATEDPARAMS (max_sga_size 700, parallelism 3) targetdefs./dirdef/target_int.def userid ogg_extract, password ogg_extract rmthost raclinux1, mgrport 7809

rmttrail./dirdat/2x table test3.state; table test3.customer; table test3.customer_detail; table test3.customer target test3_t.customer_detail1, sqlexec( id statelookup, query " select state_desc from test3.state where state_id = :state_code", PARAMS(state_code=state_code)), colmap( usedefaults, customer_id = @NUMSTR(NID), First_name = @STREXT(Name,1,@STRFIND(Name," ")), Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)), NID = NID, Tel_country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA ", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"), Tel_area = @STREXT (Tel, 1, 3), Tel_number = @STREXT (Tel, 4, 10), country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "USA", "UK"), Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "K Y", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"), State_name = @GETVAL(statelookup.state_desc), zip1 = @strext(zip,1,5), zip2 = @strext(zip,6,10));;

GGSCI (raclinux1.gj.com) 2> view params repint replicat repint --reperror(default,ignore) SETENV (ORACLE_SID = "RACDB1") userid ogg_replicat, password ogg_replicat sourcedefs./dirdef/source_int.def discardfile./dirrpt/repint.dsc, purge map test3.state, target test3_3.state; map test3.customer, target test3.customer; map test3.customer, target test3_t.customer_detail1, sqlexec( id statelookup1, query " select state_desc from test3_t.state where state_id = :state_code", PARAMS(state_code=state_code)), colmap( usedefaults, ADDRESS_CITY = ADRESS_CITY, customer_id = @NUMSTR(NID), First_name = @STREXT(Name,1,@STRFIND(Name," ")), Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)), NID = NID, Tel_country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"), Tel_area = @STREXT (Tel, 1, 3), Tel_number = @STREXT (Tel, 4, 10), country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "L A", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "USA", "UK"),

Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "S C", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"), State_name = @GETVAL(statelookup1.state_desc), zip1 = @strext(zip,1,5), zip2 = @strext(zip,6,10)); map test3.customer, target test3_t.customer_detail, sqlexec( id statelookup, query " select state_desc from test3_t.state where state_id = :state_code", PARAMS(state_code=state_code)), colmap( usedefaults, ADDRESS_CITY = ADRESS_CITY, customer_id = @NUMSTR(NID), First_name = @STREXT(Name,1,@STRFIND(Name," ")), Last_name = @STREXT(Name,@STRFIND(Name," "),@STRLEN(NAme)), NID = NID, Tel_country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"), Tel_area = @STREXT (Tel, 1, 3), Tel_number = @STREXT (Tel, 4, 10), country = @IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "L A", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "USA", "UK"), Tel_c_code =@IF (@VALONEOF(state_code,"AL","AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", "HI", "ID", "IL", "IN", "IA", "KS", "KY", "LA", "ME", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "OH", "OK", "OR", "PA", "RI", "S C", "SD", "TN", "TX", "UT", "VT", "VA", "WA", "WV", "WI", "WY"), "+1", "+44"), State_name = @GETVAL(statelookup.state_desc),

zip1 = @strext(zip,1,5), zip2 = @strext(zip,6,10)); GGSCI (raclinux1.gj.com) 3> Errors resolved during the configuration 2012-10-25 16:29:02 INFO OGG-02000 Ignoring option ASMUSER because it is incompatible with an integrate d capture configuration. 2012-10-25 16:29:02 INFO OGG-02000 Ignoring option ASMPASSWORD because it is incompatible with an integ rated capture configuration. 2012-10-25 16:29:02 WARNING OGG-02064 Oracle compatibility version 11.2.0.0.0 has limited datatype support for integrated capture. Version 11.2.0.3 required for full support. 2012-10-25 16:29:04 WARNING OGG-02045 Database does not have streams_pool_size initialization parameter co nfigured. 2012-10-25 16:29:18 INFO OGG-02036 Integrated capture successfully attached to logmining server OGG$EXT INT. 2012-10-25 16:29:30 INFO OGG-01226 Socket buffer size set to 27985 (flush size 27985). 2012-10-25 16:29:30 INFO OGG-01055 Recovery initialization completed for target file./dirdat/2x000004, at RBA 1097. 2012-10-25 16:29:30 INFO OGG-01478 Output file./dirdat/2x is using format RELEASE 11.2.

2012-10-25 16:29:30 INFO OGG-01026 Rolling over remote file./dirdat/2x000005. 2012-10-25 16:29:30 INFO OGG-01053 Recovery completed for target file./dirdat/2x000005, at RBA 1036. 2012-10-25 16:29:30 INFO OGG-01057 Recovery completed for all targets. *********************************************************************** ** Run Time Messages ** *********************************************************************** 2012-10-25 16:29:57 WARNING OGG-00869 ORA-00600: internal error code, arguments: [KGHFRE3], [0x08AB26040], [], [], [], [], [], [], [], [], [], [] ORA-26914: Unable to communicate with GoldenGate capture process "OGG$CAP_EXTINT" from outbound server "OGG$ EXTINT". References 1. Best Practice - Oracle GoldenGate 11gr2 integrated extract and Oracle Data Guard - Switchover/Fail-over Operations Version 1.1 Document ID: 1436913.1 2. Oracle GoldenGate Best Practices: Configuring Downstream Integrated Extract Document ID: 1485620.1 3. Oracle GoldenGate Oracle Installation and Setup Guide Release 11.2.1 E35957-01 Oracle Installation and Setup Guide 4. OGG Administrator's Guide 5. OGG Reference Guide