Oracle GoldenGate. Frank Bommarito. RMOUG Training Days February 11-13, 2013

Transcription

1 Oracle GoldenGate Frank Bommarito RMOUG Training Days February 11-13, 2013

2 About DBAK Oracle Database, Technology and E-Business Suite applications Co-founded in 2005 Colorado owned and operated Average 15 Years of Oracle Expertise Awards Top 250 Private Companies, 2011 and 2012 CoBIZ Magazine Fastest-Growing Private Companies 2012 Finalist, Denver Business Journal Colorado Companies to Watch 2012 Finalist, Colorado Office of Economic Development and International Trade Emerging Business of the Year, 2008 South Metro Denver Chamber of Commerce 100+ Clients 200+ Implementations, Upgrades, Conversions, Support Projects Oracle Gold Partner Oracle Accelerate Solution Hosted Financials Specialized - Oracle Database - Oracle Enterprise Manager 2

3 Agenda Topology High Level Overview GoldenGate updates Misc Stuff Questions and Wrap up 3

4 Who am I? Frank Bommarito, DBAK Co-founder and Chief Technology Officer of DBAK, Frank is an expert in designing, delivering, tuning, and troubleshooting advanced technology, large-scale, highly complex systems using Oracle technology. With more than 20 years of Oraclebased experience, including Oracle Database, Oracle Applications, Oracle Application Server, and Oracle's development tools, Frank has participated in implementing, training, and documenting solutions that include high availability, disaster recovery, data warehousing, online transaction processing, and process automation. Frank is a frequent speaker at industry conferences including OAUG and Oracle OpenWorld. 4

5 How It Works: Modular Architecture Capture: Committed changes are captured (and can be filtered) as they occur by reading the transaction logs. Trail files: Stages and queues data for routing. Pump: Distribute data for routing to multiple targets. Route: Data is compressed, encrypted for routing to targets. Delivery: Applies data with transaction integrity, transforming the data as required. Capture Source Trail Pump Pump LAN / WAN / Internet (TCP/IP) Target Trail Delivery Source Database(s) Delivery Target Trail Bi-directional Source Trail Capture Target Database(s) 5

6 Heterogeneous Platforms Databases Capture: Oracle DB2 Microsoft SQL Server Sybase ASE Teradata Enscribe SQL/MP SQL/MX O/S and Platforms Windows 2000, 2003, XP Linux Sun Solaris HP NonStop HP-UX HP TRU64 HP OpenVMS IBM AIX IBM z/os Delivery: All listed above, plus: HP Neoview, Netezza, Greenplum, and any ODBC compatible databases ETL products JMS message queues MySQL TimesTen 6

7 Why GoldenGate? Fast Efficient Easy to troubleshoot Easy to implement Reality Check it is software 7

8 Common Issues Long Running Jobs Network outages New tables Validation of data DDL Logging 8

9 Simple Setup Install Software (download and unzip done!) Configure in the database user to read/write data Few system settings no downtime typically alter database archivelog; alter database force logging; alter system set recyclebin=off scope=both; alter database add supplemental log data; Create manager Configure Extract/Replicat Done! 9

10 How to start? RMAN Backup/Restore Initial Load (GG Process) Export/Import CTAS 10

11 What do I get? Setup TNS to take advantage in case of failure OS Upgrade no problem DB Upgrade no problem Platform switch no problem Auditing of data no problem DDL (Create table) no problem Troubleshooting very clear Learning to ride a bike took some time with years of pleasure why not take the bridge! 11

12 What GoldenGate is not A database backup (physical versus logical) (Veridata) Tied to Oracle Tied to a specific OS In need of extensive maintenance Hard to operationally control Hard to master 12

13 Can I? Use one database with many destinations? Have one destination with from many sources? Use one source for many purposes (i.e. Failover/HA/Auditing) Use a source differently than the target? Eliminate other tools (ETL data guard backups)? 13

14 What about the others? GoldenGate versus Streams GoldenGate versus Data Guard GoldenGate versus SharePlex Summary it is a single tool that does all of these things and does not have the negative view that some of these have. 14

15 Where to use? Low RTO/RPO Want to maximize usage of D/R Need to offload without complexity Have a need for the MAA Want to try new performance techniques without a solid commitment Want to try upgrades to see what occurs 15

16 What is so different? Relatively Non-intrusive Low impact Sub-second latency Open/Heterogeneous Modular access Maintains transactional integrity Not HA not DR not CA it is all three 16

17 Am I on an Island? Often used with Standby Often used with Other ETL tools Often used with RMAN/Tape backups Often used with storage solutions The nice kid that places well with others Copies the tables create the views local 17

18 Miscellaneous Stuff 9.2 and greater for Oracle Most data types but a few exceptions 256 bit encryption of all data Blowfish is used Initial load can use a where clause (SQLPREDICATE) Active/Active must EXCLUDE the GG_MANAGER user Oracle Product but no need for Oracle 18

19 Any GUI? GG Management Pack Define, configure, manage, and report on Oracle GoldenGate processes deployed across your business Utilizes Oracle GoldenGate Management Services architecture Suffers from a few issues getting better every week. Key features: Centralized management of any number of Oracle GoldenGate processes Rich-client and Web-based interfaces for increased visibility Alert notifications, integration with 3rd-party monitoring products Real-time feedback on Oracle GoldenGate processes Zero-impact implementation without affecting performance 19

20 Veridata warm and fuzzy High-speed data comparison solution Audits, identifies, and reports data discrepancies between in-use databases Benefits: Reduce risk exposure and act on data discrepancies sooner Simplify comparison No disruption to business systems Easy to install not really as fast as one would hope. Key features: Support for large data volumes Comparisons ran while data sources are kept online Unparalleled speed and efficiency Selective comparison options Unattended execution Flexible reporting 20

21 Reality Check Problem Tables GGS_MARKER and GGS_DDL_HIST grow forever when DDL support is enabled Solution Add this to the manager and restart PURGEMARKERHISTORY MINKEEPDAYS 3, MAXKEEPDAYS 5, FREQUENCYMINUTES 30 This example keeps all rows that have not been modified in the past three days and deletes them when they have not been modified for at least five days. The purge frequency is 30 minutes. See note for more details 21

22 Reality Check Problem Initial loads of large tables from Oracle to Oracle take a long time if the fetch size is larger than the table generally 400k rows or more SOURCEISTABLE is documented as being best Solution Avoid sourceistable for Oracle to Oracle. CTAS across a database link of a 40M row table took < 5 minutes Same table with sourceistable took many days 22

23 Reality Check Problem Data replication stops and logs indicates that a KEY is NULL. Log shows Where PK_ID = Solution Trandata was removed from the table. Must re-enable and manually fix the missing rows. select * from DBA_LOG_GROUP_COLUMNS where table_name = '<TN here> Note the best way to restart is to skip the bad SCN (also reported in the LOG) see the next section for the syntax. start replicat rep_la, aftercsn

24 Reality Check Doc states recycle bin off Force Logging But I only want a few tables now what? 24

25 Questions? 25

26 Contact Frank Bommarito