DELICIOUS : ORACLE ENTERPRISE DATA QUALITY, ORACLE GOLDEN GATE AND ORACLE DATA INTEGRATOR

Transcription

1 DELICIOUS : ORACLE ENTERPRISE DATA QUALITY, ORACLE GOLDEN GATE AND ORACLE DATA INTEGRATOR ON ORACLE EXADATA Gürcan Orhan Data Migration & DWH & BI & Information Architect Wipro Advanced Technologies & Solutions - Turkey

2 +20 years of IT experience. +12 years of DWH experience. +9 years of Oracle Data Integrator experience, +7 years of Oracle Warehouse Builder experience. Sybase Power Designer, CA ERwin Data Modeler OBIEE, Cognos, Microstrategy, Business Objects, Tableau WHO AM I? Oracle Excellence Awards - Technologist of the Year 2011 : Enterprise Architect (Oracle Magazine Editors Choice of Awards, Enterprise Architect) DWH & BI Chair : TROUG (Turkish Oracle User Group) Published Customer Snapshot for Published video about (Oracle Media Network) Published OTN Podcasts about Data Warehousing and ODI ODI and the Evolution of Data Integration Articles in OTech Magazine, SearchSoftwareQuality.com Presenter in Oracle Open World since 2010 (5 times in a row ) Presenter in ODTUG Kscope Presenter in UKOUG, OUGF, other OUGs and OTN LA Tour Presenter in various universities in Turkey First time in OTN Yathra Tour & India... " gurcan_orhan

3 Sustained Growth $6.91 FY US $ Billion in Revenues (16% CAGR over 6 years) Partner to Leaders Active Global Clients (150+ fortune 500 customers) * Figures based on Q2 results for Global IT Business WIPRO TODAY Leaders Company for leaders, globally** Attracts the Best Talent ,000+ Top 25 Workforce Global Presence 61 Countries! Recognized as one of the world s most ethical companies by Ethisphere! # 1 in the MSCI Emerging Markets Index and # 2 in the Global 500 listing of Newsweek's Green Rankings 2012 ** List announced by Aon Hewitt, The RBL Group and Fortune Wipro demerged its IT and Non IT business in April 2013, consolidated revenues for FY stand at $7.95 Billion

4 DISCLAIMER The opinions expressed in this presentation and on the following slides are solely those of the presenter and not necessarily those of Wipro Corporation and its affiliates. Some of the information within this presentation, including product names, product versions and patch numbers, are collected from Oracle s corporate web site and should be checked again before acting on any information. You should consider the appropriateness of the information having regard to these matters, any relevant offer document. The examples included in these presentations are intended for discussion purposes only. The following is intended to outline Oracle's general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle's products remains at the sole discretion of Oracle Corporation.

5 MY DEFINITION OF ODI ODI (Oracle Data Integrator) is a tool, that can talk, or learn how to talk, with any database system, or any operating system, in its own language. This is the power of ODI.

6 INGREDIENTS

7 RECIPE

8 EDQ : THE SALAMI

9 EDQ : INTEGRATED SOLUTION FOR ALL DQ Broadest DQ offering Best of breed capabilities for both Party Data and Product Data Profiling, standardization, matching, case management, governance Most usable DQ offering Completely integrated offering designed to work together Designed for business and technical users Transparent operation and results no black boxes Pervasive operation for enterprise quality governance Within legacy systems and MDM Hubs As part of migration/system load On data entry/capture As part of data movement/transfer

10 EDQ : INTEGRATED SOLUTION FOR ALL DQ Process metrics Remediation Case Management Quality metrics Party (individuals, households) match Entity match Semantic (category) match Statistical match Match review Merge/survivorship Global parse Category parse Extract Transform Address verification & geocoding Substitute Enrich Classify Statistics Patterns Phrases Duplicates Completeness Max/min values

11 EDQ : OVERVIEW

12 EDQ : KEY ATTRIBUTES FOR SUCCESS RELEVANCE ACCURACY INTEGRITY CONSISTENCY COMPLETENESS VALIDITY TIMELINESS ACCESSIBILITY COMPLIANCE Is it relevant to its intended purpose? Is it correct and objective, and can it be validated? Does it have a coherent, logical structure? Is it consistent and easily to understand? Does it provide all the information required? Is it within acceptable parameters for the business? Is it up to date and available whenever required? Can it be easily accessed and exported to the target application? Does it comply with regulatory standards?

13 RECIPE

14 OGG : THE SAUCE Oracle s standard tool for data replication Sub-second speed

15 OGG : ARCHITECTURE Oracle s strategic tool for data replication and real-time Capture data based on transaction log Handle the distribution and the delivery of committed data in real-time Heterogeneous systems Uni-directional or bi-directional

16 OGG : STRENGTHS Sub-second speed Reliability Low impact on source and target systems Platform independent universal data format Easy to deploy - simple configuration of parameter files Conflict detection and resolution Data encryption

17 Real-time data integration Zero-downtime migration Disaster Recovery - Data Protection Data Distribution OGG : USES

18 RECIPE

19 EXADATA : A DATABASE MACHINE

20 EXADATA : A DATABASE MACHINE

21 EXADATA : MIGRATION STEPS 35 % 23 % 50 %

22 EXADATA : MORE TO DO Beware of Partitioned tables related data blocks remains uncompressed on HCC after updating rows Use CTAS, IAS and Exchange Partition methods for performance

23 EXADATA : MORE TO DO (NOT) EXISTS instead of (NOT) IN Search code, modify related code execute faster

24 EXADATA : MORE TO DO Don t drop your indexes If you are not using them for performance issues

25 RECIPE

26 ODI : THE CHEESE Oracle s standard tool for data integration Can read and write from various source / target technologies Talks the language of the technology Extensible through drivers, Knowledge Modules and ODI Tools

27 ODI : E-L-T APPROACH Reduce network transfer Reduce hardware costs Set-based operations Leverage target platform capabilities Referential Integrity Check

28 ODI : HOW DOES IT WORK? Declarative design, separation of business logic and technical implementation Reusability Data Quality

29 ODI : OVERVIEW

30 ODI : OVERVIEW

31 In ODI Studio In the repositories : at design-time ODI : LINEAGE

32 ODI : LINEAGE In ODI Studio In the repositories : at design-time and at run-time

33 ODI : LINEAGE In ODI Studio In the repositories : at design-time and at run-time

34 ODI : LINEAGE In ODI Studio In the repositories : at design-time and at run-time

35 ODI : EASE OF CHANGE

36 ODI : EASE OF CHANGE

37 ODI : EASE OF CHANGE interactivehelper.performaction(new InterfaceActionSetKM(myIkm, odiinterface.gettargetdatastore(), KMType.IKM, new KMOptionRetainerLazy())); KMOptionRetainerCopy KMOptionRetainerHomonymy interactivehelper.performaction(new InterfaceActionSetKMOptionValue ( odiinterface.gettargetdatastore(), KMType.IKM, DETECTION_STRATEGY, NOT_EXISTS ));

38 RECIPE

39 PIZZA TIME : EASE OF USAGE Performance Oracle premier/platinum support Pre-installed, fast time2market Mission critical high-availability

40 PIZZA TIME : EASE OF CONTROL Control replication in ETL process Control session properties in ETL process Control data quality in ETL process Control database maintenance in ETL process

41 PIZZA TIME : EASE OF WORKLOAD Start loading DWH immediately. No more extraction phase Alerts for data type, column size changes No ETL crash because of Data Quality errors Let Exadata finish ETL process as fast as can be

42 EDQ OdiEnterpriseDataQuality CDC OdiWaitForData OdiWaitForLogData OdiWaitForTable OdiRefreshJournalCount OdiRetrieveJournalData PIZZA TIME : ODI TOOLS

43 PIZZA TIME : ODI AND OGG ARCHITECTURE Knowledge Module support for OGG JKM Oracle to Oracle Consistent (OGG) Automatic deployment of OGG configuration files Integration with ODI CDC framework Uses same design and deployment approach as other ODI CDC KMs

44 PIZZA TIME : ODI AND GG SET UP 1. Model configuration 2. Datastore(s) selection 3. Model journalization 4. Deploy GG files

45 PIZZA TIME : ODI AND GG INTEGRATION

46 No extra hardware ODI Agent on Exadata JVM ODI Repository on Exadata DB PIZZA TIME : ODI AND EXADATA Generic or Specific IKMs IKM SQL Control Append_BIG_DATA IKM Oracle PEL IKM Oracle Incremental Update (MERGE) For Exadata NOT EXISTS MERGE (no staging)

47 PIZZA TIME!

48 PIZZA TIME : DELICIOUS

49 gurcan_orhan