Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

Transcription

1 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

2 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 2 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

3 Magdalena Bartnik Ugo Pollio Oracle Data Integration Moving Data Oracle to Transform Data Integration Platform Business Moving Data to Transform Business 3 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

4

5 Program Agenda Three New Disruptors in Data Integration Zero Downtime and Continuous Availability Real-time Analytics Trusted Business Insight GG, ODI & EDQ Putting it all Together Architecture Principles and Best Practices Use cases 5 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

6 New Requirements in Data Integration Any Data, Any Source Zero Downtime, High Availability Real-time Analytics 6 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

7 Big Data Drives Big Benefits Any Data, Any Source Your Data: Decisions based on your data Transactions Documents Oracle SAP Big Data: Decisions based on all data relevant to you Social Data Machine-Generated Data 7 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

8 Oracle Strategy for Integrating Big Data Oracle Data Integrator and Big Data Connectors Any Data, Any Source Bulk Data, E-LT Big Data Transformation OLTP OLAP Legacy Unstructured 8 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

9 Convergence of Real-time and Availability Zero Downtime, High Availability Real-Time Enterprise To supply real-time data source systems have to be continuously available Systems that require 24/7 availability contain critical data to be shared in real time 9 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

10 Oracle GoldenGate Real-time Data Integration and Continuous Availability Zero Downtime, High Availability Log-based change data capture, distribution, transformation, and delivery Real-time Replication Supports heterogeneous databases and operating systems Bidirectional replication without distance limitation Transactional integrity. Reliable data delivery and fast recovery after interruptions OLTP OLAP Legacy 10 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

11 Oracle GoldenGate 11gR2 Real-time Data Integration and Continuous Availability GoldenGate 11gR2 Integrated Capture Enhanced Conflict Detection & Resolution Security & Performance Monitoring, Management Expanded Heterogeneity (iseries) Efficiently capture changed data in high-volume and high-throughput implementations Optimized for Oracle Database 11g and Oracle Exadata environments Reduce the time required to detect and resolve data conflicts Increased support for business-critical and heterogeneous systems 11 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

12 GoldenGate 11gR2 Differentiator: Heterogeneity Databases Oracle GoldenGate Capture and Delivery: Oracle DB2 Microsoft SQL Server Sybase ASE Teradata Enscribe SQL/MP SQL/MX New MySQL v 5.5 JMS message queues Oracle GoldenGate Delivery: All listed above, plus: New TimesTen, Postgres Netezza, Greenplum, HP Neoview ETL products O/S and Platforms Linux Sun Solaris v11 Windows 2000, 2003, XP HP NonStop HP-UX IBM AIX v7.1 IBM z Series New zlinux IBM i Series New New 12 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

13 Oracle Strategy for Real-time Analytics Real-time Analytics Business Analytics Data Quality (Product & Customer Data) Real-time Replication Bulk Data, E-LT Big Data Transformation Data Services, Federation OLTP OLAP Legacy Unstructured 13 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

14 Real Time Analytics Delivering maximum performance at a lower cost GoldenGate ODI 11G EDQ Decrease batch windows for data acquisition Enable efficiency Active vs Archive data Business Decision happen in Real-Time Reliable data, trustable insights Moving towards the real-time enterprise via real-time data delivery Optimized for Oracle Database 11g and Oracle Exadata environments Handles continuous data loading while users run reports Efficiently capture changed data in high-volume and high-throughput implementations Transform data into valuable information 14 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

15 Real Time Analytics Delivering maximum performance at a lower cost Real-time Analytics Real-time Replication Super fast data refresh: GoldenGate for real-time replication Super Fast data movement : Data Integrator for Movement & Transformation Fast Aggregates and view refresh: Inmemory optimizations in Times Ten OLTP Legacy 15 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

16 How Oracle GoldenGate Works Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Pump: distributes data for routing to target(s). Route: data is compressed, encrypted for routing to target(s). Delivery: applies data with transaction integrity, transforming the data as required. Source Oracle & Non-Oracle Database(s) Bi-directional Target Oracle & Non-Oracle Database(s) 16 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

17 Oracle GoldenGate Veridata Data Comparison without Downtime A high-speed, low impact data comparison solution Identifies and reports data discrepancies between heterogeneous databases without interrupting their availability Benefits: Reduce financial/legal risk exposure Speed and simplify IT work in comparing data sources No disruption to business systems Improved failover to backup systems Confident decision-making and reporting 17 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

18 Management Pack for Oracle GoldenGate Monitoring Oracle GoldenGate Monitor Advanced monitoring, alerting (SNMP support), lag graphs, and historical repository. Based on new agent infrastructure, which will be the foundation for all future integration projects. Supports GoldenGate Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

19 Management Pack for Oracle GoldenGate Monitoring Enterprise Manager Plug-in Oracle Preferred monitoring tool, covers entire Oracle stack Database, Storage, FMW etc For customers, who prefers to drill into DB sessions, OS stats, etc, apart from monitoring GoldenGate process Included in Management Pack 19 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

20 Oracle Data Integrator 11g Fastest ETL/E-LT, Simpler Time-to-Value, Lowest TCO Oracle Data Integrator Enterprise Edition Legacy Sources Application Sources OLTP DB Sources Best in class performance, scalability Lowered TCO, lightweight deployment Improved usability and manageability Heterogeneous Open, standards-based Any Data Warehouse Any Planning System 20 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

21 ODI Oracle s Strategic ELT/ETL ODI is Faster Fastest E-LT Bulk/Batch Performance Faster Real Time integration (sub-second) with CDC, Replication, and SOA Faster Project Setup, Design and Delivery ODI is Simpler Simpler Setup, Configuration, Management, and Monitoring Simpler way to do Mapping using Declarative SQL Interfaces Simpler Deployment with Fewer Hardware Devices Simpler extensibility with Knowledge Module code templates ODI is Saves Money (Lower TCO, Higher ROI) Less Hardware & Energy Costs with E-LT Architecture Less Time Wasted on Unnecessary ETL Mappings, Scripting, and Training Less Integration Overhead Integrating with Applications, SOA, and Management 21 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

22 Problem - Shrinking ETL Batch Windows 2 LOAD 3 1 TRANSFORM EXTRACT Challenge ETL Processing Exceeds Business Batch windows due to increased data volumes Older data 22 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

23 Optimized Data Loading through E-LT The key to improved performance and reduced costs Conventional ETL Architecture Extract Transform Load E-LT provides flexible architecture for optimized performance Benefits Next Generation Architecture E-LT Extract Load Leverage Set-based transformations Improved performance for loading, no network hop Takes advantage of existing hardware Transform Transform 23 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

24 ODI-EE Declarative Design Improved Developer Productivity and Lower Maintenance Costs Specify ETL Data Flow Graph Developer must define every step of Complex ETL Flow Logic And significant development and maintenance efforts Declarative Set-based Design Simplifies the number of steps Automatically generates the Data Flow whatever the sources and target DB Benefits Significantly reduce the learning curve Shorter implementation times Streamline access to non-it pros Conventional ETL Design ODI Declarative Design 1 2 Define What You Want Automatically Generate Dataflow Define How: Built-in Templates 24 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

25 Knowledge Modules Hot-Pluggable: Modular, Flexible, Extensible Benefits Tailor to existing best practices Ease administration work Reduce cost of ownership 25 25

26 Trusted Business Insight with EDQ Master Data Management Enterprise Data Quality (Product & Customer Data) Profile & Audit Parsing & Standardize Match & Merge Watchlist Screening Data Movement & Transformation OLTP OLAP Legacy Unstructured 26 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

27 Practical Applications and Value Drivers of DQ Revenue Generation & Customer Retention Better Operations IT Agility Risk Management & Compliance Cross-Sell / Up-Sell 1 st Time Accurate Deliveries Lower Operations & IT Costs Law Enforcement / Privacy Differentiated Service Delivery Accelerate New Product Introduction Accelerate New IT Projects Fraud Prevention / Compliance DQ/MDM as Foundation or Rationalization for CRM An add-on to CRM or ERP Technology Foundation Foundation or Rationalization for SCM/ ERP/Fulfillment Marketing/ Compliance Application 27 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

28 Data Quality Maturity Progression DQ + MDM Multiple Domains Greater coverage, greater impact DQ + MDM Single Domain Hubs persist and sync data leveraged impact DQ Process/ System DQ Masters data in legacy applications with repeatable process DQ Tools Project based benefits are short lived CHAOS! No tools, no process the Wild West! 28 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

29 Enterprise Data Quality Integrated Solution for All Data Quality Problems Common Ac ccess/ui Govern Monitor effectiveness & resolve problems Match Identify & merge duplicates Standardize Drive conformance to standards Profile Quickly understand data content Broadest DQ offering Best of breed capabilities for both Party Data and Product Data Profiling, standardization, matching, case management, governance Most usable DQ offering Completely integrated offering designed to work together Designed for business and technical users Transparent operation and results no black boxes Pervasive operation for enterprise quality governance Within legacy systems and MDM Hubs As part of migration/system load On data entry/capture As part of data movement/transfer 29 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

30 Putting it all Together: Oracle Data Integration Any Data, Any Source Private Cloud Public Cloud Zero Downtime, High Availability Oracle SOA Suite, BPM Suite, Oracle Business Analytics Oracle Enterprise Data Quality Real-time Analytics Oracle GoldenGate Oracle Data Integrator Big Data Connectors Oracle Cloud Application Foundation Data Service, Federation On Premise, On Cloud OLTP OLAP Legacy Unstructured 30 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

31 Oracle Engineered Systems Any Data, Any Source Oracle Exalytics In-Memory Machine Zero Downtime, High Availability Oracle Exadata Oracle Big Data Appliance

32 If you recognize your company in one of this questions it s time to contact us My batch windows aren t long enough anymore I m moving to Exa... How do I migrate? I want to transform my architecture but retain legacy My architecture is heterogeneous and complex I need continuous availability and maximum HA I can t take much/any downtime My reporting needs are killing my production systems My DB/DW is mission critical but I need to upgrade Oracle Data Integration Solutions How can I reduce risk in my data integration plans? 32 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

33 Architecture Architecture Principles Principles and and Best Best Practices Practices

34 Your Enterprise Architecture An Architects Approach to Emerging Areas Addresses Solution Feasibility Time Cost Addresses Risks Skills Experience Investment Addresses Goals Aligned to Value Leverages Processes Leverages Infrastructure

35 The Information Architecture Spectrum Evaluating Economic and Architecture Tradeoffs Data Realms Structure Volume Security Master data Transactions Analytical data Metadata Structured Medium- High Database, app, & user access Storage & Retrieval Modeling Integration Consumption RDBMS/ SQL Pre-defined relationalor dimensional modeling ETL/ELT, CDC, Replication, Message BI & Statistical Tools, Operational Applications Reference data Structured Low- Platform XML/ xquery Flexible & ETL/ELT, System-based and Semi- Medium security Extensible Message data Structured consumption Documents and Content Unstructured High File system based File System / Search Free Form OS-levelfile movement Content Mgmt

36 The Information Architecture Spectrum Evaluating Economic and Architecture Tradeoffs Data Realms Structure Volume Security Master data Transactions Analytical data Metadata Structured Medium- High Database, app, & user access Storage & Retrieval Modeling Integration Consumption RDBMS/ SQL Pre-defined relationalor dimensional modeling ETL/ELT, CDC, Replication, Message BI & Statistical Tools, Operational Applications Reference data Structured Low- Platform XML/ xquery Flexible & ETL/ELT, System-based and Semi- Medium security Extensible Message data Structured consumption Documents and Content Unstructured High File system based File System / Search Free Form OS-levelfile movement Content Mgmt Big Data -Weblogs -Sensors - Social Media Structured, Semi- Structured, Unstructured High Filesystem & database Distributed FS /nosql Flexible (Key Value) Hadoop, MapReduce, ETL/ELT, Message BI & Statistical Tools

37 Traditional Information Architecture Structured Reference & Master Data Transaction Data DBMS (OLTP) ETL/ELT CDC Real-Time DB Rep Data Warehouse Data Marts ODS Reporting & Dashboards EPM / BI Apps Advanced Analytics Visual Discovery Management Security, Governance Key Architecture Principles Treat Data as an Asset Quality, Timeliness, Accuracy Information Metadata Key EA Responsibility Governance - Center of Excellence, Training - Technical Standards Transparency, Compliance

38 Big Data Information Architecture Unstructured Machine Generated Social Media Text, Image Video, Audio Distributed File System Key-Value Data Store MapReduce Solutions Sandboxes Data Marts Visualization & Discovery Statistical Analytics Structured Data Correlation Visualization & Discovery DW Key Architecture Principles Treat Data as an Asset Low density, variable schema Quality, Timeliness, Accuracy Drive to zero latency Information Metadata Privacy Key EA Responsibility Governance Center of Excellence, Training Technical Standards Transparency, Compliance

39 The Big Data Divide Structured Unstructured Sem miured structu Master & Ref Data Biz Txn Data Machine Generated Social Media Text, Image Video, Audio DBMS (OLTP) Distributed File System Key-Value Data Store ETL/ELT CDC Real-Time DB Rep MapReduce Solutions Sandboxes Data Warehouse Data Marts ODS Custom Code Reporting & Dashboards EPM / BI App Advanced Analytics Visual Discovery Management Security, Governance

40 Driving to an Integrated Architecture Capture Store/Process Integrate Organize Analyze Govern Structured Master & Ref Data Transaction Data Machine Generated Social Media Text, Image Video, Audio DBMS (OLTP) Hadoop Cluster w MapReduce Key-Value Data Store Unstructured Sem miured structu DB Replic ETL/ELT ChangeDC Real-Time Message- Based ODS Data Warehouse Data Marts Streaming (CEP Engine) Reporting & Dashboards Alerting EPM BI Applications Text Analytics and Search In-Database Analytics Advanced Analytics Visual Discovery Managem ment Security, Go overnance

41 Best Practices Adopt an Enterprise Architecture Approach Align Big Data initiative with specific business goals Ensure centralized IT strategy for standards and governance Use a center of excellence to minimize training and risk Expand Your Information Architecture Embrace data diversity and long tail data analysis Correlate big data and structured data Provide high performance and scalable analytical sandboxes

42 Oracle Data Integration

43 Driving to an Integrated Architecture Capture Store/Process Integrate Organize Analyze Govern Structured Master & Ref Data Transaction Data Machine Generated Social Media Text, Image Video, Audio DBMS (OLTP) Hadoop Cluster w MapReduce Key-Value Data Store DB Replication Unstructured Sem miured structu ETL/ELT Oracle Data CDC Integration Real-Time Message- Based ODS Data Warehouse Data Marts Streaming (CEP Engine) Reporting & Dashboards Alerting EPM BI Applications Text Analytics and Search In-Database Analytics Advanced Analytics Visual Discovery Managem ment Security, Go overnance

44 Oracle Data Integrator Bulk Data Processing and Big Data Transformation Legacy Sources Application Sources Oracle Data Integrator CEP, Data Services Any Data Warehouse Relational and Non-Relational Optimized E-LT Architecture Declarative design for Lower TCO Extensible Knowledge Modules for enriching functionality (i.e. big data transformation) Any Planning System

45 Oracle GoldenGate Real-time Data Integration for Big Data Legacy Sources Application Sources Oracle GoldenGate Zero Downtime Upgrade and Migration Highly Available, Disaster Recovery CEP Integration, Real-time Events Relational and Non-Relational Log-based change data capture, distribution, transformation, and delivery Bidirectional replication Reliable data delivery, transaction integrity Data Synchronization across the Enterprise Real-time BI/Data Warehousing, MDM

46 Oracle Enterprise Data Quality Data Quality for Big Data Legacy Sources Application Sources Oracle Enterprise Data Quality Cleansing, Match, Merge, Standardization Governance, Profiling, Auditing Relational and Non-Relational Data Preparation, Case Management Complete and best-of-breed approach for both customer and product data Unifies business-oriented interfaces for ease-of-use, lower TCO Reduces risk, high match rates with low false positives

47 Oracle Data Integration Complete Offering for Enterprise Data Integration Legacy Sources Application Sources Relational and Non-Relational Oracle Enterprise Data Quality Oracle Data Integrator Oracle GoldenGate Complete and best-of-breed approach to address enterprise integration Maximum performance with lower cost of ownership, ease of use, and reliability. Certified for leading technologies to deliver fast time to value Oracle customers report: 80% lower TCO Five times higher performance 70% reduction in development costs

48 DIS Use Cases Migrations& Consolidations ODI EDQ Heterogeneous Source Systems OGG OGG OGG ADG ODI Zero Downtime Migrations & Upgrades Active/Active High Availability Query Off-Loading New DB/HW/OS/APP ADG ADG Fully Active Distributed DB ADG Disaster Recovery OGG Reporting Database OGG ODI EDQ ADG OGG ODI EDQ ODS Real-Time & Batch BI Data Warehouse OGG ODI Data Distribution Global Data Centers BPM OGG ODI SOA/EDA BPM CEP

49 Data Integration in the Marketplace Sorted by Vertical Markets (sample customers only) Communications Finance / Banking Media Services Energy/Industrial Insurance / Health Retail Other 49 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

50 Join the Oracle Community or #OracleDataIntegration Facebook facebook.com/oracledataintegration LinkedIn Oracle Data Integration Oracle blog blogs.oracle.com/dataintegration Oracle.com oracle.com/goto/dataintegration 50 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

51 51 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration

52 52 Copyright 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration