About Gluent. we liberate enterprise data. We are long term Oracle Database & Data Warehousing guys long history of performance & scaling

Transcription

1 About Gluent We are long term Oracle Database & Data Warehousing guys long history of performance & scaling The world is changing we help customers to get the best out of both worlds! About 20 people in distributed teams in US & UK we liberate enterprise data Gluent Data Platform enables all applications to access all enterprise data, at anytime, on demand, and without any code migration or rewrites

2 Why Getting Started

3 Many conference presentations get too technical, too fast 5

4 What is ODI?

5 What is ODI? E-LT (Extract, Load, Transform) vs ETL Heterogeneous sources and targets Built to integrate all data Flexibility Reusable code templates (Knowledge Modules) Open API ODI will adapt to your data warehouse - and not the other way around Flow based mappings 7

6 Where did ODI come from? Early 2000s: Sunopsis formed 2006: Oracle acquires Sunopsis. Late 2013: Oracle Data Integrator 12c released, combining ODI and OWB features. 2010: Oracle Data Integrator 11g released. Oracle s first real update to ODI. 2012: ODI for Big Data first released 8

7 ODI 12c Architecture and Components *image source: 9

8 ODI 12c Architecture and Components *image source: 9

9 Installation and Configuration

10 Choose your installation type wisely Know the different types of installs Standard - used for Standalone Agent or on client machine (ODI Studio only) Enterprise - JEE Agent or Colocated Agent, ODI Console, FMC plugin 11

11 Configuration is half the battle 12

12 Configuration is half the battle CONFIGURATION 12

13 ODI Studio Configuration ODI_HOME/odi/studio/odi.sh exe Connect to existing repository If none exists, run RCU first Boom, you re in! 13

14 Standalone or JEE Agent Configuration ODI_HOME/oracle_common/common/bin/config.sh exe Run the WebLogic Server configuration wizard Standalone Agent Create domain outside of ODI Oracle Home JEE Agent Add credentials mapping Create / configure admin server, nodemanager, topology, etc. 14

15 ODI Agent - how it works ODI Agent 3. Agent passes code to source/target databases and updates repository execution logs Source database 1. Execution request via http(s) 2. Agent communicates with ODI repository to generate code and gather source/target connection ODI repository Target database 15

16 Getting Started

17 Topology, Contexts, Models, and Datastores - Oh my! Data Server Connection to the data source Physical Schema Database schema Work schema Logical Schema Abstracts the physical connection/schema from the logical Development occurs against the Logical Schema Model Metadata object tied to Logical Schema Represents database schema, file directory, XML document, etc Datastore Logical representation of table, file, XML element, etc. All of these objects must be created and setup prior to any mapping development can occur. 17

18 Developer s Quickstart There s another way to get started - quickly! New Model and Topology Objects wizard - ODI In ODI 11g, the ODI QuickStart was a 10 step checklist, where each step led to another section in the documentation 18

19 Topology, Contexts, Models, and Datastores 19

20 Logical Constraints on the metadata 20

21 Logical Constraints on the metadata 20

22 Logical Constraints on the metadata 20

23 Logical Constraints on the metadata 20

24 Mappings and Knowledge Modules

25 Mappings - Logical Datastore component Split output to multiple target tables Join component Mappings are built from datastore and components to logically form the ETL 22

26 Mappings - Physical Physical implementations of the mappings are represented based on location of the datastore. Knowledge Modules are selected to generate the code. 23

27 Mappings - Physical ODI will adapt to your data warehouse - and not the other way around Physical implementations of the mappings are represented based on location of the datastore. Knowledge Modules are selected to generate the code. 23

28 Knowledge Modules - the secret sauce *image source: Fusion Middleware Developing Knowledge Modules with Oracle Data Integrator 24

29 Knowledge Modules - the secret sauce *image source: Fusion Middleware Developing Knowledge Modules with Oracle Data Integrator 24

30 Knowledge Modules - the secret sauce *image source: Fusion Middleware Developing Knowledge Modules with Oracle Data Integrator 24

31 Orchestration Packages and Load Plans

32 Packages Group like objects together Perform specialized tasks (looping, etc) 26

33 Load Plans - the top level orchestration object Executes Scenarios (execution objects that are essentially compiled code) Steps can be run in parallel or serially If - then - else step can evaluate a variable to determine path Exception steps can be customized and executed on failure 27

34 More flexibility Procedures

35 Procedures for maintenance tasks and one-off functions Reusable components for admin tasks DDL, file archive scripts, exception handling, etc Moving or transforming data, it probably should be a mapping Creating a complex, recurring task for data manipulation, probably a Knowledge Module Images source: 29

36 Other notable features Integration with GoldenGate (change data capture) and Enterprise Data Quality (for data quality) ODI for Big Data ODI Cloud Service 30

37 Recap Oracle Data Integrator is an E-LT / data integration tool built to integrate all data flexible. ODI will adapt to your data warehouse - and not the other way around Choose your installation wisely and understand configuration is more involved Many objects must be created prior to Mapping development Use the Developer s Quickstart! Flow based mappings separate business logic from physical implementation Knowledge Modules are the secret sauce of ODI More info: mrainey.co/odi12c-getting-started 31

38 we liberate enterprise data thank you! 32