Rittman Mead Consulting Automatic promotion and versioning with Oracle Data Integrator 12c

Rittman Mead Consulting Automatic promotion and versioning with Oracle Data Integrator 12c Jérôme Françoisse DOAG 2014 Konferenz + Ausstellung - 20th September 2014 1

Jérôme Françoisse Consultant for Rittman Mead Oracle BI/DW Architect/Analyst/Developer ODI Trainer Providing ODI support on OTN Forums ODI 12c Beta Program Member Blogger at http://www.rittmanmead.com/blog/ Email : jerome.francoisse@rittmanmead.com Twitter : @JeromeFr 2

Rittman Mead Oracle Gold partner with offices in Europe, US (Atlanta), Australia, India and South Africa World leading specialist partner for technical excellence, solutions delivery and innovation in Oracle BI Provide consulting, training, global managed services for customers around the world 120+ consultants including 1 Oracle ACE Director and 3 Oracle ACEs All expert in Oracle BI, DW, EPM and Analytics tech Skills in broad range of supporting Oracle tools: OBIEE, OBIA, ODIEE, Essbase, Oracle OLAP, GoldenGate, Exadata, Endeca Blog : http://www.rittmanmead.com/blog/ Twitter : @rittmanmead 3

ODI 12c Key Components ODI Studio (the main development IDE) Designer Navigator Operator Navigator Topology Navigator Security Navigator ODI Agents (used for executing mappings and packages) ODI Console (web-based console for running published packages) ODI Repositories (contain definitions of models, interfaces and other objects) Designer Model data sources Define projects Create interfaces Release scenarios Standalone Agent Orchestrate interfaces and scenarios Operator View running interfaces and packages Operate production Repository Information Systems (Databases, files, applications) 4

ODI 12c Key Components ODI Studio (the main development IDE) Designer Navigator Operator Navigator Topology Navigator Security Navigator ODI Agents (used for executing mappings and packages) ODI Console (web-based console for running published packages) ODI Repositories (contain definitions of models, interfaces and other objects) Models Projects Work Repository (Development) Security Topology Versioning Master Repository Repository (Production) 5

ODI Architecture 6

Physical and Logical Topology Physical Contexts Logical DEV_OP DEV_SCOTT SH HR TEST_OP SCOTT HR SCOTT PROD_OP SH HUM_RES 7

Physical and Logical Topology Physical Contexts Logical DEV_OP DEV_SCOTT SH HR SCOTT TEST_OP SCOTT SH HR HR PROD_OP SCOTT SH HUM_RES 7

Physical and Logical Topology Physical Contexts Logical DEV_OP DEV_SCOTT SH HR DEV SCOTT TEST_OP SCOTT TEST SH HR HR SCOTT PROD_OP SH PROD HUM_RES 7

How many repositories do I need? Security Topology Versioning Master Repository Models Projects Work Repository (Development) Repository (Production) 8

How many repositories do I need? Security Topology Versioning Master Repository Models Projects Work Repository (Development) Repository (Test) Repository (Production) 9

How many repositories do I need? Security Topology Versioning Master Repository Security Topology Versioning Master Repository Models Projects Work Repository (Development) Repository (Test) Repository (Production) 10

How many repositories do I need? Security Topology Versioning Security Topology Versioning Security Topology Versioning Master Repository Master Repository Master Repository Models Projects Work Repository (Development) Repository (Test) Repository (Pre-Production) Repository (Production) 11

How many repositories do I need? Security Topology Security Topology Security Topology Versioning Master Repository Versioning Master Repository Versioning Master Repository Models Models Projects Projects Work Repository (Development) Repository (Test) Repository (Pre-Production) Work Repository (Hotfix) Repository (Production) 12

How many repositories do I need? Security Topology Versioning Master Repository Security Topology Versioning Master Repository Security Topology Versioning Master Repository Models Models Projects Projects Work Repository (Development) Repository (Test) Repository (Smoke Test) Repository (Pre-Production) Work Repository (Hotfix) Repository (Production) 13

Export Security Topology Versioning Master Repository Models Projects Work Repository (Development) Repository (Production) 14

Export - Manually Security Topology Versioning Master Repository Models Projects Work Repository (Development) Repository (Production) 15

Automation XKCD.com 16

Automation XKCD.com 17

Export - Automated within a package 18

Versioning - Create a version 19

Versioning - Comparison 20

Versioning - Restore a version 21

SubVersion GIT 22

Export Security Topology Versioning Master Repository Security Topology Versioning Master Repository Security Topology Versioning Master Repository Models Models Projects Projects Work Repository (Development) Repository (Test) Repository (Smoke Test) Repository (Pre-Production) Work Repository (Hotfix) Repository (Production) 23

Automation XKCD.com 24

ODI SDK - Groovy Scripting 25

ODI SDK - Groovy Scripting /* (...) Connection to the source repository (...) Creation of a collection of mappings (...) Instanciation of an OdiScenarioGeneratorImpl object */ 25

ODI SDK - Groovy Scripting /* (...) Connection to the source repository (...) Creation of a collection of mappings (...) Instanciation of an OdiScenarioGeneratorImpl object */ // Generating Scenarios For(Object mapping : mappings) { Mapping odimap = (Mapping) mapping; String scenname = odimap.getname(); OdiScenario newscen = gene.generatescenario(odimap, scenname, newversion); } 25

ODI SDK - Groovy Scripting // Generating Scenarios For(Object mapping : mappings) { Mapping odimap = (Mapping) mapping; String scenname = odimap.getname(); OdiScenario newscen = gene.generatescenario(odimap, scenname, newversion); } /* (...) Creation of a collection of the generated scenarios */ // Exporting Scenarios for (Object scen : scenariocollection) { OdiScenario odiscen =(OdiScenario)scen ; export.exporttoxml(odiscen, ExportPath, OverWrite, RecursiveExport, Encoding); } 25

ODI SDK - Groovy Scripting /* (...) Creation of a collection of the generated scenarios */ // Exporting Scenarios for (Object scen : scenariocollection) { OdiScenario odiscen =(OdiScenario)scen ; export.exporttoxml(odiscen, ExportPath, OverWrite, RecursiveExport, Encoding); } /* (...) Connection to the target repository (...) Creation of a list of XML files. */ 25

ODI SDK - Groovy Scripting // Exporting Scenarios for (Object scen : scenariocollection) { OdiScenario odiscen =(OdiScenario)scen ; export.exporttoxml(odiscen, ExportPath, OverWrite, RecursiveExport, Encoding); } /* (...) Connection to the target repository (...) Creation of a list of XML files. */ // Importing Scenarios for (String filename : XMLFiles) { import.importobjectfromxml( import.import_mode_synonym_insert_update, filename, true); } 25

GIT - Commit and Push git add. -A git commit -m "My commit Message" git push 26

... 27

Further reading Blog post by Uli Bethke (@ubethke) : ODI: AUTOMATING DEPLOYMENT OF SCENARIOS TO PRODUCTION IN ORACLE DATA INTEGRATOR http://sonra.io/odi-automating-deployment-of-scenarios-in-oracle-data-integrator/ Blog post by kdevendr : Export and Importing Scenarios based on Project (or)folder http://odiexperts.com/export-and-importing-scenarios-based-on-project-orfolder/ ODI SDK Public API : https://docs.oracle.com/middleware/1213/odi/reference-java-api/index.html David Allan : ODI - Accelerator Launchpad, getting Groovy and much more https://blogs.oracle.com/dataintegration/entry/odi_accelerator_launchpad_getting_groovy 29

I m speaking at Tech14 Register now to join me: www.tech14.ukoug.org #ukoug_tech14 30