Oracle GoldenGate 12.3

Transcription

1

2 Oracle GoldenGate 12.3 Was ist neu an der Microservices Architecture Thomas Niewel Systemberater Oracle Deutschland B.V. & Co K.G. November 2017

3 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 3

4 Agenda Oracle GoldenGate Oracle GoldenGate Classic Architecture vs. Microservices Architecture WebPage Interfaces Directory Structure Rest Api

5 Capture Trail Pump Route Deliver Oracle GoldenGate Use cases Fully Active Distributed DB New DB/ HW/OS/APP Global Data Centers High Availability & Disaster Recovery Zero Downtime Upgrades & Data Migration Data Synchronization Across the Enterprise GoldenGate Real-time Data Delivery Big Data, DW & Marts Application Offloading Message Bus & Data Grid Data Streaming Real-time BI, Hadoop Data Staging, Data Ingestion Query & Report Offloading Event Driven Architecture, SOA/JMS, Coherence Real-time Analytics & Massive Parallelization

6 Oracle GoldenGate Flexible Deployment Models Unidirectional Query Offloading Zero-Downtime Migration Bi-Directional Hot Standby or Active-Active for HA Peer-to-Peer Load Balancing Multi-Master Broadcast Data Distribution Integration/Consolidation Data Warehouse JMS, Flat File Oracle GoldenGate Application Adapters HDFS,HIVE,FLUME,Kafka, HBASE. JDBC Oracle GoldenGate for Big Data

7 GoldenGate: Heterogeneous Replication 7

8 Agenda Oracle GoldenGate Oracle GoldenGate Classic Architecture vs. Microservices Architecture WebPage Interfaces Directory Structure Rest Api

9 OGG 12.3 Overview 2 Installation Packages Classic Architecture (CA) Microservices Architecture (MA) Upgrading to 12.3 Classic Architecture as usual Microservices Architecture must be fresh install 9

10 Oracle GoldenGate Classic Architecture DYNAMICPORTLIST

11 GoldenGate 12.3Microservices Architecture Administration, Distribution, Receiver, Metrics Services with RESTful Service Interfaces Source Database Admin Server Metrics Server HTTPS RESTFul Service Interfaces Metrics Server Admin Server Target Database Distribution Server Receiver Server Extract Trail Files Source Service Manager Service Manager Trail Files Target Replicat

12 Classic Architecture vs. Microservices Architecture Admin Server Replacement of the mgr process Distribution Server Replacement of the pump process Receiver Server Replacement of the collector process Metrics Server Real-time performance monitoring 14

13 Common Key Attributes and Benefits of Each Service Enables remote configuration, monitoring, and administration Provides formal RESTful service interfaces Industry-standard communications protocol: HTTPS, JSON Enables Applications to embed GoldenGate Secure Authorize identity of clients with username / password or SSL client certificates No web server needed 15

14 GoldenGate 12.3Microservices Architecture Setup with oggca.sh

15 GoldenGate 12.3Microservices Architecture Setup with oggca.sh

16 Solutions For Everyone Administrators Traditionalists DevOps Simple to use Web interfaces Get up and running quickly Lower learning curve Command Line access via adminclient run anywhere Similar GoldenGate commands as in GGSCI Interaction the same as previous releases, just thin client Uses RESTful APIs Run anywhere Roll your own monitoring

17 Security Model Standard TLS/SSL based authentication and Client Authorization Clients SSL User Certificate HTTPS User/ Passwd Client Authorization Identity via SSL user certificates or via username/password Roles SSL Server Certificate GoldenGate Services Authorized Clients Security User Administrator Operator User

18 Cloud Ready Remote and secure administration and data communication in cloud environments Internal Network DMZ GoldenGate Services Deployment Reverse Proxy HTTPS wss GoldenGate Services Deployment Remote Administration and Configuration No OS access required after installation Secure administration and data transport Works with all standard HTTP proxy and reverse proxy servers

19 Agenda Oracle GoldenGate Oracle GoldenGate Classic Architecture vs. Microservices Architecture WebPage Interfaces Directory Structure Rest Api

20 WebPage Interfaces Access via URL http(s)://<hostname>:<port>

21 Oracle GoldenGate Service Manager Access via URL (http(s)://<hostname>:<port> i.e. Use Administration User for access: i.e. oggadmin/******** Maintains inventory and configuration information about all deployments Watchdog process for Other OGG Services Runs as a daemon Auto-Restart

22 Oracle GoldenGate ServiceManager Diagnostic Page Provides access to responses that the ServiceManager receives $OGG_DEPLOYMENT/var/log /ServiceManager.log

23 Oracle GoldenGate Administration Server - Housekeeping Example: Security Roles

24 Oracle GoldenGate Administration Server - Housekeeping

25 Oracle GoldenGate Administration Server - Overview Access via URL (http(s)://<hostname>:<port>) i.e. Manage Extracts/Replicats Create, delete, start/stop, reports, statistics View critical events/messages View notifications per page

26 Oracle GoldenGate Administration Server - Overview Access via URL (http(s)://<hostname>:<port>) i.e. Manage Extracts/Replicats Create, delete, start/stop, reports, statistics View critical events/messages View notifications per page

27 Oracle GoldenGate Administration Server - Configuration Example: Create Extract process

28 Oracle GoldenGate Administration Server - Configuration Example: Create Extract process

29 Oracle GoldenGate Administration Server - Configuration Example: Create Extract process (cont ed)

30 Oracle GoldenGate Administration Server - Configuration Example: Create Extract process

31 Oracle GoldenGate Administration Server - Details

32 Oracle GoldenGate Administration Server - Details

33 Oracle GoldenGate Administration Server - Details

34 Oracle GoldenGate Distribution Server Access via URL (http(s)://<hostname>:<port>) i.e. Replacement for the Extract Pump process Quickly identify source, trail file, and destination Distributes one or more trails to one or more destination Does not support transformations Supports multiple transport protocols Tunable per network path

35 Communication Protocol Industry standard HTTP(S) initiated full-duplex streaming protocol Distribution Service wss udt ws ogg Receiver Service Extract Trail Files Trail Files Replicat SSL based security Can seamlessly traverse through HTTP forward/reverse proxy servers

36 Oracle GoldenGate Distribution Server - Setup Supports 4 transfer protocols OGG, WS, WSS, UDT Note: For WS/WSS, target credential needs to be setup that connects to the receiver service UDT UDP transport for wide-area networks Faster than TCP over wide-area with high latency OGG inter-operability with traditional deployments

37 Oracle GoldenGate Distribution Server - Details Quickly see path information Status Source/Target trails DB Name Trail name Configure Application Options Configure TCP Options

38 Oracle GoldenGate Distribution Server - Details Quickly see path information Status Source/Target trails DB Name Trail name Configure Application Options Configure TCP Options

39 Oracle GoldenGate Receiver Server Access via URL (http(s)://<hostname>:<port>) i.e. Replacement for Collector Single source that handles multiple incoming trail files View statistics of path Tune network parameters

40 Oracle GoldenGate Receiver Server - Details See Network Statistics and File IO Statistics Statistics include: Trail file name Port Bytes received Bytes written out Wait times

41 Performance Metrics Server Metrics Service Separate the metrics collection and repository from the administrative layer All GoldenGate processes send metrics via UDP to the metrics server Metrics server provide RESTful API to query metrics. Responses in either XML format (12.2) JSON format Supports integration with third party tools MUST have Oracle Management Pack for Oracle GoldenGate

42 Oracle GoldenGate Metric Server Metrics captured in real-time No Repository

43 Oracle GoldenGate Administration Client (adminclient) New thin client command line tool Offers similar functionality as GGSCI, now includes rlwrap Runs from $OGG_HOME/bin $OGG_HOME can be on any machine Supported on Windows and Linux

44 AdminClient (connecting) sf_vm]$./adminclient Oracle GoldenGate Administration Client for Oracle Version OGGCORE_MAIN_PLATFORMS_ Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved. Linux, x64, 64bit (optimized) on Mar :30:38 Operating system character set identified as UTF-8. GG (not connected) 1> connect deployment Atlanta as oggadmin password ******** OGG ( deployment_1) 2> Address and Port for Service Manager Deployment to connect to Administrator name and password

45 Administration Client (adminclient) Interact with multiple deployments from a single command line tool Runs from any machine using RESTful calls

46 AdminClient (deployments)

47 AdminClient (help)

48 Agenda Oracle GoldenGate Oracle GoldenGate Classic Architecture vs. Microservices Architecture WebPage Interfaces Directory Structure Rest Api

49 Directory Overview Old File System layout Classic Architecture $OGG_HOME is read-write Can support multiple process groups but only one installation Everything in one directory CREATE SUBDIRS Upgrade path - overwrite binaries, manual moving file

50 Directory Overview New File System layout Microservices Architecture $OGG_HOME should be read-only One to Multiple deployments can share same $OGG_HOME Deployment Homes can be customized One step upgrade New terms/variables to be used DEPLOYMENT_HOME $OGG_ETC_HOME / $OGG_VAR_HOME

51 New Directory Structure Number of Deployments: (ServiceManager + N)

52 Oracle GoldenGate Deployment Home etc example Deployment Homes $OGG_ETC_HOME $OGG_VAR_HOME

53 Oracle GoldenGate Deployment Home var example Deployment Homes $OGG_ETC_HOME $OGG_VAR_HOME

54 Agenda Oracle GoldenGate Oracle GoldenGate Classic Architecture vs. Microservices Architecture WebPage Interfaces Directory Structure Rest Api

55 Oracle GoldenGate RESTful APIs Representational State Transfer (RESTful) Application Program Interface (API) Uses HTTP requests GET, PUT, POST, and DELETE Can be used with many different programming languages PL/SQL Perl Shell JSON Etc Simple interaction from command line using CURL

56 Oracle GoldenGate RESTful APIs JSON Documents JavaScript Object Notation (JSON) An open-standard file format used to transmit data objects consisting of attributevalue pairs and array data types Media Type: application/json Example create credential alias: { } "$schema":"ogg:credentials", "userid":"ggate", "password":"ggate"

57 Oracle GoldenGate RESTful APIs GET GET option allows you to retrieve items back from a service Example GET associated services curl -u gg123:gg123 -H "Content- Type:application/json" -H "Accept:application/json" -X GET yments/ggsrc/services python - mjson.tool... "$schema": "ogg:collection", "items": [ { "$schema": "ogg:collectionitem", "links": [ { "href": " "mediatype": "application/json", "rel": "parent" }, { "href": " vr", "mediatype": "application/json", "rel": "canonical" } ], "name": "adminsrvr" }, {...

58 Oracle GoldenGate RESTful APIs POST GET option allows you to retrieve items back from a service Example POST stats command curl -u gg123:gg123 -H "Content- Type:application/json" -H "Accept:application/json" -X POST cts/ggsrc/command -d '{"command": STATS"}' python - mjson.tool "$schema": "ogg:stats", "stats": [ { "operations": [ { "count": 0, "operation": "insert" }, { "count": 3, "operation": "update" }, { "count": 0, "operation": "delete" }, { "count": 0, "operation": "discard" }, { "count": 3, "operation": "operations" } ], "since": " T15:29:09.298Z", "sourcetable": "GG123.GG_HEARTBEAT_SEED", "targettable": "GG123.GG_HEARTBEAT_SEED" }...

59 Oracle GoldenGate RESTful APIs POST GET option allows you to retrieve items back from a service Example POST stop extract curl -u gg123:gg123 -H "Content- Type:application/json" -H "Accept:application/json" -X POST cts/ggsrc/command -d '{"command":"stop"}' python - mjson.tool } "messages": [], "response": { "$schema": "er:commandresult", "reply": "PENDING", "replydata": {} }

60 Additional Information Documentation OTN

61 Oracle Cloud Workshops auf der DOAG Konferenz 2017 Im Foyer Shanghai-Seoul Einstieg in die jeweilige Session jederzeit möglich! Dienstag Uhr Oracle Datenbank und GoldenGate in der Oracle Cloud Uhr IT Systems Management mit der Oracle Management Cloud Mittwoch Uhr Agile Entwicklung von Microservices in der Oracle Cloud Uhr Oracle Datenbank und GoldenGate in der Oracle Cloud Donnerstag Uhr IT Systems Management mit der Oracle Management Cloud Uhr Agile Entwicklung von Microservices in der Oracle Cloud 65

62

63