GoldenGate for Audited NonStop Enscribe to Oracle Using a Data Pump for CDC

Transcription

1 GoldenGate for Audited NonStop Enscribe to Oracle Using a Data Pump for CDC GoldenGate for Audited NonStop Enscribe to Oracle Using a Data Pump for CDC Objective Upon completion of this lesson, you will be able to keep two heterogeneous databases synchronized; in this case audited NonStop to Oracle. During this lesson, you will learn how to: Prepare your environment to configure the GoldenGate processes Configure and execute the initial data load Configure and start the change capture of database operations Configure and start the data pump Extract process. Configure and start the change delivery of database operations Audited NonStop to Oracle data pump configuration NonStop <source> Source database Manager TMF Audit Audserv Primary Extract Trails Data Pump Extract Network LUW server <target> Collector Manager Trails Oracle Replicat Target <schema> Page 1

2 GoldenGate for Audited NonStop Enscribe to Oracle Using a Data Pump for CDC The diagram illustrates a configuration using a primary Extract process creating a local trail. This trail is read by a data pump Extract, which then creates a remote trail on the target. The Replicat process applies the changes from the remote trail to the target database. The source data is in an audited data base on the HP NonStop and the target in an Oracle database on a Windows or UNIX platform. Overview of Tasks Prepare the Environment In order to execute this lesson, the GoldenGate application must be installed on both the source and target systems. The installation includes a sample database and scripts to generate initial data as well as subsequent update operations. The source and target tables are created and loaded with initial data. The GoldenGate Manager processes are started so that other processes may be configured and started. Initial Data Load To initially load data between heterogeneous databases, use the GoldenGate application to perform your initial data synchronization while the application remains active. This lesson demonstrates the direct method using Extract to pull data from the source files and sending the data directly to Replication the target system. <source> NonStop Windows or UNIX <target> server Network Oracle Source database Initial Extract Initial Replicat Target <schema> Configure Change Capture For audited tables, the Extract process is configured to capture change data directly from the TMF audit trails and store the changes in a data queue known as a GoldenGate local trail. A data pump Extract reads the local trail sending the data to a remote GoldenGate trail on the target. Configure Change Delivery Once the tables have been initially loaded with data and the Extract process is capturing all operations, the Replicat process is configured to deliver the captured operations to the target database. Page 2

3 Prepare the Environment Exercise 1. Prepare the Environment Objective The goals of this exercise are to: Configure and start the Manager processes. Prepare the source and target sample databases. Prepare the source NSK environment 1. Configure a GLOBALS parameter file on the source A GLOBALS parameter can be used to uniquely identify each GoldenGate instance when multiple environments exist on single NSK node. Use the two-character <prefix> that was defined in the Lab Preparation. Execute the following commands on the <source> system unless a <prefix> is not needed for your installation. TACL> VOLUME <install vol>.<install subvol> TACL> TEDIT GLOBALS ADD DEFINE =GGS_PREFIX, CLASS MAP, FILE $<prefix> 2. Configure the Manager process on the source system The Manager process is responsible for starting and monitoring all of the other GoldenGate processes. You will always have one Manager process per GoldenGate instance. TACL> VOLUME <install vol>.<install subvol> TACL> RUN GGSCI GGSCI> TEDIT PARAMS MGRPARM GoldenGate Manager Parameter file TCPIPPROCESSNAME <tcpip process> PORT <port> GGSCI> START MANAGER, CPU 1, PRI 180 Page 3

4 Prepare the Environment Verify the results: GGSCI> INFO MANAGER 3. Create and load source files The installation includes scripts to create sample files and generate initial data. Note that your <prefix> may need to be substituted for the GG in subvolume GGSSOU used in this section. Move the script to the source data volume. TACL> VOLUME <install vol>.ggssou TACL> FUP DUP <install subvol>.demoldeo,* Run the scripts TACL> VOLUME <install vol>.ggsddl TACL> DDL/IN <install vol>.<install subvol>.demoeddl/ Create files and turn audit on TACL> VOLUME <install vol>.ggssou TACL> FUP /IN <install vol>.<install subvol>.demofups/ TACL> FUP ALTER ECUSTMER, AUDIT TACL> FUP ALTER ECUSTORD, AUDIT Load the files. TACL> RUN <install vol>.<install subvol>.demoldeo Within DEMOLDEO the following prompts and replies are issued: To perform an initial load enter (I), to update the database enter (U) or to exit the program enter (E).?I To perform an initial load enter (I), to update the database enter (U) or to exit the program enter (E).?E Verify the results: TACL> FUP COPY ECUSTMER TACL> FUP COPY ECUSTORD 4. Generate source definitions The DEFGEN utility is an interactive program that generates a source definition file describing the layouts of the files/tables that are to be replicated. Page 4

5 Prepare the Environment TACL> VOLUME <install vol>.<install subvol> TACL> RUN DEFGEN Within DEFGEN, the following prompts and replies are issued: Enter definitions filename (or Exit): File/Table to create definition for: Include DDL record definition (Y/N)? DDL Dictionary: DDL record or definition name Definition retrieved. File/Table to create definition for: Include DDL record definition (Y/N)? DDL Dictionary: DDL record or definition name Definition retrieved. <install vol>.ggsdef.ecustdef <install vol>.ggssou.ecustmer Y <install vol>.ggsddl ECUSTOMER-REC <install vol>.ggssou.ecustord Y <install vol>.ggsddl ECUSTORD-REC File/Table to create definition for (or Exit): EXIT 5. Move the source definitions to the target After exiting, FTP the file <install-vol>.ggsdef.ecustdef in ASCII format to the target <install location>/dirdef/source.def. 6.. Generate target database scripts based on your source layouts The DDLGEN utility is an interactive program that generates a script containing table create statements for your target database based on the layouts of your source files/tables. TACL> VOLUME <install vol>.<install subvol> TACL> RUN DDLGEN d <install vol>.ggsdef.ecustdef This initiates an interactive DDLGEN session with the following prompts and replies. Enter definitions filename (or Exit): <install vol>.ggsdef.ecustsql DDL template file name (or Exit): TMPLORA File/Table to create definition for (or Exit): <install vol>.ggssou.ecustord Definition retrieved. File/Table to create definition for (or Exit): <install vol>.ggssou.ecustmer Definition retrieved File/Table to create definition for (or Exit): EXIT 7. Move the scripts After exiting, FTP the file <install-vol>.ggsdef.ecustsql in ASCII format to the target file <install location>/dirsql/ecust.sql. Page 5

6 Prepare the Environment Prepare your target Oracle environment Configure the Manager 1. Configure the Manager process on the target system Execute the following commands on the <target> system. Start the command interface Shell> cd <install location> Shell> ggsci Specify the port that the Manager should use. GGSCI> EDIT PARAMS MGR GoldenGate Manager Parameter file PORT <port> Start Manager GGSCI> START MANAGER Verify the results: GGSCI> INFO MANAGER Create the tables 2. Create target Oracle tables Execute the following commands on the <target> system to run the script that creates the tables. Shell> cd <install location> Shell> sqlplus <login>/<password> Verify the results: SQL> desc tcustmer; SQL> desc tcustord; SQL> exit Page 6

7 Initial Data Load: NonStop Enscribe to Oracle Exercise 2. Initial Data Load: NonStop Enscribe to Oracle Objective The goals of this exercise are to: Configure and execute the initial data load capture Configure and execute the initial data delivery Configure initial load 1. Add the initial load capture batch task group Execute the following commands on the <source> system to add an Extract named EINI<unique id> 1. GGSCI> ADD EXTRACT EINI<unique id>, SOURCEISTABLE Verify the results: GGSCI> INFO EXTRACT *, TASKS 2. Configure the initial load capture parameter file GGSCI> EDIT PARAMS EINI<unique id> GoldenGate Initial Data Capture for ECUSTMER and ECUSTORD EXTRACT EINI<unique id> RMTHOST <target>, MGRPORT <port> RMTTASK REPLICAT, GROUP RINI<unique id> TABLE <install vol>.ggssou.ecustmer; TABLE <install vol>.ggssou.ecustord; 1 The process names used in lab exercises, for example EINIBD, are made up of 1) one character for the GoldenGate process (E for Extract, R for Replicat); 2) three or four to describe the process type (INI for initial data load, ORA for capture from or delivery to an Oracle database, etc.) and 3) two characters to create a unique identifier (usually your initials).. Page 7

8 Initial Data Load: NonStop Enscribe to Oracle 3. Add the initial load delivery batch task group Execute the following commands on the <target> system. GGSCI> ADD REPLICAT RINI<unique id>, SPECIALRUN Verify the results: GGSCI> INFO REPLICAT *, TASKS 4. Configure the initial load delivery parameter file Execute the following commands on the <target> system. GGSCI> EDIT PARAMS RINI<unique id> Change Delivery parameter file for TCUSTMER and TCUSTORD changes REPLICAT RINI<unique id> USERID <login>, PASSWORD <password> SOURCEDEFS./dirdef/source.def DISCARDFILE./dirrpt/RINI<unique id>.dsc, PURGE MAP <install vol>.ggssou.ecustmer, TARGET <owner/schema>.ecustmer; MAP <install vol>.ggssou.ecustord, TARGET <owner/schema>.ecustord; Note: Remember to use <db type>.def if you renamed the source.def when you transferred it to the <target>. 5. Execute the initial load process GGSCI> START EXTRACT EINI<unique id> Verify results: GGSCI> VIEW REPORT EINI<unique id> Page 8

9 Configure Change Capture using a Data Pump Exercise 3. Configure Change Capture using a Data Pump Objective The goals of this exercise are to: Configure and add the Extract process that will capture changes. Add the local trail that will store these changes. Configure and add a data pump Extract to read the local trail and create a remote trail on the target. Start the two Extract processes. Configure the primary Extract 1. Create the Extract parameter file Execute the following commands on the <source> system to define an Extract named EENS<unique id> to pull data from the Enscribe TMF audit trails and route these changes to a GoldenGate trail on the local system. GGSCI> EDIT PARAMS EENS<unique id> Change Capture parameter file to capture ECUSTMER and ECUSTORD Changes EXTRACT EENS<unique id> FETCHCOMPS EXTTRAIL <install vol>.ggsdat.<local trail> TABLE <install vol>.ggssou.ecustmer; TABLE <install vol>.ggssou.ecustord; Note: Record the two characters selected for your <local trail>:. You will need this information to set up the data pump Extract that will define and create the remote trail to be read by Replicat. 2. Add the Extract group Page 9

10 Configure Change Capture using a Data Pump GGSCI> ADD EXTRACT EENS<unique id>, BEGIN NOW, CPU 1, PRI 160 Verify results: GGSCI> INFO EXTRACT EENS<unique id> 3. Define the GoldenGate local trail Execute the following commands on the <source> system to add the trail declared in the EENS<unique id> Extract parameters. This will be located on the source system. GGSCI> ADD EXTTRAIL <install vol>.ggsdat.<local trail>, EXTRACT EENS<unique id>, MEGABYTES 5 Configure the data pump 4. Create the data pump parameter file. Execute the following commands on the <source> system to define a data pump Extract named EPMP<unique id> to pull data from the local GoldenGate trail and route these changes to the remote GoldenGate on the target. GGSCI> EDIT PARAMS EPMP<unique id> Data Pump parameter file to read the local trail of ECUSTMER and ECUSTORD Changes EXTRACT EPMP<unique id> TCPIPPROCESSNAME <tcpip process> PASSTHRU RMTHOST <target>, MGRPORT <port> RMTTRAIL./dirdat/<remote trail> TABLE <install vol>.ggssou.ecustmer; TABLE <install vol>.ggssou.ecustord; Note: Record the two characters selected for your <remote trail>:. You will need this information later when you define the remote trail that is read by Replicat. 5. Add the data pump Extract group GGSCI> ADD EXTRACT EPMP<unique id>, EXTTRAILSOURCE <install vol>.ggsdat.<local trail> Verify results: GGSCI> INFO EXTRACT EPMP<unique id> Page 10

11 Configure Change Capture using a Data Pump 6. Define the GoldenGate remote trail Execute the following commands on the <source> system to add the trail declared in the EPMP<unique id>extract parameters. This will be located on the target system. GGSCI> ADD RMTTRAIL./dirdat/<remote trail>, EXTRACT EPMP<unique id>, MEGABYTES 5 Start the Extract processes 7. Start the primary Extract process GGSCI> START EXTRACT EENS<unique id> Verify the results: GGSCI> INFO EXTRACT EENS<unique id> GGSCI> VIEW REPORT EXTRACT EENS<unique id> 8. Start the data pump Extract process GGSCI> START EXTRACT EPMP<unique id> Verify the results: GGSCI> INFO EXTRACT EPMP<unique id> GGSCI> VIEW REPORT EXTRACT EPMP<unique id> Discussion points 1. Using the FETCHCOMPS parameter What action is triggered by the FETCHCOMPS parameter? What type of input record is affected? Page 11

12 Configure Change Capture using a Data Pump 2. Using PASSTHRU What is the function of the PASSTHRU parameter? 3. Identifying the local extract trail A data pump moves data between an Extract and a Replicat. How does it know to read the local extract trail instead of a transaction log? 4. Identifying a remote system What parameters are used to identify the remote target system? 5. The advantage of a data pump What is the advantage of using a data pump when updating a remote system? Page 12

13 Configure Change Delivery Exercise 4. Configure Change Delivery Objective The goals of this exercise are to: Set up the checkpoint table on the target system. Create a named group that includes the Replicat process and the checkpoint tables. Configure the Replicat group by adding parameters. Start the Replicat group. Set up the checkpoint table 1. Create a GLOBALS file on the target system Execute the following commands on the <target> system. Create and edit the GLOBALS parameter file to add the checkpoint table. Shell> cd <install location> Shell> ggsci GGSCI> EDIT PARAMS./GLOBALS In the text editor, type: CHECKPOINTTABLE <owner/schema>.ggschkpt Record the checkpoint table owner and name, then save and close the file. Table owner name Note: You could name the table anything you want, but for training purposes we are using ggschkpt. Verify that the GLOBALS file was created in the root GoldenGate directory, and remove any file extension that was added. 2. Activate the GLOBALS parameters For the GLOBALS configuration to take effect, you must exit the session in which the changes were made. Execute the following command to exit GGSCI. Page 13

14 Configure Change Delivery GGSCI> EXIT 3. Add a Replicat checkpoint table On the <target> system, execute the following commands in GGSCI: Shell> cd <install location> Shell> ggsci GGSCI> DBLOGIN USERID <login>, PASSWORD <password> GGSCI> ADD CHECKPOINTTABLE Configure Change Delivery 4. Add the Replicat group Execute the following command on the <target> system to add a delivery group named RORA<unique id>. GGSCI> ADD REPLICAT RORA<unique id>, EXTTRAIL./dirdat/<trail id> Note: Refer to your Extract set up for the correct two-character <trail id>. 5. Create Replicat parameter file Execute the following commands on the <target> system to bring up the parameter file in the editor. GGSCI> EDIT PARAM RORA<unique id> Type in the following parameters Change Delivery parameter file to apply TCUSTMER and TCUSTORD Changes REPLICAT RORA<unique id> USERID <login>, PASSWORD <password> HANDLECOLLISIONS SOURCEDEFS./dirdef/source.def DISCARDFILE./dirrpt/RORA<unique id>.dsc, PURGE MAP <source vol>.ggssou.ecustmer, TARGET <owner/schema>.ecustmer; MAP <source vol>.ggssou.ecustord, TARGET <owner/schema>.ecustord; 6. Start the Replicat process GGSCI> START REPLICAT RORA<unique id> Verify the results: GGSCI> INFO REPLICAT RORA<unique id> Page 14

15 Configure Change Delivery Discussion points Search in the Windows/UNIX Reference Guide for the information on the following questions. 1. When to use HANDLECOLLISIONS For which stage of GoldenGate processing change capture, initial data load, or change delivery would you use HANDLECOLLISIONS? What does it do? 2. What information is supplied by SOURCEDEFS 3. What is the purpose of the DISCARDFILE? Page 15

16 Generate Activity and Verify Results Exercise 5. Generate Activity and Verify Results Objective The goals of this exercise are to: Execute miscellaneous update, insert, and delete operations on the source system. Verify the delivery of the changes to the target Generate database operations 1. Execute miscellaneous update, insert, and delete operations TACL> VOLUME <install vol>.ggssou TACL> RUN <install vol>.<install subvol>.demoldeo Within DEMOLDEO the following prompts and replies are issued: To perform an initial load enter (I), to update the database enter (U) or to exit the program enter (E).?U To perform an initial load enter (I), to update the database enter (U) or to exit the program enter (E).?E Verify change capture and delivery 2. Verify the results on your source system Execute the following commands on the <source> system to view the contents of the source Enscribe files. TACL> FUP COPY <install vol>.ggssou.ecustmer TACL> FUP COPY <install vol>.ggssou.ecustord 3. Verify your results on the target system Execute the following commands on the <target> system to verify the target data. Shell> cd <install location> Page 16

17 Generate Activity and Verify Results Shell> sqlplus <userid>/<password> SQL> select * from ecustmer; SQL> select * from ecustord; SQL> exit Shell> ggsci GGSCI> SEND REPLICAT RORA<unique id>, REPORT GGSCI> VIEW REPORT RORA<unique id> Turn off error handling 4. Turn off initial load error handling for the running delivery process GGSCI> SEND REPLICAT RORA<unique id>, NOHANDLECOLLISIONS 5. Remove initial load error handling from the parameter file GGSCI> EDIT PARAMS RORA<unique id> Remove the HANDLECOLLISIONS parameter Page 17