GoldenGate for NonStop SQL/MP to SQL Server

Transcription

1 GoldenGate for NonStop SQL/MP to SQL Server GoldenGate for NonStop SQL/MP to SQL Server Objective Upon completion of this lesson, you will be able to keep two heterogeneous databases synchronized, in this case NonStop audited SQL/MP to Microsoft SQL Server. During this lesson, you will learn how to: Prepare your environment to configure the GoldenGate processes Configure and execute the initial data load Configure and start the change capture of database operations Configure and start the change delivery of database operations NonStop Audited to MS SQL Server configuration The following diagram illustrates the configuration for replicating a NonStop audited database to an MS SQL Server target. NonStop <source> Source database Manager Windows server <target> Manager TMF Audit Trail Audserv Network Extract Collector Trails Replicat MS SQL Server <dsn> Target <schema> Page 1

2 GoldenGate for NonStop SQL/MP to SQL Server Overview of Tasks Prepare the Environment In order to execute this lesson, the GoldenGate application must be installed on both the source and target systems. The installation includes a sample database and scripts to generate initial data as well as subsequent update operations. The source and target tables are created and loaded with initial data. The GoldenGate Manager processes are also started so that other processes may be configured and started. Initial Data Load To initially load data between heterogeneous databases, use the GoldenGate application to perform your initial data synchronization while the application remains active. This lesson demonstrates using Extract to pull data from the source files and sending the data directly to Replicat on the target system. <source> NonStop Network <target> server MS SQL Server <dsn> Source database Initial Extract Initial Replicat Target <schema> Configure Change Capture For audited tables, the Extract process is configured to capture change data directly from the TMF audit trails and store the changes in a data queue known as GoldenGate remote trails. Configure Change Delivery Once the tables have been initially loaded with data and the Extract process is capturing all operations, the Replicat process is configured to deliver the captured operations to the target database. Page 2

3 Prepare the Environment Exercise 1. Prepare the Environment Objective The goals of this exercise are to: Configure and start the Manager processes. Prepare the source and target sample databases. Generate a data definition and copy it to the source system. Prepare the source NSK environment 1. Configure a GLOBALS parameter file on the source Execute the following commands on the <source> system. A GLOBALS parameter can be used to uniquely identify each GoldenGate instance when multiple environments exist on single NSK node. Use the two-character <prefix> that was defined in the Lab Preparation. Execute the following commands on the <source> system unless a <prefix> is not needed for your installation. TACL> VOLUME <install vol>.<install subvol> TACL> TEDIT GLOBALS ADD DEFINE =GGS_PREFIX, CLASS MAP, FILE $<prefix> 2. Configure the Manager process on the source system Execute the following commands on the <source> system. The Manager process is responsible for starting and monitoring all of the other GoldenGate processes. You will always have one manager process per GoldenGate instance. TACL> VOLUME <install vol>.<install subvol> TACL> RUN GGSCI GGSCI> TEDIT PARAMS MGRPARM -- GoldenGate Manager Parameter file TCPIPPROCESSNAME <tcpip process> PORT <port> GGSCI> START MANAGER, CPU 1, PRI 180 Page 3

4 Prepare the Environment Verify the results: GGSCI> INFO MANAGER 3. Create and load source files Execute the following commands on the <source> system. The installation includes scripts to create sample tables and generate initial data. Note that your <prefix> may need to be substituted for the GG in subvolumes GGSSOU and GGSDEF used in this section. TACL> SQLCI >> VOLUME <install vol>.ggssou; >> CREATE CATALOG; >> OBEY <install vol>.<install subvol>.demosql; >> OBEY <install vol>.<install subvol>.demolds (INITIAL); Verify the results: >> SELECT * FROM TCUSTMER; >> SELECT * FROM TCUSTMER; 4. Generate source definitions Execute the following commands on the <source> system. The DEFGEN utility is an interactive program that generates a source definition file describing the layouts of the files/tables that are to be replicated. TACL> VOLUME <install vol>.<install subvol> TACL> RUN DEFGEN Within DEFGEN, the following prompts and replies are issued: Enter definitions filename (or Exit): <install vol>.ggsdef.tcustdef File/Table to create definition for (or Exit): <install vol>.ggssou.tcustmer Definition retrieved. File/Table to create definition for (or Exit): <install vol>.ggssou.tcustord Definition retrieved. File/Table to create definition for (or Exit): EXIT 5. Move the source definitions to the target After exiting, FTP the file <install-vol>.ggsdef.tcustdef in ASCII format to the target <install location>/dirdef/source.def. 6. Generate target database scripts based on your source layouts Execute the following commands on the <source> system. The DDLGEN utility is an interactive program that generates a script containing table create statements for your target database based on the layouts of your source files/tables. Page 4

5 Prepare the Environment TACL> VOLUME <install vol>.<install subvol> TACL> RUN DDLGEN This initiates an interactive DDLGEN session with the prompts and replies listed below. Select the appropriate template name for your database from the table below. DDLGEN prompts: Enter definitions filename (or Exit): <install vol>.ggsdef.tcustsql DDL template file name (or Exit): <template name> File/Table to create definition for (or Exit): <install vol>.ggssou.tcustord Definition retrieved. File/Table to create definition for (or Exit): <install vol>.ggssou.tcustmer Definition retrieved File/Table to create definition for (or Exit): EXIT Template names: TMPLDB2 TMPLDB2M TMPLMSS TMPLORA TMPLSYB TMPLTDM DB2 for Windows and UNIX DB2 for the z/os and OS/39 MS SQL Server Oracle Sybase NonStop SQL/MP 7. Move the scripts After exiting, FTP the file <install-vol>.ggsdef.tcustsql in ASCII format to the target file <install location>/dirsql/tcust.sql. Prepare the SQL Server 2005 target system Install Manager 1. Configure Manager process on the target Execute the following commands on the <target> system. Note! The port numbers must be unique if you are installing GoldenGate more than once on the same server. Shell> cd <install location> Shell> ggsci Page 5

6 Prepare the Environment GGSCI> EDIT PARAMS MGR In the parameter file, enter the following parameter, then save and close the file. -- GoldenGate Manager Parameter file PORT <port> Start Manager. GGSCI> START MANAGER Verify the results: GGSCI> INFO MANAGER Steps to create the database 2. Create the database From Start>Programs, run SQL Server Management Studio. Enter the name of the local SQL Server instance, then click connect. Note: These instructions assume SQL Server Authentication which requires the more complex, double layer, of authentication. For only Windows Authentication, you can leave out most of the login and password entries. This displays the main SQL Server Management Studio dialog box. Right, right-click Database and select the New Database option. Page 6

7 Prepare the Environment The New Database dialog box appears. In the Name box, type the <database> from the Lab Preparation worksheet. Click OK to add the database. The main SQL Server Management Studio dialog box reappears. 3. Create SQL Server logins and users Right-click Security and select the New > Login. Page 7

8 Prepare the Environment The New Login dialog box appears. For Name, type the <login>. Select Authentication, If you selected SQL Server Authentication, type the <password> in the two Password fields. Under Defaults, select your <database> for Database, and leave Language set to the default. Page 8

9 Prepare the Environment Do not click OK. Instead, from Select a page in the upper left-hand box, double click User Mapping. In the upper Users mapped to this login area, activate the checkmark next to your database. In the lower Database role membership for <database> area,, check the box for the db_owner role. Click OK to exit. The next step creates a schema that you will associate with the new user. While you have the database selected, click on the New Query button in the main menu. The right side of the dialog displays the SQL Query interface. Type in: create schema <owner/schema>; Page 9

10 Prepare the Environment And press the Execute button from the toolbar. Now you need to associate the new user and the schema that you have created with the new database. Right click on the Security option that is under the database and select New > User. The New Database User dialog box appears. Enter the name of your user and press the browse button labeled with three dots ( ). Page 10

11 Prepare the Environment The Select Login dialog displays. Press the Browse button. Select your new user <login> from the list. Then press OK. Your selected user now displays in the object names box. Press OK to exit from Select Login. Back in the New Database User dialog, enter the <schema> you added as the default schema and press OK. 4. Create the ODBC system data source name The GoldenGate Extract process connects to a SQL Server database through an ODBC (Open Database Connectivity) connection. Both the Extract and Replicat components of GoldenGate require that a system data source name (DSN) be established, which stores the information about how to connect to the SQL Server. Click Start > Settings > Control Panel. Double-click Administrative Tools. Double-click Data Sources (ODBC) to open the ODBC Data Source Administrator dialog box. Click the System DSN tab, and then click the Add button. The Create New Data Source dialog box appears. Note: For SQL Server 2000, you may use any ODBC driver supported by the database. For SQL Server 2005, however, you must use the ODBC driver that is included with Microsoft Data Access Components (MDAC). Both Extract and Replicat will abend if the SQL Native Client ODBC driver that is provided with SQL Server 2005 is used. Page 11

12 Prepare the Environment Select the SQL Server driver and then click Finish. The Create a New Data Source to SQL Server dialog box appears. For Name, type <dsn>. For a description, type "GG Training Data source" when creating this source DSN. This field is optional. For Which SQL Server do you want to connect to, select the server name. Click Next. Answer the question about how SQL Server should verify the login by selecting the option that you entered when creating the <login>. Then type <login> in Login ID and <password> in Password. Page 12

13 Prepare the Environment Click Next. Make certain the default database is set to the database that you created; otherwise select Change the default database to: and select it. Leave the other settings to their defaults to use ANSI. Click Next. Leave the next dialog box set to the defaults, and click Finish. In the confirmation, click Test Data Source to test the connection. Close the confirmation dialog box and the Create a New Data Source box. Page 13

14 Prepare the Environment 5. Create the practice tables Execute the following commands on the <target> system. Go to SQL Server Management Studio, select your database and select New Query from the toolbar. If the correct database is not displayed in the drop down box (showing ggsdata above), click on the arrow and select it from the list. Click the File > Open and navigate to the demo_mss_create.sql script. Click Open to open the script in the New Query window. Execute the script by clicking the Execute Query button on the toolbar. Verify the results using the following commands in the input window: sp_help tcustmer go sp_help tcustord go Page 14

15 Initial Data Load Exercise 2. Initial Data Load using Direct Load Method Objective The goals of this exercise are to: Configure and execute the initial data load capture Configure and execute the initial data delivery Configure initial load 1. Add the initial load capture batch task group Execute the following commands on the <source> system to add an Extract group named EINI <unique id> 1. GGSCI> ADD EXTRACT EINI<unique id>, SOURCEISTABLE Verify the results: GGSCI> INFO EXTRACT *, TASKS 2. Configure the initial load capture parameter file Execute the following commands on the <source> system. GGSCI> EDIT PARAMS EINI<unique id> GoldenGate Initial Data Capture -- for TCUSTMER and TCUSTORD -- EXTRACT EINI<unique id> RMTHOST <target>, MGRPORT <port> RMTTASK REPLICAT, GROUP RINI<unique id> TABLE <install vol>.ggssou.tcustmer; TABLE <install vol>.ggssou.tcustord; 1 The process names used in lab exercises, for example EINIBD, are made up of 1) one character for the GoldenGate process (E for Extract, R for Replicat); 2) three or four to describe the process type (INI for initial data load, ORA for capture from or delivery to an Oracle database, etc.) and 3) two characters to create a unique identifier (usually your initials).. Page 15

16 Initial Data Load 3. Add the initial load delivery batch task group Execute the following commands on the <target> system. GGSCI> ADD REPLICAT RINI<unique id>, SPECIALRUN Verify the results: GGSCI> INFO REPLICAT *, TASKS 4. Configure the initial load delivery parameter file Execute the following commands on the <target> system. GGSCI> EDIT PARAMS RINI<unique id> Change Delivery parameter file for -- TCUSTMER and TCUSTORD changes -- REPLICAT RINI<unique id> TARGETDB <dsn>, USERID <login>, PASSWORD <password> SOURCEDEFS./dirdef/source.def DISCARDFILE./dirrpt/RINI<unique id>.dsc, PURGE MAP <install vol>.ggssou.tcustmer, TARGET <owner/schema>.tcustmer; MAP <install vol>.ggssou.tcustord, TARGET <owner/schema>.tcustord; 5. Execute the initial load process Execute the following commands on the <source> system. GGSCI> START EXTRACT EINI<unique id> Verify results using GGSCI: GGSCI> VIEW REPORT EINI<unique id> And using the SQL Query Analyzer on the <target>. Query Analyzer> SELECT * FROM <owner/schema>.tcustmer Query Analyzer> SELECT * FROM <owner/schema>.tcustord Page 16

17 Configure Change Capture Exercise 3. Configure Change Capture Objective The goals of this exercise are to: Configure and add the Extract process that will capture changes for audited tables. Add the trail that will store the changes. Start the capture process. Configure change capture 1. Create the Extract parameter file Execute the following commands on the <source> system to define an Extract to pull data from the TMF audit trails and route these changes to GoldenGate trails. GGSCI> TEDIT PARAMS ESQL<unique id> Change Capture parameter file to capture -- TCUSTMER and TCUSTORD Changes -- EXTRACT ESQL<unique id> TCPIPPROCESSNAME <tcpip process> RMTHOST <target>, MGRPORT <port> RMTTRAIL./dirdat/<trail id> TABLE <install vol>.ggssou.tcustmer; TABLE <install vol>.ggssou.tcustord; Note: Record the two characters selected for your <trail id>:. You will need this for step 3 and later when you set up the Replicat. 2. Add the Extract group Execute the following commands on the <source> system. GGSCI> ADD EXTRACT ESQL<unique id>, BEGIN NOW, CPU 1, PRI Define the GoldenGate trails Execute the following commands on the <source> system to add the trail declared in the ESQL<unique id> Extract parameters. Page 17

18 Configure Change Capture GGSCI> ADD RMTTRAIL./dirdat/<trail id>, EXTRACT ESQL<unique id>, MEGABYTES 5 4. Start the Extract process Execute the following commands on the <source> system. GGSCI> START EXTRACT ESQL<unique id> Verify the results: GGSCI> INFO ESQL<unique id> GGSCI> VIEW REPORT ESQL<unique id> Discussion points 1. Identifying a remote system What parameters are used to identify the remote target system? 2. Sizing the GoldenGate trail Where do you set how large a GoldenGate trail file may get before it rolls to the next file? What option do you use? Page 18

19 Configure Change Delivery Exercise 4. Configure Change Delivery Objective The goals of this exercise are to: Set up the checkpoint table on the target system. Create a named group that includes the Replicat process and the checkpoint tables. Configure the Replicat group by adding parameters. Start the Replicat group. Set up checkpoints 1. Edit a GLOBALS file on the target system Execute the following commands on the <target> system. Edit the GLOBALS parameter file to add the checkpoint table. Shell> cd <install location> Shell> ggsci GGSCI> EDIT PARAMS./GLOBALS In the text editor, type: CHECKPOINTTABLE <owner/schema>.ggschkpt Record the checkpoint table owner and name, then save and close the file. Table owner name Note: You could name the table anything you want, but for training purposes we are using ggschkpt. 2. Activate the GLOBALS parameters For the GLOBALS configuration to take effect, you must exit the session in which the changes were made. Execute the following command to exit GGSCI. GGSCI> EXIT Page 19

20 Configure Change Delivery 3. Add a Replicat checkpoint table Execute the following commands on the <target> system. This step adds the checkpoint table that you specified when you created the GLOBALS parameter file. It will be created in the <dsn> database. Run GGSCI on the target. Shell> cd <install location> Shell> ggsci Execute the following commands in GGSCI. GGSCI> DBLOGIN SOURCEDB <dsn>, USERID <login>, PASSWORD <password> GGSCI> ADD CHECKPOINTTABLE Configure delivery 4. Add the Replicat checkpoint group Execute the following commands on the <target> system to create the Replicat group named RMSS<unique id>. GGSCI> ADD REPLICAT RMSS<unique id>, EXTTRAIL./dirdat/<trail id> Note: Refer to your Extract set up for the correct two-character <trail id>. 5. Create Replicat parameter file Execute the following command on the <target> system to edit the Replicat parameter file. GGSCI> EDIT PARAM RMSS<unique id> Add the following lines to the parameter file. REPLICAT RMSS<unique id> TARGETDB <dsn>, USERID <login>, PASSWORD <password> HANDLECOLLISIONS SOURCEDEFS./dirdef/source.def DISCARDFILE./dirrpt/RMSS<unique id>.dsc, PURGE MAP <install vol>.ggssou.tcustmer, TARGET <owner/schema>.tcustmer; MAP <install vol>.ggssou.tcustord, TARGET <owner/schema>.tcustord; 6. Start the Replicat process Execute the following commands on the <target> system. Execute the following command in GGSCI to start the Replicat process. Page 20

21 Configure Change Delivery GGSCI> START REPLICAT RMSS<unique id> Verify the results: GGSCI> INFO REPLICAT RMSS<unique id> Discussion points Search in the Windows//UNIX Reference Guide for the information on the following questions. 1. When to use HANDLECOLLISIONS When would you use HANDLECOLLISIONS? What does it do? 2. What information is supplied by SOURCEDEFS? 3. What is the purpose of the DISCARDFILE? Page 21

22 Generate Activity and Verify Results Exercise 5. Generate Activity and Verify Results Objective The goals of this exercise are to: Execute miscellaneous update, insert, and delete operations on the source system. Verify the delivery of the changes to the target. Turn off the error handling used for the initial load. Generate database operations 1. Execute miscellaneous update, insert, and delete operations Execute the following commands on the <source> system. TACL> SQLCI >> VOLUME <install vol>.ggssou; >> O <install vol>.<install subvol>.demolds (UPDATE); Verify change capture and delivery 2. Verify your results on the source system Execute the following commands on the <source> system. >> SELECT * FROM TCUSTMER; >> SELECT * FROM TCUSTORD; >> EXIT; Verify your target SQL Server 2005 results 3. Verify your results on the SQL Server 2005 target Verify and record processing statistics on the <target> with the following command in GGSCI. GGSCI> SEND REPLICAT RMSS<unique id>, REPORT GGSCI> VIEW REPORT RMSS<unique id> Verify the data by executing the following commands in the New Query window of SQL Server Management Studio on the source: Page 22

23 Generate Activity and Verify Results select * from TCUSTMER; select * from TCUSTORD; Turn off error handling 4. Turn off initial load error handling for the running delivery process Execute the following commands on the <target> system. GGSCI> SEND REPLICAT RMSS<unique id>, NOHANDLECOLLISIONS 5. Remove initial load error handling from the parameter file GGSCI> EDIT PARAMS RMSS<unique id> Remove the HANDLECOLLISIONS parameter. Page 23

24 Exercise Name Page 24