enterprise professional expertise distilled Implementer's guide Oracle GoldenGate 11g

Transcription

1 Oracle GoldenGate 11g Implementer's guide Design, install, and configure high-performance data replication solutions using Oracle GoldenGate John P. Jeffries PUBLISHING enterprise professional expertise distilled BIRMINGHAM - MUMBAI

2 Preface 1 Chapter 1: Getting Started 9 What is GoldenGate? 9 Oracle GoldenGate evolution 11 Oracle GoldenGate solutions 11 Oracle GoldenGate technology overview 13 The capture process (Extract) 13 Trail files 14 Data Pump 14 Server Collector 14 The Apply process (Replicat) 14 The Manager process 15 GGSCI 15 Process data flow 16 Oracle GoldenGate architecture 16 One-to-One 17 One-to-Many 18 Many-to-One 19 Cascading 20 Bi-directional (Active-Active) 21 Bi-directional (Active-Passive) 22 Supported platforms and databases 22 Oracle GoldenGate topology 24 Process topology 25 The rules 25 Position 26 Statistics 26 Design considerations 27 Choosing a solution 28

3 Network 28 Database schema 29 What to Replicate? 29 Object mapping Initial Load 31 and data selection 29 CSN co-ordination 31 Trail file format 31 Summary 32 Chapter 2: Installing and Preparing GoldenGate 33 Prerequisites 34 Downloading the software 34 Software requirements 35 Hardware requirements 36 Memory 36 CPU 36 Network 37 Disk 37 Software installation 37 Installing GoldenGate 38 The subdirectories 41 dirchk 41 dirdat 41 dirdef 42 dirpcs 42 dirprm 42 dirrpt 42 dirsql 42 dirtmp 43 Preparing the environment 43 Preparing the database for data replication 43 Enabling supplemental logging 43 Preparing the operating system 45 Creating the initial configuration 45 Creating the GoldenGate administrator 46 The Manager process 46 The Extract process 47 Creating and starting an Extract process 48 The Data Pump process 49 The Replicat process 50 Creating and starting a Replicat process 51 Configuration summary 52 Uninstalling GoldenGate from Linux/UNIX 52 Summary 53

4 Chapter 3: Design Considerations 55 Replication methods 56 Active-active 56 Active-passive 57 Cascading 58 Physical Standby 59 Networking 60 Surviving network outages 60 Redundant networks 60 Non-functional requirements (NFRs) 62 Latency 62 Availability 63 Backup and recovery 63 Hardware considerations 64 Computing architectures 64 Grid computing 65 Single server 65 Clusters 66 Machines 66 The x86-64 Linux Server 66 The Database Machine 67 Scaling up and out 67 Changed data management 68 Point in Time Recovery (PITR) 69 Oracle Recovery Manager (RMAN) 69 Flashback 69 SAN Snapshot 70 Summary 70 Chapter 4: Configuring Oracle GoldenGate 71_ GoldenGate parameters 72 Configuring the Initial Load 74 Choosing a method 74 File to Replicat 75 File to database utility 77 Direct Load 78 Direct Bulk Load 80 Performing the Initial Load 81 Example architecture 81 File to Replicat method 82 Configuring Initial Data Capture 82 Configuring Initial Data Delivery 85 Configuring Change Data Capture 87 Configuring Change Delivery 89

5 Testing Change Data Capture and Delivery 92 Stopping GoldenGate processes 94 More about trail files 96 The trail 96 Trail file purging 96 Configuring the Manager process 97 Summary 97 Chapter 5: Configuration Options 99 Using BATCHSQL 99 SQL cache 100 Exceptions 101 When to use BATCHSQL 101 Data compression 102 Compressing the Data Stream 102 The COMPRESS option 103 Oracle table compression 104 Security features 105 Data encryption 105 Password encryption 108 Default method 108 Named method 109 Event Actions 110 Event Records 110 Bi-directional configuration options 112 Loop detection 113 Conflict detection 113 Conflict resolution 114 Oracle sequences 114 Oracle triggers 115 Heterogeneous environments 115 Specific options 116 Microsoft SQL Server 116 IBM DB2 116 The DEFGEN utility 116 DDL replication 117 The DDL parameter 117 Filtering 117 Mapping options 118 Initial setup 119 Known issues 121 Using DUMPDDL 122

6 DDL OPTIONS 124 Summary 125 Chapter 6: Configuring GoldenGate for HA 127 GoldenGate on RAC 127 Shared storage 128 Automatic Storage Management Cluster File System (ACFS) 128 Database File System (DBFS) 128 Configuring Clusterware for GoldenGate 129 The Virtual IP 129 Creating a GoldenGate application 131 Increasing system resilience 135 GoldenGate on Exadata 135 Configuration 136 Creating the Extract process 136 Creating the Replicat process 138 Failover 138 Automatic failover 138 Manual failover 139 Relocating a service 139 Relocating a VIP 139 Summary 140 Chapter 7; Advanced Configuration 141 Mapping your data 142 Column mapping 142 Using the COLMAP option 143 The COLMATCH option 144 Using the DEFGEN utility 144 Data selection and filtering 148 The WHERE clause 148 The FILTER clause 149 Loop detection 150 Active-active 150 Cascade 151 Data transformation 151 Truncation and extraction 152 Concatenation 152 Substitution 152 Case changing 152 Numeric conversions 153 Date conversions 153 DDL support 154

7 The SQLEXEC parameter 155 Data lookups 155 Executing stored procedures 155 Executing SQL 156 Executing DML 157 Handling errors 157 Scheduling jobs 158 Using and defining macros 159 User tokens 160 User Exits 161 Calling C routines 161 Sample User Exits 162 Source files explained 162 Using logic in data replication 163 Licensing 165 Summary 166 Chapter 8: Managing Oracle GoldenGate 167 Command level security 168 The CMDSEC file 168 Trail file management 169 Managing process startup 170 Managing TCP/IP errors 171 The tcperrs file 171 Reporting and statistics 172 Monitoring errors 172 Monitoring latency 173 A bespoke solution to monitoring and reporting 174 Measuring throughput 181 Data throughput 181 Operation throughput 183 Summary 184 Chapter 9: Performance Tuning 185 Before tuning GoldenGate 186 Online redo 186 Large objects (LOBs) 187 Base lining 187 Balancing the load across parallel process groups 188 Considerations for using parallel process groups 189 Splitting large tables into row ranges across process groups 189 The RANGE function 189

8 Adding Replicats with RANGE function 190 Configuring multiple parallel process groups 194 Source system configuration 195 Parallel process parameter files 195 Creating extract parallel process groups 196 Target system configuration 198 Parallel process parameter files 198 Creating Replicat parallel process groups 200 Improving Replicat throughput 201 New releases 202 DBFS enhancements 202 New redo log OCI API 203 Tuning the network 203 Linux TCP tuning 204 Configuring a Bequeath connection 205 Summary 207 Chapter 10: Troubleshooting GoldenGate 209 Troubleshooting tips 209 Replication not working? 210 The CHECKPARAMS parameter 211 Adjusting the start point 212 Altering Extract processes in RAC environments 213 Checking process checkpoints 213 Investigating network issues 215 TCP/IP 215 SQL*Net 216 Investigating Oracle errors 218 Exception handling 219 Creating an Exceptions handler 219 Viewing Exceptions 221 Before and after images 222 Handling Oracle Sequences 223 Using LOGDUMP 224 Opening files 225 Viewing the header record 226 Viewing the transaction record 228 Miscellaneous commands 229 Filtering records 231 Upgrading GoldenGate 231 Summary 232

9 Appendix A: GGSC) Commands 233 Appendix B: GoldenGate Installed Components 243 Appendix C: The Future of Oracle GoldenGate 247 Index 249