Oracle Exadata Recipes

Transcription

1 Oracle Exadata Recipes A Problem-Solution Approach John Clarke Apress-

2 Contents About the Author About the Technical Reviewer Acknowledgments Introduction xxxiii xxxv xxxvii xxxix Part 1: Exadata Architecture 1 Chapter 1: Exadata Hardware Identifying Exadata Database Machine Components 3 Problem 3 Solution 3 How It Works Displaying Storage Server Architecture Details 10 Problem 10 Solution 10 How It Works Displaying Compute Server Architecture Details 14 Problem 14 Solution 14 How It Works Listing Disk Storage Details on the Exadata Storage Servers 17 Problem 17 Solution 17 How It Works 21 vii

3 1-5. Listing Disk Storage Details on the Compute Servers 23 Problem 23 Solution 23 How It Works Listing Flash Storage on the Exadata Storage Servers 27 Problem 27 Solution 27 How It Works Gathering Configuration Information for the InfiniBandSwitches 29 Problem 29 Solution 29 How It Works 32 Chapter 2: Exadata Software Understanding the Role of Exadata Storage Server Software 33 Problem 33 Solution 33 How It Works Validating Oracle 11gR2 Databases on Exadata 35 Problem 35 Solution 35 How It Works Validating Oracle 11gR2 Grid Infrastructure on Exadata 38 Problem 38 Solution 38 How It Works Locating the Oracle Cluster Registry and Voting Disks on Exadata 42 Problem 42 Solution 42 How It Works 43 viii

4 2-5. Validating Oracle 11gR2 Real Application Clusters Installation and Database Storage on Exadata 44 Problem 44 Solution 44 How It Works Validating Oracle 11gR2 Real Application Clusters Networking on Exadata 47 Problem 47 Solution 47 How It Works 52 a Chapter 3: How Oracle Works on Exadata Mapping Physical Disks, LUNs, and Cell Disks on the Storage Servers 53 Problem 53 Solution 53 How It Works Mapping ASM Disks, Grid Disks, and Cell Disks 58 Problem 58 Solution 58 How It Works Mapping Flash Disks to Smart Flash Storage 61 Problem 61 Solution 61 How It Works Identifying Cell Server Software Processes 62 Problem 62 Solution 63 How It Works Tracing Oracle I/O Requests on Exadata Compute Nodes 65 Problem 65 Solution 65 How It Works 65 ix

5 3-6. Validating That Your Oracle RAC Interconnect Is Using Inf iniband 68 Problem 68 Solution 68 How It Works Tracing cellsrv on the Storage Servers 71 Problem 71 Solution 71 How It Works 73 Part 2: Preparing for Exadata 75 Chapter 4: Workload Qualification Quantifying I/O Characteristics of Your Current Database 77 Problem 77 Solution 77 How It Works Conducting a Smart Scan Fit Analysis Using AWR 81 Problem 81 Solution 81 How It Works Conducting a Smart Scan Fit Analysis Using Exadata Simulation 87 Problem 87 Solution 87 How It Works Performing a Hybrid Columnar Compression Fit Assessment 93 Problem 93 Solution 93 How It Works 95 Chapter 5: Sizing Exadata Determining CPU Requirements 98 Problem 98 Solution 98 How It Works 107 x

6 5-2. Determining lops Requirements 108 Problem 108 Solution 108 How It Works Determining I/O Bandwidth Requirements 113 Problem 113 Solution 113 How It Works Determining ASM Redundancy Requirements 119 Problem 119 Solution 119 How It Works Forecasting Storage Capacity 122 Problem 122 Solution 122 How It Works Planning for Database Growth 125 Problem 125 Solution 125 How It Works Planning for Disaster Recovery 133 Problem 133 Solution 133 How It Works Planning for Backups 134 Problem 134 Solution 134 How It Works Determining Your Fast Recovery Area and RECO Disk Group Size Requirements 136 Problem 136 Solution 136 How It Works 138 xi

7 Chapter 6: Preparing for Exadata Planning and Understanding Exadata Networking 139 Problem 139 Solution 139 How It Works Configuring DNS 143 Problem 143 Solution 143 How It Works Running checkip.sh 144 Problem 144 Solution 144 How It Works Customizing Your InfiniBand Network Configuration 147 Problem 147 Solution 147 How It Works Determining Your DATA and RECO Storage Requirements 148 Problem 148 Solution 148 How It Works Planning for ASM Disk Group Redundancy 149 Problem 149 Solution 149 How It Works Planning Database and ASM Extent Sizes 152 Problem 152 Solution 152 How It Works 152 xii

8 6-8. Completing the Pre-Delivery Survey 153 Problem 153 Solution 153 How It Works Completing the Configuration Worksheet 154 Problem 154 Solution 154 How It Works 156 Part 3: Exadata Administration 157 Chapter 7: Administration and Diagnostics Utilities Logging in to the Exadata Compute and Storage Cells Using SSH 159 Problem 159 Solution 159 How It Works Configuring SSH Equivalency 161 Problem 161 Solution 161 How It Works Locating Key Configuration Files and Directories on the Cell Servers 162 Problem 162 Solution 162 How It Works Locating Key Configuration Files and Directories on the Compute Nodes 164 Problem 164 Solution 164 How It Works Starting and Stopping Cell Server Processes 165 Problem 165 Solution 165 How It Works 166 xiii

9 7-6. Administering Storage Cells Using CellCLI 167 Problem 167 Solution 167 How It Works Administering Storage Cells Using deli 168 Problem 168 Solution 168 How It Works Generating Diagnostics from the ILOM Interface 171 Problem 171 Solution 171 How It Works Performing an Exadata Health Check Using exachk 173 Problem 173 Solution 173 How It Works Collecting Compute and Cell Server Diagnostics Using the sundiag.sh Utility 176 Problem 176 Solution 176 How It Works Collecting RAID Storage Information Using the MegaCLI utility 178 Problem 178 Solution 178 How It Works Administering the Storage Cell Network Using ipconf 179 Problem 179 Solution 179 How It Works Validating Your InfiniBand Switches with the CheckSWProfile.sh Utility 181 Problem 181 Solution 181 How It Works 182 xiv

10 7-14. Verifying Your InfiniBand Network Topology 182 Problem 182 Solution 182 How It Works Diagnosing Your InfiniBand Network 183 Problem 183 Solution 183 How It Works Connecting to Your Cisco Catalyst 4948 Switch and Changing Switch Configuration 187 Problem 187 Solution 187 How It Works 188 Chapter 8: Backup and Recovery Backing Up the Storage Servers 189 Problem 189 Solution 189 How It Works Displaying the Contents of Your CELLBOOT USB Flash Drive 191 Problem 191 Solution 191 How It Works Creating a Cell Boot Image on an External USB Drive 192 Problem 192 Solution 193 How It Works Backing Up Your Compute Nodes Using Your Enterprise Backup Software 195 Problem 195 Solution 195 How It Works 196 XV

11 8-5. Backing Up the Compute Servers Using LVM Snapshots 196 Problem 196 Solution 197 How It Works Backing Up Your Oracle Databases with RMAN 201 Problem 201 Solution 201 How It Works Backing Up the InfiniBand Switches 202 Problem 202 Solution 203 How It Works Recovering Storage Cells from Loss of a Single Disk 205 Problem 205 Solution 205 How It Works Recovering Storage Cells from Loss of a System Volume Using CELLBOOT Rescue 210 Problem 210 Solution 210 How It Works Recovering from a Failed Storage Server Patch 222 Problem 222 Solution 222 How It Works Recovering Compute Server Using LVM Snapshots 223 Problem 223 Solution 223 How It Works Reimaging a Compute Node 229 Problem 229 Solution 229 How It Works 231 xvi

12 8-13. Recovering Your InfiniBand Switch Configuration 232 Problem 232 Solution 232 How It Works Recovering from Loss of Your Oracle Cluster Registry and Voting Disks 235 Problem 235 Solution 235 How It Works 237 Chapter 9: Storage Administration Building ASM Disk Groups on Exadata 239 Problem 239 Solution 239 How It Works Properly Configuring ASM Disk Group Attributes on Exadata 246 Problem 246 Solution 246 How It Works Identifying Unassigned Grid Disks 248 Problem 248 Solution 248 How It Works Configuring ASM Redundancy on Exadata 248 Problem 248 Solution 248 How It Works Displaying ASM Partner Disk Relationships on Exadata 253 Problem 253 Solution 253 How It Works 254 xvii

13 9-6. Measuring ASM Extent Balance on Exadata 254 Problem 254 Solution 254 How It Works Rebuilding Cell Disks 258 Problem 258 Solution 258 How It Works Creating Interleaved Cell Disks and Grid Disks 274 Problem 274 Solution 274 How It Works Rebuilding Grid Disks 282 Problem 282 Solution 282 How It Works Setting smart_scan_capable on ASM Disk Groups 286 Problem 286 Solution 286 How It Works Creating Flash Grid Disks for Permanent Storage 292 Problem 292 Solution 292 How It Works 296 Chapter 10: Network Administration Configuring the Management Network on the Compute Nodes 297 Problem 297 Solution 297 How It Works 299 xviii

14 10-2. Configuring the Client Access Network 299 Problem 299 Solution 299 How It Works Configuring the Private Interconnect on the Compute Nodes 301 Problem 301 Solution 302 How It Works Configuring the SCAN Listener 304 Problem 304 Solution 304 How It Works Managing Grid Infrastructure Network Resources 306 Problem 306 Solution 306 How It Works Configuring the Storage Server Ethernet Network 309 Problem 309 Solution 309 How It Works Changing IP Addresses on Your Exadata Database Machine 312 Problem 312 Solution 312 How It Works 330 Chapter 11: Patching and Upgrades Understanding Exadata Patching Definitions, Alternatives, and Strategies 331 Problem 331 Solution 331 How It Works 333 xix

15 11-2. Preparing to Apply Exadata Patches 333 Problem 333 Solution 333 How It Works Patching Your Exadata Storage Servers 335 Problem 335 Solution 335 How It Works Patching Your Exadata Compute Nodes and Databases 342 Problem 342 Solution 342 How It Works Patching the InfiniBand Switches 349 Problem 349 Solution 349 How It Works Patching Your Enterprise Manager Systems Management Software 352 Problem 352 Solution 352 How It Works 353 HChapter 12: Security Configuring Multiple Oracle Software Owners on Exadata Compute Nodes 355 Problem 355 Solution 355 How It Works Installing Multiple Oracle Homes on Your Exadata Compute Nodes 359 Problem 359 Solution 359 How It Works 361 XX

16 12-3. Configuring ASM-Scoped Security 362 Problem 362 Solution 362 How It Works Configuring Database-Scoped Security 364 Problem 364 Solution 364 How It Works 367 HPart 4: Monitoring Exadata 369 HChapter 13: Monitoring Exadata Storage Cells Monitoring Storage Cell Alerts 371 Problem 371 Solution 371 How It Works Monitoring Cells with Active Requests 376 Problem 376 Solution 376 How It Works Monitoring Cells with Metrics 381 Problem 381 Solution 381 How It Works Configuring Thresholds for Cell Metrics 389 Problem 389 Solution 389 How It Works Using deli with Special Characters 391 Problem 391 Solution 391 How It Works 393 xxi

17 13-6. Reporting and Summarizing metrichistory Using R 393 Problem Solution How It Works Reporting and Summarizing metrichistory Using Oracle and SQL 399 Problem Solution How It Works Detecting Cell Disk I/O Bottlenecks 400 Problem Solution How It Works Measuring Small I/O vs. Large I/O Requests 402 Problem Solution How It Works Detecting Grid Disk I/O Bottlenecks 404 Problem 404 Solution 404 How It Works Detecting Host Interconnect Bottlenecks 406 Problem 406 Solution 406 How It Works Measuring I/O Load and Waits per Database, Resource Consumer Group, and Resource Category 407 Problem 407 Solution 407 How It Works 409 xxii

18 Chapter 14: Host and Database Performance Monitoring Collecting Historical Compute Node and Storage Cell Host Performance Statistics 411 Problem 411 Solution 411 How It Works Displaying Real-Time Compute Node and Storage Cell Performance Statistics 414 Problem 414 Solution 414 How It Works Monitoring Exadata with Enterprise Manager 417 Problem 417 Solution 417 How It Works Monitoring Performance with SQL Monitoring 421 Problem 421 Solution 421 How It Works Monitoring Performance by Database Time 425 Problem 425 Solution 425 How It Works Monitoring Smart Scans by Database Time and AAS 432 Problem 432 Solution 432 How It Works Monitoring Exadata with Wait Events 435 Problem 435 Solution 435 How It Works 439 xxiii

19 14-8. Monitoring Exadata with Statistics and Counters 440 Problem 440 Solution 440 How It Works Measuring Cell I/O Statistics for a SQL Statement 442 Problem 442 Solution 442 How It Works 444 Part 5: Exadata Software 445 Chapter 15: Smart Scan and Cell Offload Identifying Cell Offload in Execution Plans 447 Problem 447 Solution 447 How It Works Controlling Cell Offload Behavior 450 Problem 450 Solution 450 How It Works Measuring Smart Scan with Statistics 451 Problem 451 Solution 451 How It Works Measuring Offload Statistics for Individual SQL Cursors 455 Problem 455 Solution 455 How It Works Measuring Offload Efficiency 457 Problem 457 Solution 457 How It Works 460 xxiv

20 15-6. Identifying Smart Scan from Trace Files 461 Problem 461 Solution 461 How It Works Qualifying for Direct Path Reads 463 Problem 463 Solution 463 How It Works Influencing Exadata's Decision to Use Smart Scans 469 Problem 469 Solution 469 How It Works Identifying Partial Cell Offload 473 Problem 473 Solution 473 How It Works Dealing with Fast Object Checkpoints 475 Problem 475 Solution 475 How It Works 475 Chapter 16: Hybrid Columnar Compression Estimating Disk Space Savings for HCC 477 Problem 477 Solution 477 How It Works Building HCC Tables and Partitions 479 Problem 479 Solution 479 How It Works 482 XXV

21 16-3. Contrasting Oracle Compression Types 482 Problem 482 Solution 482 How It Works Determining the Compression Type of a Segment 487 Problem 487 Solution 487 How It Works Measuring the Performance Impact of HCC for Queries 488 Problem 488 Solution 489 How It Works Direct Path Inserts into HCC Segments 492 Problem 492 Solution 492 How It Works Conventional Inserts to HCC Segments 493 Problem 493 Solution 494 How It Works OMLand HCC 495 Problem 495 Solution 495 How It Works Decompression and the Performance Impact 502 Problem 502 Solution 502 How It Works 504 xxvi

22 Chapter 17: I/O Resource Management and Instance Caging Prioritizing I/O Utilization by Database 505 Problem 505 Solution 505 How It Works Limiting I/O Utilization for Your Databases 507 Problem 507 Solution 507 How It Works Managing Resources within a Database 508 Problem 508 Solution 508 How It Works Prioritizing I/O Utilization by Category of Resource Consumers 511 Problem 511 Solution 511 How It Works Prioritizing I/O Utilization by Categories of Resource Consumers and Databases 513 Problem 513 Solution 513 How It Works Monitoring Performance When IORM Is Enabled 517 Problem 517 Solution 517 How It Works Obtaining IORM Plan Information 521 Problem 521 Solution 521 How It Works 525 xxvii

23 17-8. Controlling Smart Flash Cache and Smart Flash Logging with IORM 526 Problem 526 Solution 526 How It Works Limiting CPU Resources with Instance Caging 527 Problem 527 Solution 527 How It Works 527 (Chapter 18: Smart Flash Cache and Smart Flash Logging Managing Smart Flash Cache and Smart Flash Logging 529 Problem 529 Solution 530 How It Works Determining Which Database Objects Are Cached 532 Problem 532 Solution 532 How It Works Determining What's Consuming Your Flash Cache Storage 534 Problem 534 Solution 534 How It Works Determining What Happens When Querying Uncached Data 536 Problem 536 Solution 536 How It Works Measuring Smart Flash Cache Performance 541 Problem 541 Solution 541 How It Works 546 xxviii

24 18-6. Pinning Specific Objects in Smart Flash Cache 546 Problem 546 Solution 546 How It Works Quantifying Benefits of Smart Flash Logging 549 Problem 549 Solution 549 How It Works 550 S Chapter 19: Storage Indexes Measuring Performance Impact of Storage Indexes 553 Problem, 553 Solution 553 How It Works Measuring Storage Index Performance with Not-So-Well-Ordered Data 557 Problem 557 Solution 557 How It Works Testing Storage Index Behavior with Different Query Predicate Conditions 559 Problem 559 Solution 560 How It Works Tracing Storage Index Behavior 564 Problem 564 Solution 565 How It Works Tracing Storage Indexes When More than Eight Columns Are Referenced 568 Problem 568 Solution 568 How It Works 572 xxix

25 19-6. Tracing Storage Indexes when DML Is Issued against Tables 573 Problem 573 Solution 573 How It Works Disabling Storage Indexes 575 Problem 575 Solution 575 How It Works Troubleshooting Storage Indexes 576 Problem 576 Solution 576 How It Works 576 Part 6: Post Implementation Tasks 577 Chapter 20: Post-Installation Monitoring Tasks Installing Enterprise Manager 12c Cloud Control Agents for Exadata 579 Problem 579 Solution 579 How It Works Configuring Enterprise Manager 12c Cloud Control Plug-ins for Exadata 592 Problem 592 Solution 592 How It Works Configuring Automated Service Requests 592 Problem 592 Solution 592 How It Works 597 Chapter 21: Post-Install Database Tasks Creating a New Oracle RAC Database on Exadata 599 Problem 599 Solution 599 How It Works 601 XXX

26 21-2. Setting Up a DBFS File System on Exadata 601 Problem 601 Solution 601 How It Works Configuring HugePages on Exadata 606 Problem 606 Solution 606 How It Works Configuring Automatic Degree of Parallelism 610 Problem 610 Solution 610 How It Works Setting I/O Calibration on Exadata 612 Problem 612 Solution 612 How It Works Measuring Impact of Auto DOP and Parallel Statement Queuing 614 Problem 614 Solution 614 How It Works Measuring Auto DOP and In-Memory Parallel Execution 620 Problem 620 Solution 621 How It Works Gathering Optimizer Statistics on Exadata 625 Problem 625 Solution 626 How It Works 630 Index 633 xxxi