<Insert Picture Here> Oracle MAA und RAC Best Practices und Engineered Systems

Transcription

1 <Insert Picture Here> Oracle MAA und RAC Best Practices und Engineered Systems Jörg Eggelsmann Leitender Systemberater STCC Nord DB

2 Agenda Maximum Availability Architecture: MAA RMAN Recovery Advisor Data Guard Grid Infrastructure ASM RAC RAC experiences Oracle Database Appliance 2 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

3 Traditional Database HA Clusterware 3 rd party or OS-dependent 3 rd party Volume Manager Idle Failover Server Idle DR Servers integrating disparate components? Monolithic often idle hardware Human Errors? Storage mirroring Corruptions? 3 rd Party Backup Software Upgrades? 3 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

4 Oracle Integrated Software Environment Address both Planned and Unplanned Downtime Oracle Maximum Availability Architecture Media and Storage Failures Server Failure Database, System, Cluster, & Site Outages Human or Application Error Flexible Maintenance and Migrations Maintainance 4 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

5 Oracle Integrated Software Environment Address both Planned and Unplanned Downtime Oracle Maximum Availability Architecture Automatic Storage Management Recovery Manager (RMAN), Oracle Secure Backup Storage failure Data recovery Backups Oracle RAC Instance failure Server failure Rolling maintenance Performance scale-out Online instance relocation Consolidation Oracle Clusterware Data Guard Database failure System failure Site failure Zero data loss Automatic failover Corruption protection Rolling upgrade Read-only offload Backup offload Flashback Fast point-in-time recovery Granular repair of logical corruptions Transaction Table Database GoldenGate Heterogeneous migrations Bi-directional and multi-master replication Zero downtime maintenance Online Redefinition and Reorganization 5 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

6 Agenda Maximum Availability Architecture: MAA RMAN Recovery Advisor Data Guard Grid Infrastructure ASM RAC RAC experiences Oracle Database Appliance 6 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

7 Data Recovery Advisor Reduces downtime by eliminating confusion list failure, advise failure, repair failure 7 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

8 Agenda Maximum Availability Architecture: MAA RMAN Recovery Advisor Data Guard Grid Infrastructure ASM RAC RAC experiences Oracle Database Appliance 8 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

9 Oracle Data Guard Physical & Logical Standby Logical Standby Database Production Database Physical Standby Database t=-1h Data Guard Broker Enterprise Manager Grid Control Physical Standby Database 9 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

10 Active Data Guard More than Data Protection Real-time Queries Real-time Reporting Continuous Redo Shipment and Apply Offload Fast Incremental Backups Production Database Active Standby Database 10 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

11 Active Data Guard More than Data Protection Real-time Queries Real-time Reporting Continuous Redo Shipment and Apply Automatic Block Repair Offload Fast Incremental Backups Production Database Active Standby Database 11 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

12 Data Guard: additional Options Rolling Upgrade with Logical Standby RAC Single Instances Transient Logical Standby Convert a Physical Standby to a Logical (and Back) Rolling Upgrades Snapshot Standby Use as test system in conjunction with Real Application Testing 12 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

13 Agenda Maximum Availability Architecture: MAA RMAN Recovery Advisor Data Guard Grid Infrastructure ASM RAC RAC experiences Oracle Database Appliance 13 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

14 Oracle Clusterware The Basis for Oracle RAC (and more) Node 1 Node 2 Node Oracle RAC... App A Oracle ASM / Cloud File System Oracle Clusterware Protected Node n Protected App B Consolidated Pool of Storage with Automatic Storage Management (ASM) The foundation for Oracle Real Application Clusters and the Oracle Cloud File System The infrastructure for the management of all kinds of applications 14 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

15 Agenda Maximum Availability Architecture: MAA RMAN Recovery Advisor Data Guard Grid Infrastructure ASM RAC RAC experiences Oracle Database Appliance 15 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

16 ASM / ACFS Enhancements Extending ASM to manage ALL data High Performance LVM Load Balancing Striping Redundancy Online Maintenance EM, ASMCA, CLI 16 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

17 Agenda Maximum Availability Architecture: MAA RMAN Recovery Advisor Data Guard Grid Infrastructure ASM RAC RAC experiences Oracle Database Appliance 17 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

18 Oracle Real Application Clusters (RAC) HR SALES DWH HR Active/active Cluster Highly available Scale-out Workload Management Consolidation 18 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

19 The basic RAC idea: 1 database, n instances Public Lan Public Lan Node 1 Node 2 Node... Node n-1 Node n Server Interconnect (with Switch) Shared Storage 19 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

20 The basic RAC idea: 1 database, n instances Public Lan Public Lan Node 1 Node 2 Node... Node n-1 Node n Oracle RAC Server Interconnect (with Switch) Oracle ASM / ACFS Oracle Clusterware Shared Storage 20 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

21 The basic RAC idea: 1 database, n instances All instances concurrently working on the same database Public Lan Public Lan Node 1 Node 2 Node... Node n-1 Node n Server Interconnect (with Switch) Shared Cache Shared Storage 21 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

22 Extended RAC Active/Active Cluster Host based Mirroring Protects from Site-Failure Third Voting Disk (NFS or iscsi) DB files (ASM) OCR Voting Disk ASM Mirror DB files (ASM) OCR Voting Disk 22 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

23 MAA with Oracle Oracle Cluster Infrastructure RAC active/active Cluster RAC Reader Farm Components from Oracle Oracle ASM Secure Backup Active Data Guard Use of all resources, RAC scale-out Human Error Protection Corruption Handling Rolling Upgrade 23 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

24 What next? Is Oracle RAC suitable for Database HA? Factors: Outage time? Workload balancing needed? Scale-out needed? Raised Complexity Is your application suited for RAC? Scalability Performance Coverage of Disaster Szenarios? Use in conjunction with SE 24 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

25 Agenda Maximum Availability Architecture: MAA RMAN Recovery Advisor Data Guard Grid Infrastructure ASM RAC RAC experiences Oracle Database Appliance 25 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

26 RAC Best Practices? 26 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

27 RAC Recommendations Use the Oracle Stack (Oracle Clusterware, Oracle RAC, and Automatic Storage Management) Use mirrored OCR and at least 3 voting disks Interconnect should be UDP over GigE For network interface high availability and scalability use OS NIC bonding Use the cluster verification utility for install, configuration, and component checking Use RACCheck (Doc ). Use Cluster Managed Services, Load Balancing Advisory, and Fast Connection Failover for workload management 27 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

28 RAC Pitfalls Unsupported Stack of Components Firewall Issues Interconnect is not exclusive Weak bandwidth an latency Jumbo Frames: Check NICs and Switches Mismatched MTU sizes in the communication path Don t forget to size right the UDP Buffer Separate the Redo Logs physically DNS Issues 28 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

29 RAC and Applications: Scalability If your application will scale transparently on SMP, then it is realistic to expect it to scale well on RAC, without having to make any changes to the application code. 29 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

30 RAC and Performance There are no special principles and practices how to develop RAC applications RAC If an application does not scale on one node it will not work faster in a cluster Poorly written SQL will not work better in a cluster Standard SQL optimization solutions solve > 80% performance problems 30 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

31 RAC Top 5 Performance Issues High counts of lost blocks(gc lost blocks, gc current/cr lost blocks) High log file sync waits High Waits on Mutexes High gc buffer busy, enq: TX -row lock contention, enq: TX - index contention, enq: TX - ITL allocate entry waits High CPU and memory consumption (Doc ID ) 31 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

32 RAC gc cr block lost is significant wait event Faulty or poorly seated cables/cards/switches Poorly sized UDP receive (rx) buffer sizes / UDP buffer socket overflows Mismatched MTU sizes in the communication path Misconfigured Jumbo Frames Interconnect LAN non-dedicated NIC force full duplex and duplex mode mismatch More: Doc ID Copyright 2011, Oracle and/or its affiliates. All rights reserved.

33 RAC High log file sync waits (Log file parallel write) Disk I/O: higher throughput and better response time (LFPW 1-2ms) Reduce number of commits (Batch Commit) NOLOGGING / UNRECOVERABLE possible? COMMIT NOWAIT option possible?(document ) Apply fixes for the known oracle bugs (Document ) Ensure that LMS processes are running in RT class 33 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

34 RAC - High Waits on Mutexes (pin S wait on X, pin S, Mutex X in AWR) Reduce Hard Parses, Invalidations and Reloads Use Bind Variables (Document ) High Version Counts Make SQL sharable (eg. CURSOR_SHARING) (Document ) Fix Oracle Bugs. Document , Document Note: Note: Copyright 2011, Oracle and/or its affiliates. All rights reserved.

35 RAC High gc buffer busy enq: TX - Index Contention (Re)create the index as reverse key index Hash partition the index Index key generation: Increase cache of sequence, cache noorder enq: TX - ITL allocate entry waits is no free ITL slot in the block Increase initrans value Table enq: TX - row lock contention, exclusive Lock Row level Lock SQL/Application Problem: Commit or roll back 35 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

36 RAC High CPU and memory consumption High CPU Utilization Identify and tune the SQLs Document Apply fix for the bug(s) Document Implement hugepages on Linux x86-64 (Large SGA) Document High Memory Utilization Ensure cursors are explicitly closed by application Avoid very large hash join and/or sort operation Apply fix for the applicable bug(s). Document Copyright 2011, Oracle and/or its affiliates. All rights reserved.

37 Miscellaneous Partition your data (partition options): tables Indexes Create tablespaces: locally managed, ASSM Do not run connection/disconnection continuously Do not use commit often High log file sync waits But do not use commit too seldom row lock contention 37 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

38 Agenda Maximum Availability Architecture: MAA RMAN Recovery Advisor Data Guard Grid Infrastructure ASM RAC RAC experiences Oracle Database Appliance 38 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

39 RAC Installation and Maintainance I m an experienced Oracle DBA, I have lots of Linux experience and scores of RAC installations under my belt. I also work for one of the best and more experienced RAC teams in the world. So when a customer asked how long it will take to install 2 node RAC, we answered 2 hours. It took over 30 hours 39 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

40 Oracle Database Appliance RAC in a Box Oracle Database Appliance certified by SAP 40 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

41 Fully Redundant Hardware 2 x dual-socket Oracle Linux servers Two 6-core Intel Xeon processors X5675 per server node 192 GB main memory 12 TB raw disk storage: 4TB or 6 TB eff. 292 GB solid state storage Built-in redundancy Storage, network, power and cooling Hot swappable 41 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

42 Highly Reliable Software Oracle Database 11.2 Enterprise Edition Real Application Clusters (RAC) RAC One Node Single Instance Oracle Grid Infrastructure Automatic Storage Management (ASM) Oracle Clusterware Oracle Linux Oracle Appliance Manager (pre-installed) Simple one-button installation, patching, and diagnostics 42 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

43 Oracle Appliance Manager Modules Configurator Module Deployment Module Storage Management Module Patching Module Validation & Diagnostic Tools Module # oakcli h Usage: oakcli show disk oakcli locate oakcli apply oakcli deploy oakcli update - show storage, core_config_key, expander, controller, diskgroup, - locates a disk - applies the core_config_key - deploys the Unbreakable Database Appliance - updates the Unbreakable Database Appliance 43 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

44 Pay-as-you-Grow Database Licensing Purchase single hardware configuration No hardware upgrades required License database software just for the cores you use Year 1 Year 2 Year 3 Start with minimum of 2 processor core licenses Scale to maximum of 24 processor core licenses Core factor is 0.5 Cores are disabled/enabled in BIOS 44 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

45 Sun ZFS Storage Appliance ODA Use Cases Expand capacity beyond 4 TB limit dnfs accessed file protocol (10 GbE) Storage Tiering Put frequently accessed tables on internal SAS disk, and less frequently used tables on ZFSSA Enable Hybrid Columnar Compression (HCC) Backup of ODA hosted databases, FRA Snap & Clone: Data files, RMAN Backups, 45 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

46 What Could You Do with 2000 Extra Hours? 46 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

47 Resources Oracle 11.2 Dokumentation Oracle Databse HA Oracle MAA MAA Best Practices html MAA Database Best Practices 47 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

48 Resources Data Guard 11g Installation and Configuration Best Practices on Oracle RAC Client Failover Best Practices for Data Guard 11g Release 2 Active Data Guard Best Practices Active Data Guard Best Practices Rolling Database Upgrades for Physical Standby Databases using Transient Logical Standby 11g Rolling Database Upgrades using Data Guard SQL Apply 48 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

49 49 Copyright 2011, Oracle and/or its affiliates. All rights reserved. Q&A