From Dataguard to Sharding High Availibility in the Oracle DB

Transcription

1 From Dataguard to Sharding High Availibility in the Oracle DB Matthias Fuchs, Principal Consultant, esentri AG Nürnberg DOAG Konferenz 16. November Nürnberg

2 From Data Guard to Sharding - High Availability in the Oracle DB > Data Guard - Introduction and Features > Implementation - Experiences and Examples out of customer projects > Theory Potential and Limitations > Oracle Sharding webscale Availability 2

3 Bio Matthias Fuchs Principal > > > > 10+ years Experience with Oracle Database, Middleware, SOA OCP Engineered Systems > Exadata > Exalogic Link with esentri 3

4 Data Guard - Introduction and Features 4

5 Data Guard Overview Datacenter 1 Primary Database Redo Transport Datacenter 2 Active Dataguard Read only Data Guard Broker Cloud Control 5

6 Data Guard Conclusion > transactional consistent copy of the primary database > transmitting redo data > single-instance Oracle database or an Oracle RAC > It isn t > Different Versions Physical - see valid for Standby first Patching Readme > Difference OS Dok ID: Microsoft Windows (32-bit), 10 - Linux (32-bit), 8 - Microsoft Windows IA (64-bit), 20 Solaris X-86 6

7 Data Guard Types of standby Databases > Physical > Logical > identical copy of the primary database > a physical standby database can receive and apply redo while it is open for read-only access > a physical standby database can be used to install eligible one-off patches, patch set updates (PSUs), and critical patch updates (CPUs), in rolling fashio > Contains the same logical information as the production database > kept synchronized with the primary database through SQL > The flexibility of a logical standby database lets you upgrade Oracle Database software (patch sets and new Oracle Database releases) > Snapshot Standby > fully updatable standby database > a snapshot standby database does not apply the redo data that it receives > updatable snapshot of a physical standby database > redo data received by a snapshot standby database is not applied until it is converted back into a physical standby 7

8 Data Guard Protection Modes > Maximum Availability > highest level of data protection that is possible without compromising the availability > If the primary does not receive acknowledgement from at least one synchronized standby Maximum Perfomance > SYNC/AFFIRM versus SYNC/NOAFFIRM (FastSync) > NetTimeout > Maximum Performance > This protection mode provides the highest level of data protection that is possible without affecting the performance of a primary database > ASYNC/NOAFFIRM > Standby hat keinen Einfluss auf Primary > Maximum Protection > Oracle recommends that a minimum of two standby databases be used to protect a primary database > SYNC/AFFIRM > Zero Data Loss 8

9 12c Transport Services > Synchronous redo transport > *.log_archive_dest_2= service=astandby LGWR SYNC.. > *.log_archive_dest_3= service=bstandby LGWR SYNC.. > Cascaded Redo Transport > Only physical standby databases can cascade redo > Oracle Active Data Guard option Sync/Affirm Async/Noaffirm 9

10 12c Transport Services Far Sync > Far Sync Instances - little disk and processing resources > manages a control file, receives redo, archives redo logs > No data files > Oracle Active Data Guard Far Sync feature Sync/Affirm Async/Noaffirm Control, Log Files No Data Files, No media Recovery Zero Data Loss target Open Read onlymöglich 1

11 Features 12c > Automatic Block Repair High Availability > Active Data Guard automatically performs block media recovery that is transparent to the application. Active Data Guard repairs physical corruption on a primary database using a good version of the block retrieved from the standby. Conversely, corrupt blocks detected on the standby database are automatically repaired using the good version from the primary database. > Application Continuity > Application Continuity is a new application-independent capability for Oracle Database 12c that recovers incomplete requests from an application perspective and masks many system, communication, and hardware failures, and storage outages from the end-user. It also ensures that end-user transactions are executed no more than once. Application Continuity is included with Active Data Guard. > Global Data Services > Oracle Global Data Services (GDS) is a new capability for Oracle Database 12c that extends familiar RAC-style connect-time and run-time load balancing, service failover and workload management capabilities to a collection of replicated databases, be it within a single datacenter or across multiple datacenters. GDS is included with the Active Data Guard. 1

12 Features 12.2 > A new database initialization parameter, DATA_GUARD_SYNC_LATENCY, enables you to define the maximum amount of time (in seconds) that the primary database may wait before disconnecting subsequent destinations after at least one synchronous standby has acknowledged receipt of the redo > The new ENABLED_PDBS_ON_STANDBY initialization parameter enables you to specify a subset of pluggable databases (PDBs) for replication on a physical standby of a multitenant container database (CDB). In releases prior to Oracle Database 12c Release 2 ( ), you had to specify either all PDBs or none > Oracle Database In-Memory column store (IM column store) is now supported on standby databases in Oracle Active Data Guard (ADG) environments. > You can now use the Oracle Diagnostic Pack with an Oracle Active Data Guard standby database that is open read-only 1

13 Implementation - Experiences and Examples out of customer projects 13

14 Aus der Praxis - Aufbau Topic Logging Static Listener Entries Password File Duplicate from active Cluster Registry Single -> RAC Check Beschreibung ALTER DATABASE FORCE LOGGING Connect to closed instances possible, DGMGR not needed since 12c, - Clusterware ASM! duplicate target database for standby from active database spfile parameter_value_convert 'chicago','boston' set db_unique_name='boston' Register Standby Database Resources with Clusterware... First time a lot todo (DBCA, rconfig, manual) DGMGRL -> alter system set dg_broker_start=true sid='*'; show configuration verbose; show DATABASE verbose <prod><stdb> Doc ID

15 Multitenenat > One database on standby database, many pluggable databases > ENABLED_PDBS_ON_STANDBY -> R12.2 > PDB$SEED > Template DB > Read only > Listener > All service are available, pluggable? > Move Pluggable between containers > Optimization Move data files 12c > Online tablespace move > Online table move 12c! 1

16 Multitenenat PDB$SEED + Start UP > Save State of Containers CREATE OR REPLACE TRIGGER open_pdbs AFTER STARTUP ON DATABASE BEGIN EXECUTE IMMEDIATE 'ALTER PLUGGABLE DATABASE ALL OPEN'; END open_pdbs; / > Working with PDB$SEED select con_name, state from dba_pdb_saved_states; ALTER PLUGGABLE DATABASE PDBP2 SAVE STATE; select con_name, state from dba_pdb_saved_states; CON_NAME STATE PDBP2 OPEN alter pluggable database pdb$seed close * ERROR at line 1: ORA-65017: seed pluggable database may not be dropped or altered SQL> alter session set "_oracle_script"=true SQL> alter pluggable database pdb$seed close; SQL> alter pluggable database pdb$seed open read write; Pluggable database altered. 1

17 Redologs, Size, Backup > show parameter "_deferred_log_dest_is_valid" -> false > If you want to delete archivelogs even if they are not applied at standby > Multi process for redo recovery > RECOVER STANDBY DATABASE PARALLEL > Block checking > Message Size > See best practice > Performance > OLTP Online Transaktion Processing > ODS Operational Data Store 1

18 Performance NoLogging Data guard? Primary RMAN Catalog Standby Inc Backup, Archive Backup Force Logging on Inc Restore Apply Archivelog zu SCN Snapshotdatabase Änderungen auf SPARSE Disks Nach jedem Appl neu erstellen RESTORE STANDBY CONTROLFILE FROM SERVICE; RECOVER DATABASE NOREDO; (CATALOG!) RECOVER DATABASE FROM SERVICE primary_db NOREDO SECTION SIZE 120M; -> Logs Force Logging 1

19 Theory Potential and Limitations 19

20 Availability Availability =!"#$%&!"#$%&'()*+$#%& > Downtime: > Planed > Unplanned > Availability 24x7 means in most cases > Planed and unplanned downtimes influence Availability 2

21 ACID > Defined by Jim Gray end of 1970s > A transaction is a transformation of state which has the properties of atomicity (all or nothing), durability (effects survive failures) and consistency (a correct transformation) > Basic for all relational database systems > Atomic: > closed > All or nothing > Consistent > Transactions leave consistent state > Normalization, foreign keys > Isolated > Transactions can run parallel > One transaction doesn t see changes of running other transactions > Durable > All changes after commit are persistent > Even soft or hardware failures don t change commited data 21

22 ACID

23 CAP Theorem > Eric Brewer aus dem Jahre 2000 > In 2000, Eric Brewer outlined the CAP conjecture, which was later granted theorem status when a mathematical proof was provided. > The CAP theorem says that in a distributed database system, you can have at most only two of Consistency, Availability, and Partition tolerance > Consistency > every user of the database has an identical view of the data at any given instant. > Availability > event of a failure, the database remains operational. > Partition Tolerance > the database can maintain operations in the event of the network s failing between two segments of the distributed system > Example: > AP Domain Name System > CA Relationales Datenbank Management System > CP Banking-System C A P 23

24 CAP Theorem No Go Consistency Strict Consistency Partition Toleranz Availibility Eventual Consistency 24

25 Sharding App Server > Application complexity > Es muss das richtige Shard angesprochen werden > Während des Wachstums werden die Teile unterschiedlich verteilt - Maintenance > Crippled SQL > it is not possible to issue a SQL statement that operates across shards > Complex joins > Loss of transactional integrity - CID transactions against multiple shards are not possible > Operational complexity. Memcache Shard A-F Shard G-P Shard Q-Z Data Tier 2

26 Webscale, Enterprise Partition Tolerant, Availability Example Cassandra 1. Write 3. write I A B 2. write Write Decisions: All Quorum (Each, Local) One One, Two, Three (Local) ANY H C G D Different rack Separate data center F E difference per insert 2

27 Oracle Sharding webscale Availability 27

28 Sharding at Oracle > Extreme Scalability. Sharding eliminates performance bottlenecks and makes it possible to linearly scale performance and capacity by adding shards. > Fault Containment. Sharding is a shared nothing architecture that eliminates single points of failure, such as shared disk, SAN, and clusterware, and provides strong fault isolation the failure or slow-down of one shard does not affect the performance or availability of other shards. > Global Data Distribution. Sharding makes it possible to store particular data close to its consumers and satisfy regulatory requirements when data must be located in a particular jurisdiction. > Rolling Upgrades. Applying configuration changes on one shard at a time does not affect other shards, and allows administrators to first test the changes on a small subset of data. > Simplicity of Cloud Deployment. Sharding is well suited to deployment in the cloud. Shards may be sized as required to accommodate whatever cloud infrastructure is available and still achieve required service levels. 2

29 Partitioning Table T1 Table T1 Table T1 Partition 2

30 Sharding Table T1 Table T1 Shard 3

31 CREATE SHARDED TABLE Customers ( CustNo NUMBER NOT NULL, Name VARCHAR2(50), Address VARCHAR2(250), CONSTRAINT RootPK PRIMARY KEY(CustNo) ) PARTITION BY CONSISTENT HASH (CustNo) PARTITIONS AUTO TABLESPACE SET ts1; Oracle Sharding Materialized View Replication(duplicated tables - read-only materialized view.) CREATE DUPLICATED TABLE Products ( StockNo NUMBER PRIMARY KEY, Description VARCHAR2(20), Price NUMBER(6,2)) ); 3

32 Sharding Objects > Sharded Table A sharded table is a table that is partitioned into smaller and more manageable pieces among multiple databases, called shards. > Sharded Table Family A sharded table family is a set of tables that are sharded in the same way. > Duplicated Tables In addition to sharded tables, an SDB can contain tables that are duplicated on all shards. > Non-Table Objects Created on All Shards In addition to duplicated tables, other schema objects, such as users, roles, views, indexes, synonyms, functions, procedures, and packages, and non-schema database objects, such as tablespaces, tablespace sets, directories, and contexts, can be created on all shards. > DDL Execution in a Sharded Database To create a schema in an SDB, you must issue DDL commands on the shard catalog database, which validates the DDLs and executes them locally first. > DDL Syntax Extensions for the Sharded Database Oracle Sharding introduces changes to the SQL DDL statements. DDL statements with this syntax can only be executed against a sharded database. 3

33 Archtektur Übersicht App Tier Application Server Weblogic Routing Tier Directors (GSM) Shard Catalog Global Data Services Catalog Service Data Tier DB 3

34 Shard Groups and Replikation > ADD SHARDGROUP -shardgroup shardgroup1 -region dc1 -deploy_as primary > ADD SHARDGROUP -shardgroup shardgroup2 -region dc1 -deploy_as active_standby > ADD SHARDGROUP -shardgroup shardgroup3 -region dc2 -deploy_as active_standby > CREATE SHARD -shardgroup shardgroup1 -destination host01 > CREATE SHARD -shardgroup shardgroup1 -destination host02 > CREATE SHARD -shardgroup shardgroup1 -destination host03 >... > CREATE SHARD -shardgroup shardgroup3 -destination host09 > Can be extended with shard spaces! 3

35 Global Data Services > Uses read-only global services exclusively > Uses read/write global services, but is programmed to resolve update conflicts if those services are simultaneously offered by more than one database > Can tolerate replicated data that is potentially stale due to replication lag 3

36 GDS Directors - SHD > GDSCTL> create shardcatalog > GDSCTL> add gsm -gsm sharddirector1 Shard Director Shard Catalog 3

37 Sharding Routing > Direct Routing to a Shard Oracle clients and connections pools are able to recognize sharding keys specified in the connection string for high performance data dependent routing. A shard routing cache in the connection layer is used to route database requests directly to the shard where the data resides. > Queries and DMLs via Proxy Routing in an SDB Sharding supports routing for queries that do not specify a sharding key. This allows the flexibility for any database application to execute SQL statements (including SELECT and DML) in a system where tables are sharded or duplicated without the need to specify the shards where the query should be executed. > When the application cannot pass the sharding key during connect > When the application needs to access data from sharded tables residing on multiple shards > SQL queries typically used in reporting such as - Aggregates on sales data 3

38 Oracle Sharding Pro & Cons What we liked Auto-configuration (Create and Manage Shards and Routing) Out of box support for Shard Catalog, Routing layer, Common Tables for Lookup Cross-shared query support (Not supported in custom sharding) Familiar SQL for sharding management Two level sharding (not supported in other technologies such as NoSQL) Consistent hashing to prevent large data movement as new shards are added Dynamically changing the number of shards Things to be careful of Sharding requires major rework in Data Model and Data Design Duplicated Table Make sure not to overuse; Not for write-heavy use cases Enable connection pool limits per Shard John Kanagaraj, OOW 3

39 Thank You! 39