Exadata Resource Management Konrad HÄFELI Senior Solution Manager Infrastructure Engineering teile und herrsche! BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 1
Unser Unternehmen Trivadis ist führend bei der IT-Beratung, der Systemintegration, dem Solution-Engineering und der Erbringung von IT-Services mit Fokussierung auf und Technologien im D-A-CH-Raum. Unsere Leistungen erbringen wir aus den strategischen Geschäftsfeldern: B E T R I E B Trivadis Services übernimmt den korrespondierenden Betrieb Ihrer IT Systeme. 2
Mit über 600 IT- und Fachexperten bei Ihnen vor Ort Hamburg 12 Trivadis Niederlassungen mit über 600 Mitarbeitenden 200 Service Level Agreements Düsseldorf Frankfurt Stuttgart Freiburg München Basel Brugg Bern Zürich Lausanne Wien Mehr als 4'000 Trainingsteilnehmer Forschungs- und Entwicklungsbudget: CHF 5.0 Mio. / EUR 4.0 Mio. Finanziell unabhängig und nachhaltig profitabel Erfahrung aus mehr als 1'900 Projekten pro Jahr bei über 800 Kunden 3 3 3 Stand 12/2013
AGENDA 1. Introduction 2. Database Resource Manager 3. Exadata IO Resource Manager 4. Tests 5. Tooling 6. Conclusion 4
Introduction 5
Introduction Exadata is known for its enormous performance This is optimal for consolidation Concurrency forces bottlenecks but the resource cake is not endless How can I guarantee everybody's piece? Resource Management 6
Database Resource Management The goal of resource management is to optimize resources allocation among different users / groups or programs Example: Online users get more power than batch users Example: batch gets many resources during the night and little during the day Sounds familiar!?! DBRM was introduced in Oracle 8i It s not just about CPU (more introduced over the coming versions) Also helps to manage runaway queries, parallel queries, idle sessions, undo and temp space Instance caging Limits, and takes automatic actions if a limit is exceeded 7
Exadata Resource Management Intra-Database via DBRM Inter-Database is introduced with IORM on Exadata 8
Database Resource Manager 9
Overview DBRM Oracle Resource Manager is organized with Resource Plans Directives Consumer Groups Ref: Oracle 11gR2 Database Administrator s guide 10
DBRM Resource Plans A resource plan regulates, how specific resources are shared among the consumer groups. It also regulates how unused resources are redistributed Resource Plans get activated on Oversubscription Only one resource plan is active at a specific time It can be enabled manually by parameter Or it can be enabled by a scheduler window 11
DBRM Directives A resource plan directive assigns values/rules to the resource plan. There are many parameters available, specifying values for such as cpu consumption up to 8 levels number of concurrently active sessions parallel degree limit amount of undo information in KB for a session Maximum idle time for a blocking session Amount of MB transferred (Read/Write) by a session before action specified in SWITCH_GROUP ist taken 12
DBRM Consumer Groups A consumer group is a collection of sessions that have similar requirements Consumer groups can also be created to represent certain kinds of workload There exist predefined consumer groups in the oracle server, for example SYS_GROUP and the default group for sessions OTHER_GROUPS A consumer group mapping rule defines, how sessions are mapped to consumer groups, for ex: service_name oracle_user client_program client_machine 13
DBRM Manipulation (1) The Resource Manager can be switched on- or off online SQL> alter system set resource_manager_plan = DEFAULT_PLAN sid= * ; SQL> select plan from dba_rsrc_plans order by 1; PLAN ------------------------------ APPQOS_PLAN DAYTIME_PLAN DEFAULT_MAINTENANCE_PLAN DEFAULT_PLAN DSS_PLAN ETL_CRITICAL_PLAN INTERNAL_PLAN INTERNAL_QUIESCE MIXED_WORKLOAD_PLAN ORA$AUTOTASK_HIGH_SUB_PLAN ORA$AUTOTASK_SUB_PLAN 14
DBRM Manipulation (2) BEGIN DBMS_RESOURCE_MANAGER.CREATE_PENDING_AREA(); DBMS_RESOURCE_MANAGER.CREATE_PLAN('DAYTIME_PLAN', 'Resource plan for managing all applications between 9 am and 5 pm'); DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP('CRITICAL_APPS', 'Critically important Apps'); DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP('ONLINE_APPS', 'Online Apps'); DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP('DEVELOPMENT', 'Developpers'); DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE('DAYTIME_PLAN', 'CRITICAL_APPS', 'Allocation for CRITICAL_APPS', MGMT_P1 => 60); DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE('DAYTIME_PLAN', 'ONLINE_APPS', 'Allocation for ONLINE_APPS', MGMT_P1 => 25); DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE('DAYTIME_PLAN', 'DEVELOPMENT', 'Allocation for DEVELOPMENT', MGMT_P1 => 10); DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE('DAYTIME_PLAN', 'OTHER_GROUPS', 'Allocation for default group', MGMT_P1 => 5); DBMS_RESOURCE_MANAGER.VALIDATE_PENDING_AREA(); DBMS_RESOURCE_MANAGER.SUBMIT_PENDING_AREA(); END; / 15 15
DBRM Components Ratio Based Priority Based Reports wants 40% -> Okay? - Yes, if not oversubscribed 16
DBRM Instance Caging Small but useful new feature of Oracle 11gR2 Limits (only) the amount of CPU s a Instance can use One Instance will not be able to allocate all the CPU resources of a server Instance Caging has no impact to the CPU s that has to be licensed Two different approaches: Over Provisioning Partitioning SQL> alter system set resource_manager_plan='some plan'; SQL> alter system set cpu_count=5; 17
Exadata IO Resource Manager 18
Exadata I/O Resource Management (1) Consumer Groups / Categories I/O Metadata: DB Name, Request Type, Consumer Group Resource Manager 19
Overview Exadata IORM (1) Adds the possibility to prioritize I/O s Can be used on cell only (dbplan, Inter Database IORM) or in combination with Resource Manager within the DB (catplan, Intra Database Plan) Hard limits (limit) or soft limits (allocation) possible Limits the I/O requests only if there is I/O pressure on the cell (soft limits) Without pressure, every database or consumer group is able to use 100% of the I/O throughput of a cell (soft limits) IORM Plans must be implemented and activated on every cell. Use dcli for that. 20
Overview Exadata IORM (2) IORM Plans can have multiple levels Sublevels get what is left over from the upper levels DB Plans and Category Plans always need a OTHER group Database Role specific settings possible IORM can be activated, deactivated or altered online oracle@exadb01:bin/ [rdbms1120]./dcli -g mycells cellcli -e 'alter iormplan active' exacel01.trivadis.com: IORMPLAN successfully altered exacel02.trivadis.com: IORMPLAN successfully altered exacel03.trivadis.com: IORMPLAN successfully altered 21
IORM I/O Scheduling Traditional Way Traditional I/O schedulers are black boxes No possibility to influence I/O requests are processed in FIFO order RDBMS I/O Request Traditional Storage Server Disk Queue H L H L L L L High Priority Workload Low Priority Workload 22
IORM I/O Scheduling Exadata Way I/O Requests are executed based on the priorisation Low priority Requests can be queued if needed Exadata RDBMS I/O Request H H I/O Resource Manager Disk Queue H H L H L H H L L L L L 23
Inter- / Intra Database Resource Management Intra DB RM controls the resources within one database Tools to use: - dbms_resource_manager - Consumer Groups and Plan Directives: CPU and I/O resources - (Consumer groups represent collections of users within a database) Inter DB RM controls the resources between multiple databases Tools to use: - (Instance Caging: CPU Resources) - IORM: I/O Resources (dbplan and/or catplan on cell) - dbms_resource_manager Categories: I/O resources - The category s must be present in the DB and on each cell - (categories represent collections of consumer groups across all databases) 24
Categories in Databases DBMS_RESOURCE_MANAGER.CREATE_CATEGORY( CATEGORY => 'prio1', COMMENT => 'Priority 1 consumer group'); DBMS_RESOURCE_MANAGER.CREATE_CATEGORY( CATEGORY => 'prio2', COMMENT => 'Priority 2 consumer group'); DBMS_RESOURCE_MANAGER.CREATE_CATEGORY( CATEGORY => 'prio3', COMMENT => 'Priority 3 consumer group'); DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP( consumer_group => 'CRITICAL_APPS', comment => 'Critically important Apps', category => 'PRIO1'); DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP( consumer_group => 'ONLINE_APPS', comment => 'Online Apps', category => 'PRIO2'); DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP( consumer_group => 'DEVELOPMENT', comment => 'Developpers', category => 'PRIO3'); 25
IORMPLAN on Cells dcli -g mycells cellcli -e \ "ALTER IORMPLAN catplan=\( \ \(name=administrative, level=1, allocation=80\), \ \(name=prio1, level=2, allocation=60\), \ \(name=prio2, level=2, allocation=30\), \ \(name=prio3, level=2, allocation=10\), \ \(name=other, level=3, allocation=100\) \ \), \ dbplan=\( \ \(name=rmtest1, level=1, allocation=60\), \ \(name=rmtest11, level=1, allocation=10\), \ \(name=other, level=2, allocation=100\) \ \)" 26
IORM Priorities Cell 1 Categories High 70% Low 30% Inter Database Database 1 60% 42% Database 2 40% 28% Database 1 60% 18% Database 2 40% 12% Intra Database CG 1 50% 21% CG2 50% 21% CG 3 50% 14% CG 4 50% 14% CG 5 75% 13.5% CG 6 25% 4.5% CG 7 80% 9.6% CG 8 20% 2.4% 27
New feature (1) 11.2.3.2.1 - I/O Resource Management Support for More Than 32 Databases supports share-based plans which allocate resources based on shares instead of percentages which can support up to 1024 databases, and up to 1024 directives for interdatabase plans 11.2.3.3 - Network Resource Management Network Resource Management automatically and transparently prioritizes critical database network messages fabric ensuring fast response times for latency critical operations. Prioritization is implemented in the database and the different InfiniBand adapters, This feature is enabled by default, and requires no configuration or management 28
New feature (2) 12.1.0.1 - IORM Support for Container Databases and Pluggable Databases By using CDB plans and PDB plans, I/O Resource Management (IORM) provides the ability to manage I/O resource utilization among different PDBs as well as manage the workloads within each PDB... DBMS_RESOURCE_MANAGER.CREATE_CDB_PLAN( plan => ''NEWCDB_PLAN ', comment => 'CDB resource plan for newcdb'); DBMS_RESOURCE_MANAGER.CREATE_CDB_PLAN_DIRECTIVE( plan => 'NEWCDB_PLAN', pluggable_database => 'SALESPDB', shares => 3, utilization_limit => 100);... 29
Kundenbeispiel: Datenbankkonsolidierung auf Exadata Exadata Database Machine X4-2 Half Rack 3 aktivierte DB Servern (anstatt 4) 3 Storage Server Zellen = 36 Disks (anstatt 7*12=84) Exadata Database Machine X4-2 Quarter Rack 2 DB Servern 3 Storage Server Zellen = 36 Disks Server 1 PROD Server 2 PROD Server 3 Server 1 PROD Server 2 PROD Prod DB s Server 1 DB1-Prod. DBn-Prod Failover DB s von Server 1 DB1-Prod DBn-Prod Test Entwicklung Integration Prod DB s Server 1 DB1-Prod. DBn-Prod Failover DB s von Server 1 DB1-Prod DBn-Prod Failover DB s von Server 2 DB(a)-Prod DB(z)-Prod Prod DB s Server 2 DB(a)-Prod. DB(z)-Prod Failover DB s von Server 2 DB(a)-Prod DB(z)-Prod Prod DB s Server 2 DB(a)-Prod. DB(z)-Prod Standby-Dataguard 1 Standby-Dataguard 2 30
Tests 31
DBRM and IORM Tests Test Environment Test runs 80 parallel sessions running a create table as select on a 3GB table exadb1 I1_RMTEST1 I1_RMTEST11 exadb2 I2_RMTEST1 I2_RMTEST11 Oracle ASM1 Oracle GI Oracle ASM2 Oracle GI Exadata DATA FRA DATA FRA 32
MB/s DBRM and IORM Tests Results Inter-Database IORM (1) 2 Databases, 1 Instance each, no IORM Rivalry is visible nicely 2 DBs, 1 Instance each without IORM 600 500 400 300 200 100 RMTEST1 Instance 1 RMTEST11 Instance 2 0 1 3 5 7 9 11 13 15 17 min 33 33
DBRM and IORM Tests Results Inter-Database IORM (2) 2 Databases, 1 Instance each, IORM Hard limit (limit=50 and limit=20, level 1 of max possible throughput) Hard limit: RMTEST11 does not get the full throughput even if RMTEST1 is idle 2 DBs, RMTEST1 Limit 50, RMTEST11 Limit 20, both Level 1 600 500 400 MB/s 300 200 RMTEST1, Instance 1 RMTEST11, Instance 2 100 0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 min 34 34
DBRM and IORM Tests Results Inter-Database IORM (3) 2 Databases, 1 Instance each, IORM Soft limit Soft limit: RMTEST1 has 3x more resources (alloc=30 and alloc=10) As soon as RMTEST1 is finish, RMTEST11 gets all the resources 2 DBs, no Limits RMTEST1 alloc 30 RMTEST11 alloc 10, both Level 1 MB/s 800 700 600 500 400 300 200 100 0 1 3 5 7 9 11 13 15 17 19 min RMTEST1, Instance 1 RMTEST11, Instance 2 35 35
DBRM and IORM Tests Results Intra-Database IORM (1) 1 Database, 2 Instances consumer group CRITICAL_APPS (category prio 1 ): 60% on DB consumer group DEVELOPMENT (category prio 3 ): 10% on DB Category prio 1 : alloc 30 Category prio 3 : alloc 5 1 DB, 2 Instances, Schema 1 60% (CRITICAL_APPS), Schema 2 10% (DEVELOPMENT) (like Test 9) with IORMPLAN, Schema 1 alloc 30 (Prio 1), Schema 2 alloc 5 (Prio 3) 700 600 500 400 300 RMTEST1, Instance 1 RMTEST1, Instance 2 200 100 0 1 3 5 7 9 11 13 15 17 19 21 36 36
DBRM and IORM Tests Results Intra-Database IORM (2) 2 Databases, 4 Instances consumer group CRITICAL_APPS (category prio 1 ): 60% on DB consumer group DEVELOPMENT (category prio 3 ): 10% on DB Category prio 1 Level 2: alloc 60 on cell Category prio 3 Level 2: alloc 10 on cell RMTEST1: alloc 60% on cell RMTEST11: alloc 10% on cell CATEGORY is evaluated first RMTEST1, Instance 1 gets more resources because of the priorisation MB/s 800 700 600 500 400 300 200 100 0 2 DBs, 4 Instances, Schema 1 60% (CRITICAL_APPS), Schema 2 10% (DEVELOPMENT) with IORM Level 2, Schema 1 alloc 60, Schema 2 alloc 10, Instance 1 alloc 60, Instance 2 alloc 10 1 4 7 10 13 16 19 22 25 28 31 34 37 40 min RMTEST1, Instance 1 RMTEST1, Instance 2 RMTEST11, Instance 1 RMTEST11, Instance 2 37 37
Tooling 38
Resource Management Challenges Kind of complicated Has to be changed according the systems needs New Database New Applications New User Flexible and easy to use Has to be reproducible Monitorable Resilient Write a tool with a configuration file 39 39
Resource Management Tool Start Script Read conf Generate SID code SID list done Apply code to SID-list Apply code to CELL-list Errorcheck, mail, exit 40 40
Resource Management Tool Configuration Sections Intra Database Category's Intra Database ConsumerGroups Intra Database IORM catplan Intra Database Resource Plan Intra Database Resource Plan Directives Intra Database Consumer Group Mapping Intra Database Consumer Group Grants Inter Database IORM dbplan # Primary databases dbplan name=rmtest1, level=1, allocation=60, role=primary dbplan name=rmtest11, level=1, allocation=10, role=primary dbplan name=ecazt, level=1, allocation=10, role=primary # Standby databases dbplan name=rmtest1, level=2, allocation=60, role=standby 41 41
Resource Management Tool Generate script (1) # *************************************************************** * # * M A I N * # *************************************************************** * # *-------------------------------------------------------------- * clear # --------------------------------- # Exadata IORM # --------------------------------- getcatplans getdbplans gencellcommand getrmplans getrmcategorys getrmconsgroups getrmplandirectives getrmcgmappings getrmcggrants 42 42
Resource Management Tool Generate script (2).. else genrmscript applyrmscript fi done fi checklog # *************************************************************** * # * E N D * # *************************************************************** * sendmail 43 43
Conclusion 44
Oracle Exadata Resource Management Conclusion (1) Resource Management in Consolidation Environments is a MUST DBRM and IORM work very well KISS! Keep it simple! First implement a good CATEGORY Plan CRITICAL_APPS, BATCH_LOADING, DEVELOPMENT, REPORTS etc. Second implement a Database IORM Plan Only limit the Consumer Groups within a database if needed, it s easier to use category s within a database only A generator Script helps to manage the needed PL/SQL Code 45 45
Oracle Exadata Resource Management Conclusion (2) How can you keep the smile on your Database? With Resource Management! 46
Weitere Informationen... www.trivadis.com/exa 47
Fragen und Antworten... Konrad Häfeli Senior Solution Manager Infrastructure Engineering konrad.haefeli@trivadis.com BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN