Database Consolidation with Oracle Exadata

Transcription

1 Database Consolidation with Oracle Exadata Techniques and Best Practices Manish Shah Sr. Principal Product Manager, Exadata Oracle Corporation Ajit Singh Director, Data Technology Platforms New York Stock Exchange (NYSE: ICE) Copyright 2015, Oracle and/or its affiliates. All rights reserved.

2 Program Agenda Why Consolidate On Exadata How To Be Successful With Consolidation On Exadata Customer Success Story - NYSE Copyright 2015, Oracle and/or its affiliates. All rights reserved. 2

3 Program Agenda Why Consolidate On Exadata How To Be Successful With Consolidation On Exadata Customer Success Story - NYSE Copyright 2015, Oracle and/or its affiliates. All rights reserved. 3

4 Odds Of dying in a lifetime* 1 in 98 1 in People 250 People *2008 National Safety Council Report Copyright 2015, Oracle and/or its affiliates. All rights reserved. 4

5 Redundant Components Redundant Database Servers Redundant Storage Redundant Connectivity Continuous Application Availability through hardware Faults FANs PDUs Batteries Switch Disk Flash Motherboards DIMMs Redundant and hot swappable power supply Copyright 2015, Oracle and/or its affiliates. All rights reserved. 5

6 Overview of Exadata Specific HA Features Reduced HA Brownouts Fast node detection and failover Automatic detection of Exadata storage failures with low application impact Automatic detection of Exadata network failures with low application impact Data Protection Automatic Hard Disk scrub and repair Exadata H.A.R.D Quality Of Service I/O Latency capping for read and write operations Exadata Cell I/O Timeout Threshold Health Factor for predictive failed disk Identification of underperforming disks and automatic removal I/O Resource Management Network Resource Management Copyright 2015, Oracle and/or its affiliates. All rights reserved. 6

7 Performance Exadata Smart Flash Logging Active Bonding Network Exadata Smart Write Back Flash Cache Persistent After Cell Restarts Management Flash and Disk Life Cycle Management Alerts Cell Alert Summary LED Notification for Storage Server Disk Removal Drop Hard Disk for replacement Drop BBU for replacement Minimize or eliminate False disk failures Copyright 2015, Oracle and/or its affiliates. All rights reserved. 7

8 How many more databases can be consolidated on Exadata? x86 40 DB 160 DB X86 Exadata Avg. Response Time 177 ms 132 ms 4X Density With Faster Response Time Exadata How many more PDB can be consolidated than non- CDB? Non-CDB 168 non- CDB PDB 252 PDB Avg. Response Time 212 ms 14 ms 50% Higher Density Copyright 2015, Oracle and/or its affiliates. All rights reserved. 8

9 The Exadata Community Effect Exadata Takes Standardization to the Next Level Standard Technologies Standard Configurations Standard Integration Standard Tuning Standard Support The Scalability of Telecom, The Availability of Banking, The Security of Government Oracle Public Cloud Oracle Development & Support 1,000 s of Customers and Partners Server Vendor A Storage Vendor B Network Vendor C Database Vendor D OS Vendor E VM Vendor F The New Global Standard vs A Company Standard Copyright 2015, Oracle and/or its affiliates. All rights reserved. 9

10 Program Agenda Why Consolidate On Exadata How To Be Successful With Consolidation On Exadata Customer Success Story - NYSE Copyright 2015, Oracle and/or its affiliates. All rights reserved. 10

11 8 Key Success Factors Copyright 2015, Oracle and/or its affiliates. All rights reserved. 11

12 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 12

13 Oracle MAA Availability Tiers Availability Service Levels for Unplanned and Planned Outages PLATINUM GOLD SILVER BRONZE Zero Outage for Platinum Ready Applications Zero data loss Comprehensive HA and Disaster Protection Recovery in seconds with zero or near-zero data loss High Availability (HA) for Recoverable Local Outages Backups plus redo for Oracle data protection Basic Service Restart Backups plus redo for Oracle data protection Copyright 2015, Oracle and/or its affiliates. All rights reserved. 13

14 Platinum Tier Zero Application Outage for Platinum Ready Applications Site A Backups RAC Application Continuity RAC Site B Backups Outages masked from applications, inflight transactions preserved Application Continuity Zero data loss failover, LAN or WAN Active Data Guard / Far Sync Data Guard sync Bi-directional replication and zero downtime maintenance Oracle GoldenGate Online patching for applications Tape Edition-based Redefinition GoldenGate Tape Automated workload management Oracle Global Data Services Copyright 2015, Oracle and/or its affiliates. All rights reserved. 14

15 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 15

16 Determine Strategy to move to Multitenant How many CDBs should you have? What criteria should you use to determine # of CDBs? Prod v/s Non-prod? Business Functions? Business SLAs / Common maintenance window? All of the above? It Depends! No. I am not going to say that. Copyright 2015, Oracle and/or its affiliates. All rights reserved. 16

17 Goal ABC Company 100 Databases 25 Prod 75 Non Prod 5 DW 20 OLTP 10 DW 65 OLTP Copyright 2015, Oracle and/or its affiliates. All rights reserved. 17

18 Start With Database Categorization Apply Reference Architecture ABC Company 100 Databases 25 Prod 75 Non Prod 5 DW 20 OLTP 10 DW 65 OLTP Metal Tier Prod Non Prod DW OLTP DW OLTP Platinum 1 5 Gold 2 10 Silver Bronze 8 45 Total Copyright 2015, Oracle and/or its affiliates. All rights reserved. 18

19 Guidelines for Separating PDBs into CDBs Separate CDBs for OLTP and DW workloads Separate CDBs for different 12c versions. Limit the number of Oracle database homes to a maximum of 5. Separate CDBs for Prod and Non-Prod HA Tier. Consider separate CDB for PDBs that don t need flash performance so you can turn IORM Flash cache and Flash log off for that CDB when needed. Keep no more than two variants per patch set (e.g Apr PSU and July PSU) Copyright 2015, Oracle and/or its affiliates. All rights reserved. 19

20 Start With Database Categorization ABC Company 100 Databases 25 Prod 75 Non Prod 5 DW 20 OLTP 10 DW 65 OLTP Apply Reference Architecture Metal Tier Prod Non Prod DW OLTP DW OLTP Platinum 1 5 Gold 2 10 Silver Bronze 8 45 Total Apply CDB Guidelines Copyright 2015, Oracle and/or its affiliates. All rights reserved. 20

21 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 21

22 Engineered Systems Capacity Utilization and Planning (ES-CUP) Program Details Free program offered by Oracle for Exadata Customers New Customers Analyze existing workload recommend Exadata machine configuration Existing Customers Recommend solutions to address current challenges Provide roadmap for ongoing capacity planning exercise Engagement Process Contact your regional Oracle Sales Team Copyright 2015, Oracle and/or its affiliates. All rights reserved. 22

23 Engineered Systems Capacity Utilization and Planning ES-CUP Step 1 Step 2 Step 3 Step 4 Kick-Off Data Collection Analyze Readout ACTIVITIES Conf-call to review objectives Document engagement scope Planning and timeline Customer to describe near-term growth estimates Oracle send data collection scripts Customer execute data collection scripts and send logs to Oracle Customer execute Exachk and send logs to Oracle Oracle to analyze information Crosscheck with Best Practices Generate Exadata Capacity Utilization and Planning document Generate roadmap Remote or onsite delivery of Exadata Capacity Utilization and Planning Review objectives Knowledge transfer Documentation shared Copyright 2015, Oracle and/or its affiliates. All rights reserved.

24 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 24

25 Consolidation Workbench Enterprise Manager ABC Company 100 Databases 25 Prod 75 Non Prod 5 DW 20 OLTP 10 DW 65 OLTP Copyright 2015, Oracle and/or its affiliates. All rights reserved. 25

26 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 26

27 CDB1 CDB2 Exadata Resource Management P1 P2 P3 Instance 1 P4 P5 P6 Instance 1 CDB1 CDB2 P1 P2 P3 Instance 2 P4 P5 P6 Instance 2 Compute Node 1 Compute Node 2 Resources To Be Managed CPU Memory Storage Node 1 Storage Node 2 Storage Node 3 Flash Disk Exadata Quarter Rack X5-2 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 27

28 Exadata Resource Management CDB1 CDB2 P1 P2 P3 Instance 1 P4 P5 P6 Instance 1 HOW DO I ALLOCATE HOW DO I ALLOCATE HOW DO I ALLOCATE CPU FLASH DISK BETWEEN SPACE AND BANDWIDTH BETWEEN IO REQUESTS BETWEEN Compute Node CDB1 CDB2 Storage Node 1 P1 P2 P3 P4 P5 P6 Storage Node 2 Storage Node 3 OLTP BATCH OLTP BATCH OLTP BATCH OLTP BATCH OLTP BATCH OLTP BATCH Copyright 2015, Oracle and/or its affiliates. All rights reserved. 28

29 Most Important For Consolidation Purpose CDB1 P1 P2 P3 Instance 1 Between CDBs PDBs Within CDBs Workloads Within PDBs CDB2 P4 P5 P6 Instance 1 CPU Instance Caging Configured on database CDB Resource Plan Configured on database Database Resource Plan Configured on database Compute Node Storage Node 1 FLASH Inter-Database IORM Configured on Storage Cell CDB Resource Plan Configured on database Database Resource Plan Configured on database Storage Node 2 Storage Node 3 DISK Inter-Database IORM Configured on Storage Cell CDB Resource Plan Configured on database Database Resource Plan Configured on database Copyright 2015, Oracle and/or its affiliates. All rights reserved. 29

30 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 30

31 Best practices for Exadata planned maintenance Leverage Exachk for simple software planning Leverage zero downtime software maintenance Usage Out of Place and Standby First Updating Leverage Lights Out Patching with notification function Take advantage of Exadata Engineered defaults Copyright 2015, Oracle and/or its affiliates. All rights reserved. 31

32 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 32

33 OVM On Exadata Use Cases and Benefits OVM on Exadata was created for databases but it is a full fledge feature that can be used for any valid reason a customer wants. It is available at no additional cost. Caveats: Exadata is optimized to be a Database Machine. The primary goal of OVM on Exadata is to provide a higher level of Isolation between databases. Exadata is not a middleware consolidation platform. Consider PCA* or SuperCluster for that purpose. Recommendation by Oracle: Focus on consolidating database with perhaps some light consolidation of other applications. An example of light consolidation would be something like: "I want to consolidate 30 databases, but I also need a few cores to run some ETL, management, and other tools and don't want to deploy another server just for these". *Private Cloud Appliance Copyright 2015, Oracle and/or its affiliates. All rights reserved. 33

34 OVM On Exadata Use Cases Isolation Dedicated DB Servers VM VM Virtual Machines Efficiency Many DBs in one Server DATABASE IN-MEMORY FINANCE VM SALES VM SUPPLY CHAIN VM Database 12c Multitenant VMs have good Isolation but poor efficiency and high management Separate OS, memory, CPUs, and patching Isolation without need to trust DBA, System Admin Database consolidation in a single OS is highly efficient but less isolated DB Resource manager isolation adds no overhead; must trust admins to configure Best strategy is to combine VMs with database native consolidation Multiple trusted DBs/PDBs in a VM Few VMs per server to limit overhead Copyright 2015, Oracle and/or its affiliates. All rights reserved. 34

35 No Additional Cost X2-2, X3-2, X4-2, X5-2 DB 11.2 and 12c SALES FINANCE SUPPLY CHAIN OVM On Exadata Use Cases Use Cases: Hosting, Cloud, Cross department consolidation OVMs are trusted partitions use for license cost reduction/compliance (e.g. create a VM cluster with few cores to match what you are licensed for best for Oracle options e.g. partitioning) Government compliance for workload isolation PCI/DSS requirements can be met by using OVM, VLAN Tagging and IB Partitioning. Host a small app or a tool that needs isolation Copyright 2015, Oracle and/or its affiliates. All rights reserved. 35

36 OVM On Exadata Configuration Guidelines Exadata OVM allows licensing of Virtual CPUs (Trusted Partitions)* At least 40% of usable cores on the machine must be licensed for Oracle software Cannot use both OVM and CoD to license less than 40% of cores A server must be virtualized or non-virtualized not mixed OVM has no License cost; support is included with premier hardware support VM1A 12 cores (24 vcpu) RAC DB EE Virtualized Cluster VM2A 24 cores (48 vcpu) Adv Security RAC DB EE VM1AB 12 cores (24 vcpu) RAC DB EE Virtualized Cluster VM2B 24 cores (48 vcpu) Adv Security RAC DB EE 14 cores DB IM DB EE Non-Virtualized 60% CoD * Copyright 2015, Oracle and/or its affiliates. All rights reserved. 36

37 OVM On Exadata Configuration Guidelines Hardware X2-2 X3-2 X4-2 X5-2 Physical Memory /Node 72 GB 256 GB 256 GB 256 GB (default/max) 144 GB (max) 512 GB (max) 512 GB (max) 768 GB (max) Min Mem per VM Max Mem per VM Mem Defaults (GB) 8 GB min + DB or app memory 512 GB Small - 16; Medium - 32; Large 64 (Templates adjustable at install time) Physical CPU threads/node* Min vcpus per VM 2 Max vcpus per VM Physical Threads - 2 Default Template vcpus Small - 2; Medium - 4; Large 8 (Templates adjustable at install time) Total Available Disk Space for all DomU Minimum Used Disk Space per VM 700 GB 700 GB 1.6 TB 1.6 TB 100 GB Sparse allocated per VM for initial deployment. 90 GB required to run one Oracle database version (8 GB required per DB Version). 10 GB available for any other usage. *1 thread = 1 vcpu Copyright 2015, Oracle and/or its affiliates. All rights reserved. 37

38 OVM On Exadata Example Exadata Half Rack Split into Physical and Virtual Clusters Node 1 (Physical) Node 2 (Physical) Node 3 (Virtual) Node 4 (Virtual) OVM Cluster 1 Physical Cluster OVM Cluster 2 DBFS RECO DATA DBFS RECO DATA DBFS RECO DATA Copyright 2015, Oracle and/or its affiliates. All rights reserved. 38

39 OVM On Exadata Example Node 3 (Virtual) Node 4 (Virtual) Dom U1 OVM Cluster 1 Dom U1 OVM Cluster 2 Dom U2 Dom U2 Dom 0 Dom 0 DBFS RECO DATA DBFS RECO DATA Copyright 2015, Oracle and/or its affiliates. All rights reserved. 39

40 OVM On Exadata Software Architecture Comparison Database Server: Bare Metal / Physical versus OVM Bare Metal / Physical Database Server Oracle GI/DB homes Exadata (Linux, fw) OVM Database Server dom0 Exadata (Linux, Xen, fw) domu-1 domu-3 Oracle GI/DB domu-2 homes Oracle Exadata GI/DB (Linux) homes Oracle Exadata GI/DB (Linux) homes Exadata (Linux) No change to Storage Grid, Networking between OVM and Bare Metal Copyright 2015, Oracle and/or its affiliates. All rights reserved.

41 OVM Patching on Exadata Must maintain patching on both Dom0 and DomU Grid Infrastructure and RDBMS patching is handled in the same manner as non-virtualized system Exadata Software updates on DomU may include updates to both, Exadata Platform Software and OS or it may include update to any one component Dom0 only needs Exadata Software Updates OVM On Exadata Maintenance and Patching Copyright 2015, Oracle and/or its affiliates. All rights reserved. 41

42 New York Stock Exchange Oracle Exadata Our Journey Ajit Singh, Director, Data Technology Platforms

43 New York Stock Exchange (NYSE) and Intercontinental Exchange (ICE) NYSE - world s largest stock exchange by market capitalization ($28 trillion) NYSE is owned by Intercontinental Exchange (ICE) - Architects of the world s markets 11 exchanges, 7 clearing houses 12,000 + listed contracts and securities Diverse markets span futures & options on interest rates, commodities, indexes and FX, as well as equities and equity options. We transact on average in <200 mics.

44 DATA ENVIRONMENT Data volume: ~50-60 billion transactions per day (at peak) Data size: ~ 15 TB of data per day (at peak) All data for equities, ETFs, options markets and the feed to the street Data for trading system latency, other markets Data services responsible for managing all the data and data platforms including Exadata November 17,

45 Oracle Database environment on Exadata Oracle - primary transactional database platform supporting all NYSE businesses OLTP and Data Mart environments databases, over 500 instances Total database size TB Daily transaction volume - ~1000 million transactions Large transactional databases: ~10-20 TB databases generating about ~ 300 GB of log per day Large Data Mart: >100 TB in size (after HCC compression for historical data) November 17,

46 CONSOLIDATION TO EXADATA 2004: 2006: 2014: 17 Exadata 12 for NYSE

47

48 NYSE Exadata footprint

49 EXADATA FOR CONSOLIDATION Some examples: Dev: ~60 databases consolidated on a full rack of X3-2 with an average size of 1.6 TB QA: ~70 QA consolidated on to a full rack of X3-2 with an average size of 2 TB Prod: ~ 50 databases consolidated on a full rack of X3-2 with an average size of 9.5 TB Mix of OLTP and data marts November 17,

50 Leveraging exadata for performance Performance gains of 2x 25x Trade Reports database 180,000 messages/sec - ~ 50% increase in processing speed For the same application, completion time decreased by 66% Data Mart - daily transactions of 2 TB applied to standby in real-time Data Loads processing times decreased by 50% Data Mart hybrid columnar compression - 4x compression gain vs ACO x compression achieved on raw data using HCC Full Backup time of a ~150 TB data mart reduced by 85% (over Infiniband switches) Utilize SmartScan capabilities November 17,

51 Migration Consolidation planning Database load compatibility analysis setup appropriate resources parameters, limits Migration to Exadata using Data Guard, RMAN Closing and RMAN Duplicate Plan hardware - Dedicated network Test it out connectivity, test with application and RAT testing for performance validation November 17,

52 CONSIDERATIONS Coordinate downtime Risk Mitigation: Manage capacity Oracle Homes 2 Load Isolation Instance caging, IORM deployed to manage resources consolidated Security isolation Rolling upgrades of databases Rolling Exadata bundle patches cell servers, components, compute nodes and clusterware Observe utilization Re-evaluate database parameters November 17,

53 BENEFITS OF CONSOLIDATION Reduced time to market Platform as a service Database as a service Reduced cost Hardware, licensing operations

54 Copyright 2015, Oracle and/or its affiliates. All rights reserved. 54