Deep Dive: Snap Clone and Data Refresh Solutions using Oracle Enterprise Manager CON6982

Transcription

1 Deep Dive: Snap Clone and Data Refresh Solutions using Oracle Enterprise Manager CON6982 Martin Peña Senior Director, Product Management Subhadeep Sengupta Consulting Product Manager Janardhana Korapala, Database Admin Consultant, Dell Inc Presented with

2 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle.

3 Agenda Snap Clone Overview Features and Use Cases Customer Use Case: Dell - Snap Clone with EM13 Q & A

4 Hurdles to Rapid Development and Deployment Innovation is a key requirement for business growth Increasing the speed of application development Fast moving application/database requirements require systematic approach to dev/test System/application patch/upgrades can introduce risk to dev/test Custom application/development require deployment in less time Efficient and up-to-date copies of mission-critical databases for agile application development Multiple clones of production databases/systems often required to support dev/test activities Dev/test and cloning of application and databases must be non-disruptive to production database 62% manage more than 5 DB copies, 34% manage more than 20 IOUG Database Storage Survey, 2014

5 Challenges to Development Agility Today Lengthy and complex database provisioning QA Engineer Manager DBA Network Admin Storage Admin DBA QA Engineer Request DB Approval Request HW & Storage Setup OS & Network Allocate Storage Configure Cluster & DB Access DB

6 Challenges with Cloning Today Impeding your efforts to innovate Lack of automation Scripts are not a viable solution for an agile development process Traditional cloning techniques are slow Existing storage vendor solutions are DBA unfriendly Data explosion often outpaces storage capacity Inhibits ITs ability to provide clones for development and testing Archaic processes that require multiple users to share a single clone, or only supports fixed refresh cycles Different priorities between DBAs and Storage administrators

7 Enterprise Manager Snap Clone Space Efficient Over 90% storage savings Rapid Cloning ~ 2 minutes to provision a 1 TB database Storage Agnostic Supports NAS and SAN storage Data Refresh Refresh from production Integrated DB Lifecycle Mgmt. Masking, subsetting, lineage and association Complete On Premise & Cloud Solutions Production DB Create Test Master Test Master Masking & Sub-setting Provision Thin Clones Functional Copies User can create additional copies for functional testing without consuming space

8 Enterprise Manager Snap Clone Development Agility Storage savings, rapid provisioning, Self Service Powerful Storage Support Supports ALL storage vendors (NAS & SAN) Data Refresh Empower the user to make adhoc clones and restores Database Cloning across Clouds For Compute Services and Database Cloud Services

9 Snap Clone Solution Development Agility

10 Data Reduction via Thin Clones Traditional Clones Full copy of the original database 3TB for original + clones Thin Clones Only changed blocks in new clone space 1.015TB for original + clones Base Image 1TB Data Files Base Image 1TB Data Files Clone A 1TB Data Files Clone A 5GB Data Files Pointers to existing blocks New data Clone B 1TB Data Files Clone B 10GB Data Files Pointers to existing blocks New data

11 Oracle Database as a Service Snap Clone Leverages DBaaS Framework and DB Management Efficiency Consolidated, shared platform maintained with automation Accessibility Automated request channel via self-service Dedicated Database Pluggable Database Database Schema Database Elasticity Agile scale up, scale down, relocation of services Accountability Meter usage, quota, compliance and governance Empty OR Full Clone OR Snap clones Data Integration API integration with 3 rd party tools, change requests, dev workflows Physical, Virtual, or Engineered Systems (Hybrid Cloud) On Premise & On Oracle Cloud Infrastructure

12 Snap Clone Solution Powerful Storage Support

13 Data Cloning Technology Options Leverage your Existing Investments RMAN Restore RMAN Duplicate Data Pump Database Native Hardware Solutions Software Solutions Vendor Specific Vendor Agnostic Storage Agnostic ACFS

14 Snap Clone on Exadata Leveraging Sparse Clones Fast, space-efficient snapshot copies of database Available for Self-Service consumption. Integration with pluggable database enables creation of Snap Clone with a Single Click All Exadata features work on snapshots (smart scans, smart flash cache, resource management ) Initiated by using Database snapshot integration features SNAPSHOT COPY (for pluggable databases) CLONEDB (for non-container databases) 10 x Snap Clones created concurrently from 60TB database in 15 minutes.

15 Hierarchy of Snap Clones on Exadata T1 DEV/TEST EXADATA - ASM CLUSTER T2 T3 P T1-R/O T2-R/O DB6 DB1 DB2 DB3 ASM SPARSE DG DB4 DB5 ASM SPARSE DG T3-R/O DB7

16 Snap Clone with Oracle ZSF Storage Appliance Engineered for optimizing database storage Oracle ZS Storage Appliance Optimized Oracle Database efficiency with HCC Optimized RMAN backup and recovery Optimized data reuse in development and test DBAs manage snapshots and clones directly

17 Snap Clone using Solaris File System (ZFS) Uses a single stock Solaris image physical or virtual [and NOT the Sun ZS3 Appliance ] Supports any kind storage NAS or SAN For SAN, mount luns as raw disk and format with ZFS filesystem Does NOT require the snapshot/clone licenses from the storage vendor, these features are available for free Additional features include compression, deduplication, IO caching, etc HA has to be handled externally either via Solaris Clusters, or by using HA features of the underlying hypervisor

18 Snap Clone Solution Agile Data Refresh

19 Data Cloning begins from PRODUCTION DBS TEST MASTER SNAP SHOTS SNAP CLONES P Data Guard S Change Config. (SI/RAC) RMAN Backup Mask,Change Config. (SI/RAC) Test Master Snapshot RMAN incremental data backup RMAN backup of archive logs Data Archive

20 Deployment Scenarios Snap Clones using Standby Production Database Continuous or Discrete Replication Standby / Test Master Database Private backups (snapshots) for SSA user Replication Types: Continuous Discrete Technology Data Guard, Golden Gate RMAN, Snap Mirror, import/export Data Refresh Automatic and instantaneous Manual and at scheduled intervals Masking and Subsetting Not possible At source (in production), or in place at test master

21 Agile Data Refresh Time Travel Refresh, Rewind across Snapshots Scheduled or Manual Storage Snapshots or RMAN Backups of the test master database, called Profiles Test Master t0 t1 t2. tn Test master is regularly refreshed with current data from production Clones Data Refresh Clones can be created from any profile Each user gets a personal read-write database clone Private backups

22 Data Cloning & Refresh flow It s Not Cloudy any more Non Self Service (Admin) flows Scheduled Snapshot/RMAN backup creation Clone from a library snapshots / backups / Image copies Clone from a snapshot to a point-in-time or SCN 1-click Refresh of clones from source REST APIs & Emcli verbs

23 Snap Clone Solution Coming soon on Oracle Cloud

24 Data Cloning & Refresh for DevOps 49 clones/month 60X faster with 180TB savings over EMC Production Masking Test Master Snap Clones 60,000 refreshes per year supporting DevOps for Developers

25 Summary

26 Oracle Snap Clone Scalable Supports 1 to 1,000s of clones Protects your existing Storage Investments Choice between hardware and software solution Use of trusted technologies like data guard for test master refresh Integrated DB Management with Enterprise Manager Oracle s flagship management product for all your database needs In sync with DB releases (e.g. support for PDBs on day one) Secure and role based access control; used by Fortune 1,000 customers Protection from unnecessary point tools; reduce TCO

27 Customer Overview Key Pains & Challenges 70,000+ full DB refreshes/year cloning issues/day Majority refreshes scheduled; 10% on demand Customer Thumbnail One of the largest European financial services groups, with group net income of 23 billion ,000 employees, 76 countries, and 32 million clients worldwide Solution Enterprise Manager Snap Clone on NetApp Refresh takes couple of minutes Cloning from Standby databases; no impact on production Using REST APIs; zero manual intervention

28 Customer Overview Key Pains & Challenges Average 5 days to provision UAT Database(s) for Dev Team Sprawling Computing Resources Limited Space in Data center Customer Thumbnail Largest mobile phone company in Egypt in terms of active subscribers 36 million customer, 22k employees Solution Storage Reduction on Exadata, ZFS Storage Appliance DB provisioning time reduced from 4 Days to <1 hour for 1TB DB Integrated Data Masking Resource Pooling, Rapid Elasticity, Metering and Chargeback

29 Customer Overview Key Pains & Challenges Long provisioning cycle Quality concerns due to limited data set Not possible to create training environments on demand Customer Thumbnail International container transportation, logistics and terminal companies 320 offices in 70 countries Fleet of more than 300 ships Solution Delivered comprehensive service catalog Enable self services for application owners to request new or modify environments. Implement DBaaS Snap Clone. 90% Storage Savings demonstrated on both EMC and ZFS storage. Data Refresh with Data Guard /Standby DB.

30 Customer Overview Key Pains & Challenges Dozens of concurrent development projects Synchronization of application and DB deployments Cloning frequency Customer Thumbnail Founded ,180 employees from over 50 countries Recognized as one of the world's most important banks; one of Fortune Magazine's most admired companies. API for automation and integration Solution Full Clones using RM Backup Provisioning for Performance, Stress, UAT testing Snap Clones using ZFS SA and Oracle Database Appliances Exadata Sparse Disk Snapshots provide Exadata performance on thin provisioned clones

31 Snap Clone Solutions Using OEM13c Janardhana Korapala, Dell Global Marketing

32 Agenda Oracle Enterprise Manager at Dell Overview Oracle Enterprise Manager at Dell Enterprise Manager in action at Dell New Enterprise Manager Architecture Snap Clone at Dell Business Drivers How Snap Clone Works? Snap Clone Pilot Architecture Snap Clones Interface Snap Clones (Self-Service) Proof of Concept Summary & Next Steps Q & A Global Marketing

33 Oracle Enterprise Manager at Dell Global Marketing

34 Overview Oracle Enterprise Manager at Dell App/DB Servers Monitored in Production Enterprise Manager App/DB Servers Monitored in Non- Production Enterprise Manager 29K Database Targets 57K Application Targets (Middleware/EBS/Siebel/PS) 7.4K URL Monitoring 2.6K Agents 5 Downstream Application 1.4K UDM DB/Host 350 Reports 15 Jobs / Automation 3K Grouping 1.5K User Global Marketing

35 Enterprise Manager in action at Dell Monitored Targets Oracle Enterprise Manager Dell In-house Applications Deployed in 2007 Pilot Test 2011 Deployed in 2012 Pilot Test 2013 Deployed in 2015 Pilot Test 2013 Deployed in 2016 Oracle DB Monitoring 10g, 11g, 12c Oracle SOA Monitoring 10g, 11g Oracle EBS Monitoring 11g, 12c Oracle SIEBEL Monitoring 10g, 11g, 12c, 13c URL Monitoring N/A Agent Installation Groups Targets Properties UDM Incident Rule Notify Method Targets Discovery Admin Group Compliance Policies Baseline Metric Reports User Access RMAN Backup DMMR Database Metadata Repository DBCM Database Compliance Mgmt KPI Key Performance Indicator DMP Database Maintenance Planner Deployed in 2007 Deployed in 2010 Deployed in 2010 Deployed in 2013 Deployed in 2012 Pilot Test & Deployed in 2015 Convert to ServiceNow Dec/2016 Automation Service Improvement Since EMCLI Jobs Automation OMR (Database) Incident (Remedy) Ticketing Notification SMTP Server Incident (ServiceNow) Incident Json NT Authentication PAC Group Global Marketing

36 New Enterprise Manager Architecture From 12c Level 3 to 13c Level 4 Oracle Management Agent Agent Agent Agent 13c Agent Agent Agent Agent Agent Agent Production Application Server Cluster (U.S., Austin, PC1) Production Application Server Cluster (U.S., Austin, PC1) Dell 13g M630's Oracle Management Service (13c) DR Application Server Cluster (U.S., Austin, PS3) Dell 13g M630's Primary Server Additional OMS Additional OMS Shared Storage Additional OMS Primary Server RAM 128GB CPU 36 Core /u01 250GB Additional OMS RAM 128GB CPU 36 Core /u01 250GB Additional OMS RAM 128GB CPU 36 Core /u01 250GB Additional OMS RAM 128GB CPU 36 Core /u01 250GB Additional OMS RAM 128GB CPU 36 Core /u01 250GB 400GB Rsync 400GB Primary Server RAM 128GB CPU 36 Core /u01 250GB Additional OMS RAM 128GB CPU 36 Core /u01 250GB Additional OMS RAM 128GB CPU 36 Core /u01 250GB Additional OMS RAM 128GB CPU 36 Core /u01 250GB Additional OMS RAM 128GB CPU 36 Core /u01 250GB em12p.prd.amer.dell.com (U.S., Austin, PC1) Primary Cluster DB (U.S., Austin, PC1) Dell 13g M630's Oracle Management Repository (13c) DR Cluster DB (U.S., Austin, PS3) Dell 13g M630's Oracle Database 12c EM12P1 EM12P2 EM12P3 EM12P4 EM12P1.Prd RAM 128GB CPU 36 Core /u01 100GB EM12P2.Prd RAM 128GB CPU 36 Core /u01 100GB EM12P3.Prd RAM 128GB CPU 36 Core /u01 100GB EM12P4.Prd RAM 128GB CPU 36 Core /u01 100GB DataGuard EM12P1.DR RAM 128GB CPU 36 Core /u01 100GB EM12P2.DR RAM 128GB CPU 36 Core /u01 100GB EM12P3.DR RAM 128GB CPU 36 Core /u01 100GB EM12P4.DR RAM 128GB CPU 36 Core /u01 100GB Archive: 512GB, FRA: 512GB, Data: 3TB Support Disaster Recovery Zero downtime for OEM rolling patching Archive: 512GB, FRA: 512GB, Data: 3TB Global Marketing

37 Snap Clone at Dell Global Marketing

38 Business Drivers Data issues for testing Data refresh is manual Existing solution (Optim) unsupportable, expensive Too many database copies Data masking via custom scripts cumbersome Virtual Database cloning solution Testing possible with Real production data Global Marketing

39 How Snap Clone Works? Continuous or Discrete Replication Test master is regularly refreshed with current data from production Source Database Standby / Test Master Database Scheduled or Manual Storage Snapshots or RMAN Backups of the test master database, called Profiles T0 T1 TN Source A Source A Source A Source B Source B Source B Clones can be created from a library of profiles Source C User A User B Each user gets a personal read-write database clone Global Marketing

40 SnapClone Pilot Architecture Oracle Enterprise Manager 13c Snap Clone Appliance Test Master DB Pool Scheduled or Manual Storage Snapshots or RMAN Backups OEM Agent OEM Agent OEM Agent San Storage Pool 50 TB 50 TB Source Database Source Database Source Database 50 TB Global Marketing

41 Snap Clones Interface One Test Master 4 Single Instance Snap Clones running from storage snapshots of Test Master DB mounted via NFS Global Marketing

42 Snap Clones (Self-Service) One Test Master All Datafiles are thin clones of parent files from Test Master Five (5) Snap Clones and many more can be created as system resources allow (CPU & Memory) Snap Clone Analyzer: >95% Storage Savings using Snap Clone vs. Full Clones Global Marketing

43 Proof of Concept Summary Database Cloud Capabilities Private Cloud DBaaS Self Service Enablement: Service Portal, 3 rd Party Automation, Self Service Performance Monitoring, Service Instance Snap Shots/Rollback, Resource Collaboration User Governance: Role Based Service Catalog and Quota Control, Show-back Value Platform as a Service (PaaS) PaaS: Platform Virtualization, Automated Resource Allocation, Resource Governance, Resource Optimization, Cloud Resource Monitoring, Service Refresh/Control, Service Consumption Monitoring PaaS Enterprise Integration RESTful Web Service API: All PaaS Functionality Exposed via Web Service API for Dell Enterprise Automation, Custom Service Creation Global Marketing

44 Next Steps Enterprise Automation Push Button Application Deployment Push Button Application Deployment Full Function Application Deployment, Dell Internal SaaS MWaaS Continued Focus on Platform Virtualization Orchestration Integration Templates Application Blueprints SDDC Support PaaS IT Service Management Global Marketing

45 Enterprise Manager Page on O.com Database as a Service Page on OTN Additional Resources Join the Conversation Twitter.com/oracle_em Facebook.com/oracleenterprisemanager Blogs.Oracle.com/OEM Oracle.com/newsletters Cloud Administration Guide (Documentation) MOS Note : Enterprise Manager Cloud Control Recommended Plug-Ins and Patches for Database as a Service (DBaaS) Related Videos Youtube.com/OracleEnterpriseMgr

46 Copyright 2016, Oracle and/or its affiliates. All rights reserved. 46