Experience of being a Cloud DBA

Size: px

Start display at page:

Download "Experience of being a Cloud DBA"

Frank Allen
5 years ago
Views:

1 Experience of being a Cloud DBA Satyendra Pasalapudi Associate Practice Director Apps Associates LLC April 26, 2016 Copyright Apps Associates LLC. 1

2 About Me Satyendra Kumar Pasalapudi Associate Practice Director IMS & Cloud Practice at Apps Associates Co-Founder & President of Copyright Apps Associates LLC. 2

of Services AWS Premier Consulting Partner Oracle and Managed Service Partner Competencies Salesforce Cloud Alliance Partner MuleSoft

3 About Apps Associates Global Reach for Global Customers Founded in 2002, 650+ employees US, Europe, Asia US (East, Central, West) HQ in Boston, Massachusetts Strategic Partnerships, Certifications, Credentials Oracle Platinum Partner Oracle Specialized Across Our Portfolio of Services AWS Premier Consulting Partner Oracle and Managed Service Partner Competencies Salesforce Cloud Alliance Partner MuleSoft Partner CMMI Level 3 Appraised Cloud Services Majority Investment in Pre-Sales, Demand Gen, Labs & Solutions Copyright Apps Associates LLC. 3

4 Agenda What is Cloud Virtualization the mother of Cloud Oracle on Cloud Storage Considerations Customer Case Study Copyright Apps Associates LLC. 4

5 Cloud Copyright Apps Associates LLC. 5

Generational Change for Enterprise (IT) Cloud supports mission critical workloads 87% of Enterprises use Cloud for Mission Critical Applications Cloud use in the enterprise continues to grow Half of

6 Generational Change for Enterprise (IT) Cloud supports mission critical workloads 87% of Enterprises use Cloud for Mission Critical Applications Cloud use in the enterprise continues to grow Half of the Enterprises say they will use cloud for at least 75% of their workloads by 2018 No one cloud fits all More than half (53 %) of enterprises use two(2) to four(4) cloud providers Source: Verizon 2016 State of the Market: Enterprise Cloud report Copyright Apps Associates LLC. 6

Cloud Probable to Inevitable Service Management Network Perimeter Risk Based Security Controls Self Service and Automation Financial Transparency GE undergoing most important transformation in 140

7 Cloud Probable to Inevitable Service Management Network Perimeter Risk Based Security Controls Self Service and Automation Financial Transparency GE undergoing most important transformation in 140 year history 9000 Applications to AWS & to 4000 Applications 300 ERPs (two years back) to more manageable 34 Data Centers to 4 Data Centers By US$15b of Software Revenue Changes People - Reduce Outsourcing Technology - Build Approach for things that matter 20% of Applications in Cloud as of today 70% of Applications by 2020 in Cloud Source: AWS 2015 Keynote Oct OOW Keynote with Mark Hurd Oct Copyright Apps Associates LLC. 7

8 Hybrid Cloud Framework SCOM HR SALES FIN PLANNING DW / BI PROCUREMENT Copyright Apps Associates LLC. 8

9 What is Cloud Copyright Apps Associates LLC. 9

10 It is a abstraction of computer hardware resources What is Virtualization? Virtualization technology emulates real or physical computing resources, such as desktop computers and servers, processors and memory, storage systems, networking, and individual applications. Virtualization players in market A hypervisor also known as a virtual machine manager (VMM) allows multiple guest operating systems to run concurrently on a single physical host computer. Types Hardware Virtualization Full Software Virtualization Para virtualization Hardware-assisted software virtualization Component or resource virtualization Copyright Apps Associates LLC. 10

11 Reasons for Virtualization? Server Consolidation Server Provisioning Backup/Restore Hosting/ Cloud Computing Copyright Apps Associates LLC. 11

12 Types of Hypervisors Type 1 Native (Bare metal) Type 2 Hosted Guest OS Guest OS Guest OS Guest OS Guest OS Guest OS Hypervisor Server Hypervisor Operating System Oracle VM VMware vsphere KVM Xen Hyper-V Server VirtualBox VMware Fusion VMware Player VMware Workstation Copyright Apps Associates LLC. 12

picture Only Virtual Machine Monitor (VMM) / Hypervisor talks to hardware

13 Virtualization in a nutshell Virtualization : Running multiple machines on a single hardware Real hardware invisible to OS OS only sees an abstracted out picture Only Virtual Machine Monitor (VMM) / Hypervisor talks to hardware Traditional Environment Virtualized Environment Copyright Apps Associates LLC. 13

14 Pros and Cons Full Virtualization Pros Disaster recovery, failover Virtual appliance deployment Legacy code on non-legacy hardware Cons LATENCY of core four resources RAM performance reduced 25% to 75% Disk I/O degraded from 5% to 20% Network performance decreased up to 10% CPU privileged instruction dings nearing 1% to 7% Copyright Apps Associates LLC. 14

15 Database I/O Patterns Cache data in memory from disk to achieve fast query response For OLTP/ DSS workloads, memory should be big enough to hold as much as possible Buffer Cache SGA 100% cache hit ratio is ideal for OLTP workloads Copyright Apps Associates LLC. 16

16 Database I/O Patterns CPU time is mainly consumed by I/O wait Cache miss Cache Hit Copyright Apps Associates LLC. 17

17 Database I/O Patterns 1. Data Size is increasing Huge amounts of data 2. Not possible to cache all data SGA 2. User Volume is increasing... Buffer Cache 3. Many I/O operations Copyright Apps Associates LLC. 18

18 Running Oracle On Cloud Cloud Evaluation Considerations Copyright Apps Associates LLC. 19

19 Key Considerations for running on Oracle Cloud Security Performance Compliance Oracle Database Partner Ecosystem Hybrid Cloud Connectivity Licensing & Supportability Copyright Apps Associates LLC. 20

20 Enterprise Class Oracle Database Architecture Copyright Apps Associates LLC. 21

21 Storage Scalability SSD / PIOPS Striving to match SAN like performance Compute Scalability Vertical scaling massive CPU cores Horizontal Scaling clusters Oracle RAC on Cloud Network Scalability Bandwidth constraints ( With in the Cloud ) Leased lines to Cloud ( Hybrid Cloud ) Architectural Considerations Copyright Apps Associates LLC. 22

22 AWS Storage Options Summary Storage Option General Database Applications Speed Durability Instance Store Swap Temporary files Reports cache, web server cache Very low latency Very low; volatile EBS PIOPS / SSD Data files, redo logs Low latency Highly Durable but always backup EBS Boot volume Binaries, archive logs Binaries Moderate latency Highly Durable but always backup Amazon S3 Backups Backups Backups Longer latency Very high durability Amazon Glacier Long-term backups Long-term backups Long-term backups Restore times of 3 5 hours Very high durability Copyright Apps Associates LLC. 23

23 Storage Best Practices for Oracle Oracle Automatic Storage Management feature is the best option to optimize storage performance on AWS SAME (Stripe And Mirror Everything) feature of Oracle ASM will help distributing entire data across multiple ebs volumes ASM along wit PIOPS EBS volumes can provide best in class storage performance on AWS Oracle Secure Backup for Cloud Module will enable backups directly from Oracle Database/ASM to S3 using RMAN RMAN channels can be optimized for optimal system throughput and Storage performance Copyright Apps Associates LLC. 24

24 Sample disk layout for Oracle Database using ASM Storage Options for Oracle PROD DEV TEST Amazon S3 Bucket +PROD_DATA +PROD_FRA +DEV_DATA Life Cycle Policies Striping Striping 4 EBS Volumes 4 EBS Volumes 4000 PIOPS 1000 PIOPS ASM instance with 16,000+ IOPS Striping 4 EBS Volumes 500 PIOPS Amazon Glacier Copyright Apps Associates LLC. 25

25 Typical Storage Config for Oracle Database Storage Type Comments Binaries EBS volume Standard volume; EBS snapshots enabled Data Files Redo Log Files Archive Log Files Backup Files PIOPS EBS / SSD volumes PIOPS EBS / SSD volumes EBS volumes EBS volumes / Amazon S3 Striped across multiple volumes using ASM or any other technology PIOPS of 1000 or above Use separate EBS volumes for each group Preferable to use ASM disk groups Standard volumes for normal database workloads PIOPS for highly transactional environments Standard EBS volumes for local backups Use OSB / other technology to push to Amazon S3 Copyright Apps Associates LLC. 26

26 Backup on Cloud Use Amazon S3 for all backups Snapshot backup mechanism non-data file volumes Use RMAN to backup the database to Amazon S3 using Oracle Secure Backup Cloud Module (OSB) Archive data to Amazon Glacier for long term storage Copyright Apps Associates LLC. 27

27 Huge Pages

28 Huge Pages

29 Huge Pages

3 Default 72 GB RAM HugePages pool 24 GB RAM Huge

4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K

30 3 Default 72 GB RAM HugePages pool 24 GB RAM Huge Pages 48 GB HugePages 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 4K 2 MB 4K 4K 4K 4K 2 MB 2 MB 2 MB 2 MB

31 Huge Pages Advantages of HugePages HugePages are not swappable, in other words they are always pinned, so no swapping overhead. HugePages can be allocated on-the-fly but must be reserved first by setting the kernel parameters HugePages are reserved at system startup AMM (Automatic Memory Management) and HugePages are not compatible, AMM should be disabled so that Database can utilize HugePages. 11gR2 logs HugePages information in alert.log.

32 Huge Pages on Cloud Under the covers, Amazon EC2 uses a hypervisor to (potentially) run many virtual machines on a given physical server. EC2 instances essentially come in two flavors of virtualization: Para virtualization (PVM) and Hardware virtualization (HVM). The vast majority of EC2 AMIs use PVM, but for a variety of reasons, only EC2 instances using HVM can allocate huge pages.

33 1 st Generation: Full virtualization (Binary rewriting) Software Based VMware and Microsoft Virtual Machine Virtual Machine Dynamic Translation Operating System Evolution of Virtualization Types 2 nd Generation: Para virtualization Cooperative virtualization Modified guest VMware, Xen VM Hypervisor VM 3 rd Generation: Siliconbased (Hardware-assisted) virtualization Unmodified guest VMware and Xen on virtualization-aware hardware platforms Virtual Machine Hypervisor Virtual Machine Hardware Hardware Hardware Time Virtualization Logic

34 Smart Flash Cache

35 How Smart Cache Works

36 Customer Case Study Copyright Apps Associates LLC. 39

37 Customer Case Study Oracle E-Business Suite R Oracle Database 11g ( ) Database size 1.5 TB OEL bit 24 cores with 98GB RAM x 2 Nodes RAC on ODA 2 - Middle tier application nodes 8 cores with 16Gb RAM RHEL bit App Tier DB 11g Primary Datacenter App Tier Production DB Tier Customer via Internet Customer WAN

38 Key Considerations AWS Sizing Compute & Storage Right sizing Storage IOPS Instance type selection RAC vs Non-RAC Oracle ASM Oracle Specific Techniques for Migration Oracle Data Guard for incremental data transfer Oracle Flash Cache with ephemeral SSD volumes Shared Storage Sharing batch job log & out files across applications Avoiding NFS as single point of failure RTO / RPO Considerations Build Data Guard based local DR for better RTO Elastic Load Balancers with EBS Copyright Apps Associates LLC. 41

Oracle DataGuard App Node 3 DB Tier S Private Subnet AZ-2 Customer via

39 Cloud Architecture App Tier DB 11g App Tier Production DB Tier Primary Datacenter App Node 1 DB Tier M Private Subnet AZ-1 App Node 2 US East Oracle DataGuard App Node 3 DB Tier S Private Subnet AZ-2 Customer via Internet App Node 4 Customer WAN Copyright Apps Associates LLC. 42

40 Key Considerations Current Production DB on RAC cluster with 2 nodes On AWS, DB tier will be migrated to single node (No RAC) To ensure RTO / RPO guidelines, cross AZ DB high availability needs to be built using Data guard Application tier high availability should also be considered in proposed design Overall solution to be optimized as per AWS high availability guidelines Copyright Apps Associates LLC. 43

41 On Premise Database Tier Configuration Source environment Oracle Database Appliance Internal storage for Database files on ASM ; SSD for REDO log files Backup storage from SAN Average CPU utilization < 25% ( out of 24 cores) Instance caging implemented to limit DB CPU utilization to 8 cores Database Configuration sga_target 18G pga - 10G Total memory allocation 56G across two nodes Copyright Apps Associates LLC. 44

42 Database Instances On Cloud Instance Family Memory Optimized Instance Type r3.4xlarge Processor Arch 64-bit Physical Processor Intel Xeon E v2 vcpu ( Cores ) 16 Memory 122GB SSD ( Ephemeral ) 320GB EBS Optimization Available (2000 Mbps) Network Performance High Virtualization HVM Only Oracle Flash Cache feature availability Yes Price Performance (on-demand) $1.400 std price Ref: Copyright Apps Associates LLC. 47

43 DB Server Config DB Server r3.4xlarge with EBS Optimization Storage OS Disk 50G - Root (depends on AMI) (Magnetic/SSD) Oracle Binaries 100G (Magnetic) Backup disk - 1 TB SSD EBS Volume Concurrent Log & Out stored on DB tier with NFS to App tiers - 200G SSD Automatic Storage Management Production Non-Prod DATA (2TB / instance) 1TB x 2 SSD Disks 750 GB x 3 SSD Disks Or 1TB x 2 SSD Disks REDO 50GB x 1 Disk with 1000 PIOPS 50GB x 1 Disk SSD FLASH (Archives) ( <200G/day) 250GB x 1 SSD 250GB x 1 SSD Copyright Apps Associates LLC. 48

44 Oracle EBS Architecture Summary on Cloud Role Instance Type AZ CPU Cores Memory EBS Database Tier r3.4xlarge Primary EBS Local DR Tier R3.4xlarge/m3.xlarge DR 16/ 4 122/15 Application Node 1 M3.xlarge Primary 4 15 Application Node 2 M3.xlarge Primary 4 15 Application Node 3 M3.xlarge DR 4 15 Application Node 4 (Passive) M3.xlarge DR 4 15 Load Balancer ELB Copyright Apps Associates LLC. 49

45 Cloud DBA Core DBA Storage = + + Network + 75% Admin 15% Admin 5% Security Admin 5 % Copyright Apps Associates LLC. 50

46 Copyright Apps Associates LLC. 51

47 @pasalapudi Thank You!

Advanced Architectures for Oracle Database on Amazon EC2

Advanced Architectures for Oracle Database on Amazon EC2 Abdul Sathar Sait Jinyoung Jung Amazon Web Services November 2014 Last update: April 2016 Contents Abstract 2 Introduction 3 Oracle Database Editions