EXADATA CLOUD MACHINE

Size: px

Start display at page:

Download "EXADATA CLOUD MACHINE"

Allan Holt
5 years ago
Views:

1 EXADATA CLOUD MACHINE WHAT S IT REALLY LIKE? ANDY COLVIN ACCENTURE ENKITEC GROUP RMOUG 2018

2 ANDY COLVIN Professionally 10+ years at Enkitec/Accenture Oracle ACE Co-author of Expert Oracle Exadata Background in systems, network, DBA Unprofessionally Husband / father / travel agent Arsenal supporter Beer snob BBQ aficionado Twitter - acolvin@gmail.com Blog - blog.oracle-ninja.com

3 EXADATA CLOUD OFFERINGS

5 EXADATA CLOUD OFFERINGS TRADITIONAL EXADATA Exadata exactly the way you ve known it Traditional on premises Exadata Standard Exadata, the way it s been done for years You own the asset, and can manage it to your needs Hire partners for managed services Install additional software Copyright 2017 Andy Colvin. All rights reserved. 5

6 EXADATA CLOUD OFFERINGS EXADATA CLOUD SERVICE Exadata for the Cloudy McCloudface folks Exadata running in Oracle's public cloud Great if your applications are already in Oracle's cloud Administrative tasks are performed via Oracle's cloud console or API System deployment Database creation Configuration of backups Patching Equipment is owned by Oracle Exadata resides in an Oracle datacenter Supports either OCI or classic interfaces/apis Copyright 2017 Andy Colvin. All rights reserved. 6

7 EXADATA CLOUD OFFERINGS EXADATA CLOUD MACHINE Just like Oracle cloud in your datacenter Exadata running in your datacenter If your applications can't move to public cloud, this is a solution Assets are owned and maintained by Oracle Includes all benefits of Exadata Cloud service in your datacenter Subscription pricing Bursting Scalable Inexpensive cost of database management packs You *do* give up some control by choosing Customer Only Oracle has access log in to physical equipment (cells, compute nodes, switches) Copyright 2017 Andy Colvin. All rights reserved. 7

8 EXADATA CLOUD OFFERINGS COMPARING TRADITIONAL VS CLOUD Hosted Exadata Owned capital expense Customer supports entire stack Elastic configurations Bare metal or virtualized configuration Extreme Flash or High Capacity storage Monitoring / alerting performed by Oracle Platinum Services Exadata Cloud Subscription model Oracle manages infra components Cookie-cutter sizes Only supports virtualization single cluster Only high capacity storage is offered at this time Monitoring / alerting performed by Oracle Cloud Operations Copyright 2017 Andy Colvin. All rights reserved. 8

9 EXADATA CLOUD MACHINE ACCESS AND MANAGEMENT

10 EXADATA CLOUD MACHINE ORACLE CLOUD CONTROL PLANE One small thing. Exadata Cloud Machine requires the Oracle "Cloud Control Plane" The control plane is what makes an Exadata Cloud Machine "cloudy" This is provided via an Oracle Cloud Machine ("OCM") The OCM is the mechanism for providing billing/utilization metrics, identity management, API access, and virtual networking management One OCM can manage multiple Exadata Cloud Machines in a single metro area If you are running geographically distant sites, each site needs an OCM OCM gives access to all of the remaining Oracle Cloud offerings OCM can be used to provide IaaS, database cloud service (non-exadata), object storage, and more Copyright 2017 Andy Colvin. All rights reserved. 10

11 EXADATA CLOUD MACHINE ORACLE CLOUD CONTROL PLANE Connecting the control plane OCM connectivity can be a challenge The control plane is what makes an Exadata Cloud Machine "cloudy" This is provided via an Oracle Cloud Machine ("OCM") The OCM is the mechanism for providing billing/utilization metrics, identity management, API access, and virtual networking management One OCM can manage multiple Exadata Cloud Machines in a single metro area If you are running geographically distant sites, each site needs an OCM OCM gives access to all of the remaining Oracle Cloud offerings OCM can be used to provide IaaS, database cloud service (non-exadata), object storage, and more Copyright 2017 Andy Colvin. All rights reserved. 11

12 EXADATA CLOUD MACHINE ADVANCED SUPPORT GATEWAY Oracle connects via a support gateway The Advanced Support Gateway ("ASG") allows Oracle access to your network The ASG serves the same purpose as a platinum gateway in a traditional hosted environment All traffic in and out of the ASG is encrypted with SSL The ASG logically resides in a DMZ network range, with specific firewall ports open externally and internally This 1U server should be placed in a rack outside of the OCM and ECM frames You will need 1 ASG per location Copyright 2017 Andy Colvin. All rights reserved. 12

14 EXADATA CLOUD MACHINE MANAGEMENT MODEL Exadata Cloud Machine domu Compute Node 1 (VM) fs hr dw domu Compute Node 2 (VM) Managed by Customer dom0 Compute Node 1 dom0 Compute Node 2 Managed by Oracle Cloud Operations IB Switch 1 IB Switch 2 Exadata Storage Server 1 Exadata Storage Server 2 Exadata Storage Server n Copyright 2017 Andy Colvin. All rights reserved. 14

15 EXADATA CLOUD OFFERINGS ADMINISTRATIVE ACCESS Component Type Customer Access? Oracle Access? Database Yes No Compute Node Virtual Machine Yes No Compute Node Hypervisor No Yes Exadata Storage Servers No Yes InfiniBand Switches No Yes Management Switch No Yes Power Distribution Units No Yes Copyright 2017 Andy Colvin. All rights reserved. 15

16 EXADATA CLOUD MACHINE CONFIGURATION

17 EXADATA CLOUD OFFERINGS RACK CONFIGURATIONS Eighth Quarter Half Full Compute Nodes OCPU Max RAM 240GB 720GB 720GB 720GB Storage Servers Raw Disk 42.7TB 85.4TB 170.9TB 341.7TB Flash Capacity 19.2TB 38.4TB 76.8TB 153.6TB Copyright 2017 Andy Colvin. All rights reserved. 17

18 EXADATA CLOUD MACHINE NETWORK CONFIGURATION Exadata Cloud Machine separates networks by ownership domu compute nodes use networks routable to customer Customer management (administration) Database client access Database backup Storage InfiniBand partition Cluster InfiniBand partition All other components reside on a private management network Oracle manages these components, so they aren't accessible to you Compute node management / ILOM Storage server management / ILOM InfiniBand Switches Copyright 2017 Andy Colvin. All rights reserved. 18

19 EXADATA CLOUD MACHINE NETWORK CONFIGURATION Exadata Cloud Machine TOR Switches Client Switches domu Compute Node 1 (VM) domu Compute Node 2 (VM) OCM dom0 Compute Node 1 dom0 Compute Node 2 IB Switch 1 IB Switch 2 P D U 1 Exadata Storage Server 1 Exadata Storage Server 2 Exadata Storage Server 3 P D U 2 Networks Customer Admin Database Client Backup InfiniBand Cloud Admin Copyright 2017 Andy Colvin. All rights reserved. 19

20 EXADATA CLOUD MACHINE NETWORK CONNECTIVITY REQUIREMENTS Newer Exadata Cloud Machine models include top of rack switches High speed connectivity Connect with 10GbE or via MPO for 40gbps 10GbE required at a minimum Systems include top-of-rack ("TOR") switches Reduced cabling to each rack Run cables to the TOR switches TOR switches provide cabling to compute nodes Reduced cabling to each rack Run cables to the TOR switches Copyright 2017 Andy Colvin. All rights reserved. 20

4-way MPO to 10GbE connection Cables provided by

22 EXADATA CLOUD MACHINE DNS CONFIGURATION DNS is controlled/maintained by Oracle You are assigned a <cloudid.>oraclecloudatcustomer.com domain Your DNS servers must be configured to forward to DNS servers inside of the OCM You can create your own internal aliases, but the Oracle DNS is the source of truth Copyright 2017 Andy Colvin. All rights reserved. 22

23 EXADATA CLOUD MACHINE TAILORING TO FIT YOUR NEEDS As cloud factor increases, customizability decreases All database homes must be owned by "Oracle" user Problematic when sharing DR and nonproduction Groundwork is laid by installing GI as "grid" user, so may be supported in future Diskgroups must follow a predetermined pattern: 80% DATA, 20% RECO, 0% SPARSE 40% DATA, 60% RECO, 0% SPARSE 60% DATA, 20% RECO, 20% SPARSE only available in Oracle public cloud 35% DATA, 50% RECO, 15% SPARSE only available in Oracle public cloud There are also diskgroups for ACFS filesystems and the cluster files No custom directory structures for Oracle homes must follow Oracle's guidelines Copyright 2017 Andy Colvin. All rights reserved. 23

24 EXADATA CLOUD MACHINE IMPLEMENTATION TIMELINE The cloud is agile, but it rolls in slowly Plan including Oracle's responsible tasks included 33 steps By comparison, traditional Exadata plan includes 6 steps Most time is spent planning, installing, and configuring OCM Additional time is needed to add system to Oracle's management infrastructure Think of this as doing an entire Platinum Services implementation before using your new Exadata Copyright 2017 Andy Colvin. All rights reserved. 24

25 EXADATA CLOUD MACHINE IMPLEMENTATION STEPS High level implementation steps (*planned*) 1. Customer orders Exadata and Oracle Cloud Machines 2. ECM/OCM/ASG are shipped from the factory 3. Customer and Oracle complete PCMA form and OEDA a. PCMA includes information about cloud and management networks, DNS domain, as well as ASG configuration information b. Without the PCMA, nothing can be done 4. Oracle takes PCMA information, sends to engineers for deployment 5. Field engineers perform installation of ASG, OCM, and ECM a. Connect racks to power and network b. Install ASG in separate rack and prepare for connectivity 6. ACS remotely configures OCM and ECM for deployment, databases begin to get created 7. Oracle enables service subscription, billing begins Typical timeframe of 30 days between steps 5 and 7 Copyright 2017 Andy Colvin. All rights reserved. 25

EXADATA CLOUD MACHINE PUTTING IT ALL TOGETHER

27 EXADATA CLOUD MACHINE A REAL WORLD STORY

28 CASE STUDY CASE STUDY BACKGROUND Existing Exadata customer, making the cloud jump Large direct retail company Currently owns 2 Exadata racks X2-2 quarter rack development / DR X3-2 quarter rack production 3 key applications on Exadata E-Business Suite handling all ERP Web backend database for public and private sites Warehouse distribution management system Small IT shop 4 DBAs 5 Linux administrators Copyright 2017 Andy Colvin. All rights reserved. 28

29 CASE STUDY WHY NOW? Exadata systems were due for a refresh Deploying a new datacenter in 2017 End of support lifecycle nearing Extended support was nearing for the X2-2 Desire to move to newer equipment that requires less maintenance No more battery replacements Worry about flash card EOL More powerful hardware in the X6-2 34X flash capacity and 3X flash IOPS of X2-2 4X flash capacity and 1.9X flash IOPS of X3-2 Copyright 2017 Andy Colvin. All rights reserved. 29

30 CASE STUDY DECISION TIME To go cloud, or not to go cloud Decision was between buying new Exadata racks, or cloud at customer Public cloud was not an option due to security requirements and desire to host applications inhouse Decision was between buying new Exadata racks, or cloud at customer Public cloud was not an option due to security requirements and desire to host applications inhouse Copyright 2017 Andy Colvin. All rights reserved. 30

31 CASE STUDY DECISION FACTORS Hosted Exadata No difference from current setup More flexibility for configuration Speedy deployment Nothing new to learn pro Customer Great from an accounting perspective Access to all licensed options Elevated support from Oracle They can tell the other groups about how "cloud" they are Large capital expenditure Pay per licensed option Customer owns a depreciating asset con Very few customers running Oracle customer Early adopter tax More restrictive configuration requirements Lack of total control Longer deployment model Copyright 2017 Andy Colvin. All rights reserved. 31

32 WHAT DID THEY DO?!?!?

34 CASE STUDY HERE COMES THE CLOUD Purchase is made in late May 2017 Customer chooses two Exadata X6-2 quarter racks One in existing datacenter One in new datacenter to be built out in mid/late 2017 They see themselves having plenty of lead time for implementation They also need a control plane There is no desire to use it for cloud services, though All applications are either on VMware or bare metal Oracle sweetens the deal to include one OCM at each location Copyright 2017 Andy Colvin. All rights reserved. 34

35 CASE STUDY IMPLEMENTATION High level implementation steps (*actual*) 1. Customer orders Exadata and Oracle Cloud Machines (May) 2. Customer is sent multiple copies of PCMA (June, July) a. Network and security requirements for the OCM and ASG are discussed for the first time, customer's security team does not initially agree to the security model. b. Customer doesn't have necessary 10GbE ports in legacy datacenter for OCM and ECM c. During this time, power for the OCM is changed, no more small single-phase PDUs customer balks at power cost for OCM, may remove one OCM from the order. 3. Customer and Oracle work PCMA form and assuage security's fears (August) a. Nothing can be done until new datacenter is online mid-october 4. Exception for smaller PDUs granted, ECM/OCM/ASG are shipped from the factory (September/October) 5. Network team gets new datacenter online (November) 6. Installation begins the week of December 4, Service activated February 2018 Copyright 2017 Andy Colvin. All rights reserved. 35

36 CASE STUDY LESSONS LEARNED What have they learned? Documentation wasn't easy to find Look at for documentation - *not* Oracle Cloud Machine Too much new tech at once can be overwhelming Network and security teams were in the middle of deploying new datacenter when cloud services came in to play this created significant delays to the project Customer had trouble implementing platinum services before going with cloud at customer was more difficult Would they do it the same way now? Copyright 2017 Andy Colvin. All rights reserved. 36

37 WRAPPING UP

38 I M GETTING EXADATA CUSTOMER WHAT WILL I DO?!?! It s going to take a little while to get started Implementation timelines are noticeably longer The first implementation at a site requires building out an OCM this takes time Think of it as a full platinum deployment *before* you set up the Exadata Subsequent machines are deployed very quickly Similar to a traditional Exadata deployment provide network and power, do your homework, and it s simple! Copyright 2017 Andy Colvin. All rights reserved. 38

39 I M GETTING EXADATA CLOUD WHAT WILL I DO?!?! Don t worry about performance differences between traditional and cloud It s still the same Exadata rack Fewer cells, but still same performance features OVM on Exadata doesn t show a performance impact Testing shows that a single OVM cluster performs on par with You don t have to worry about noisy neighbors Each Exadata (public cloud or customer) is dedicated to a single cluster Storage servers are not shared Compute nodes aren t shared Copyright 2017 Andy Colvin. All rights reserved. 39

40 I M GETTING EXADATA CLOUD WHAT WILL I DO?!?! You ll still have plenty of work to do Oracle doesn t manage the database layer Oracle has ownership of physical compute nodes, storage servers, and InfiniBand There s plenty of room for you to manage what s inside the VM Patching is still a challenge New tools! Patching framework is (mostly) automated, but requires frequent updates to the toolkit (exadbcpatchmulti) Oracle is backlogged on patching cloud infrastructure patches aren t available instantaneously Copyright 2017 Andy Colvin. All rights reserved. 40

Database Level 100. Rohit Rahi November Copyright 2018, Oracle and/or its affiliates. All rights reserved.

Database Level 100. Rohit Rahi November Copyright 2018, Oracle and/or its affiliates. All rights reserved. Database Level 100 Rohit Rahi November 2018 1 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated