Migrating Your Database to Oracle Cloud

Transcription

1 Migrating Your Database to Oracle Cloud Exploring the Options - PRO2520 Simon Pane & Ivica Arsov October 25, 2018

2 About Me Pythian Principal Consultant Working with Oracle since version 6 IOUG Board of Directors Oracle Certified Professional: Database 8, 8i, 9i, 10g, 11g, 12c Oracle Certified Expert Oracle Certified Partner Specialist Oracle ACE MOS Communities Simon_DBA The Pythian Group Inc.,

3 Ivica Arsov Senior Database Consultant Oracle Certified Master 12c & 11g Oracle ACE Blogger Twitter: IvicaArsov Blog: The Pythian Group Inc.,

4 Conference and/or Webcast Speakers For The Pythian Group Inc.,

5 PYTHIAN A global IT company that helps businesses leverage disruptive technologies to better compete. Our services and software solutions unleash the power of cloud, data and analytics to drive better business outcomes for our clients. Our 20 years in data, commitment to hiring the best talent, and our deep technical and business expertise allow us to meet our promise of using technology to deliver the best outcomes faster. The Pythian Group Inc.,

6 20 Years in Business 400+ Pythian Experts in 35 Countries 350+ Current Clients Globally The Pythian Group Inc.,

7 PYTHIAN TIMELINE Remote Database Management Services Oracle, Microsoft SQL Server, MySQL Cloud emerges, DevOps practice established Hadoop practice established First Cloud Managed Service Expanded Open Source databases Cassandra, MongoDB Cloud partnerships with Google, AWS, Microsoft Analytics practice established Competencies grow with Cloud partners Data, Machine Learning, Migrations, DevOps 11,000 database systems under Pythian management Analytics as a Service launches Completed one of the world s most complex Cloud Migrations The Pythian Group Inc.,

8 AGENDA Considerations: There s more to consider than the technology Solution Alternatives: Which technical solution is most applicable under what circumstances The Pythian Group Inc.,

9 Where to Start

10 NFRs Drive the Technology Not the Other Way Around Define the NFRs and Business Requirements NFR = Non-Functional Requirements Examples: downtime, encryption, security Need to understand parameters in which we re working How much resource time to invest vs software procurement or licensing costs Then move on to the technical fun part: The technical solutions that will satisfy the NFRs The Pythian Group Inc.,

11 Technical Architecture & Design Questions Should the cloud migration include: A database version upgrade? Architectural changes (i.e. to/from RAC or to/from ASM)? Edition change? Consolidation (i.e. at the server level or DB multitenancy)? Data purging (leave some old stuff behind)? Inclusion of additional options/features (cloud may includes features not licensed on-premises)???? Cloud IaaS vs DBaaS (ADW, ATP)? The Pythian Group Inc.,

12 Fundamental Approach Decisions Which is most aligned with the NFRs, business requirements and technical direction? Physical Conceptually simple Like for like (lift and shift) Near-zero downtime options Logical More complex implementation Upgrade and change Subset or consolidation Near-zero downtime options Advanced Custom solutions Mixed technology solutions Manual transfers The Pythian Group Inc.,

13 Underpinning Common Layers Networking and Security

14 Cloud Networking - VCN and VNICs Virtual Cloud Network - VCN Customizable virtual private network Virtual Network Interface Cards (VNICs) Enables a compute instance to be connected to a VCN You have complete control over your network environment IPv6 is not currently supported The Pythian Group Inc.,

15 Cloud Networking - VCN and VNICs Oracle cloud instances can connect to: The Internet (CIDR blocks can be specified/filtered) Your on-premises data center using an IPSec VPN tunnel connection or FastConnect Instances in peered VCNs Oracle Cloud Infrastructure services such as Object Storage The Pythian Group Inc.,

16 Cloud Networking - VCN peering Peering: connecting multiple virtual cloud networks (VCNs) VCN peering traffic is encrypted Local VCN peering (within regions) Remote VCN peering (across regions) Local VCN peering is supported in all Oracle Cloud Infrastructure regions Remote VCN peering is supported between: Ashburn and Phoenix (US) Frankfurt and London (EU) The Pythian Group Inc.,

17 Database Cloud Service - Network Encryption Native Oracle Net encryption is used by default: [oracle@iarsov admin]$ cat sqlnet.ora... SQLNET.ENCRYPTION_SERVER=REQUIRED SQLNET.CRYPTO_CHECKSUM_SERVER=REQUIRED SQLNET.ENCRYPTION_TYPES_SERVER=(AES256,AES192,AES128) SQLNET.CRYPTO_CHECKSUM_TYPES_SERVER=(SHA1) SQLNET.ENCRYPTION_CLIENT=REQUIRED SQLNET.CRYPTO_CHECKSUM_CLIENT=REQUIRED SQLNET.ENCRYPTION_TYPES_CLIENT=(AES256,AES192,AES128) SQLNET.CRYPTO_CHECKSUM_TYPES_CLIENT=(SHA1) The Pythian Group Inc.,

18 Database Cloud Service - Network Encryption Session encryption information is visible from V$SESSION_CONNECT_INFO SQL> select network_service_banner 2 from v$session_connect_info 3* where sid in (select distinct sid from v$mystat) NETWORK_SERVICE_BANNER Oracle Bequeath NT Protocol Adapter for Linux: Version Production Authentication service for Linux: Version Production Encryption service for Linux: Version Production AES256 Encryption service adapter for Linux: Version Production Crypto-checksumming service for Linux: Version Production SHA1 Crypto-checksumming service adapter for Linux: Version Production 6 rows selected. The Pythian Group Inc.,

19 JDBC Connections Older JDBC drivers are supported with some tweaks Disabling encryption may be required Error log java.sql.sqlexception: Io exception: Oracle Error ORA java/lang/throwable.<init>(ljava/lang/string;)v+4 (Throwable.java:85)... oracle/jdbc/driver/physicalconnection.<init>(ljava/lang/string;ljava/lang/string;ljava/lang/string ;Ljava/lang/String;Ljava/ oracle/jdbc/driver/t4cconnection.<init>(ljava/lang/string;ljava/lang/string;ljava/lang/string;ljav a/lang/string;ljava/util/ oracle/jdbc/driver/t4cdriverextension.getconnection(ljava/lang/string;ljava/lang/string;ljava/lang /String;Ljava/lang/String oracle/jdbc/driver/oracledriver.connect(ljava/lang/string;ljava/util/properties;)ljava/sql/connect ion;+831 (OracleDriver.ja FATAL ERROR in native method: native code passed a null object to a non-static method The Pythian Group Inc.,

20 Older JDBC Connections SQLNET.ORA Tweaks Disable (comment out): #SQLNET.ENCRYPTION_SERVER = required #SQLNET.CRYPTO_CHECKSUM_SERVER = required #SQLNET.CRYPTO_CHECKSUM_TYPES_SERVER = (SHA1) #SQLNET.ENCRYPTION_TYPES_SERVER = (AES256, AES192, AES128) Enable SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8 SQLNET.ALLOWED_LOGON_VERSION_SERVER= , 00000, "No common encryption or data integrity algorithm" // *Cause: The client and server have no algorithm in common for // either encryption or data integrity or both. // *Action: Choose sets of algorithms that overlap. In other words, // add one of the client's algorithm choices to the server's // list or vice versa. The Pythian Group Inc.,

21 Physical Migration Options

22 Maybe the Easiest Definitely the Most Complete Physical migrations (block by block) have numerous advantages: Guarantees completeness nothing left behind Might be technically simple Might have low downtime Caveats: Platform / Endian Database size and/or redo volume The Pythian Group Inc.,

23 Backup and Recovery Backup using the Oracle Cloud Backup Agent Download agent, configure RMAN to use as a virtual tape device Backup pieces go to the Oracle Cloud Storage Classic Service Restore into the cloud instance (also using the agent) Problems / Issues: Time to transfer if DB is very large No DB changes (upgrade after restore if required) Downtime for restoration Most suitable for non-prod Data sub-setting possible but only via PITR The Pythian Group Inc.,

24 Minimizing Cut-Over Downtime using Replication Add physical replication: Data Guard with a cloud based target Manual standby for SE is possible Problems / Issues: Data Guard requires EE No ability to subset data No consolidation options Easy & Reliable No version or platform changes The Pythian Group Inc.,

25 Dbvisit Standby for Standard Edition (or EE) Dbvisit Standby Additional management control for SE or other benefits Network encryption and network compression Easy to install and use Creates web console GUI A few ports need to be opened between source and target Takes care of everything including creation of the standby DB creates a backup, copies, restores Advanced architectures: supports multiple standbys and cascading The Pythian Group Inc.,

26 Dbvisit Standby Makes the Actual Go-Live Simple The Pythian Group Inc.,

27 Cross - Platform Migration (XTTS) Solution for cross-endian migrations - a MAA best practice recommendation: An enhancement over the normal TTS Replacement for Data Pump if that does not meet your maintenance window The process involves combination of Data Pump for meta-data and RMAN for file conversions Cons: Complexity: higher level of expertise is needed compared to Data Pump Larger time investment is needed to test the migration Validation of the database and application required Some downtime still needed The Pythian Group Inc.,

28 Full Transportable Export/Import Database migration made faster and easier: Use Data Pump to move metadata over a database link quickly The performance of the Transportable Tablespaces mechanism Applicable to migrating non-cdb into a PDB, PDB to PDB, PDB to a non-cdb: Export option is also backported to (with a patch set) Only user-defined object data and metadata is extracted from administrative tablespaces The data segments inside tablespace data files are not read Export operation only extracts metadata describing objects within each data file The Pythian Group Inc.,

29 Cloning a PDB with Encrypted Data in United Mode How to avoid ORA : SQL> create pluggable database REMOTE_CLONE from PDB2@L1 KEYSTORE IDENTIFIED BY "*******"; create pluggable database REMOTE_CLONE from PDB2@L1 KEYSTORE IDENTIFIED BY "*******" * ERROR at line 1: ORA-00283: recovery session canceled due to errors ORA-28374: typed master key not found in wallet The Pythian Group Inc.,

30 Cloning a PDB with Encrypted Data in United Mode As per Oracle documentation: When you clone a PDB, you must make the master encryption key of the source PDB available to cloned PDB. This allows a cloned PDB to operate on the encrypted data. To perform the clone, you do not need to export and import the keys because Oracle Database transports the keys for you even if the cloned PDB is in a remote CDB. However, you will need to provide the keystore password of the CDB where you are creating the clone. However, if the source PDB does not have its own master key we can encounter ORA How can a PDB not have its own master key? The Pythian Group Inc.,

31 Cloning a PDB with Encrypted Data in United Mode Possible scenarios: If the source PDB has its own master key, then everything would work fine. The master key will be copied over to cloned PDB. If the source PDB does not have its own master key (no records in V$ENCRYPTION_KEYS) and no recovery is needed (during clone operation) the clone operation will succeed. But, afterwards any data manipulation over encrypted data will fail. If the source PDB does not have its own master key (no records in V$ENCRYPTION_KEYS) and during clone operation a recovery needs to be done over encrypted data, the clone operation will fail. The Pythian Group Inc.,

32 Logical Migration Options

33 More Flexible but at the Price of Complexity Pros: Cross platform and version (upgrade and migrate) Custom changes (i.e. table and database structure) Selective data sub-setting Migration between endian formats Problems / Issues: Extended downtimes High complexity Good for small databases The Pythian Group Inc.,

34 Data Pump Conventional Export/Import Applicable regardless of endian format Can be used to migrate data on schema/table level Business logic migration (PL/SQL) Steps (check/test DB_LINK import on-premises to cloud): On-premises export Transfer Data Pump files Import Data Pump files Verify data The Pythian Group Inc.,

35 Continuous Sync Tools Oracle provided plus 3rd party tools Pros: Cross-platform / version Some ETL capability Issues: Costs Complexity Schema/table structure compatibility The Pythian Group Inc.,

36 Don t Forget: A Robust Balancing Technique is Essential! The Pythian Group Inc.,

37 Advanced Options Out-of-the-box Ideas (pun intended)

38 Custom Pulls Through DB_LINKs Unique Use Cases A build your own replacement to Data Pump Insert As Select (IAS) through a database link Customize hints for performance: On both the SELECT and INSERT parts of the statement Direct path inserts via the APPEND hint Allows for full DDL customization: PCTFREE, PCTUSED, PARALLEL, etc. Enable constraints with NOVALIDATE if acceptable Use hidden function to manage partitioned table locking The Pythian Group Inc.,

39 Oracle Cloud Infrastructure Data Transfer Appliance Each Data Transfer Appliance supports 150TB of data 30 days period to copy your data and ship back to Oracle Can be extended by contacting Oracle with a SR All data is encrypted by default Encryption passphrase is kept in a separate location It s free service Shipping costs are covered by Oracle The Pythian Group Inc.,

40 Last Resort: Import a Virtual Image if Possible In some (limited) circumstances, it may be possible to convert to or import existing VMDK images: The Pythian Group Inc.,

41 WRAP UP! The Pythian Group Inc.,

42 Summary Remember what wags what: Start with the NFRs and Business Requirements Then consider Architecture and Design Common characteristics/benefits: Network encryption between sites The Pythian Group Inc.,

43 Summary Choose the approach Weigh the pros and cons of each Downtime considerations Remember the edge case solutions Custom IAS solutions have proven effective but is a lot of work Physical replication is still possible with SE Lift and ship The Pythian Group Inc.,

44 COLLABORATE is the largest user-led, user-focused event for the Oracle community. Featuring 1,000+ education and networking opportunities. Discover solutions for wherever you are in your technology journey. COLLABORATE 19 April 7-11, 2019 Gonzalez Convention Center San Antonio, Texas Registration is now open!

45 Thanks and Q&A For Follow-ups: The Pythian Group Inc.,