Backup to Oracle Cloud. Charles Kim, Viscosity North America, Oracle ACE Director NZAOUG July 2017

Transcription

1 To Oracle Cloud and Beyond Backup to Oracle Cloud Charles Kim, Viscosity North America, Oracle ACE Director NZAOUG July 2017

2 Cloud Experience Agenda 09:30 10:00 Check-in and Registration and cloud account setup for each student 10:00 10:30 Introduction to Oracle Cloud and SSH Security 10:30 11:00 Hands on Lab Workshop 11:00 11:30 Oracle Storage Cloud and Database Backup Cloud Lecture :30 12:00 Hands on Workshop 12:00 13:00 Provided lunch and work through lunch as needed :00 14:30 Upgrade to Oracle Database 12c Release 2: How and when to upgrade or migrate databases or workloads to the Oracle Cloud, How to migrate databases to Oracle Cloud with zero-downtime, Achieve hybrid cloud with the latest Oracle hardware on-premise platform :30 15:00 Database Cloud Lecture 15:00 16:00 Hands on Lab Workshop 16:00 16:30 Oracle Storage Cloud Storage Appliance Lecture, ZFS Storage Integration with Oracle Cloud :30 17:00 Final Questions and Answers 2

3 Agenda Review of Oracle Cloud Storage Offereings Why Oracle Database Backup as a Service Backup & Restore On-Premise to Oracle Public Cloud (OPC) RMAN Configuration RMAN Backups / Restore from OPC Storage Cloud Object Storage Archive Storage Storage Cloud Software Appliance Architecture Setup/Configuration and lessons learned

4 Oracle Cloud Leadership IaaS Data Centers World Wide 33 Billion Daily Transactions 70 Million Users with Triple Digit Growth 19 of 20 Top Cloud Providers run Oracle 15 Years Managing Enterprise Clouds

5 Oracle IaaS Storage: Highest Performance and Durability Local NVMe Block Object Archive File & Database Backup Storage Cloud Software Appliance Hardware Physical Servers & Storage Core Data Center & Access Network Data Center Regions & Availability Domains POSIX NFS SMB, NAS WebDav

6 Oracle Cloud Database Backup Service Instant Off-Site Protection with RMAN Direct Database Backup to the Cloud Backup to Disk & Cloud Disaster Recovery Copy in the Cloud Backup to Tape & Cloud Database as a Service Backup in the Cloud

7 Database Backup Service On Premise to Cloud, Cloud to Cloud, Both Platform as a Service Instant Offsite Storage Cost-Effective On-demand Scalability End-to-End Security On-Premises Databases 10.2 and above Encryption Compression Oracle Database Backup Cloud Service 3-Way Protection 24x7 Data Availability Oracle DBaaS / Exadata Cloud Service Database RMAN Used

8 Introduction -Oracle Database Backup Cloud Service ODBS complements your existing backup strategy Use familiar RMAN interface to perform backup & restore operations Provides an off-site storage location No up-front capital costs or hardware expenses. Storage management & data transfer complexities are handled by the service Pay-as-you-go, elastic self-provisioning mode - Scale capacity on demand Create secure, encrypted backups in an off-premises location Oracle ONLY accepts secured encrypted backups 17

9 Backup/Recovery General Guidelines Following are general guidelines customers typically follow to address RTO and RPO requirements: Local - Fast Recovery Area (FRA): Typically stores backups for up to 7 days External Storage (NAS): up to 30 days Tape media (if available): 1 to 6 months Tape vaulting (offsite storage): months to years 18

10 RMAN Step By Step Configuration 19

11 Database Cloud Backup Service Easy as (4 and 5) 1. Subscribing to the Backup service 2. Download the Oracle Database Cloud Backup Module 3. Install the Oracle Database Cloud Backup Module 4. Configure RMAN Backup Channel 5. Backup the database Oracle Cloud Downloads: 20

12 Recovery Manager (RMAN) as tool of choice Oracle database integrated backup & recovery tool No new learning curve for the DBAs ASO and Compression bundled w/ Oracle Database Backup Cloud with no additional cost Perform SET, CONFIGURE, BACKUP, RESTORE & RECOVER operations at various levels Database, data file, tablespace, table, block Options: Backs up to disk then to the cloud Backup directly to the cloud Requires Media Management Layer (MML) knowledge to backup to cloud Simple commands RMAN> BACKUP DATABASE; RMAN> RESTORE DATABASE;

13 Backup & Recovery to OPC What: Create secure, encrypted backup off site Oracle only accepts encrypted backups Benefits: Create secure, encrypted backups in an offpremises location Store backups using reliable 3-way mirroring Familiar using simple standard RMAN commands Scale capacity on demand Some Best Practices: Weekly Full Daily Incremental Archive to Oracle Storage Cloud Use multiple RMAN channels for higher parallelism Use multi-section backups Example: BACKUP DEVICE TYPE SBT DATABASE SECTION SIZE 1g; Oracle recommends using MEDIUM compression for cloud backups Use the RMAN Fast Incremental Backup feature (based on Block Change Tracking)

14 Additional Best Practices Increase PARALLELISM (until you reach acceptable network throughput or hit max throughput capable with the existing network) RMAN> CONFIGURE DEVICE TYPE 'SBT_TAPE' PARALLELISM 4 BACKUP TYPE TO BACKUPSET; To determine network throughput for a specific time period, use RMAN network analyzer, see MOS note To diagnose Oracle Cloud Backup Performance, see MOS note » Consider scheduling additional archived log backups to reduce RPO» Download and run Installer at least once every two months to pick up latest RMAN SBT module» Configure CONTROLFILE AUTOBACKUP ON 23

15 Database Backup Cloud Pricing As of April Cloud Product Database Backup Cloud Pricing * ASO Encrypted Backups * RMAN Compression Non-Metered Pricing $33.00 / TB / Month Metered Pricing $ GB per Month (1TB = $26.40/Month) Up to 1TB $ GB per Month ($24.20 / Month) >5000 TB < 1GB Free Data Transfer Rates 1 GB to 9.99 TB.12 per GB per month Next 40TB Next 100TB Next 350 TB 1TB = $120 per month.09 per GB / month.07 per GB / month.05 per GB / month 24

16 Oracle Database Backup Service Support Matrix Oracle Database Operations Systems (64bit only) RMAN Compression RMAN Encryption 10gR2 ( ), 11g, 12c(EE, SE, SE1) Linux, Solaris x86 64, SPARC, Windows, AIX, HP UX, zlinux HIGH, MEDIUM, BASIC, LOW Password, TDE, Dual mode Oracle Database Backup Service - FAQ (Doc ID ) * Standard Editions require a patch for the bug

17 RMAN Operations Supported on Oracle Database Backup Cloud Service 26

18 Configure Database Cloud Backup Module Download the OPC Install Jar File $ java -jar opc_install.jar - servicename Storage - identitydomain ViscTech - opcid charles.kim@viscosityna.com - opcpass 'mypass*%' -walletdir $ORACLE_HOME/dbs/opc_wallet - libdir $ORACLE_HOME/lib - libplatform linux64 orcl > cat /u01/app/oracle/product/ /dbhome_1/dbs/opcorcl.ora $ ls -ltr $ORACLE_HOME/dbs/opc_wallet/* $ORACLE_HOME/lib/libopc.so Unzip the installer. It contains opc_install.jar and a README file Oracle Database Cloud Backup Module Install Tool, build Oracle Database Cloud Backup Module credentials are valid. Oracle Database Cloud Backup Module wallet created in directory /u01/app/oracle/product/12.1.0/dbhome_1/dbs/opc_wallet. Oracle Database Cloud Backup Module initialization file /u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcorcl.ora created. Downloading Oracle Database Cloud Backup Module Software Library from file opc_linux64.zip. Downloaded bytes in 16 seconds. Transfer rate was bytes/second. Download complete. OPC_HOST= OPC_WALLET='LOCATION=file:/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opc_wallet CREDENTIAL_ALIAS=storage_opc' -rw-rw-r-- 1 oracle oracle 9068 Oct 18 06:17 /u01/app/oracle/product/12.1.0/dbhome_1/dbs/opc_wallet/cwallet.sso -rw-rw-r-- 1 oracle oracle Oct 18 06:17 /u01/app/oracle/product/12.1.0/dbhome_1/lib/libopc.so 27

19 Configure RMAN High Level Steps 1. Configure RMAN Channels run { configure retention policy to recovery window of 30 days; configure channel device type 'sbt_tape' MAXPIECESIZE 2G FORMAT 'VCE_%d_%U' PARMS 'SBT_LIBRARY=libopc.so, ENV=(OPC_PFILE=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcVCE.ora)'; configure encryption for database on; configure device type 'sbt_tape' parallelism 2 backup type to backupset; configure backup optimization on; configure compression algorithm medium' as of release 'default' optimize for load true; configure default device type to sbt_tape; }

20 RMAN Backup to OPC $ cat rman_backup.ksh rman target / <<EOF set encryption on identified by oracle only; run { backup as compressed backupset incremental level 0 tag 'onprem DB' filesperset 1 (database); backup as compressed backupset skip inaccessible tag 'onprem ARC' (archivelog all not backed up 2 times); backup tag 'onprem CTL' (current controlfile); } EOF 29

21 Restore Controlfile set decryption identified by oracle; Run { allocate channel t1 type 'sbt_tape' PARMS 'SBT_LIBRARY=libopc.so, ENV=(OPC_PFILE=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcorcl.ora)'; set dbid= ; restore controlfile from autobackup; release channel t1; } 30

22 Restore Database set decryption identified by oracle; run { allocate channel t1 type 'sbt_tape' PARMS 'SBT_LIBRARY=libopc.so, ENV=(OPC_PFILE=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcorcl.ora)'; allocate channel t2 type 'sbt_tape' PARMS 'SBT_LIBRARY=libopc.so, ENV=(OPC_PFILE=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcorcl.ora)'; allocate channel t3 type 'sbt_tape' PARMS 'SBT_LIBRARY=libopc.so, ENV=(OPC_PFILE=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcorcl.ora)'; restore database; recover database; release channel t1; release channel t2; release channel t3; } 31

23 RMAN> list backup summary; using target database control file instead of recovery catalog List of Backups =============== Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag B 0 A SBT_TAPE 05-SEP YES ONPREM DB TO CLOUD 2 B 0 A SBT_TAPE 05-SEP YES ONPREM DB TO CLOUD 3 B 0 A SBT_TAPE 05-SEP YES ONPREM DB TO CLOUD 4 B 0 A SBT_TAPE 05-SEP YES ONPREM DB TO CLOUD 5 B 0 A SBT_TAPE 05-SEP YES ONPREM DB TO CLOUD 6 B 0 A SBT_TAPE 05-SEP YES ONPREM DB TO CLOUD 7 B A A SBT_TAPE 05-SEP YES ONPREM ARC TO CLOUD 8 B F A SBT_TAPE 05-SEP NO ONPREM CTL TO CLOUD 9 B 1 A DISK 17-SEP YES LOCALDISK ONPREM DB 10 B 1 A DISK 17-SEP YES LOCALDISK ONPREM DB 11 B 1 A DISK 17-SEP YES LOCALDISK ONPREM DB 12 B 1 A DISK 17-SEP YES LOCALDISK ONPREM DB 13 B 1 A DISK 17-SEP YES LOCALDISK ONPREM DB 14 B 1 A DISK 17-SEP YES LOCALDISK ONPREM DB 15 B A A DISK 17-SEP YES TAG T B A A DISK 17-SEP YES TAG T B A A DISK 17-SEP YES TAG T B F A DISK 17-SEP NO LOCALDISK ONPREM CTL 19 B 1 A SBT_TAPE 17-SEP YES ONPREM DB TO CLOUD 20 B 1 A SBT_TAPE 17-SEP YES ONPREM DB TO CLOUD 21 B 1 A SBT_TAPE 17-SEP YES ONPREM DB TO CLOUD 22 B 1 A SBT_TAPE 17-SEP YES ONPREM DB TO CLOUD 32

24 Oracle Database Cloud 33

25 Oracle Database Cloud Service 34

26 Create Service 35

27 Oracle Database Cloud Service Cloud Storage Container Cloud Storage Container: 36

28 SSH Public Key for VM Access $ chmod 600 privatekey publickey cat ssh.ksh ssh opc@ i privatekey 37

29 Oracle Database Cloud Service Confirmation 38

30 Additional Storage for DBaaS & IaaS 39

31 40

32 Oracle Cloud Home Page 41

33 Storage Containers As of April Cloud Product Storage Cloud Service Storage Cloud Software Appliance Non-Metered Pricing $30.00 / TB / Month Free Metered Pricing $0.024 GB per Month (1TB = $24/Month) Up to 1TB $0.022 GB per Month >5000 TB OOW 16 Screens: Show New Screens Later 42

34 Oracle Cloud Archive Storage 1 TB / Month = $1 +Need to look at Outbound traffic pricing 43

35 Bulk Data Transfer Services 44

36 Amazon Storage Pricing 45

37 Storage Cloud Service Initial Screen 46

38 Creating a Container 47

39 Upload Objects 48

40 Large Object Load 49

41 Create Archive Storage

42 Check Status > Restore File 51 51

43 Restore File Details 52

44 Restore File Details Up to 4 Hrs Later 53

45 Oracle Storage Cloud Upload CLI Demo later Support for both object storage and archive storage Optimized uploads through segmentation and parallelization Automatic checksum verification on upload Upload individual files, groups of files, and entire directories Automatic retry on failures Resume interrupted uploads of large files 54

46 Oracle Cloud Storage - Archive Customers can use Archive containers to store data that won't be accessed for a while or that will be accessed infrequently. It is ideally suited for data sets such as corporate financial records, medical and pharmaceutical archives, cultural preservation content, insurance records and digital film masters. CloudBerry Explorer provides a sleak user interface to Cloud Archive Storage, allowing easy movement of data from local folder to Cloud Archive An Archive container is a container for which the X-Storage-Class metadata field is set to Archive. curl -v -X PUT -H 'X-Auth-Token: token -H 'X-Storage-Class: Archive' 55

47 Archive Storage Create Upload Status Restore Status / Tracking Download curl -v -X PUT -H "X-Auth-Token: AUTH_tk23be53ac28c3e443de38367f87643eb9" -H "X-Storage-Class: Archive" curl -v -X PUT -H "X-Auth-Token: AUTH_tk23be53ac28c3e443de38367f87643eb9" -T myfile foo.storage.oraclecloud.com/v1/storage-myidentitydomain/myfirstarchivecontainer/myobject curl -v -X HEAD -H "X-Auth-Token: AUTH_tk23be53ac28c3e443de38367f87643eb9" foo.storage.oraclecloud.com /v1/storage-myidentitydomain/myfirstarchivecontainer/myobject curl -v -X POST -H "X-Auth-Token: AUTH_tk23be53ac28c3e443de38367f87643eb9" foo.storage.oraclecloud.com /v0/storagemyidentitydomain/myfirstarchivecontainer/myobject?restore curl -v -X GET -H "X-Auth-Token: AUTH_tk23be53ac28c3e443de38367f87643eb9" foo.storage.oraclecloud.com /v0/storagemyidentitydomain/myfirstarchivecontainer?jobs&jobid=a75c8bbf53224a88738e68d6628acd83a4b30 0e4 curl -v -X GET -H "X-Auth-Token: AUTH_tk23be53ac28c3e443de38367f87643eb9" -o myobject1 /myfirstarchivecontainer/myobject 56

48 Back-up to the Cloud Scalable, Redundant, High Available Backup Storage Solution On Premise Backup Oracle Database Backup Service Writing to tape $5,700 (tape library) $0 Storage of tapes $2,400 (1 TB / month) $0 Storage $50,000 (ZFS SA) $ 1,584 $33/TB for 12 months) Hardware $7,000 (tape library) $0 Maintenance $15,000 (HW patches/updates) $0 DBA $15,600 (3 $100/hr) $ 100 (download Backup Client) Total for year 1: $95,700 $1,684 Customer Profile: Scenarios 1. Customer must purchase additional storage on premise and offsite to maintain data archive compliance 2. New to Oracle database and has not acquired skill set to implement best practices for database backup and recovery 3. Failed recovery due to incomplete backups Confidential 57

49 Oracle Storage Cloud Software Appliance Instant On Ramp to Oracle Storage Cloud Services Simple to Install and Use Secure with Encryption and TLS/SSL Scalable Open Standards

50 OSCSA Working on a step-by-step blog on Oracle Storage Cloud Software Appliance From Oracle Linux Installation to setting up Docker to logging into the Management Console Download the OSCSA As of July 30, 2017: Oracle Storage Cloud Software Appliance (Release ) Supports Standard and Archive Storage 59

51 Oracle Storage Cloud Software Appliance 60

52 OCSCA Contents for On-Premise $ gunzip oscsa-onprem tar.gz -rw-r--r-- 1 Dobby staff Jul 30 07:56 oscsa-onprem tar $ tar -tvf oscsa-onprem tar drwxrwxr-x 0 paasusr paasusr 0 Jun 27 16:07./ -rw-rw-r-- 0 paasusr paasusr Jun 27 16:07./oscsa_gw:1.2.1.tar -rwxrwxr-x 0 paasusr paasusr Jun 27 16:07./oscsa -rwxrwxr-x 0 paasusr paasusr Jun 27 16:07./oscsa-install.sh -rwxrwxr-x 0 paasusr paasusr Jun 27 16:07./oscsa-config.sh -rwxrwxr-x 0 paasusr paasusr 1905 Jun 27 16:07./helper-install-oscsa.sh -rwxrwxr-x 0 paasusr paasusr Jun 27 16:07./oscsa-control.sh -rw-rw-r-- 0 paasusr paasusr Jun 27 16:07./OSCSA_GATEWAY_README.txt -rw-rw-r-- 0 paasusr paasusr 2057 Jun 27 16:07./OSCSA_RELEASE_NOTES.txt 61

53 Storage Cloud Software Appliance 62

54 OSCSA Installation oscsa]#./oscsa-install.sh -h oscsa]#./oscsa-install.sh -a Checking that docker is installed and using the correct version Pass found docker version Docker version , build 57bf6fd WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning. ************************************* Checking host prerequisites ************************************* Installing to destination /opt/oscsa_gateway/ Previous install detected, checking version OSCSA version is already installed. Are you sure you want reinstall? (Y N) Y Stopping OSCSA Copied install scripts Copied OSCSA image Starting configuration script Enter the mount location for data cache /OPCcache/data Enter the mount location for meta data /OPCcache/meta Enter the mount location for log file information /OPCcache/log Enter the docker network mode (host or bridge), Hit <Enter> for the default bridge mode. Detected linux operating system Checking kernel version Pass kernel version el7uek.x86_64 found Checking NFS version Pass found NFS version 4 ************************************* All prerequisites have been met ************************************* ************************************* Begin install ************************************* Enter the host port to use for the Administrative Web Interface. Hit <Enter> to use dynamic p Enter the host port to use for NFS access. Hit <Enter> to use dynamic port mapping Enter the host port to use for the optional HTTP REST service. Hit <Enter> to use dynamic p Writing configuration Importing image Please run 'oscsa up' to start the software appliance ************************************* For additional details, please see (/opt/oscsa_gateway/oscsa_gateway_readme.txt) file 63

55 Oracle Storage Cloud Software Appliance Tips for 1 st timers Oracle Linux ships with UEK 3.8 rpm -e kernel-uek el7uek.x86_64 rpm -e kernel-uek-firmware el7uek.noarch Requires Oracle Linux 7 Requires Docker Some Best Practices (Min 16GB RAM) For filesystems with the number of files up to 5 million, the required amount of memory is 32 GB. For large filesystems with the number of files up to 10 million, the required amount of memory is 64 GB Storage Cloud Software Appliance has a UEL 4+ requirement Must upgrade to UEK 4.1 yum install kernel-uek * Will pull in the kernel UEK firmware RPM New learning curve for old timers Not Red Hat 7 but OL 7 with UEK New technology for old timers * It is one of the hottest technologies now Do not modify or maintain OSCSA container with curl REST API Will cause inconsistency --> not recoverable 1. Start with a read cache setting that is 50% of the size of the local storage volume 2. Provision a volume that can accommodate the local cache and ingest new files (upload buffer) without ever becoming more than 80% full 3. Multiple disks (hard disk drives or solid state drives) in a RAID10 4. Enable read-ahead on the volume. 5. Ensure that the number of objects stored in an appliance filesystem doesn t exceed one million (10,000,000) 6. bin-pack or zip small files before writing them to the appliance. 7. However, for optimal performance, ensure that each filesystem is hosted on a dedicated appliance 64

56 Oracle Storage Cloud Software Appliance Backing Up Data Using Oracle Storage Cloud Software Appliance System Requirements for the appliance 4A ECA14BDFD3A.htm#CSSGU-GUID-678DFE1B-B37C-4A ECA14BDFD3A Best Practices for Oracle Storage Cloud Software Appliance 44C3-9C1C-3D htm#CSSGU-GUID-00ADC51A-19FC-44C3-9C1C- 3D

57 Login to the Storage Cloud Software Appliance

58 Create a File System to Sync to OPC

59 Create a File System to Sync to OPC

60 NFS Mount the OPC Software Appliance File System on Client Server # mount -t nfs -o vers=4,port=32770 orabkups:/docs /docs 69

61 Managing the Appliance Start Oracle Storage Cloud Software Appliance Stop Oracle Storage Cloud Software Appli View the latest logs View details and how to access the management console Configure Oracle Storage Cloud Software Appliance to use a proxy server to connect to OPC Configure Oracle Storage Cloud Software Appliance to use SSL to access the management console Determine the version of Oracle Storage Cloud Software Appliance Allocate memory for Oracle Storage Cloud Software Appliance in the appliance host View help for the available commands: oscsa up oscsa down oscsa log oscsa info oscsa configure proxy [http_proxy_server https_proxy_server] oscsa configure ssl true oscsa version oscsa configure memory memory_in_gb oscsa help 70

62 ZFS Storage Appliance: Converged Cloud Storage for Modern IT Handles consolidation with high performance, efficiency, security and management ease Optimizes for demanding applications and databases that run your business Integrates public cloud extensions for all applications and users 71

63 Cloud Converged Storage Puts the Cloud to Work Today Increases IT agility with data and application mobility Diagnostics Public Cloud Storage Support End-to-end visibility Test/dev cloning Seamless experience Backup & Recovery Archive On-Premises High-Performance Storage Appliances ZFS Storage Appliance Extends on-premises storage with unlimited, on-demand cloud capacity Improves storage resilience with geographical dispersion Simplifies IT infrastructure by eliminating stand-alone gateways and cloud exit taxes

64 Follow Us Online! Facebook.com/ViscosityNA Viscosity North America 73

65