OPTIMIZE ORACLE 12C WITH ZFS STORAGE MIKE GANGLER, LEAD ORACLE DBA SECURE-24

Transcription

1 OPTIMIZE ORACLE 12C WITH ZFS STORAGE MIKE GANGLER, LEAD ORACLE DBA SECURE-24

2 Performance. Innova.on. Trust. We are the Managed Cloud. 2 Secure-24 Confidential Internal Copyright 2015 Secure-24 and/or its affiliates. All rights reserved.

3 ABOUT MIKE GANGLER Oracle ACE with robust database creden5als DBA for over 28 years, working with Oracle since version 4 Lead and Senior Database Specialist at Secure-24 Currently serving on the board of the Southeast Michigan Oracle Professionals (SEMOP) group Charter member of the Board of Directors for the Interna5onal Oracle Users Group (IOUG) Follow me on my Blog hqp://mjgangler.wordpress.com and on 3

4 Join for as low as $150 SELECT Journal Resource Center IOUG Press Webinars & Podcasts IOUG Forum 5 Minute Briefing Communi.es Educa.on Plus get access to IOUG s content library, peer-to-peer networking, and more! Corporate options also available!

5 SOUTHEASTERN MICHIGAN ORACLE PROFESSIONALS (SEMOP) Technology and Database Professionals - hqp:// Contact: Mike Gangler mjgangler@gmail.com Meet monthly 2 nd Tuesday of the month

6 TODAY S DISCUSSION Learn how Secure-24 leverages Oracle ZFS Storage, Oracle Intelligent Storage Protocol (OISP), Templates and Oracle 12C RAC to implement a Private Cloud for Oracle 12c RAC applica5ons. This pairing of Database and Storage hardware through Oracle s VCA has enabled Secure-24 to take advantage of the unique integra5on capabili5es such as OISP which enables automa5c and dynamic tuning of storage se^ngs such as record size. 6 Secure-24 Confidential Internal Copyright 2015 Secure-24 and/or its affiliates. All rights reserved.

7 SECURE-24 TECHNOLOGY GOALS Simplicity Simplify the environment with new technology Scalability Technology should be able to grow and scale as the company grows. Automa5on Technology should have the ability to be automated and repeatable process Overall reduc5on of cost Technology should drive down the overall cost Solu5on driven by Customer requirements and demands. This is the reason for looking at the technology

8 BUSINESS CASE Simplify Oracle 12c RAC deployments Eliminate complexity around storage tuning Decrease support 5me through transparency and simplicity Fully Oracle Support technology stack

9 ORACLE PRIVATE CLOUD APPLIANCE Hardware Specifica5ons Overview» High speed InfiniBand network mul.ple 40 Gbps 40Gb/s Server-to-Server Compute Nodes Fabric Interconnect Ethernet Switch InfiniBand Expansion InfiniBand Expansion Fabric Interconnect Compute Nodes Mgmt Nodes Management Storage Networking Management Oracle X86 Servers Up to 25 nodes CPU: Up to 900 cores Memory: Up to 6.4TB aggregate Supports any Linux, Solaris and Windows applica.ons Network Infrastructure Oracle Fabric Interconnect F1-15 (Virtual Network) Oracle Switches Fully configured fabric networking, no setup required Integrates with exis.ng Ethernet and storage networks Oracle X86 Servers Oracle ZFS Storage Appliance Redundant controllers, high performance virtual storage» InfiniBand connected storage» Dedicated management systems included» Oracle VM and Oracle Linux included» Tested and Cer.fied by Oracle» So_ware Defined networking through Fabric Interconnect 9

10 WHY WE SELECTED PRIVATE CLOUD APPLIANCE Fast Power-on to Produc5on - Automa5on 2 hours vs. 1 month on legacy infrastructure Mixed workloads - Scalable Oracle DB, Oracle Applica5ons, Microsoe, Custom apps More efficient Licensing Reduc5on of cost Trusted Par55ons Licensing Rapid Applica5on Deployment Simplicity and automa5on Pre-built Oracle VM templates Simplified Support - Simplicity One Hand to Shake (One Throat to Choke) Lower Acquisi5on and Deployment Costs Impressive performance Customer requirements

11 ORACLE 12C RAC DESIGN OVERVIEW Oracle PCA and ZS3-2 Storage Single Node or mul5ple nodes with RAC X3-2 Oracle 12c Enterprise Edi5on Oracle ZFS storage array with OS 8.3+ ZFS Storage Flash enabled performance storage OISP (Oracle Intelligent Storage Protocol) for storage op5miza5on Z F S Z S 3-2 SP R E AR PS C R IT IC AL MAJ O R MIN O R US E R VIRTUAL COMPUTE APPLIANCE Database connec5ons made from Oracle using Direct NFS Z F S Z S 3-2 SP R E AR PS C R IT IC AL MAJ O R MIN O R US E R No ASM

12 DESIGN OVERVIEW Local OS disks Stored on Oracle VM data repository Data, Logs, and Vo5ng disks presented via NFS and dnfs RAC VM 1 RAC SI VM 2 VM VM dnfs Oracle VM Server Pool VM ZFS Storage Oracle VM Servers NAS, SAN, iscsi

13 ZFS POWERED ZFS Hybrid Array with 3 types of caching DRAM, SLC Write FLASH, MLC Read FLASH OISP Op.mal Record Size & Latency The Oracle dnfs client passes the op.mal record size to the ZFS Storage Appliance for each NFSv4 write request The Oracle dnfs client passes the op.mal write bias to the ZFS Storage Appliance for each NFSv4 write request.. OISP MOS Doc ID ZFS Admin Guide hqp:// docs.oracle.com/cd/e51475_01/pdf/ E52872.pdf

14 ZFS SETUP Create our 3 shares OCR Vo5ng Disk This will be raw NFS v3 hqps://docs.oracle.com/database/121/cwadd/ votocr.htm#cwadd92368 We leave Write Bias as Latency and Record Size as 128K (Generic) Data and Logs files This is where the magic of OISP comes in. Instead of having to create mul5ple different volumes for redo, archive, control, data, temp etc. We simple create 2 volumes for Logs and Data Write Bias and Record Size will be set via database cue s through OISP and op5mized for the database.

15 STANDARD NFS ARCHITECTURE

16

17

18 DNFS AND OISP

19 LINUX CONFIGURATION SETUP dnfs Fairly simple example # cat oranfstab.example server: nfs_version: nfsv4 export: /export/u02 mount: /u02 export: /export/u03 mount: /u03 dnfs is fairly straight forward in 12c setup guide hqps://docs.oracle.com/database/121/cwlin/ storage.htm#cwlin279 This is where NFSv4 is setup Many other op5ons are possible, but simple is possible too Old paper, s5ll valid - hqp:// technetwork/ar5cles/directnfsclient-11gr1- twp pdf Most dnfs guides are for 11g, 12c is the same except dnfs is Na5vely part of 12c Troubleshoo5ng (Doc ID )

20 DNFS LINUX CONFIGURATION SETUP dnfs Enable: Ln s $ORACLE_HOME/lib/libnfsodm11.so $ORACLE_HOME/lib/libodm11.so Disable Ln s $ORACLE_HOME/lib/libodmd11.so $ORACLE_HOME/lib/libodm11.so

21 DNFS LINUX CONFIGURATION SETUP dnfs Verify bash-3.2$ ps -ef grep dbw oracle :53? 00:00:00 ora_dbw0_myinst1 oracle :53? 00:00:00 ora_dbw1_myinst1 oracle :04 pts/0 00:00:00 grep dbw bash-3.2$ /usr/sbin/lsof -p COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME oracle oracle cwd DIR 104, /u02/app/oracle/product/12.1.0/dbs oracle oracle rtd DIR 104, / oracle oracle txt REG 104, /u02/app/oracle/product/12.1.0/bin/oracle... oracle oracle 32u IPv TCP mydbhost:64553->mynfshost:nfs (ESTABLISHED) oracle oracle 33u IPv TCP mydbhost:21549->mynfshost:nfs (ESTABLISHED) -- dnfs setup guide hqps://docs.oracle.com/database/121/cwlin/storage.htm#cwlin279

22 LINUX CONFIGURATION SETUP Oracle Linux 6 uek Kernel U5lize dtrace in Linux Ksplice for non-disrup5ve kernel patching UEK kernel op5miza5on for Oracle DB Setup Mount Points Setup Vo5ng disk Must be NFSv3 Not Part of dnfs Mount op5ons important (Doc ID ) rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,noac,vers=3,5meo=600,ac5meo=0 Setup Logs and Data Must be NFSv3 (dnfsv4 setup to come later) Mount op5ons important (Doc ID ) rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,ac5meo=0,vers=3,5meo=600

23 LINUX CONFIGURATION SETUP Install Oracle Pre-Install RPM Op5onal but makes life easier This will download all required dependencies, create the oracle user id, modify sysctl.conf and modify limits.conf. hqps://blogs.oracle.com/ wim/entry/ easily_install_oracle_rdbms_1 2cr1 Test snmp (needed for OISP) Doc ID # snmpget -v1 -c public # yum install oracle-rdbmsserver-12cr1-preinstall SNMPv2- SMI::enterprises = STRING: "Sun ZFS Storage 7330" Setup for dnfs Oracle wants us to tune for dnfs, so we will # vi /etc/sysctl.conf # # hqps://docs.oracle.com/database/ 121/CWLIN/ storage.htm#chdefhfc #7.3.4 Checking TCP Network Protocol Buffer for Direct NFS Client # net.ipv4.tcp_rmem = net.ipv4.tcp_wmem =

24 ORACLE 12C INSTALLATION Oracle 12c Enterprise Edi5on Install Oracle Confirm dnfs and OISP setup $ egrep I '"Sun ZFS" Direct' alert_*.log Found NAS server Sun ZFS Storage 7420 Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0 Direct NFS: channel id [0] path [ ] to filer [ ] via local [] is UP Direct NFS: channel id [1] path [ ] to filer [ ] via local [] is UP Direct NFS: mount complete dir /export/oisp/redo on nfsport 2049 using NFS version 4 Direct NFS: mount complete dir /export/oisp/redo on nfsport 2049 using NFS version 4 Found NAS server indicates SNMP has found the ZFSSA. Requirement for OISP. ODM/Direct NFS indicates that dnfs has been ac5vated using NFSv4. Apply required patch Patch Resolves issue with DB Analy5cs

25 TYPICAL DEPLOYMENT Typical Process For Creating and Deploying New Applications 1 to 3 weeks 1-5 days 1-2 days 1-5 days 1 day 1-2 days Start

26 PCA RAPID DEPLOYMENT Pre-made, cer5fied virtual machine templates, including applica5on, opera5ng system, and database available at hqps://edelivery.oracle.com Save Weeks/Days in Configura5on Time Pre-built, pre-configured Templates Applica5on, Middleware, and DB Oracle DB, JD Edwards, EBS, PeopleSoe, and many more. 90+ templates E-Delivery Download from Oracle.com JDE JD Edwards File File JDE VM 1 Import to Oracle VM Manager Start-up in Oracle VM Pool Oracle VM Servers Server Customize &Pool Save 2 as Golden Image

27 APPLICATION TEMPLATES & ASSEMBLIES Oracle VM Assembly Produc.on Environment: SOA_PROD_1 Deployed: Produc.on Environment SOA_PROD_1 SOA_PROD_1 VM VM VM VM VM WebLogic Server VM Deployment Policies Configura.on Metadata DB OVF-based Deploy Oracle VM Servers Server Pool 2 Storage Repository Deploy applica.on into Oracle VM Manager as Golden Image Clone, ini.alize, and start-up instance Up and running in hours vs. days or weeks with liile risk 3 Guest Addi.ons Allows direct integra.on between guest so_ware and virtualiza.on layer Assists in orchestra.on of complex, mul.- VM deployments

28 NOTES Oracle Database 11g RAC with ZFS Storage Appliance and dnfs on SPARC: hqp:// Op5mizing Storage for Oracle Database 11gR2 with ZFS Storage Appliance: hqp:// How to enable OISP on ZFS Storage Appliance for 12c: hqps://support.oracle.com/epmos/faces/documentdisplay?id= ANALYST DATA hqp://

29 VISIT SECURE-24 S TABLE! Enter for a chance to win a $250 Amazon Gie Card! Meet with other S-24 execu5ves and technical resources Discuss your organiza5on s Cloud Strategy for 2016 Learn more about our capabili5es with Oracle s Virtual Compute Appliance Learn about our newest product launch: Engineering Desktops

30 QUESTIONS Mike Gangler Blog : hips://mjgangler.wordpress.com JDE mjgangler Server Pool 2