Provisioning Oracle RAC in a Virtualized Environment, Using Oracle Enterprise Manager Oracle RAC SIG Web Seminar Thursday, August 5, 2010 Kai Yu, Sr. System Engineer Consultant Dell Global Solutions Engineering
About Authors Kai Yu Senior System Engineer, Dell Oracle Solutions Lab 15 years Oracle DBA and Solutions Engineering Specialized in Oracle RAC, Oracle EBS and OVM Oracle ACE Director, Oracle papers author/presenter IOUG Oracle RAC SIG President Rajat Nigam Senior Product Manager, Oracle Corporation Focus areas Enterprise Manager s Datacenter Automation solution Provisioning, Patching, Upgrade, Migration End user experience of Enterprise Manager install and upgrade
Agenda Introduction to Oracle Enterprise Manager and Oracle VM Oracle Virtualization Infrastructure Implementation Provisioning Oracle 11g RAC using Oracle Enterprise Manager Extending Oracle 11g RAC using Oracle Enterprise Manager QA
Introduction to Oracle Enterprise Manager and Oracle VM New Challenges for IT organization Quick deployment of Oracle service at low cost Deploy using tested and patched gold images An ability to response demand in short order Server consolidation and partition Oracle Enterprise Manager Grid Control Life cycle management of the entire IT stack Oracle Database/ RAC Provisioning and Patching Oracle VM Provisioning and Patching
Introduction to Oracle Enterprise Manager and Oracle VM Automated provisioning, patching, lifecycle management Gold images based & reference based deployment
RAC PROVISIONING
Oracle VM Technology Oracle VM Server: Xen Hypervisor, management domain dom0 Linux kernel with support of devices, IO, networking, etc. VM Server Pool: an autonomous region of VM servers VM: guest OS with applications running on domu
Oracle VM and Oracle RAC
VM and Virtualization Management VM Management Pack features Lifecycle Automation for Guest Virtual Machines
OVM Infrastructure Implementation Dell/Oracle Joint POC Project: Oracle Grid/ RAC Provisioning Expand the Grid to include virtualization Implement the Grid on Commodity hardware and SAN storage Grid control as the unified management for the grid Oracle Grid Architecture Physical Grid and Virtual Grid Based on Commodity Hardware Dell Blade Servers Focus of this presentation: Establish the VM infrastructure for running 11g RAC on VM Provision 11g RAC on VMs using Enterprise Manager Grid Control
OVM Infrastructure Implementation Virtualization Infrastructure Design VM Servers: Dell blade servers Shared storage: Dell EqualLogic iscsi SAN Unified Management: Grid Control with VM Management
OVM Infrastructure Implementation Prepare VM Infrastructure for RAC VM server installation Network configuration in VM servers Configure Shared storage in VM servers Configure Enterprise Manage Grid control Enable VM management pack to Grid control Create guest VMs for the RAC nodes Configuration network on guest VMs Present the shared storage to guest VMs as the virtual storage for Oracle homes, OCRs/Votingdisks, ASM disk volumes.
OVM Infrastructure Implementation Oracle VM server Installation Enable virtualization for CPUs on BIOS Install Oracle VM server 2.1.2 Configure dom0 memory by editing /boot/grub/menu.lst Secure OVM agent is running: service ovs-agent status Oracle VM server network configuration
OVM Infrastructure Implementation Customize the default Xen bridge configuration Disable the default Xen bridge settings. Create ifcfg-ethn, incfg-bond0, ifcfg-xenbr0, ifcfg-xenbr1 Restart the network services Oracle VM server shared storage configuration Shared storage partitions added into OVM repositories /dev/mapper/ovs_data4p1; /dev/mapper/ovs_data6p1 Additional disk partitions for guest VMs through physical disks mappings for RAC shared disks 2 X OCRs, 3 X Votingdisks, ASM disk volumes Detailed Information in Building an Oracle Grid with Oracle VM on Dell Blade Servers and Dell EqualLogic iscsi Storage, Dell Oracle Solutions Whitepaper
OVM Infrastructure Implementation Grid Control Management Infrastructure Install EM Grid control 10.2.0.3,Upgraded to 10.2.0.5 Enable VM Management Pack: metalink#781879.1 Connect the VM servers to Grid Control Connect VM servers to EM Grid control Create VM server pool with the VM server
OVM Infrastructure Implementation Create Guest VMs using the VM template Download/Register OVM_EL5U2_X86_64_PVM_10GB Create VMs from the template
OVM Infrastructure Implementation Specify the parameters for the guest VM Guest VM is created and running on virtual server kblade10
OVM Infrastructure Implementation Present xenbr0 and xenbr1 as eth0 (public) and eth1(private) vm.cfg: vif=['bridge=xenbr0,mac=00:16:3e:11:8e:ce,type=netfront', 'bridge=xenbr1,mac=00:16:3e:50:63:25,type=netfront', ] Add the additional disk and shared storages to the VMs Attached the shared disks from VM repositories
OVM Infrastructure Implementation Virtual disk partitions storage configuration Partitions Size File system Raw devices /dev/xvda1 104391 Os /dev/xvda2 13526730 OS /dev/xvdb1 15360000 /u0 /dev/xvdc1 21486906 ASM disk /dev/raw/raw1 /dev/xvdd1 297171 ocr1 /dev/raw/raw2 /dev/xvdd2 297202 votingdisk1 /dev/raw/raw3 /dev/xvdd3 297202 ASM spfile /dev/raw/raw4 /dev/xvde1 297171 ocr2 /dev/raw/raw5 /dev/xvde2 297202 votingdisk2 /dev/raw/raw6 /dev/xvde3 297202 votingdisk2 /dev/raw/raw7
OVM Infrastructure Implementation vm.cfg: disk = [''file:/ovs/shareddisk/racdb.img,xvdc,w!', b. Attached the physical partitions to the guest VM vm.cfg: disk = [' phy: /dev/mapper/vmracdbp1, xvdc,w!',
Provision Oracle 11g RAC using EM Install/configure Grid control Agent on RAC node VMs Grid control agent 10.2.5 installed on two guest VMs Disable firewall services Check host/cluster preferred credentials on the guest VMs
Provision Oracle 11g RAC using EM Setup software library: Deployment Provisioning Administration: Software Library configuration Load the Oracle RAC shipped software to the software library Download 11g software; Deployment Components
Provision Oracle 11g RAC using EM Upload Oracle Database binary and Oracle clusterware binary
Provision Oracle 11g RAC using EM Prepare guest VM nodes for RAC provisioning Install/configure Grid control Agent on RAC node VMs Prepare shared devices: 2 X OCR and 3 X votingdisk, ASM disk /udev/rules.d/65-raw.rules and 89-raw.rules: map raw devices Change kernel parameters for RAC install Extend OS partition by adding another 20 GB disks using LVM Prepare /etc/hosts to include public, private, VIPs: hostnames/ips
Provision Oracle 11g RAC using EM In Grid control, click the Deployments tab. On the Deployments page, in the Deployment Procedure Manager section, click RAC Provisioning Procedures.
Provision Oracle 11g RAC using EM Provision Oracle RAC on guest VM Start the provisioning procedure Select source from Software library Click Schedule deployment and specify the source of software
Provision Oracle 11g RAC Using EM Select Target Host:
Provision Oracle 11g RAC using EM Target: clusterware/asm RAC Database details
Provision Oracle 11g RAC using EM Shared Storage configuration for RAC Submit the provisioning job and check the job status
Provision Oracle 11g RAC using EM Check the Job status and ensure it completed successfully Results: 1)11g R1 Clusterware installed and configured 2) ASM home installed, the instance configured 3) 11g R1 RAC home installed and RAC Database created and started up.
Extend Oracle 11g RAC using EM MANAGER Two ways to extend RAC One Click Extend Deployment Procedure Clone from the existing RAC node s Template Prepare the guest VM for the new node Create a new guest VM Same network, disks, kernel settings, rpms. disable firewall Install Grid control agent 10.2.0.5 Add host and cluster preferred credentials of Oracle users Schedule the provisioning procedure to extend the RAC Deployment Provisioning Deployment Procedure Manager Provisioning Procedure RAC
Extend Oracle 11g RAC using EM Select One Click Extend Cluster Database, Click Schedule Deployment
Extend Oracle 11g RAC using EM Select the RAC database to extend Specify the reference host
Extend Oracle 11g RAC using EM Specify the new node to add and the user credentials
Extend Oracle 11g RAC using EM Check the job status: procedure completed successfully
Extend Oracle 11g RAC using EM Verify the RAC extension results: Check clusterware status: [root@bvmrac2 bin]#./crsctl check crs Cluster Synchronization Services appears healthy Cluster Ready Services appears healthy Event Manager appears healthy Check database service: [oracle@bvmrac2 bin]$ srvctl status database -d vmracdb Instance vmracdb1 is running on node bvmrac1 Instance vmracdb2 is running on node bvmrac2
Summary Use Grid control to manage the virtual servers Use Grid Control Provisioning pack to provision 11g RAC on VMs and extend the RAC to additional VM node. Coming Oracle OpenWorld 2010 Presentations S316318: Oracle RAC/Oracle VM Automated Provisioning with Oracle Enterprise Manager 11g S316263: Monitoring and Diagnosing Oracle RAC Performance with Oracle Enterprise Manager 11g Dell Oracle Whitepapers: www.dell.com/oracle Efficient Deployment of Oracle RAC in a Virtualized Environment Using Enterprise Manager Provisioning Pack Building an Oracle Grid with Oracle VM on Dell Blade Servers and Dell EqualLogic iscsi Storage,
QA
Thank You and Visit my Oracle Blog at http://kyuoracleblog.wordpress.com/