Build two node Oracle RAC 12c ( ) cluster on OEL 6.5 using Oracle VM Virtual Box

Transcription

1 Build two node Oracle RAC 12c ( ) cluster on OEL 6.5 using Oracle VM Virtual Box In the article you will have a look at how to use some Oracle Virtual Box features to build two node Oracle 12c ( ) RAC system on Oracle Enterprise Linux (OEL 6.5). The setup will implement a role separation with different users for Oracle RDBMS and Oracle GI that is, user oracle and grid are used respectively in order to split the responsibilities between DBAs and storage administrators. An overview to Oracle Virtualization solutions can be seen here. You can see how to use Oracle VM Virtual Box to build a two node 11gR2 RAC cluster on Solaris 10 and Solaris 11. For information related to building a RAC 11gR2 cluster on OEL 5.X/6.X without GNS click here and here. For information how to build a two node Oracle RAC 11gR with GNS (DNS, DHCP) and HAIP look here. For information how to upgrade Oracle RAC cluster on OEL 6.1 to Oracle RAC cluster 12c on OEL6.5 look here. In the article you will see how to configure Linux in an Oracle VM Virtual Box Virtual machines, install Oracle GI, Oracle RDBMS and will create a RAC enabled Multitenant Container Database (CDB). The following software will be used: 1. Oracle VM Virtual Box download from here. 2. Oracle 12c (database, grid, examples) download from OTN here. 3. Oracle OEL 6.5 download from Oracle Software Delivery Cloud (former Edelivery) here. Two virtual machines will be created and used. 1. OEL65A for RAC node oel65a 2. OEL65B for RAC node oel65b The OEL65A/OEL65B Virtual Machines will be configured for RAC nodes each with: 4096MB RAM 80GB bootable disk (Disk space will be with a fixed size preallocation using LVM) NIC bridged for public interface in RAC with address /12 (first IP on oel65a and second IP on node oel65b). These are public interface in RAC. NIC bridged for private interface in RAC with address /12 (first IP on oel65a and second IP on node oel65b). These are private interface in RAC. NIC bridged for private interface in RAC with address /72 (first IP on oel65a and second IP on node oel65b). These are private interface in RAC.

2 NIC bridged for private interface in RAC with address /82 (first IP on oel65a and second IP on node oel65b). These are private interface in RAC. 5 10GB attached shared disks for the ASM storage. (High Redundancy ASM disk group DATA will be deployed for CRS/Vote and data files). 5 2GB attached shared disks for the ASM storage. (External Redundancy ASM disk group FRA will be deployed as flash/fast recovery area). The interfaces IP addresses will be as show in the table 1 below. VM OEL65A RAC node oel65a VM OEL65B RAC node oel65b eth eth eth eth The /etc/hosts file for oel65a/oel65b will be as follows: oel65a.gj.com oel65a oel65a-priv.gj.com oel65a-priv oel65a-vip.gj.com oel65a-vip oel65a-priv1.gj.com oel65a-priv oel65a-prib2.gj.com oel65a-priv oel65b.gj.com oel65b oel65b-priv.gj.com oel65b-priv oel65b-vip.gj.com oel65b-vip oel65b-priv1.gj.com oel65b-priv oel65b-prib2.gj.com oel65b-priv scan.gj.com scan scan.gj.com scan scan.gj.com scan The article will cover the following topics 1. Create an OEL65A VM with OEL 6.5 as guest OS for node oel65a. 2. Configure the OEL61A VM to meet the prerequisites for GI and RAC 12c deployment. 3. Clone OEL61A to OEL61B and attach the shared disks. 4. Change IP and hostname on OEL65B 5. Install GI 12c on oel65a and oel65b. 6. Install RAC RDBMS 12c on oel65a and oel65b. 7. Create a Multitenant Container Database (CDB) database RACDB oel65a and oel65b. 8. Verify database creation and create a service.

3 1. Create an OEL65A VM with OEL 6.5 as guest OS for node oel65a In this section you will look at how to create a guest OEL 6.5 VM using Oracle VM Virtual Box. 1.1 Select New icon or Machine->New 1.2 Select the RAM size. The final value I used is 4096MB. Larger values turned out to cause VM instability. The error below from the VM log was avoided by reducing the VM RAM from 5000MB to 4096MB. run OUI 00:26: VM: Raising runtime error 'HostMemoryLow' (fflags=0x2) 00:26: AssertLogRel D:\tinderbox\win- 4.3\src\VBox\VMM\VMMR3\VMM.cpp(1756) int cdecl VMMR3EmtRendezvous(struct VM *,unsigned int,int ( cdecl *)(struct VM *,struct VMCPU *,void *),void *):!pvcpu->vmm.s.finrendezvous 00:26: Console: VM runtime error: fatal=false, errorid=hostmemorylow message="unable to allocate and lock memory. The virtual machine will be paused. Please close applications to free up memory or close the VM"

4 00:26: Changing the VM state from 'RUNNING' to 'GURU_MEDITATION'. 1.3 Select Create Virtual Hard Drive now

5 1.4 Select the drive type VDI

6 1.5 Select Dynamically allocated

7 1.6 Select the HD size of 80G or more.

8 1.7 The VM is created

9 1.8 Add the OEL 65 ISO image so that you can boot and install the OEL 65.

10 1.9 Configure the Clipboard and mouse integration 1.10 Configure all adapters as bridged

11 1.11 Start the VM and select Install

12 1.12 Select Skip

13 1.13 Press Next

14 1.14 Select language

15 1.15 Select keyboard

16 1.16 Select Basic Storage Devices

17 1.17 Select discard to reformat

18 1.18 Enter hostname

19 1.19 Enter IP addresses, Net masks and gateways. For example eth1 you will have the configuration below. Repeat the configuration for all interfaces eth0,eth2,eth3

20 1.20 Click Next after configuring all interfaces ethx and the hostname.

21 1.21 Select a city in the time zone.

22 1.22 Enter root password.

23 1.23 Select use all space

24 1.24 Write the changes to disk

25 1.25 I am going to do a full install and will pick all rpms.

26 1.26 I selected everything

27 1.27 Wait for the install to complete.

28 1.28 Press Forward

29 1.29 Continue

30 1.30 You can register any time. I opted for register later.

31 1.31 Press Forward

32 1.32 Create a user.

33 1.33 Setup date and time

34 1.34 I did not enable kdump

35 1.35 The login screen comes up after a reboot

36 1.36 Install Virtual Box additions

37 1.37 Press Run

38 1.38 Wait for the installation to completes

39 1.39 Once installation is done power reboot the VM

40 1.40 This concludes OEL65 installation 2. Configure the OEL61A VM to meet the prerequisites for GI and RAC 12c deployment. 2.1 Modify the UEKR3 kernel and add divider=10 by editing /etc/grub.conf. This will speed up the OEL guest VM. ~]# cat /etc/grub.conf # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/mapper/vg_oel65a-lv_root # initrd /initrd-[generic-]version.img #boot=/dev/sda default=0

41 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Oracle Linux Server Unbreakable Enterprise Kernel ( el6uek.x86_64) root (hd0,0) kernel /tboot.gz logging=vga,serial,memory numa=off transparent_hugepage=never module /vmlinuz el6uek.x86_64 ro root=/dev/mapper/vg_oel65a-lv_root intel_iommu=on amd_iommu=on rd_no_luks LANG=en_US.UTF-8 rd_no_md SYSFONT=latarcyrheb-sun16 rd_lvm_lv=vg_oel65a/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_lvm_lv=vg_oel65a/lv_root rd_no_dm rhgb quiet divider=10 module /initramfs el6uek.x86_64.img title Oracle Linux Server Unbreakable Enterprise Kernel ( el6uek.x86_64.debug) root (hd0,0) kernel /tboot.gz logging=vga,serial,memory numa=off transparent_hugepage=never module /vmlinuz el6uek.x86_64.debug ro root=/dev/mapper/vg_oel65a-lv_root intel_iommu=on amd_iommu=on rd_no_luks LANG=en_US.UTF-8 rd_no_md SYSFONT=latarcyrheb-sun16 rd_lvm_lv=vg_oel65a/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_lvm_lv=vg_oel65a/lv_root rd_no_dm rhgb quiet module /initramfs el6uek.x86_64.debug.img title Oracle Linux Server Red Hat Compatible Kernel ( el6.x86_64) root (hd0,0) kernel /tboot.gz logging=vga,serial,memory numa=off transparent_hugepage=never module /vmlinuz el6.x86_64 ro root=/dev/mapper/vg_oel65a-lv_root intel_iommu=on amd_iommu=on rd_no_luks LANG=en_US.UTF-8 rd_no_md SYSFONT=latarcyrheb-sun16 rd_lvm_lv=vg_oel65a/lv_swap crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_lvm_lv=vg_oel65a/lv_root rd_no_dm rhgb quiet module /initramfs el6.x86_64.img title Oracle Linux Server Unbreakable Enterprise Kernel ( el6uek.x86_64) root (hd0,0) kernel /tboot.gz logging=vga,serial,memory numa=off transparent_hugepage=never module /vmlinuz el6uek.x86_64 ro root=/dev/mapper/vg_oel65a-lv_root intel_iommu=on amd_iommu=on rd_no_luks LANG=en_US.UTF-8 rd_no_md SYSFONT=latarcyrheb-sun16 rd_lvm_lv=vg_oel65a/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_lvm_lv=vg_oel65a/lv_root rd_no_dm rhgb quiet module /initramfs el6uek.x86_64.img title Oracle Linux Server Unbreakable Enterprise Kernel ( el6uek.x86_64.debug) root (hd0,0) kernel /tboot.gz logging=vga,serial,memory numa=off transparent_hugepage=never module /vmlinuz el6uek.x86_64.debug ro root=/dev/mapper/vg_oel65a-lv_root intel_iommu=on amd_iommu=on rd_no_luks LANG=en_US.UTF-8 rd_no_md SYSFONT=latarcyrheb-sun16 rd_lvm_lv=vg_oel65a/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_lvm_lv=vg_oel65a/lv_root rd_no_dm rhgb quiet module /initramfs el6uek.x86_64.debug.img

42 title Oracle Linux Server Red Hat Compatible Kernel ( el6.x86_64) root (hd0,0) kernel /tboot.gz logging=vga,serial,memory numa=off transparent_hugepage=never module /vmlinuz el6.x86_64 ro root=/dev/mapper/vg_oel65a-lv_root intel_iommu=on amd_iommu=on rd_no_luks LANG=en_US.UTF-8 rd_no_md SYSFONT=latarcyrheb-sun16 rd_lvm_lv=vg_oel65a/lv_swap crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_lvm_lv=vg_oel65a/lv_root rd_no_dm rhgb quiet module /initramfs el6.x86_64.img You have new mail in /var/spool/mail/root ~]# 2.2 Run the command yum install oracle-rdbms-server-12cr1- preinstall. The output is in the Annex A. Note that you need to temporary enable internet access by a) configure any interface to obtain a DHCP address and b) modify the corresponding adaptor to NAT. After successful installation revert to the original configuration. 2.3 If you have not install all packages make sure that the following packages are installed. binutils el6 (x86_64) compat-libcap (x86_64) compat-libstdc el6 (x86_64) compat-libstdc el6.i686 gcc el6 (x86_64) gcc-c el6 (x86_64) glibc el6 (i686) glibc el6 (x86_64) glibc-devel el6 (x86_64) glibc-devel el6.i686 ksh libgcc el6 (i686) libgcc el6 (x86_64) libstdc el6 (x86_64) libstdc el6.i686 libstdc++-devel el6 (x86_64) libstdc++-devel el6.i686 libaio el6 (x86_64) libaio el6.i686 libaio-devel el6 (x86_64) libaio-devel el6.i686 libxext-1.1 (x86_64) libxext-1.1 (i686) libxtst (x86_64) libxtst (i686) libx (x86_64) libx (i686) libxau (x86_64) libxau (i686) libxcb-1.5 (x86_64) libxcb-1.5 (i686) libxi-1.3 (x86_64) libxi-1.3 (i686)

43 make el6 sysstat el6 (x86_64) 2.4 The oracle-rdbms-server-11gr2-preinstall takes care of the oracle user. Similarly modify /etc/security/limits.conf to address the grid user. # oracle-rdbms-server-11gr2-preinstall setting for nofile soft limit is 1024 grid soft nofile 1024 # oracle-rdbms-server-11gr2-preinstall setting for nofile hard limit is grid hard nofile # oracle-rdbms-server-11gr2-preinstall setting for nproc soft limit is 2047 grid soft nproc 2047 # oracle-rdbms-server-11gr2-preinstall setting for nproc hard limit is grid hard nproc # oracle-rdbms-server-11gr2-preinstall setting for stack soft limit is 10240KB grid soft stack # oracle-rdbms-server-11gr2-preinstall setting for stack hard limit is 32768KB grid hard stack Add to /etc/pam.d/login a line session required pam_limits.so 2.6 Enabling the Name Service Cache Daemon chkconfig --level 35 nscd on service nscd start 2.7 Configure NTP You have two options for time synchronization: an operating system configured network time protocol (NTP), or Oracle Cluster Time Synchronization Service. Oracle Cluster Time Synchronization Service is designed for organizations whose cluster servers are unable to access NTP services. If you use NTP, then the Oracle Cluster Time Synchronization daemon (ctssd) starts up in observer mode. If you do not have NTP daemons, then ctssd starts up in active mode and synchronizes time among cluster members without contacting an external time server. So there are two options: For enabling the NTP make sure that /etc/sysconfig/ntpd has the line modified to include -x.

44 OPTIONS= -x -u ntp:ntp -p /var/run/ntpd.pid For disabling the NTP make sure that the NTP service is stopped and disabled for auto-start and there is not configuration file. /sbin/service ntpd stop chkconfig ntpd off mv /etc/ntp.conf to /etc/ntp.conf.org In the article NTP is enabled. 2.8 Set in /etc/ssh/sshd_config the following line LoginGraceTime Create OS groups. The masked are create by oracle-rdbmsserver-11gr2-preinstall #groupadd -g oinstall #groupadd -g dba groupadd -g oper groupadd -g backupdba groupadd -g asmdba groupadd -g dgdba groupadd -g kmdba groupadd -g asmadmin groupadd -g asmoper 2.10 Create users. useradd -u g oinstall -G asmadmin,asmdba grid ##useradd -u g oinstall -G dba,backupdba,dgdba,kmdba,asmdba asmoper oracle usermod -g oinstall -G dba,backupdba,dgdba,kmdba,asmdba,asmoper oracle 2.11 Create Directories mkdir -p /u01/app/12.1.0/grid mkdir -p /u01/app/grid mkdir -p /u01/app/oracle chown -R grid:oinstall /u01 chown oracle:oinstall /u01/app/oracle chmod -R 775 /u01/ chown oracle:oinstall /u01/app/oracle

45 2.12 List disks dev]# ls sd* sda sda1 sda2 dev]# 2.13 Create shared disks From the host OS ( Windows in my case) run the following commands. VBoxManage createhd --filename e:\vb\asm1.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm2.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm3.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm4.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm5.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm6.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm7.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm8.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm9.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm10.vdi --size format VDI -- variant Fixed VBoxManage createhd --filename e:\vb\asm11.vdi --size format VDI -- variant Fixed VBoxManage storageattach OEL65a --storagectl "SATA" --port 1 --device 0 -- type hdd --medium asm1.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 2 --device 0 -- type hdd --medium asm2.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 3 --device 0 -- type hdd --medium asm3.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 4 --device 0 -- type hdd --medium asm4.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 5 --device 0 -- type hdd --medium asm5.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 6 --device 0 -- type hdd --medium asm6.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 7 --device 0 -- type hdd --medium asm7.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 8 --device 0 -- type hdd --medium asm8.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 9 --device 0 -- type hdd --medium asm9.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 10 --device 0 -- type hdd --medium asm10.vdi --mtype shareable VBoxManage storageattach OEL65a --storagectl "SATA" --port 11 --device 0 -- type hdd --medium asm11.vdi --mtype shareable

46 2.14 List and format the disks Initially you will have dev]# ls -l sd* brw-rw root disk 8, 0 May 12 16:37 sda brw-rw root disk 8, 1 May 12 16:37 sda1 brw-rw root disk 8, 2 May 12 16:37 sda2 brw-rw root disk 8, 16 May 12 16:37 sdb brw-rw root disk 8, 32 May 12 16:37 sdc brw-rw root disk 8, 48 May 12 16:37 sdd brw-rw root disk 8, 64 May 12 16:37 sde brw-rw root disk 8, 80 May 12 16:37 sdf brw-rw root disk 8, 96 May 12 16:37 sdg brw-rw root disk 8, 112 May 12 16:37 sdh brw-rw root disk 8, 128 May 12 16:37 sdi brw-rw root disk 8, 144 May 12 16:37 sdj brw-rw root disk 8, 160 May 12 16:37 sdk brw-rw root disk 8, 176 May 12 16:37 sdl [root@oel65a dev]# For each disk from /dev/sdb to /dev/sdl perform the following procedure. [root@oel65a dev]# fdisk /dev/sdb WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1305, default 1): Using default value 1 Last cylinder, +cylinders or +size{k,m,g} (1-1305, default 1305): Using default value 1305 Command (m for help): p Disk /dev/sdb: 10.7 GB, bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of * 512 = bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xeede9dba Device Boot Start End Blocks Id System /dev/sdb Linux

47 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@oel65a dev]# At the end you will have: [root@oel65a dev]# ls -l /dev/sd* brw-rw root disk 8, 0 May 12 16:37 /dev/sda brw-rw root disk 8, 1 May 12 16:37 /dev/sda1 brw-rw root disk 8, 2 May 12 16:37 /dev/sda2 brw-rw root disk 8, 16 May 12 16:46 /dev/sdb brw-rw root disk 8, 17 May 12 16:46 /dev/sdb1 brw-rw root disk 8, 32 May 12 16:46 /dev/sdc brw-rw root disk 8, 33 May 12 16:46 /dev/sdc1 brw-rw root disk 8, 48 May 12 16:46 /dev/sdd brw-rw root disk 8, 49 May 12 16:46 /dev/sdd1 brw-rw root disk 8, 64 May 12 16:47 /dev/sde brw-rw root disk 8, 65 May 12 16:47 /dev/sde1 brw-rw root disk 8, 80 May 12 16:47 /dev/sdf brw-rw root disk 8, 81 May 12 16:47 /dev/sdf1 brw-rw root disk 8, 96 May 12 16:47 /dev/sdg brw-rw root disk 8, 97 May 12 16:47 /dev/sdg1 brw-rw root disk 8, 112 May 12 16:48 /dev/sdh brw-rw root disk 8, 113 May 12 16:48 /dev/sdh1 brw-rw root disk 8, 128 May 12 16:48 /dev/sdi brw-rw root disk 8, 129 May 12 16:48 /dev/sdi1 brw-rw root disk 8, 144 May 12 16:48 /dev/sdj brw-rw root disk 8, 145 May 12 16:48 /dev/sdj1 brw-rw root disk 8, 160 May 12 16:49 /dev/sdk brw-rw root disk 8, 161 May 12 16:49 /dev/sdk1 brw-rw root disk 8, 176 May 12 16:49 /dev/sdl brw-rw root disk 8, 177 May 12 16:49 /dev/sdl1 [root@oel65a dev]# 2.15 Configure ASMlib and ASmlib disks by runnin the following commands in an ordered manner. [root@oel65a dev]# /usr/sbin/oracleasm configure -i Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: grid Default group to own the driver interface []: asmadmin Start Oracle ASM library driver on boot (y/n) [n]: y Scan for Oracle ASM disks on boot (y/n) [y]: y Writing Oracle ASM library driver configuration: done [root@oel65a dev]#

48 /usr/sbin/oracleasm init dev]# /usr/sbin/oracleasm init Creating /dev/oracleasm mount point: /dev/oracleasm Loading module "oracleasm": oracleasm Configuring "oracleasm" to use device physical block size Mounting ASMlib driver filesystem: /dev/oracleasm dev]# /usr/sbin/oracleasm createdisk DISK1 /dev/sdb1 /usr/sbin/oracleasm createdisk DISK2 /dev/sdc1 /usr/sbin/oracleasm createdisk DISK3 /dev/sdd1 /usr/sbin/oracleasm createdisk DISK4 /dev/sde1 /usr/sbin/oracleasm createdisk DISK5 /dev/sdf1 /usr/sbin/oracleasm createdisk DISK6 /dev/sdg1 /usr/sbin/oracleasm createdisk DISK7 /dev/sdh1 /usr/sbin/oracleasm createdisk DISK8 /dev/sdi1 /usr/sbin/oracleasm createdisk DISK9 /dev/sdj1 /usr/sbin/oracleasm createdisk DISK10 /dev/sdk1 /usr/sbin/oracleasm createdisk DISK11 /dev/sdl1 dev]# /usr/sbin/oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... dev]# /usr/sbin/oracleasm listdisks DISK1 DISK10 DISK11 DISK2 DISK3 DISK4 DISK5 DISK6 DISK7 DISK8 DISK9 dev]# 2.16 Setting the Disk I/O Scheduler on Linux Disk I/O schedulers reorder, delay, or merge requests for disk I/O to achieve better throughput and lower latency. Linux has multiple disk I/O schedulers available, including Deadline, Noop, Anticipatory, and Completely Fair Queuing (CFQ). For best performance for Oracle ASM, Oracle recommends that you use the Deadline I/O Scheduler. On each cluster node, enter the following command to ensure that the Deadline disk I/O scheduler is configured for use: # echo deadline > /sys/block/${asm_disk}/queue/scheduler echo deadline > /sys/block/sdb/queue/scheduler echo deadline > /sys/block/sdc/queue/scheduler echo deadline > /sys/block/sdd/queue/scheduler echo deadline > /sys/block/sde/queue/scheduler

49 echo deadline > /sys/block/sdf/queue/scheduler echo deadline > /sys/block/sdg/queue/scheduler echo deadline > /sys/block/sdh/queue/scheduler echo deadline > /sys/block/sdi/queue/scheduler echo deadline > /sys/block/sdj/queue/scheduler echo deadline > /sys/block/sdk/queue/scheduler echo deadline > /sys/block/sdl/queue/scheduler 2.17 Disable SElinux. Edit /etc/selinux/config and enable the line SELINUX=disabled 2.18 Disable iptables chkconfig iptables off 2.19 Install the rpm running rpm -Uvh cvuqdisk rpm 2.20 Set rp filtering by seting the rp_filter of the private address to 2 (loose filtering), the public address to 1 (strict filtering), using the following entries in /etc/sysctl.conf: net.ipv4.conf.eth2.rp_filter = 2 net.ipv4.conf.eth2.rp_filter = 2 net.ipv4.conf.eth1.rp_filter = 1 net.ipv4.conf.eth0.rp_filter = Shutdown the VM 2.22 This Concludes the OEL65A preparation for 12c installation. 3. Clone OEL61A to OEL61B. 3.1 Clone the OEL65a.vdi by issuing the following command. E:\vb>VBoxManage clonehd OEL65a.vdi OEL65b.vdi 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Clone hard disk created in format 'VDI'. UUID: 5b4a256c-d6c9-4b32-8d02- c5a1b10e3 1d5 E:\vb> 3.2 Create OEL65b VM and add the 4 NIC as in OEL65a 3.3 Attach the shared disks

50 VBoxManage storageattach OEL65b --storagectl "SATA" --port 1 --device 0 - -type hdd --medium asm1.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 2 --device 0 - -type hdd --medium asm2.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 3 --device 0 - -type hdd --medium asm3.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 4 --device 0 - -type hdd --medium asm4.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 5 --device 0 - -type hdd --medium asm5.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 6 --device 0 - -type hdd --medium asm6.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 7 --device 0 - -type hdd --medium asm7.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 8 --device 0 - -type hdd --medium asm8.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 9 --device 0 - -type hdd --medium asm9.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 10 --device 0 --type hdd --medium asm10.vdi --mtype shareable VBoxManage storageattach OEL65b --storagectl "SATA" --port 11 --device 0 --type hdd --medium asm11.vdi --mtype shareable 3.4 Startup the OEL65B VM 3.5 This concludes OEL65B clone 4. Change IP and hostname on OEL65B 4.1 Change the hostname in /etc/sysconfig/network [root@oel65a ~]# cat /etc/sysconfig/network NETWORKING=yes HOSTNAME=oel65b.gj.com GATEWAY= NTPSERVERARGS=iburst [root@oel65a ~]# 4.2 Change the IPs of the network interfaces Issue the following command #ifconfig a grep eth You will see eth4,eth5,eth6,eth7 I the etc/sysconfig/network-scripts you will have [root@oel65b network-scripts]# pwd /etc/sysconfig/network-scripts [root@oel65b network-scripts]# ls -l ifcfg-eth* -rw-r--r-- 1 root root 294 May 13 17:09 ifcfg-eth0

51 -rw-r--r-- 1 root root 295 May 13 11:58 ifcfg-eth1 -rw-r--r-- 1 root root 294 May 13 11:58 ifcfg-eth2 -rw-r--r-- 1 root root 294 May 13 17:10 ifcfg-eth3 [root@oel65b network-scripts]# Edit the ifcfg-eth0 and modify the following lines using the HWADDR from the first line of #ifconfig a grep eth. HWADDR=08:00:27:4F:66:B2 IPADDR= Edit the ifcfg-eth1 and modify the following lines using the HWADDR from the second line of #ifconfig a grep eth HWADDR=08:00:27:77:2B:DB IPADDR= Edit the ifcfg-eth2 and modify the following lines using the HWADDR from the third line of #ifconfig a grep eth HWADDR=08:00:27:BC:10:F5 IPADDR= Edit the ifcfg-eth3 and modify the following lines using the HWADDR from the fourth line of #ifconfig a grep eth HWADDR=08:00:27:BC:10:F5 HWADDR=08:00:27:BC:10:F5 4.3 Reboot the VM 4.4 This concludes the network reconfiguration. 5. Install GI 12c on oel65a and oel65b. 5.1 Create.bash_proile for user grid and user oracle on both oel65a and oel65b. Look at Annex A for the profiles. 5.2 Run ckuvfy to make sure that the prerequisites for GI installations are met. You will find the output in the Annex A../runcluvfy.sh stage -pre crsinst -n oel65a,oel65b./runcluvfy.sh stage -post hwos -n oel65a,oel65b 5.3 runinstall from the grid 12c stage area and configure ssh.

52 5.4 Run OUI from the grid 12c stage area Select Skip and press Next. Select install and configure GI for a cluster.

53 Select Configure a standard Cluster

54 Select a Typical Installation

55 Enter the scan name and the public and virtual hostnames.

56 Select the interfaces and what they are used for.

57 Enter the software locations and ASM passwords.

58 Specify the pattern to use to discover the disks

59 Specify the 10g disk for a high redundancy disk group.

60 Note that the disks are as follows

61 Specify the inventory

62 Select an option to manually run the root scripts.

63 Wait for the checks to complete. Review and select Ignore All and press Next.

64 The errors are as follows PRVF-9802 : Attempt to get udev information from node "oel65a" failed PRVF-9802 : Attempt to get udev information from node "oel65b" failed PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b The only note INS PRVF-9802 PRVF-5184 PRVF-5186 After Successful Upgrade to 11gR2 Grid Infrastructure (Doc ID ) suggest that the disk should be managed by UDEV. I wonder why ASMlib does not do the job. As I am running it using Oracle VM Virtual Box I will ignore the error. Note : PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes (Doc ID ) addresses the PRVF-5636 problem. I will ignore it for the same reason as stated above. Review and press Install.

65 Wait until prompted to execute scripts as root.

66 Execute the scripts as root.

67 ~]# /u01/app/orainventory/orainstroot.sh Changing permissions of /u01/app/orainventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/orainventory to oinstall. The execution of the script is complete. ~]# ~]# /u01/app/orainventory/orainstroot.sh Changing permissions of /u01/app/orainventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/orainventory to oinstall. The execution of the script is complete. ~]# ~]# /u01/app/ /grid/root.sh Performing root user operation for Oracle 12c The following environment variables are set as:

68 ORACLE_OWNER= grid ORACLE_HOME= /u01/app/ /grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin... Copying oraenv to /usr/local/bin... Copying coraenv to /usr/local/bin... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/ /grid/crs/install/crsconfig_params 2014/05/14 15:09:12 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert 2014/05/14 15:10:27 CLSRSC-330: Adding Clusterware entries to file 'oracleohasd.conf' CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-2672: Attempting to start 'ora.evmd' on 'oel65a' CRS-2672: Attempting to start 'ora.mdnsd' on 'oel65a' CRS-2676: Start of 'ora.evmd' on 'oel65a' succeeded CRS-2676: Start of 'ora.mdnsd' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'oel65a' CRS-2676: Start of 'ora.gpnpd' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'oel65a' CRS-2672: Attempting to start 'ora.gipcd' on 'oel65a' CRS-2676: Start of 'ora.cssdmonitor' on 'oel65a' succeeded CRS-2676: Start of 'ora.gipcd' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'oel65a' CRS-2672: Attempting to start 'ora.diskmon' on 'oel65a' CRS-2676: Start of 'ora.diskmon' on 'oel65a' succeeded CRS-2676: Start of 'ora.cssd' on 'oel65a' succeeded ASM created and started successfully.

69 Disk Group DATA created successfully. CRS-2672: Attempting to start 'ora.storage' on 'oel65a' CRS-2676: Start of 'ora.storage' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'oel65a' CRS-2676: Start of 'ora.crsd' on 'oel65a' succeeded CRS-4256: Updating the profile Successful addition of voting disk 9d8b11d713204f34bf32e2445c8c751d. Successful addition of voting disk f791aee91e474f58bfae1efacb87c4ce. Successful addition of voting disk 4f5bcd1ebae74f1dbf5a7edcb2c86bef. Successful addition of voting disk d329f f08bfc6b265f4007b98. Successful addition of voting disk fdeda4f2ebf334a123938d781. Successfully replaced voting disk group with +DATA. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk grou p ONLINE 9d8b11d713204f34bf32e2445c8c751d (/dev/oracleasm/disks/disk1) [DATA] 2. ONLINE f791aee91e474f58bfae1efacb87c4ce (/dev/oracleasm/disks/disk2) [DATA] 3. ONLINE 4f5bcd1ebae74f1dbf5a7edcb2c86bef (/dev/oracleasm/disks/disk3) [DATA] 4. ONLINE d329f f08bfc6b265f4007b98 (/dev/oracleasm/disks/disk4) [DATA] 5. ONLINE fdeda4f2ebf334a123938d781 (/dev/oracleasm/disks/disk5) [DATA] Located 5 voting disk(s). CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'oel65a' CRS-2673: Attempting to stop 'ora.crsd' on 'oel65a' CRS-2677: Stop of 'ora.crsd' on 'oel65a' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'oel65a' CRS-2673: Attempting to stop 'ora.evmd' on 'oel65a' CRS-2673: Attempting to stop 'ora.storage' on 'oel65a' CRS-2673: Attempting to stop 'ora.mdnsd' on 'oel65a' CRS-2673: Attempting to stop 'ora.gpnpd' on 'oel65a' CRS-2677: Stop of 'ora.storage' on 'oel65a' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'oel65a' CRS-2677: Stop of 'ora.mdnsd' on 'oel65a' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'oel65a' succeeded CRS-2677: Stop of 'ora.evmd' on 'oel65a' succeeded CRS-2677: Stop of 'ora.ctssd' on 'oel65a' succeeded CRS-2677: Stop of 'ora.asm' on 'oel65a' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'oel65a' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'oel65a' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'oel65a' CRS-2677: Stop of 'ora.cssd' on 'oel65a' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'oel65a' CRS-2677: Stop of 'ora.gipcd' on 'oel65a' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'oel65a' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'oel65a' CRS-2672: Attempting to start 'ora.evmd' on 'oel65a' CRS-2676: Start of 'ora.evmd' on 'oel65a' succeeded CRS-2676: Start of 'ora.mdnsd' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'oel65a' CRS-2676: Start of 'ora.gpnpd' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'oel65a' CRS-2676: Start of 'ora.gipcd' on 'oel65a' succeeded

70 CRS-2672: Attempting to start 'ora.cssdmonitor' on 'oel65a' CRS-2676: Start of 'ora.cssdmonitor' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'oel65a' CRS-2672: Attempting to start 'ora.diskmon' on 'oel65a' CRS-2676: Start of 'ora.diskmon' on 'oel65a' succeeded CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'oel65a' CRS-2676: Start of 'ora.cssd' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'oel65a' CRS-2672: Attempting to start 'ora.ctssd' on 'oel65a' CRS-2676: Start of 'ora.ctssd' on 'oel65a' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.asm' on 'oel65a' CRS-2676: Start of 'ora.asm' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.storage' on 'oel65a' CRS-2676: Start of 'ora.storage' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'oel65a' CRS-2676: Start of 'ora.crsd' on 'oel65a' succeeded CRS-6023: Starting Oracle Cluster Ready Services-managed resources CRS-6017: Processing resource auto-start for servers: oel65a CRS-6016: Resource auto-start has completed for server oel65a CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2014/05/14 15:17:53 CLSRSC-343: Successfully started Oracle clusterware stack CRS-2672: Attempting to start 'ora.asm' on 'oel65a' CRS-2676: Start of 'ora.asm' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.data.dg' on 'oel65a' CRS-2676: Start of 'ora.data.dg' on 'oel65a' succeeded 2014/05/14 15:19:58 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster... succeeded You have new mail in /var/spool/mail/root [root@oel65a ~]# [root@oel65b grid]# pwd /u01/app/ /grid [root@oel65b grid]# /u01/app/ /grid/root.sh Performing root user operation for Oracle 12c The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/ /grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin... Copying oraenv to /usr/local/bin... Copying coraenv to /usr/local/bin... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/ /grid/crs/install/crsconfig_params 2014/05/14 15:20:54 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful 2014/05/14 15:21:21 CLSRSC-330: Adding Clusterware entries to file 'oracleohasd.conf'

71 CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'oel65b' CRS-2672: Attempting to start 'ora.evmd' on 'oel65b' CRS-2676: Start of 'ora.mdnsd' on 'oel65b' succeeded CRS-2676: Start of 'ora.evmd' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'oel65b' CRS-2676: Start of 'ora.gpnpd' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'oel65b' CRS-2676: Start of 'ora.gipcd' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'oel65b' CRS-2676: Start of 'ora.cssdmonitor' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'oel65b' CRS-2672: Attempting to start 'ora.diskmon' on 'oel65b' CRS-2676: Start of 'ora.diskmon' on 'oel65b' succeeded CRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'oel65b' CRS-2676: Start of 'ora.cssd' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'oel65b' CRS-2672: Attempting to start 'ora.ctssd' on 'oel65b' CRS-2676: Start of 'ora.ctssd' on 'oel65b' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.asm' on 'oel65b' CRS-2676: Start of 'ora.asm' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.storage' on 'oel65b' CRS-2676: Start of 'ora.storage' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'oel65b' CRS-2676: Start of 'ora.crsd' on 'oel65b' succeeded CRS-6017: Processing resource auto-start for servers: oel65b CRS-2673: Attempting to stop 'ora.listener_scan1.lsnr' on 'oel65a' CRS-2672: Attempting to start 'ora.ons' on 'oel65b' CRS-2677: Stop of 'ora.listener_scan1.lsnr' on 'oel65a' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'oel65a' CRS-2677: Stop of 'ora.scan1.vip' on 'oel65a' succeeded CRS-2672: Attempting to start 'ora.scan1.vip' on 'oel65b' CRS-2676: Start of 'ora.scan1.vip' on 'oel65b' succeeded CRS-2672: Attempting to start 'ora.listener_scan1.lsnr' on 'oel65b' CRS-2676: Start of 'ora.ons' on 'oel65b' succeeded CRS-2676: Start of 'ora.listener_scan1.lsnr' on 'oel65b' succeeded CRS-6016: Resource auto-start has completed for server oel65b CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2014/05/14 15:26:57 CLSRSC-343: Successfully started Oracle clusterware stack 2014/05/14 15:27:21 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster... succeeded [root@oel65b grid]#

72 The errors can be ignored. The errors are listed below logs]# cat installactions _ pm.log grep ERROR INFO: INFO: ERROR: [Result.addErrorDescription:607] PRVF-7530 : Sufficient physical memory is not available on node "oel65b" [Required physical memory = 4GB ( KB)] INFO: INFO: ERROR: [Result.addErrorDescription:607] PRVF-7530 : Sufficient physical memory is not available on node "oel65a" [Required physical memory = 4GB ( KB)] INFO: ERRORMSG(oel65b): PRVF-7530 : Sufficient physical memory is not available on node "oel65b" [Required physical memory = 4GB ( KB)] INFO: ERRORMSG(oel65a): PRVF-7530 : Sufficient physical memory is not available on node "oel65a" [Required physical memory = 4GB ( KB)] INFO: INFO: ERROR: [Result.addErrorDescription:607] PRVF-9802 : Attempt to get udev information from node "oel65a" failed INFO: INFO: ERROR: [Result.addErrorDescription:607] PRVF-9802 : Attempt to get udev information from node "oel65b" failed INFO: INFO: ERROR: [Result.addErrorDescription:618] PRVF-9802 : Attempt to get udev information from node "oel65b" failed INFO: INFO: ERROR: [Result.addErrorDescription:618] PRVF-9802 : Attempt to get udev information from node "oel65a" failed INFO: INFO: ERROR: [Result.addErrorDescription:618] PRVF-9802 : Attempt to get udev information from node "oel65b" failed INFO: INFO: ERROR: [Result.addErrorDescription:618] PRVF-9802 : Attempt to get udev information from node "oel65a" failed

73 INFO: ERRORMSG(oel65b): PRVF-9802 : Attempt to get udev information from node "oel65b" failed INFO: ERRORMSG(oel65a): PRVF-9802 : Attempt to get udev information from node "oel65a" failed INFO: INFO: ERROR: [Result.addErrorDescription:607] PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b INFO: INFO: ERROR: [Result.addErrorDescription:607] PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b INFO: ERRORMSG(oel65b): PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b INFO: ERRORMSG(oel65a): PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b INFO: INFO: ERROR: [Result.addErrorDescription:618] PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b INFO: INFO: ERROR: [Result.addErrorDescription:618] PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b INFO: INFO: ERROR: [Result.addErrorDescription:607] Check for integrity of file "/etc/resolv.conf" failed INFO: INFO: ERROR: [Result.addErrorDescription:607] Check for integrity of file "/etc/resolv.conf" failed INFO: ERRORMSG(oel65b): PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b INFO: ERRORMSG(oel65b): Check for integrity of file "/etc/resolv.conf" failed INFO: ERRORMSG(oel65a): PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b INFO: ERRORMSG(oel65a): Check for integrity of file "/etc/resolv.conf" failed INFO: ERROR: INFO: ERROR: INFO: ERROR: INFO: ERROR: INFO: ERROR: [root@oel65a logs]# INFO: ERROR: INFO: PRVG-1101 : SCAN name "scan" failed to resolve INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "scan" (IP address: ) failed INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "scan" (IP address: ) failed INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "scan" (IP address: ) failed INFO: ERROR: INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "scan" INFO: Checking SCAN IP addresses...

74 5.5 Verify that GI is installed properly bin]#./crsctl check cluster -all ************************************************************** oel65a: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** oel65b: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** bin]# bin]#./crsctl status resource -t Name Target State Server State details

75 Local Resources ora.data.dg ONLINE ONLINE oel65a STABLE ONLINE ONLINE oel65b STABLE ora.listener.lsnr ONLINE ONLINE oel65a STABLE ONLINE ONLINE oel65b STABLE ora.asm ONLINE ONLINE oel65a STABLE ONLINE ONLINE oel65b Started,STABLE ora.net1.network ONLINE ONLINE oel65a STABLE ONLINE ONLINE oel65b STABLE ora.ons ONLINE ONLINE oel65a STABLE ONLINE ONLINE oel65b STABLE Cluster Resources ora.listener_scan1.lsnr 1 ONLINE ONLINE oel65b STABLE ora.listener_scan2.lsnr 1 ONLINE ONLINE oel65a STABLE ora.listener_scan3.lsnr 1 ONLINE ONLINE oel65a STABLE ora.cvu 1 ONLINE ONLINE oel65a STABLE ora.oc4j 1 OFFLINE OFFLINE STABLE ora.oel65a.vip 1 ONLINE ONLINE oel65a STABLE ora.oel65b.vip 1 ONLINE ONLINE oel65b STABLE ora.scan1.vip 1 ONLINE ONLINE oel65b STABLE ora.scan2.vip 1 ONLINE ONLINE oel65a STABLE ora.scan3.vip 1 ONLINE ONLINE oel65a STABLE [root@oel65a bin]# 5.6 This concludes the GI installation. 6. Install RAC RDBMS 12c on oel65a and oel65b 6.1 Login as oracle and run the OUI to install Oracle 12c binaries only. Skip security updates.

76 Skip software updates.

77 Select Install software only.

78 Select RAC database.

79 Select all nodes.

80 Select languages.

81 Select enterprise Edition.

82 Verify the locations.

83 Verify the OS groups.

84 Wait for the check to complete. The errors are listed below. I can ignore them. Single Client Access Name (SCAN) - This test verifies the Single Client Access Name configuration. Error: - PRVG-1101 : SCAN name "scan" failed to resolve - Cause: An attempt to resolve specified SCAN name to a list of IP addresses failed because SCAN could not be resolved in DNS or GNS using 'nslookup'. - Action: Check whether the specified SCAN name is correct. If SCAN name should be resolved in DNS, check the configuration of SCAN name in DNS. If it should be resolved in GNS make sure that GNS resource is online. - PRVF-4657 : Name resolution setup check for "scan" (IP address: ) failed - Cause: Inconsistent IP address definitions found for the SCAN name identified using DNS and configured name resolution mechanism(s). - Action: Look up the SCAN name with nslookup, and make sure the returned IP addresses are consistent with those defined in NIS and /etc/hosts as configured in /etc/nsswitch.conf by reconfiguring the latter. Check the Name Service Cache Daemon (/usr/sbin/nscd) by clearing its cache and restarting it. -

85 PRVF-4657 : Name resolution setup check for "scan" (IP address: ) failed - Cause: Inconsistent IP address definitions found for the SCAN name identified using DNS and configured name resolution mechanism(s). - Action: Look up the SCAN name with nslookup, and make sure the returned IP addresses are consistent with those defined in NIS and /etc/hosts as configured in /etc/nsswitch.conf by reconfiguring the latter. Check the Name Service Cache Daemon (/usr/sbin/nscd) by clearing its cache and restarting it. - PRVF-4657 : Name resolution setup check for "scan" (IP address: ) failed - Cause: Inconsistent IP address definitions found for the SCAN name identified using DNS and configured name resolution mechanism(s). - Action: Look up the SCAN name with nslookup, and make sure the returned IP addresses are consistent with those defined in NIS and /etc/hosts as configured in /etc/nsswitch.conf by reconfiguring the latter. Check the Name Service Cache Daemon (/usr/sbin/nscd) by clearing its cache and restarting it. Check Failed on Nodes: [oel65b, oel65a] Verification result of failed node: oel65b Details: - PRVF-4664 : Found inconsistent name resolution entries for SCAN name "scan" - Cause: The nslookup utility and the configured name resolution mechanism(s), as defined in /etc/nsswitch.conf, returned inconsistent IP address information for the SCAN name identified. - Action: Check the Name Service Cache Daemon (/usr/sbin/nscd), the Domain Name Server (nslookup) and the /etc/hosts file to make sure the IP address for the SCAN names are registered correctly. Back to Top Verification result of failed node: oel65a Details: - PRVF-4664 : Found inconsistent name resolution entries for SCAN name "scan" - Cause: The nslookup utility and the configured name resolution mechanism(s), as defined in /etc/nsswitch.conf, returned inconsistent IP address information for the SCAN name identified. - Action: Check the Name Service Cache Daemon (/usr/sbin/nscd), the Domain Name Server (nslookup) and the /etc/hosts file to make sure the IP address fo r the SCAN names are registered correctly. Back to Top Task resolv.conf Integrity - This task checks consistency of file /etc/resolv.conf file across nodes Check Failed on Nodes: [oel65b, oel65a] Verification result of failed node: oel65b Details: - PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b - Cause: The DNS response time for an unreachable node exceeded the value specified on nodes specified. - Action: Make sure that 'options timeout', 'options attempts' and 'nameserver' entries in file resolv.conf are proper. On HPUX these entries will be 'retrans', 'retry' and 'nameserver'. On Solaris these will be 'options retrans', 'options retry' and 'nameserver'. Make sure that the DNS server responds back to name lookup request within the specified time when looking up an unknown host name. - Check for integrity of file "/etc/resolv.conf" failed - Cause: Cause Of Problem Not Available - Action: User Action Not Available Back to Top Verification result of failed node: oel65a Details: -

86 PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: oel65a,oel65b - Cause: The DNS response time for an unreachable node exceeded the value specified on nodes specified. - Action: Make sure that 'options timeout', 'options attempts' and 'nameserver' entries in file resolv.conf are proper. On HPUX these entries will be 'retrans', 'retry' and 'nameserver'. On Solaris these will be 'options retrans', 'options retry' and 'nameserver'. Make sure that the DNS server responds back to name lookup request within the specified time when looking up an unknown host name. - Check for integrity of file "/etc/resolv.conf" failed - Cause: Cause Of Problem Not Available - Action: User Action Not Available Back to Top Review and start the installation. Wait until prompted to run scripts as root.

87

88 bin]# /u01/app/oracle/product/12.1.0/db_1/root.sh Performing root user operation for Oracle 12c The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/12.1.0/db_1 Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. [root@oel65a bin]# [root@oel65b grid]# /u01/app/oracle/product/12.1.0/db_1/root.sh Performing root user operation for Oracle 12c The following environment variables are set as:

89 ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/12.1.0/db_1 Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. [root@oel65b grid]# Exit 6.2 This concludes the 12c binaries installation. 7. Create a Multitenant Container Database (DB) database RACDB oel65a and oel65b 7.1 Login as grid and start asmca to create FRA disk group

90 Click create and select disks for the FRA disk group. Press OK to create the group.

91 Wait until confirmation.

92 Now you have the disk groups

93 7.2 Login as oracle and start dbca. Select create a database.

94 Select Advanced Mode

95 Select options that interest you.

96 Select a CDB option.

97 Select all nodes.

98 Select the crosschecked options

99 Select a password.

100 Specify the storage.

101 Skip the Vault configurations.

102 Configure AMM

103 Configure character sets.

104 Select what to create.

105 Wait for the prerequisite checks to complete.

106 Review the components to be installed.

107 Wait for the creation to complete.

108

109 You can change the passwords. Note the URL

110 Login to the URL

111 Examine what is inside.

112 7.3 This concludes the CDB creation. 8. Verify database creation and create a service 8.1 Perform Verifications [root@oel65a bin]#./crsctl status resource -t Name Target State Server State details Local Resources ora.data.dg ONLINE ONLINE oel65a STABLE ONLINE ONLINE oel65b STABLE ora.fra.dg ONLINE ONLINE oel65a STABLE ONLINE ONLINE oel65b STABLE ora.listener.lsnr ONLINE ONLINE oel65a STABLE