raw]$ uname -a Linux laguna el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux

Size: px

Start display at page:

Download "raw]$ uname -a Linux laguna el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux"

Milo Barton
6 years ago
Views:

1 PART 1 of 4 - GRID INFRASTRUCTURE INSTALLATION Environment details: Red Hat Enterprise Linux Server release 6.4 (Santiago) Oracle PSU4 Preinstallation Steps ## Information provided by OS admin: [root@laguna2:/u01/orainst]# nslookup laguna-scan.turkcell.tgc Server: Address: #53 Name: laguna-scan.turkcell.tgc Address: Name: laguna-scan.turkcell.tgc Address: Name: laguna-scan.turkcell.tgc Address: [root@laguna2:~]# oracleasm listdisks CONF1 CONF2 CONF3 CONF4 CONF5 PROD1 PROD2 PROD3 Laguna1.turkcell.tgc Root pass: ## OS version [grid@laguna1 raw]$ uname -a Linux laguna el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux [grid@laguna1 raw]$ cat /etc/*-release LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics- 4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Red Hat Enterprise Linux Server release 6.4 (Santiago) root:oinstall /u01

2 root:oinstall /u01/app oracle ORACLE_BASE oracle:oinstall /u01/app/oracle oracle ORACLE_HOME oracle:oinstall /u01/app/oracle/product/11.2.0/dbhome_1 grid ORACLE_HOME root:oinstall /u01/app/11.2.0/grid grid $ORACLE_BASE grid:oinstall /u01/app/grid #### Creating required directories #### mkdir /u01/app chown root:oinstall /u01/app ## Create grid OB directory on both nodes. mkdir /u01/app/grid chown grid:oinstall /u01/app/grid ## Create grid OH directory on both nodes. mkdir -p /u01/app/11.2.0/grid chown -R grid:oinstall /u01/app/11.2.0/grid ## Create oracle OB directory on both nodes. mkdir /u01/app/oracle chown oracle:oinstall /u01/app/oracle ## Create oracle OH directory on both nodes. mkdir -p /u01/app/oracle/product/11.2.0/dbhome_1 chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/dbhome_1 ## additioal rights &ownership chmod -R 775 /u01 chown grid:oinstall /u01 mkdir /u01/app chown root:oinstall /u01/app mkdir /u01/app/oracle chown oracle:oinstall /u01/app/oracle mkdir -p /u01/app/oracle/product/11.2.0/dbhome_1 chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/dbhome_1 mkdir -p /u01/app/11.2.0/grid chown -R grid:oinstall /u01/app/11.2.0/grid mkdir /u01/app/grid chown grid:oinstall /u01/app/grid chmod -R 775 /u01/ chown grid:oinstall /u01

3 ## Continue from below installation documentation. ##Oracle Grid Infrastructure Installation Guide 11g Release 2 (11.2) for Linux E ## Manual preinstallation checks, on both nodes. #Verify System Requirements grep MemTotal /proc/meminfo grep SwapTotal /proc/meminfo [root@laguna1:/u01/app]# grep MemTotal /proc/meminfo MemTotal: kb [root@laguna1:/u01/app]# [root@laguna1:/u01/app]# grep SwapTotal /proc/meminfo SwapTotal: kb [root@laguna2:/u01/orainst]# grep MemTotal /proc/meminfo MemTotal: kb [root@laguna2:/u01/orainst]# grep SwapTotal /proc/meminfo SwapTotal: kb [root@laguna2:/u01/orainst]# # Ensure you have at least 6.5 GB of space for the Oracle Grid Infrastructure /u01 disk # Ensure that you have at least 1 GB of space in /tmp [root@laguna1:/u01/app]# df -h /tmp Filesystem Size Used Avail Use% Mounted on /dev/mapper/vgroot-lvroot 20G 4.1G 15G 22% / [root@laguna2:/u01/orainst]# df -h /tmp Filesystem Size Used Avail Use% Mounted on /dev/mapper/vgroot-lvroot 20G 4.1G 15G 22% / [root@laguna1:/u01/app]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vgroot-lvroot 20G 4.1G 15G 22% / tmpfs 64G 0 64G 0% /dev/shm /dev/sda1 485M 87M 373M 19% /boot /dev/mapper/oravg-u01lv 99G 7.0G 87G 8% /u01 [root@laguna2:/u01/orainst]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vgroot-lvroot 20G 4.1G 15G 22% / tmpfs 64G 0 64G 0% /dev/shm

4 /dev/sda1 485M 87M 373M 19% /boot /dev/mapper/oravg-u01lv 99G 5.9G 88G 7% /u01 ## 2.Advanced Installation Oracle Grid Infrastructure for a Cluster Preinstallation Tasks Navigate to /u01/orainst/grid, unzipped GI installation files folder. [grid@laguna1 grid]$ /u01/orainst/grid/runcluvfy.sh stage -pre crsinst -n laguna1,laguna2 -fixup -verbose /u01/orainst/cvu/bin/cluvfy stage -pre crsinst -n laguna1,laguna2 -fixup -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "laguna1" Destination Node Reachable? laguna1 yes laguna2 yes Result: Node reachability check passed from node "laguna1" Checking user equivalence... Check: User equivalence for user "grid" Node Name Status laguna2 failed laguna1 failed Result: PRVF-4007 : User equivalence check failed for user "grid" ERROR: User equivalence unavailable on all the specified nodes Verification cannot proceed Pre-check for cluster services setup was unsuccessful on all the nodes. [grid@laguna1 grid]$ id grid uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1020(asmadmin),1021(asmdba),1022(asmoper) [grid@laguna2 grid]$ id grid uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1020(asmadmin),1021(asmdba),1022(asmoper) ==> SSH connectivity issue ### setup grid passwordless ssh connection for each node. example below is for grid@laguna1 to grid@laguna2. ### [grid@laguna1 grid]$ cd ~ [grid@laguna1 ~]$ cd.ssh [grid@laguna1.ssh]$ ls -ltr total 0 [grid@laguna1.ssh]$

5 ls -las total 8 4 drwx grid oinstall 4096 May 8 14:30. 4 drwx grid oinstall 4096 May 8 14:40.. [grid@laguna1.ssh]$ [grid@laguna1.ssh]$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/grid/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/grid/.ssh/id_rsa. Your public key has been saved in /home/grid/.ssh/id_rsa.pub. The key fingerprint is: 95:9f:af:20:5e:ef:28:22:bf:42:dc:b0:91:1b:c7:33 grid@laguna1 The key's randomart image is: +--[ RSA 2048] o o = E.... O os o =.... o. o...o +. +oo [grid@laguna1.ssh]$ ls -las total 16 4 drwx grid oinstall 4096 May 8 14:54. 4 drwx grid oinstall 4096 May 8 14: rw grid oinstall 1675 May 8 14:54 id_rsa 4 -rw-r--r-- 1 grid oinstall 394 May 8 14:54 id_rsa.pub [grid@laguna1.ssh]$ ssh ssh ssh-add ssh-agent ssh-copy-id sshd ssh-keygen sshkeyscan [grid@laguna1.ssh]$ ssh-copy-id Usage: /usr/bin/ssh-copy-id [-i [identity_file]] [user@]machine [grid@laguna1.ssh]$ ssh-copy-id -i id_rsa.pub grid@laguna2.turkcell.tgc The authenticity of host 'laguna2.turkcell.tgc ( )' can't be established. RSA key fingerprint is e5:c8:18:3f:af:7b:a2:85:d0:25:21:da:f2:03:81:e2. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'laguna2.turkcell.tgc, ' (RSA) to the list of known hosts... grid@laguna2.turkcell.tgc's password: Now try logging into the machine, with "ssh 'grid@laguna2.turkcell.tgc'", and check in:.ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. [grid@laguna1.ssh]$ ssh grid@laguna2 The authenticity of host 'laguna2 ( )' can't be established. RSA key fingerprint is e5:c8:18:3f:af:7b:a2:85:d0:25:21:da:f2:03:81:e2. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'laguna2' (RSA) to the list of known hosts.

6 .. ~]$ grid]$ export DISPLAY= :0.0 grid]$ ### setup oracle passwordless ssh connection for each node. example below is for to ### ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/oracle/.ssh/id_rsa): /home/oracle/.ssh/id_rsa already exists. Overwrite (y/n)? y Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_rsa. Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. The key fingerprint is: d8:31:d3:34:07:f5:58:3e:9e:5d:c4:99:cb:46:fc:bf oracle@laguna2 The key's randomart image is: +--[ RSA 2048] oo o.+ o o = *. +.. * + o +. Bo. S + o.. E [oracle@laguna2.ssh]$ [oracle@laguna2.ssh]$ ssh-copy-id -i id_rsa.pub oracle@laguna1.turkcell.tgc The authenticity of host 'laguna1.turkcell.tgc ( )' can't be established. RSA key fingerprint is cf:28:91:95:74:1c:4e:eb:80:5f:02:22:a9:26:55:1a. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'laguna1.turkcell.tgc, ' (RSA) to the list of known hosts. oracle@laguna1.turkcell.tgc's password: Now try logging into the machine, with "ssh 'oracle@laguna1.turkcell.tgc'", and check in:.ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. [oracle@laguna2.ssh]$ ssh laguna1 The authenticity of host 'laguna1 ( )' can't be established. RSA key fingerprint is cf:28:91:95:74:1c:4e:eb:80:5f:02:22:a9:26:55:1a. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'laguna1' (RSA) to the list of known hosts. [oracle@laguna1 ~]$

7 ssh-copy-id -i id_rsa.pub ### setup root passwordless ssh connection for each node. example below is for to ### ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 06:f5:74:89:2b:6b:7e:3e:a9:54:4f:1f:15:e7:62:e9 The key's randomart image is: +--[ RSA 2048] o... oo... + o... o o So.. E.o. o.. o [root@laguna2:~]# [root@laguna2:~]# cd.ssh [root@laguna2:~/.ssh]# ls -las total 20 4 drwx root root 4096 May 21 13:50. 4 dr-xr-x root root 4096 May 21 00: rw root root 1675 May 21 13:50 id_rsa 4 -rw-r--r-- 1 root root 394 May 21 13:50 id_rsa.pub 4 -rw-r--r-- 1 root root 629 May 8 17:19 known_hosts [root@laguna2:~/.ssh]# [root@laguna2:~/.ssh]# ssh-copy-id -i id_rsa.pub root@laguna1.turkcell.tgc The authenticity of host 'laguna1.turkcell.tgc ( )' can't be established. RSA key fingerprint is cf:28:91:95:74:1c:4e:eb:80:5f:02:22:a9:26:55:1a. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'laguna1.turkcell.tgc' (RSA) to the list of known hosts. root@laguna1.turkcell.tgc's password: Now try logging into the machine, with "ssh 'root@laguna1.turkcell.tgc'", and check in:.ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting. [root@laguna2:~/.ssh]#

8 ### 2.8 additional manual package check ### rpm -qa grep elfutils-libelf-devel rpm -qa grep elfutils-libelf-devel rpm -qa grep gcc rpm -qa grep gcc-c++ rpm -qa grep glibc rpm -qa grep glibc-common rpm -qa grep glibc-devel rpm -qa grep glibc-headers rpm -qa grep kernel-headers rpm -qa grep ksh rpm -qa grep libaio rpm -qa grep libaio-devel rpm -qa grep libgcc rpm -qa grep libgomp rpm -qa grep libstdc++ rpm -qa grep libstdc++-devel rpm -qa grep make rpm -qa grep sysstat rpm -qa grep unixodbc rpm -qa grep unixodbc-devel rpm -qa grep glibc-common glibc-common el6.x86_64 rpm -qa grep glibc-devel glibc-devel el6.x86_64 rpm -qa grep glibc-headers glibc-headers el6.x86_64 compat-glibc-headers x86_64 rpm -qa grep kernel-headers kernel-headers el6.x86_64 rpm -qa grep ksh rpm -qa grep ksh mksh-39-7.el6.x86_64 ksh el6.x86_64 rpm -qa grep libaio libaio-devel el6.x86_64 libaio el6.x86_64 rpm -qa grep libaio-devel libaio-devel el6.x86_64 rpm -qa grep libgcc libgcc el6.i686 libgcc el6.x86_64 rpm -qa grep libgomp libgomp el6.x86_64 rpm -qa grep libstdc++ compat-libstdc el6.i686 libstdc++-devel el6.x86_64 compat-libstdc el6.x86_64 libstdc el6.x86_64 libstdc el6.i686 rpm -qa grep libstdc++-devel libstdc++-devel el6.x86_64 rpm -qa grep make makebootfat el6.x86_64 make el6.x86_64 rpm -qa grep sysstat sysstat el6.x86_64 rpm -qa grep unixodbc unixodbc-devel el6_3.x86_64

9 unixodbc el6_3.x86_64 rpm -qa grep unixodbc-devel unixodbc-devel el6_3.x86_64 ## Verifying UDP and TCP Kernel Parameters cat /proc/sys/net/ipv4/ip_local_port_range cat /proc/sys/net/ipv4/ip_local_port_range ## Check Resource Limits for the Oracle Software Installation Users ## grid]$ ulimit -Sn grid]$ ulimit -Hn grid]$ ulimit -Su 8192 grid]$ ulimit -Hu grid]$ ulimit -Ss grid]$ ulimit -Hs unlimited grid]$ oracleasm listdisks CONF1 CONF2 CONF3 CONF4 CONF5 PROD1 PROD2 PROD3 PROD4 oracleasm status Checking if ASM is loaded: yes Checking if /dev/oracleasm is mounted: yes grid]$./runinstaller Starting Oracle Universal Installer... Checking Temp space: must be greater than 120 MB. Actual MB Passed Checking swap space: must be greater than 150 MB. Actual 4095 MB Passed Checking monitor: must be configured to display at least 256 colors. Actual Passed Preparing to launch Oracle Universal Installer from /tmp/orainstall _ pm. Please grid]$

10 Installation Steps

15 Admin123

[root@laguna1:~]# /u01/app/orainventory/orainstroot.sh Changing permissions of /u01/app/orainventory. Adding read,write permissions for group. Removing read,write,execute permissions for world.

24 /u01/app/orainventory/orainstroot.sh Changing permissions of /u01/app/orainventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/orainventory to oinstall. The execution of the script is complete. /u01/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin... Copying oraenv to /usr/local/bin... Copying coraenv to /usr/local/bin... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory

25 User ignored Prerequisites during installation OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert Adding Clusterware entries to upstart CRS-2672: Attempting to start 'ora.mdnsd' on 'laguna1' CRS-2676: Start of 'ora.mdnsd' on 'laguna1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'laguna1' CRS-2676: Start of 'ora.gpnpd' on 'laguna1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'laguna1' CRS-2672: Attempting to start 'ora.gipcd' on 'laguna1' CRS-2676: Start of 'ora.cssdmonitor' on 'laguna1' succeeded CRS-2676: Start of 'ora.gipcd' on 'laguna1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'laguna1' CRS-2672: Attempting to start 'ora.diskmon' on 'laguna1' CRS-2676: Start of 'ora.diskmon' on 'laguna1' succeeded CRS-2676: Start of 'ora.cssd' on 'laguna1' succeeded ASM created and started successfully. Disk Group CONFIG created successfully. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. CRS-4256: Updating the profile Successful addition of voting disk 6dae7976ff034f6abf23a9eeb07866c5. Successful addition of voting disk 1d807080e86c4fe2bf0acf6d2f3dc3fe. Successful addition of voting disk 453f08f20d4b4f92bff4d6bcf979b655. Successful addition of voting disk 4fe8507cb22f4ff6bf2a6934eb56a330. Successful addition of voting disk 832af fd0bf2e8cf4a865b9b7. Successfully replaced voting disk group with +CONFIG. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group ONLINE 6dae7976ff034f6abf23a9eeb07866c5 (ORCL:CONF1) [CONFIG] 2. ONLINE 1d807080e86c4fe2bf0acf6d2f3dc3fe (ORCL:CONF2) [CONFIG] 3. ONLINE 453f08f20d4b4f92bff4d6bcf979b655 (ORCL:CONF3) [CONFIG]

26 4. ONLINE 4fe8507cb22f4ff6bf2a6934eb56a330 (ORCL:CONF4) [CONFIG] 5. ONLINE 832af fd0bf2e8cf4a865b9b7 (ORCL:CONF5) [CONFIG] Located 5 voting disk(s). CRS-2672: Attempting to start 'ora.asm' on 'laguna1' CRS-2676: Start of 'ora.asm' on 'laguna1' succeeded CRS-2672: Attempting to start 'ora.config.dg' on 'laguna1' CRS-2676: Start of 'ora.config.dg' on 'laguna1' succeeded Configure Oracle Grid Infrastructure for a Cluster... succeeded [root@laguna1:~]# [root@laguna2:~]# /u01/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin... Copying oraenv to /usr/local/bin... Copying coraenv to /usr/local/bin... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation OLR initialization - successful Adding Clusterware entries to upstart CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node laguna1, number 1, and is terminating An active cluster was found during exclusive startup, restarting to join the cluster Configure Oracle Grid Infrastructure for a Cluster... succeeded [root@laguna2:~]#

27 ### Oracle Cluster Verification Utility NTP related error: There are three problems: 1) ntpd daemon is dead. Restarting ntpd process resolved the problem. service ntpd status service ntpd stop service ntpd start 2) ntp server entry is not valid /etc/ntp.conf Deleting invalid server names, inserting valid server, resolved. 3) time is beyond drift value: adjust date according to ntp server. ntpdate <ntp server ip> ntpdate After getting NTP related error: INFO: CTSS is in Observer state. Switching over to clock synchronization checks using NTP INFO: Starting Clock synchronization checks using Network Time Protocol(NTP)... INFO: NTP Configuration file check started... INFO: NTP Configuration file check passed INFO: Checking daemon liveness... INFO: Liveness check passed for "ntpd" INFO: Check for NTP daemon or service alive passed on all nodes INFO: NTP daemon slewing option check failed on some nodes INFO: Check failed on nodes: INFO: laguna2 INFO: PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x" [root@laguna2:~/.ssh]# service ntpd status ntpd dead but pid file exists [root@laguna2:~/.ssh]# ^C [root@laguna2:~/.ssh]# service ntpd stop Shutting down ntpd: [FAILED] [root@laguna2:~/.ssh]# service ntpd start Starting ntpd: [ OK ] [root@laguna2:~/.ssh]# [root@laguna2:~/.ssh]# service ntpd status ntpd (pid 47784) is running... [root@laguna2:~/.ssh]# service ntpd stop Shutting down ntpd: [ OK ] [root@laguna2:~/.ssh]# service ntpd start Starting ntpd: [ OK ] [root@laguna2:~/.ssh]# service ntpd status ntpd (pid 48323) is running... [root@laguna2:~/.ssh]# vi /etc/ntp.conf ### NTP server ntp.conf file was containing illegal server entries. [root@laguna2:~/.ssh]# service ntpd stop Shutting down ntpd: [ OK ] ### ntpdate command for updating date accoring to supplied ntp server.

[root@laguna2:~/.ssh]# ntpdate 22 May 09:26:29 ntpdate[61843]: no servers can be used, exiting [root@laguna2:~/.ssh]# ntpdate 10.200.100.

28 ntpdate 22 May 09:26:29 ntpdate[61843]: no servers can be used, exiting ntpdate May 09:26:40 ntpdate[62471]: step time server offset sec ### After pressing Retry it successfully finished the step: Oracle Cluster Verification Utility

29 [It is not over, continueing in other page...]

30 ### It worths seeing the change in ifconfig output before and after GI installation: ###BEFORE ifconfig bond0 Link encap:ethernet HWaddr AC:16:2D:B0:29:74 inet addr: Bcast: Mask: inet6 addr: fe80::ae16:2dff:feb0:2974/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets:68500 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (68.5 MiB) TX bytes: (64.2 MiB) bond1 eth0 eth1 eth2 eth3 lo Link encap:ethernet HWaddr AC:16:2D:B0:29:76 inet addr: Bcast: Mask: inet6 addr: fe80::ae16:2dff:feb0:2976/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:69690 errors:0 dropped:0 overruns:0 frame:0 TX packets:224 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (3.9 MiB) TX bytes:10704 (10.4 KiB) Link encap:ethernet HWaddr AC:16:2D:B0:29:74 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets:68500 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (62.4 MiB) TX bytes: (64.2 MiB) Interrupt:69 Link encap:ethernet HWaddr AC:16:2D:B0:29:74 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (6.1 MiB) TX bytes:0 (0.0 b) Interrupt:73 Link encap:ethernet HWaddr AC:16:2D:B0:29:76 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:34843 errors:0 dropped:0 overruns:0 frame:0 TX packets:224 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (1.9 MiB) TX bytes:10704 (10.4 KiB) Interrupt:77 Link encap:ethernet HWaddr AC:16:2D:B0:29:76 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:34847 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (1.9 MiB) TX bytes:0 (0.0 b) Interrupt:81 Link encap:local Loopback inet addr: Mask:

31 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:96832 errors:0 dropped:0 overruns:0 frame:0 TX packets:96832 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (22.1 MiB) TX bytes: (22.1 MiB) ### AFTER ifconfig bond0 Link encap:ethernet HWaddr AC:16:2D:B0:29:74 inet addr: Bcast: Mask: inet6 addr: fe80::ae16:2dff:feb0:2974/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (240.1 MiB) TX bytes: (4.6 GiB) bond0:1 bond0:3 bond0:4 bond1 bond1:1 eth0 eth1 Link encap:ethernet HWaddr AC:16:2D:B0:29:74 inet addr: Bcast: Mask: UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 Link encap:ethernet HWaddr AC:16:2D:B0:29:74 inet addr: Bcast: Mask: UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 Link encap:ethernet HWaddr AC:16:2D:B0:29:74 inet addr: Bcast: Mask: UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 Link encap:ethernet HWaddr AC:16:2D:B0:29:76 inet addr: Bcast: Mask: inet6 addr: fe80::ae16:2dff:feb0:2976/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (1.8 GiB) TX bytes: (1.3 GiB) Link encap:ethernet HWaddr AC:16:2D:B0:29:76 inet addr: Bcast: Mask: UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 Link encap:ethernet HWaddr AC:16:2D:B0:29:74 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (228.0 MiB) TX bytes: (4.6 GiB) Interrupt:69 Link encap:ethernet HWaddr AC:16:2D:B0:29:74 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0

32 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (12.0 MiB) TX bytes:0 (0.0 b) Interrupt:73 eth2 eth3 lo Link encap:ethernet HWaddr AC:16:2D:B0:29:76 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (1.8 GiB) TX bytes: (1.3 GiB) Interrupt:77 Link encap:ethernet HWaddr AC:16:2D:B0:29:76 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:68709 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (3.9 MiB) TX bytes:0 (0.0 b) Interrupt:81 Link encap:local Loopback inet addr: Mask: inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (225.0 MiB) TX bytes: (225.0 MiB) ### 2 scan names and vip is up in first node. [root@laguna1:~]# nslookup laguna1vip.turkcell.tgc Server: Address: #53 Name: laguna1vip.turkcell.tgc Address: [root@laguna1:~]# nslookup laguna-scan Server: Address: #53 Name: laguna-scan.turkcell.tgc Address: Name: laguna-scan.turkcell.tgc Address: Name: laguna-scan.turkcell.tgc Address: [root@laguna1:~]#

33 ### The orher san ip and laguna2vip is up in laguna2 node. nslookup laguna2vip.turkcell.tgc Server: Address: #53 Name: laguna2vip.turkcell.tgc Address: ifconfig bond0 Link encap:ethernet HWaddr AC:16:2D:BC:8D:50 inet addr: Bcast: Mask: inet6 addr: fe80::ae16:2dff:febc:8d50/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (3.2 GiB) TX bytes: (67.7 MiB) bond0:1 bond0:2 bond1 bond1:1 eth0 eth1 eth2 Link encap:ethernet HWaddr AC:16:2D:BC:8D:50 inet addr: Bcast: Mask: UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 Link encap:ethernet HWaddr AC:16:2D:BC:8D:50 inet addr: Bcast: Mask: UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 Link encap:ethernet HWaddr AC:16:2D:BC:8D:52 inet addr: Bcast: Mask: inet6 addr: fe80::ae16:2dff:febc:8d52/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (1.3 GiB) TX bytes: (1.8 GiB) Link encap:ethernet HWaddr AC:16:2D:BC:8D:52 inet addr: Bcast: Mask: UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 Link encap:ethernet HWaddr AC:16:2D:BC:8D:50 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (12.0 MiB) TX bytes:0 (0.0 b) Interrupt:71 Link encap:ethernet HWaddr AC:16:2D:BC:8D:50 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (3.2 GiB) TX bytes: (67.7 MiB) Interrupt:75 Link encap:ethernet HWaddr AC:16:2D:BC:8D:52

34 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (1.3 GiB) TX bytes: (1.8 GiB) Interrupt:79 eth3 lo Link encap:ethernet HWaddr AC:16:2D:BC:8D:52 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:68909 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (3.9 MiB) TX bytes:0 (0.0 b) Interrupt:83 Link encap:local Loopback inet addr: Mask: inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (95.8 MiB) TX bytes: (95.8 MiB) Post Installation Steps Back Up the root.sh Script for both nodes. cp /u01/app/11.2.0/grid/root.sh /u01/orainst/root.shbackup_gi_inst Creating the Fast Recovery Area Disk Group grid]$ cd /u01/app/11.2.0/grid/bin bin]$ bin]$ bin]$./asmca

37 A.7 About Using CVU Cluster Healthchecks After Installation bin]$ cd /u01/orainst/cvu/bin bin]$./cluvfy comp healthcheck -collect cluster -bestpractice deviations Verifying OS Best Practice Verifying Hardware Clock synchronization at shutdown...not met Verifying Clusterware Best Practice *********************************************************************************** ******* Summary of environment *********************************************************************************** ******* Date (mm/dd/yyyy) : 05/23/2013 Time (hh:mm:ss) : 16:40:45 Cluster name : laguna-cluster Clusterware version : Grid home : /u01/app/11.2.0/grid Grid User : grid Operating system : Linux el6.x86_64 *********************************************************************************** ******* System recommendations *********************************************************************************** ******* Verification Check : Hardware Clock synchronization at shutdown Verification Description : Checks whether Hardware Clock is synchronized with the system clock during system shutdown Verification Result : NOT MET Verification Summary : Check for Hardware Clock synchronization at shutdown failed Node Status Expected Value Actual Value laguna2 FAILED HWCLOCKSYNC=1 HWCLOCKSYNC=0 laguna1 FAILED HWCLOCKSYNC=1 HWCLOCKSYNC=0 [grid@laguna1 bin]$

38 bin]$./cluvfy comp healthcheck -collect cluster -mandatory -save - savedir /u01/orainst/cvu/bin [grid@laguna1 bin]$./cluvfy comp healthcheck -collect cluster -mandatory -save - savedir /u01/orainst/cvu/bin cvucheckreport_ txt

STEP BY STEP ORACLE RAC 12C (FLEX ASM) INSTALLATION ON LINUX X86_64 (VM)

STEP BY STEP ORACLE RAC 12C (FLEX ASM) INSTALLATION ON LINUX X86_64 (VM) V1.0 Contents 1. Prepare Operating System 2. GRID Infrastructure Installation 3. RDBMS Installation 4. Container Database Creation