Upgrade Grid Infrastructure from to on Linux

Transcription

1 Upgrade Grid Infrastructure from to on Linux Created by : Hans Camu Date : 29 August In this article I will describe the steps to upgrade an Oracle Grid Infrastructure cluster on Linux to version In this article a 2- node cluster with nodes RAC1 and RAC2 is used for this purpose. Unlike most other articles I will not describe how to use the OUI to install the software and do the upgrade. For this purpose I will execute a software only installation of the GI software (for which the OUI will be used, but there it ends) and then also apply some patches before the actual upgrade. The upgrade itself will be executed using the silent option of the config.sh tool. I will show you how to check the prerequisites to upgrade the Grid Infrastructure software. The upgrade of the RDBMS software and the upgrade of the database will NOT be discussed here, but in a next article. Before you start upgrading make sure patch : 11203:ASM UPGRADE FAILED ON FIRST NODE WITH ORA is installed. Without this patch you will not be able to perform a successful upgrade to This patch is available on top of the base version and a version is available for GI PSU 1 until GI PSU 5. The patch is included from GI PSU 6 and above. To upgrade to the following steps are involved: 1. Meet the prerequisites 2. Perform a software only installation of the base software 3. Apply latest GI PSU ( ) and, if needed, some additional patches to the GI software 4. Run the cluster verification utility to check the prerequisites for the upgrade 5. Run the config.sh in silent mode to configure the cluster using the new software 6. Run the rootupgrade.sh to actually upgrade to version Page 1 van 26

2 1. Meet the prerequisites Before we can start, the following patches need to be downloaded: Patch : Oracle patch set Patch : GI PSU 7 (July 2012) Patch : The latest OPatch version Patch or PSU 6 or later: a prerequisite patch for Before we start we want to know exactly at which version we are running at this moment: oracle@rac1::/home/oracle $ /u01/app/grid/ /bin/crsctl query crs softwareversion Oracle Clusterware version on node [rac1] is [ ] oracle@rac1::/home/oracle $ /u01/app/grid/ /bin/crsctl query crs releaseversion Oracle High Availability Services release version on the local node is [ ] oracle@rac1::/home/oracle $ /u01/app/grid/ /bin/crsctl query crs activeversion Oracle Clusterware active version on the cluster is [ ] As you can see, we are running version at this moment. 2. Perform a software only installation of the base software As from 11gR2 an upgrade of GI can only be executed as an out- of- place upgrade. This means the current GI_HOME will not be updated, but a new GI_HOME will be crated. The upgrade itself will be done by switching from the old to the new ORACLE_HOME and activate the new software. This is done by executing an upgrade script (rootupgrade.sh) as root. Before software only installation, create the new GI_HOME directory and set it s permissions. root@rac1::/root $ mkdir /u01/app/grid/ $ chown oracle:dba /u01/app/grid/ $ chmod 755 /u01/app/grid/ Now start the OUI to install the new GI software: oracle@rac1::/home/oracle $ cd /stagedir/ /grid oracle@rac1::/stagedir/ /grid $./runinstaller Page 2 van 26

3 Click Next. Choose the option Install Oracle Grid Infrastructure Software Only. Click Next. Page 3 van 26

4 Click Next (choose another language first, if you want to). Select the name for all groups and click Next. Page 4 van 26

5 It is recommended to define separate OS groups, but I chose not to do so. Click Yes. Specify the GI_HOME directory. Is this directory the software will be installed. Click Next. Page 5 van 26

6 Wait while some verification checks are executed. Click Install to start installing the GI software. Page 6 van 26

7 Wait while the software is installed. The root.sh script must be executed now: Page 7 van 26

8 $ /u01/app/grid/ /root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/grid/ Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. To configure Grid Infrastructure for a Stand-Alone Server run the following command as the root user: /u01/app/grid/ /perl/bin/perl - I/u01/app/grid/ /perl/lib -I/u01/app/grid/ /crs/install /u01/app/grid/ /crs/install/roothas.pl To configure Grid Infrastructure for a Cluster execute the following command: /u01/app/grid/ /crs/config/config.sh This command launches the Grid Infrastructure Configuration Wizard. The wizard also supports silent operation, and the parameters can be passed through the response file that is available in the installation media. Now click OK. Page 8 van 26

9 Click Close to exit the installer. 3. Apply latest GI PSU ( ) and, if needed, some additional patches to the GI software The base GI software is now installed. The next step is to apply some additional patches. We will start will updating the OPatch utility to the latest version. $ cd /stagedir/ /patches/zip $ unzip p _112000_linux-x86-64.zip -d /u01/app/grid/ Archive: p _112000_linux-x86-64.zip creating: /u01/app/grid/ /opatch/oplan/ inflating: /u01/app/grid/ /opatch/oplan/readme.html inflating: /u01/app/grid/ /opatch/oplan/readme.txt creating: /u01/app/grid/ /opatch/oplan/jlib/ inflating: /u01/app/grid/ /opatch/oplan/jlib/oplan.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/oracle.oplan.classpath.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/automation.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/osysmodel.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/emrepodrivers.jar creating: /u01/app/grid/ /opatch/oplan/jlib/apache-commons/ inflating: /u01/app/grid/ /opatch/oplan/jlib/apachecommons/commons-cli-1.0.jar creating: /u01/app/grid/ /opatch/oplan/jlib/jaxb/ inflating: /u01/app/grid/ /opatch/oplan/jlib/jaxb/activation.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/jaxb/jaxb-api.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/jaxb/jaxb-impl.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/jaxb/jsr173_1.0_api.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/osysmodel-utils.jar inflating: /u01/app/grid/ /opatch/oplan/jlib/crsproductdriver.jar inflating: /u01/app/grid/ /opatch/oplan/oplan Page 9 van 26

10 replace /u01/app/grid/ /opatch/docs/faq? [y]es, [n]o, [A]ll, [N]one, [r]ename: A inflating: /u01/app/grid/ /opatch/docs/faq inflating: /u01/app/grid/ /opatch/docs/users_guide.txt inflating: /u01/app/grid/ /opatch/docs/prereq_users_guide.txt inflating: /u01/app/grid/ /opatch/jlib/opatch.jar inflating: /u01/app/grid/ /opatch/jlib/opatchsdk.jar inflating: /u01/app/grid/ /opatch/jlib/oracle.opatch.classpath.jar inflating: /u01/app/grid/ /opatch/jlib/oracle.opatch.classpath.unix.jar inflating: /u01/app/grid/ /opatch/jlib/oracle.opatch.classpath.windows.jar inflating: /u01/app/grid/ /opatch/opatchprereqs/opatch/opatch_prereq.xml inflating: /u01/app/grid/ /opatch/opatchprereqs/opatch/rulemap.xml inflating: /u01/app/grid/ /opatch/opatchprereqs/opatch/runtime_prereq.xml inflating: /u01/app/grid/ /opatch/opatchprereqs/oui/knowledgesrc.xml inflating: /u01/app/grid/ /opatch/opatchprereqs/prerequisite.properties inflating: /u01/app/grid/ /opatch/opatch inflating: /u01/app/grid/ /opatch/opatch.bat inflating: /u01/app/grid/ /opatch/opatch.pl inflating: /u01/app/grid/ /opatch/opatch.ini inflating: /u01/app/grid/ /opatch/opatchdiag inflating: /u01/app/grid/ /opatch/opatchdiag.bat inflating: /u01/app/grid/ /opatch/emdpatch.pl inflating: /u01/app/grid/ /opatch/readme.txt creating: /u01/app/grid/ /opatch/ocm/bin/ inflating: /u01/app/grid/ /opatch/ocm/bin/emocmrsp creating: /u01/app/grid/ /opatch/ocm/doc/ creating: /u01/app/grid/ /opatch/ocm/lib/ inflating: /u01/app/grid/ /opatch/ocm/lib/emocmclnt-14.jar inflating: /u01/app/grid/ /opatch/ocm/lib/emocmclnt.jar inflating: /u01/app/grid/ /opatch/ocm/lib/emocmcommon.jar inflating: /u01/app/grid/ /opatch/ocm/lib/http_client.jar inflating: /u01/app/grid/ /opatch/ocm/lib/jcert.jar inflating: /u01/app/grid/ /opatch/ocm/lib/jnet.jar inflating: /u01/app/grid/ /opatch/ocm/lib/jsse.jar inflating: /u01/app/grid/ /opatch/ocm/lib/log4j-core.jar inflating: /u01/app/grid/ /opatch/ocm/lib/osdt_core3.jar inflating: /u01/app/grid/ /opatch/ocm/lib/osdt_jce.jar inflating: /u01/app/grid/ /opatch/ocm/lib/regexp.jar inflating: /u01/app/grid/ /opatch/ocm/lib/xmlparserv2.jar extracting: /u01/app/grid/ /opatch/ocm/ocm.zip inflating: /u01/app/grid/ /opatch/ocm/ocm_platforms.txt creating: /u01/app/grid/ /opatch/crs/ creating: /u01/app/grid/ /opatch/crs/log/ extracting: /u01/app/grid/ /opatch/crs/log/dummy inflating: /u01/app/grid/ /opatch/crs/auto_patch.pl inflating: /u01/app/grid/ /opatch/crs/crsconfig_lib.pm inflating: /u01/app/grid/ /opatch/crs/crsdelete.pm inflating: /u01/app/grid/ /opatch/crs/crspatch.pm inflating: /u01/app/grid/ /opatch/crs/installpatch.excl inflating: /u01/app/grid/ /opatch/crs/oracss.pm inflating: /u01/app/grid/ /opatch/crs/patch112.pl inflating: /u01/app/grid/ /opatch/crs/s_crsconfig_defs inflating: /u01/app/grid/ /opatch/crs/s_crsconfig_lib.pm Now OPatch is up to date we can install the latest GI PSU. Will writing this article the latest GI PSU is PSU 7 (patch ). This patch must be installed in 2 steps. The first step will install the GI part of the PSU and the second the RDBMS part of the PSU. This part must also be installed on the GI_HOME. Page 10 van 26

11 $ /u01/app/grid/ /opatch/opatch napply -oh /u01/app/grid/ local /stagedir/ /patches/gi/ / Oracle Interim Patch Installer version Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/grid/ Central Inventory : /u01/app/orainventory from : /u01/app/grid/ /orainst.loc OPatch version : OUI version : Log file location : /u01/app/grid/ /cfgtoollogs/opatch/opatch _ PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: Do you want to proceed? [y n] y User Responded with: Y All checks passed. Provide your address to be informed of security issues, install and initiate Oracle Configuration Manager. Easier for you if you use your My Oracle Support address/user Name. Visit for details. address/user Name: <ENTER> You have not provided an address for notification of security issues. Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: y Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/grid/ ') Is the local system ready for patching? [y n] y User Responded with: Y Backing up files... Applying interim patch ' ' to OH '/u01/app/grid/ ' Patching component oracle.crs, Patching component oracle.usm, Verifying the update... OPatch found the word "warning" in the stderr of the make command. Please look at this stderr. You can re-run this make command. Stderr output: ins_srvm.mk:68: warning: overriding commands for target `libsrvm11.so' ins_srvm.mk:31: warning: ignoring old commands for target `libsrvm11.so' ins_srvm.mk:71: warning: overriding commands for target `libsrvmocr11.so' Page 11 van 26

12 ins_srvm.mk:34: warning: ignoring old commands for target `libsrvmocr11.so' ins_srvm.mk:74: warning: overriding commands for target `libsrvmhas11.so' ins_srvm.mk:37: warning: ignoring old commands for target `libsrvmhas11.so' Patch successfully applied. OPatch Session completed with warnings. Log file location: /u01/app/grid/ /cfgtoollogs/opatch/opatch _ PM_1.log OPatch completed with warnings. The warning can safely be ignored. Now we can install the RDBMS part of the PSU: $ /u01/app/grid/ /opatch/opatch napply -oh /u01/app/grid/ local /stagedir/ /patches/gi/ / Oracle Interim Patch Installer version Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/grid/ Central Inventory : /u01/app/orainventory from : /u01/app/grid/ /orainst.loc OPatch version : OUI version : Log file location : /u01/app/grid/ /cfgtoollogs/opatch/opatch _ PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: Do you want to proceed? [y n] y User Responded with: Y All checks passed. Provide your address to be informed of security issues, install and initiate Oracle Configuration Manager. Easier for you if you use your My Oracle Support address/user Name. Visit for details. address/user Name: <ENTER> You have not provided an address for notification of security issues. Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: y Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/grid/ ') Is the local system ready for patching? [y n] Page 12 van 26

13 y User Responded with: Y Backing up files... Applying sub-patch ' ' to OH '/u01/app/grid/ ' Patching component oracle.rdbms.rsf, Patching component oracle.rdbms, Patching component oracle.rdbms.dbscripts, Verifying the update... Applying sub-patch ' ' to OH '/u01/app/grid/ ' ApplySession: Optional component(s) [ oracle.sysman.console.db, ], [ oracle.sysman.oms.core, ] not present in the Oracle Home or a higher version is found. Patching component oracle.rdbms.rsf, Patching component oracle.rdbms, Patching component oracle.sdo.locator, Verifying the update... Applying sub-patch ' ' to OH '/u01/app/grid/ ' ApplySession: Optional component(s) [ oracle.sysman.console.db, ], [ oracle.network.cman, ] not present in the Oracle Home or a higher version is found. Patching component oracle.rdbms.rsf, Patching component oracle.rdbms, Patching component oracle.rdbms.dbscripts, Patching component oracle.network.rsf, Patching component oracle.network.listener, Verifying the update... Composite patch successfully applied. Log file location: /u01/app/grid/ /cfgtoollogs/opatch/opatch _ PM_1.log OPatch succeeded. The GI PSU is now successfully installed. We will install one other additional patch. This is a patch for a bug I have run into, but this can be any other patch you need, based on which patched are already installed in you current GI_HOME and not fixed yet in So the next is just an example and can be skipped. oracle@rac1::/stagedir/ /patches $ /u01/app/grid/ /opatch/opatch apply -oh /u01/app/grid/ local /stagedir/ /patches/ Oracle Interim Patch Installer version Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/grid/ Page 13 van 26

14 Central Inventory : /u01/app/orainventory from : /u01/app/grid/ /orainst.loc OPatch version : OUI version : Log file location : /u01/app/grid/ /cfgtoollogs/opatch/opatch _ PM_1.log Applying interim patch ' ' to OH '/u01/app/grid/ ' Verifying environment and performing prerequisite checks... All checks passed. Provide your address to be informed of security issues, install and initiate Oracle Configuration Manager. Easier for you if you use your My Oracle Support address/user Name. Visit for details. address/user Name: <ENTER> You have not provided an address for notification of security issues. Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: y Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/grid/ ') Is the local system ready for patching? [y n] y User Responded with: Y Backing up files... Patching component oracle.rdbms, Verifying the update... Patch successfully applied Log file location: /u01/app/grid/ /cfgtoollogs/opatch/opatch _ PM_1.log OPatch succeeded. The GI software is now installed and patched to meet your requirements. But it s still only available on this node. To make it available to all the other nodes in the cluster, we will create a tar- file of the new GI software and then distribute it. Because no root scripts are whatever have run yet, no node specific directories or log files are in the new GI_HOME yet. So the complete directory can be zipped. oracle@rac1::/home/oracle $ cd /u01/app/grid $ ls -ltr total 8 drwxr-xr-x 71 root dba 4096 Aug 28 16: drwxr-xr-x 59 oracle dba 4096 Aug 28 17: oracle@rac1::/u01/app/grid $ tar -zcvf /stagedir/ /gi_ tgz / /evm/ Page 14 van 26

15 /mesg/kfndgus.msb /log/ /log/crs/ Now the GI_HOME can be deployed to all the other nodes in the cluster. First create the new GI_HOME directory and set it s permissions. root@rac1::/root $ mkdir /u01/app/grid/ $ chown oracle:dba /u01/app/grid/ $ chmod 755 /u01/app/grid/ And now unzip the file. oracle@rac2::/home/oracle $ tar -xvf /stagedir/ /gi_ tgz -C /u01/app/grid / /evm/ /evm/lib/ /evm/lib/libevmd.a /evm/init/ /evm/admin/ /mesg/ /mesg/kfndgus.msb /log/ /log/crs/ Register the new GI_HOME in the orainventory: oracle@rac2::/home/oracle $ /u01/app/grid/ /oui/bin/runinstaller -attachhome - noclusterenabled ORACLE_HOME=/u01/app/grid/ ORACLE_HOME_NAME=Ora11g_gridinfrahome2 CLUSTER_NODES=rac1,rac2 "INVENTORY_LOCATION=/u01/app/oraInventory" LOCAL_NODE=`hostname -s` Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 8191 MB Passed The inventory pointer is located at /etc/orainst.loc The inventory is located at /u01/app/orainventory Please execute the 'null' script at the end of the session. 'AttachHome' was successful. If you have more nodes in your cluster, repeat these 3 steps on all nodes! Page 15 van 26

16 4. Run the cluster verification utility to check the prerequisites for the upgrade Before we will run the actual upgrade, we can now first determine if we meet al prerequisites to execute a successful upgrade. For this purpose we run the Cluster Verification Utility. oracle@rac1::/u01/app/grid $ cd /stagedir/ /grid oracle@rac1::/stagedir/ /grid $./runcluvfy.sh stage -pre crsinst -upgrade -n all -rolling - src_crshome /u01/app/grid/ dest_crshome /u01/app/grid/ dest_version Performing pre-checks for cluster services setup Checking node reachability... Node reachability check passed from node "rac2" Checking user equivalence... User equivalence check passed for user "oracle" Checking CRS user consistency CRS user consistency check successful Checking node connectivity... Checking hosts config file... Verification of the hosts config file successful Check: Node connectivity for interface "eth0" Node connectivity passed for interface "eth0" TCP connectivity check passed for subnet " " Check: Node connectivity for interface "eth1" Node connectivity passed for interface "eth1" TCP connectivity check passed for subnet " " Checking subnet mask consistency... Subnet mask consistency check passed for subnet " ". Subnet mask consistency check passed for subnet " ". Subnet mask consistency check passed. Node connectivity check passed Checking multicast communication... Checking subnet " " for multicast communication with multicast group " "... Check of subnet " " for multicast communication with multicast group " " passed. Checking subnet " " for multicast communication with multicast group " "... Check of subnet " " for multicast communication with multicast group " " passed. Page 16 van 26

17 Check of multicast communication passed. Checking OCR integrity... OCR integrity check passed Checking ASMLib configuration. Check for ASMLib configuration passed. Total memory check passed Available memory check passed Swap space check passed Free disk space check passed for "rac2:/u01/app/grid/ " Free disk space check passed for "rac1:/u01/app/grid/ " Free disk space check passed for "rac2:/tmp" Free disk space check passed for "rac1:/tmp" Check for multiple users with UID value 500 passed User existence check passed for "oracle" Group existence check passed for "dba" Membership check for user "oracle" in group "dba" [as Primary] passed Run level check passed Hard limits check passed for "maximum open file descriptors" Soft limits check passed for "maximum open file descriptors" Hard limits check passed for "maximum user processes" Soft limits check passed for "maximum user processes" Check for Oracle patch " " in home "/u01/app/grid/ " passed There are no oracle patches required for home "/u01/app/grid/ ". System architecture check passed Kernel version check passed Kernel parameter check passed for "semmsl" Kernel parameter check passed for "semmns" Kernel parameter check passed for "semopm" Kernel parameter check passed for "semmni" Kernel parameter check passed for "shmmax" Kernel parameter check passed for "shmmni" Kernel parameter check passed for "shmall" Kernel parameter check passed for "file-max" Kernel parameter check passed for "ip_local_port_range" Kernel parameter check passed for "rmem_default" Kernel parameter check passed for "rmem_max" Kernel parameter check passed for "wmem_default" Kernel parameter check passed for "wmem_max" Kernel parameter check passed for "aio-max-nr" Package existence check passed for "make" Package existence check passed for "binutils" Package existence check passed for "gcc(x86_64)" Package existence check passed for "libaio(x86_64)" Package existence check passed for "glibc(x86_64)" Package existence check passed for "compat-libstdc++-33(x86_64)" Package existence check passed for "elfutils-libelf(x86_64)" Package existence check passed for "elfutils-libelf-devel" Package existence check passed for "glibc-common" Package existence check passed for "glibc-devel(x86_64)" Package existence check passed for "glibc-headers" Package existence check passed for "gcc-c++(x86_64)" Package existence check passed for "libaio-devel(x86_64)" Package existence check passed for "libgcc(x86_64)" Package existence check passed for "libstdc++(x86_64)" Package existence check passed for "libstdc++-devel(x86_64)" Package existence check passed for "sysstat" Package existence check passed for "ksh" Check for multiple users with UID value 0 passed Page 17 van 26

18 Current group ID check passed Starting check for consistency of primary group of root user Check for consistency of root user's primary group passed Package existence check passed for "cvuqdisk" Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... No NTP Daemons or Services were found to be running Clock synchronization check using Network Time Protocol(NTP) passed Core file name pattern consistency check passed. User "oracle" is not part of "root" group. Check passed Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes File "/etc/resolv.conf" does not have both domain and search entries defined domain entry in file "/etc/resolv.conf" is consistent across nodes search entry in file "/etc/resolv.conf" is consistent across nodes All nodes have one search entry defined in file "/etc/resolv.conf" The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes UDev attributes check for OCR locations started... UDev attributes check passed for OCR locations UDev attributes check for Voting Disk locations started... UDev attributes check passed for Voting Disk locations Time zone consistency check passed Checking VIP configuration. Checking VIP Subnet configuration. Check for VIP Subnet configuration passed. Checking VIP reachability Check for VIP reachability passed. Checking Oracle Cluster Voting Disk configuration... ASM Running check passed. ASM is running on all specified nodes Oracle Cluster Voting Disk configuration check passed Clusterware version consistency passed Pre-check for cluster services setup was successful. The check must be successful. If not, then the prerequisites that are not met must be fixed first. Rerun the check until it ends successful. Page 18 van 26

19 5. Run the config.sh in silent mode to configure the cluster using the new software To configure the cluster we must run Oracle Clusterware Configuration Wizard. This time we will not run the GUI, but will execute a silent configuration. When using this option, several parameters must be passed to the config.sh and I will first discuss these parameters: -silent The tool will execute in a silent, non- interactive, mode ORACLE_HOSTNAME The hostname the tool will run from INVENTORY_LOCATION Specifies the location which holds the inventory files SELECTED_LANGUAGES Specify the languages in which the components will be installed oracle.install.option Specify the installation option ORACLE_BASE Specify the complete path of the Oracle Base ORACLE_HOME Specify the complete path of the Oracle Home oracle.install.asm.osdba The DBA_GROUP is the OS group which is to be granted OSDBA privileges oracle.install.asm.osoper The OPER_GROUP is the OS group which is to be granted OSOPER privileges oracle.install.asm.osasm The OSASM_GROUP is the OS group, which is to be granted OSASM privileges. This must be different than the previous two oracle.install.crs.config.clusternodes Specify a list of public node names, and virtual hostnames that have to be part of the cluster. The list should a comma- separated list of nodes. Each entry in the list should be a colon- separated string that contains 2 fields oracle.install.crs.upgrade.clusternodes Specify nodes for Upgrade oracle.install.asm.upgradeasm For RAC- ASM only. Value should be 'true' while upgrading Cluster ASM of version 11gR2( ) and above Page 19 van 26

20 Run the config.sh in silent mode to configure Grid Infrastructure : $ /u01/app/grid/ /crs/config/config.sh -silent ORACLE_HOSTNAME=rac1.camoraict.com INVENTORY_LOCATION=/u01/app/oraInventory SELECTED_LANGUAGES=en oracle.install.option=upgrade ORACLE_BASE=/u01/app/oracle ORACLE_HOME=/u01/app/grid/ oracle.install.asm.osdba=dba oracle.install.asm.osoper=dba oracle.install.asm.osasm=asmadmin oracle.install.crs.config.clusternodes=rac1:rac1-vip,rac2:rac2-vip oracle.install.crs.upgrade.clusternodes=rac1,rac2 oracle.install.asm.upgradeasm=true As a root user, execute the following script(s): 1. /u01/app/grid/ /rootupgrade.sh Execute /u01/app/grid/ /rootupgrade.sh on the following nodes: [rac1, rac2] Successfully Setup Software. 6. Run the rootupgrade.sh to actually upgrade to version When you execute the rootupgrade.sh script (as the root user of course), you actual upgrade is performed. It is highly recommended to umount all the ACFS mounts before you execute the script. If there are some open file handles to an ACFS filesystem, the filesystem can t be unmounted automatically, resulting in the upgrade to fail! root@rac1::/root $ mount /dev/mapper/systemvg-rootlv on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/mapper/systemvg-usrlv on /usr type ext3 (rw) /dev/mapper/systemvg-tmplv on /tmp type ext3 (rw) /dev/mapper/systemvg-homelv on /home type ext3 (rw) /dev/mapper/systemvg-varlv on /var type ext3 (rw) /dev/mapper/u01vg-u01lv on /u01 type ext3 (rw) /dev/sda1 on /boot type ext3 (rw) tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /software on /media/sf_/software type vboxsf (gid=503,rw) /dev/asm/v_orabackup-238 on /orabackup1 type acfs (rw) /dev/asm/v_software-238 on /software type acfs (rw) root@rac1::/root $ umount.acfs /orabackup1 root@rac1::/root $ umount.acfs /software root@rac1::/root $ mount /dev/mapper/systemvg-rootlv on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) Page 20 van 26

21 /dev/mapper/systemvg-usrlv on /usr type ext3 (rw) /dev/mapper/systemvg-tmplv on /tmp type ext3 (rw) /dev/mapper/systemvg-homelv on /home type ext3 (rw) /dev/mapper/systemvg-varlv on /var type ext3 (rw) /dev/mapper/u01vg-u01lv on /u01 type ext3 (rw) /dev/sda1 on /boot type ext3 (rw) tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /software on /media/sf_/software type vboxsf (gid=503,rw) Now run the rootupgrade.sh script on the first node as user root: $ /u01/app/grid/ /rootupgrade.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/grid/ Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Relinking oracle with rac_on option Using configuration parameter file: /u01/app/grid/ /crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation ASM upgrade has started on first node. CRS-2791: Starting shutdown of Oracle High Availability Servicesmanaged resources on 'rac1' CRS-2673: Attempting to stop 'ora.crsd' on 'rac1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1' CRS-2673: Attempting to stop 'ora.listener_scan1.lsnr' on 'rac1' CRS-2673: Attempting to stop 'ora.dgacfs.dg' on 'rac1' CRS-2673: Attempting to stop 'ora.dgdata.dg' on 'rac1' CRS-2673: Attempting to stop 'ora.dgfra.dg' on 'rac1' CRS-2673: Attempting to stop 'ora.dggrid.dg' on 'rac1' CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac1' CRS-2673: Attempting to stop 'ora.listener.lsnr' on 'rac1' CRS-2677: Stop of 'ora.listener_scan1.lsnr' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1' CRS-2677: Stop of 'ora.listener.lsnr' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac1' CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2' CRS-2677: Stop of 'ora.rac1.vip' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.rac1.vip' on 'rac2' CRS-2677: Stop of 'ora.registry.acfs' on 'rac1' succeeded CRS-2677: Stop of 'ora.dgdata.dg' on 'rac1' succeeded CRS-2677: Stop of 'ora.dgfra.dg' on 'rac1' succeeded CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded Page 21 van 26

22 CRS-2672: Attempting to start 'ora.listener_scan1.lsnr' on 'rac2' CRS-2676: Start of 'ora.rac1.vip' on 'rac2' succeeded CRS-2677: Stop of 'ora.dgacfs.dg' on 'rac1' succeeded CRS-2676: Start of 'ora.listener_scan1.lsnr' on 'rac2' succeeded CRS-2677: Stop of 'ora.dggrid.dg' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac1' CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'rac1' CRS-2677: Stop of 'ora.ons' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1' CRS-2677: Stop of 'ora.net1.network' on 'rac1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.crf' on 'rac1' CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1' CRS-2673: Attempting to stop 'ora.evmd' on 'rac1' CRS-2673: Attempting to stop 'ora.asm' on 'rac1' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1' CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded CRS-2677: Stop of 'ora.drivers.acfs' on 'rac1' succeeded CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac1' CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1' CRS-2673: Attempting to stop 'ora.diskmon' on 'rac1' CRS-2677: Stop of 'ora.diskmon' on 'rac1' succeeded CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1' CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed CRS-4133: Oracle High Availability Services has been stopped. OLR initialization - successful Replacing Clusterware entries in inittab clscfg: EXISTING configuration version 5 detected. clscfg: version 5 is 11g Release 2. Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Configure Oracle Grid Infrastructure for a Cluster... succeeded Check if the ACFS mounts are back again: oracle@rac1::/home/oracle $ mount /dev/mapper/systemvg-rootlv on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/mapper/systemvg-usrlv on /usr type ext3 (rw) /dev/mapper/systemvg-tmplv on /tmp type ext3 (rw) /dev/mapper/systemvg-homelv on /home type ext3 (rw) /dev/mapper/systemvg-varlv on /var type ext3 (rw) /dev/mapper/u01vg-u01lv on /u01 type ext3 (rw) Page 22 van 26

23 /dev/sda1 on /boot type ext3 (rw) tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /software on /media/sf_/software type vboxsf (gid=503,rw) /dev/asm/v_orabackup-238 on /orabackup1 type acfs (rw) /dev/asm/v_software-238 on /software type acfs (rw) Now you can run the upgrade script on the other nodes in the cluster, one by one. Here the same, make sure all ACFS mounts are umounted first! $ umount.acfs /orabackup1 $ umount.acfs /software root@rac2::/root $ mount /dev/mapper/systemvg-rootlv on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/mapper/systemvg-usrlv on /usr type ext3 (rw) /dev/mapper/systemvg-tmplv on /tmp type ext3 (rw) /dev/mapper/systemvg-homelv on /home type ext3 (rw) /dev/mapper/systemvg-varlv on /var type ext3 (rw) /dev/mapper/u01vg-u01lv on /u01 type ext3 (rw) /dev/sda1 on /boot type ext3 (rw) tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /software on /media/sf_/software type vboxsf (gid=503,rw) Now run the rootupgade.sh script on this node: root@rac2::/root $ /u01/app/grid/ /rootupgrade.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/grid/ Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Relinking oracle with rac_on option Using configuration parameter file: /u01/app/grid/ /crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation CRS-2791: Starting shutdown of Oracle High Availability Servicesmanaged resources on 'rac2' CRS-2673: Attempting to stop 'ora.crsd' on 'rac2' CRS-2790: Starting shutdown of Cluster Ready Services-managed Page 23 van 26

24 resources on 'rac2' CRS-2673: Attempting to stop 'ora.listener_scan3.lsnr' on 'rac2' CRS-2673: Attempting to stop 'ora.listener_scan2.lsnr' on 'rac2' CRS-2673: Attempting to stop 'ora.listener.lsnr' on 'rac2' CRS-2673: Attempting to stop 'ora.oc4j' on 'rac2' CRS-2673: Attempting to stop 'ora.cvu' on 'rac2' CRS-2673: Attempting to stop 'ora.dgacfs.dg' on 'rac2' CRS-2673: Attempting to stop 'ora.dgdata.dg' on 'rac2' CRS-2673: Attempting to stop 'ora.dgfra.dg' on 'rac2' CRS-2673: Attempting to stop 'ora.dggrid.dg' on 'rac2' CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac2' CRS-2677: Stop of 'ora.listener_scan3.lsnr' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rac2' CRS-2677: Stop of 'ora.listener_scan2.lsnr' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac2' CRS-2677: Stop of 'ora.listener.lsnr' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.rac2.vip' on 'rac2' CRS-2677: Stop of 'ora.scan3.vip' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.scan3.vip' on 'rac1' CRS-2677: Stop of 'ora.scan2.vip' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.scan2.vip' on 'rac1' CRS-2677: Stop of 'ora.rac2.vip' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.rac2.vip' on 'rac1' CRS-2677: Stop of 'ora.registry.acfs' on 'rac2' succeeded CRS-2677: Stop of 'ora.dgdata.dg' on 'rac2' succeeded CRS-2677: Stop of 'ora.dgfra.dg' on 'rac2' succeeded CRS-2676: Start of 'ora.scan3.vip' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.listener_scan3.lsnr' on 'rac1' CRS-2676: Start of 'ora.rac2.vip' on 'rac1' succeeded CRS-2676: Start of 'ora.scan2.vip' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.listener_scan2.lsnr' on 'rac1' CRS-2677: Stop of 'ora.dgacfs.dg' on 'rac2' succeeded CRS-2676: Start of 'ora.listener_scan3.lsnr' on 'rac1' succeeded CRS-2676: Start of 'ora.listener_scan2.lsnr' on 'rac1' succeeded CRS-2677: Stop of 'ora.cvu' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.cvu' on 'rac1' CRS-2677: Stop of 'ora.oc4j' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.oc4j' on 'rac1' CRS-2676: Start of 'ora.cvu' on 'rac1' succeeded CRS-2676: Start of 'ora.oc4j' on 'rac1' succeeded CRS-2677: Stop of 'ora.dggrid.dg' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac2' CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'rac2' CRS-2677: Stop of 'ora.ons' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'rac2' CRS-2677: Stop of 'ora.net1.network' on 'rac2' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac2' has completed CRS-2677: Stop of 'ora.crsd' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2' CRS-2673: Attempting to stop 'ora.evmd' on 'rac2' CRS-2673: Attempting to stop 'ora.asm' on 'rac2' CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac2' CRS-2677: Stop of 'ora.evmd' on 'rac2' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac2' succeeded CRS-2677: Stop of 'ora.drivers.acfs' on 'rac2' succeeded CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac2' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac2' succeeded Page 24 van 26

25 CRS-2673: Attempting to stop 'ora.cssd' on 'rac2' CRS-2677: Stop of 'ora.cssd' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.diskmon' on 'rac2' CRS-2673: Attempting to stop 'ora.crf' on 'rac2' CRS-2677: Stop of 'ora.diskmon' on 'rac2' succeeded CRS-2677: Stop of 'ora.crf' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2' CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2' CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2' has completed CRS-4133: Oracle High Availability Services has been stopped. OLR initialization - successful Replacing Clusterware entries in inittab clscfg: EXISTING configuration version 5 detected. clscfg: version 5 is 11g Release 2. Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Started to upgrade the Oracle Clusterware. This operation may take a few minutes. Started to upgrade the CSS. Started to upgrade the CRS. The CRS was successfully upgraded. Oracle Clusterware operating version was successfully set to ASM upgrade has finished on last node. PRKO-2116 : OC4J is already enabled Configure Oracle Grid Infrastructure for a Cluster... succeeded Check if the ACFS mounts are back again: oracle@rac2::/home/oracle $ mount /dev/mapper/systemvg-rootlv on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/mapper/systemvg-usrlv on /usr type ext3 (rw) /dev/mapper/systemvg-tmplv on /tmp type ext3 (rw) /dev/mapper/systemvg-homelv on /home type ext3 (rw) /dev/mapper/systemvg-varlv on /var type ext3 (rw) /dev/mapper/u01vg-u01lv on /u01 type ext3 (rw) /dev/sda1 on /boot type ext3 (rw) tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /software on /media/sf_/software type vboxsf (gid=503,rw) /dev/asm/v_orabackup-238 on /orabackup1 type acfs (rw) /dev/asm/v_software-238 on /software type acfs (rw) Repeat the above steps for all nodes in the cluster and make sure all nodes are upgraded! Page 25 van 26

26 Now the upgrade script has been successfully executed on all nodes in the cluster we must check that we now are really running the version: $ /u01/app/grid/ /bin/crsctl query crs softwareversion Oracle Clusterware version on node [rac1] is [ ] $ /u01/app/grid/ /bin/crsctl query crs releaseversion Oracle High Availability Services release version on the local node is [ ] $ /u01/app/grid/ /bin/crsctl query crs activeversion Oracle Clusterware active version on the cluster is [ ] Now we are running the correct upgraded version it is save to remove the binaries of the old version! Congratulations, you have successfully upgraded your Grid Infrastructure cluster. Page 26 van 26