Oracle 11gR2 Two Node Real Application Cluster installation step by step guide on Linux using Vmware Server 2.0

Transcription

1 Oracle 11gR2 Two Node Real Application Cluster installation step by step guide on Linux using Vmware Server 2.0 Mohamed Azar. Database Administrator, Blogger, Contributor,Book Reviewer. reach me at Page 1

2 Step 1: Pre-requisites Before Start my installation, the below recommanded Memory Configuration needed for you. In this case onle you can smoothly install RAC. If you re plan to Install Two node RAC on your virtual machine, Each node want atleast minimum 2GB RAM as per Oracle 11gR2 Installation recommand. So you need atelast minimum 6GB RAM for your laptop. 4 GB for Virtual Machine, remaining for your windows. Recommanded Browser : Mozilla Firefox 3.5 or Minimum version. The upgrade version nor support for Vmware Server 2.0. Internet Explorer 7 or 6, I hope IE 9 not support for VMware Server 2.0. Step 2: Download Software Download VMware Server and Install ware%20server%20download&aq=f&aqi=g10&aql=&oq= Download Oracle Enterprise Linux 32 Bit Download Oracle Software Grid Infrastructure and Database Software html Step 3: Install Oracle Enterprise Linux We need to Install Oracle Enterprise Linux for Vmware Server infrastructure environment. Follow procedure for further installation. Create Virtual Machine Name of the Virtual Machine racha1 Page 2

3 Choose Linux Operating System Red Hat Enterprise Linux 5(32-bit). Page 3

4 Allocate Virtual Machine RAM memory 2048MB (Oracle 11gR2 recommand) Create New Virtual DISK (Hard Disk), Size 30 GB Page 4

5 Add a Network Adapter first with bridged Network (RAC Environment We need to configure TWO adapter, Add another adapter later ). Page 5

6 If you want to use Linux CD Choose Physical Drive OR If you plan to use ISO image file, you can use ISO image and after chooses it, you can select your first disc ISO image using browser option. The software file should be in Virtual Machine Datastore. Page 6

7 Add a USB Controller Page 7

8 Add another one Network Adapter using with bridged network. Now new virtual machine hardware configuration finished. Page 8

9 Power on Virtual Machine : Page 9

10 Click on console window, the new remote console window will appear. If it s not appear, it will ask you to install REMOTE plugin console. Just you install it. Click Skip option. Page 10

11 Page 11

12 Page 12

13 Page 13

14 Page 14

15 Select eth0 put it manual IP, disable IPv6 Your server name with localdomain name Page 15

16 Select eth1 ipaddress Page 16

17 Page 17

18 GNOME Desktop Environment Editors Graphical Internet Text-based Internet Development Libraries Development Tools Server Configuration Tools Administration Tools Base System Tools X Window System Page 18

19 Page 19

20 Page 20

21 Page 21

22 Page 22

23 Page 23

24 Page 24

25 Page 25

26 Page 26

27 Page 27

28 Installation Completed. Step 4: Install VMware tools, Click right side panel Install VMware Tools Page 28

29 Go to Linux node 1 machine, Install vmware tools like below [root@racha1 ~]# cd /media/vmware\ Tools/ [root@racha1 ~]# cp VMwareTools tar.gz /tmp cd /tmp [root@racha1 tmp]# tar -zxvf VMwareTools tar.gz [root@racha1 tmp]# cd vmware-tools-distrib/ [root@racha1 vmware-tools-distrib]# ls bin doc etc FILES INSTALL installer lib vmware-install.pl [root@racha1 vmware-tools-distrib]#./vmware-install.pl Creating a new VMware Tools installer database using the tar4 format. Installing VMware Tools. In which directory do you want to install the binary files? [/usr/bin] What is the directory that contains the init directories (rc0.d/ to rc6.d/)? [/etc/rc.d] **Here choose virtual machine screen resolution also and after installation complete, Reboot machine. vmware-toolbox Step 5 : Oracle Installation Prerequisites. Make sure the shared memory file system is big enough for Automatic Memory Manager to work. Page 29

30 ~]# umount tmpfs ~]# mount -t tmpfs shmfs -o size=1500m /dev/shm ~]# vi /etc/fstab tmpfs /dev/shm tmpfs size=1500m 0 0 Install RPM Package : [root@racha1 rpm]# ls binutils el5.i386.rpm compat-libstdc i386.rpm elfutils-libelf-devel el5.i386.rpm elfutils-libelf-devel-static el5.i386.rpm gcc el5.i386.rpm gcc-c el5.i386.rpm glibc-devel i386.rpm glibc-headers i386.rpm kernel-headers el5.i386.rpm libacl-devel el5.i386.rpm libaio-devel i386.rpm libgomp el5.i386.rpm libstdc++-devel el5.i386.rpm sysstat el5.i386.rpm unixodbc i386.rpm unixodbc-devel i386.rpm [root@racha1 rpm]# rpm -Uvh unixodbc-* warning: unixodbc i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159 Preparing... ########################################### [100%] 1:unixODBC ########################################### [ 50%] 2:unixODBC-devel ########################################### [100%] [root@racha1 rpm]# Likewise Install Other rpm packages also. Add ipaddress in /etc/hosts file [root@racha1 ~]# vi /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail localhost.localdomain localhost # Public racha1.localdomain racha1 Page 30

31 racha2.localdomain racha2 # Private racha1-priv.localdomain racha1-priv racha2-priv.localdomain racha2-priv # Virtual racha1-vip.localdomain racha1-vip racha2-vip.localdomain racha2-vip # SCAN rac-scan.localdomain rac-scan The SCAN address should not really be defined in the hosts file. Instead is should be defined on the DNS to round-robin between 3 addresses on the same subnet as the public IPs. For this installation, we will compromise and use the hosts file. More Detail about SCAN &p_cat=documents_new&p_user=YURY&p_company= /etc/sysctl.conf [root@racha1 ~]# vi /etc/sysctl.conf fs.aio-max-nr = fs.file-max = kernel.shmall = kernel.shmmax = kernel.shmmni = 4096 # semaphores: semmsl, semmns, semopm, semmni kernel.sem = net.ipv4.ip_local_port_range = net.core.rmem_default= net.core.rmem_max= net.core.wmem_default= net.core.wmem_max= [root@racha1 ~]# /sbin/sysctl -p net.ipv4.ip_forward = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = Page 31

32 kernel.msgmax = fs.aio-max-nr = fs.file-max = kernel.shmall = kernel.shmmax = kernel.shmmni = 4096 kernel.sem = net.ipv4.ip_local_port_range = net.core.rmem_default = net.core.rmem_max = net.core.wmem_default = net.core.wmem_max = [root@racha1 ~]# [root@racha1 ~]# vi /etc/security/limits.conf oracle soft nproc 2047 oracle hard nproc oracle soft nofile 1024 oracle hard nofile [root@racha1 ~]# vi /etc/pam.d/login session required pam_limits.so [root@racha1 ~]# cat /etc/selinux/config # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - SELinux is fully disabled. SELINUX=disabled # SELINUXTYPE= type of policy in use. Possible values are: # targeted - Only targeted network daemons are protected. # strict - Full SELinux protection. SELINUXTYPE=targeted [root@racha1 ~]# Page 32

33 The Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes. In this case we will deconfigure NTP. ~]# service ntpd stop Shutting down ntpd: [FAILED] ~]# chkconfig ntpd off ~]# mv /etc/ntp.conf /etc/ntp.conf.org ~]# rm /var/run/ntpd.pid rm: cannot lstat `/var/run/ntpd.pid': No such file or directory User and Group Add ~]# groupadd -g 500 oinstall ~]# groupadd -g 501 dba ~]# groupadd -g 502 oper ~]# groupadd -g 503 asmadmin ~]# useradd -u g oinstall -G dba,oper,asmadmin oracle ~]# passwd oracle Changing password for user oracle. New UNIX password: BAD PASSWORD: it is based on a dictionary word Retype new UNIX password: passwd: all authentication tokens updated successfully. [root@racha1 ~]# Make Directory [root@racha1 ~]# mkdir -p /u01/app/11.2.0/grid [root@racha1 ~]# mkdir -p /u01/app/oracle/product/11.2.0/dbhome_1 [root@racha1 ~]# chown -R oracle:oinstall /u01 [root@racha1 ~]# chmod -R 775 /u01 Bash Profile Log on as Oracle User. # Oracle Settings TMP=/tmp; export TMP TMPDIR=$TMP; export TMPDIR Page 33

34 ORACLE_HOSTNAME=racha1.localdomain; export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC; export ORACLE_UNQNAME ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1; export ORACLE_HOME ORACLE_SID=RAC1; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH; export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p ulimit -n else ulimit -u n fi fi Shutdown NODE 1 machine. [root@racha1 ~]#shutdown h now Create three shared DISK Making directory for sharing disk in host OS. Page 34

35 C:\Users\Azar>mkdir d:\shareasm On the VMware Infrastructure web access panel, click Add Hardware and click Add hard disk Page 35

36 10GB allocate for new harddisk Page 36

37 Create other two hard disk as the same above procedure. After added three disks, Then see your racha1.vmx file, you need to modify like this. Page 37

38 Now Restart the NODE 1 racha1 machine and after restarted, partition your disks like below [root@racha1 ~]# cd /dev/ [root@racha1 dev]# ls sd* sda sda1 sda2 sdb sdc sdd [root@racha1 dev]# fdisk sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: Page 38

39 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1305, default 1): Using default value 1 Last cylinder or +size or +sizem or +sizek (1-1305, default 1305): Using default value 1305 Command (m for help): p Disk sdb: 10.7 GB, bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of * 512 = bytes Device Boot Start End Blocks Id System sdb Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@racha1 dev]# Do It as same again procedure for other remaining Disks. ASMLIB Install Download ASMLIB rpm package for your OS requirements [root@racha1 ~]# cp /media/azar/asmlib/*.rpm /install/asmlib [root@racha1 ~]# cd /install/asmlib [root@racha1 asmlib]# ls Page 39

40 oracleasm el el5.i686.rpm oracleasmlib el5.i386.rpm oracleasm-support el5.i386.rpm asmlib]# rpm -Uvh oracleasm-support el5.i386.rpm warning: oracleasm-support el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159 Preparing... ########################################### [100%] 1:oracleasm-support ########################################### [100%] asmlib]# rpm -Uvh oracleasm el el5.i686.rpm warning: oracleasm el el5.i686.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159 Preparing... ########################################### [100%] 1:oracleasm el########################################### [100%] asmlib]# rpm -Uvh oracleasm el5-debuginfo el5.i686.rpm warning: oracleasm el5-debuginfo el5.i686.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159 Preparing... ########################################### [100%] 1:oracleasm el########################################### [100%] ~]# oracleasm configure -i Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: oracle Default group to own the driver interface []: dba Start Oracle ASM library driver on boot (y/n) [n]: y Scan for Oracle ASM disks on boot (y/n) [y]: y Writing Oracle ASM library driver configuration: done [root@racha1 ~]# /usr/sbin/oracleasm init Creating /dev/oracleasm mount point: /dev/oracleasm Loading module "oracleasm": oracleasm Mounting ASMlib driver filesystem: /dev/oracleasm Page 40

41 ~]# /usr/sbin/oracleasm createdisk DISK1 /dev/sdb1 Writing disk header: done Instantiating disk: done ~]# /usr/sbin/oracleasm createdisk DISK2 /dev/sdc1 Writing disk header: done Instantiating disk: done ~]# /usr/sbin/oracleasm createdisk DISK3 /dev/sdd1 Writing disk header: done Instantiating disk: done Unzip Grid Infrastruture Software ~]# chown -R oracle:oinstall /install ~]# cd /install install]# ls asmlib linux_11gr2_grid.zip rpm install]# unzip linux_11gr2_grid.zip Shutdown Node 1 racha1 [root@racha1 install]# shutdown -h now Copy racha1 the same location and rename folder racha2 Edit the contents of d:\racha2\racha1.vmx displayname = "racha2" Page 41

42 In the VMware Infrastructure Web Access Console, select the "Virtual Machine > Add Virtual Machine to Inventory" and browser for the d:\racha2\racha1.vmx Start the racha2 machine, just choose i_copied it option and click ok. Now the another cloned copy of node1 virtual machine running. Log on to the virtual machine racha2 System > Administration > Network Remove the devices with the "%.bak" nicknames. To do this, highlight a device, deactivate, then delete it. This will leave just the regular "eth0" and "eth1" devices. Highlight the "eth0" interface and click the "Edit" button on the toolbar. Page 42

43 Page 43

44 Eth1 Configure Add ip address, subnet mask and Don t add default gateway ip address here. Page 44

45 If you want restart your network service,you need to use below commands. service network restart Edit Bash_profile of Node 2 Oracle user path correct to ORACLE_SID and HOSTNAME [root@racha1 ~]# su - oracle [oracle@racha1 ~]$ vi.bash_profile ORACLE_HOSTNAME=racha2.localdomain; export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC; export ORACLE_UNQNAME ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1; export ORACLE_HOME ORACLE_SID=RAC2; export ORACLE_SID Page 45

46 PING NETWORK : Node 1: [root@racha1 ~]# ping -c 3 racha2 PING racha2.localdomain ( ) 56(84) bytes of data. 64 bytes from racha2.localdomain ( ): icmp_seq=1 ttl=64 time=62.2 ms 64 bytes from racha2.localdomain ( ): icmp_seq=2 ttl=64 time=0.196 ms 64 bytes from racha2.localdomain ( ): icmp_seq=3 ttl=64 time=0.150 ms --- racha2.localdomain ping statistics packets transmitted, 3 received, 0% packet loss, time 4053ms rtt min/avg/max/mdev = 0.150/20.874/62.276/ ms [root@racha1 ~]# ping -c 3 racha2-priv PING racha2-priv.localdomain ( ) 56(84) bytes of data. 64 bytes from racha2-priv.localdomain ( ): icmp_seq=1 ttl=64 time=0.923 ms 64 bytes from racha2-priv.localdomain ( ): icmp_seq=2 ttl=64 time=0.180 ms 64 bytes from racha2-priv.localdomain ( ): icmp_seq=3 ttl=64 time=0.141 ms --- racha2-priv.localdomain ping statistics packets transmitted, 3 received, 0% packet loss, time 3159ms rtt min/avg/max/mdev = 0.141/0.414/0.923/0.360 ms [root@racha1 ~]# Node 2: [root@racha2 ~]# ping -c 3 racha1 PING racha1.localdomain ( ) 56(84) bytes of data. Page 46

47 64 bytes from racha1.localdomain ( ): icmp_seq=1 ttl=64 time=0.500 ms 64 bytes from racha1.localdomain ( ): icmp_seq=2 ttl=64 time=0.160 ms 64 bytes from racha1.localdomain ( ): icmp_seq=3 ttl=64 time=0.161 ms --- racha1.localdomain ping statistics packets transmitted, 3 received, 0% packet loss, time 2862ms rtt min/avg/max/mdev = 0.160/0.273/0.500/0.161 ms [root@racha2 ~]# ping -c 3 racha1-priv PING racha1-priv.localdomain ( ) 56(84) bytes of data. 64 bytes from racha1-priv.localdomain ( ): icmp_seq=1 ttl=64 time=0.430 ms 64 bytes from racha1-priv.localdomain ( ): icmp_seq=2 ttl=64 time=0.176 ms 64 bytes from racha1-priv.localdomain ( ): icmp_seq=3 ttl=64 time=0.166 ms SSH Configuration : Manual SSH configuration not need for Oracle 11gr2, because of During grid infrastructure It will ask SSH setup. But here I just configured ssh manually. Node 1 : [root@racha1 ~]# su - oracle [oracle@racha1 ~]$ mkdir ~/.ssh [oracle@racha1 ~]$ chmod 700 ~/.ssh [oracle@racha1 ~]$ /usr/bin/ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/oracle/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_rsa. Page 47

48 Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. The key fingerprint is: cf:36:68:1b:be:18:51:7d:7a:67:ee:b7:ee:8e:ca:0b Node 2: ~]# su - oracle [oracle@racha2 ~]$ mkdir ~/.ssh [oracle@racha2 ~]$ chmod 700 ~/.ssh [oracle@racha2 ~]$ /usr/bin/ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/oracle/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_rsa. Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. The key fingerprint is: 7b:69:c1:7d:7e:b1:e8:62:72:1d:3d:ee:9b:d6:26:c6 oracle@racha2.localdomain Node 1: [oracle@racha1 ~]$ cd ~/.ssh [oracle@racha1.ssh]$ cat id_rsa.pub >> authorized_keys [oracle@racha1.ssh]$ scp authorized_keys racha2:/home/oracle/.ssh The authenticity of host 'racha2 ( )' can't be established. RSA key fingerprint is fd:dd:51:77:0d:46:69:f8:55:fe:2c:53:77:75:90:e0. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'racha2, ' (RSA) to the list of known hosts. oracle@racha2's password: Page 48

49 authorized_keys 100% KB/s 00:00 Node 2: ~]$ cd ~/.ssh cat id_rsa.pub >> authorized_keys scp authorized_keys racha1:/home/oracle/.ssh The authenticity of host 'racha1 ( )' can't be established. RSA key fingerprint is fd:dd:51:77:0d:46:69:f8:55:fe:2c:53:77:75:90:e0. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'racha1, ' (RSA) to the list of known hosts. password: authorized_keys 100% KB/s 00:00 Check below for each node Node 1: ssh date racha1 ssh racha2 date ssh racha2-priv date ssh racha1-priv date Node 2: ssh date racha1 ssh racha2 date ssh racha2-priv date ssh racha1-priv date Cluvfy pre requisites check 11gR2 we would probably use the "runcluvfy.sh" utility in the clusterware root directory to check the prerequisites have been met. Page 49

50 grid]$./runcluvfy.sh stage -pre crsinst -n racha1,racha2 -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "racha1" Destination Node Reachable? racha1 yes racha2 yes Result: Node reachability check passed from node "racha1" Checking user equivalence... Check: User equivalence for user "oracle" Node Name Comment racha1 passed racha2 passed Result: User equivalence check passed for user "oracle" Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes Result: Clock synchronization check using Network Time Protocol(NTP) passed Pre-check for cluster services setup was successful. Oracle Grid Infrastructure installation [oracle@racha1 grid]$ Install Grid Infrastructure [root@racha1 ~]# chown -R oracle:oinstall /install/grid [root@racha1 ~]# xhost + access control disabled, clients can connect from any host [root@racha1 ~]# su - oracle Page 50

51 ~]$ cd /install/grid grid]$ ls doc response runcluvfy.sh sshsetup welcome.html install rpm runinstaller stage grid]$./runinstaller Starting Oracle Universal Installer... Checking Temp space: must be greater than 80 MB. Actual MB Checking swap space: must be greater than 150 MB. Actual 4031 MB Passed Passed Checking monitor: must be configured to display at least 256 colors. Actual Passed Preparing to launch Oracle Universal Installer from /tmp/orainstall _ am. Please grid]$ Page 51

52 Add Second node hostname and virtual ip name Page 52

53 Test SSH Connectivity, Here I was already configured SSH. Page 53

54 Choose ASM storage Select Disks, If its not show, choose change recovery path and add correct disk path here. These disks are should be oracle permission. Page 54

55 Page 55

56 Page 56

57 Run the above screenshot shown root script as a root user in both nodes like below. Node 1: [root@racha1 ~]# /u01/app/orainventory/orainstroot.sh Changing permissions of /u01/app/orainventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/orainventory to oinstall. The execution of the script is complete. [root@racha1 ~]# Node 2: [root@racha2 ~]# /u01/app/orainventory/orainstroot.sh Changing permissions of /u01/app/orainventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Page 57

58 Changing groupname of /u01/app/orainventory to oinstall. The execution of the script is complete. ~]# Node 1: ~]# /u01/app/11.2.0/grid/root.sh Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin... Copying oraenv to /usr/local/bin... Copying coraenv to /usr/local/bin... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed :43:57: Parsing the host name :43:57: Checking for super user privileges :43:57: User has super user privileges Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory LOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP Page 58

59 profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert Adding daemon to inittab CRS-4123: Oracle High Availability Services has been started. ohasd is starting CRS-2672: Attempting to start 'ora.gipcd' on 'racha1' CRS-2672: Attempting to start 'ora.mdnsd' on 'racha1' CRS-2676: Start of 'ora.gipcd' on 'racha1' succeeded CRS-2676: Start of 'ora.mdnsd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'racha1' CRS-2676: Start of 'ora.gpnpd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racha1' CRS-2676: Start of 'ora.cssdmonitor' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'racha1' CRS-2672: Attempting to start 'ora.diskmon' on 'racha1' CRS-2676: Start of 'ora.diskmon' on 'racha1' succeeded CRS-2676: Start of 'ora.cssd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'racha1' CRS-2676: Start of 'ora.ctssd' on 'racha1' succeeded ASM created and started successfully. DiskGroup DATA created successfully. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. CRS-2672: Attempting to start 'ora.crsd' on 'racha1' CRS-2676: Start of 'ora.crsd' on 'racha1' succeeded CRS-4256: Updating the profile Successful addition of voting disk 0cf9af f43bf110fdde04e726a. Successfully replaced voting disk group with +DATA. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group ONLINE 0cf9af f43bf110fdde04e726a (/dev/oracleasm/disks/disk1) [DATA] Located 1 voting disk(s). CRS-2673: Attempting to stop 'ora.crsd' on 'racha1' CRS-2677: Stop of 'ora.crsd' on 'racha1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'racha1' Page 59

60 CRS-2677: Stop of 'ora.asm' on 'racha1' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'racha1' CRS-2677: Stop of 'ora.ctssd' on 'racha1' succeeded CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'racha1' CRS-2677: Stop of 'ora.cssdmonitor' on 'racha1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'racha1' CRS-2677: Stop of 'ora.cssd' on 'racha1' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'racha1' CRS-2677: Stop of 'ora.gpnpd' on 'racha1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'racha1' CRS-2677: Stop of 'ora.gipcd' on 'racha1' succeeded CRS-2673: Attempting to stop 'ora.mdnsd' on 'racha1' CRS-2677: Stop of 'ora.mdnsd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.mdnsd' on 'racha1' CRS-2676: Start of 'ora.mdnsd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'racha1' CRS-2676: Start of 'ora.gipcd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'racha1' CRS-2676: Start of 'ora.gpnpd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racha1' CRS-2676: Start of 'ora.cssdmonitor' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'racha1' CRS-2672: Attempting to start 'ora.diskmon' on 'racha1' CRS-2676: Start of 'ora.diskmon' on 'racha1' succeeded CRS-2676: Start of 'ora.cssd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'racha1' CRS-2676: Start of 'ora.ctssd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'racha1' CRS-2676: Start of 'ora.asm' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'racha1' CRS-2676: Start of 'ora.crsd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.evmd' on 'racha1' CRS-2676: Start of 'ora.evmd' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'racha1' CRS-2676: Start of 'ora.asm' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.data.dg' on 'racha1' CRS-2676: Start of 'ora.data.dg' on 'racha1' succeeded CRS-2672: Attempting to start 'ora.registry.acfs' on 'racha1' CRS-2676: Start of 'ora.registry.acfs' on 'racha1' succeeded racha1 2011/07/25 14:57:56 /u01/app/11.2.0/grid/cdata/racha1/backup_ _ olr Preparing packages for installation... cvuqdisk Configure Oracle Grid Infrastructure for a Cluster... succeeded Updating inventory properties for clusterware Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 4031 MB Passed Page 60

61 The inventory pointer is located at /etc/orainst.loc The inventory is located at /u01/app/orainventory 'UpdateNodeList' was successful. ~]# Node 2: ~]# /u01/app/11.2.0/grid/root.sh ]Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/11.2.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Creating ] directory... Copying dbhome to ]... Copying oraenv to ]... Copying coraenv to ]... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed :35:41: Parsing the host name :35:41: Checking for super user privileges :35:41: User has super user privileges Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params LOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Page 61

62 Adding daemon to inittab CRS-4123: Oracle High Availability Services has been started. ohasd is starting CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node racha1, number 1, and is terminating An active cluster was found during exclusive startup, restarting to join the cluster CRS-2672: Attempting to start 'ora.mdnsd' on 'racha2' CRS-2676: Start of 'ora.mdnsd' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'racha2' CRS-2676: Start of 'ora.gipcd' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'racha2' CRS-2676: Start of 'ora.gpnpd' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racha2' CRS-2676: Start of 'ora.cssdmonitor' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'racha2' CRS-2672: Attempting to start 'ora.diskmon' on 'racha2' CRS-2676: Start of 'ora.diskmon' on 'racha2' succeeded CRS-2676: Start of 'ora.cssd' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'racha2' CRS-2676: Start of 'ora.ctssd' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.drivers.acfs' on 'racha2' CRS-2676: Start of 'ora.drivers.acfs' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.asm' on 'racha2' CRS-2676: Start of 'ora.asm' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'racha2' CRS-2676: Start of 'ora.crsd' on 'racha2' succeeded CRS-2672: Attempting to start 'ora.evmd' on 'racha2' CRS-2676: Start of 'ora.evmd' on 'racha2' succeeded Page 62

63 racha2 2011/07/26 09:41:40 /u01/app/11.2.0/grid/cdata/racha2/backup_ _ olr Preparing packages for installation... cvuqdisk Configure Oracle Grid Infrastructure for a Cluster... succeeded Updating inventory properties for clusterware Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 4031 MB Passed The inventory pointer is located at /etc/orainst.loc The inventory is located at /u01/app/orainventory 'UpdateNodeList' was successful. [root@racha2 ~]# We expect the verification phase to fail with an error relating to the SCAN, assuming you are not using DNS. Page 63

64 Page 64

65 Check Cluster status for both nodes ~]$ export GRID_HOME=/u01/app/11.2.0/grid ~]$ export PATH=/u01/app/11.2.0/grid/bin ~]$ crsctl check cluster -all ************************************************************** racha1: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** racha2: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ~]$ olsnodes racha1 racha2 ~]$ crs_stat -t -v Name Type R/RA F/FT Target State Host ora.data.dg ora...up.type 0/5 0/ ora...er.lsnr ora...er.type 0/5 0/ ONLINE ONLINE racha1 ONLINE ONLINE racha1 ora...n1.lsnr ora...er.type 0/5 0/0 ONLINE ONLINE racha2 Page 65

66 ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE racha1 ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE racha1 ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora...network ora...rk.type 0/5 0/ ONLINE ONLINE racha1 ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE racha1 ora.rac.db ora...se.type 0/2 0/1 ONLINE ONLINE racha1 ora...sm1.asm application 0/5 0/0 ONLINE ONLINE racha1 ora...a1.lsnr application 0/5 0/0 ONLINE ONLINE racha1 ora.racha1.gsd application 0/5 0/0 OFFLINE OFFLINE ora.racha1.ons application 0/3 0/0 ONLINE ONLINE racha1 ora.racha1.vip ora...t1.type 0/0 0/0 ONLINE ONLINE racha1 ora...sm2.asm application 0/5 0/0 ONLINE ONLINE racha2 ora...a2.lsnr application 0/5 0/0 ONLINE ONLINE racha2 ora.racha2.gsd application 0/5 0/0 OFFLINE OFFLINE ora.racha2.ons application 0/3 0/0 ONLINE ONLINE racha2 ora.racha2.vip ora...t1.type 0/0 0/0 ONLINE ONLINE racha2 ora...ry.acfs ora...fs.type 0/5 0/ ONLINE ONLINE racha1 ora.scan1.vip ora...ip.type 0/0 0/0 ONLINE ONLINE racha2 ~]$ Oracle 11gR2 database Installation Go to Node 1 Unzip Oracle 11g Software database. [oracle@racha1 install]$ unzip linux_11gr2_database_1of2.zip Page 66

67 install]$ unzip linux_11gr2_database_2of2.zip Database Installation ~]# xhost + access control disabled, clients can connect from any host [root@racha1 ~]# su - oracle [oracle@racha1 ~]$ cd /install/database [oracle@racha1 database]$ ls doc install response rpm runinstaller sshsetup stage welcome.html [oracle@racha1 database]$./runinstaller Starting Oracle Universal Installer... Checking Temp space: must be greater than 80 MB. Actual 9852 MB Passed Checking swap space: must be greater than 150 MB. Actual 3887 MB Passed Checking monitor: must be configured to display at least 256 colors. Actual Passed Preparing to launch Oracle Universal Installer from /tmp/orainstall _ pm. Please wait... Page 67

68 Page 68

69 Page 69

70 Page 70

71 Page 71

72 During Database creation, suppose to be you may face listener issues like below words cluster database creation requires default listener configured In this case, you need to create listener using netca from node 1. Page 72

73 Page 73

74 The below errors showing, we need to set Unique name in Node 2 environment and start dbconsole. However this is not issues, you can start further process. Page 74

75 Node 1: ~]# /u01/app/oracle/product/11.2.0/dbhome_1/root.sh Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/11.2.0/dbhome_1 Enter the full pathname of the local bin directory: [/usr/local/bin]: The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: Page 75

76 The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. Finished product-specific root actions. [root@racha1 ~]# Node 2: [root@racha2 ~]# /u01/app/oracle/product/11.2.0/dbhome_1/root.sh Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/11.2.0/dbhome_1 Enter the full pathname of the local bin directory: [/usr/local/bin]: The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: Page 76

77 Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. Finished product-specific root actions. [root@racha2 ~]# Two Node RAC Installation Successfully completed. Just Check Node 1: [root@racha1 ~]# su - oracle [oracle@racha1 ~]$ export ORACLE_SID=RAC1 [oracle@racha1 ~]$ sqlplus Page 77

78 SQL*Plus: Release Production on Tue Jul 26 13:00: Copyright (c) 1982, 2009, Oracle. All rights reserved. Enter user-name: / as sysdba Connected to: Oracle Database 11g Enterprise Edition Release Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> select name from v$database; NAME RAC SQL> select instance_name from v$instance; INSTANCE_NAME RAC1 SQL> create user test identified by test; User created. Page 78

79 SQL> grant connect,resource to test; Grant succeeded. SQL> conn test/test; Connected. SQL> create table emp(empno number); Table created. SQL> insert into emp values(001); 1 row created. SQL> insert into emp values(002); 1 row created. SQL> insert into emp values(003); 1 row created. SQL> insert into emp values(004); 1 row created. Page 79

80 SQL> commit; Commit complete. SQL> select * from emp; EMPNO Node 2: [root@racha2 ~]# su - oracle [oracle@racha2 ~]$ export ORACLE_SID=RAC [oracle@racha2 ~]$ export ORACLE_SID=RAC2 [oracle@racha2 ~]$ sqlplus SQL*Plus: Release Production on Tue Jul 26 13:06: Copyright (c) 1982, 2009, Oracle. All rights reserved. Enter user-name: / as sysdba Connected to: Oracle Database 11g Enterprise Edition Release Production Page 80

81 With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options ;SQL>select instance_name from v$instance; INSTANCE_NAME RAC2 SQL> conn test/test; Connected. SQL> select * from emp; EMPNO Page 81

82 More Details About RAC, my recommended book Page 82