The article performs the following task:-

Transcription

1 This RAC installation article is based on RAC Attack hands-on learning lab The article performs the following task:- 1. Deploying Oracle Grid Infrastructure 12cR1 ( ) 2. Deploying Oracle database software 12cR1 ( ) 3. Using Oracle ASM with udev rules Below the hardware details Windows 7 64 bit 8 CPUs 16Gb memory, 8GB is used for the RAC 50Gb for VDI (Dynamic Allocation) 30Gb for ASM disk (Fixed Size) Below are the software components The VirtualBox for Windows: The Oracle Linux (OEL 7.2 OS -- V iso) : The Oracle Database 12c R1 for Linux x86-64: 1. Complete the Installation of Oracle VM VirtualBox 2. Run VirtualBox Manager. 3. From the main screen, choose File > Preferences > Network. 4. Double click on VirtualBox Host-Only Ethernet Adapter. VirtualBox will suggest a default address of Please update the settings as shown and click Ok twice. 12cR1 RAC OEL 7.2 Page 1

2 IPv4 Address: IPv4 Network Mask: In VirtualBox, from the main screen, click the New icon in the upper left hand corner. 12cR1 RAC OEL 7.2 Page 2

3 6. Set the memory size. I have set to 4G. 7. Create virtual hard drive. 12cR1 RAC OEL 7.2 Page 3

4 8. Leave VDI selected and click next. 9. Leave dynamically allocated selected and click Next. 12cR1 RAC OEL 7.2 Page 4

5 10. Type in the full path or choose through the browse icon the Location for the disk file to be created. Have named the file racnode1.vdi and all the vdi files created are in location D:\rac12c\. 12cR1 RAC OEL 7.2 Page 5

6 11. From the main screen, select the virtual machine racnode1 and click the Settings icon in the upper left hand corner. 12cR1 RAC OEL 7.2 Page 6

7 12. Open the Storage sub-menu. Delete the Empty disk under the IDE Controller. 13. Click on the Add CD/DVD/Optical Device. 12cR1 RAC OEL 7.2 Page 7

8 14. Locate and open the Oracle_Linux_7_64.iso (V iso) file. 12cR1 RAC OEL 7.2 Page 8

9 15. Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to: dropdown to Host-only Adapter. 16. Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to Internal Network and type the name rac-priv in the Name field. 12cR1 RAC OEL 7.2 Page 9

10 17. Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to NAT. 18. Select the Shared Folders tab. Click Add Shared Folder. Enter the path where you have downloaded the Oracle installation media and enter a name for your folder. 12cR1 RAC OEL 7.2 Page 10

11 19. Open the USB sub-menu. Uncheck the Enable USB Controller check box. 20. Click on Start to boot the virtual machine. 12cR1 RAC OEL 7.2 Page 11

12 21. The Oracle Linux install screen will automatically come up. 22. Select the language and click on Continue. 12cR1 RAC OEL 7.2 Page 12

13 23. Next is the Installation Summary screen. 24. Select the Installation Destination. If you are happy with automatic partitioning of the whole disk, click Done button to return to previous screen. 12cR1 RAC OEL 7.2 Page 13

14 25. On Software Selection screen select Server with GUI for your Base Environment. Then select extra software you want to install by selecting it in Add-Ons for Selected Environment. For Oracle software installation and general development on Oracle Linux 7 please choose the below option and click Done button. 12cR1 RAC OEL 7.2 Page 14

15 26. Select the Network & Host Name section from Installation Summary. 12cR1 RAC OEL 7.2 Page 15

16 27. Select Ethernet (enp0s3), the Public IP and click on Configure. In earlier releases of OEL we had eth0, eth1, eth2, which is now enp0s3, enp0s8, enp0s9 a. Click the Connect Automatically checkbox in General Tab b. Click on IPv4 Settings and select Manual method. c. Click Add and type for the address and for Netmask d. Type , in DNS servers and racattack in search domain. e. In Hostname type racnode1.racattack. f. Click on Save 12cR1 RAC OEL 7.2 Page 16

17 28. Select Ethernet (enp0s8) and click on Configure a. Click on Connect Automatically checkbox in General Tab b. Move to IPv4 Settings, change to Manual method and click on Add c. Type in Address and in Netmask d. Save the settings 12cR1 RAC OEL 7.2 Page 17

18 29. Select Ethernet (enp0s9) and click on Configure a. Click on Connect Automatically checkbox in General Tab b. Move to IPv4 Settings, change to Automatic (DHCP) addresses only method 12cR1 RAC OEL 7.2 Page 18

19 12cR1 RAC OEL 7.2 Page 19

20 30. Click on Begin Installation and the screen changes to CONFIGURATION window. 31. Set the ROOT PASSWORD and USER CREATION while the installation is going on. 12cR1 RAC OEL 7.2 Page 20

21 32. Completion Screen: - After the Oracle Linux is successfully installed, go ahead and reboot it. After reboot you would need to complete License Information. 12cR1 RAC OEL 7.2 Page 21

22 12cR1 RAC OEL 7.2 Page 22

23 33. Turn off and disable the Firewall. systemctl and systemd have replaced chkconfig and init under OEL Change the setting of SELinux to disabled by editing /etc/selinux/config. 12cR1 RAC OEL 7.2 Page 23

24 35. Check the ntpd status. Either configure NTP, or make sure it is not configured so that CTSSd can synchronize the times of RAC node. In my installation I de-configured NTP. # systemctl disable ntpd # mv /etc/ntp.conf /etc/ntp.conf.orig 36. Verify that all the network interfaces are up. 12cR1 RAC OEL 7.2 Page 24

25 37. Configure DNS. 12cR1 RAC OEL 7.2 Page 25

26 Original /etc/named.conf file before any changes - Add to listen-on port 53 and /24 to allow-query and allowtransfer. 12cR1 RAC OEL 7.2 Page 26

27 At the end of file add the zone and reverse zone as shown below. Create the zone file for the racattack domain on racnode1. 12cR1 RAC OEL 7.2 Page 27

28 Create the reverse zone file Verify the changes made to /etc/named.conf file as shown below. In case of any errors, rectify it and re-check. 12cR1 RAC OEL 7.2 Page 28

29 Enabled the named service and started it Check the forward and reverse lookup using nslookup command. 12cR1 RAC OEL 7.2 Page 29

30 12cR1 RAC OEL 7.2 Page 30

31 38. Install the database preinstall package oracle-rdbms-server-12cr1-preinstall. Select the Devices > Optical Drives > oracle_linux_7_64.iso 12cR1 RAC OEL 7.2 Page 31

32 Run the following commands. Make a backup copy of file, delete the content and add the below lines to /etc/yum.repos.d/publicyum-ol7.repo. Install the oracle-rdbms-server-12cr1-preinstall package. It performs all prerequisite setup like installing required rpms, creating oracle user, required groups, setting parameters in /etc/sysctl.conf etc. 12cR1 RAC OEL 7.2 Page 32

33 One thing to note is, Oracle user limit does not reflect values in /etc/security/limit.conf if oraclerdbms-server-12cr1-preinstall RPM is used. The file to be checked is /etc/security/limits.d/oraclerdbms-server-12cr1-preinstall.conf. 12cR1 RAC OEL 7.2 Page 33

34 39. Unmount/Eject the device (oracle_linux_7_64.iso) and Make the VirtualBox guest additions available to the OS by clicking Devices->Install Guest Additions. 12cR1 RAC OEL 7.2 Page 34

35 40. After the guest addition, the Shared Folders added in step 18 should be visible. The folder is visible as /media/sf_12cr Create the directories in which the Oracle Software will be installed and add oracle to vboxsf group. 12cR1 RAC OEL 7.2 Page 35

36 42. Create Virtualbox Shared Storage (ASM Disks) a. Shutdown the server. b. In the VirtualBox Manager, select the machine racnode1, click Settings -> Storage. c. Select Controller SATA and click on the Add Hard Disk button. Click Create new disk. 12cR1 RAC OEL 7.2 Page 36

37 12cR1 RAC OEL 7.2 Page 37

38 43. Repeat the same steps to create three more disks: 44. Make all the asm devices sharable: from VirtualBox Manager, click File -> Virtual Media Manager. 12cR1 RAC OEL 7.2 Page 38

39 Repeat the same above steps for all the asm*.vdi disk. 12cR1 RAC OEL 7.2 Page 39

40 45. Start the racnode1 and you must see 4 disks named /dev/sd*. 46. Once the new disks are visible in the server, add primary partition on each of them using fdisk. 12cR1 RAC OEL 7.2 Page 40

41 47. Add the following to the "/etc/scsi_id.config" file to configure SCSI devices as trusted. Create the file if it doesn't already exist. # echo options=-g > /etc/scsi_id.config Check if scsi_id returns data. # /usr/lib/udev/scsi_id g u d /dev/sdb Prepare the /etc/udev/rules.d/99-oracle-asmdevices.rules by running the below script. #!/bin/bash #file ~/createoracleasmudevrules.sh i=1 # ol7 / rhel7 cmd="/usr/lib/udev/scsi_id -g -u -d" for disk in sdb sdc sdd sde ; do cat <<EOF >> /etc/udev/rules.d/99-oracle-asmdevices.rules KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="$cmd /dev/\$parent", \ RESULT=="`$cmd /dev/$disk`", SYMLINK+="asm-disk$i", OWNER="oracle", GROUP="dba", \ MODE="0660" EOF i=$(($i+1)) done 12cR1 RAC OEL 7.2 Page 41

42 Load updated block device partition tables. Test the rules are working as expected. Restart UDEV Service. 12cR1 RAC OEL 7.2 Page 42

43 48. Clone VirtualBox VM a. Shutdown the VM. # shutdown h now b. Clone the disk racnod1.vdi: from VirtualBox Manager, click File -> Virtual Media Manager. 12cR1 RAC OEL 7.2 Page 43

44 12cR1 RAC OEL 7.2 Page 44

45 Once the copy is complete, click Close to exit from the Virtual Media Manager. 12cR1 RAC OEL 7.2 Page 45

46 49. In VirtualBox, click New icon in the upper left corner to create the new VM. 12cR1 RAC OEL 7.2 Page 46

47 12cR1 RAC OEL 7.2 Page 47

48 50. Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to: dropdown to Host-only Adapter. 51. Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to Internal Network and type the name rac-priv in the Name field. 52. Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to NAT. 12cR1 RAC OEL 7.2 Page 48

49 53. Open the USB sub-menu. Uncheck the Enable USB Controller check box. 54. Select the Shared Folders tab. Click Add Shared Folder. Enter the path where you have downloaded the Oracle installation media and enter a name for your folder. 12cR1 RAC OEL 7.2 Page 49

50 55. Click on Controller SATA and Choose existing disk to add the asm disk vdi file. 56. Repeat the operation to add all asm disks and then click OK to save the modifications. 12cR1 RAC OEL 7.2 Page 50

51 57. Review the summary of new virtual machine. 58. Start the racnode2 and login as root. 12cR1 RAC OEL 7.2 Page 51

52 59. On the top right had side end you would see Ethernet setting. Choose one and click on Wired Settings. 60. Below is the screen-shot for enp0s3 (Public IP). Click on the right end bottom mark. 12cR1 RAC OEL 7.2 Page 52

53 61. Change the Address to for enp0s In the similar way change the Address to (Private-IP) for enp0s8. 12cR1 RAC OEL 7.2 Page 53

54 63. For enp0s9 uncheck Connect Automatically 64. On terminal as root type nmtui (NetworkManager Text User Interface) 65. Change the Hostname 12cR1 RAC OEL 7.2 Page 54

55 66. Delete the UUID entry for ifcfg-enp0s[389]. # sed i e /UUID/d ifcfg-enp0s[389] 12cR1 RAC OEL 7.2 Page 55

56 67. Reboot the server and then verify all the network interfaces are up and ASM devices are visible. 68. Check the firewall status (should be inactive) and stop the DNS. 12cR1 RAC OEL 7.2 Page 56

57 69. Delete the racattack.zone and racattack.revzone from /var/named. 70. Edit the /etc/named.conf and change.51 to.52 and the end of the file as below. 12cR1 RAC OEL 7.2 Page 57

58 71. Review the /etc/resolv.conf. 72. Start DNS and check the forward and reverse lookup using nslookup command. 12cR1 RAC OEL 7.2 Page 58

59 73. Check that both the master on racnode1 and slave on racnode2 DNS servers are working. Run all the below statements to verify correct configuration of both DNS servers :- ~]# racnode1.racattack ~]# racnode2.racattack ~]# racnode1-vip.racattack ~]# racnode2-vip.racattack ~]# racnode1-priv.racattack ~]# racnode2-priv.racattack ~]# rac-cluster-scan.racattack ~]# racnode1.racattack ~]# racnode2.racattack ~]# racnode1-vip.racattack ~]# racnode2-vip.racattack ~]# racnode1-priv.racattack ~]# racnode2-priv.racattack ~]# rac-cluster-scan.racattack 12cR1 RAC OEL 7.2 Page 59

60 74. When both nodes have started, check they can both ping all the public and private IP addresses using the following commands. 12cR1 RAC OEL 7.2 Page 60

61 75. Install the Grid Infrastructure a. Install the following package for the Oracle Grid media as root user on both the nodes. b. Setup the passwordless ssh for as Oracle user../sshusersetup.sh user oracle hosts racnode1 racnode2 nopromptpassphrase advanced exverify logfile /home/oracle/sshsetup.log c. At this point you can run the runcluvfy.sh (optional) 12cR1 RAC OEL 7.2 Page 61

62 d. Run the Grid Infrastructure runinstaller as Oracle grid]./runinstaller e. Below are the screen-shots for the configuration. 12cR1 RAC OEL 7.2 Page 62

63 12cR1 RAC OEL 7.2 Page 63

64 f. In the SCAN Name box, enter rac-cluster-scan.racattack. In the central box, only the first node is displayed. Click on Add and enter the network names of the second node. Public Hostname: - racnode2.racttack Virtual Hostname: - racnode2-vip.racattack Since the SCAN Name chosen is more than 15 character, Oracle proposes different name as Cluster Name. Click Yes. 12cR1 RAC OEL 7.2 Page 64

65 g. Check the correct addresses are set as below. h. Make sure the Software Location is correct and change Cluster Registry Storage Type to Oracle Automatic Storage Management. Enter SYSASM password and confirm. Click Yes. 12cR1 RAC OEL 7.2 Page 65

66 i. Set the redundancy to external, click the Change Discovery Path button and set the path to /dev/asm-disk*. I have changed the disk group name to DATA01.Once completed click on Next. 12cR1 RAC OEL 7.2 Page 66

67 j. If you want the root script to run automatically, enter the relevant credentials. I prefer to run them manually. 12cR1 RAC OEL 7.2 Page 67

68 k. Wait while the prerequisite checks complete. l. The below checks can be ignored for this type of installation. 12cR1 RAC OEL 7.2 Page 68

69 m. Save the response file and review the summary. Click the Install button. n. Installation progress 12cR1 RAC OEL 7.2 Page 69

70 o. When prompted, run the configuration scripts in each node. Once both the scripts are run on all the nodes, click OK. 12cR1 RAC OEL 7.2 Page 70

71 p. The installation continues and may take longtime to complete. MGMTDB creation may take some time. 12cR1 RAC OEL 7.2 Page 71

72 q. Optional step: - You can increase the timeout of CRS before it causes fencing of the node. 12cR1 RAC OEL 7.2 Page 72

73 r. You can check the crs patch details. 76. As oracle user add new diskgroup for FRA The asmca opens on the Disk Groups tab, showing the DATA01 Diskgroup created by the installer. Click on create and enter FRA in Disk Group Name. Choose External (node) redundancy. Check one disk to create 10GB diskgroup. Click OK 12cR1 RAC OEL 7.2 Page 73

74 77. After the diskgroup is created 78. Install Database Software a. You can use cluvfy to perform prerequisite checks (Optional) cluvfy stage pre dbinst n racnode1,racnode2 d /u01/app/oracle/product/ /db_1 -verbose 12cR1 RAC OEL 7.2 Page 74

75 b. Make sure all nodes are started and then login to racnode1 as oracle user and start the Oracle Installer. 12cR1 RAC OEL 7.2 Page 75

76 c. Make sure both nodes are selected, then click Next button 12cR1 RAC OEL 7.2 Page 76

77 12cR1 RAC OEL 7.2 Page 77

78 d. Make sure the Oracle Base and Software Location are as needed 12cR1 RAC OEL 7.2 Page 78

79 e. For my installation I ignored both the warnings. 12cR1 RAC OEL 7.2 Page 79

80 f. Review the summary information and click the Install button. g. Run the configuration script on each node. When the script has been run on each node, click OK button. 12cR1 RAC OEL 7.2 Page 80

81 79. Create Database a. Run the Database Configuration Assistance (DBCA) 12cR1 RAC OEL 7.2 Page 81

82 b. Global Database Name:- cdbrac, Pluggable Database Name:- PDB 12cR1 RAC OEL 7.2 Page 82

83 c. Review the Create Database Summary and Click Finish 12cR1 RAC OEL 7.2 Page 83

84 12cR1 RAC OEL 7.2 Page 84

85 12cR1 RAC OEL 7.2 Page 85