... Setting up IBM BladeCenter S to install Oracle Real Application Clusters 11g Release 1 with RSSM on Microsoft Windows Server 2008

Transcription

1 Setting up IBM BladeCenter S to install Oracle Real Application Clusters 11g Release 1 with RSSM on Microsoft Windows Server Christopher Chung IBM Oracle International Competency Center November 2009 Copyright IBM Corporation, All Rights Reserved. All trademarks or registered trademarks mentioned herein are the property of their respective holders

2 Table of contents Abstract...1 Overview of IBM BladeCenter S solution...1 Prerequisites...1 Setting up BladeCenter S for Oracle Real Application Clusters 11g Release Installing Microsoft Windows Server...1 Steps for Installing Microsoft Windows Server using ServerGuide... 3 Configuring Storage Configuration Manager (SCM)...17 Launching SCM to configure the Disk System...24 Prepare for Network...37 Oracle Cluster Ready Services...40 Automatic Storage Manager...46 Installing Oracle Real Application Clusters 11g Release Database Configuration Assistant (DBCA)...55 Summary...60 About the author...60 Appendix 1 More information about set up...61 Preparing for BladeCenter S Preparing for Oracle Clusterware installation Appendix 2 Pre-checks for cluster services setup report...64 Trademarks and special notices...66

3 Abstract This white paper documents the procedure that is used to install and configure Oracle Real Application Clusters 11g Release 1 on IBM BladeCenter S servers with a RAIDed Shared Storage Module (RSSM) and the Microsoft Windows Server operating system. Overview of IBM BladeCenter S solution The BladeCenter S SAS RAID controller module is the industry-first shared storage feature for the IBM BladeCenter, it brings enterprise-class storage capabilities to smaller firms and branch offices that lack the IT staff and budgets to manage expanding volumes of business information. This chassis not only provides datacenter-proven blade technology, highly energy efficient power supplies, integrated SAS or SATA storage, it also allows the storage array or hard drives to be networked together as a SAN which can be shared by the blades in the BladeCenter S. When combined with the RSSM, or RAIDed shared storage module, the BladeCenter S system can be an excellent choice to run Oracle Real Application Clusters with RSSM, which fulfills the Oracle Real Application Clusters requirement for shared storage. This white paper is meant to increase the readers understanding of how Oracle Real Application Clusters 11g Release 1 with the BladeCenter S can be deployed on Microsoft Windows Server. One possible example is shown. The example shown in this paper has not been extensively tested in a production environment. It is left to the reader to take this information and use it wisely in their own unique deployment environment. Prerequisites This document assumes knowledge of the Oracle Real Application Clusters installation process. Setting up BladeCenter S for Oracle Real Application Clusters 11g Release 1 The steps to prepare the BladeCenter S with RSSM for Oracle Real Application Clusters 11g Release 1 are: 1. Install Microsoft Windows Server 2. Configure Storage Configuration Manager 3. Prepare for the Oracle Real Application Clusters 11g Release 1 installation 4. Install CRS software 5. Install Oracle Real Application Clusters 11g Release 1 software Installing Microsoft Windows Server IBM ServerGuide is a tool that steps through the process of installing Microsoft Windows Server operating systems. It provides device drivers and other system components based on detecting installed options. The built-in intelligence of ServerGuide recognizes machine types and models as well as ServeRAID adapters and other hardware criteria. Based on the gathered hardware information, ServerGuide offers installation and configuration choices tailored to your system. 1

4 The steps are: 1. Set the server date and time: ServerGuide provides a Date and time selection panel so you don t have to use the server setup (F1) to access these settings. 2. Clear hard disk drivers: If your server has existing partitions, you can select to remove all of the partitions on all the attached hard disk drives, if the server also has a SeverRAID configuration, you can return the configuration to factory-default settings. 3. On figure SeverRAID controllers and adapters: If the server has an IBM SeverRAID controller or adapter, or an integrated SCSI controller with RAID capabilities, you can configure the server for a RAID implementation (SeverRAID Manager is available in English only). 4. Create and format Network Operating System partitions: Based on the NOS you are installing, ServerGuide assists you in selecting the file system type and partition size. If you did not clear hard disk drives, this task must create a new NOS partition. If partitions are on the startup hard disk drive, these partitions are removed. Note: This task is not available if you click Other network operating system in the Operating System Version panel. 5. View the Server summary report: After the setup process completes, ServerGuide provides a summary of your server configuration and hardware settings. If you want to make changes to your selections, you can return to the applicable panels in the ServerGuide wizard. 6. Install the device drivers and network operating system: If you are installing Microsoft Windows Servers, ServerGuide can install device drivers based on your server configuration and NOS. ServerGuide provides device drivers for a wide range of adapters. The wizard stores the information about the server model, service processor, hard disk controllers, and network adapters. Then, ServerGuide checks the ServerGuide Setup and Installation CD for newer device drivers required for your server. Also, information about your server is stored and later passed to the NOS installation program. 7. Additionally, ServerGuide assists you in gathering general network settings, network adapter configuration information, network protocol information, and IP address assignments, as well as display, time zone, and other Windows selections. Then, using the Widows CD that you provide, ServerGuide installs Microsoft Windows Server using the selections and information you provided. Note: For additional information about your server model, see the documentation included with your IBM server. Servers announced since early 2009 no longer ship with ServerGuide CDs. This applies to BladeCenter blade models. However you can download the latest ServerGuide code from the IBM website at: GUIDE&brandind=

5 Steps for Installing Microsoft Windows Server using ServerGuide This section shows the panels for the procedure to install Microsoft Windows Server using the System Management tool, ServerGuide. Press the media tray assignment button on the blade to assign the DVD drive to this server. Insert the ServerGuide into the DVD drive, and boot the blade server from the DVD. The ServerGuide Wizard starts. Reboot the computer. When prompted for an installation language and other regional options, make your selection and press Next. 3

6 Accept the program licensing agreement. 4

7 Review steps the SeverGuide would go through. Select the network operating system and version to install. 5

8 Set date and time. Configure RAID adapter. 6

9 Bypass RAID configuration. A partition must be created and formatted. 7

10 Review summary report. Microsoft Windows Server installation preparation begins by copying necessary files. 8

11 Insert the oprerating system CD/DVD. ServerGuide is searching for specified NOS in CD/DVD. 9

12 ServerGuide has finished the preparation phase and unattended installation of Microsoft Windows Server begins. When prompted for an installation language and other regional options, make your selection and press Next. 10

13 Accept the license terms and press Next. The installation now begins, extracting, uncompressing, and copying the files takes about 20 minutes. Microsoft Windows Server takes up approximately 10 GB of hard drive space. 11

14 The installation process will reboot your computer, after the reboot you'll be prompted with the new Microsoft Windows Server type of login screen. Press CTRL+ALT+DEL to log in. Click Other user. 12

15 The default Administrator is blank, provide with a password at least 7 characters long press Enter. Once you're logged on, you will be greeted by an assistant for the initial server configuration, and after performing some initial configuration tasks you will be able to start working. 13

16 At this time, this is a good point to apply Windows updates from Microsoft. Go to Windows Update in Control Panel for Internet Explorer or programs that ship with Windows. Updates are additions to software that can help prevent or fix problems, improve how your computer works, or enhance your computing experience. You need to restart the server since Windows can t update files and services while the system is using them. 14

17 The next important task is to activate the Windows license by clicking Activate Windows now. Once you have activated the license, you will see the following screen. 15

18 You can see an icon in the bottom right corner indicating the product is genuine Microsoft software. Like any other server, server core machines must be properly configured to be able to communicate on your network. Some of these settings include: configuring an IP address configuring an administrator's password configuring a server name enabling remote MMC snap-in management enabling remote RDP connections enabling remote Windows Firewall management enabling remote shell management activating the server joining a domain configuring Windows Updates configuring error reporting adding server roles and features Once you have configured these setting, the Microsoft Windows Server installation is complete and you are ready to set up the RSSM. 16

19 Configuring Storage Configuration Manager (SCM) To start this step, go to the directory where the installation package was placed. Execute setupwin32.exe by double clicking to start the installer Press Next. 17

20 Accept the terms in the license agreement and press Next. Stand alone SCM installation will be implemented as in our case to manage all devices including the IBM ServerRAID MR Controller, press Next. 18

21 Specify the installation path for SCM and press Next. It s important to understand that if you install on a non-supported platform, the Manager and Agent for the SAS RAID Controller Module will not be installed. 19

22 Choose the BladeCenter S installation option and click Next. Default ports can be overridden or you can accept the default port values, press Next. 20

23 The Installation summary report is shown. Check mark the Create Shortcut to SCM in Program menu option. The Installation summary report display continues. Press Install. 21

24 The Installation Progress panel will appear. Make sure reboot the system after successful installation to allow correct operation of SCM. 22

25 At this point, the system requires a reboot for correct operation. Press Finish. 23

26 Launching SCM to configure the Disk System To support the Oracle Database with Real Application Clusters, your database files must be stored on disks that are shared by both nodes in your cluster. The shared disks must be attached to both nodes in your cluster and both nodes must be able to read and write to them. This section will show how to define and configure two shared volumes for the database files. There are two ways you can launch SCM locally: Start Program IBM Storage Configuration Manager IBM Storage Configuration Manager Open a web browser with To login from remote server, replace localhost with a fully qualified hostname or IP address. For example: You must log in before you can access IBM Storage Configuration Manager. In order to log into IBM Storage Configuration Manager: In the User ID field, enter your operating system user ID. Notes: You must be in the Administrator group of the operating system in order to access RAID controller functions. If you install IBM Storage Configuration Manager on a Windows server that is configured to more than one domain, you must log in with a user ID in the following format: username@domainname, where username is your operating system user ID and domainname is your domain name. In the Password field, enter your operating system password. Click Log In. After a failed log in attempt, you can return to the Log In page by clicking OK. The Welcome page is the first page that is displayed after you log in to the IBM Storage Configuration Manager. From the Welcome page, click a link in the navigation panel to view a particular page, or under Getting Started, click the twistie next to BC S RAID SAS Module and then go directly to the Initial Setup Wizard, the All Resources, Storage, or Physical View pages by clicking the links in the center of the Welcome page. 24

27 The Initial Set up Wizard will lead you through the steps required to get up and running with IBM BladeCenter S SAS RAID modules chassis. You will add the chassis to IBM Storage Configuration Manager (SCM), set it up and configure logical storage. To get started with the BladeCenter S SAS RAID Modules, it is necessary to add SAS RAID modules to the Storage Configuration Manager. This is a required first task. You cannot do anything until you add modules to the Storage Configuration Manager. Use this page to add SAS RAID Modules. 25

28 26

29 In the Configuration section for the IBM BladeCenter S SAS RAID Controller Module, use the storage management tasks to create and allocate logical storage and disk drive management. You can create volumes from either the existing pool or you can create a new pool. In our case, we created a storage pool using the advanced method. Based on RAID level, select drives and add to the new storage pool until the required capacity is reached. In our case, the RAID level is 5, therefore minimum of 3 disks are used. Note: Storage pool names are treated as case insensitive and must be unique. 27

30 In our case, the RAID level is 5 and 3 disks are used out of 5 total disks as a minimum requirement. A newly created storage pool indicates functioning normally. 28

31 Define Volumes and add them to the list of volumes to be created. Hosts can be mapped to volumes using the Storage configuration wizard or stand-alone page. 29

32 In order to access the volumes from a host, a volume mapping must be created for that host. A confirmation of configuration is shown, it can be either accepted by selecting Finish or you can go back to make any required changes. 30

33 Once the Finish button is clicked the remaining wizard steps will start executing. When complete, you will see a summary of all tasks again, this time with completion status. After the second pool with its associated volumes has been defined and created, you can view all volumes that have been configured under POOL2, and there are other actions available from this page: Properties - this option is enabled when a single volume is selected. This task launches the volume properties page with the general tab selected. Expand Volumes - this option is enabled when one or more volumes are selected. Unmap Hosts - this option is enabled when a single volume is selected and that volume is mapped to a defined host. Delete Volumes - this option is enabled when one or more volumes are selected. 31

34 You can view currently defined storage pools information and there are other actions available from this page: Properties - this option is enabled when a single volume is selected. This task launches the volume properties page with the general tab selected. Expand Storage Pools - this option is enabled when a valid pool is selected. A pool must have a RAID level other than RAID 0 in order to be valid. Delete Storage Pool - this option is enabled when one pool that you would like to delete is selected. Assign Global Hot Spare - this option is always enabled. Assign Dedicated Hot Spare - this option is enabled when a single pool that is not RAID 0 is selected. Unassign Dedicated Hot Spare - this option is enabled when a single pool is selected and that pool has a dedicated hot spare. 32

35 You can view all available hosts, blades that are currently in the chassis, and there are other actions available from this page: Properties - this option is enabled when a single host is selected. This task launches the host properties page with the general tab selected. Unmap Volumes - this option is enabled when a single host or host port is selected. This link launches the host properties page with the Mapped Volumes tab active. Discover New Hosts - this option is always enabled. Create Host - this option is always enabled. Edit Host name - this option is enabled when a single host port is selected and launches the Edit Host Name page. 33

36 You can view information about all available disks, and there are other actions available from this page: Properties - this option is enabled when a single drive is selected. This task launches the drive properties page with the general tab selected. See Disk Properties for more information. Assign as a Dedicated Hot Spare - this option is enabled when a single unconfigured or global hot spare drive is selected and storage pools exist. Unassign as Hot Spare - this option is enabled when a single global or dedicated hot spare drive is selected. Migrate Data - this option is enabled when a single configured drive is selected. Oracle Clusterware provides overall management of the cluster activities. Oracle Clusterware requires that Oracle Cluster Registry and Voting disks be located in logical drives on shared disks. Once all needed volumes are defined and mapped to a given host, the next step is to configure the volumes within Microsoft Windows Server. To open Disk Management, you can start by running Diskmgmt.msc from a command line, or Server Manager console, in the Storage section. 34

37 The new disks need to set as online first, and then initialized one disk at a time. In order to set an offline disk to online, right-click the disk you want to bring online, and select Online from the Action menu. Once online, right-click on the disk icon, and choose the Master Boot Record (MBR) option as the partition style. When using Disk Management, a regular partition on a basic disk is called a simple volume and this is the type of volume we want to use on all our volumes. The New Volume Wizard for that specific type of volume will open. Adjust the size of the new volume. In our case, we used a default value, which is the maximum available space from each of the selected disks. Select a drive letter for the new volume. Select the formatting option as a raw device by clicking on the Do not format this volume option and click Next. Even when mounting the volume rather than creating a new drive, you can choose your format type without regard to the underlying format of the mount point. Click Next. On the confirmation page, if all the options are correct, click Finish to create and format the volume. 35

38 In order to create Logical Partitions in Windows, follow the steps below: From the command prompt, type the diskpart command. Under DISKPART prompt, select the disk to create extended partition as: DISKPART> select disk <disk_number> DISKPART> create partition extended Proceed to Computer Management and Disk Management, then select the disk, and right-click. Set as New simple partition and assign a Drive letter. Select Do not format option and click Finish. Repeat the same for all the partitions in the list. Once this is done, the disks will be listed as Logical Drive as shown in the figure below 36

39 Prepare for Network The two nodes in the cluster must be able to communicate with each other and with external clients using the TCP/IP protocol. Communication between clients and the nodes in the cluster is across the public network. Both nodes need a network adapter configured for the public network. To enable availability and failover, a virtual IP (VIP) address is also required for each of your nodes. A VIP address can be moved between nodes in case of a failure. CRS manages the VIP addresses for you. To support a virtual IP address, both nodes require an unused IP address that is compatible with the public network's subnet and netmask. VIP address should not be attached to a physical network interface at OS level. Only Public and Private addresses must be reported by ipconfig command but not the VIP address. The virtual IP address and host name should also be registered in the domain name system (DNS). When RAC initializes, it will automatically add the VIP to each machine and the RAC will manage it. For communication between the instances running on the two nodes, a private network is required. This private network connects only the nodes in the cluster and cannot be accessed from outside the cluster. By issuing the ipconfig command in the DOS windows, you can determine the IP addresses and names for two public and two private IP addresses. Windows IP Configuration Ethernet adapter Local Area Connection 2: Connection-specific DNS Suffix. : Link-local IPv6 Address..... : fe80::8c30:9822:e9e1:64d4%11 IPv4 Address : Subnet Mask : Default Gateway : Ethernet adapter Local Area Connection: Connection-specific DNS Suffix. : Link-local IPv6 Address..... : fe80::d056:c353:a89f:d51%10 IPv4 Address : Subnet Mask : Default Gateway :

40 Windows IP Configuration Ethernet adapter Local Area Connection 2: Connection-specific DNS Suffix. : Link-local IPv6 Address..... : fe80::c07b:ce62:eeef:59de%11 IPv4 Address : Subnet Mask : Default Gateway : Ethernet adapter Local Area Connection: Connection-specific DNS Suffix. : Link-local IPv6 Address..... : fe80::39de:b339:522f:c811%10 IPv4 Address : Subnet Mask : Default Gateway : You can confirm that the private IP addresses are listed in the hosts file, which is in the C:\Windows\System32\drivers\etc directory. # Copyright (c) Microsoft Corp. # # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. # # This file contains the mappings of IP addresses to host names. Each # entry should be kept on an individual line. The IP address should # be placed in the first column followed by the corresponding host name. # The IP address and the host name should be separated by at least one # space. # # Additionally, comments (such as these) may be inserted on individual # lines or following the machine name denoted by a '#' symbol. # # For example: # # rhino.acme.com # source server # x.acme.com # x client host localhost ::1 localhost BCS1WS #Oracle cluster node BCS2WS #Oracle cluster node BCS1WS-priv #Oracle cluster node 1 Private BCS2WS-priv #Oracle cluster node 2 private BCS1WS-vip #Oracle cluster node 1 virtual BCS2Ws-vip #Oracle cluster node 2 virtual Note: Virtual IP is listed in the host file, it can neither be pinged nor reported by ipconfig command before the installation of CRS; VIP can be pinged and also reported by ipconfig only after the installation of CRS. You can confirm that the two nodes can actually communicate by executing a ping test using the IP address or alias name for the other node s private IP address and public IP address. Or you can run runcluvfy (Cluster Verification Utility), which is a part of pre-checks for cluster services setup on the node connectivity report, and you can see the public and private network interface names, the public and virtual IP addresses and the private IP address for both nodes. 38

41 Interface information for node "bcs1ws" Name IP Address Subnet Gateway HW Address MTU Local Area Connection On-link 00:1A:64:5A:89:8E 1500 Local Area Connection On-link 00:1A:64:5A:89: Interface information for node "bcs2ws" Name IP Address Subnet Gateway HW Address MTU Local Area Connection On-link 00:1A:64:5A:88: Local Area Connection On-link 00:1A:64:5A:88:

42 Oracle Cluster Ready Services This section will describe how to install and set up Oracle Real Application Clusters 11g Release 1 on a Microsoft Windows Server two-node cluster with Oracle Automatic Storage Management. There will be two phases in this installation: install and configure the Cluster Ready Services (CRS) software, and then the installation of the Oracle Real Application Clusters 11g Release 1 software. Run setup.exe from the Oracle Cluster Ready Services CD-ROM and select the Oracle Clusterware option. 40

43 Take the default location or specify File Location page, enter the name of the Oracle home and directory. Then press Next. The installer verifies the envrionment meets all of the minimum requirements for installing and configuring the Oracle Clusterware. 41

44 Enter a cluster name and public/private node name for both nodes. Then press Next. Select Public for the first Local Area Connection interface and Private for the Local Area Connection 2 interface and the press Next. 42

45 Designate each disk in the list where they are being used for storing the OCR and voting disk. A list of disks can be viewed in the Disk Management screen. 43

46 Configure one of three CFS partions. Configure one of two OCR disks. 44

47 Configure one of three CSS Voting disks. Review the list of products being installed and press Install. 45

48 Automatic Storage Manager To enable disk discovery during the database installation, the logical drives used to store the database files must be stamped with an ASM header. To stamp the logical drives for ASM, run the asmtoolg.exe in the $ORACLE_HOME\BIN directory and the following asmtool operation screen appears. Accept the default option as Add or change label and press Next. Select disk(s) to stamp and optionally modify the prefix. Press Next. 46

49 Press Next. Press Finish. 47

50 Under the ASM link name column below, disk names created by the tool begins with the prefix ORCLDISK for identification purposes. 48

51 Installing Oracle Real Application Clusters 11g Release 1 Run setup.exe from the Oracle Database 11g Release 1 CD-ROM. Select the Configure Automatic Storage Management option and press Next. 49

52 Press Select All for both nodes to be managed by ASM diskgroups. Press Next. Provide the password for SYS and choose the option of Create initialization parameter file with the file location information. Press Next. 50

53 A message appears informing that DBCA will create and start the ASM instance. Press OK. The ASM Instance Creation panel appears, then the ASM Disk Group page. 51

54 Press the Create New button on the ASM Disk Groups page. On the Create Disk group panel, select Show Candidates, then select the disks with the \\.\ORCLDISKDAA0 and \\.\ORCLDISKDATA1 path, and press OK. 52

55 On the Create Disk Group panel, enter DATA as Disk Group Name. Press Stamp Disks only if the disks never have been used for ASM. The DATA Disk Group Name has been created and we are ready to create second Disk Group. 53

56 On the Create Disk Group panel, enter FLASH as Disk Group Name for the last disk group. Press Stamp Disks only if the disks never have been used for ASM. The FLASH Disk Group Name has been created, next press Finish to wrap up the ASM Disk Group creation. 54

57 Database Configuration Assistant (DBCA) The Oracle Universal Installer was used to install the Oracle Database 11g Release 1 software and create a database. Run setup.exe from the Oracle Database 11g Release 1 CD-ROM and once the Select a Product to install panel shows up, select the Oracle Database 11g Release 1 option and press Next. Select the Enterprise Edition option and press Next. 55

58 Specify a base location for Oracle software and configuration related files. Press Next. Provide the address which will be informed of security issues, install the product and initiate configuration manager. Press Next. h 56

59 The Cluster Installation mode is selected by default. Select the additional node that is to be part of this installation and press Next. Press Next once product specific prerequestite checks complete successfully. 57

60 Select the Install Software Only option and press Next. Review the list of software components to be installed. Press Install. 58

61 The install progress panel appears. When the installation completes, press Exit. 59

62 Summary This paper provides information on installing Microsoft Windows Server and setting up the RSSM module for use with Oracle Real Application Clusters 11g Release 1. It includes the panels to install OCR, ASM and to run DBCA. About the author Christopher Y. Chung is an IBM Power Systems and IBM System x Advanced Technical Specialist at the IBM Oracle International Competency Center (ICC) in Pleasanton, California. He has been working as an on-site specialist at the ICC for over 10 years. His experience includes producing technical papers, performing benchmarks with major RDBMS platforms and developing tuning guides for PeopleSoft applications on multiple IBM hardware platforms. Christopher also provides advanced technical support for PeopleSoft Enterprise with Power Systems and System x to technical field specialists worldwide. He has responsibility for managing and administrating IBM hardware/software systems which includes technical support for Oracle performance and benchmark engineers on Oracle s PeopleSoft campus. The author wants to thank and acknowledge the contribution of Jayashree P RamaRao and Prasad U Inchal from IBM India. They completed a detailed technical review of the first edition of this document and provided input to improve the usability of the text for the readers. 60

63 Appendix 1 More information about set up Since Oracle Real Application Clusters require shared disk storage among blades, the new storage technology for BladeCenter S called RAIDed shared storage module (RSSM) is required to provide the necessary hardware support for Oracle Real Application Clusters 11g Release 1. For BladeCenter S chassis set up, please refer to Implementing the IBM BladeCenter S Chassis which can be downloaded at There is also a BladeCenter Product Publications Quick Reference document that can provide very useful information. It can be found at: _136620&cmibm_em=dm:0: Preparing for BladeCenter S The SAS RAID Controller Module Firmware must be updated using either the Storage Configuration Manager procedure, or the Command Line Firmware Update procedure: Storage Configuration Manager Software Download Link: Software and device drivers - IBM Storage Configuration Manager Storage Configuration Manager Firmware Package Link: SAS RAID Controller Firmware Update Package (Storage Configuration Manager) Command Line Firmware Update Download Package and Procedure: SAS RAID Controller Firmware Update Package (Command Line) The main SAS RAID Controller Module Web page includes links to all the information required for the setup process: SAS RAID Controller Model Web page Preparing for Oracle Clusterware installation 1. Make sure that the blade servers are compatible with BladeCenter S. Please refer to the SAS RAID Module Interoperability Guide for more information. 2. Physically install SAS daughter cards on all blades that will be sharing the disks for Oracle Clusterware. 3. Update all BIOS and firmware on blades, SAS daughter cards and Ethernet switches. Details for version and download locations are included in the SAS RAID Module Interoperability Guide. The following link is for the SAS Expander Card Drives specifically. 4. Install RSSM and perform RSSM firmware updates using either Storage Configuration Manager (SCM) or Command Line (CLI). a. SCM (or the latest version) can be downloaded from this link (note that SCM will NOT update disk drive module firmware). b. The SCM firmware package can be found at this link. Firmware included in the package: i. SAS RAID Module Firmware ii. SAS Switch Module Firmware 61

64 iii. Battery Backup (BBU) Firmware iv. Disk Storage Module (DSM) Firmware c. Command Line (CLI includes driver firmware). Command Line software / firmware package (one package for both) available at this link. Firmware included in the package: v. SAS RAID Module Firmware vi. SAS Switch Module Firmware vii. Battery Backup (BBU) Firmware viii. Disk Storage Module (DSM) Firmware ix. Disk Drive Module (DDM) Firmware 5. Install Microsoft Windows Server on all blade servers by using ServerGuide for your appropriate blade models. This link provides you with the appropriate ServerGuide to download based on your blade models and operating systems. For this paper we have downloaded version v This ServerGuide assists with the configuration of blades and installation processes of the operating system. It will download the latest system device drivers and other Ethernet drivers needed for the operating system. This version supports most BladeCenter chassis and System x servers with 64-bit or 32-bit Windows operating systems. Note: If you need to configure the blade servers for installing blade server operating systems on the volumes (LUNs) created in the BladeCenter SAS RAID Controller Module (RSSM), please follow the instructions in the manual SAS RAID Module Detailed Host Attachment Guide (Remote Boot Included). 6. Download and install Microsoft Windows Hot Fixes; you can find this information in the SAS RAID Module Interoperability Guide or on the Subsystem Device Driver Device Specific Module (SDDDSM) download page. 7. Install the SDDDSM package for RSSM on the Windows blade server. This package is based on Microsoft MPIO technology and it is specifically designed to support IBM storage devices and particularly the multipath configuration of IBM storage devices. It is recommended that you follow this link to get the most current information about the latest release of SDDDSM. The main functions of SDDDSM are as follows: i. Enhanced data availability ii. Dynamic I/O load-balancing across multiple paths iii. Automatic path failover protection iv. Concurrent download of licensed internal code v. Path-selection policies for the host system Links and Information can also be found in the SAS RAID Module Interoperability Guide. This is a direct link to the SDDDSM drivers web page. Note: If SDD is installed it needs to be uninstalled prior to installing SDDDSM. In addition, MPIO is shipped with SDDDSM for Windows Server. 8. The IBM BladeCenter S chassis has default zoning configured allowing all blade servers to communicate with all disks. It is recommended to maintain the default zone settings either in SCM or Advanced Management Module (AMM). Note: Zoning for the IBM BladeCenter S SAS RAID Controller Module is set up automatically. Altering the zoning disables communication between the storage modules and blade servers. 9. Create logical configuration on RSSM storage using SCM. a. Create storage volumes to be shared by Oracle Real Application Clusters nodes. b. Discover the hosts in SCM and map those volumes to the blade server. This can also be performed using the RSSM CLI. 62

65 63

66 Appendix 2 Pre-checks for cluster services setup report C:\ToolBox\oracle11g\database>runcluvfy stage -pre crsinst -n bcs1ws,bcs2ws -verbose 1 file(s) copied. C:\Windows\temp\cvtmphome\oui\lib\win32\msvcp71.dll C:\Windows\temp\cvtmphome\oui\lib\win32\msvcr71.dll 2 file(s) copied. Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "BCS1WS" Destination Node Reachable? bcs2ws yes bcs1ws yes Result: Node reachability check passed from node "BCS1WS". Checking user equivalence... Check: User equivalence for user "Administrator" Node Name Comment bcs1ws passed bcs2ws passed Result: User equivalence check passed for user "Administrator". Checking administrative privileges... Administrative privileges check passed. Checking node connectivity... Interface information for node "bcs1ws" Name IP Address Subnet Gateway Def. Gateway HW Address MTU Local Area Connection On-link UNKNOWN 00:1A:64:5A:89:8E 1500 Local Area Connection On-link UNKNOWN 00:1A:64:5A:89: Interface information for node "bcs2ws" Name IP Address Subnet Gateway Def. Gateway HW Address MTU Local Area Connection On-link UNKNOWN 00:1A:64:5A:88: Local Area Connection On-link UNKNOWN 00:1A:64:5A:88: Check: Node connectivity of subnet " " Source Destination Connected? bcs1ws:local Area Connection 2 bcs2ws:local Area Connection 2 yes Result: Node connectivity check passed for subnet " " with node(s) bcs1ws,bcs2ws. Check: Node connectivity of subnet " " Source Destination Connected? Check: Node connectivity of subnet " " 64

67 Source Destination Connected? bcs1ws:local Area Connection 2 bcs2ws:local Area Connection 2 yes Result: Node connectivity check passed for subnet " " with node(s) bcs1ws,bcs2ws. Check: Node connectivity of subnet " " Source Destination Connected? bcs1ws:local Area Connection bcs2ws:local Area Connection yes Result: Node connectivity check passed for subnet " " with node(s) bcs1ws,bcs2ws. Interfaces found on subnet " " that are likely candidates for VIP: bcs1ws Local Area Connection: bcs2ws Local Area Connection: Interfaces found on subnet " " that are likely candidates for a private interconnect: bcs1ws Local Area Connection 2: bcs2ws Local Area Connection 2: Result: Node connectivity check passed. Checking system requirements for 'crs'... Check: Operating system version Node Name Available Required Comment bcs1ws Windows Server Windows Server passed bcs2ws Windows Server Windows Server passed Result: Operating system version check passed. Check: Total memory Node Name Available Required Comment bcs1ws 3.25GB ( KB) 1GB ( KB) passed bcs2ws 3.25GB ( KB) 1GB ( KB) passed Result: Total memory check passed. Check: Swap space Node Name Available Required Comment bcs1ws 6.68GB ( KB) 1GB ( KB) passed bcs2ws 6.68GB ( KB) 1GB ( KB) passed Result: Swap space check passed. Check: System architecture Node Name Available Required Comment bcs1ws 32-bit 32-bit passed bcs2ws 32-bit 32-bit passed Result: System architecture check passed. Check: Free disk space in "C:\Users\ADMINI~1\AppData\Local\Temp\2" dir Node Name Available Required Comment bcs1ws 45.65GB ( KB) 400MB (409600KB) passed bcs2ws 46.22GB ( KB) 400MB (409600KB) passed Result: Free disk space check passed. System requirement passed for 'crs' Pre-check for cluster services setup was successful. 65

68 Trademarks and special notices Copyright. IBM Corporation All rights reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, BladeCenter, Power Systems, and System x are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both: Microsoft, Windows, Windows Server, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. The information provided in this document is distributed AS IS without any warranty, either express or implied. The information in this document may include technical inaccuracies or typographical errors. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. Any references in this information to non-ibm Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. 66