Best Practices When Deploying Microsoft Windows Server 2008 R2 or Microsoft Windows Server 2008 SP2 on HP ProLiant DL980 G7 Servers

Transcription

1 Best Practices When Deploying Microsoft Windows Server 2008 R2 or Microsoft Windows Server 2008 SP2 on HP ProLiant DL980 G7 Servers Technical white paper Table of contents Introduction... 2 OS Support Matrix for the HP ProLiant DL980 G7 Server... 3 Hyper-V Support Matrix... 4 Hyper-threading + Windows Server 2008 R2 + Hyper-V version Additional Software Components Recommended When Running Windows on the DL HP Smart Update CD... 5 HP ProLiant DL980 System Providers (for Windows Server 2008 R2 Users Only)... 6 Hardware Best Practices for the DL980 When Running Windows... 8 Boot Adapter Location... 9 Power Consumption Settings... 9 Windows Server 2008 R2 Best Practices for the DL Enable Hyper-threading to Maximize Resource Usage for Large Applications (but NOT for Hyper-V) Hyper-threading + Windows Server 2008 R2 + Hyper-V version Enable 44-bit Mode after OS Installation Install Microsoft Hotfix on Systems with More Than 1 TB RAM Workaround When Enabling Hyper-V on Systems with More Than 512 GB RAM Windows Server 2008 SP2 Best Practices for the DL Disable Hyper-threading to Keep Logical Processor Count at 64 Maximum Hyper-threading + Windows Server 2008 SP2 + Hyper-V version Workaround for Possible Driver Conflict When Logging in Using ilo For more information... 20

2 Introduction This purpose of this white paper is to facilitate the successful deployment and configuration of Microsoft Windows Server 2008 R2 and Windows Server 2008 Service Pack 2 (SP2) operating systems on HP ProLiant DL980 G7 servers. It includes a description of additional recommended HP software components for the DL980, and best practices you can use when deploying Microsoft Windows on the DL980. Topics include: Operating System support matrix Hyper-V support matrix Additional software components recommended Hardware best practices Windows Server 2008 R2 best practices Windows Server 2008 SP2 best practices The HP ProLiant DL980 G7 server is HP s latest 8 socket, x86-based, 64-bit scale up platform. Available in 4-, 6-, and 8-core configurations, it can support up to 128 logical processors (with 8 Intel Xeon 7500/6500 series processors x 8 cores x 2 threads per core, with Intel s Hyperthreading enabled), 2 TB of physical memory (RAM), and up to 16 PCIe/PCI-X I/O cards. In addition, the DL980 utilizes Non-Uniform Memory Architecture (NUMA) and Intel QuickPath Interconnect (QPI) technology. For a general overview of the technical specifications for this server, see the HP ProLiant DL980 G7 Server Data Sheet at: For detailed technical specifications, see the server QuickSpecs at: The DL980 uses HP PREMA architecture which incorporates a new node controller design with Smart CPU caching and redundant system fabric. Combined with the Microsoft Windows Operating System, these features provide a solution that is fully capable of supporting the most demanding, dataintensive workloads, with the reliability, availability, and efficiency needed to run all of your businesscritical applications with confidence. For a detailed discussion of the HP PREMA architecture, see the technical overview at: 2

3 Figure 1: Front view of the DL980 G7 server OS Support Matrix for the HP ProLiant DL980 G7 Server To run Windows and Windows applications optimally, you must understand how to leverage the capabilities and resources of the DL980 server. A good starting point is to review the following OS Support Matrix. HP ProLiant DL980 G7 Windows Operating System Support Matrix Operating Notes System Microsoft Windows Server 2008 R2 Enterprise Edition Microsoft Windows Server 2008 R2 Data Center Edition Microsoft Windows Server 2008 Enterprise x64 Edition Microsoft Windows Server 2008 Data Center x64 Edition Maximum CPUs Supported by OS 8 Sockets 256 Logical Processors 64 Sockets 256 Logical Processors 8 Sockets 64 Logical Processors 64 Sockets 64 Logical Processors Maximum Memory (RAM) 2 TB 2 TB 1 TB Requires Service Pack 2 (SP2) or greater. 1 TB Requires Service Pack 2 (SP2) or greater. The matrix shows that Windows Server 2008 R2 is necessary to make full use of the DL980 G7 s maximum processors and memory. Note that Hyper-V is also fully supported in both operating systems on the DL980 G7. More information on Hyper-V is found in the next section. 3

4 Hyper-V Support Matrix Hyper-V is a Microsoft technology that enables the partitioning of a single physical machine into multiple virtual machines (also called logical servers). Based on the hypervisor architecture, Hyper-V allows the creation and management of virtual machines, each one isolated from the others, running the same or different operating systems. Hyper-V is supported on the ProLiant DL980 G7 server running either Windows Server 2008 R2 or Windows Server 2008 SP2. It was first made available as a download for Windows Server 2008 (known as Hyper-V version 1). A more feature-rich version was released later with Windows Server 2008 R2 (known as Hyper-V version 2). The following table illustrates the major differences between these two versions: Hyper-V: Version 1 versus Version 2 Hyper-V Feature Version 1 Version 2 Maximum CPUs/Host 24 Logical 64 Logical Maximum Memory/Host 1 TB 1 TB Maximum Guests Maximum Virtual CPUs/Guest 4 4 Maximum Virtual Memory/Guest 64 GB 64 GB Maximum Virtual CPUs/Host For specific recommendations regarding the use of Hyper-V version 1 and version 2 in combination with hyper-threading, see the following sections: Hyper-threading + Windows Server 2008 R2 + Hyper-V version 2 Hyper-threading + Windows Server 2008 R2 + Hyper-V version 2 (on page 11) Hyper-threading + Windows Server 2008 SP2 + Hyper-V version 1 (on page 18) In addition, Hyper-V version 2 includes several new features such as live migration, storage hot addition and removal, jumbo frame and TCP chimney, and CPU core parking. For a complete summary of this newer version, see the following website: 4

5 Additional Software Components Recommended When Running Windows on the DL980 In addition to the software found on the HP Smart Start CD (with ProLiant Support Pack minimum recommended version 8.60), HP provides additional components for running Windows on the HP ProLiant DL980 G7 that ensure optimal performance, availability, and manageability of the solution. These two additional components are the HP Smart Update CD and the HP ProLiant DL980 System Providers for Windows. To download these components, along with other drivers and software, go to the HP DL980 G7 Support web page: From this page, select either Microsoft Windows Server 2008 R2 or Microsoft Windows Server 2008 x64. Then look in the section marked Software CD-ROM for the HP Smart Update package, or in the section marked Software System Management for the HP System Providers package. HP Smart Update CD One of the challenges when administering servers in mission critical environments is the sourcing and installation of important system updates. The HP Smart Update CD provides a comprehensive source for operating system and security updates, created by Microsoft and recommended by HP, all of which are thoroughly tested to ensure their compatibility on the DL980. Distributing updates in this way offers the following advantages for the administering IT professional: It provides a single source for all regularly published, tested fixes It is the quickest way to update the system, minimizing down time It is the only way to update systems that are not connected to the Internet Also known as hotfixes, these updates are grouped into 3 categories in the Smart Update package: Operating System, Security, and Clustering. Each category provides a collection of self-extracting, self-installing system updates. HP recommends that you regularly update your system with the most recent Operating System and Security updates. However, you only need to install the Clustering updates if your server will function as a node in a Microsoft Failover Cluster. Updates are provided for both Windows Server 2008 R2 and Windows Server 2008 SP2, in both English and Japanese. The Smart Update ISO package for the DL980 G7 can be found on the HP FTP site, here: ftp://ftp.hp.com/pub/softlib2/software1/cd/p /v63104 To install the updates, follow these steps: 1. Download the Smart Update ISO package and burn it to CD. 2. Insert the CD locally (or use ilo3 Virtual media), on the target server, or remotely using an ilo3 Integrated Remote Console. 3. Select the desired OS (Windows Server 2008 SP2 or Windows Server 2008 R2). 4. Navigate to the desired category (Operating System, for example) using the links on the left side of the page. 5. Click the Install button and the HP Setup screen displays. 6. Click Install again and a command window displays, listing each update as it installs. These can be large updates, taking several minutes each. 7. When prompted, click Reboot to complete the installation for that category. You must reboot after each category finishes, to ensure correct updating of the system registry. 8. Repeat to install the Security updates (highly recommended). 9. Repeat again to install the Clustering updates (if your server will function as a cluster node). 5

6 HP ProLiant DL980 System Providers (for Windows Server 2008 R2 Users Only) On DL980s running Windows Server 2008 R2, the HP ProLiant System Providers detect and report the current method for addressing system memory. The default RBSU (ROM-Based Setup Utility) method for addressing memory on the DL980 G7 is 40-bit. However, the recommended setting for running Windows Server 2008 R2 is 44-bit. Therefore, Windows Server 2008 R2 users must change this setting to 44-bit in order to make full use of the available address space and to reduce data faults that can occur if the less-optimum 40-bit setting is used. The procedure for making this change is described in the section, Enable 44-bit Mode after OS Installation (on page 11). Note HP recommends that you install the HP ProLiant DL980 System Providers on DL980s running Windows Server 2008 R2 only. At this time the System Providers are not supported on systems running Windows Server 2008 SP2. Click the Download button on this page to download the HP ProLiant DL980 System Providers: odtypeid=15351&prodseriesid= &prodnameid= &swenvoid=4064&swlang=13 &mode=2&taskid=135&switem=mtx-2b1d827b26d2497eb05e3b3666. Also on this page are the installation instructions and release notes for the System Providers. Once installed, the DL980 System Providers detect the current memory addressing mode, and if the non-optimum 40-bit mode is found, report it to the Windows System log (see Figure 2). Figure 2: Windows System log screen with 40-bit mode warning 6

7 Verifying Installation of the ProLiant DL980 System Providers You can confirm the presence of the DL980 System Providers on systems running Windows Server 2008 R2 by looking at the Windows Services window for the HP ProLiant DL980 System Providers entry (see Figure 3). Figure 3: Verifying System Providers installation (Services window) Another way to confirm positive installation is to check the Windows System log for the following event (see Figure 4): Figure 4: Verifying System Providers installation (System log) To confirm the System Providers are functional after installation, using HP s System Management Homepage (SMH) software, follow these steps: 1. From a system with a network connection to the target server, open a web browser window. 2. Enter the following URL into the browser s address bar: where machine is the IP address or computer name of the target server. If it is the first time using this link, a Security Alert dialog box displays and asks if you want to trust this server. Click Yes to continue, and the Sign In page displays. 3. Enter your user name and password, and click Sign In to access SMH. The SMH Main page displays. 4. In the Advanced Server Health Information section, click Advanced Server Settings. 7

8 5. Locate the System Address Mode setting in the Advanced Server Settings window. It indicates either 44-bit mode (see Figure 5) or 40-bit mode (see Figure 6), depending on how the system is currently configured. Either one of these values confirms that the System Providers are functional. Figure 5: SMH Advanced Server Settings window; 44-bit enabled Figure 6: SMH Advanced Server Settings window; 40-bit enabled SMH is a web-based interface that simplifies single-system management of HP servers running HP-UX, Linux, and Microsoft Windows operating systems. Use it to view in-depth hardware configuration and status data, performance metrics, system thresholds, diagnostics, and software version control information using a single intuitive interface. HP SMH aggregates and displays information provided by the Web Agents and other HP Web-enabled System Management software, including: HP Insight Diagnostics Array Configuration Utility Version Control Agents Hardware Best Practices for the DL980 When Running Windows This section describes hardware recommendations to follow when running Windows on the DL980. These apply to both Windows Server 2008 R2 and Windows Server 2008 SP2. 8

9 Boot Adapter Location On the DL980 G7, slots 11 through 7 are the first ones to be scanned during the boot phase. Therefore, the boot adapter must be installed in one of these slots in order for it to be enumerated first. Once that is accomplished, boot order is prioritized by the BIOS settings, as usual. Power Consumption Settings To take advantage of Windows ability to adjust server power consumption (and reduce costs) based on workload, follow these steps: 1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU). 2. Select Power Management Options. 3. Select HP Power Regulator. 4. Select OS Control Mode (see Figure 7). Figure 7: Enabling OS Control Mode using RBSU 5. Exit RBSU using the Esc key and reboot the system. 9

10 Windows Server 2008 R2 Best Practices for the DL980 When running Windows Server 2008 R2 on the HP ProLiant DL980 G7 Server there are several recommended best practices that are fully explained in this section: Enable Hyper-threading to maximize resource usage (unless using Hyper-V) Enable 44-bit mode after OS installation Implement Hyper-V workaround for systems with more than 512GB RAM Enable Hyper-threading to Maximize Resource Usage for Large Applications (but NOT for Hyper-V) As noted in the OS Matrix earlier, Windows Server 2008 R2 can support up to 128 logical processors on the DL980 (8 sockets x 8 cores x 2 threads/core, with hyper-threading enabled). Therefore, it is highly recommended that hyper-threading be enabled on DL980s running Windows Server 2008 R2, in order to take full advantage of that processing power. Hyper-threading is controlled by the system BIOS, and is enabled by default on the DL980 G7. If, however, it was disabled for some reason, you should follow these steps to enable it: 1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU). 2. Select System Options. 3. Select Intel Hyperthreading Options (see Figure 8). Figure 8: Enabling Hyper-threading using RBSU 4. Select Enabled to turn Hyper-threading ON. 5. Exit RBSU using the Esc key and reboot the system. 10

11 For more information about the RBSU, see the HP ROM-Based Setup Utility User Guide: Hyper-threading + Windows Server 2008 R2 + Hyper-V version 2 Windows Server 2008 R2 supports Hyper-V version 2. But as stated earlier in the Hyper-V support matrix, version 2 supports a maximum of 64 logical processors. In other words, Hyper-V can only use the first 64 processors it sees. So on DL980s running Windows Server 2008 R2 with the Hyper-V role enabled, you must turn hyper-threading OFF instead (the procedure is the same as described previously, except that you select Disabled in Step 4 instead). Enable 44-bit Mode after OS Installation The default RBSU method for addressing memory on the DL980 G7 is 40-bit. However, the optimal setting for running Windows Server 2008 R2 is 44-bit. Therefore, Windows Server 2008 R2 users must change this setting to 44-bit to make full use of the available address space and reduce the decoding faults and data coherency issues that can occur if the less-optimum 40-bit setting is used. To enable the 44-bit System Address Mode, follow these steps: 1. During the boot process, confirm that the system is in the default 40-bit mode by looking at the RBSU splash screen. It indicates a System Address Mode of 40-bit (see Figure 9). Figure 9: Confirming System Address Mode reported by RBSU 2. Install the Windows Server 2008 R2 operating system. 3. Install the Windows Server 2008 R2 operating system updates, using the Smart Update CD, as described in the section, HP Smart Update CD (on page 5). 11

12 4. Reboot the system. 5. Enable a System Address Mode of 44-bit as follows: a. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU). b. Select Advanced Options. c. Select Advanced Performance Tuning Options. d. Select Address Mode 44-bit (see Figure 10). Figure 10: Enabling 44-bit System Address Mode using RBSU e. Ignore the Warning message: Caution: Enabling 44-bit Address Mode may cause problems with an operating system if it does not support 44-bit memory addressing. f. Select Enabled to turn 44- bit memory addressing ON. 6. Exit RBSU using the Esc key and reboot the system. 7. Confirm the new 44-bit System Address Mode by looking at the RBSU splash screen during system boot, as described in Step 1. Install Microsoft Hotfix on Systems with More Than 1 TB RAM Windows Server 2008 R2 can sometimes fail to install or run on systems with more than 1 TB of RAM and 44-bit addressing mode enabled because some physical addresses may go beyond 40 bits. This can generate a bug check for some specific memory conditions, causing the installation to fail. Microsoft has documented the issue and its resolution in Knowledge Base Article KB980598, titled Windows Server 2008 R2 cannot be installed or started on a computer that has 1 TB or more of RAM, found here: This hotfix is included on the Smart Update CD in the Windows Server 2008 R2 updates, and it must be installed to solve this specific problem. 12

13 Workaround When Enabling Hyper-V on Systems with More Than 512 GB RAM In Windows Server 2008 R2, if you enable the Hyper-V role on DL980s with between 512 GB and 1 TB of RAM, these systems will hang after rebooting and never return back to the Windows Login prompt. A Microsoft hotfix must be installed, but it cannot be installed until the Hyper-V role is enabled, and once enabled, you run into the problem. There are several workarounds, all of which involve a temporary reduction in the amount of memory made available to Windows. Use one of these following temporary solutions to reduce system memory: Method #1: Enable Mirrored Memory using RBSU Method #2: Decrease maximum memory in Windows using the MSCONFIG utility Method #3: Reduce addressable memory with a Windows boot option After the memory is reduced to less than 512 GB using one of the previous methods, you must enable Hyper-V, then apply the Microsoft hotfix for the problem ( reboot again, and then finally, restore the system memory back to its original value. Method #1: Enable Mirrored Memory using RBSU Mirrored Memory provides maximum protection against uncorrectable memory errors that might otherwise result in system failure. To enable Mirrored Memory using RBSU, follow these steps: 1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU). 2. Select System Options. 3. Select Advanced Memory Protection. 4. Select Mirrored Memory with Advanced ECC Support (see Figure 11). 13

14 Figure 11: Enabling Mirrored Memory using RBSU 5. Select Enabled to turn Mirrored Memory ON. 6. Exit RBSU using the Esc key. 7. Reboot the system. Note Enabling or disabling mirrored memory will increase your system boot time, since any change invokes a mandatory test of all system memory during the boot process. Method #2: Decrease maximum memory using MSCONFIG utility To decrease maximum memory in Windows using the MSCONFIG utility, follow these steps: 1. At the Windows desktop, click Start Run. 2. Type msconfig and press Enter. 3. Select the Boot tab. 4. Click the Advanced Options button. 5. Select the Maximum Memory check box and lower the value; for example, to something like 48GB (see Figure 12). 14

15 Figure 12: Decreasing Maximum Memory using MSCONFIG utility 6. Click OK and exit the MSCONFIG utility. 7. Reboot the system. Method #3: Reduce addressable memory with a Windows boot option To decrease maximum memory in Windows with a temporary boot option, follow these steps: 1. Enter the following command at the Windows command line: bcdedit /enum Various system boot entries will display. 2. Locate the Windows Boot Loader boot entry and write down its GUID number, for example: ba85e57e-8d4e-11df-b805-c70cfe Use the following command to create a new Windows Boot Loader entry (by copying the current one): bcdedit /copy GUID /d Less Memory Config, where GUID is the GUID number noted in the previous step, and Less Memory Config is a descriptive name given to the new boot entry that will assign less memory to the system. 4. Run the bcdedit /enum command again to verify that the new entry was created. Notice that the new entry is exactly the same as the current one, except it has been assigned a new GUID number. For example, the new boot entry has the following GUID: ba85e580-8d4e-11df-b805-c70cfe Alter the new boot entry so it will assign less memory to the system. For example, for a system with 1 TB RAM, use the following command to give it less than 512 GB, by removing 640 GB: bcdedit /set ba85e580-8d4e-11df-b805-c70cfe removememory Reboot the system. The Windows Boot Manager screen displays, showing the new boot entry you just created. Scroll down to select the new entry (Less Memory Config) that will boot the system with reduced memory (see Figure 13). 15

16 Figure 13: Windows Boot Manager screen showing new boot entry For a complete description of the BCDEdit command and an explanation of all available options, see the following document: After you reduce the system memory using one of the previous methods, follow these steps: 1. Enable the Hyper-V role. 2. Download and install the Microsoft Hyper-V hotfix for this problem: 3. Reboot the system and remove the memory restriction, depending on the method previously used: a. Method #1: Press F9 during system boot, disable Mirrored Memory within RBSU, and reboot. b. Method #2: After system boot, use MSCONFIG utility to remove the maximum memory limit, and reboot. c. Method #3: Let the system reboot using its default boot option (with no memory restriction). 16

17 Windows Server 2008 SP2 Best Practices for the DL980 This section describes several recommended best practices when running Windows Server 2008 SP2 on the HP ProLiant DL980 G7 Server. Disable Hyper-threading to Keep Logical Processor Count at 64 Maximum Hyper-threading refers to Intel s proprietary technology for increasing parallel computational power (processor multi-tasking) by allowing the OS to see and address each physical processor core as if it were 2 virtual processors. It also enables the OS and applications to share work between those 2 virtual processors whenever possible, thereby making use of resources that would otherwise sit idle. As noted in the OS Matrix, Windows Server 2008 SP2 supports up to 64 logical processors on the DL980 (8 sockets x 8 cores). Therefore, if Hyper-threading is enabled on these systems, it increases the logical processor count to 128 logical processors, or well above that limit. This wastes processing power because the system simply cannot recognize processors above the 64 processor limit. For example, on a DL980 G7 with 8 sockets, 8 cores per socket, and Hyper-threading enabled, Windows Server 2008 with SP2 sees only the first 16 threads in Processor #1, the next 16 threads in Processor #2, and so on, until it reaches Processor #4 (16 threads x 4 processors = 64 threads or logical processors). Any processor after the fourth processor is a wasted resource. In this example, four entire processors, or half of the available processing power, sit idle. Hyper-threading is controlled by the system BIOS, and is enabled by default on the DL980 G7. To disable it, follow these steps: 1. During system boot, press F9 to run the ROM-Based Setup Utility (RBSU). 2. Select System Options. 3. Select Intel Hyperthreading Options (see Figure 14). 17

18 Figure 14: Disabling Hyper-threading using RBSU 4. Select Disabled to turn Hyper-threading OFF. 5. Exit RBSU using the Esc key and reboot the system. For more information about the RBSU, see the HP ROM-Based Setup Utility User Guide: Hyper-threading + Windows Server 2008 SP2 + Hyper-V version 1 Windows Server 2008 SP2 supports Hyper-V version 1. But as stated earlier in the Hyper-V support matrix, version 1 supports a maximum of 24 logical processors. In other words, Hyper-V can only use the first 24 processors it sees. So on DL980s running Windows Server 2008 SP2 with the Hyper-V role enabled, there is an even greater need to turn hyper-threading OFF. Workaround for Possible Driver Conflict When Logging in Using ilo Another issue, although intermittent, occurs when logging onto the system using ilo (HP s Integrated Lights Out a remote console technology). The problem has usually been seen in systems with 1TB of RAM, and results from a conflict with the ATI2dvag video driver. The resulting blue screen displays the following stop code: 0x000000EA (0xFFFFFA86B9A91860, 0xFFFFFA86B9B64910, 0xFFFFFA86B9AF37C0, 0x ) To resolve this issue, you must add the following two keys to the system registry: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Watchdog\Display\ DisableBugCheck REG_DWORD 0x1 18

19 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Watchdog\Display\ EaRecovery REG_DWORD 0x0 19

20 For more information For an overview of the HP Proliant DL980 G7 server: For more information about the HP DL980 G7 server with HP PREMA Architecture: html To download the recommended components described in this document, along with other drivers and software, visit the HP DL980 G7 Support web page: For a complete library of all documentation supporting the DL980 G7 server: (click on the link, HP ProLiant DL980 G7 Server series ) For more information on the Intel Xeon processor 7500 series: Share with colleagues

21 Copyright 2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Microsoft and Windows are US registered trademarks of Microsoft Corporation. AM A, October 2010