Technical White Paper iscsi Boot November 11, 2004

Transcription

1 Technical White Paper iscsi Boot November 11, 2004 SN Rev A 11/04 Page 1 of 12

2 Table of Contents I. Executive Summary...3 II. Booting The Basics...3 Booting Options...3 Boot Methods Pros and Cons...4 III. iscsi Boot with Software Initiators vs. HBAs...6 Applications...6 IV. An overview of how boot from SAN works...7 V. Implementation Considerations...8 Operating System Dependencies...9 Microsoft Windows Dependencies...9 Linux Dependencies...9 VI. Recovery Details...10 Appendix A Finding the HBA iscsi Name and IP Address...11 Appendix B Configuring the QLogic HBA for SAN boot...11 SN Rev A 11/04 Page 2 of 12

3 I. Executive Summary Two constant driving forces within IT departments are the need to reduce IT costs, and increase the availability of data. These needs have pushed organizations to use many smaller, low cost servers in an effort to get the greatest computing power per dollar spent. With this trend comes increased complexity to manage the environment. This trend toward larger numbers of servers has also driven more IT departments to put more of their servers and storage on storage area networks (SANs). Increasingly, there is interest in allowing serves to boot from the SAN as this boot configuration can both reduce the cost of ownership and simplify SAN management, while increasing the availability of these servers. Fibre Channel SANs have supported booting for several years. In iscsi SANs, host bus adapters (HBAs), as opposed to software initiators, are the best means to implement booting from the SAN. In addition to the enhancements to CPU performance through the processing offload provided by an HBA, the SAN boot advantages will make the investment in HBAs cost effective. This white paper will explain what SAN boot is, its advantages, and how it works in iscsi. It will also explain the hardware requirements and applications, and conclude with specific information about QLogic s iscsi HBA and its support of SAN boot. II. Booting The Basics Booting a system is an iterative process for a system to get itself from power up to a ready state. The process starts with loading a small amount of code (e.g. BIOS for IA32 processors, or Fcode for SPARC) that in turn directs the CPU to load additional code, specifically the operating system. The boot process starts with the following: Power On Self Test (POST) checks that the hardware is operating properly by checking voltages and checking that any local drives are operating. Locate bootable devices finds all devices, such as floppy and hard drives, as well as network cards, that can be the source of operating system code. Choose the boot device from the selection of devices found in the last step, one is selected, either through basic rules set in the system or through a setting from the user. Load boot code after choosing the boot device, the system will locate the Operating System (OS) and load it to memory. Booting Options Booting can be accomplished from an internal or direct attached storage (DAS) drive, over a LAN, or over a SAN. SN Rev A 11/04 Page 3 of 12

4 Most commonly, system booting is completed with an internal or DAS drive. In this simplest configuration, there only needs to be a boot drive present on the system. There is no reliance on network connectivity. The second option is booting over a LAN, known as remote boot, in which a system boots from a boot server connected somewhere across a local area network. LAN boot has been in use in Unix systems since the late 1970s. LAN boot depends on code stored in non-volatile memory on the network interface card (NIC) that will direct the system to the boot server for a boot image. Booting from a SAN, also a type of remote boot, relies on code stored on an HBA the way booting over a LAN relies on an NIC. In this case, the code stored on the HBA will direct the system to a logical unit on a storage array where the boot image is found. In this regard, SAN boot is a merging of the network aspect of LAN boot and the storage aspect of local boot (i.e., no boot server is required). Boot Methods Pros and Cons Local boot is the simplest method, but requires managing boot images on every machine. In deployments of hundreds of servers, the multitude of images to back up and maintain can be very time consuming. This method also lacks the other advantages realized with LAN and SAN boot, as outlined below. Remote boot methods, both SAN- and LAN-based, enjoy significant advantages derived from separating boot images from the servers. Booting over the LAN reduces purchase costs of the servers and centralizes management. However, LAN-based booting has not experienced wide adoption on servers. NICs become more complex, as the NIC needs the network stack from the OS to communicate with the network (discussed below). In addition, there is a greater security risk, as the boot code is subject to hacking on the Ethernet. Although long supported in Unix installations, Microsoft has never widely supported LAN boot, since supporting storage boot over a NIC requires a different system BIOS. Booting over a SAN provides all of the advantages of LAN booting, and since it uses a standard storage interface (SCSI), it is simpler to implement. From an implementation standpoint, booting over a SAN requires significant up-front configuration work. HBAs must be configured properly and a logical unit number (LUN) assigned to ensure that the systems are finding the proper images for booting. No single step is inherently complex, but in aggregate the process needs to be carefully managed. Advantages to SAN boot are: Lower hardware costs: The installed cost of diskless servers is lower since they take less rack/floor space, have no disk drive, use smaller enclosures and can have smaller power supplies. Blade serves are the ultimate example of this type of cost reduction through density. Those implemented in a diskless configuration take greatest advantage of this form factor. Reduced Hardware Operating cost: By reducing power and size, diskless systems require less cooling, use less energy and require smaller back-up power systems (e.g., UPS). Greater reliability: With fewer moving parts, diskless systems are less prone to failure. SN Rev A 11/04 Page 4 of 12

5 Simplified Management: Upgrades and patches can be implemented in one location with instant access for all servers. Centralized upgrades are faster and eliminate the need to install upgrades on every server individually. This results in far fewer mistakes and service interruptions. Greater Software Integrity and Availability: Software upgrades are implemented more quickly and are generally maintained on highly available redundant systems such as RAIDs. These centralized systems are also backed up with higher frequency, resulting in more current and more readily available software. Quick Server Swaps: In the event of a server failure, diskless systems can be swapped out without the need to reconstruct the booting, operating and application environments. Spare systems only need to be installed and configured to boot from the remote disk, greatly reducing the potential down time from the failure. Faster Disaster Recovery: When stored on a SAN, boot information along with production applications and data are more easily duplicated in disaster recovery sites through methods such as mirroring. In the event of a catastrophic failure, the disaster recovery site can quickly duplicate the primary site s functionality and come on line. Configuration Flexibility: With the operating environment on the SAN, servers can be added and deleted quickly to adjust to varying computer demand. In this way, IT departments can provide capacity on demand to meet seasonal changes or special project requirements. Implementation Simplicity: When using an HBA (the NIC alternative is discussed below), booting from a SAN uses the same SCSI storage interface used for a local drive, which simplifies the protocol implementation. Broad Support: SAN boot is supported by all three major operating systems: Windows, Linux and Solaris The table below summarized the comparison of three boot methods. Boot Method Local LAN Pros Most basic configuration Very Secure Lower hardware purchase cost no disk, less space Reduced operating cost less power/cooling Greater HW reliability no disk failures Simplified management centralized upgrades Cons Management complexity - many images to support Cost higher hardware system cost Security prone to hacking over the LAN Complexity - NIC and protocol implementation OS Support limited Windows support System BIOS update SN Rev A 11/04 Page 5 of 12

6 SAN Greater SW integrity/availability fewer backups Faster recovery no need to rebuild systems Better disaster recovery simplified site duplication Lower hardware purchase cost no disk, less space Reduced operating cost less power/cooling Greater HW reliability no disk failures Simplified management centralized upgrades Greater SW integrity/availability fewer backups Faster recovery no need to rebuild servers Better disaster recovery simplified site duplication Protocol Simplicity standard storage interface Greater Security SAN isolation prevents hacking Broad support all Operating Systems Appears to the host OS as a local disk. Common interrupt vector as a disk drive. No system BIOS modifications required. required Initial configuration complexity III. iscsi Boot with Software Initiators vs. HBAs Unlike with Fibre Channel, iscsi connections can be made with either an HBA or a NIC + software stack. Implementing SAN boot is similar for iscsi HBAs and Fibre Channel HBAs. Since the protocol stack runs on the HBA, it is ready to communicate with the network and download the operating system code to the host system. For a software initiator to implement iscsi SAN boot, the NIC must overcome a Catch-22 situation in which the NIC needs to load an OS to have a network stack, but with the OS stored across the network it does not yet have the means to communicate with the network. There is a complex iterative process for finding a boot server, then finding the OS files and loading portions of the stack so further portions can be downloaded. Although possible, supporting this method is very difficult. Microsoft, the dominant supplier of software initiators, only supports iscsi Boot with HBAs. Applications Most applications can benefit from the advantages derived from SAN boot. Those that benefit most greatly will be large installations or those that have critical requirements for minimal downtime and quick disaster recovery. Some examples follow: SN Rev A 11/04 Page 6 of 12

7 , database, and web servers located in data centers (or server farms ). Commerce servers that handle revenue-critical applications. Enterprise servers that provide access to critical network resources and services. Server blade installations, where small servers built on cards plug into specially designed, stackable chassis. These types of environments typically offer shared management, storage, networking, power, and cooling between servers, thus reducing TCO even further. IV. An overview of how boot from SAN works Booting from a SAN is achieved by configuring the server with its own virtual boot device. The boot device allows the server to fetch boot-loading instructions from remote storage and execute them. This is most often configured on the HBA installed in the server. The boot device is mapped as a logical unit number (LUN) on the remote storage. Depending on an organization s requirements, all LUNs might boot servers with the same profile. Alternatively, some LUNs may be mapped to Microsoft Windows NT servers, others mapped to Microsoft Windows 2000 servers, and still others mapped to Solaris servers. Each server is then configured to point to the appropriate LUN at the specific controller from which the server is to boot. Setting up a LUN to serve as a boot disk is partially configured on the HBA and partially configured on the array. With the QLogic SANblade 4010 iscsi HBA, the configuration procedure involves these steps: 1. Choose Selectable Boot option from a range of targets provided by the HBA. Refer to Appendix B for a more detailed explanation of this process. 2. Select an iscsi storage device from which to boot. 3. Select the LUN associated with the selected iscsi target. Organizations with fully redundant and highly available SAN configurations may have two or more HBAs or multi-ported HBAs installed in each server attached to the SAN. For these installations, the procedure above is repeated for the remaining HBAs so that they point to the same LUN, either via the same array or a different array. The configuration of such an environment is out of the scope of this document, as there are many approaches. After performing the procedure above, the server is powered up and the following sequence occurs: 1. During the boot process, the option BIOS from the HBA is initialized. 2. The BIOS finds the boot LUN at the storage unit and configures it as a boot device. 3. A booting protocol tells the server BIOS to scan the network for the appropriate LUN for the OS image. 4. When the server locates the appropriate LUN, the server CPU executes the OS boot image. SN Rev A 11/04 Page 7 of 12

8 V. Implementation Considerations To implement a boot-from-san configuration with a RAID controller: The server must have an HBA equipped with a boot BIOS that can support booting from a storage device attached to the SAN. That is, the boot BIOS must support iscsi boot and selectable boot capabilities. In many instances it is necessary for the server s internal IDE hard drives to be disconnected, or disabled in the system BIOS, to allow the boot selection process to work correctly. This is strictly a limitation of the system BIOS and not the HBA s boot BIOS. Some systems may allow internal IDE drives to stay connected, or enabled in the system BIOS, if they correctly support the bootstrap protocol Typically, the host needs to have a boot order of CD-ROM, diskette, and then drive 0 (or to where ever the HBA re-maps). This allows the installation process of the particular OS to proceed correctly. After the OS is installed, this can be changed to have drive 0 (or to where ever the HBA re-maps) to be first boot device. The SAN must be installed and configured, and the storage visible to the host. The diagnostic features available in the QLogic HBA BIOS can be utilized to verify that the links are working properly before installing the OS. The storage system must have at least one available LUN for booting the server(s). There is typically a logical drive-to-lun association on the storage device. The LUN used must be appropriate for the OS from which it is intending to boot. For instance, Linux and Windowsbased OSs require the boot LUN to be LUN 0. Also, some RAID units have a host type associated with the logical drive. This host type must be configured for the correct OS, so that the correct format of the inquiry data is sent to the host. The storage must be configured to allow the server exclusive access to the LUN; access is typically assigned via the iscsi HBA s iscsi name and IP address on the storage. No other server may have access to or view this LUN. Refer to Appendix A to determine the iscsi name and IP address for QLogic HBAs. For High Availability (HA) configurations, such as multi-hba or multi-port RAID, setups must adhere to at least the following general rules: Only one path to the storage must be visible to the OS during OS installation. To be more precise, this rule refers to the endpoints between the storage and the host, not the fabric connections in-between, for example, as in multi-switch configurations. If a RAID subsystem has multiple controller ports, as most do, only one RAID controller port can be enabled or connected to the SAN during the OS install. The same rule applies to the HBA side: Multiple HBAs and multi-port HBAs installed in a server must have only one HBA port connected to the SAN during the OS install process. The reasoning behind this restriction is that the OS may interpret these additional paths to the storage as separate SN Rev A 11/04 Page 8 of 12

9 storage devices, thus compromising data integrity. Additional storage paths (controller ports/hba connections) can be added later after higher-layer filter drivers are installed/configured. If the current driver already supports multi-path IO, it must first be configured to handle these additional paths properly, such as Linux HBA driver. Also, the additional pathways must be configured to allow alternate pathways for SAN boot -- for example, HBA configuration, HA driver configuration, and RAID configuration. If these general rules are not followed, several things can happen. The worst-case scenario is data integrity could be compromised. Secondly, hosts might be unable to boot or fail-over to an alternate pathway once the primary path has failed. Only general rules have been provided, due to the sheer number of configurations and variables available. To ensure a problem-free SAN environment, refer to the integrator s expertise for solutions that account for all possible scenarios in these types of configurations. Other requirements depend on the configuration that an organization uses or plans to use. Operating System Dependencies Various operating systems require certain guidelines to be followed to allow servers to boot from a SAN. Microsoft Windows Dependencies For servers to boot from a SAN in a Microsoft Windows environment: Hosts must be segregated The SAN must either be configured in a switched environment or be directly attached from each host to one of the storage subsystem's iscsi ports. LUN 0 Microsoft Windows NT and Windows 2000 both require the storage unit to appear as LUN 0 to the server. Exclusive access to the bootable disk The host must have exclusive access to the logical disk from which it is booting. No other host on the SAN should be able to detect or have access to the same logical disk. This can be accomplished by using a LUN-management facility such as LUN masking, zoning or a combination of these methods. Linux Dependencies Linux has the same dependencies as Microsoft Windows NT/2000. SN Rev A 11/04 Page 9 of 12

10 VI. Recovery Details Observe the following recovery details regarding booting from a SAN: If a server is replaced by the same model/type server with the HBA from the old server, the system only needs to be rebooted; no reconfiguration is required. The only exceptions to this are if the old server had some unique settings to enable SAN boot, such as System BIOS Startup options, or internal IDE drives were disconnected or disabled; then these settings would need to be migrated to the new server. If an HBA in the server is replaced with a different HBA, the new HBA and storage will need to be reconfigured. Configure the new HBA to boot the server from the appropriate storage and LUN. Refer to Appendix B for these steps. In addition, be sure the access rights on the RAID ports are updated to reflect the change in the iscsi name and IP Address of the new HBA. Refer to RAID vendor-specific documentation for these steps. Refer to Appendix A to determine the QLogic iscsi name and IP Address. If a server is replaced by a different model/type with the same HBA, from the old server, only the server will need to be reconfigured. This reconfiguration may be automatic or manual depending on the OS. For instance, with Windows 2000, a plug-and-play OS, the OS reconfigures itself and prompts the user to insert the drivers diskette/cd for the new hardware found, without requiring the reconfiguring of applications, such as Exchange Server. On Linux, Windows NT, and other non-plug-and-play OSs, the server will need to be manually reconfigured to recognize the new hardware, such as video, network, etc. Refer to the OS vendor s specific documentation on how to reconfigure a server for new hardware. If a server is replaced by a different model/type with a new HBA, the new HBA, storage, and server will need to be reconfigured. Configure the new HBA to boot the new server from the appropriate storage and LUN. Refer to Appendix B for these steps. For storage, be sure the access rights on the RAID ports are updated to reflect the change in the iscsi Name and IP Address of the new HBA. Refer to RAID vendor-specific documentation for these steps. Refer to Appendix A to determine the QLogic HBAs iscsi Name and IP Address. For the server, reconfiguration may be automatic or manual depending on the OS. Remember that these are general rules. There may be some additional steps required depending on what other drivers are being used and special settings configured within the OS and/or applications. SN Rev A 11/04 Page 10 of 12

11 Appendix A Finding the HBA iscsi Name and IP Address For new installations, the iscsi name and IP address must be configured with the SANsurfer GUI. For HBAs already configured, return to the GUI or follow these steps: 1. Reboot system with QLogic HBA. 2. Hit Ctrl-Q when QLogic HBA banner appears. 3. Select Configuration Settings. 4. Then select Host Adapter Settings. 5. Write down the iscsi Name and IP Address Appendix B Configuring the QLogic HBA for SAN boot 1. Have the HBA directly connected, or via a switch, to the storage device. Be sure the storage device and, if used, switch are both powered up and completely initialized before proceeding to the next step. Be sure all devices are visible to all other devices on the switch. If VLANs are enabled, the HBA and storage device must be on the same VLAN. Refer to the switch vendor documentation on switch configuration. 2. Be sure the storage device allows HBA access. Refer to the vendor-specific documentation to determine how to enable this access. 3. Install the BIOS on the card from the SANsurfer CD or check QLogic web site for the latest BIOS. 4. Start the system; press Ctrl-Q or Alt-Q (REMOVE ALT-Q) when BIOS screen for the QLA4010 pops up. 5. Select the "Configuration Settings" menu. 6. Select the Host Adapter Settings. 7. Enable "Host Adapter BIOS". 8. Hit <Escape> to exit host adapter settings screen. 9. Enter the "iscsi Boot Settings" 10. Enable "iscsi Boot" For first time configurations in which no devices were configured using the SANsurfer GUI, continue with steps Select "Primary Boot Device Settings". 12. Select "Target IP". 13. Enter the IP address of the target device. 14. Hit Escape to save the changes. The HBA will query all the devices attached. 15. Select the "Configuration Settings" menu. 16. Select the "iscsi Boot Settings" option. SN Rev A 11/04 Page 11 of 12

12 17. Select Target IP address and hit <Enter> 18. Select the target device from which you want to boot in the Select iscsi Device list. 19. HBA will query all the devices and an IQN name will now appear under the iscsi name. 20. Exit the BIOS and save the configuration. 21. Restart the system. Have Windows install CD in the CD drive. 22. When prompted to hit "F6" to install additional drivers hit F Insert the QLA4010 Windows driver diskette and proceed through the OS install as if attached to a local device. For first time configuration with devices configured using the SANsurfer GUI, follow steps 11b 16b 11b. Highlight the "LUN and Target IP next to the Primary location. 12b. Hit <Enter> to see a list of devices. Select the boot device. 13b. Exit the BIOS and save the configuration. 14b. Restart the system. Have Windows install CD in the CD drive. 15b. When prompted to hit "F6" to install additional drivers hit F6. 16b. Insert the QLA4010 Windows driver diskette and proceed through the OS install as if attached to a local device This example accounts for using a system with one single-port HBA. If multiple-port HBAs or multiple HBAs are used in the system during the HBA configuration phase, the user must ensure the correct configuration of the boot HBA by connecting only the HBA that will perform SAN boot. SN Rev A 11/04 Page 12 of 12