RAID on Motherboard (ROMB) Considerations Using Intel I/O Processor

Transcription

1 RAID on Motherboard (ROMB) Considerations Using Intel I/O Processor Application Note June 2002 Document Number:

2 Information in this document is provided in connection with Intel products. No license, express or implied, by estoppel or otherwise, to any intellectual property rights is granted by this document. Except as provided in Intel's Terms and Conditions of Sale for such products, Intel assumes no liability whatsoever, and Intel disclaims any express or implied warranty, relating to sale and/or use of Intel products including liability or warranties relating to fitness for a particular purpose, merchantability, or infringement of any patent, copyright or other intellectual property right. Intel products are not intended for use in medical, life saving, or life sustaining applications. Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked reserved or undefined. Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The Intel I/O processor may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order. Copies of documents which have an ordering number and are referenced in this document, or other Intel literature may be obtained by calling or by visiting Intel's website at Copyright Intel Corporation, 2002 AlertVIEW, i960, AnyPoint, AppChoice, BoardWatch, BunnyPeople, CablePort, Celeron, Chips, Commerce Cart, CT Connect, CT Media, Dialogic, DM3, EtherExpress, ETOX, FlashFile, GatherRound, i386, i486, icat, icomp, Insight960, InstantIP, Intel, Intel logo, Intel386, Intel486, Intel740, IntelDX2, IntelDX4, IntelSX2, Intel ChatPad, Intel Create&Share, Intel Dot.Station, Intel GigaBlade, Intel InBusiness, Intel Inside, Intel Inside logo, Intel NetBurst, Intel NetStructure, Intel Play, Intel Play logo, Intel Pocket Concert, Intel SingleDriver, Intel SpeedStep, Intel StrataFlash, Intel TeamStation, Intel WebOutfitter, Intel Xeon, Intel XScale, Itanium, JobAnalyst, LANDesk, LanRover, MCS, MMX, MMX logo, NetPort, NetportExpress, Optimizer logo, OverDrive, Paragon, PC Dads, PC Parents, Pentium, Pentium II Xeon, Pentium III Xeon, Performance at Your Command, ProShare, RemoteExpress, Screamline, Shiva, SmartDie, Solutions960, Sound Mark, StorageExpress, The Computer Inside, The Journey Inside, This Way In, TokenExpress, Trillium, Vivonic, and VTune are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. *Other names and brands may be claimed as the property of others. 2 Application Note

3 Contents Contents 1.0 Introduction Document Organization I/O Processor Architecture Overviews Dual PCI Bus I/O Processor Architecture (Intel I/O Processor) Dual-PCI Bus System Architecture Single PCI Bus I/O Processor Architecture (Intel I/O Processor) Single PCI Bus System Architecture Hardware Design Considerations Interrupt Routing Initialization Device Select (IDSEL) Eliminating IDSEL Race Conditions with Host Software Design Considerations Interrupt Control Configuring and Initializing I/O Controllers (IDSEL) Configuring the I/O Controller as Private Discovering the PCI I/O Controller Private Memory Space Conclusion...18 A B C D Microsoft WHQL* Considerations...19 ROMB Application Workload...20 Zero-Memory ROMB Design with Intel I/O Processor...22 Discrete Logic Example for IDSEL Control...23 Application Note 3

4 Contents Figures 1 Intel I/O Processor Functional Block Diagram Dual PCI Bus ROMB System Architecture Intel I/O Processor Architecture Single PCI Bus ROMB System Architecture Intel I/O Processor Interrupt Routing IDSEL Steering Implementation Inbound Address Detection RAID-5 ROMB Typical Workload Data Flow Discrete Logic Example for IDSEL Control...23 Tables 1 PCI Interrupt Routing Select Register (address = FFFF E2ECh) GPIO Output Enable Register (address=ffff E7C4h) GPIO Output Data Register (address = FFFF E7CCh) WHQL Testing (Windows 2000*) Using HCT Version 9.5 with Update ROMB RAID-5 Typical Workload Throughput Boot Sequence and Initialization Code Modifications Application Note

5 Introduction 1.0 Introduction The purpose of this application note is to describe the hardware and software modifications required to migrate an existing intelligent I/O application from a dual PCI bus architecture I/O processor to the single PCI bus architecture of the Intel I/O processor based on Intel XScale microarchitecture (ARM* architecture compliant). The application that will be described in this application note will be RAID on Motherboard (ROMB). An intelligent I/O application implemented with a single PCI bus I/O processor requires minimal modifications in both hardware and software. This application note discusses the details for both the hardware and software modifications necessary to control the additional logic and the added functions in the Intel I/O processor (80321) to obtain the equivalent functionality. Throughout this application note, references are made to the 'PCI bus'. Unless specifically indicated, this reference includes both the PCI and PCI-X bus functionality of the Reference Documents MROMB Design Considerations Using the Intel I/O Processor Application Note, Doc.# RAIDIOS (RAID I/O Steering) Design Guide, Doc. # Intel I/O processor Developer s Manual, Doc. # Document Organization This document is divided into three main sections, each with its intended audience. Section 2.0, I/O Processor Architecture Overviews, gives the general reader a summary of the differences between dual PCI bus architecture I/O processors and the single PCI bus architecture of the It also describes the system level implementation of the ROMB application. Section 3.0, Hardware Design Considerations, provides the system level H/W designer with the interrupt routing details and external logic required to control the IDSEL signal to implement the ROMB application using the Section 4.0, Software Design Considerations, provides the system level S/W designer with the modifications required to migrate a ROMB application implemented from a dual PCI bus I/O processor architecture to the single PCI bus and new features of the Appendix A, Microsoft WHQL* Considerations, addresses potential WHQL certification considerations Appendix B, ROMB Application Workload, addresses PCI-X bandwidth concerns when using the Appendix C, Zero-Memory ROMB Design with Intel I/O Processor, explains the usage of the without dedicated flash memory and SDRAM. Appendix D, Discrete Logic Example for IDSEL Control, shows an example circuit for controlling the IDSEL. Application Note 5

6 I/O Processor Architecture Overviews 2.0 I/O Processor Architecture Overviews To understand the hardware and software modifications necessary to port the ROMB application, it is best to review the individual architectures of an Intel I/O processor with a dual PCI bus and the single PCI bus In addition, sample system architecture implementations of both I/O processors will be discussed. While some functional units in the dual PCI bus architecture are not available in the design, new functions have been added to the architecture that software can utilize and obtain the equivalent capability. The dual PCI bus I/O processor that will be used for this document is the Intel I/O processor (80303). 2.1 Dual PCI Bus I/O Processor Architecture (Intel I/O Processor) The implements a dual PCI bus architecture through an on-board PCI-PCI bridge. For each PCI bus, Address Translation Units (ATU) and Direct Memory Access (DMA) units provide communication and data movement capabilities between the applicable PCI bus and the I/O processor core and local memory (flash or SDRAM). The dual PCI bus architecture diagram of the is shown in Figure 1. Figure 1. Intel I/O Processor Functional Block Diagram Local Memory (SDRAM, Flash) I 2 CSerialBus Core Processor Intel I/O Processor Memory Controller Bus Interface Unit I 2 CBus Interface Application Accelerator Internal Arbitration 64-bit Internal Bus Messaging Unit Two DMA Channels Address Translation Unit Performance Monitoring Unit One DMA Channel Address Translation Unit 64-bit/32-bit Primary PCI Bus PCI to PCI Bridge 64-bit/32-bit Secondary PCI Bus Secondary PCI 6 Application Note

7 I/O Processor Architecture Overviews 2.2 Dual-PCI Bus System Architecture An intelligent I/O application such as ROMB uses the primary bus as the interconnect bus to the host system PCI bus. This interconnect could be on the same PCI bus segment as the system PCI expansion slots or, for host chipsets that support multiple PCI bus segments, the ROMB could be dedicated to its own PCI bus segment. The secondary bus downstream from the PCI-PCI Bridge serves as the 'private' interconnect bus between the I/O processor and the I/O controller(s) (typically SCSI or Fibre channel). Flash and DRAM memory connected to the I/O processor local bus serves as storage for both execution code and data and RAID write-through and write-back data cache. A typical system architecture diagram of a ROMB application is shown in Figure 2. Figure 2. Dual PCI Bus ROMB System Architecture Host Processor Host Processor System Memory Host Bridge / Chipset Note: Some chipsets may support multiple peer busses. 64-Bit/66-Bit MHz PCI Slots IOP Flash and DRAM Intel I/O Processor SCSI Controller ROMB Subsystem A Application Note 7

8 I/O Processor Architecture Overviews 2.3 Single PCI Bus I/O Processor Architecture (Intel I/O Processor) The is a single function PCI device that integrates the Intel XScale core with intelligent peripherals. The consolidates, into a single component: Intel XScale core PCI - Local Memory Bus Address Translation Unit I 2 O* Messaging Unit DMA Controller Peripheral Bus Interface Unit Integrated Memory Controller Performance Monitor Application Accelerator I 2 C Bus Interface Unit Synchronous Serial Port Unit Eight General Purpose Input Output (GPIO) ports The PCI bus is capable of 133 MHz operation in PCI-X mode as defined by the PCI-X Addendum to the PCI Local Bus Specification, Revision 1.0a to the PCI Local Bus Specification, Revision 1.0. Also, the processor supports a 66 MHz conventional PCI mode as defined by the PCI Local Bus Specification, Revision 2.2. The addition of the Intel XScale core brings intelligence to the PCI bus application bridge. The is a single function PCI device. This function represents the address translation unit. The address translation unit is an 'application bridge' as definedbythepci-x Addendum to the PCI Local Bus Specification, Revision 1.0a to the PCI-X Addendum to the PCI Local Bus Specification, Revision 1.0a. The contains PCI configuration space accessible through the PCI bus. For detailed descriptions of the feature set, refer to the Intel I/O Processor Developer s Manual. 8 Application Note

9 I/O Processor Architecture Overviews As a single PCI bus architecture, the does not implement a PCI-to-PCI Bridge or the associated secondary PCI bus units (ATU, DMA). In addition, several control functions implemented in the PCI-to-PCI Bridge for controlling the Private Memory and configuring secondary I/O controllers as public or private is also not available. However, the architecture does incorporate new functions that allow an existing ROMB application to maintain the equivalent functionality. The single PCI bus architecture diagram of the is shown in Figure 3. Figure 3. Intel I/O Processor Architecture 72-Bit I/F 32-Bit I/F I 2 C Serial Bus Serial Bus Intel XScale Core DDR I/F Unit PBI Unit (Flash) I 2 C Bus Interface Application Accelerator SSP Serial Bus 64-bit Internal Bus Messaging Unit Address Translation Unit Two DMA Channels Performance Monitoring Unit PCI-X Bus (133 MHz) A The new functions added to the architecture applicable to the ROMB migration are as follows: Additional ATU Inbound Translation Windows BAR1, BAR2 and BAR3. Complete details of the architecture are covered in the Intel I/O Processor Developer s Manual. Application Note 9

10 I/O Processor Architecture Overviews 2.4 Single PCI Bus System Architecture The single PCI bus system architecture of a ROMB application using the is show in Figure 4. As shown, the and the I/O controller(s) are both connected to the same PCI bus segment. As with the dual PCI bus implementation, the ROMB application could be connected to the same bus segment as the PCI expansion slots or on a dedicated segment (as shown). The one difference with the implementation is the implementation induces two physical loads on the PCI bus vs. the 80303, which only induces one. All communications and data movement capabilities between the and the I/O controller(s) is through the single ATU or the two DMA units. In the absence of the PCI-to-PCI bridge and its associated capabilities as implemented in the 80303, additional logic is required to control the IDSEL signal to the I/O controllers and 'hide' the device from the host processor/operating system. This is explained in section 3.0, Hardware Design Considerations. In addition, the provides BAR1, BAR2 and BAR3 to offset the absence of SMBR and SMLR used by the PCI-to-PCI bridge. This is explained in section 4.3, Private Memory Space. Figure 4. Single PCI Bus ROMB System Architecture Host Processor Host Processor System Memory Host Bridge / Chipset SCSI Controller PCI-X 64-Bit/133 MHz PCI-X ROMB Subsystem Intel I/O Processor IOP Flash and DRAM A Application Note

11 Hardware Design Considerations 3.0 Hardware Design Considerations Three capabilities that were provided in the are the control of the PCI interrupts, the IDSEL to the I/O controllers and the capability to create a private memory address space for the controllers. A PCI-to-PCI Bridge supporting these functions is not implemented in the 80321, however, additional features have been implemented that help address the absence of these exact capabilities. The following sections detail the new features and the hardware design modifications required to implement the equivalent functionality. 3.1 Interrupt Routing In the standard implementation of an onboard PCI I/O device, its interrupts are routed to the system interrupt controller. Any interrupt from the I/O device requiring service by its host driver will be received by the host interrupt controller and serviced appropriately. To implement ROMB with the 80321, I/O device interrupt signal(s) must be routed to the so it can service the request instead of the system interrupt controller. The has four external interrupt input signals (XINT0#, XINT1#, XINT2#, and XINT3#) for this purpose. The interrupt out signal(s) from the I/O device can be connected to one of these external interrupt pins to be serviced by the The PCI Interrupt Routing Select Register (PIRSR - Table 1) in the can be used to direct the interrupt to the Intel XScale core by setting the appropriate bit in firmware. An example is shown in Figure 5. INTA# of the I/O device is connected to XINT0# of the 80321, firmware needs to set PIRSR.0 to direct the interrupt request to the processor core. With PIRSR.0 = '1', any interrupt from the I/O device will then be serviced by the RAID firmware running on the Figure 5. Intel I/O Processor Interrupt Routing System Interrupt Controller Intel I/O Processor PIRSR.0 I/O Device INTx#/ P_INT# P_INT# 0 1 XINT0# INTA# Intel XScale Microarchitecture A Application Note 11

12 Hardware Design Considerations Table 1. PCI Interrupt Routing Select Register (address = FFFF E2ECh) Bit Default Description XINT3# Select Bit 0 = Interrupt routed to P_INTD# pin. 1 = Interrupt routed to Intel XScale core interrupt controller input. XINT2# Select Bit 0 = Interrupt routed to P_INTC# pin. 1 = Interrupt routed to Intel XScale core interrupt controller input. XINT1# Select Bit 0 = Interrupt routed to P_INTB# pin. 1 = Interrupt routed to Intel XScale core interrupt controller input. XINT0# Select Bit 0 = Interrupt routed to P_INTA# pin. 1 = Interrupt routed to Intel XScale core interrupt controller input. 12 Application Note

13 Hardware Design Considerations 3.2 Initialization Device Select (IDSEL) When the system boots after a reset, the host BIOS initiates a PCI bus scan to find all the PCI components installed in the system. The system uses the IDSEL (Initialization Device Select) signal to address the I/O device when assigning the necessary resources. Without special control over the IDSEL signal during configuration cycles, the host and the may both attempt to configure the same I/O device. By taking control of IDSEL, the can execute configuration cycles to the slave I/O device (SCSI) and properly hide the slave I/O device (SCSI) from the host and operating system initiated configuration cycles. The has eight integrated general purpose input output (GPIO) pins, referred to as GPIO[7:0]. These pins can be used by the Intel XScale core to control the IDSEL to the I/O device. The output function of the GPIO pins is controlled by two registers as shown in Table 2 and Table 3. The output enables are mapped on a per bit basis to each of the data bits in the GPIO Output Data Register. When a bit of the GPIO Output Enable Register is cleared, the corresponding data bit value in the GPIO Output Data Register will be actively driven on the appropriate GPIO pin. To enable a particular GPIO pin as an output following the deassertion of P_RST#, a weak pull-down needs to be connected to the GPIO pin to overdrive the internal weak pull-up device. Table 2. GPIO Output Enable Register (address=ffff E7C4h) Bit Default Description 7:0 1 When clear, bit 7:0 of the GPIO Output Data Register will be enabled onto the GPIO[7:0] pin. Table 3. GPIO Output Data Register (address = FFFF E7CCh) Bit Default Description 7:0 0 This bit value will be driven on the GPIO[7:0] pin if bit 7:0 of the GPOE register is cleared. Application Note 13

14 Hardware Design Considerations The IDSEL signal is used as a chip select during configuration cycles initiated by the host BIOS, operating system or The GPIO can be driven high, in firmware, thereby hiding the host I/O device from the system. When the intends to perform configuration cycles on the PCI bus segment of the I/O device, the GPIO pin should be asserted low. The effect of these two operations is that the I/O device will be initialized and controlled by the More care must be taken with the gate chosen to control IDSEL since most host bridge controllers do not use PCI address stepping. With IDSEL being a synchronous signal with respect to CLK, the switch used must be a sub nanosecond propagation delay device. In Figure 6, a mux/demux bus switch is used to enable/disable IDSEL to the I/O device. Figure 6. IDSEL Steering Implementation AD[11+x] R1 I/O Device IDSEL PCI Bus AD[64:0] R2 GNT# CPLD or Discrete Logic GPIO[x] P_GNT# Intel I/O Processor B The host BIOS does not require any modifications to accommodate this implementation. All the responsibility for I/O device configuration and resourcing falls to the firmware Eliminating IDSEL Race Conditions with Host Care must be taken to ensure that the IDSEL for the I/O device is never enabled unless the has been granted the PCI bus for the configuration cycle it s attempting. One solution is to use P_GNT# signal, with CPLD based logic or discrete logic, to ensure that the IDSEL for the I/O device is only enabled when the is the PCI bus master. When the firmware drives the GPIO signal, used to enable the IDSEL of the I/O device, the P_GNT# logic will guarantee that the host (BIOS or OS) will not see the I/O device in any bus scan scenario. So with this implementation, the PCI bus scan order doesn t matter. The PCI bus can be scanned either way, increment or decrement. See Appendix D, Discrete Logic Example for IDSEL Control, for an example circuit. 14 Application Note

15 Software Design Considerations 4.0 Software Design Considerations Section 3.0, Hardware Design Considerations on page 11, discussed the system hardware design changes required to control the interrupt and IDSEL. In addition, the also adds new features to account for the absence of the Private Memory capability provided by the PCI-to-PCI bridge. The following sections describe the ROMB software design changes that are required to implement the hardware features. 4.1 Interrupt Control Internal interrupt routing for the is identical to that of the with the only difference being the naming convention of the inputs. For the 80303, the interrupt inputs used the naming convention of the PCI Local Bus Specification, Revision 2.2, INTA# - INTD#, however, internally inside the 80303, they were referenced as XINT0# - XINT3#. The uses the XINT0# - XINT3# naming convention only. Routing of these interrupts to either the Intel XScale core or to the PCI bus is controlled by the PCI Interrupt Routing Select Register (PIRSR - see Table 1). As shown in Figure 5,iftheI/O controller is to be controlled by the 80321, the corresponding PIRSR.X bit must be set to a '1' to route the interrupt to the Intel XScale core. If the is disabled on the motherboard, the I/O device interrupt will be forwarded to the system interrupt controller through the because the PIRSR default value is '0' which routes the interrupt to the P_INTx# signals. 4.2 Configuring and Initializing I/O Controllers (IDSEL) The PCI-PCI Bridge of the implements the capability to make devices on the secondary PCI bus downstream from the bridge either public (visible) or private (invisible) to the host processor. This capability provides the I/O processor with the ability to hide any devices it intends to control from the host processor during initialization configuration cycles and prevent conflicting control between the host and the I/O processor. This public/private capability is implemented by enabling (public) or disabling (private) the Initialization Device Select (IDSEL) signals during configuration cycles initiated by the host processor. This capability is implemented on the by using the GPIO outputs and external control logic as described in Section 3.2, Initialization Device Select (IDSEL) onpage Configuring the I/O Controller as Private The ROMB firmware executing on the must configure the GPIO output to a '1' to hide all I/O controllers under its control. Hiding the device should be executed immediately after boot to prevent the host BIOS from discovering the same controller and attempting to load its device driver resulting in a resource conflict. Application Note 15

16 Software Design Considerations Discovering the PCI I/O Controller Two methods can be implemented for discovering the I/O controllers that are to be controlled by the The first method is to hard code the PCI slot values in the firmware based on the hardware design of the ROMB implementation. The second method is by dynamic discovery during initialization. Before the begins any configuration cycles, it should hold off the host using PCI retries. Hard coded slot value(s) is the simplest to implement but lacks the flexibility to migrate the firmware to different platforms. The dynamic discovery process, provides the ROMB implementation with the greatest flexibility. In this method, the ROMB firmware can identify the I/O device it needs to control by completing a bus scan with GPIO high (i.e., the I/O devices IDSEL is off), and then comparing it to a bus scan with GPIO low (i.e., the I/O device IDSEL is on). Once the I/O device(s) is identified, the firmware must then properly resource the targeted system I/O device. Upon completion of all the necessary bus scans, GPIO is again toggled high, hiding the device from the system. The firmware then stops retrying the host-initiated configuration cycles and the system continues it's normal boot process. The must make certain not to interfere with system configuration cycles. This can be done during board layout as described in the hardware section of this paper. 16 Application Note

17 Software Design Considerations 4.3 Private Memory Space The does not have an integrated PCI-to-PCI Bridge, therefore the and the I/O device could be sharing the same PCI bus segment with other PCI devices. To avoid address conflicts, the includes new inbound ATU registers that can be used for various purposes. With the 80303, the PCI-to-PCI bridge provided a private memory space using registers SMBR and SMBL. This memory space could be used for private communications between the controlled I/O device and the 80303, or it could be used as a pool to allocate PCI memory space to the controlled I/O device. The Inbound ATU Base Address Register 1 (BAR1) and the ATU BAR1 Limit Register define a block of memory addresses for the inbound BAR1 translation window. This window is used merely to allocate memory on the PCI bus and the ATU will not claim any PCI bus transactions to this memory range. Think of BAR1 as a Proxy BAR that acts on behalf of the hidden I/O device who cannot request PCI memory on its own. Following configuration of the I/O controller(s), the ROMB firmware determines the total amount of PCI address space required for all controllers and requests that amount, rounded to the next logical PCI address size, in the BAR1 Limit Register. Once the BAR1 has been programmed, that address space can be allocated to the individual I/O controllers. Note that an interrupt is raised whenever BAR1 is changed, this allows the I/O device to be reconfigured appropriately, by the firmware, if and when the host changes the base address of the I/O device during post-configuration. The example in Figure 7 shows which addresses would be claimed by the inbound ATU of the Notice the ATU will not claim PCI bus transactions in the ATU BAR1 memory window, but will use this window to communicate to the I/O device as a private device. Another benefit of having the Inbound ATU BAR1 registers, is to allow a smaller memory range allocation for the I/O device. For example, if the primary ATU window needed to be 32 Mbytes and memory needed to be allocated for the I/O device, the next memory increment is 64 Mbytes. If the I/O device only needs 1 Mbyte of memory, then 31 Mbytes is being allocated unnecessarily. In addition to BAR0 and BAR1 shown below there are two additional Inbound ATU BAR registers,bar2andbar3.bar2canbeusedinthesamewayasbar0toprovideasecond window into the IOP local memory for flexibility in environments where there may be a maximum PCI window size requirement. The full window (including the bottom 4Kbytes) is available for use. Think of BAR2 as a Complimentary BAR, it does not need to be implemented if the designer chooses to only use BAR0 for access to local memory. BAR3 should be used for private communications, between the I/O device and the IOP, only when the traffic can be isolated via an external bridge or other isolation mechanism. BAR3 is not seen by the host and therefore the firmware is free to choose any range of addresses the designer chooses, thus it should only be used in conjunction with an external bridge or other isolation mechanism. Think of BAR3 as being the Hidden BAR. Typically, an external bridge would be more applicable for an adapter card that uses the 80321, (see RAID Adapter Card Considerations Using an Intel I/O processor for more information). Application Note 17

18 Conclusion Figure 7. Inbound Address Detection PCI Address Inbound ATU BAR1 + Value of BAR1 Limit Register Inbound ATU BAR1 Inbound ATU BAR0 + Value of BAR0 Limit Register Inbound ATU BAR0 BAR1 Inbound Translation Window BAR0 Inbound Translation Window Not Claimed by ATU Not Claimed by BAR1 Not Claimed by ATU Claimed by BAR0 Not Claimed by ATU A Conclusion The integration of interrupt steering, general purpose I/O and extended ATU registers to the creates an excellent solution for RAID on motherboard (ROMB) applications. System designers may use the information in this document as a basis for implementing ROMB designs with the As with any design, Intel recommends that customers perform thorough validation of their application hardware and software prior to production. 18 Application Note

19 Microsoft WHQL* Considerations Appendix A Microsoft WHQL* Considerations In the past, less complete architectures (SISL) identified various challenges with implementing a single PCI bus architecture and passing WHQL* certification tests. This application note describes the necessary components that should prevent this implementation from being a roadblock to WHQL certification. The architecture defined in this application note has been prototyped, including the dynamic discovery of the I/O device, and has passed 1 the following Microsoft pre-certification tests for WHQL: Table 4. WHQL Testing (Windows 2000*) Using HCT Version 9.5 with Update Pass Test Name Description X X X X X X X X Memory AWE (Server Test Suite) Syscache (Server Test Suite) NTFS File I/O (Server Test Suite) Stress Disk Test (Server Test Suite) Stress System Test (Server Test Suite) NTFS File I/O (RAID Test Suite) Syscache (RAID Test Suite) Device Stress (RAID Test Suite) The AWE test tests the Win32 Address Windowing Extensions on machines with more than 4 gigabytes (GB) of memory (RAM). The Address Windowing Extensions provide user applications with 32-Bit virtual addressing to greater than 32-bit regions of physical memory. This test reads, writes, and verifies data with and without the Windows NT* buffer feature. Testing halts when the SysCache test detects data corruption. This test uses various file I/O system calls to test the NTFS. The test reads, writes, and verifies a 1-MB pattern for 30 minutes by creating multiple threads for the drive you select. This ensures that the drive correctly reads and writes data. A drive partition can use any file system format and can include CD-ROM drives. A thread that tests a CD-ROM drive reads from, but does not write to, that drive. This test puts an abnormal load on the system by simultaneously stressing disk access, memory management, GDI operations, module management, and system caching. This test uses various file I/O system calls to test the NTFS. This test reads, writes, and verifies data with and without the Windows NT buffer feature. Testing halts when the SysCache test detects data corruption. SdStress tests storage API commands and stresses the hard disks and CD-ROMs in a system. 1. Note that pre-certification tests passing does not insure that the pre-tested system would pass official testing but it is the best indication- short of submitting a production system. Intel does not guarantee that a customer's final implementation based on this application note will pass WHQL. The data above is provided to help alleviate potential concerns with WHQL testing prior to initial design/concept approval. Application Note 19

20 ROMB Application Workload Appendix B ROMB Application Workload ROMB designs based on the share the PCI-X segment with other devices. The PCI-X segment bandwidth is therefore shared between the RAID application and the other PCI device. While this could result in contention during simultaneous peak throughput of the two devices, the typical usage of these devices, is supported comfortably on the shared PCI-X segment. The following analysis will examine the bandwidth demands of a typical ROMB workload to determine the bus bandwidth remaining for the other PCI device. A typical ROMB application workload is a mix of random 8 KB I/O workload with 67% disk reads and 33% disk writes. The typical RAID level is RAID-5 with six disks per channel. Under this configuration and workload, the I/O rate is dictated primarily by the drives (or number of drives) and not the I/O Processor, PCI bus or SCSI bus. Figure 8. RAID-5 ROMB Typical Workload Data Flow PCI Device Intel I/O Processor Chipset msg PCI Bus 8 MB/s Write cmd 16 old 16 new SCSI 16 MB/s Read A Under this workload, the I/O data flow for the PCI traffic is illustrated in Figure 8. The values in the figure represent a high forecast of the data bandwidth used for the I/O data. Disk reads are transferred directly from the SCSI devices to host/system memory via the chipset across the shared PCI-X bus. This data transfer is represented by the straight arrow at the bottom of the figure. RAID-5 Disk writes require data transfers from both host memory and the SCSI bus. These data transfers are shown as the curved arrows. RAID-5 data transfers include write data from host memory to the 80321, read of old data/parity from SCSI bus and write new data/parity from the IOP to the SCSI bus. 20 Application Note

21 ROMB Application Workload Assuming an optimistic performance forecast with disks capable of 1000 IOPs for random I/Os, and six disks per channel, a maximum of 6000 IOPs per channel could be supported. Based on 8 KB random I/Os, a maximum bandwidth to the disks on a single SCSI channel would be 48 MB/s per channel. Using the 67% read percentage of the typical workload, the application throughput would be 8 MB/s of writes and 16 MB/s of read traffic. The 8 MB/s of disk writes result in an additional 16 MB/s of read and 16 MB/s of writes across the PCI-X bus to the SCSI device. The total bandwidths for this case and a 2-channel case are summarized in Table 5 below. Table 5. ROMB RAID-5 Typical Workload Throughput Ch/disk Application SCSI Bus PCI Used PCI Remain 1xU320 / 6 24 MB/s 48 MB/s 70 MB/s 730 MB/s 2xU320 /12 48 MB/s 96 MB/s 140 MB/s 660 MB/s Under this typical RAID-5 workload, the majority of the PCI-X 100 MHz bandwidth remains for use by other applications and the other I/O device on the PCI-X bus. The PCI used total includes 20% overhead for messaging, SCSI commands, and data transfer overhead cycles. The result is that 730 MB/s of the 800 MB/s bandwidth is available on a ROMB implementation under this typical load. A total of 660 MB/s remain for the two channel case. This illustrates how, for a based ROMB implementation, the majority of the PCI-X bandwidth still remains for other I/O devices which may share that PCI-X segment. Application Note 21

22 Zero-Memory ROMB Design with Intel I/O Processor Appendix C Zero-Memory ROMB Design with Intel I/O Processor The supports low cost ROMB designs. There may be applications where extreme cost cutting or board space limitations require a minimal ROMB implementation. The allows designers to meet this design objective with a ROMB design requiring no memory. This zero-memory ROMB design eliminates the cost of the Flash memory device and SDRAM (DIMM and socket), and significantly reduces the I/O subsystem footprint. The hardware design is simpler, with only the and motherboard SCSI controller to implement as devices on a PCI-X bus segment. There are some modifications to the boot sequence and initialization code to support this zero-memory ROMB design. The modifications are summarized in Table 6: Table 6. Boot Sequence and Initialization Code Modifications Function ROMB Zero-Memory ROMB Codestore StoredinIOPFlash StoredinsystemBIOSFlash Execution RAM Local IOP SDRAM System RAM accessed across PCI bus via ATU Boot RAID IOP Core processor self boots from local Flash Resources mapped to local SDRAM, SCSI device mapped in PCI via ATU System BIOS configures IOP to boot via PCI, then releases core processor from reset Resources and SCSI device mapped in PCI via ATU The system BIOS is responsible for booting the I/O processor using the following sequence. 1. Processor powers up with the core held in reset. 2. BIOS loads the desired reset vector into mechanism registers. 3. BIOS enables the mechanism registers to respond to 0x0h boot address. 4. BIOS configures the processor ATU appropriately. 5. BIOS allocates system memory and copies IOP code to system memory. 6. BIOS releases the core processor from reset. 7. Core processor boots to system memory address and begins program execution across PCI-X bus. Core processor continues to process code fetches, which are retrieved from system memory via the ATU and PCI-X bus based on the configuration performed by the BIOS. Detailed register configuration settings required for Zero-Memory ROMB design with the 80321: Disable PBI Window 0 in the PBI Limit Register (PBLR) Enable Intel Xscale Core PCI Bus Boot mode (PBCR bit 3) Write a short branch into PMBR0 register (branch to 0x20h) Write a long branch into PMBR1 and PMBR2 registers (branch to system memory address via ATU window) Setup ATU outbound window translate value OMWTVR0 If system memory address is above 4 GB, also program upper 32-bit address in the OUMWTVR0 Boot Core: PCI Configuration and Status Register (PCSR bit 1) 22 Application Note

23 Discrete Logic Example for IDSEL Control Appendix D Discrete Logic Example for IDSEL Control Figure 9. Discrete Logic Example for IDSEL Control VCC 4.7K PCI_GNT# GPIO[x] 7402 PR# D Q 7474 To IDSEL Switch Logic Q CL# PCI_CLK PCI_RST# A Application Note 23

24 Discrete Logic Example for IDSEL Control This page intentionally left blank. 24 Application Note