Oracle <Insert Picture Here>

Transcription

1 Slide 1 Oracle <Insert Picture Here>

2 Slide 2 <Insert Picture Here> WZT-5160 SPARC T3 Based Servers Architecture and Features Welcome to the SPARC T3 Based Servers Architecture and Features module. This content will cover the overall architecture and major features of Oracle s SPARC T3-1, SPARC T3-2, and SPARC T3-4 rackmount servers along with Oracle s SPARC T3-1B server blade.

3 Slide 3 Objectives Identify the features of the servers Explain the architecture of the servers List the server s FRUs and CRUs 3 Upon completion of this module, you should be able to: 1 Identify the features of the servers 2 Explain their architecture and 3 List their FRUs and CRUs

4 Slide 4 Additional Resources Oracle, Inc., SPARC T3-1, T3-2 and T3-4 Servers Administration Guide, Revision A, June 2010, Part Number xx. Oracle, Inc., SPARC T3-1 Servers Installation Guide, Revision A, Oct 2010, Part Number xx. Oracle, Inc., SPARC T3-2 Servers Installation Guide, Revision A, Sept 2010, Part Number xx. Oracle, Inc., SPARC T3-4 Servers Installation Guide, Revision A, Aug 2010, Part Number xx. Oracle, Inc., SPARC T3-1B Servers Installation Guide, Revision A, Aug 2010, Part Number xx. Oracle, Inc., SPARC T3-1 Servers Service Manual, Revision A, Oct 2010, Part Number xx. Oracle, Inc., SPARC T3-2 Servers Service Manual, Revision A, Aug 2010, Part Number xx. Oracle, Inc., SPARC T3-4 Servers Service Manual, Revision A, Aug 2010, Part Number xx. 4 The following references are 1 the technical documentation for the SPARC T3 based server platforms.

5 Slide 5 Additional Resources Oracle, Inc., SPARC T3-1B Servers Service Manual, Revision A, Sept 2010, Part Number xx. Oracle, Inc., SPARC T3-1 Servers Product Notes, Revision A, Oct 2010, Part Number xx. Oracle, Inc., SPARC T3-2 Servers Product Notes, Revision A, Aug 2010, Part Number xx. Oracle, Inc., SPARC T3-4 Servers Product Notes, Revision A, Aug 2010, Part Number xx. Oracle, Inc., SPARC T3-1B Servers Product Notes, Revision A, Aug 2010, Part Number xx. Oracle, Inc., Oracle Integrated Lights Out Manager (ILOM) 3.0 Web Interface Procedures Guide, Revision A, May 2010, Part Number, xx. Oracle, Inc., Oracle Integrated Lights Out Manager (ILOM) 3.0 CLI Procedures Guide, Revision A, May 2010, Part Number xx. Oracle, Inc., Oracle Integrated Lights Out Manager (ILOM) 3.0 SNMP and IPMI Procedures Guide, Revision A, May 2010, Part Number xx. 5 The following references are 1 the rest of the technical documentation for the SPARC T3 based server platforms.

6 Slide 6 Which are the Servers? 1 CPU socket SPARC T3-1 (Serpa) 2 CPU sockets SPARC T3-2 (Solana) 4 CPU sockets 1 CPU socket SPARC T3-1B (Jumilla) SPARC T3-4 (Seville) 6 There are four servers that support the new SPARC T3 CPU chip. 1 Oracle s SPARC T3-1 Server, code named Serpa, is a 2U rack mount server that supports one CPU socket. 2 Oracle s SPARC T3-2 Server, code named Solana, is a 3U rack mount server that supports 2 CPU sockets. 3 Oracle s SPARC T3-4 Server, code named Seville, is a 5U rack mount server that supports up to 4 CPU sockets. 4 Oracle s SPARC T3-1B Server, code named Jumilla is a server blade that supports one CPU socket and is supported on the Sun Blade 6000 Modular System.

7 Slide 7 Which is the CPU chip? NOTE: The manner in which the s and s are packaged is platform dependent 7 All four servers support the SPARC T3 CPU chip. 1 Each CPU chip has 16 cores and 8 threads per core for a total of 128 threads per chip that runs off of a 1.65 GHz clock. 2 Each core has 16 KBytes of instruction cache, 8KBytes of data cache, a floating point and graphics unit along with a cryptographic accelerator. 3 All the cores share 6 MBytes of level 2 cache that is accessed through the crossbar circuitry. 4 The chip s memory interface consists of two memory controllers which support two memory channels each. External to the chip, the memory channels interface to the DDR3 s through memory buffers referred to as Buffer on Board chips or s. Each memory channel supports two s and each has two channels that support two slots each for a total of 16 slots. Populating the slot with 8 Gbyte s gives you a maximum of 128 Gbytes of memory per CPU chip. 5 Note, the manner in which the s and s are packaged is platform dependent so we will discuss each packaging scheme when we discuss the individual platforms later in this presentation. 6

8 Slide 8 Which is the CPU chip? 8 Two coherence units, that provide 3 links each, are also available to provide shared memory paths out to other CPUs that may be on the server. 1 In terms of I/O, the CPU chip provides two 8 lane PCIe generation 2 interconnects 2 and two XAUI 10 Gigabit Ethernet interfaces. These interfaces will be discussed in more detail later in this presentation. 3

9 Slide 9 Which I/O is Supported? CPU and Memory Subsystem x1 x4 Switch 0 IDT 48H12G2 Partitionable (48 lane) Switch 1 IDT 64H16G2 Partitionable (64 lane) x1 x4 9 There are several industry standard I/O interconnects and buses supported by the SPARC T3 based servers. 1 The main I/O protocol used on the servers is PCIe generation 2 interconnects which originate at the CPUs and are expanded within the server 2 using two switch chips. Depending on the server, the two switch chips used could be 48 or 64 lane types, as displayed here, 3 and can support up to 8 lane interconnects within the server architecture. The 4 and 8 lane interconnects support mainly ports while other I/O devices are supported directly or indirectly through the interconnects. 4

10 Slide 10 Which I/O is Supported? CPU and Memory Subsystem XAUI Switch 0 IDT 48H12G2 Partitionable (48 lane) Switch 1 IDT 64H16G2 Partitionable (64 lane) LSI SAS 2008 SAS/SATA Controller RAID 0 and 1 to-pci Bridge PCI-to-USB Bridge Intel EB MAC/PHY REM (on SPARC T3-1B) Ethernet Adapter(s) 10-Gbit Ethernet ports 10 The interconnects also support other interconnects and buses, through the use of bridge chips and daughter cards, 1 such as LSI controller chip, USB controller chips, 1Gbit Ethernet controller chip and the RAID Expansion Module or REM board on the SPARC T3-1B blade server. The SPARC T3 CPUs also 2 provides two XAUI interfaces that are used to provide 10-Gbit Ethernet ports through an Ethernet adapter slot. The number of ports and adapter slots are dependent on the server platform. Since the I/O infrastructure is different for each server platform we will defer that discussion to that section of this presentation. 3

11 Slide 11 Server Features Comparison Jumilla Solana Serpa Seville Features SPARC T3-1B SPARC T3-1 SPARC T3-2 SPARC T3-4 Size (RU) 1 slot blade 2RU 3RU 5RU Cores/Threads 16 / 128 (1 CPU) 16 / 128 (1 CPU) 32 / 256 (2 CPUs) 64 / 512 (4 CPUs) Memory (max) 128 GBytes 128 GBytes 256 GBytes 512 GBytes Slots 2 EM slots 2 NEM ports GbE / 10 GbE ports 2 / 2 4 / 2 4 / 4 4 / 8 HDD or SSD (max) LDoms (recommended) Service Processor ILOM 3.0 ILOM 3.0 ILOM 3.0 ILOM A summary of the server features as displayed in this table for a side by side comparison. A copy of this table is available in PDF format that you can download by selecting the Server Features Comparison entry under the attachment tab. 1

12 Slide 12 Specific Server Architecture and Features SPARC T3-1B SPARC T3-2 Jumilla Solana SPARC T3-4 Seville SPARC T3-1 Serpa 12 Now you have a choice to make. Select one of the four arrow icons. This will branch you to that specific server s architecture and features. Note, you will be given the option to cover all the other server s information later on. 1

13 Slide 13 Front of the SPARC T3-1 Server Front CD/DVD (2)USB CD/DVD 6 7 (2)USB 8-disk config System Indicators Locator LED (white) / Button Service Action Required LED (amber) Power/OK LED (green) Power Button Fault Indicators Power Supply Fault LED (amber) Temperature Threshold LED (amber) Fan Fault LED (amber) Disk slots label 13 The SPARC T3-1 server s front panel has the following features. 1 On the top left are the system indicators that consist of the white locator LED and button assembly used as a beacon to locate the server within a data center, an amber service action required LED that is on when a hardware failure has occurred within the server, a green power OK LED that is used to determine the power state of the server and the power button that is used to turn on the power to the server s host. 2 There are 16 disk drive slots available on the SPARC T3-1 server that can support either 8 or 16 SAS or SATA hard disk drives or solid state disk drives, known as HDDs or SSDs. The numbers represent their slot number as well as their SAS or SATA IDs. 3 On the right of the faceplate above disk slots 10 and 13, you have the CD/DVD drive along with USB ports 2 and 3. 4 On the far right are the fault indicators and the disk drive labels. The fault indicators consist of the power supply Fault LED, the amber temperature threshold LED and amber fan fault LED. 5

14 Slide 14 Rear of the SPARC T3-1 Server 10-GbE Rear Power Supplies 1+1 redundancy hot swap VAC or VAC Gold+ efficiency rating NET0 NET3 NET MGT SER MGT System Indicators (duplicate) (2) USB Video port 14 The SPARC T3-1 server s rear panel has the following features. 1 On the left are two 1+1 redundant hot swappable power supplies that run on 110 or 220 VAC which has a gold+ efficiency rating. At 110 VAC it generates up to 1100 watts of power and at 220 VAC it generates up to 1200 watts of power. 2 There are six slots that support adapters with slots 0 and 3 able to support XAUI adapters. Below slot 0, there is 3 a serial and network management port that are connected to the service processor and a duplicate set of system indicators. The same indicators we went over on the front faceplate. To the right of the network management port 4 are four 1-Gbit Ethernet ports, numbered 0 to 3 left to right, that are used by server host. To the right of these ports 5 are USB ports 0 and 1 and a video port. These ports can be used to connect a console keyboard, mouse and a monitor. 6

15 Slide 15 Inside the SPARC T3-1 Server power supply riser riser Adapter Slot 0 and 3 Adapter Slot 1 and 4 memory SAS/SATA cable harness memory CPU fan assemblies disk backplane and slots riser Adapter Slot 2 and 5 memory SAS/SATA cable harness memory 15 The key components of the SPARC T3-1 server includes 1 the CPU, its memory and 2 the adapter cards within the risers. The other components are 3 the fan assemblies, 4 the power supply subsystem, 5 the disk backplane and slots along with the SAS/SATA cabling harness. To display a list of FRUs and CRUs click on the SPARC T3-1 FRUs and CRUs entry within the attachment tab. For a closer look at the inside of the server select SPARC T3-1 3-D entry from the attachment tab. Using the control panel that comes with it, you can see the server in any of 360 degree orientations, zoom in and out on components, and remove components to view other components within the enclosure. 6

16 Slide 16 SPARC T3-1 CPU and Memory A B C D SPARC T3 Densities: 2, 4 or 8-Gbyte registered s Speed: 1066 MT/sec 16 The SPARC T3-1 server has four memory channels each supporting a Buffer on Board or Bob. 1 Each Bob has two channels that supports two slots. 2 Each slot supports either a 2, 4 or 8-Gbyte registered 3 that operate at 1066 Megatransfers per second. Note, the population and replacement rules will be covered in the Installation and Replacement module of this course. 4

17 Slide 17 SPARC T3-1 I/O Subsystem Memory Subsystem disk1 disk3 disk5 disk7 disk9 disk11 disk13 disk15 disk0 disk2 disk4 disk6 disk8 disk10 disk12 disk14 disk0 disk1 disk2 disk3 disk4 disk5 disk6 disk7 SAS/SATA Expander SAS/SATA Expander 4 A B SPARC T3 4 A B B3 A0 (2) XAUI LSI SAS2008 SAS/SATA Controller 0 RAID 0,1,0/1 LSI SAS2008 SAS/SATA Controller 0 RAID 0,1,0/1 DVD SATA XAUI port 3 () XAUI port 0 (x16) A4 B1 Switch 0 48 ports port 4 () port 1 (x16) B5 A0 Intel 82576EB MAC/PHY 0 A2 A4 Intel 82576EB MAC/PHY 1 Quad 1-GbE A2 C x1 Switch 1 64 ports x4 x4 port 5 () port 2(x16) x1 USB Hub Front side USBs B1 B5 B3 / PCI Bridge PLX PEX8112 PCI / USB Bridge NEC upd Internal USB C Service Processor MAX3241 BCM5221 video SERMGT NETMGT USB Hub Rear side USBs 17 The I/O subsystem of the SPARC T3-1 servers originates directly from the SPARC T3 CPU through 1 two 8 lane interconnects that interface to a 48 port and an 64 port switch. 2 Each switch supports 3 slots. All six slots support 8 lane adapters. Slots 0,1 and 2 can also support 16 lane adapters, physically but not electrically. 3 Also, slots 0 and 3 are connected to the XAUI interfaces of the CPU chip and can support 10-Gigabit Ethernet ports. 4 Other 8 lane interconnects support the internal SAS or SATA disk drives through two LSI SAS/SATA controllers. Depending on disk backplane and the cable harness used, either 8 or 16 disks can be supported through a primary and a secondary path. One of the controllers also supports the CD/DVD drive. 5 The 64 port switch provides two 4 lane interconnects that supports two Ethernet controllers that hosts four 1-Gbit Ethernet ports located on the rear of the server. 6 The 64 port switch also supports the server s five USB ports and 7 the service processor s interface to the CPU through two separate one lane interconnects. 8

18 Slide 18 Front of the SPARC T3-2 Server System Indicators and buttons: Locator LED/Button assembly (white) Service Action Required LED (amber) OK/Power LED (green) Power button Fault indicators: SP, Fan, CPU, Memory, Power supplies and temperature Fan baffle Disk Slots for (6) SAS/SATA disks USB 2 & 3 video port CD/DVD drive 18 The SPARC T3-2 server s front panel has the following features. 1 On the top left are the system indicators that consist of the white locator LED and button assembly used as a beacon to locate the server within a data center, an amber service action required LED that is on when a hardware failure has occurred within the server, a green power OK LED that is used to determine the power state of the server and the power button that is used to turn on the power to the server s host. For hardware failure isolation there is amber fault LED for the service processor, fan module, CPU, memory, power supplies and temperature threshold. 2 There are 6 disk drive slots available on the SPARC T3-2 server that can support SAS or SATA hard disk drives or solid state disk drives, better known as HDDs and SSDs. The numbers represent their slot number as well as their SAS or SATA IDs. 3 On the bottom right of the faceplate you have the CD/DVD drive while on 4 the bottom left are USB ports 2 and 3 and the video port. 5

19 Slide 19 Rear of the SPARC T3-2 Server Rear video SER MGT NET MGT Power Supplies N+1 redundancy hot swap VAC Network Module (NM) slot (2) USB for a Quad 10-GbE adapter System Indicators (duplicate) Quad 1-GbE ports NET0 NET3 19 The SPARC T3-2 server s rear panel has the following features. 1 On the left are two N+1 redundant hot swappable power supplies that run on 220 VAC which can generate up to 2000 watts of power. 2 These 10 slots support adapters. Every slot supports 8 lane adapters except for slots 4 and 5 which support 4 lane PCIe adapters. 3 A dedicated slot, referred to as the network module slot or NM, is able to support a Quad 10-Gigabit Ethernet adapter. Between the power supplies and slot 0 are 4 a duplicate set of indicators we went over on the front faceplate. 5 A serial management port, a network management port and a video port, that are connected to the service processor, are located between the NM slot and slot 5. Below the management ports are 6 four host based 1-Gigabit Ethernet ports and USB ports 0 and 1. The video port and two USB ports can be used to connect a monitor, keyboard and mouse for the system console when needed. 7

20 Slide 20 Inside of the SPARC T3-2 Server Power Supply Disk Drive slots Filler adapter slots and NM slot CPU1 Memory Riser Board 1 Memory Riser Board 0 Filler Filler Fan Modules CPU0 Memory Riser Board 1 Memory Riser Board 0 Filler 20 The key components of the SPARC T3-2 server includes 1 two CPUs and up to two memory riser boards per CPU. Notice that the open board slots need a filler to avoid disruption of the cooling paths. 2 The adapter slots and the network module slot are located towards the rear of the motherboard. 3 The disk drive slots and power supplies are located on the right side of the server chassis. To the right of the memory riser boards are 4 the fan assemblies that provide the cooling for all of the server s internal components. To display a list of FRUs and CRUs click on the SPARC T3-2 FRUs and CRUs entry within the attachment tab. For a closer look at the inside of the server select SPARC T3-2 3-D entry from the attachment tab. Using the control panel that comes with it, you can see the server in any 360 degree orientation, zoom in and out on components, and remove components to view other components within the enclosure. 5

21 Slide 21 SPARC T3-2 CPU and Memory Memory Riser Board 1 Memory Riser Board 1 CPU0 CPU1 Memory Riser Board 0 D C SPARC T3 SPARC T3 D C Memory Riser Board 0 A B A B Densities: 2, 4 or 8-Gbyte registered s Speed: 1066 MT/sec 21 The SPARC T3-2 server has four memory channels per CPU with each channel supporting a Buffer on Board or. 1 Each has two channels that supports two slots. 2 The memory subsystem for each CPU is packaged in two memory riser cards with each supporting two s and eight slots. 3 Each slot supports either a 2, 4 or 8-Gbyte registered 4 that operate at 1066 Megatransfers per second. Note, the population and replacement rules will be covered in the Installation and Replacement module. 5

22 Slide 22 Memory Subsystem Memory Subsystem SPARC T3-2 I/O Subsystem disk0 disk1 disk2 disk3 disk4 disk5 A0 CPU1 SPARC T3 CPU0 PCIe port 0 () A4 PCIe port 4 (x4) SPARC T3 SAS/SATA Expander PCIe A1 (2) XAUI PCIe PCIe PCIe (2) XAUI 4 PCIe port 1 () B5 PCIe port 5 (x4) 4 D E A2 PCIe port 2() B6 PCIe port 6() LSI SAS2008 SAS/SATA Controller 0 RAID 0,1,0/1 A3 E DVD SATA PCIe port 3 () B7 PCIe port 7 () PCIe Switch 0 64 ports A4 A3 A0 Intel 82576EB MAC/PHY 0 A1 A3 D C Intel 82576EB MAC/PHY 1 Quad 1-GbE x1 PCIe Switch 1 64 ports x4 B5 x4 B6 x1 USB Hub Front side USBs B7 B8 B9 PCIe / PCI Bridge PLX PEX8112 PCI / USB Bridge NEC upd Internal USB C Service Processor MAX3241 BCM5221 video SERMGT NETMGT USB Hub Rear side USBs B8 B9 (4) XAUI PCIe port 8 () PCIe port 9 () NM Quad XAUI 22 The I/O subsystem of the SPARC T3-2 server originates directly from the SPARC T3 CPUs through 1 four 8 lane interconnects that interface to the two 64 port switches to create I/O paths from each CPU to both switches. 2 Each switch supports 5 slots. Eight of the ten slots support 8 lane adapters while slot 4 and 5 which support 4 lane adapters. 3 Also, the XAUI interfaces of the CPU chips are connected to a network module slot that supports an 10-Gigabit Ethernet adapter with up to 4 ports. 4 Other 8 lane interconnects support eight internal SAS or SATA disk drives through a LSI SAS/SATA controller using a primary and a secondary path to each device. The LSI controller also supports the SATA CD/DVD drive. 5 One of the 64 port switches provides two 4 lane interconnects that supports two Ethernet controllers that hosts four 1-Gbit Ethernet ports located on the rear of the server. 6 The same 64 port switch also supports the server s five USB ports and 7 the service processor s interface to the CPU through two separate one lane interconnects. 8

23 Slide 23 Front of the SPARC T3-4 Server A PM1 CPU2 & 3 C B 1 0 PM0 CPU 0 & D A. System Indicators and buttons: B. Storage Locator LED/Button assembly (white) (8) SAS/SATA disks Service Action Required LED (amber) OK/Power LED (green) Power button Fault indicators: temperature, Fan, and EMs C. Ports video NET MGT (2) USB D. Power Supplies: 2+2 redundancy VAC/unit 2000 watts/unit 23 The unique feature of the SPARC T3-4 server it that all its components are accessible from either the front or rear of the unit. The SPARC T3-4 server s front panel features are displayed. 1 The CPU and memory is packaged on two processor modules that contain 2 CPU sockets and 32 sockets each. 2 On the left are the system indicators that consist of the white locator LED and button assembly used as a beacon to locate the server within a data center, an amber service action required LED that is on when a hardware failure has occurred within the server, a green power OK LED that is used to determine the power state of the server and the power button that is used to turn on the power to the server s host. For hardware failure isolation there is amber fault LED for the temperature threshold, fan module and Express Modules. Note, each system module has their own set of indicators for further isolation. 3 There are 8 disk drive slots available on the SPARC T3-4 server s main module that support SAS or SATA hard disk drives or solid state disk drives, known as HDDs and SSDs. The numbers represent their slot number as well as their SAS or SATA IDs. 4 To the left of the main module are the video port, network management port and the two USB ports. 5 Finally, on the bottom of the server chassis are the 4 power supply slots. When populated, the server supports a 2+2 power supply redundancy with individual supplies that run at 200 to 240 VAC and can generate up to 2000 watts of power. 6

24 Slide 24 Rear of the SPARC T3-4 Server PM0 CPU0 PM1 CPU2 PM0 CPU1 PM1 CPU3 24 The SPARC T3-4 server s rear panel has the following features. The 16 slots that support adapters are referred to as Express Modules or EMs. 1 Assuming both processor modules are installed, the EM slots are directly accessible by the individual CPUs as you see here. 2

25 Slide 25 Rear of the SPARC T3-4 Server PM0 CPU0 PM0 CPU1 25 If processor module 1 is not present and a PM filler is in its place, the two remaining CPUs from PM0 directly access the EM slots as you see here. So, for proper operation of the server PM0 needs to be present in a two or four CPU configuration. 1 On the top are the fan assembles that allows for front-to-rear air flow that cools the internal components of the server. 2 On the left of the chassis are the four AC power connectors that provides power to the four power supplies in the front chassis. 3

26 Slide 26 Rear of the SPARC T3-4 Server System Indicators 10-GbE ports 4-7 SER MGT 10-GbE ports 0-3 NET MGT GbE NET To the right of the AC connectors is the rear I/O module, 1 referred to as RIO, which contains the system indicators and ports. 2 The system indicators are a duplicate of the ones on the front chassis. 3 The ROI also supports 10-Gigabit Ethernet ports, that originate from the CPUs, through QSFP connections. 4 It also supports 1-Gigabit Ethernet ports, that originate on the onboard Kawela controller chips and are controlled by the service processor. 5 The other two RJ-45 connectors support the serial and network management interfaces to the service processor or SP. To display a list of FRUs and CRUs click on the SPARC T3-4 FRUs and CRUs entry within the attachment tab. For a closer look at the inside of the server select SPARC T3-4 3-D entry from the attachment tab. Using the control panel that comes with it, you can see the server in any 360 degree orientation, zoom in and out on components, and remove components to view other components within the enclosure. 6

27 Slide 27 Inside of the SPARC T3-4 PM CPU0 memory CPU0 CPU0 memory CPU0 memory CPU0 memory CPU1 memory CPU1 CPU1 memory CPU1 memory CPU1 memory 27 The key components of the SPARC T3-4 processor module includes two CPUs and their corresponding memory, as you can see here. 1

28 Slide 28 Inside of the SPARC T3-4 Main Module SP REM0 disks REM1 disks 28 The key components of the SPARC T3-4 main module includes two RAID Expansion Modules, 8 disks and the service processor, as you can see here. 1

29 Slide 29 SPARC T3-4 Processor Module 0 CPU0 CPU1 D C SPARC T3 SPARC T3 D C A B A B 29 The SPARC T3-4 server has two identical processor modules. PM0 contains two CPU chips, CPU0 and CPU1, with four memory channels per CPU with each channel supporting 1 a Buffer on Board or. Each has two channels that supports two sockets for a total of 32 sockets within PM0. 2

30 Slide 30 SPARC T3-4 Processor Module 1 CPU2 CPU3 D C SPARC T3 SPARC T3 D C A B A B Densities: 2, 4 or 8-Gbyte registered s Speed: 1066 MT/sec 30 PM1 contains CPU2 and CPU3 along with 32 sockets. 1 Each socket supports either a 2, 4 or 8-Gbyte registered 2 with speeds of up to 1066 Megatransfers per second. Note, the population and replacement rules will be covered in the Installation and Replacement module. 3

31 Slide 31 SPARC T3-4 CPU and I/O memory PM0 memory memory PM1 memory CPU0 CPU1 CPU2 CPU3 Switch 0 Switch 2 Switch 1 Switch 3 31 The connection scheme between the CPU and the I/O was designed to get the most redundancy in the system. Notice that each PM has access to all four switches and thus have access to the complete I/O subsystem. Each CPU also has a coherency link to each of the other CPUs, as you see here. 1

32 Slide 32 SPARC T3-4 CPU and I/O memory CPU0 PM0 memory CPU1 PM1 Filler Module Switch 0 Switch 2 Switch 1 Switch 3 32 So if PM1 is removed and replaced with a filler card, PM0 will still have access to the complete I/O subsystem. The filler module serves to maintain air flow to the other components in the server but also provides a pass-thru connection on the coherency connection between CPU0 and CPU1 for PM0. 1

33 Slide 33 SPARC T3-4 CPU Main I/O Board Switch 0 x4 x4 Switch 2 Switch 1 Switch 3 REM0 (3) to slots (5) to slots (5) 8 lane (3) 8 lane REM1 BP x1 CPU0 CPU1 CPU2 CPU3 XAUI XAUI XAUI XAUI BP HDD 0-3 PCIe / PCI Bridge PLX PEX8112 x1 Service Processor MAX3241 BCM5221 video Kawela Kawela Intel Intel 82576EB 82576EB MAC/PHY MAC/PHY Gbit Network Board HDD 4-7 USB Hub USB Hub SERMGT NETMGT Quad 1-GbE Eight 10-GbE ports PCI / USB Bridge NEC upd SW2 SW2 SW0 SW2 SW2 SW0 SW2 SW0 SW1 SW3 SW1 SW3 SW1 SW1 SW3 SW1 Front side USBs (FIO) Internal USB Rear side USBs (RIO) The four 64 port switches support 1 the 16 8 lane slots that are referred to as Express Modules. 2 switch 0 and 3 each connects to a RAID Expansion Module. Each REM supports a four slot backplane that can house hard disk drives or solid state drives, referred to as HDDs and SSDs. 3 Switch 0 also supports five USB connectors through a one lane connection. Two in the front I/O assembly, two in the rear I/O board and one on the main I/O module. 4 The four 1-Gigabit Ethernet ports on the RIO is supported through two Kawela Ethernet controllers that is connected through two four lane interconnects, 5 while a one lane interconnect supports the service processor. 6 There are also eight 10-Gigabit Ethernet ports, that are supported directly by the XAUI interfaces on each of the CPUs, that are resident on the 10-Gigabit Ethernet Network Board. 7

34 Slide 34 Front of the SPARC T3-1B 3 2 System Indicators and buttons: Locator LED/Button assembly (white) Ready-to-Remove LED (blue) Service Action Required LED (amber) OK/Power LED (green) Power button Non-maskable Interrupt button (NMI) 1 0 P/N Option #4622A (2)USB Serial video management port 34 The SPARC T3-1B server blade is supported on the Sun Blade 6000 chassis. As with all the Sun Blade 6000 server blades, it contains some standard features. 1 The system indicators consist of the white locator LED and button assembly used as a beacon to locate the server within a data center, a blue ready-to-remove LED that is on when the server blade is ready to be removed from its slot, an amber service action required LED that is on when a hardware failure has occurred within the server blade, a green power OK LED that is used to determine the power state of the server and the power button that is used to turn on the power to the server s host. Also, a non-maskable interrupt button that can be used to send a non-maskable interrupt to the CPU. 2 The server blade has four hard disk drive or solid state drive slots, referred to as HDDs and SSDs. All the server blade s ports come from 3 the Universal Connector Port that requires an octopus cable, referred to as a dongle cable, to access its serial management port, its two USB ports and the video port. 4

35 Slide 35 Inside of the SPARC-1B memory Disk Slots 0 &1 REM memory FEM memory memory CPU SP Disk Slots 2 & 3 35 The key components of the SPARC-1B server blade includes 1 a single CPU and up to 16 slots. 2 The RAID Expansion Module or REM supports 3 the four disk slots that are at the front of the server blade. Another daughter board is 4 the Fabric Expansion Module or FEM that supports the host s network ports through a Network Expansion Module or NEM. 5 This server blade also has a service processor, known as the SP, that can be easily replaced as we will see later in this course. To display a list of FRUs and CRUs click on the SPARC-1B FRUs and CRUs entry within the attachment tab. For a closer look at the inside of the server select SPARC T3-1B 3-D entry from the attachment tab. Using the control panel that comes with it, you can see the server in any 360 degree orientation, zoom in and out on components, and remove components to view other components within the enclosure. 6

36 Slide 36 SPARC T3-1B CPU and Memory A B C D SPARC T3 Densities: 2, 4 or 8-Gbyte registered s Speed: 1066 MT/sec 36 The SPARC T3-1B server blade has four memory channels each supporting a Buffer on Board or Bob. 1 Each Bob has two channels that supports two slots. 2 Each slot supports either a 2, 4 or 8- Gbyte registered 3 that operate at 1066 Megatransfers per second. Note, the population and replacement rules will be covered in the Installation and Replacement module of this course. 4

37 Slide 37 SPARC T3-1B I/O Subsystems Memory Subsystem SPARC T3 (2) XAUI Switch 0 48 ports A0 A1 C x1 Switch 1 48 ports x4 x1 B0 B1 C Service Processor MAX3241 BCM5221 video SERMGT NETMGT / PCI Bridge PLX PEX8112 disk0 disk1 disk2 disk3 REM SAS2 SAS2 NEM0 NEM1 Intel A0 A1 B0 B1 XAUI XAUI 82576EB MAC/PHY 0 1-GbE 1-GbE EM0 NEM0 EM1 NEM1 NEM0 NEM1 NEM0 NEM1 USB Hub UCP USBs PCI / USB Bridge NEC upd Internal USB 37 The I/O subsystem of the SPARC T3-1B server blade originates directly from the SPARC T3 CPU through 1 two 8 lane interconnects that interface to two 48 port switches. Each switch provides 2 two 8 lane interconnects that are routed to the NEMs and EMs, as you can see here. 3 The CPU also provides two XAUI interfaces that are routed to NEM slots to produce 10-Gigabit Ethernet ports. 4 The onboard Ethernet controller generates 1-Gigabit Ethernet interfaces that is also routed to the NEM slots switch 0 also supports 5 the RAID Expansion Module or REM and the onboard LSI disk controller through individual 8 lane interconnects. The REM supports the four onboard disk slots that can house either hard disk drives or solid state drives, referred to as HDDs and SSDs. 6 switch 1 supports three USB ports through a one lane interconnect. Two USBs through the UCP and the other internally on the server motherboard. 7 The service processor is also connected through a one lane interconnect. 8

38 Slide 38 Specific Server Architecture and Features Return 38 You have just completed the specific architecture and features of one of the servers. If you want to cover another of the servers click on the return icon. If not, click on the play button at the bottom of the screen. 1

39 Slide 39 Software Stack NOTE: Solaris 10 Update 8 coupled with MU9 was added and is now the minimum supported OS version. 39 The SPARC T3-1, 2, 4 and -1B servers run Solaris 10 and Open Solaris. These servers will release with a minimum of Solaris Update 9 running the Sun4v architecture. As you can see, the software stack includes 1 the management software OBP and ILOM along with hypervisor which covers the hardware platform and presents a virtual platform to the Solaris OS. ILOM runs as a application of an embedded Linux OS. 2 Note, the IPMI software is embedded within ILOM but you need the IPMITool user interface software to be able to access its CLI. It comes under Solaris 10 within /usr/sfw/bin directory. Also note, that Fault Management Architecture Event Transport Module is also embedded with ILOM and assists in error data collection for the OS level FMA. 3 POST provides hardware host platform testing that verifies that the host system is capable of booting and running the Solaris OS. It runs on initial power on of the server as well as after a system error. More information about this software stack will be covered throughout this course. 4 Note, Solaris 10 Update 8 coupled with Maintenance Update 9 was added and is now the minimum supported OS version. 5

40 Slide 40 PROPERTIES On passing, 'Finish' button: Goes to Next Slide On failing, 'Finish' button: Goes to Next Slide Allow user to leave quiz: At any time User may view slides after quiz: At any time User may attempt quiz: Unlimited times

41 Slide 41 Summary Identify the features of the servers Explain the architecture of the servers List the server s FRUs and CRUs 41 Now that you completed this module, you should be able to 1 identify and explain the features and architecture of the servers. You should also be able to 2 list its FRUs and CRUs. This completes this module.

42 Slide 42 Oracle is the Information Company 42

43 Slide 43 Oracle 43