Storage Systems. The text highlighted in green in these slides contain external hyperlinks. 1 / 12

Size: px

Start display at page:

Download "Storage Systems. The text highlighted in green in these slides contain external hyperlinks. 1 / 12"

Dwain Perry
5 years ago
Views:

1 Storage Systems Compared to the performance parameters of the other components we have been studying, storage systems are much slower devices. Typical access times to rotating disk storage devices are in the millisecond range. Fortunately, organization (both hardware and software) and technology can help mitigate this performance disparity. Probably the largest assist comes from DMA and the actual hiding of this latency (by prefetching or by multitasking). RAID organizations, disk caches, and flash/solid state disks can also help improve overall performance. The text highlighted in green in these slides contain external hyperlinks. 1 / 12

2 Typical System-I/O Setup CPU-memory bus Cache CPU Bus adapter Main memory I/O bus I/O controller I/O controller I/O controller Disk Disk Graphics output Network FIGURE 6.15 A typical interface of I/O devices and an I/O bus to the CPU-memory bus. 2 / 12

3 Program Controlled I/O ReadWait: if (ReadDeviceBusy) then goto ReadWait; else move CharacterRead to desired memory location; ReadDeviceBusy = false; end 3 / 12

4 Direct Memory Access (DMA) Package and handoff I/O requests to device electronics for concurrent access with other executing tasks. Basic process: 1. Application: issues I/O request 2. O/S: if (device available) package and send request to device & start device else queue request 3. I/O: process request stealing memory cycles 4. O/S: place job requesting I/O on blocked list 5. O/S: return to normal job scheduling 6. I/O: interrupt CPU when I/O complete 7. CPU: respond to interrupt, invoking O/S interrupt service routine 8. O/S: recognize interrupt & move blocked job back to read-to-run list; post any other pending request to the device 9. O/S: return to normal job scheduling 4 / 12

5 Redundant Array of Inexpensive Disks (RAID) Most significant types from the base classification: RAID 0: unchanged (nicknamed JBOD for just a bunch of disks); in practice this is often configured to interleave the data on disk for higher performance RAID 1: mirroring (exact copies of the data; attacking reliability, but also improving read performance) RAID 4: parity (placed on an extra disk; the parity disk rapidly becomes a bottleneck) RAID 5: distributed parity (same as 4 but with distribution of parity bits) 5 / 12

6 RAID 0/RAID 4/RAID 5 RAID 0: consider RAID 4 w/o parity disk P P P P P2 8 9 P P3 12 P P4 P P P5 FIGURE 6.28 Block-interleaved parity (RAID 4) versus distributed block-interleaved parity (RAID 5). 6 / 12

7 RAID: Additional derivatives RAID 10 (RAID 1 + RAID 0): mirror disks and then interleave the mirrors. RAID 01 (RAID 0 + RAID 1): interleave data then mirror. RAID 6: add additional parity disks to allow detecting/recovering from multiple failures. 7 / 12

8 Faults, Errors, and Failures Fault: a mistake/issue that can be triggered to throw an error. The fault can be of human origin (e.g., a programming mistake) or a physical fault. Faults are further classified into transient faults, intermittent faults, or permanent faults. Error: when a fault negatively affects the operation of the device to deliver incorrect data, potentially triggering a failure. Failure: the actual behavior deviates from the specified behavior (so if ECC corrects the error, no system failure occurs). 8 / 12

9 Case Study of Failure: The Tertiary Disk Project 20 PCs networked by switched Ethernet containing 368 disks; monitored over 18 months. Component Quantity # failures % failed SCSI Controller SCSI Cable SCSI Disk IDE/ATA disk Disk Enclosure backplane Disk Enclosure power Ethernet Controller Ethernet Switch Ethernet Cable CPU/motherboard / 12

10 X86 Block Diagram (separate northbridge) 10 / 12

11 X86 Block Diagram (onchip northbridge) 11 / 12

I/O Backplane and Bandwidths of Sun Fire x4150 DIMMs B3 B2 B1 B0 USB to IDE IDE CD/DVD Intel Xeon 5100/5300 TM XEON TM XEON Intel Xeon 5100/5300 FSB 1333 MT/s FSB 1333 MT/s C0 C1 C2 C3 10.

3 GB/s DIMMs MCH Blackford 5000P DIMMs 5.3 GB/s 5.

12 I/O Backplane and Bandwidths of Sun Fire x4150 DIMMs B3 B2 B1 B0 USB to IDE IDE CD/DVD Intel Xeon 5100/5300 TM XEON TM XEON Intel Xeon 5100/5300 FSB 1333 MT/s FSB 1333 MT/s C0 C1 C2 C GB/s Dual FSB to MCH 10.5 GB/s Channel C Channel B 5.3 GB/s 5.3 GB/s DIMMs MCH Blackford 5000P DIMMs 5.3 GB/s 5.3 GB/s Channel D Channel A A3 A2 A1 A0 PCIe ESI (PCIe) PCIe x8 PCIe x8 PCI-E SAS/RAID Controller PCIe x8 USB IOH ESB-2 PCIe x4 PCI 32-bit 33 MHz 2x USB USB Hub AST2000 Q GP 0608 TAN A2 1x Internal USB 2.0 2x Rear USB 2.0 2x Front USB 2.0 2x 1GB Ethernet 2 & 3 2x 1GB Ethernet 0 & 1 Serial RJ-45 Management 10/100 Ethernet VGA Video D0 D1 D2 D3 DIMMs PCIe x16-2 PCIe x16-1 PCIe x16-0 8x SAS HDDs FIGURE 6.17 Logical connections and bandwidths of components in the Sun Fire x The three PCIe connectors allow x16 boards to be plugged in, but it only provides eight lanes of bandwidth to the MCH. Source: Figure 5 of SUN FIRE X4150 AND X4450. SERVER ARCHITECTURE (see Copyright 2009 Elsevier, Inc. All rights reserved. 12 / 12

SUN FIRE X4150 AND X4450 SERVER ARCHITECTURE

SUN FIRE X4150 AND X4450 SERVER ARCHITECTURE Breakthrough Density Using Sun and Intel Technology White Paper October 2007 SunWIN # 508684 Table of Contents Executive Summary................................................