Storage System COSC UCB

Transcription

1 Storage System COSC UCB

2 I/O and Disks Over the years much less attention was paid to I/O compared with CPU design. As frustrating as a CPU crash is, disk crash is a lot worse. Disks are mechanical devices, bottleneck and according to Amdahl s law, diminishes the progress in CPU s Types: hard disks, optical disks, and tapes UCB

3 I/O system Processor interrupts Cache Memory - I/O Bus Main Memory I/O Controller I/O Controller I/O Controller Disk Disk Graphics Network UCB

4 I/O System UCB

5 Magnetic Disks Arm Head Sector Inner Track Outer Track Actuator Platter Information recorded on several platters (both sides usually). Bits are recorded sequentially on tracks. Tracks are divided into sectors Heads are connected to arms that moves to position itself over the required track by actuators. Heads could be fixed (one per track) or movable. Cylinder is all the tracks under the head. Areal density: bits per square inch UCB

6 Magnetic Disks Typical values: Rotation speed ,000 RPM. Number of platters Diameter inches. Number of tracks per surface 5,000-30,000 Sectors: typically 512 bytes. Disk latency: seek timerotation timetransfer timecontroller overhead UCB

7 Disk Performance Rotation delay: average is halfway around the disk, for 10,000RPM, the average rotation latency is 60*0.5/10000=3.0 ms. Seek time: average number of tracks arm moves. Typically 8ms. (overlaps with rotation time). Transfer time: 3-60 MB per second. Read ahead is used to cache nearby sectors in the disk cache (0.1-4 MB). More sectors may be on the outer track compared to inner track, constant bit density. (although not quite constant). Another solution is zone-bit recording UCB

8 Optical Disks Compact Disks CD ROM, DVD ROM (Digital Versatile Disk). CD-RW, and WORM (Write Once Read Many) Magneto Optical Disks: uses an optical laser to enhance the capabilities of a magnetic disc system. Reading is optical: direction of magnetization can be detected by a polarized laser light. The disc is coated with a material whose polarity can be altered only at high temp. Laser is used to heat tiny spots and then applying a magnetic field UCB

9 Magnetic Tapes. Same idea as magnetic disks. Sequential access. Helical scan tapes: information is recorded using a tape reader that spins much faster than the tape, and is recorded on a diagonal to the tape (one limit to tape drivers is the speed the tape can be spun without jamming). Tapes wear off: Automated tape library: A robot is used to automate loading and changing the tapes UCB

10 RAID Redundant Arrays of Inexpensive Disks. Used to improve the system performance and reliability. The idea is to distribute the data among more than one physical disk UCB

11 RAID Level 0 Is a misnomer, no redundancy Strips are distributed among many disks A strip is a block, or a sector Fast access since we can read many strips at the same time UCB

12 RAID Level 1 Mirroring Each disk is fully duplicated onto its mirror. Expensive 100% overhead Must write twice (but in parallel UCB

13 RAID Level 2 Raid level 2 performs stripping with a strip size of 1 bit or 1 byte. Must have extra disks to store error correcting codes. No commercial product was released for Level 1 RAID UCB

14 RAID Level 3 Data are stripped in small units. One extra parity disk. Spindles are synchronized (head is over the same sector in each disk). Works fine for access of big sequential data. Only 1 I/O request can be done at a time (not very good for transaction based environment) UCB

15 RAID Level 4 Similar to 3, but blocks are distributed on disks instead of bits. One disk is used for parity (potential bottleneck). Penalty for small write requests. For large writes, the parity is calculated from the written data (otherwise from the difference between old and new data) UCB

16 RAID Level 5 Parity is distributed over all disks (avoiding potential bottleneck) To write Strip 0 S 0. Must do Read S0, and P0 EXOR(S 0,S 0 ) and EXOR with P 0 write to P 0 Write S UCB

17 RAID Level 6 Uses 2 extra disks. Parity is distributed over all disks 2 different parity are used, one is the EX- OR, one is another parity. 3 disk failure in the MTTF in order to loose data, UCB

18 Reliability A fault creates one or more error. Errors are latent. Latent errors become effective once activated. If the error affects delivered service, a component failure. Module Reliability: is a measure of continuous service (MTTF) Availability = MTTF/(MTTFMTTR) UCB

19 Reliability If a collection of modules have an exponentially distributed lifetimes, then The overall failure rate is the some of failure rates for individual component. The failure rate, is the reciprocal of the MTTF UCB

20 Reliability Assume the following 10 disks, each has MTTF of 1,000,000 hours 1 SCSI controller, MTTF of 500,000 hours 1 Power supply, 250,000 hours MTTF 1 fan, with 200,000 hours MTTF 1 SCSI Cable with 1,000,000 hours MTTF Failure rate MTTF system system 1 1 = , = 1,000,000 hours = 1,000, ,000 = 43,500 hours = 5 years 1 200, UCB

21 Example 2500 MIPS CPU for $20, byte-wide interleaved memory 10ns 1000MB/sec I/O bus with room for 20 Ultra3 SCSI busses and controllers Wide Ultra SCSI bus 160MB/sec (can support up to 15 disks pr bus called SCSI strings) A $500 Ultra3 SCSI controller.3 ms delay OS uses 50,000 instructions per IO Large 80, or 40GB disk $10 per GB $1500 enclosure power and cooling to 8 large disks or 12 small disks 15,000 RPM 5 msec seek 40MB/sec disk Storage must be 1920GB, with 32KB per IO UCB

22 Example CPU Memory 1000M B/B/sec I/O BUS SCSI controller SCSI controller SCSI controller 160MB/sec SCSI BUS UCB

23 Example Find the cost of IOPS (IO Per Sec, for both small and large disks (assume 100% utilization). We have a chain of components, CPU, OS, Memory, different busses, disks, and controllers. The performance of the system is limited by the weakest link in the chain UCB

24 Example CPU 50,000 IOPS 25000MIPS Maximum IOPS = = 50,000 Instructions/IO 50,000IOPS For Memory 50,000 IOPS 16 /(10ns) Maximum IOPS = = 32KB per I/O 50,000IOPS UCB

25 Example I/O Bus 31, MB / sec Maximum IOPS = = 32KB per I/O 31,250IOPS SCSI Controllers 2,000IOPS (this is per SCSI, we may have more than one) 32KB Time to transfer a block = = 0.2msec 160MB/sec Total time per block = = 0.5msec. 1 5msec = 2000IOPS UCB

26 Example Now, the disks themselves. I/O time = 5 msec ,000ROM 32KB 40MB / sec = 7.8msec. Maximum IOPS per disk is 128, that is of course assuming 100% utilization. Now how many disks we need, total capacity is 1920GB, so we need 24 large disks or 48 small disks. We have to be sure do we have enough SCSI strings UCB

27 Example If we used 24 disks, the max. number of IOPS is 24 x 128=3072. If we use 48 disks the ma. Is 6144 IOPS For choice 1 (24 large disks) We need 24/15 = 2 SCSI strings. For choice 2 (small disks) We need 48/15=4 SCSI strings Both are O.K. although three enclosure for 2 SCSI controllers may not be the best way to go, increase them to 3 SCSI controllers UCB

28 Example The limit is 3072 for large disks and 6144 for small disks. Now for the cost Large disks 20,0003*$50024(80x10)1500*3=$45,200 Small disks 20,0003*$50048(40x10)1500*4=47, UCB

29 Example Calculate reliability assuming Component CPU/Memory MTTF (hours)disk 1,000,000 Disk 1,000,000 SCSI controller 500,000 Power supply 200,000 SCSI cable 1,000,000 Enclosure 1,000,000 Fan 200, UCB

30 Example Consider failure rate Big disk 1 Big disk = 1,000,000 = 67 1,000,000 MTTF = 14,925 hours Small disk CPU disks controller enclosure power supply fan 24 1,000, , ,00, ,00, ,00,000 cables 3 2,00,000 Big disk MTTF = = 1 1,000, ,000,000 = 9524 hours 48 1,000, , ,00, ,00, ,00, ,00, UCB

31 Example --Availability The configuration of the previous example has changed due to limitations on the utilization of the different parts. We did not cover that, but here is the final configuration for the next example (availability). 80GB disks 4 strings, 4 enclosures, 24 disks 40GB disks 8 strings, 4 enclosures, 48 disks UCB

32 Example -- Availability What about availability If we have n component, each has MMTF, the total MMTF n = MTTF/n For RAID, one data is lost if a second disk failed before the first disk is repaired. Probability of that is MTTR/MTTF MTDL = MTTF MTTR MTTF disk disk / disk /( G N 1) G is the number of disks in the group protected by a parity,n is the number of disks in the system UCB

33 Orthogonal RAID String Controller... String Controller... Array Controller String Controller... String Controller... String Controller... String Controller... If a string controller fails, the system is available If a string controller fails, all disks in the group fails and data is lost UCB

34 Example -- Availability SO far, for large disks 4 strings 24 disks 4 enclosures For small disks 8 strings 48 disks 4 enclosures UCB

35 Example --Availability I/O BUS SCSI SCSI SCSI SCSI I/O BUS SS CC SIS 6I6 S C S I6 S C S I6 S C S I6 S C S I6 S C S I6 S C S I6 S C S I UCB

36 Example --Availability Large Disks 4 enclosures, each one SCSI controller and 6 disks ADD 1 enclosure, controller and 6 disks Small disks 4 enclosures, each has 2 controllers with 12 disks ADD 1 enclosure 2 controllers with 12 disks each UCB

37 Example --Availability Now to calculate the MTTF per enclosure Disks SCSI controller power fan cable enclosure Enclosure Failure Rate big = 6 1,000, , , , ,000, ,000,000 = 20 1,000,000 Enclosure Failure Rate small = 6 1,000, , , , ,000, ,000,000 = 29 1,000,000 MTTF big = 50,000 hours MTTF small = 34,500 hours UCB

38 Example -- Availability Now, consider the MTDL, note that G=N=5. MTTR=24 hours. MTDL MTDL big small = 2 50, = 2 34, = 5,200,000 hours = 2,500,000 hours Cost big =20,0005x$50030x(80*10)5*1500 =$54,000 Cost small =20,00010*50060*(10*40)5*1500 =$56,500 The big disk costs $10 per 1000 hours of operation The small disk costs $23 per 1000 hours of operation UCB