I/O Commercial Workloads. Scalable Disk Arrays. Scalable ICDA Performance. Outline of This Talk: Related Work on Disk Arrays.

Size: px

Start display at page:

Download "I/O Commercial Workloads. Scalable Disk Arrays. Scalable ICDA Performance. Outline of This Talk: Related Work on Disk Arrays."

Lionel Montgomery
5 years ago
Views:

1 Scalable Disk Arrays I/O Commercial Workloads David Kaeli Northeastern University Computer Architecture Research Laboratory Boston, MA Manpreet Singh William Zahavi EMC Corporation Hopkington, MA Industry is becoming increasingly dependent upon data to manage their business Sears and Walmart will have decision support databased larger than 4 TB in 2 93% of major US companies are centralizing their information in data centers > 5% of these companies are in various stages of planning large storage-area networks The scalability of data center systems (ICDAs) will be critical to the success of US business in the future Scalable ICDA Performance Current barriers to scalable performance in cached I/O systems include: Bus-based interconnect topologies Centralized cache Centralized cache coherency mechanisms Our approach is to provide parallelism by: Introducing scalable interconnect components and topologies Decentralizing cache memory and cache management Constraints: Fault tolerance Scalability Leverage advances in commodity technology Outline of This Talk: ICDA Architecture Related work Studying I/O commercial workloads Interconnection topologies Simulation results Conclusions and future work ICDA Architecture Related Work on Disk Arrays Host Host Cache Memory Interconnection Network Disk Disk RAID (Redundant Array of Inexpensive Disks) (Gibson 87) data striping small write problem declustered parity caching, destaging and prefetching Network Attached Secure Disk remove fileserver bottleneck by attaching drives directly to the network Tertiary Disks a switched networks of PCs, each conect to a dual channel SCSI disk array IDISK embeds processor and cache in the disk controller

2 Summary of Industry-standard Storage Systems ICDA Modeling Philosophy Company Storage system model Maximum capacity Maxinum Cache Throughput # of Hosts Ciprico Ruggedized 7 2 TB none MB/sec single Ciprico 69 Ultra SCSI 288 GB none 4 MB/sec single Dell 65F, 651F 2 TB 1 GB MB/sec two Dell 2S,21S 2.3 TB none 8 MB/sec - IBM 7133 Serial Disk 3.5 GB/host none 16 MB/sec Multiple Compaq ESA disks 512 MB 8 MB/sec two DG CLARiiON 1.58 TB 1.5 GB 2 MB/sec - HP E Disk XP GB MB/sec 32 EMC Symmetrix TB 16 GB - 32 Model existing and new architectures at different level of abstraction Model the interaction of prefetching and destaging algorithms, network contention Evaluate cache partitioning and interconnection topologies Drive simulation based on real commercial workloads Exploit existing EMC tools for workload generation Develop a modeling framework which allows for easy implementation, maintenance and modifications Components of ICDAmodel Host adapter: a buffer between the hosts and the internal interconnection fabric of an ICDA Disk adapter: a buffer between the interconnection fabric and the disk devices Interconnection fabric: interconnect host adapters, disk adapters and cache partitions,models the particular technology, topology, and routing policy, as prescribed by the particular interconnect used Cache memory: non-coherent cache partitions that store both the directory and data Disks: non-volatile storage devices, with predefined physical data layout ICDAmodel Specifics ICDA is a hybrid event-based simulator The simulator is driven by a synthetic workload generator Adapter buffers, switch buffers, disks and cache controllers are modelled as queues We limit the number of concurrent accesses active in the ICDA at any one time ( IOs) 4 GB of cache memory, fully associated, LRU 96 physical disks, 418 GB of RAID5 disk space, 12 disks attached to 1 disk adapter, 2 logical volumes/disk, fixed disk latency of 7 ms Events in ICDAmodel Read Hit occurs if all the requested data in found in cache Read Miss occurs if any of the requested addresses are not found in the cache A Write operation results in a cache hit, if the write-pending threshold is reached. The write operation is complete as soon as the data is written to the cache Write Miss occurs if the cache reaches any of the writepending thresholds. Modified (dirty) cache slots are first written to the disk and then the present write data is written to the cache Prefetching captures the spatial locality in the access pattern and tries to bring data in the cache before it is accessed Destaging manages Writes to the disks Prefetching and Destaging Algorithms Prefetch next sequential line on a read miss If this prefetched line is accessed, prefetch next two sequential lines Maximum prefetch distance is limited to three cache lines Two write-pending thresholds have been defined: the system level threshold and the logical volume level threshold The system starts incurring write misses as soon as either of the write-pending thresholds is reached Prefetch and destage are low-priority operations

3 I/O Commercial Workloads Commercial workloads - OLTP: Online transaction processing system - DSS: Decision support system - DATAMINE: Data mining database application Scaled workloads -RANDOM: Contains only random references Workloads were generated by using Workload Generator from EMC Each workload is composed of following fundamental workloads: random read hit (RRH), random read miss (RRM), random write (RW), sequential read (SR) and sequential write (SW) Characteristics of commercial workloads I/O profile I/O size % of Total I/O size % of Total I/O size % of Total RRH 4KB 4% 32KB 18% 32KB % RRM 4KB 24% 32KB 18% 32KB 6% RW 4KB 16% 32KB 4% 32KB % SR 4KB % 32KB 48% 32KB % SW 4KB % 32KB 12% 32KB % Characteristics of RANDOM workloads I/O Profile I/O Size % of Total RRH 4/32 KB 29% RRM 4/32 KB 21% RW 4/32 KB 5% Interconnection Topologies Two-bus topology (EMC Symmetrix) Torus topology (SCI based) Dual ServerNet topology (ServerNet based) Fat tree topology (ServerNet based) Modified Fat tree topology (ServerNet based) Modeling 4MB buffers are modeled at each switch SCI and Servernet switches have a.25ms latency All switched networks use Wormhole Routing Our Torus topology also uses X-Y Routing Shared-bus system Torus topology

4 X-network Y-network Fat Tree topology Dual ServerNet topology Hit rates for moderate IO rates Workload Read Hits Write Hits IO Rate (IOs/sec) OLTP 65.9% 5.1% 244 DSS 21.6% 63.9% 1221 Modified Fat Tree topology DATAMINE I/O Throughput (in MB/sec) for moderate IO request rates Workloads Two Bus Torus Fat Tree Mod Fat ServerNet Hit rates for high IO rates Workload Read Hits Write Hits IO Rate (IOs/sec) OLTP 65.9% 43.% 4816 DSS 21.5% 43.6% 2641 DATAMIN E 11.8% 3.4% 2641

5 I/O Throughput (in MB/sec) for high IO request rates I/O Throughput (in MB/sec) for a Torus topology running a RANDOM workload Two Bus Torus Fat Tree Mod Fat ServerNet partitions 5 partitions 6 partitions 7 partitions 8 partitions Workloads Host IO Rates Cost Comparison of switch-based topologies Topology # of switches # of links Ave. # of ports available per switch Length of longest path Load distribution Torus U 2. 7 non-uniform Fat Tree B very-uniform Mod Fat 2 72 B very-uniform Dual Server 16 6 B uniform Network Reliability We characterize network completeness by Monte Carlo simulation. Network is complete as long as there is some non-faulty path between every pair of network endpoints. Probability of completeness is plotted for a given number of uniformly distributed random faults Fat tree and modified fat tree are shown to be the most fault tolerant among switch-based topologies. Results Summary For high IO rates, switch-based topologies can achieve an increase in delivered host throughput by as much as 4X over a bus-based topology Modified Fat Tree topology provides the highest throughput Increasing the number of cache partitions from 4 to 8 for the Torus topology increased throughput by 89% Increasing the number of cache partitions for the Modified Fat Tree topology did not significantly improve performance Conclusion and Future Work We have studied the relationship of different interconnects, the workloads run on ICDAs, and the inherent performance limitations of current ICDA interconnects The Modified Fat Tree topology employs a simple routing algorithm, great potential for scalability and higher fault tolerance as compared to other switch-based topologies considered in this work Future work will explore more variations of Fat Tree topologies Further investigation of intelligent prefetching and destaging algorithms The use of fibrechannel switching will be addressed in future work

LSI Corporation

LSI Corporation Figure 47 RAID 00 Configuration Preview Dialog 18. Check the information in the Configuration Preview Dialog. 19. Perform one of these actions: If the virtual drive configuration is acceptable, click Accept