Calypso Blind Survey 2010

Transcription

1 Calypso Blind Survey 2010 SSD Performance Comparison MLC, SLC & HDD Eden Kim, CEO Dr. Easen Ho, CTO Calypso Systems, Inc. SNIA SDC 22 September

2 Finally.. Meaningful SSD Comparisons Device Level Ordinal Ranking Steady State SSD Performance Synthetic Workload Characteristics August

3 Table of Contents August

4 Topics for Today SSS TWG Performance Test Spec Overview PTS Methodology Calypso Blind Survey Reference Test Environment Group Blinded Data Enterprise SSD v HDD Take Aways August

5 SNIA PTS TEST SPEC August

6 SNIA SSD Performance Test Spec Purpose: SSD Performance Testing Goal: Apples to Apples Comparison Scope: Device Level Synthetic Test Hardware & Software Agnostic Defines Requirements Device Level Block IO Performance NOT File System or Application NOT Endurance, Reliability, Power, Cost / GB / IOP / W August

7 SNIA SSD Performance Test Spec Key Concepts: Steady State Measurement Methodology: Pre Condition to Steady State Test Environment: Normalized Test Environment Tests: IOPS, Throughput & Latency Reporting: SNIA Recommended Report Format August

8 STEADY STATE August

9 Performance Changes over Time August

10 Write History PC Active Range Test Active Range Data Pattern What was previously written Where data was previously written Trim effects Where data is currently written What is being accessed Access Pattern Demand Intensity Throttling Architecture Manner in which data is being accessed How hard the application is driving the device How much data is being written NAND Type Controller Algorithms 10

11 1. Purge Security Erase, Sanitize, Format Unit, other proprietary methods 2. Set Conditions 3. Pre-Condition Set user selectable test parameters, such as Active Range, Data Pattern, Demand Intensity Workload independent Workload dependent 4. Run Until SS Reiterate loops until Steady State is reached, or run to a prescribed maximum number of loops 5. Collect Data Collect data from Steady State Measurement Window 6. Generate Reports Use standard report formats and include required and optional elements 11

12 Pre Conditioning to Steady State ACTIVITY PURPOSE Step 1 PURGE Security Erase/Format Unit Writing to all LBAs As if no Writes Occurred Create a Repeatable Test Starting Point Step 2 Workload Independent Pre Conditioning WIPC - 2X User Capacity SEQ 128K W Touch all LBAs Linearize Look Up Table Step 3 Workload Dependent Pre Conditioning WDPC - Test Stimulus Access Pattern In Rounds Repeat Workload Dependent Stimulus Step 4 Steady State Convergence Convergence of Workload Dependent Rounds Determine when Steady State Reached Step 5 Steady State Measurement Window 5 Rounds within 20% Excursion of Ave & 10% Slope of Curved Fit Establish Window from Which to take Measurements August

13 Workload Dependent PC All Block Sizes August

14 Steady State Convergence Tracking Block Size August

15 STEADY STATE Measurement Window 5 Rounds: w/in 20% Excursion Band AND 10% Slope August

16 Blind Survey 2010 August

17 SSD Blind Surveys 2d Annual Blind Survey SSD Performance Continues to Evolve SSD Unique Behaviors & Considerations Write History & Workloads Reference Test Environment Performance Changes over Time Application vs Device Workloads August

18 Blind Survey SSD Sample Pool (8) MLC; (8) SLC; (1) 15K RPM SAS HDD No. Drive Manufacturer Drive Model No. NAND Type & Capacity Form Factor & Interface 1. Imation MAC GB MLC Client Gb/s SATA 2. Intel X25M 160 GB MLC Client Gb/s SATA 3. Micron C GB MLC Client Gb/s SATA 4. SanDisk G3 128 GB MLC Client Gb/s SATA 5. Samsung UPM1 128 GB MLC Client Gb/s SATA 6. SandForce SF GB MLC Client Gb/s SATA 7. Toshiba N GB MLC Client Gb/s SATA 8. Smart Modular SG9XGS 128 GB MLC Enterprise Gb/s SATA 9. Dell DS1 100 GB SLC Enterprise Gb/s SATA 10. Intel X25E 64 GB SLC Enterprise Gb/s SATA 11. Micron P GB SLC Enterprise Gb/sec SATA 12. Pliant PLS LS1 200 GB SLC Enterprise Gb/s SAS 13. Pliant PLS LB1 300 GB SLC Enterprise Gb/s SAS 14. Samsung SS GB SLC Enterprise Gb/s SATA 15. Soligen D50 50 GB SLC Enterprise Proprietary 3 Gb/s SATA 16. Viking Modular Element 200 GB SLC Enterprise Gb/s SATA Santa Clara, 17. CA Seagate Savvio 500GB 15K RPM HDD Gb/s SAS August

19 Test Set Up August

20 SNIA Compliant Reference Test Platform Normalized Test Hardware & Software Standardized Metrics, Methodologies & Tests Repeatable, Accurate, Relevant Test Comparisons! SNIA PTS Compliant Test System - Calypso RTP 2.0 / CTS 6.5 August

21 Calypso Blind Survey 2010 Test Plan Standard Calypso SSD & SNIA PTS Tests WSAT (Write Saturation) & Latency Histogram** IOPS, Throughput, Latency (Client & Enterprise)* Calypso CTS 6.5 Tests Non RND Data Pattern Tests** Limited Active Range (LBA Hot Zones)** Write Saturation with Interleaved Idle** * SNIA PTS COMPLIANT ** Calypso CTS 6.5 COMPLIANT 21

22 FOB Performance Fresh-out-of-the-Box August

23 Performance Varies Among Devices August

24 15K RPM SAS HDD: Stable but low IOPS August

25 Client MLC SSDs: 50,000 to 1,000 Max IOPS August

26 Enterprise SLC: 30,000 to 13,000 Max IOPS August

27 FOB v Steady State August

28 FOB v Steady State MAX FOB IOPS v Consistent Steady State IOPS FOB v Steady State RND 4K W IOPS Drive FOB MAX IOPS ** STEADY STATE * SS / FOB Ratio HDD % MLC-1 1, % MLC-2 19, % MLC-3 50,156 2,716 54% SLC-1 55,677 19,561 35% SLC-2 28,063 1,182 42% SLC-3 21,581 7,556 35% Santa Clara, * SNIA CA PTS 0.9 Compliant August 2010 ** Calypso CTS 6.5 Compliant 28

29 SSD Asymmetric Read v Write Performance Fast Reads, Slower Writes Steady State RND 4K IOPS * Drive READS WRITES W/R Ratio HDD % MLC-A 2, % MLC-B 60,988 2, % MLC-C 25,819 4, % SLC-A 37,091 1, % SLC-B 24,273 20, % SLC-C 46,365 19, % * SNIA PTS COMPLIANT 29

30 Group SLC, MLC IOPS & Throughput August

31 RND IOPS 65% 35% R/W Mix All Block Sizes August

32 SEQ 1MB THROUGHPUT 100% R 100% W August

33 Enterprise SLC v 15K RPM SAS HDD August

34 Enterprise SLC v SAS HDD August

35 Enterprise SLC v SAS HDD August

36 Enterprise SLC v SAS HDD August

37 Enterprise SLC v SAS HDD August

38 Enterprise SLC v SAS HDD August

39 Enterprise SLC v SAS HDD August

40 Enterprise SLC v SAS HDD August

41 Survey Conclusions NON RANDOM DATA PATTERNS August

42 Non Random Data Patterns WSAT using RND Data Pattern - WSAT WSAT using Non Random (0000s) WSAT using a Compressible Data Base Binary File * SNIA PTS COMPLIANT ** Calypso CTS 6.5 COMPLIANT 42

43 SLC (BS1) Data Pa8ern Effects RND/4K Writes DP=0 DP=Database DP=RND DP=RND Write Sat IOPS Time (Minutes)

44 SLC (UMC1) Data Pa8ern Effects RND/4K Writes DP=0 DP=Database DP=RND DP=RND Write Sat IOPS Time (Minutes)

45 Sandforce Controller (CM1) Data Pa8ern Effects RND/4K Writes DP=0 DP=Database DP=RND DP=RND Write Sat IOPS Time (Minutes)

46 Sandforce Controller (MO2) Data Pa8ern Effects RND/4K Writes DP=0 DP=Database DP=RND DP=RND Write Sat IOPS Time (Minutes)

47 Survey Conclusions Take Aways August

48 Performance Take Aways SSD Performance Changes over Time Write History & Workload Affect Performance Take Test Measurements during Steady State SNIA PTS Standard Standardized Test Environment Standardized Test Metrics & Methodologies SSD Products Different Architectures, Algorithms, Different Performance August

49 Industry Take Aways SNIA means Standardized Metrics & Methodologies RTP Testers allow Apples to Apples Comparison Device Level Test provides Ordinal Ranking of Drives Controllers & NAND Performance are Improving SSDs are being integrated into the Mass Storage Ecosystem Synthetic approximation to user workloads is needed August

50 August

51 Thank You! Point Click Results Engineering Test Services SSD Testing Equipment Product Validation Professionals in Solid State Storage Test & Measurement August